Could future devices read images from our brains?
I had brain surgery 18 years ago, and since that time, brain science has become a personal passion of mine. I'm actually an engineer. And first let me say, I recently joined Google's Moonshot group, where I had a division, the display division in Google X, and the brain science work I'm speaking about today is work I did before I joined Google and on the side outside of Google.
0:36 So that said, there's a stigma when you have brain surgery. Are you still smart or not? And if not, can you make yourself smart again?
0:48 After my neurosurgery, part of my brain was missing, and I had to deal with that. It wasn't the grey matter, but it was the gooey part dead center that makes key hormones and neurotransmitters. Immediately after my surgery, I had to decide what amounts of each of over a dozen powerful chemicals to take each day, because if I just took nothing, I would die within hours. Every day now for 18 years -- every single day -- I've had to try to decide the combinations and mixtures of chemicals, and try to get them, to stay alive. There have been several close calls.
1:28 But luckily, I'm an experimentalist at heart, so I decided I would experiment to try to find more optimal dosages because there really isn't a clear road map on this that's detailed. I began to try different mixtures, and I was blown away by how tiny changes in dosages dramatically changed my sense of self, my sense of who I was, my thinking, my behavior towards people. One particularly dramatic case: for a couple months I actually tried dosages and chemicals typical of a man in his early 20s, and I was blown away by how my thoughts changed. (Laughter) I was angry all the time, I thought about sex constantly, and I thought I was the smartest person in the entire world, and —(Laughter)— of course over the years I'd met guys kind of like that, or maybe kind of toned-down versions of that. I was kind of extreme. But to me, the surprise was, I wasn't trying to be arrogant. I was actually trying, with a little bit of insecurity, to actually fix a problem in front of me, and it just didn't come out that way.
2:44 So I couldn't handle it. I changed my dosages. But that experience, I think, gave me a new appreciation for men and what they might walk through, and I've gotten along with men a lot better since then.
2:57 What I was trying to do with tuning these hormones and neurotransmitters and so forth was to try to get my intelligence back after my illness and surgery, my creative thought, my idea flow. And I think mostly in images, and so for me that became a key metric -- how to get these mental images that I use as a way of rapid prototyping, if you will, my ideas, trying on different new ideas for size, playing out scenarios. This kind of thinking isn't new. Philiosophers like Hume and Descartes and Hobbes saw things similarly. They thought that mental images and ideas were actually the same thing. There are those today that dispute that, and lots of debates about how the mind works, but for me it's simple: Mental images, for most of us, are central in inventive and creative thinking.
3:53 So after several years, I tuned myself up and I have lots of great, really vivid mental images with a lot of sophistication and the analytical backbone behind them. And so now I'm working on, how can I get these mental images in my mind out to my computer screen faster? Can you imagine, if you will, a movie director being able to use her imagination alone to direct the world in front of her? Or a musician to get the music out of his head? There are incredible possibilities with this as a way for creative people to share at light speed. And the truth is, the remaining bottleneck in being able to do this is just upping the resolution of brain scan systems.
4:38 So let me show you why I think we're pretty close to getting there by sharing with you two recent experiments from two top neuroscience groups. Both used fMRI technology -- functional magnetic resonance imaging technology -- to image the brain, and here is a brain scan set from Giorgio Ganis and his colleagues at Harvard. And the left-hand column shows a brain scan of a person looking at an image. The middle column shows the brainscan of that same individual imagining, seeing that same image. And the right column was created by subtracting the middle column from the left column, showing the difference to be nearly zero. This was repeated on lots of different individuals with lots of different images, always with a similar result. The difference between seeing an image and imagining seeing that same image is next to nothing.
5:33 Next let me share with you one other experiment, this from Jack Gallant's lab at Cal Berkeley. They've been able to decode brainwaves into recognizable visual fields. So let me set this up for you. In this experiment, individuals were shown hundreds of hours of YouTube videos while scans were made of their brains to create a large library of their brain reacting to video sequences. Then a new movie was shown with new images, new people, new animals in it, and a new scan set was recorded. The computer, using brain scan data alone, decoded that new brain scan to show what it thought the individual was actually seeing. On the right-hand side, you see the computer's guess, and on the left-hand side, the presented clip. This is the jaw-dropper. We are so close to being able to do this. We just need to up the resolution. And now remember that when you see an image versus when you imagine that same image, it creates the same brain scan.
6:39 So this was done with the highest-resolution brain scan systems available today, and their resolution has increased really about a thousandfold in the last several years. Next we need to increase the resolution another thousandfold to get a deeper glimpse. How do we do that? There's a lot of techniques in this approach. One way is to crack open your skull and put in electrodes. I'm not for that. There's a lot of new imaging techniques being proposed, some even by me, but given the recent success of MRI, first we need to ask the question, is it the end of the road with this technology? Conventional wisdom says the only way to get higher resolution is with bigger magnets, but at this point bigger magnets only offer incremental resolution improvements, not the thousandfold we need. I'm putting forward an idea: instead of bigger magnets, let's make better magnets. There's some new technology breakthroughs in nanoscience when applied to magnetic structures that have created a whole new class of magnets, and with these magnets, we can lay down very fine detailed magnetic field patterns throughout the brain, and using those, we can actually create holographic-like interference structures to get precision control over many patterns, as is shown here by shifting things. We can create much more complicated structures with slightly different arrangements, kind of like making Spirograph.
8:10 So why does that matter? A lot of effort in MRI over the years has gone into making really big, really huge magnets, right? But yet most of the recent advances in resolution have actually come from ingeniously clever encoding and decoding solutions in the F.M. radio frequency transmitters and receivers in the MRI systems. Let's also, instead of a uniform magnetic field, put down structured magnetic patterns in addition to the F.M. radio frequencies. So by combining the magnetics patterns with the patterns in the F.M. radio frequencies processing which can massively increase the information that we can extract in a single scan. And on top of that, we can then layer our ever-growing knowledge of brain structure and memory to create a thousandfold increase that we need. And using fMRI, we should be able to measure not just oxygenated blood flow, but the hormones and neurotransmitters I've talked about and maybe even the direct neural activity, which is the dream.
9:18 We're going to be able to dump our ideas directly to digital media. Could you imagine if we could leapfrog language and communicate directly with human thought? What would we be capable of then? And how will we learn to deal with the truths of unfiltered human thought? You think the Internet was big. These are huge questions. It might be irresistible as a tool to amplify our thinking and communication skills. And indeed, this very same tool may prove to lead to the cure for Alzheimer's and similar diseases.
9:58 We have little option but to open this door. Regardless, pick a year -- will it happen in five years or 15 years? It's hard to imagine it taking much longer. We need to learn how to take this step together.
10:14 Thank you.
10:16 (Applause)
