Mary Lou Jepsen: Could future devices read images from our brains?

I had brain surgery 18 years ago,

and since that time, brain science has become

a personal passion of mine.

I'm actually an engineer.

And first let me say, I recently joined

Google's Moonshot group,

where I had a division,

the display division in Google X,

and the brain science work I'm speaking about today

is work I did before I joined Google

and on the side outside of Google.

So that said, there's a stigma

when you have brain surgery.

Are you still smart or not?

And if not, can you make yourself smart again?

After my neurosurgery,

part of my brain was missing,

and I had to deal with that.

It wasn't the grey matter, but it was the gooey part dead center

that makes key hormones and neurotransmitters.

Immediately after my surgery,

I had to decide what amounts of each of over

a dozen powerful chemicals to take each day,

because if I just took nothing,

I would die within hours.

Every day now for 18 years -- every single day --

I've had to try to decide the combinations

and mixtures of chemicals,

and try to get them, to stay alive.

There have been several close calls.

But luckily, I'm an experimentalist at heart,

so I decided I would experiment

to try to find more optimal dosages

because there really isn't a clear road map

on this that's detailed.

I began to try different mixtures,

and I was blown away by how

tiny changes in dosages

dramatically changed my sense of self,

my sense of who I was, my thinking,

my behavior towards people.

One particularly dramatic case:

for a couple months I actually tried dosages

and chemicals typical of a man in his early 20s,

and I was blown away by how my thoughts changed.

(Laughter)

I was angry all the time,

I thought about sex constantly,

and I thought I was the smartest person

in the entire world, and

—(Laughter)—

of course over the years I'd met guys kind of like that,

or maybe kind of toned-down versions of that.

I was kind of extreme.

But to me, the surprise was,

I wasn't trying to be arrogant.

I was actually trying,

with a little bit of insecurity,

to actually fix a problem in front of me,

and it just didn't come out that way.

So I couldn't handle it.

I changed my dosages.

But that experience, I think, gave me

a new appreciation for men

and what they might walk through,

and I've gotten along with men

a lot better since then.

What I was trying to do

with tuning these hormones

and neurotransmitters and so forth

was to try to get my intelligence back

after my illness and surgery,

my creative thought, my idea flow.

And I think mostly in images,

and so for me that became a key metric --

how to get these mental images

that I use as a way of rapid prototyping,

if you will, my ideas,

trying on different new ideas for size,

playing out scenarios.

This kind of thinking isn't new.

Philiosophers like Hume and Descartes and Hobbes

saw things similarly.

They thought that mental images and ideas

were actually the same thing.

There are those today that dispute that,

and lots of debates about how the mind works,

but for me it's simple:

Mental images, for most of us,

are central in inventive and creative thinking.

So after several years,

I tuned myself up and I have lots of great,

really vivid mental images with a lot of sophistication

and the analytical backbone behind them.

And so now I'm working on,

how can I get these mental images in my mind

out to my computer screen faster?

Can you imagine, if you will,

a movie director being able to use

her imagination alone to direct the world in front of her?

Or a musician to get the music out of his head?

There are incredible possibilities with this

as a way for creative people

to share at light speed.

And the truth is, the remaining bottleneck

in being able to do this

is just upping the resolution of brain scan systems.

So let me show you why I think we're pretty close to getting there

by sharing with you two recent experiments

from two top neuroscience groups.

Both used fMRI technology --

functional magnetic resonance imaging technology --

to image the brain,

and here is a brain scan set from Giorgio Ganis

and his colleagues at Harvard.

And the left-hand column shows a brain scan

of a person looking at an image.

The middle column shows the brainscan

of that same individual

imagining, seeing that same image.

And the right column was created

by subtracting the middle column from the left column,

showing the difference to be nearly zero.

This was repeated on lots of different individuals

with lots of different images,

always with a similar result.

The difference between seeing an image

and imagining seeing that same image

is next to nothing.

Next let me share with you one other experiment,

this from Jack Gallant's lab at Cal Berkeley.

They've been able to decode brainwaves

into recognizable visual fields.

So let me set this up for you.

In this experiment, individuals were shown

hundreds of hours of YouTube videos

while scans were made of their brains

to create a large library of their brain reacting

to video sequences.

Then a new movie was shown with new images,

new people, new animals in it,

and a new scan set was recorded.

The computer, using brain scan data alone,

decoded that new brain scan

to show what it thought the individual was actually seeing.

On the right-hand side, you see the computer's guess,

and on the left-hand side, the presented clip.

This is the jaw-dropper.

We are so close to being able to do this.

We just need to up the resolution.

And now remember that when you see an image

versus when you imagine that same image,

it creates the same brain scan.

So this was done with the highest-resolution

brain scan systems available today,

and their resolution has increased really

about a thousandfold in the last several years.

Next we need to increase the resolution

another thousandfold

to get a deeper glimpse.

How do we do that?

There's a lot of techniques in this approach.

One way is to crack open your skull and put in electrodes.

I'm not for that.

There's a lot of new imaging techniques

being proposed, some even by me,

but given the recent success of MRI,

first we need to ask the question,

is it the end of the road with this technology?

Conventional wisdom says the only way

to get higher resolution is with bigger magnets,

but at this point bigger magnets

only offer incremental resolution improvements,

not the thousandfold we need.

I'm putting forward an idea:

instead of bigger magnets,

let's make better magnets.

There's some new technology breakthroughs

in nanoscience

when applied to magnetic structures

that have created a whole new class of magnets,

and with these magnets, we can lay down

very fine detailed magnetic field patterns

throughout the brain,

and using those, we can actually create

holographic-like interference structures

to get precision control over many patterns,

as is shown here by shifting things.

We can create much more complicated structures

with slightly different arrangements,

kind of like making Spirograph.

So why does that matter?

A lot of effort in MRI over the years

has gone into making really big,

really huge magnets, right?

But yet most of the recent advances

in resolution have actually come from

ingeniously clever encoding and decoding solutions

in the F.M. radio frequency transmitters and receivers

in the MRI systems.

Let's also, instead of a uniform magnetic field,

put down structured magnetic patterns

in addition to the F.M. radio frequencies.

So by combining the magnetics patterns

with the patterns in the F.M. radio frequencies

processing which can massively increase

the information that we can extract

in a single scan.

And on top of that, we can then layer

our ever-growing knowledge of brain structure and memory

to create a thousandfold increase that we need.

And using fMRI, we should be able to measure

not just oxygenated blood flow,

but the hormones and neurotransmitters I've talked about

and maybe even the direct neural activity,

which is the dream.

We're going to be able to dump our ideas

directly to digital media.

Could you imagine if we could leapfrog language

and communicate directly with human thought?

What would we be capable of then?

And how will we learn to deal

with the truths of unfiltered human thought?

You think the Internet was big.

These are huge questions.

It might be irresistible as a tool

to amplify our thinking and communication skills.

And indeed, this very same tool

may prove to lead to the cure

for Alzheimer's and similar diseases.

We have little option but to open this door.

Regardless, pick a year --

will it happen in five years or 15 years?

It's hard to imagine it taking much longer.

We need to learn how to take this step together.

Thank you.

(Applause)