Pawan Sinha on how brains learn to see

If you are a blind child in India,

you will very likely have to contend with

at least two big pieces of bad news.

The first bad news

is that the chances of getting treatment

are extremely slim to none,

and that's because most of the blindness

alleviation programs in the country

are focused on adults,

and there are very, very few hospitals

that are actually equipped to treat children.

In fact, if you were to be treated,

you might well end up being treated

by a person who has no medical credentials

as this case from Rajasthan illustrates.

This is a three-year-old orphan girl

who had cataracts.

So, her caretakers took her

to the village medicine man,

and instead of suggesting to the caretakers

that the girl be taken to a hospital,

the person decided to burn her abdomen

with red-hot iron bars

to drive out the demons.

The second piece of bad news

will be delivered to you

by neuroscientists, who will tell you

that if you are older than four or five years of age,

that even if you have your eye corrected,

the chances of your brain learning how to see

are very, very slim --

again, slim or none.

So when I heard these two things,

it troubled me deeply,

both because of personal reasons

and scientific reasons.

So let me first start with the personal reason.

It'll sound corny, but it's sincere.

That's my son, Darius.

As a new father,

I have a qualitatively different sense

of just how delicate babies are,

what our obligations are towards them

and how much love

we can feel towards a child.

I would move heaven and earth

in order to get treatment for Darius,

and for me to be told

that there might be other Dariuses

who are not getting treatment,

that's just viscerally wrong.

So that's the personal reason.

Scientific reason is that this notion

from neuroscience of critical periods --

that if the brain is older

than four or five years of age,

it loses its ability to learn --

that doesn't sit well with me,

because I don't think that idea

has been tested adequately.

The birth of the idea is from

David Hubel and Torsten Wiesel's work,

two researchers who were at Harvard,

and they got the Nobel Prize in 1981

for their studies of visual physiology,

which are remarkably beautiful studies,

but I believe some of their work

has been extrapolated

into the human domain prematurely.

So, they did their work with kittens,

with different kinds of deprivation regiments,

and those studies,

which date back to the '60s,

are now being applied to human children.

So I felt that I needed to do two things.

One: provide care

to children who are currently

being deprived of treatment.

That's the humanitarian mission.

And the scientific mission would be

to test the limits

of visual plasticity.

And these two missions, as you can tell,

thread together perfectly. One adds to the other;

in fact, one would be impossible without the other.

So, to implement

these twin missions,

a few years ago, I launched Project Prakash.

Prakash, as many of you know,

is the Sanskrit word for light,

and the idea is that

in bringing light into the lives of children,

we also have a chance

of shedding light on some of the

deepest mysteries of neuroscience.

And the logo -- even though it looks extremely Irish,

it's actually derived from

the Indian symbol of Diya, an earthen lamp.

The Prakash, the overall effort

has three components:

outreach, to identify children in need of care;

medical treatment; and in subsequent study.

And I want to show you a short video clip

that illustrates the first two components of this work.

This is an outreach station

conducted at a school for the blind.

(Text: Most of the children are profoundly and permanently blind ...)

Pawan Sinha: So, because this is a school for the blind,

many children have permanent conditions.

That's a case of microphthalmos,

which is malformed eyes,

and that's a permanent condition;

it cannot be treated.

That's an extreme of micropthalmos

called enophthalmos.

But, every so often, we come across children

who show some residual vision,

and that is a very good sign

that the condition might actually be treatable.

So, after that screening, we bring the children to the hospital.

That's the hospital we're working with in Delhi,

the Schroff Charity Eye Hospital.

It has a very well-equipped

pediatric ophthalmic center,

which was made possible in part

by a gift from the Ronald McDonald charity.

So, eating burgers actually helps.

(Text: Such examinations allow us to improve

eye-health in many children, and ...

... help us find children who can participate in Project Prakash.)

PS: So, as I zoom in to the eyes of this child,

you will see the cause of his blindness.

The whites that you see in the middle of his pupils

are congenital cataracts,

so opacities of the lens.

In our eyes, the lens is clear,

but in this child, the lens has become opaque,

and therefore he can't see the world.

So, the child is given treatment. You'll see shots of the eye.

Here's the eye with the opaque lens,

the opaque lens extracted

and an acrylic lens inserted.

And here's the same child

three weeks post-operation,

with the right eye open.

(Applause)

Thank you.

So, even from that little clip, you can begin to get the sense

that recovery is possible,

and we have now

provided treatment to over 200 children,

and the story repeats itself.

After treatment, the child

gains significant functionality.

In fact, the story holds true

even if you have a person who got sight

after several years of deprivation.

We did a paper a few years ago

about this woman that you see on the right, SRD,

and she got her sight late in life,

and her vision is remarkable at this age.

I should add a tragic postscript to this --

she died two years ago

in a bus accident.

So, hers is just a truly inspiring story --

unknown, but inspiring story.

So when we started finding these results,

as you might imagine, it created quite a bit of stir

in the scientific and the popular press.

Here's an article in Nature

that profiled this work,

and another one in Time.

So, we were fairly convinced -- we are convinced --

that recovery is feasible,

despite extended visual deprivation.

The next obvious question to ask:

What is the process of recovery?

So, the way we study that is,

let's say we find a child who has light sensitivity.

The child is provided treatment,

and I want to stress that the treatment

is completely unconditional;

there is no quid pro quo.

We treat many more children then we actually work with.

Every child who needs treatment is treated.

After treatment, about every week,

we run the child

on a battery of simple visual tests

in order to see how their visual skills

are coming on line.

And we try to do this for as long as possible.

This arc of development

gives us unprecedented

and extremely valuable information

about how the scaffolding of vision

gets set up.

What might be the causal connections

between the early developing skills

and the later developing ones?

And we've used this general approach to study

many different visual proficiencies,

but I want to highlight one particular one,

and that is image parsing into objects.

So, any image of the kind that you see on the left,

be it a real image or a synthetic image,

it's made up of little regions

that you see in the middle column,

regions of different colors, different luminances.

The brain has this complex task

of putting together, integrating,

subsets of these regions

into something that's more meaningful,

into what we would consider to be objects,

as you see on the right.

And nobody knows how this integration happens,

and that's the question we asked with Project Prakash.

So, here's what happens

very soon after the onset of sight.

Here's a person who had gained sight just a couple of weeks ago,

and you see Ethan Myers, a graduate student from MIT,

running the experiment with him.

His visual-motor coordination is quite poor,

but you get a general sense

of what are the regions that he's trying to trace out.

If you show him real world images,

if you show others like him real world images,

they are unable to recognize most of the objects

because the world to them is over-fragmented;

it's made up of a collage, a patchwork,

of regions of different colors and luminances.

And that's what's indicated in the green outlines.

When you ask them,

"Even if you can't name the objects, just point to where the objects are,"

these are the regions that they point to.

So the world is this complex

patchwork of regions.

Even the shadow on the ball

becomes its own object.

Interestingly enough,

you give them a few months,

and this is what happens.

Doctor: How many are these?

Patient: These are two things.

Doctor: What are their shapes?

Patient: Their shapes ...

This one is a circle,

and this

is a square.

PS: A very dramatic transformation has come about.

And the question is:

What underlies this transformation?

It's a profound question,

and what's even more amazing is how simple

the answer is.

The answer lies in motion

and that's what I want to show you in the next clip.

Doctor: What shape do you see here?

Patient: I can't make it out.

Doctor: Now?

Patient: Triangle.

Doctor: How many things are these?

Now, how many things are these?

Patient: Two.

Doctor: What are these things?

Patient: A square and a circle.

PS: And we see this pattern over and over again.

The one thing the visual system needs

in order to begin parsing the world

is dynamic information.

So the inference we are deriving from this,

and several such experiments,

is that dynamic information processing,

or motion processing,

serves as the bedrock for building

the rest of the complexity of visual processing;

it leads to visual integration

and eventually to recognition.

This simple idea has far reaching implications.

And let me just quickly mention two,

one, drawing from the domain of engineering,

and one from the clinic.

So, from the perspective of engineering,

we can ask: Goven that we know

that motion is so important for the human visual system,

can we use this as a recipe

for constructing machine-based vision systems

that can learn on their own, that don't need to be programmed

by a human programmer?

And that's what we're trying to do.

I'm at MIT, at MIT you need to apply

whatever basic knowledge you gain.

So we are creating Dylan,

which is a computational system

with an ambitious goal

of taking in visual inputs

of the same kind that a human child would receive,

and autonomously discovering:

What are the objects in this visual input?

So, don't worry about the internals of Dylan.

Here, I'm just going to talk about

how we test Dylan.

The way we test Dylan is by giving it

inputs, as I said, of the same kind

that a baby, or a child in Project Prakash would get.

But for a long time we couldn't quite figure out:

Wow can we get these kinds of video inputs?

So, I thought,

could we have Darius

serve as our babycam carrier,

and that way get the inputs that we feed into Dylan?

So that's what we did.

(Laughter)

I had to have long conversations with my wife.

(Laughter)

In fact, Pam, if you're watching this,

please forgive me.

So, we modified the optics of the camera

in order to mimic the baby's visual acuity.

As some of you might know,

babyies are born pretty much legally blind.

Their acuity -- our acuity is 20/20;

babies' acuity is like 20/800,

so they are looking at the world

in a very, very blurry fashion.

Here's what a baby-cam video looks like.

(Laughter)

(Applause)

Thankfully, there isn't any audio

to go with this.

What's amazing is that working with such

highly degraded input,

the baby, very quickly, is able

to discover meaning in such input.

But then two or three days afterward,

babies begin to pay attention

to their mother's or their father's face.

How does that happen? We want Dylan to be able to do that,

and using this mantra of motion,

Dylan actually can do that.

So, given that kind of video input,

with just about six or seven minutes worth of video,

Dylan can begin to extract patterns

that include faces.

So, it's an important demonstration

of the power of motion.

The clinical implication, it comes from the domain of autism.

Visual integration has been associated with autism

by several researchers.

When we saw that, we asked:

Could the impairment in visual integration

be the manifestation of something underneath,

of dynamic information processing deficiencies in autism?

Because, if that hypothesis were to be true,

it would have massive repercussions in our understanding

of what's causing the many different aspects

of the autism phenotype.

What you're going to see are

video clips of two children -- one neurotypical,

one with autism, playing Pong.

So, while the child is playing Pong, we are tracking where they're looking.

In red are the eye movement traces.

This is the neurotypical child, and what you see

is that the child is able to make cues

of the dynamic information

to predict where the ball is going to go.

Even before the ball gets to a place,

the child is already looking there.

Contrast this with a child

with autism playing the same game.

Instead of anticipating,

the child always follows where the ball has been.

The efficiency of the use

of dynamic information

seems to be significantly compromised in autism.

So we are pursuing this line of work

and hopefully we'll have

more results to report soon.

Looking ahead, if you think of this disk

as representing all of the children

we've treated so far,

this is the magnitude of the problem.

The red dots are the children we have not treated.

So, there are many, many more children who need to be treated,

and in order to expand the scope of the project,

we are planning on launching

The Prakash Center for Children,

which will have a dedicated pediatric hospital,

a school for the children we are treating

and also a cutting-edge research facility.

The Prakash Center will integrate health care,

education and research in a way

that truly creates the whole

to be greater than the sum of the parts.

So, to summarize: Prakash, in its five years of existence,

it's had an impact in multiple areas,

ranging from basic neuroscience

plasticity and learning in the brain,

to clinically relevant hypotheses like in autism,

the development of autonomous machine vision systems,

education of the undergraduate and graduate students,

and most importantly in the alleviation

of childhood blindness.

And for my students and I, it's been

just a phenomenal experience

because we have gotten to do interesting research,

while at the same time

helping the many children that we have worked with.

Thank you very much.

(Applause)