What intelligent machines can learn from a school of fish | Radhika Nagpal

In my early days as a graduate student,

I went on a snorkeling trip off the coast of the Bahamas.

I'd actually never swum in the ocean before,

so it was a bit terrifying.

What I remember the most is, as I put my head in the water

and I was trying really hard to breathe through the snorkel,

this huge group of striped yellow and black fish

came straight at me ...

and I just froze.

And then, as if it had suddenly changed its mind,

came towards me and then swerved to the right

and went right around me.

It was absolutely mesmerizing.

Maybe many of you have had this experience.

Of course, there's the color and the beauty of it,

but there was also just the sheer oneness of it,

as if it wasn't hundreds of fish

but a single entity with a single collective mind

that was making decisions.

When I look back, I think that experience really ended up determining

what I've worked on for most of my career.

I'm a computer scientist,

and the field that I work in is artificial intelligence.

And a key theme in AI

is being able to understand intelligence by creating our own computational systems

that display intelligence the way we see it in nature.

Now, most popular views of AI, of course, come from science fiction and the movies,

and I'm personally a big Star Wars fan.

But that tends to be a very human-centric view of intelligence.

When you think of a fish school,

or when I think of a flock of starlings,

that feels like a really different kind of intelligence.

For starters, any one fish is just so tiny

compared to the sheer size of the collective,

so it seems that any one individual

would have a really limited and myopic view of what's going on,

and intelligence isn't really about the individual

but somehow a property of the group itself.

Secondly, and the thing that I still find most remarkable,

is that we know that there are no leaders supervising this fish school.

Instead, this incredible collective mind behavior

is emerging purely from the interactions of one fish and another.

Somehow, there are these interactions or rules of engagement

between neighboring fish

that make it all work out.

So the question for AI then becomes,

what are those rules of engagement that lead to this kind of intelligence,

and of course, can we create our own?

And that's the primary thing that I work on with my team in my lab.

We work on it through theory,

looking at abstract rule systems

and thinking about the mathematics behind it.

We also do it through biology, working closely with experimentalists.

But mostly, we do it through robotics,

where we try to create our own collective systems

that can do the kinds of things that we see in nature,

or at least try to.

One of our first robotic quests along this line

was to create our very own colony of a thousand robots.

So very simple robots,

but they could be programmed to exhibit collective intelligence,

and that's what we were able to do.

So this is what a single robot looks like.

It's quite small, about the size of a quarter,

and you can program how it moves,

but it can also wirelessly communicate with other robots,

and it can measure distances from them.

And so now we can start to program exactly an interaction,

a rule of engagement between neighbors.

And once we have this system,

we can start to program many different kinds of rules of engagement

that you would see in nature.

So for example, spontaneous synchronization,

how audiences are clapping and suddenly start all clapping together,

the fireflies flashing together.

We can program rules for pattern formation,

how cells in a tissue

determine what role they're going to take on

and set the patterns of our bodies.

We can program rules for migration,

and in this way, we're really learning from nature's rules.

But we can also take it a step further.

We can actually take these rules that we've learned from nature

and combine them and create entirely new collective behaviors

of our very own.

So for example,

imagine that you had two different kinds of rules.

So your first rule is a motion rule

where a moving robot can move around other stationary robots.

And your second rule is a pattern rule

where a robot takes on a color based on its two nearest neighbors.

So if I start with a blob of robots in a little pattern seed,

it turns out that these two rules are sufficient for the group

to be able to self-assemble a simple line pattern.

And if I have more complicated pattern rules,

and I design error correction rules,

we can actually create really, really complicated self assemblies,

and here's what that looks like.

So here, you're going to see a thousand robots

that are working together to self-assemble the letter K.

The K is on its side.

And the important thing is that no one is in charge.

So any single robot is only talking to a small number of robots nearby it,

and it's using its motion rule to move around the half-built structure

just looking for a place to fit in based on its pattern rules.

And even though no robot is doing anything perfectly,

the rules are such that we can get the collective to do its goal

robustly together.

And the illusion becomes almost so perfect, you know --

you just start to not even notice that they're individual robots at all,

and it becomes a single entity,

kind of like the school of fish.

So these are robots and rules in two dimensions,

but we can also think about robots and rules in three dimensions.

So what if we could create robots that could build together?

And here, we can take inspiration from social insects.

So if you think about mound-building termites

or you think about army ants,

they create incredible, complex nest structures out of mud

and even out of their own bodies.

And like the system I showed you before,

these insects actually also have pattern rules

that help them determine what to build,

but the pattern can be made out of other insects,

or it could be made out of mud.

And we can use that same idea to create rules for robots.

So here, you're going to see some simulated robots.

So the simulated robot has a motion rule,

which is how it traverses through the structure,

looking for a place to fit in,

and it has pattern rules where it looks at groups of blocks

to decide whether to place a block.

And with the right motion rules and the right pattern rules,

we can actually get the robots to build whatever we want.

And of course, everybody wants their own tower.

(Laughter)

So once we have these rules,

we can start to create the robot bodies that go with these rules.

So here, you see a robot that can climb over blocks,

but it can also lift and move these blocks

and it can start to edit the very structure that it's on.

But with these rules,

this is really only one kind of robot body that you could imagine.

You could imagine many different kinds of robot bodies.

So if you think about robots that maybe could move sandbags

and could help build levees,

or we could think of robots that built out of soft materials

and worked together to shore up a collapsed building --

so just the same kind of rules in different kinds of bodies.

Or if, like my group, you are completely obsessed with army ants,

then maybe one day we can make robots that can climb over literally anything

including other members of their tribe,

and self-assemble things out of their own bodies.

Once you understand the rules,

just many different kinds of robot visions become possible.

And coming back to the snorkeling trip,

we actually understand a great deal about the rules that fish schools use.

So if we can invent the bodies to go with that,

then maybe there is a future

where I and my group will get to snorkel with a fish school of our own creation.

Each of these systems that I showed you

brings us closer to having the mathematical and the conceptual tools

to create our own versions of collective power,

and this can enable many different kinds of future applications,

whether you think about robots that build flood barriers

or you think about robotic bee colonies that could pollinate crops

or underwater schools of robots that monitor coral reefs,

or if we reach for the stars and we thinking about programming

constellations of satellites.

In each of these systems,

being able to understand how to design the rules of engagement

and being able to create good collective behavior

becomes a key to realizing these visions.

So, so far I've talked about rules for insects and for fish

and for robots,

but what about the rules that apply to our own human collective?

And the last thought that I'd like to leave you with

is that science is of course itself

an incredible manifestation of collective intelligence,

but unlike the beautiful fish schools that I study,

I feel we still have a much longer evolutionary path to walk.

So in addition to working on improving the science of robot collectives,

I also work on creating robots and thinking about rules

that will improve our own scientific collective.

There's this saying that I love:

who does science determines what science gets done.

Imagine a society

where we had rules of engagement

where every child grew up believing that they could stand here

and be a technologist of the future,

or where every adult

believed that they had the ability not just to understand but to change

how science and technology impacts their everyday lives.

What would that society look like?

I believe that we can do that.

I believe that we can choose our rules,

and we engineer not just robots

but we can engineer our own human collective,

and if we do and when we do, it will be beautiful.

Thank you.

(Applause)