Wii remote hacks | Johnny Lee

So, as researchers, something that we often do is use immense resources

to achieve certain capabilities, or achieve certain goals.

And this is essential to the progress of science,

or exploration of what is possible.

But it creates this unfortunate situation

where a tiny, tiny fraction of the world

can actually participate in this exploration

or can benefit from that technology.

Something that motivates me,

and gets me really excited about my research,

is when I see simple opportunities

to drastically change that distribution and make the technology accessible

to a much wider percentage of the population.

I'm going to show you two videos that have gotten a lot of attention

that I think embody this philosophy.

And they actually use the Nintendo Wii Remote.

For those of you who aren't familiar with this device,

it's a $40 video game controller.

And it's mostly advertised for its motion-sensing capabilities:

so you can swing a tennis racket, or hit a baseball bat.

But what actually interests me a lot more

is the fact that in the tip of each controller

is a relatively high-performing infrared camera.

And I'm going to show you two demos of why this is useful.

So here, I have my computer set up with the projector,

and I have a Wii Remote sitting on top of it.

And, for example, if you're in a school that doesn't have a lot money,

probably a lot of schools,

or if you're in an office environment, and you want an interactive whiteboard,

normally these cost about two to three thousand dollars.

So I'm going to show you how to create one with a Wii Remote.

Now, this requires another piece of hardware,

which is this infrared pen.

You can probably make this yourself for about five dollars

with a quick trip to the Radio Shack.

It's got a battery, a button and an infrared LED --

you guys can't see it --

but it turns on whenever I push the button.

Now, what this means is that if I run this piece of software,

the camera sees the infrared dot,

and I can register the location of the camera pixels

to the projector pixels.

And now this is like an interactive whiteboard surface.

(Applause)

So for about $50 of hardware, you can have your own whiteboard.

This is Adobe Photoshop.

(Applause)

Thank you.

(Laughter)

The software for this I've actually put on my website

and have let people download it for free.

In the three months this project has been public,

it's been downloaded over half a million times.

So teachers and students all around the world

are already using this.

(Applause)

Although it does do it for 50 dollars,

there are some limitations of this approach.

You get about 80 percent of the way there, for one percent of the cost.

Another nice thing is that a camera can see multiple dots,

so this is actually a multi-touch, interactive whiteboard system as well.

(Applause)

For the second demo,

I have this Wii Remote that's actually next to the TV.

So it's pointing away from the display, rather than pointing at the display.

And why this is interesting

is that if you put on, say, a pair of safety glasses,

that have two infrared dots in them,

they are going to give the computer an approximation of your head location.

And why this is interesting

is I have this sort of application running on the computer monitor,

which has a 3D room, with some targets floating in it.

And you can see that it looks like a 3D room.

kind of like a video game, it sort of looks 3D,

but for the most part, the image looks pretty flat,

and bound to the surface of the screen.

But if we turn on head tracking --

the computer can change the image that's on the screen

and make it respond to the head movements.

So let's switch back to that.

(Laughter)

(Applause)

So this has actually been a little bit startling

to the game-development community.

(Laughter)

Because this is about 10 dollars of additional hardware

if you already have a Nintendo Wii.

So I'm looking forward to seeing some games,

and actually Louis Castle, that's him down there,

last week announced that Electronic Arts, one of the largest game publishers,

is releasing a game in May that has a little Easter egg feature

for supporting this type of head tracking.

And that's from less than five months

from a prototype in my lab to a major commercial product.

(Applause)

Thank you.

(Applause)

But actually, to me, what's almost more interesting

than either of these two projects

is how people actually found out about them.

YouTube has really changed the way, or changed the speed,

in which a single individual

can actually spread an idea around the world.

I'm doing some research in my lab with a video camera,

and within the first week, a million people had seen this work,

and literally within days, engineers, teachers and students

from around the world were already posting their own YouTube videos

of them using my system or derivatives of this work.

So I hope to see more of that in the future,

and hope online video distribution to be embraced by the research community.

So thank you very much.

(Applause)