Golan Levin makes art that looks back at you

56,800 views ・ 2009-07-30

TED


Please double-click on the English subtitles below to play the video.

00:12
Hello! My name is Golan Levin.
0
12160
3000
00:15
I'm an artist and an engineer,
1
15160
2000
00:17
which is, increasingly, a more common kind of hybrid.
2
17160
2000
00:19
But I still fall into this weird crack
3
19160
3000
00:22
where people don't seem to understand me.
4
22160
2000
00:24
And I was looking around and I found this wonderful picture.
5
24160
4000
00:28
It's a letter from "Artforum" in 1967
6
28160
3000
00:31
saying "We can't imagine ever doing a special issue
7
31160
3000
00:34
on electronics or computers in art." And they still haven't.
8
34160
3000
00:37
And lest you think that you all, as the digerati, are more enlightened,
9
37160
5000
00:42
I went to the Apple iPhone app store the other day.
10
42160
3000
00:45
Where's art? I got productivity. I got sports.
11
45160
4000
00:49
And somehow the idea that one would want to make art for the iPhone,
12
49160
4000
00:53
which my friends and I are doing now,
13
53160
2000
00:55
is still not reflected in our understanding
14
55160
3000
00:58
of what computers are for.
15
58160
2000
01:00
So, from both directions, there is kind of, I think, a lack of understanding
16
60160
2000
01:02
about what it could mean to be an artist who uses the materials
17
62160
2000
01:04
of his own day, or her own day,
18
64160
2000
01:06
which I think artists are obliged to do,
19
66160
2000
01:08
is to really explore the expressive potential of the new tools that we have.
20
68160
4000
01:12
In my own case, I'm an artist,
21
72160
2000
01:14
and I'm really interested in
22
74160
2000
01:16
expanding the vocabulary of human action,
23
76160
2000
01:18
and basically empowering people through interactivity.
24
78160
3000
01:21
I want people to discover themselves as actors,
25
81160
3000
01:24
as creative actors, by having interactive experiences.
26
84160
4000
01:28
A lot of my work is about trying to get away from this.
27
88160
3000
01:31
This a photograph of the desktop of a student of mine.
28
91160
2000
01:33
And when I say desktop, I don't just mean
29
93160
2000
01:35
the actual desk where his mouse has worn away the surface of the desk.
30
95160
3000
01:38
If you look carefully, you can even see
31
98160
2000
01:40
a hint of the Apple menu, up here in the upper left,
32
100160
3000
01:43
where the virtual world has literally
33
103160
2000
01:45
punched through to the physical.
34
105160
2000
01:47
So this is, as Joy Mountford once said,
35
107160
4000
01:51
"The mouse is probably the narrowest straw
36
111160
2000
01:53
you could try to suck all of human expression through."
37
113160
2000
01:55
(Laughter)
38
115160
3000
01:58
And the thing I'm really trying to do is enabling people to have more rich
39
118160
3000
02:01
kinds of interactive experiences.
40
121160
2000
02:03
How can we get away from the mouse and use our full bodies
41
123160
2000
02:05
as a way of exploring aesthetic experiences,
42
125160
3000
02:08
not necessarily utilitarian ones.
43
128160
2000
02:10
So I write software. And that's how I do it.
44
130160
3000
02:13
And a lot of my experiences
45
133160
2000
02:15
resemble mirrors in some way.
46
135160
2000
02:17
Because this is, in some sense, the first way,
47
137160
2000
02:19
that people discover their own potential as actors,
48
139160
2000
02:21
and discover their own agency.
49
141160
2000
02:23
By saying "Who is that person in the mirror? Oh it's actually me."
50
143160
3000
02:26
And so, to give an example,
51
146160
2000
02:28
this is a project from last year,
52
148160
2000
02:30
which is called the Interstitial Fragment Processor.
53
150160
2000
02:32
And it allows people to explore the negative shapes that they create
54
152160
4000
02:36
when they're just going about their everyday business.
55
156160
3000
02:53
So as people make shapes with their hands or their heads
56
173160
2000
02:55
and so forth, or with each other,
57
175160
2000
02:57
these shapes literally produce sounds and drop out of thin air --
58
177160
3000
03:00
basically taking what's often this, kind of, unseen space,
59
180160
4000
03:04
or this undetected space, and making it something real,
60
184160
3000
03:07
that people then can appreciate and become creative with.
61
187160
3000
03:10
So again, people discover their creative agency in this way.
62
190160
3000
03:13
And their own personalities come out
63
193160
2000
03:15
in totally unique ways.
64
195160
3000
03:18
So in addition to using full-body input,
65
198160
3000
03:21
something that I've explored now, for a while,
66
201160
2000
03:23
has been the use of the voice,
67
203160
2000
03:25
which is an immensely expressive system for us, vocalizing.
68
205160
4000
03:29
Song is one of our oldest ways
69
209160
2000
03:31
of making ourselves heard and understood.
70
211160
3000
03:34
And I came across this fantastic research by Wolfgang Köhler,
71
214160
2000
03:36
the so-called father of gestalt psychology, from 1927,
72
216160
4000
03:40
who submitted to an audience like yourselves
73
220160
2000
03:42
the following two shapes.
74
222160
2000
03:44
And he said one of them is called Maluma.
75
224160
2000
03:46
And one of them is called Taketa. Which is which?
76
226160
2000
03:48
Anyone want to hazard a guess?
77
228160
4000
03:52
Maluma is on top. Yeah. So.
78
232160
2000
03:54
As he says here, most people answer without any hesitation.
79
234160
3000
03:57
So what we're really seeing here is a phenomenon
80
237160
2000
03:59
called phonaesthesia,
81
239160
2000
04:01
which is a kind of synesthesia that all of you have.
82
241160
2000
04:03
And so, whereas Dr. Oliver Sacks has talked about
83
243160
2000
04:05
how perhaps one person in a million
84
245160
2000
04:07
actually has true synesthesia,
85
247160
2000
04:09
where they hear colors or taste shapes, and things like this,
86
249160
2000
04:11
phonaesthesia is something we can all experience to some extent.
87
251160
2000
04:13
It's about mappings between different perceptual domains,
88
253160
3000
04:16
like hardness, sharpness, brightness and darkness,
89
256160
3000
04:19
and the phonemes that we're able to speak with.
90
259160
2000
04:21
So 70 years on, there's been some research where
91
261160
2000
04:23
cognitive psychologists have actually sussed out
92
263160
2000
04:25
the extent to which, you know,
93
265160
2000
04:27
L, M and B are more associated with shapes that look like this,
94
267160
4000
04:31
and P, T and K are perhaps more associated with shapes like this.
95
271160
4000
04:35
And here we suddenly begin to have a mapping between curvature
96
275160
2000
04:37
that we can exploit numerically,
97
277160
2000
04:39
a relative mapping between curvature and shape.
98
279160
3000
04:42
So it occurred to me, what happens if we could run these backwards?
99
282160
3000
04:45
And thus was born the project called Remark,
100
285160
2000
04:47
which is a collaboration with Zachary Lieberman
101
287160
2000
04:49
and the Ars Electronica Futurelab.
102
289160
2000
04:51
And this is an interactive installation which presents
103
291160
2000
04:53
the fiction that speech casts visible shadows.
104
293160
2000
04:55
So the idea is you step into a kind of a magic light.
105
295160
3000
04:58
And as you do, you see the shadows of your own speech.
106
298160
3000
05:01
And they sort of fly away, out of your head.
107
301160
2000
05:03
If a computer speech recognition system
108
303160
3000
05:06
is able to recognize what you're saying, then it spells it out.
109
306160
4000
05:10
And if it isn't then it produces a shape which is very phonaesthetically
110
310160
2000
05:12
tightly coupled to the sounds you made.
111
312160
2000
05:14
So let's bring up a video of that.
112
314160
3000
06:03
(Applause)
113
363160
2000
06:05
Thanks. So. And this project here,
114
365160
3000
06:08
I was working with the great abstract vocalist, Jaap Blonk.
115
368160
3000
06:11
And he is a world expert in performing "The Ursonate,"
116
371160
3000
06:14
which is a half-an-hour nonsense poem
117
374160
2000
06:16
by Kurt Schwitters, written in the 1920s,
118
376160
2000
06:18
which is half an hour of very highly patterned nonsense.
119
378160
4000
06:22
And it's almost impossible to perform.
120
382160
2000
06:24
But Jaap is one of the world experts in performing it.
121
384160
3000
06:27
And in this project we've developed
122
387160
2000
06:29
a form of intelligent real-time subtitles.
123
389160
3000
06:32
So these are our live subtitles,
124
392160
3000
06:35
that are being produced by a computer that knows the text of "The Ursonate" --
125
395160
3000
06:38
fortunately Jaap does too, very well --
126
398160
3000
06:41
and it is delivering that text at the same time as Jaap is.
127
401160
5000
06:53
So all the text you're going to see
128
413160
2000
06:55
is real-time generated by the computer,
129
415160
2000
06:57
visualizing what he's doing with his voice.
130
417160
3000
08:10
Here you can see the set-up where there is a screen with the subtitles behind him.
131
490160
3000
08:34
Okay. So ...
132
514160
2000
08:36
(Applause)
133
516160
5000
08:41
The full videos are online if you are interested.
134
521160
2000
08:43
I got a split reaction to that during the live performance,
135
523160
2000
08:45
because there is some people who understand
136
525160
2000
08:47
live subtitles are a kind of an oxymoron,
137
527160
2000
08:49
because usually there is someone making them afterwards.
138
529160
3000
08:52
And then a bunch of people who were like, "What's the big deal?
139
532160
3000
08:55
I see subtitles all the time on television."
140
535160
2000
08:57
You know? They don't imagine the person in the booth, typing it all.
141
537160
3000
09:00
So in addition to the full body, and in addition to the voice,
142
540160
3000
09:03
another thing that I've been really interested in,
143
543160
2000
09:05
most recently, is the use of the eyes,
144
545160
2000
09:07
or the gaze, in terms of how people relate to each other.
145
547160
4000
09:11
It's a really profound amount of nonverbal information
146
551160
2000
09:13
that's communicated with the eyes.
147
553160
2000
09:15
And it's one of the most interesting technical challenges
148
555160
2000
09:17
that's very currently active in the computer sciences:
149
557160
2000
09:19
being able to have a camera that can understand,
150
559160
2000
09:21
from a fairly big distance away,
151
561160
2000
09:23
how these little tiny balls are actually pointing in one way or another
152
563160
3000
09:26
to reveal what you're interested in,
153
566160
2000
09:28
and where your attention is directed.
154
568160
2000
09:30
So there is a lot of emotional communication that happens there.
155
570160
3000
09:33
And so I've been beginning, with a variety of different projects,
156
573160
4000
09:37
to understand how people can relate to machines with their eyes.
157
577160
3000
09:40
And basically to ask the questions:
158
580160
3000
09:43
What if art was aware that we were looking at it?
159
583160
5000
09:48
How could it respond, in a way,
160
588160
2000
09:50
to acknowledge or subvert the fact that we're looking at it?
161
590160
3000
09:53
And what could it do if it could look back at us?
162
593160
3000
09:56
And so those are the questions that are happening in the next projects.
163
596160
2000
09:58
In the first one which I'm going to show you, called Eyecode,
164
598160
3000
10:01
it's a piece of interactive software
165
601160
2000
10:03
in which, if we read this little circle,
166
603160
2000
10:05
"the trace left by the looking of the previous observer
167
605160
3000
10:08
looks at the trace left by the looking of previous observer."
168
608160
3000
10:11
The idea is that it's an image wholly constructed
169
611160
2000
10:13
from its own history of being viewed
170
613160
2000
10:15
by different people in an installation.
171
615160
2000
10:17
So let me just switch over so we can do the live demo.
172
617160
5000
10:22
So let's run this and see if it works.
173
622160
4000
10:26
Okay. Ah, there is lots of nice bright video.
174
626160
3000
10:29
There is just a little test screen that shows that it's working.
175
629160
2000
10:31
And what I'm just going to do is -- I'm going to hide that.
176
631160
2000
10:33
And you can see here that what it's doing
177
633160
2000
10:35
is it's recording my eyes every time I blink.
178
635160
3000
10:44
Hello? And I can ... hello ... okay.
179
644160
4000
10:48
And no matter where I am, what's really going on here
180
648160
2000
10:50
is that it's an eye-tracking system that tries to locate my eyes.
181
650160
3000
10:53
And if I get really far away I'm blurry.
182
653160
2000
10:55
You know, you're going to have these kind of blurry spots like this
183
655160
2000
10:57
that maybe only resemble eyes in a very very abstract way.
184
657160
3000
11:00
But if I come up really close and stare directly at the camera
185
660160
3000
11:03
on this laptop then you'll see these nice crisp eyes.
186
663160
2000
11:05
You can think of it as a way of, sort of, typing, with your eyes.
187
665160
4000
11:09
And what you're typing are recordings of your eyes
188
669160
2000
11:11
as you're looking at other peoples' eyes.
189
671160
2000
11:13
So each person is looking at the looking
190
673160
3000
11:16
of everyone else before them.
191
676160
2000
11:18
And this exists in larger installations
192
678160
2000
11:20
where there are thousands and thousands of eyes
193
680160
2000
11:22
that people could be staring at,
194
682160
2000
11:24
as you see who's looking at the people looking
195
684160
2000
11:26
at the people looking before them.
196
686160
2000
11:28
So I'll just add a couple more. Blink. Blink.
197
688160
3000
11:31
And you can see, just once again, how it's sort of finding my eyes
198
691160
3000
11:34
and doing its best to estimate when it's blinking.
199
694160
3000
11:37
Alright. Let's leave that.
200
697160
2000
11:39
So that's this kind of recursive observation system.
201
699160
3000
11:42
(Applause)
202
702160
2000
11:44
Thank you.
203
704160
2000
11:46
The last couple pieces I'm going to show
204
706160
2000
11:48
are basically in the new realm of robotics -- for me, new for me.
205
708160
2000
11:50
It's called Opto-Isolator.
206
710160
2000
11:52
And I'm going to show a video of the older version of it,
207
712160
3000
11:55
which is just a minute long. Okay.
208
715160
2000
12:06
In this case, the Opto-Isolator is blinking
209
726160
2000
12:08
in response to one's own blinks.
210
728160
2000
12:10
So it blinks one second after you do.
211
730160
3000
12:13
This is a device which is intended to reduce
212
733160
3000
12:16
the phenomenon of gaze down to the simplest possible materials.
213
736160
3000
12:19
Just one eye,
214
739160
2000
12:21
looking at you, and eliminating everything else about a face,
215
741160
2000
12:23
but just to consider gaze in an isolated way
216
743160
3000
12:26
as a kind of, as an element.
217
746160
3000
12:29
And at the same time, it attempts to engage in what you might call
218
749160
3000
12:32
familiar psycho-social gaze behaviors.
219
752160
2000
12:34
Like looking away if you look at it too long
220
754160
2000
12:36
because it gets shy,
221
756160
2000
12:38
or things like that.
222
758160
3000
12:41
Okay. So the last project I'm going to show
223
761160
3000
12:44
is this new one called Snout.
224
764160
3000
12:47
(Laughter)
225
767160
2000
12:49
It's an eight-foot snout,
226
769160
2000
12:51
with a googly eye.
227
771160
2000
12:53
(Laughter)
228
773160
1000
12:54
And inside it's got an 800-pound robot arm
229
774160
3000
12:57
that I borrowed,
230
777160
2000
12:59
(Laughter)
231
779160
1000
13:00
from a friend.
232
780160
2000
13:02
(Laughter)
233
782160
1000
13:03
It helps to have good friends.
234
783160
2000
13:05
I'm at Carnegie Mellon; we've got a great Robotics Institute there.
235
785160
3000
13:08
I'd like to show you thing called Snout, which is --
236
788160
2000
13:10
The idea behind this project is to
237
790160
2000
13:12
make a robot that appears as if it's continually surprised to see you.
238
792160
4000
13:16
(Laughter)
239
796160
4000
13:20
The idea is that basically --
240
800160
2000
13:22
if it's constantly like "Huh? ... Huh?"
241
802160
2000
13:24
That's why its other name is Doubletaker, Taker of Doubles.
242
804160
4000
13:28
It's always kind of doing a double take: "What?"
243
808160
2000
13:30
And the idea is basically, can it look at you
244
810160
2000
13:32
and make you feel as if like,
245
812160
2000
13:34
"What? Is it my shoes?"
246
814160
2000
13:36
"Got something on my hair?" Here we go. Alright.
247
816160
3000
14:10
Checking him out ...
248
850160
2000
14:20
For you nerds, here's a little behind-the-scenes.
249
860160
2000
14:22
It's got a computer vision system,
250
862160
2000
14:24
and it tries to look at the people who are moving around the most.
251
864160
3000
14:39
Those are its targets.
252
879160
2000
14:42
Up there is the skeleton,
253
882160
2000
14:44
which is actually what it's trying to do.
254
884160
3000
14:54
It's really about trying to create a novel body language for a new creature.
255
894160
3000
14:57
Hollywood does this all the time, of course.
256
897160
2000
14:59
But also have the body language communicate something
257
899160
2000
15:01
to the person who is looking at it.
258
901160
2000
15:03
This language is communicating that it is surprised to see you,
259
903160
2000
15:05
and it's interested in looking at you.
260
905160
3000
15:08
(Laughter)
261
908160
2000
15:10
(Applause)
262
910160
9000
15:19
Thank you very much. That's all I've got for today.
263
919160
2000
15:21
And I'm really happy to be here. Thank you so much.
264
921160
3000
15:24
(Applause)
265
924160
3000
About this website

This site will introduce you to YouTube videos that are useful for learning English. You will see English lessons taught by top-notch teachers from around the world. Double-click on the English subtitles displayed on each video page to play the video from there. The subtitles scroll in sync with the video playback. If you have any comments or requests, please contact us using this contact form.

https://forms.gle/WvT1wiN1qDtmnspy7