Rana el Kaliouby: This app knows how you feel — from the look on your face

137,190 views

2015-06-15 ・ TED


New videos

Rana el Kaliouby: This app knows how you feel — from the look on your face

137,190 views ・ 2015-06-15

TED


Please double-click on the English subtitles below to play the video.

00:12
Our emotions influence every aspect of our lives,
0
12556
4017
00:16
from our health and how we learn, to how we do business and make decisions,
1
16573
3576
00:20
big ones and small.
2
20149
1773
00:22
Our emotions also influence how we connect with one another.
3
22672
3490
00:27
We've evolved to live in a world like this,
4
27132
3976
00:31
but instead, we're living more and more of our lives like this --
5
31108
4319
00:35
this is the text message from my daughter last night --
6
35427
3134
00:38
in a world that's devoid of emotion.
7
38561
2740
00:41
So I'm on a mission to change that.
8
41301
1951
00:43
I want to bring emotions back into our digital experiences.
9
43252
4091
00:48
I started on this path 15 years ago.
10
48223
3077
00:51
I was a computer scientist in Egypt,
11
51300
2066
00:53
and I had just gotten accepted to a Ph.D. program at Cambridge University.
12
53366
4505
00:57
So I did something quite unusual
13
57871
2113
00:59
for a young newlywed Muslim Egyptian wife:
14
59984
4225
01:05
With the support of my husband, who had to stay in Egypt,
15
65599
2999
01:08
I packed my bags and I moved to England.
16
68598
3018
01:11
At Cambridge, thousands of miles away from home,
17
71616
3228
01:14
I realized I was spending more hours with my laptop
18
74844
3413
01:18
than I did with any other human.
19
78257
2229
01:20
Yet despite this intimacy, my laptop had absolutely no idea how I was feeling.
20
80486
4853
01:25
It had no idea if I was happy,
21
85339
3211
01:28
having a bad day, or stressed, confused,
22
88550
2988
01:31
and so that got frustrating.
23
91538
2922
01:35
Even worse, as I communicated online with my family back home,
24
95600
5231
01:41
I felt that all my emotions disappeared in cyberspace.
25
101421
3282
01:44
I was homesick, I was lonely, and on some days I was actually crying,
26
104703
5155
01:49
but all I had to communicate these emotions was this.
27
109858
4928
01:54
(Laughter)
28
114786
2020
01:56
Today's technology has lots of I.Q., but no E.Q.;
29
116806
4974
02:01
lots of cognitive intelligence, but no emotional intelligence.
30
121780
3176
02:04
So that got me thinking,
31
124956
2197
02:07
what if our technology could sense our emotions?
32
127153
3624
02:10
What if our devices could sense how we felt and reacted accordingly,
33
130777
4076
02:14
just the way an emotionally intelligent friend would?
34
134853
3013
02:18
Those questions led me and my team
35
138666
3564
02:22
to create technologies that can read and respond to our emotions,
36
142230
4377
02:26
and our starting point was the human face.
37
146607
3090
02:30
So our human face happens to be one of the most powerful channels
38
150577
3173
02:33
that we all use to communicate social and emotional states,
39
153750
4016
02:37
everything from enjoyment, surprise,
40
157766
3010
02:40
empathy and curiosity.
41
160776
4203
02:44
In emotion science, we call each facial muscle movement an action unit.
42
164979
4928
02:49
So for example, action unit 12,
43
169907
2925
02:52
it's not a Hollywood blockbuster,
44
172832
2038
02:54
it is actually a lip corner pull, which is the main component of a smile.
45
174870
3442
02:58
Try it everybody. Let's get some smiles going on.
46
178312
2988
03:01
Another example is action unit 4. It's the brow furrow.
47
181300
2654
03:03
It's when you draw your eyebrows together
48
183954
2238
03:06
and you create all these textures and wrinkles.
49
186192
2267
03:08
We don't like them, but it's a strong indicator of a negative emotion.
50
188459
4295
03:12
So we have about 45 of these action units,
51
192754
2206
03:14
and they combine to express hundreds of emotions.
52
194960
3390
03:18
Teaching a computer to read these facial emotions is hard,
53
198350
3901
03:22
because these action units, they can be fast, they're subtle,
54
202251
2972
03:25
and they combine in many different ways.
55
205223
2554
03:27
So take, for example, the smile and the smirk.
56
207777
3738
03:31
They look somewhat similar, but they mean very different things.
57
211515
3753
03:35
(Laughter)
58
215268
1718
03:36
So the smile is positive,
59
216986
3004
03:39
a smirk is often negative.
60
219990
1270
03:41
Sometimes a smirk can make you become famous.
61
221260
3876
03:45
But seriously, it's important for a computer to be able
62
225136
2824
03:47
to tell the difference between the two expressions.
63
227960
2855
03:50
So how do we do that?
64
230815
1812
03:52
We give our algorithms
65
232627
1787
03:54
tens of thousands of examples of people we know to be smiling,
66
234414
4110
03:58
from different ethnicities, ages, genders,
67
238524
3065
04:01
and we do the same for smirks.
68
241589
2811
04:04
And then, using deep learning,
69
244400
1554
04:05
the algorithm looks for all these textures and wrinkles
70
245954
2856
04:08
and shape changes on our face,
71
248810
2580
04:11
and basically learns that all smiles have common characteristics,
72
251390
3202
04:14
all smirks have subtly different characteristics.
73
254592
3181
04:17
And the next time it sees a new face,
74
257773
2368
04:20
it essentially learns that
75
260141
2299
04:22
this face has the same characteristics of a smile,
76
262440
3033
04:25
and it says, "Aha, I recognize this. This is a smile expression."
77
265473
4278
04:30
So the best way to demonstrate how this technology works
78
270381
2800
04:33
is to try a live demo,
79
273181
2136
04:35
so I need a volunteer, preferably somebody with a face.
80
275317
3913
04:39
(Laughter)
81
279230
2334
04:41
Cloe's going to be our volunteer today.
82
281564
2771
04:45
So over the past five years, we've moved from being a research project at MIT
83
285325
4458
04:49
to a company,
84
289783
1156
04:50
where my team has worked really hard to make this technology work,
85
290939
3192
04:54
as we like to say, in the wild.
86
294131
2409
04:56
And we've also shrunk it so that the core emotion engine
87
296540
2670
04:59
works on any mobile device with a camera, like this iPad.
88
299210
3320
05:02
So let's give this a try.
89
302530
2786
05:06
As you can see, the algorithm has essentially found Cloe's face,
90
306756
3924
05:10
so it's this white bounding box,
91
310680
1692
05:12
and it's tracking the main feature points on her face,
92
312372
2571
05:14
so her eyebrows, her eyes, her mouth and her nose.
93
314943
2856
05:17
The question is, can it recognize her expression?
94
317799
2987
05:20
So we're going to test the machine.
95
320786
1671
05:22
So first of all, give me your poker face. Yep, awesome. (Laughter)
96
322457
4186
05:26
And then as she smiles, this is a genuine smile, it's great.
97
326643
2813
05:29
So you can see the green bar go up as she smiles.
98
329456
2300
05:31
Now that was a big smile.
99
331756
1222
05:32
Can you try a subtle smile to see if the computer can recognize?
100
332978
3043
05:36
It does recognize subtle smiles as well.
101
336021
2331
05:38
We've worked really hard to make that happen.
102
338352
2125
05:40
And then eyebrow raised, indicator of surprise.
103
340477
2962
05:43
Brow furrow, which is an indicator of confusion.
104
343439
4249
05:47
Frown. Yes, perfect.
105
347688
4007
05:51
So these are all the different action units. There's many more of them.
106
351695
3493
05:55
This is just a slimmed-down demo.
107
355188
2032
05:57
But we call each reading an emotion data point,
108
357220
3148
06:00
and then they can fire together to portray different emotions.
109
360368
2969
06:03
So on the right side of the demo -- look like you're happy.
110
363337
4653
06:07
So that's joy. Joy fires up.
111
367990
1454
06:09
And then give me a disgust face.
112
369444
1927
06:11
Try to remember what it was like when Zayn left One Direction.
113
371371
4272
06:15
(Laughter)
114
375643
1510
06:17
Yeah, wrinkle your nose. Awesome.
115
377153
4342
06:21
And the valence is actually quite negative, so you must have been a big fan.
116
381495
3731
06:25
So valence is how positive or negative an experience is,
117
385226
2700
06:27
and engagement is how expressive she is as well.
118
387926
2786
06:30
So imagine if Cloe had access to this real-time emotion stream,
119
390712
3414
06:34
and she could share it with anybody she wanted to.
120
394126
2809
06:36
Thank you.
121
396935
2923
06:39
(Applause)
122
399858
4621
06:45
So, so far, we have amassed 12 billion of these emotion data points.
123
405749
5270
06:51
It's the largest emotion database in the world.
124
411019
2611
06:53
We've collected it from 2.9 million face videos,
125
413630
2963
06:56
people who have agreed to share their emotions with us,
126
416593
2600
06:59
and from 75 countries around the world.
127
419193
3205
07:02
It's growing every day.
128
422398
1715
07:04
It blows my mind away
129
424603
2067
07:06
that we can now quantify something as personal as our emotions,
130
426670
3195
07:09
and we can do it at this scale.
131
429865
2235
07:12
So what have we learned to date?
132
432100
2177
07:15
Gender.
133
435057
2331
07:17
Our data confirms something that you might suspect.
134
437388
3646
07:21
Women are more expressive than men.
135
441034
1857
07:22
Not only do they smile more, their smiles last longer,
136
442891
2683
07:25
and we can now really quantify what it is that men and women
137
445574
2904
07:28
respond to differently.
138
448478
2136
07:30
Let's do culture: So in the United States,
139
450614
2290
07:32
women are 40 percent more expressive than men,
140
452904
3204
07:36
but curiously, we don't see any difference in the U.K. between men and women.
141
456108
3645
07:39
(Laughter)
142
459753
2506
07:43
Age: People who are 50 years and older
143
463296
4027
07:47
are 25 percent more emotive than younger people.
144
467323
3436
07:51
Women in their 20s smile a lot more than men the same age,
145
471899
3852
07:55
perhaps a necessity for dating.
146
475751
3839
07:59
But perhaps what surprised us the most about this data
147
479590
2617
08:02
is that we happen to be expressive all the time,
148
482207
3203
08:05
even when we are sitting in front of our devices alone,
149
485410
2833
08:08
and it's not just when we're watching cat videos on Facebook.
150
488243
3274
08:12
We are expressive when we're emailing, texting, shopping online,
151
492217
3010
08:15
or even doing our taxes.
152
495227
2300
08:17
Where is this data used today?
153
497527
2392
08:19
In understanding how we engage with media,
154
499919
2763
08:22
so understanding virality and voting behavior;
155
502682
2484
08:25
and also empowering or emotion-enabling technology,
156
505166
2740
08:27
and I want to share some examples that are especially close to my heart.
157
507906
4621
08:33
Emotion-enabled wearable glasses can help individuals
158
513197
3068
08:36
who are visually impaired read the faces of others,
159
516265
3228
08:39
and it can help individuals on the autism spectrum interpret emotion,
160
519493
4187
08:43
something that they really struggle with.
161
523680
2778
08:47
In education, imagine if your learning apps
162
527918
2859
08:50
sense that you're confused and slow down,
163
530777
2810
08:53
or that you're bored, so it's sped up,
164
533587
1857
08:55
just like a great teacher would in a classroom.
165
535444
2969
08:59
What if your wristwatch tracked your mood,
166
539043
2601
09:01
or your car sensed that you're tired,
167
541644
2693
09:04
or perhaps your fridge knows that you're stressed,
168
544337
2548
09:06
so it auto-locks to prevent you from binge eating. (Laughter)
169
546885
6066
09:12
I would like that, yeah.
170
552951
2717
09:15
What if, when I was in Cambridge,
171
555668
1927
09:17
I had access to my real-time emotion stream,
172
557595
2313
09:19
and I could share that with my family back home in a very natural way,
173
559908
3529
09:23
just like I would've if we were all in the same room together?
174
563437
3971
09:27
I think five years down the line,
175
567408
3142
09:30
all our devices are going to have an emotion chip,
176
570550
2337
09:32
and we won't remember what it was like when we couldn't just frown at our device
177
572887
4064
09:36
and our device would say, "Hmm, you didn't like that, did you?"
178
576951
4249
09:41
Our biggest challenge is that there are so many applications of this technology,
179
581200
3761
09:44
my team and I realize that we can't build them all ourselves,
180
584961
2903
09:47
so we've made this technology available so that other developers
181
587864
3496
09:51
can get building and get creative.
182
591360
2114
09:53
We recognize that there are potential risks
183
593474
4086
09:57
and potential for abuse,
184
597560
2067
09:59
but personally, having spent many years doing this,
185
599627
2949
10:02
I believe that the benefits to humanity
186
602576
2972
10:05
from having emotionally intelligent technology
187
605548
2275
10:07
far outweigh the potential for misuse.
188
607823
3576
10:11
And I invite you all to be part of the conversation.
189
611399
2531
10:13
The more people who know about this technology,
190
613930
2554
10:16
the more we can all have a voice in how it's being used.
191
616484
3177
10:21
So as more and more of our lives become digital,
192
621081
4574
10:25
we are fighting a losing battle trying to curb our usage of devices
193
625655
3498
10:29
in order to reclaim our emotions.
194
629153
2229
10:32
So what I'm trying to do instead is to bring emotions into our technology
195
632622
3914
10:36
and make our technologies more responsive.
196
636536
2229
10:38
So I want those devices that have separated us
197
638765
2670
10:41
to bring us back together.
198
641435
2462
10:43
And by humanizing technology, we have this golden opportunity
199
643897
4588
10:48
to reimagine how we connect with machines,
200
648485
3297
10:51
and therefore, how we, as human beings,
201
651782
4481
10:56
connect with one another.
202
656263
1904
10:58
Thank you.
203
658167
2160
11:00
(Applause)
204
660327
3313
About this website

This site will introduce you to YouTube videos that are useful for learning English. You will see English lessons taught by top-notch teachers from around the world. Double-click on the English subtitles displayed on each video page to play the video from there. The subtitles scroll in sync with the video playback. If you have any comments or requests, please contact us using this contact form.

https://forms.gle/WvT1wiN1qDtmnspy7