Massive-scale online collaboration | Luis von Ahn

311,575 views ใƒป 2011-12-06

TED


ืื ื ืœื—ืฅ ืคืขืžื™ื™ื ืขืœ ื”ื›ืชื•ื‘ื™ื•ืช ื‘ืื ื’ืœื™ืช ืœืžื˜ื” ื›ื“ื™ ืœื”ืคืขื™ืœ ืืช ื”ืกืจื˜ื•ืŸ.

ืžืชืจื’ื: Ran Amitay ืžื‘ืงืจ: Hanan Rosemarin
00:15
How many of you had to fill out a web form
0
15260
2000
ื›ืžื” ืžื›ื ื›ื‘ืจ ื ืืœืฆื• ืœืžืœื ื˜ื•ืคืก ืื™ื ื˜ืจื ื˜ื™
00:17
where you've been asked to read
1
17284
1512
ื‘ื• ื”ืชื‘ืงืฉืชื ืœืงืจื•ื ืกื“ืจืช ืื•ืชื™ื•ืช ืžืขื•ื•ืชื•ืช ื›ื–ืืช?
00:18
a distorted sequence of characters like this?
2
18820
2136
ื›ืžื” ืžื›ื ื—ืฉื‘ื• ืฉื–ื” ืžืžืฉ ืžืžืฉ ืžืขืฆื‘ืŸ?
00:20
How many of you found it really annoying?
3
20980
1953
ื˜ื•ื‘. ื ื”ื“ืจ. ืื– ืื ื™ ื”ืžืฆืืชื™ ืืช ื–ื”.
00:22
(Laughter)
4
22957
1099
00:24
OK, outstanding. So I invented that.
5
24080
1736
(ืฆื—ื•ืง)
00:25
(Laughter)
6
25840
1836
ืื•, ืฉื”ื™ื™ืชื™ ืื—ื“ ืžื”ืžืžืฆื™ืื™ื.
00:27
Or I was one of the people who did it.
7
27700
1856
ืœื“ื‘ืจ ื”ื–ื” ืงื•ืจืื™ื CAPTCHA (ืงืืคืฆ'ื”)
00:29
That thing is called a CAPTCHA.
8
29580
1536
ื•ื”ื•ื ื ืžืฆื ืฉื ืขืœ ืžื ืช ืœื•ื•ื“ื ืฉืืชื, ื”ื™ื™ืฉื•ืช ืฉืžืžืœืืช ืืช ื”ื˜ื•ืคืก,
00:31
And it is there to make sure you, the entity filling out the form,
9
31140
3136
ื”ื™ื ื” ืœืžืขืฉื” ืื“ื ื•ืœื ืื™ื–ื• ืชื•ื›ื ืช ืžื—ืฉื‘
00:34
are a human and not a computer program
10
34300
1856
ืฉื ื›ืชื‘ื” ืข"ืž ืœืฉืœื•ื— ืืช ื”ื˜ื•ืคืก ืžืœื™ื•ื ื™ ื•ืžืœื™ื•ื ื™ ืคืขืžื™ื.
00:36
that was written to submit the form millions of times.
11
36180
2576
ื”ืกื™ื‘ื” ืฉื‘ื’ืœืœื” ื–ื” ืขื•ื‘ื“ ื”ื™ื ืฉืœืื ืฉื™ื,
00:38
The reason it works is because humans, at least non-visually-impaired humans,
12
38780
3656
ืœืคื—ื•ืช ืœืื ืฉื™ื ืœืœื ื‘ืขื™ื•ืช ืจืื™ื™ื”,
ืื™ืŸ ื‘ืขื™ื” ืœืงืจื•ื ืืช ื”ืื•ืชื™ื•ืช ื”ืžืขื•ื•ืชื•ืช ื”ืืœื”,
00:42
have no trouble reading these distorted characters,
13
42460
2416
ืื‘ืœ ืชื•ื›ื ื•ืช ืžื—ืฉื‘ ืคืฉื•ื˜ ืื™ื ืŸ ืขื•ืฉื•ืช ื–ืืช ืžืกืคื™ืง ื˜ื•ื‘ ืขื“ื™ื™ืŸ
00:44
whereas programs can't do it as well yet.
14
44900
1976
00:46
In the case of Ticketmaster,
15
46900
1496
ืื– ืœืžืฉืœ, ื‘ืžืงืจื” ืฉืœ ื˜ื™ืงื˜ืžืืกื˜ืจ,
00:48
the reason you have to type these characters
16
48420
2096
ื”ืกื™ื‘ื” ืฉื‘ื’ืœืœื” ืขืœื™ื›ื ืœื”ืงืœื™ื“ ืืช ื”ืื•ืชื™ื•ืช ื”ืžืขื•ื•ืชื•ืช ื”ืืœื”
00:50
is to prevent scalpers from writing a program
17
50540
2136
ื”ื™ื ืœืžื ื•ืข ืžืกืคืกืจื™ื ืœื›ืชื•ื‘ ืชื•ื›ื ื”
00:52
that can buy millions of tickets, two at a time.
18
52700
2256
ืฉื™ื›ื•ืœื” ืœืงื ื•ืช ืžืœื™ื•ื ื™ ื›ืจื˜ื™ืกื™ื, ืฉื ื™ื™ื ื‘ื›ืœ ืคืขื.
00:54
CAPTCHAs are used all over the Internet.
19
54980
1936
ืงืืคืฆ'ื•ืช ื ืžืฆืื•ืช ื‘ืฉื™ืžื•ืฉ ื‘ื›ืœ ืจื—ื‘ื™ ื”ืื™ื ื˜ืจื ื˜.
00:56
And since they're used so often,
20
56940
1576
ื•ื›ื™ื•ื•ืŸ ืฉื”ืŸ ื‘ืฉื™ืžื•ืฉ ื›ื” ืชื›ื•ืฃ,
00:58
a lot of times the sequence of random characters shown to the user
21
58540
3136
ื”ืจื‘ื” ืคืขืžื™ื ืฆื™ืจื•ืฃ ื”ืื•ืชื™ื•ืช ื”ืืงืจืื™ ื”ืžื•ืฆื’ ืœืžืฉืชืžืฉ
ืื™ื ื• ื›ืœ ื›ืš ืžื–ื”ื™ืจ
01:01
is not so fortunate.
22
61700
1216
01:02
So this is an example from the Yahoo registration page.
23
62940
2656
ื–ื• ื“ื•ื’ืžื” ืžืขืžื•ื“ ื”ื”ืจืฉืžื” ืฉืœ ื™ืื”ื•ื•.
01:05
The random characters that happened to be shown to the user
24
65620
2816
ื”ืื•ืชื™ื•ืช ื”ืืงืจืื™ื•ืช ืฉื”ื•ืฆื’ื• ืœืžืฉืชืžืฉ
ื”ื™ื• W, A, I, T ืืฉืจ, ื›ืžื•ื‘ืŸ, ืžืื™ื™ืชื•ืช ืžื™ืœื”. ("ื”ืžืชืŸ")
01:08
were W, A, I, T, which, of course, spell a word.
25
68460
2696
ืื‘ืœ ื”ื—ืœืง ื”ื›ื™ ื˜ื•ื‘ ื”ื•ื ื”ื”ื•ื“ืขื”
01:11
But the best part is the message
26
71180
2096
01:13
that the Yahoo help desk got about 20 minutes later.
27
73300
2456
ืฉืงื™ื‘ืœื” ืžืขืจื›ืช ื”ืชืžื™ื›ื” ืฉืœ ื™ืื”ื•ื• ื›ืขื‘ื•ืจ 20 ื“ืงื•ืช.
01:15
[Help! I've been waiting for over 20 minutes and nothing happens.]
28
75780
3136
"ื”ืฆื™ืœื•! ืื ื™ ืžื—ื›ื” ื›ื‘ืจ 20 ื“ืงื•ืช, ื•ื›ืœื•ื ืœื ืงื•ืจื”."
01:18
(Laughter)
29
78940
4856
(ืฆื—ื•ืง)
01:23
This person thought they needed to wait.
30
83820
1905
ื”ืžืฉืชืžืฉ ื”ื–ื” ื—ืฉื‘ ืฉืขืœื™ื• ืœื—ื›ื•ืช.
01:25
This, of course, is not as bad as this poor person.
31
85749
2407
ื•ื–ื” ืœื ื›ืœ ื›ืš ื ื•ืจื ื›ืžื• ื”ืžืฉืชืžืฉ ื”ืื•ืžืœืœ ื”ื–ื”.
01:28
(Laughter)
32
88180
2376
(ืฆื—ื•ืง)
01:30
CAPTCHA Project is something that we did at Carnegie Melllon over 10 years ago,
33
90580
3736
ืคืจื•ื™ื™ืงื˜ ืงืคืืฆ'ื” ื”ื•ื ืžืฉื”ื• ืฉื”ืชื—ืœื ื• ื›ืืŸ ื‘ืื•ื ื™ื‘ืจืกื™ื˜ืช ืงืจื ื’ื™-ืžืœื•ืŸ ืœืคื ื™ ื™ื•ืชืจ ืž10 ืฉื ื™ื,
ื•ื”ื•ื ื‘ืฉื™ืžื•ืฉ ื‘ื›ืœ ืžืงื•ื.
01:34
and it's been used everywhere.
34
94340
1456
01:35
Let me now tell you about a project that we did a few years later,
35
95820
3136
ืชื ื• ืœื™ ืœืกืคืจ ืœื›ื ืขืœ ืคืจื•ื™ื™ืงื˜ ืฉืขืจื›ื ื• ื›ืžื” ืฉื ื™ื ืื—"ื›
ืฉื”ื•ื ืžืขื™ื™ืŸ ื”ืฉืœื‘ ื”ืื‘ื•ืœื•ืฆื™ื•ื ื™ ื”ื‘ื ืฉืœ ืงืืคืฆ'ื”.
01:38
which is sort of the next evolution of CAPTCHA.
36
98980
2216
ื–ื” ืคืจื•ื™ื™ืงื˜ ืฉืื ื—ื ื• ืžื›ื ื™ื reCAPTCHA (ืจื™-ืงืืคืฆ'ื”),
01:41
This is a project that we call reCAPTCHA,
37
101220
1976
ื•ื”ื•ื ืžืฉื”ื• ืฉื”ืชื—ืœื ื• ื›ืืŸ, ื‘ืงืจื ื’ื™-ืžืœื•ืŸ,
01:43
which is something that we started here at Carnegie Mellon,
38
103220
2776
ื•ืื– ื”ืคื›ื ื• ืื•ืชื• ืœื—ื‘ืจืช ืกื˜ืืจื˜-ืืค.
01:46
then we turned it into a start-up company.
39
106020
2008
ื•ืื–, ืœืคื ื™ ื›ืฉื ื” ื•ื—ืฆื™,
01:48
And then about a year and a half ago, Google actually acquired this company.
40
108052
3588
ื’ื•ื’ืœ ืจื›ืฉื” ืืช ื”ื—ื‘ืจื”.
ืื– ืชื ื• ืœื™ ืœืกืคืจ ืœื›ื ืžืžื” ื”ืคืจื•ื™ื™ืงื˜ ื”ื–ื” ื”ืชื—ื™ืœ.
01:51
Let me tell you what this project started.
41
111664
2007
ืื– ื”ืคืจื•ื™ื™ืงื˜ ื”ื–ื” ื”ืชื—ื™ืœ ืžื”ื”ื‘ื ื” ื”ื‘ืื”:
01:53
This project started from the following realization:
42
113695
2531
ืžืกืชื‘ืจ ืฉื›200 ืžืœื™ื•ืŸ ืงืคืืฆ'ื•ืช
01:56
It turns out that approximately 200 million CAPTCHAs
43
116250
2437
ืžื•ื–ื ื•ืช ื‘ื›ืœ ื™ื•ื ืข"ื™ ืื ืฉื™ื ืžืกื‘ื™ื‘ ืœืขื•ืœื.
01:58
are typed everyday by people around the world.
44
118711
2151
02:00
When I first heard this, I was quite proud of myself.
45
120886
2484
ื›ื›ืฉืžืขืชื™ ื–ืืช ืœืจืืฉื•ื ื” ื”ื™ื™ืชื™ ื“ื™ื™ ื’ืื” ื‘ืขืฆืžื™.
ื—ืฉื‘ืชื™, ืจืื• ืื™ื–ื• ื”ืฉืคืขื” ื”ื™ืชื” ืœืžื—ืงืจ ืฉืœื™.
02:03
I thought, look at the impact my research has had.
46
123394
2341
ืื‘ืœ ืื– ื”ืชื—ืœืชื™ ืœื”ืจื’ื™ืฉ ืœื ื ืขื™ื.
02:05
But then I started feeling bad.
47
125759
1484
ื•ื–ื” ื”ืขื ื™ื™ืŸ, ื‘ื›ืœ ืคืขื ืฉืืชื ืžืžืœืื™ื ืงืืคืฆ'ื”,
02:07
Here's the thing: each time you type a CAPTCHA,
48
127267
2206
ืืชื ืœืžืขืฉื” ืžื‘ื–ื‘ื–ื™ื 10 ืฉื ื™ื•ืช ืžื–ืžื ื›ื.
02:09
essentially, you waste 10 seconds of your time.
49
129497
2339
02:11
And if you multiply that by 200 million,
50
131860
1936
ื•ืื ืžื›ืคื™ืœื™ื ืืช ื–ื” ื‘200 ืžืœื™ื•ืŸ,
02:13
you get that humanity is wasting about 500,000 hours every day
51
133820
3016
ืžืงื‘ืœื™ื ืฉื”ืื ื•ืฉื•ืช ื›ื•ืœื” ืžื‘ื–ื‘ื–ืช ื› 500,000 ืฉืขื•ืช ื›ืœ ื™ื•ื
02:16
typing these annoying CAPTCHAs.
52
136860
1536
ื‘ื”ื–ื™ื ื” ืืช ื”ืงืืคืฆ'ื•ืช ื”ืžืขืฆื‘ื ื•ืช ื”ืืœื”.
02:18
(Laughter)
53
138420
1016
ื•ืื– ื”ืชื—ืœืชื™ ืœื”ืจื’ื™ืฉ ืœื ื ืขื™ื.
02:19
So then I started feeling bad.
54
139460
1429
02:20
(Laughter)
55
140913
1803
(ืฆื—ื•ืง)
02:22
And then I started thinking, of course, we can't just get rid of CAPTCHAs,
56
142740
3496
ื•ืื– ื—ืฉื‘ืชื™, ื›ืžื•ื‘ืŸ ืื™ ืืคืฉืจ ืคืฉื•ื˜ ืœื”ื™ืคื˜ืจ ืžื”ืงืืคืฆ'ื•ืช ื”ืืœื”,
ื›ื™ ื‘ื˜ื™ื—ื•ืช ื”ืจืฉืช ื“ื™ื™ ืชืœื•ื™ื” ื‘ื”ื
02:26
because the security of the web depends on them.
57
146260
2256
ืื‘ืœ ืื– ื”ืชื—ืœืชื™ ืœื—ืฉื•ื‘, ื”ืื ื™ืฉ ื“ืจืš ื‘ื” ืืคืฉืจ ืœืจืชื•ื ืืช ื”ืžืืžืฅ ื”ื–ื”
02:28
But then I started thinking, can we use this effort
58
148540
2416
02:30
for something that is good for humanity?
59
150980
1936
ืœืžื˜ืจื” ืฉืชื”ื™ื” ื˜ื•ื‘ื” ืœืื ื•ืฉื•ืช?
02:32
So see, here's the thing.
60
152940
1496
ืื– ืจืื•, ื–ื” ื”ืขื ื™ื™ืŸ,
02:34
While you're typing a CAPTCHA, during those 10 seconds,
61
154460
2616
ื‘ื–ืžืŸ ืฉืืชื ืžืงืœื™ื“ื™ื ืงืืคืฆ'ื”, ื‘ืื•ืชืŸ 10 ืฉื ื™ื•ืช,
ืžื•ื—ื›ื ืขื•ืฉื” ื“ื‘ืจ ืžื“ื”ื™ื
02:37
your brain is doing something amazing.
62
157100
1856
02:38
Your brain is doing something that computers cannot yet do.
63
158980
2816
ืžื•ื—ื›ื ืขื•ืฉื” ื“ื‘ืจ ืฉืžื—ืฉื‘ื™ื ืขื“ื™ื™ืŸ ืื™ื ื ืžืกื•ื’ืœื™ื ืšืขืฉื•ืช
ืื– ื”ืื ื ื•ื›ืœ ื ื•ื›ืœ ืœื’ืจื•ื ืœื›ื ืœืขืฉื•ืช ืžืฉื”ื• ืžื•ืขื™ืœ ื‘ืื•ืชืŸ 10 ืฉื ื™ื•ืช?
02:41
So can we get you to do useful work for those 10 seconds?
64
161820
2696
ื“ืจืš ืื—ืจืช ืœืชืืจ ื–ืืช ื”ื™ื,
02:44
Is there some humongous problem that we cannot yet get computers to solve,
65
164540
3496
ื”ืื ื™ืฉ ื‘ืขื™ื” ืขื ืงื™ืช ืฉืขื“ื™ื™ืŸ ืื™ื ื ื• ื™ื›ื•ืœื™ื ืœืคืชื•ืจ ื‘ืืžืฆืขื•ืช ืžื—ืฉื‘ื™ื,
ืื‘ืœ ืืคืฉืจ ืœื—ืœืงื” ืœื—ืชื™ื›ื•ืช ืงื˜ื ื•ืช ืฉืœ 10 ืฉื ื™ื•ืช
02:48
yet we can split into tiny 10-second chunks
66
168060
2776
02:50
such that each time somebody solves a CAPTCHA,
67
170860
2176
ื›ืš ืฉื‘ื›ืœ ืคืขื ืฉืžื™ืฉื”ื• ืคื•ืชืจ ืงืืคืฆ'ื”
ื”ื•ื ืคื•ืชืจ ื—ืชื™ื›ื” ืงื˜ื ื” ืžื”ื‘ืขื™ื”?
02:53
they solve a little bit of this problem?
68
173060
1936
ื•ื”ืชืฉื•ื‘ื” ืœื›ืš ื”ื™ื "ื›ืŸ", ื•ื–ื” ืžื” ืฉืื ื—ื ื• ืขื•ืฉื™ื ืขื›ืฉื™ื•.
02:55
And the answer to that is "yes," and this is what we're doing now.
69
175020
3136
ื•ืžื” ืฉืื•ืœื™ ืื™ื ื›ื ื™ื•ื“ืขื™ื ื”ื•ื ืฉื‘ื–ืžืŸ ืฉืืชื ืคื•ืชืจื™ื ืงืืคืฆ'ื”,
02:58
Nowadays, while you're typing a CAPTCHA,
70
178180
1936
ืืชื ืœื ืจืง ืžืืžืชื™ื ืืช ื–ื”ื•ืชื›ื ื›ื‘ืŸ-ืื“ื,
03:00
not only are you authenticating yourself as a human,
71
180140
2429
ืื‘ืœ ืœืžืขืฉื” ืืชื ืขื•ื–ืจื™ื ืœื ื• ืœื”ืคื•ืš ืกืคืจื™ื ืœื“ื™ื’ื™ื˜ืœื™ื™ื.
03:02
but in addition you're helping us to digitize books.
72
182593
2443
ืื– ืชื ื• ืœื™ ืœื”ืกื‘ื™ืจ ืื™ืš ื–ื” ืขื•ื‘ื“.
03:05
Let me explain how this works.
73
185060
1456
ืื– ื™ืฉ ื”ืจื‘ื” ืคืจื•ื™ื™ืงื˜ื™ื ืฉืžื ืกื™ื ืœื”ืคื•ืš ืกืคืจื™ื ืœื“ื™ื’ื™ื˜ืœื™ื™ื.
03:06
There's a lot of projects trying to digitize books.
74
186540
2416
ืœื’ื•ื’ืœ ื™ืฉ ืื—ื“. ืœืืจื›ื™ื•ืŸ ื”ืื™ื ื˜ืจื ื˜ ื™ืฉ ืื—ื“.
03:08
Google has one. The Internet Archive has one.
75
188980
2136
ืืžื–ื•ืŸ, ืขื›ืฉื™ื• ืขื ื”ืงื™ื ื“ืœ, ืžื ืกื” ืœื”ืคื•ืš ืกืคืจื™ื ืœื“ื™ื’ื™ื˜ืœื™ื™ื.
03:11
Amazon, with the Kindle, is trying to digitize books.
76
191140
2496
ืื™ืš ืฉื–ื” ืขื•ื‘ื“ ื‘ืขื™ืงืจื•ืŸ
03:13
Basically, the way this works is you start with an old book.
77
193660
3176
ื–ื” ืฉืžืชื—ื™ืœื™ื ืขื ืกืคืจ ื™ืฉืŸ.
03:16
You've seen those things, right?
78
196860
1576
ืจืื™ืชื ื“ื‘ืจ ื›ื–ื”, ื›ืŸ? ื›ืื™ืœื• ืกืคืจ?
03:18
Like a book?
79
198460
1216
(ืฆื—ื•ืง)
03:19
(Laughter)
80
199700
1256
03:20
So you start with a book and then you scan it.
81
200980
2536
ืื– ืžืชื—ื™ืœื™ื ืขื ืกืคืจ, ื•ืื– ืกื•ืจืงื™ื ืื•ืชื•.
ืœืกืจื•ืง ืกืคืจ
03:23
Now, scanning a book
82
203540
1216
03:24
is like taking a digital photograph of every page.
83
204780
2376
ื–ื” ื›ืžื• ืœืฆืœื ืฆื™ืœื•ื ื“ื™ื’ื™ื˜ืœื™ ืฉืœ ื›ืœ ืขืžื•ื“ ื‘ืกืคืจ.
ืžืคื™ืงื™ื ื›ืš ืชืžื•ื ื” ืฉืœ ื›ืœ ืขืžื•ื“ ื‘ืกืคืจ.
03:27
It gives you an image for every page.
84
207180
1816
ื–ืืช ืชืžื•ื ื” ืขื ื˜ืงืกื˜ ืขื‘ื•ืจ ื›ืœ ืขืžื•ื“ ื‘ืกืคืจ.
03:29
This is an image with text for every page of the book.
85
209020
2576
ื”ืฉืœื‘ ื”ื‘ื ื‘ืชื”ืœื™ืš
03:31
The next step in the process is that the computer needs to be able
86
211620
3136
ื”ื•ื ืฉื”ืžื—ืฉื‘ ืฆืจื™ืš ืœื”ืฆืœื™ื— ืœืคืขื ื— ืืช ื›ืœ ื”ืžื™ืœื™ื ื‘ืชืžื•ื ื”.
03:34
to decipher the words in this image.
87
214780
1736
ื–ืืช ืข"ื™ ืฉื™ืžื•ืฉ ื‘ื˜ื›ื ื•ืœื•ื’ื™ื” ื”ื ืงืจืืช OCR (ืื•-ืกื™-ืืจ)
03:36
That's using a technology called OCR, for optical character recognition,
88
216540
3416
ื›ืžืœื•ืžืจ ื–ื™ื”ื•ื™ ืื•ืชื™ื•ืช ืื•ืคื˜ื™,
03:39
which takes a picture of text
89
219980
1416
ืืฉืจ ืžืขื‘ื“ืช ืชืžื•ื ื” ืฉืœ ื˜ืงืกื˜
03:41
and tries to figure out what text is in there.
90
221420
2176
ื•ืžื ืกื” ืœื”ื‘ื™ืŸ ืื™ื–ื” ื˜ืงืกื˜ ื ืžืฆื ื‘ื”.
03:43
Now, the problem is that OCR is not perfect.
91
223620
2656
ืขื›ืฉื™ื•, ื”ื‘ืขื™ื” ื”ื™ื ืฉOCR ืื™ื ื• ืžื•ืฉืœื.
ื‘ื™ื™ื—ื•ื“ ืขื‘ื•ืจ ืกืคืจื™ื ื™ืฉื ื™ื
03:46
Especially for older books
92
226300
1416
03:47
where the ink has faded and the pages have turned yellow,
93
227740
3136
ืฉื‘ื”ื ื”ื“ื™ื• ื“ื”ื” ื•ื”ื“ืคื™ื ื”ืฆื”ื™ื‘ื•,
03:50
OCR cannot recognize a lot of the words.
94
230900
1936
OCR ืœื ืžืฆืœื™ื— ืœื–ื”ื•ืช ืจื‘ื•ืช ืžื”ืžื™ืœื™ื.
03:52
For things that were written more than 50 years ago,
95
232860
2456
ืœืžืฉืœ, ื‘ื“ื‘ืจื™ื ืฉื ื›ืชื‘ื• ืœืคื ื™ ื™ื•ืชืจ ืž50 ืฉื ื”,
ื”ืžื—ืฉื‘ ืื™ื ื• ืžืฆืœื™ื— ืœื–ื”ื•ืช ื›30 ืื—ื•ื– ืžื”ืžื™ืœื™ื.
03:55
the computer cannot recognize about 30 percent of the words.
96
235340
2856
ืื– ืžื” ืฉืื ื—ื ื• ืขื•ืฉื™ื ืขื›ืฉื™ื•
03:58
So now we're taking all of the words that the computer cannot recognize
97
238220
3376
ื–ื” ืฉืื ื—ื ื• ืœื•ืงื—ื™ื ืืช ื›ืœ ื”ืžื™ืœื™ื ืฉื”ืžื—ืฉื‘ ืœื ืžืฆืœื™ื— ืœื–ื”ื•ืช
04:01
and we're getting people to read them for us
98
241620
2256
ื•ืื ื—ื ื• ื ืขื–ืจื™ื ื‘ืื ืฉื™ื ืฉื™ืงืจืื• ืœื ื• ืื•ืชืŸ
04:03
while they're typing a CAPTCHA on the Internet.
99
243900
2216
ื‘ื–ืžืŸ ืฉื”ื ืžืงืœื™ื“ื™ื ืงืืคืฆ'ื” ื‘ืื™ื ื˜ืจื ื˜.
ืื– ื‘ืคืขื ื”ื‘ืื” ืฉืืชื ืžืงืœื™ื“ื™ื ืงืืคืฆ'ื”, ื”ืžื™ืœื™ื ื”ืืœื” ืฉืืชื ืžืงืœื™ื“ื™ื
04:06
So the next time you type a CAPTCHA, these words that you're typing
100
246140
3176
ื”ืŸ ืœืžืขืฉื” ืžื™ืœื™ื ื”ืžื’ื™ืขื•ืช ืžืกืคืจื™ื ืฉื”ื•ืคื›ื™ื ืœื“ื™ื’ื™ื˜ืœื™ื™ื
04:09
are actually words from books that are being digitized
101
249340
2576
04:11
that the computer could not recognize.
102
251940
1856
ืืฉืจ ื”ืžื—ืฉื‘ ืœื ื”ืฆืœื™ื— ืœื–ื”ื•ืช.
04:13
The reason we have two words nowadays instead of one
103
253820
2456
ื•ื”ืกื™ื‘ื” ืฉื‘ื’ืœืœื” ื™ืฉ ืœื ื• ืฉืชื™ ืžื™ืœื™ื ื‘ืžืงื•ื ืื—ืช ื‘ื™ืžื™ื ืืœื”
ื”ื™ื ืฉืื—ืช ืžื”ืžื™ืœื™ื
04:16
is because one of the words
104
256300
1416
04:17
is a word that the system just got out of a book,
105
257740
2576
ื”ื™ื ืžื™ืœื” ืฉื”ืžืขืจื›ืช ืงื™ื‘ืœื” ืžืกืคืจ,
ื”ื™ื ืœื ื–ื™ื”ืชื” ืื•ืชื” ื•ื”ื™ื ื”ื•ืœื›ืช ืœื”ืฆื™ื’ ืœื›ื ืื•ืชื”.
04:20
it didn't know what it was and it's going to present it to you.
106
260340
3016
ืื‘ืœ ื›ื™ื•ื•ืŸ ืฉืื™ื ื” ื™ื•ื“ืขืช ืืช ื”ืชืฉื•ื‘ื” ื”ื™ื ืื™ื ื” ื™ื›ื•ืœื” ืœืฆื™ื™ืŸ ืืช ืชืฉื•ื‘ืชื›ื.
04:23
But since it doesn't know the answer, it cannot grade it.
107
263380
2696
ืื– ืื ื—ื ื• ืžืฆื™ื’ื™ื ืœื›ื ืžื™ืœื” ื ื•ืกืคืช,
04:26
So we give you another word,
108
266100
1376
04:27
for which the system does know the answer.
109
267500
2000
ืžื™ืœื” ืฉืขื‘ื•ืจื” ื”ืžืขืจื›ืช ื™ื•ื“ืขืช ืืช ื”ืชืฉื•ื‘ื”.
04:29
We don't tell you which one's which and we say, please type both.
110
269524
3072
ืื ื—ื ื• ืœื ืžื’ืœื™ื ืœื›ื ืื™ื–ื• ื”ื™ื ืื™ื–ื•, ื•ืžื‘ืงืฉื™ื ืื ื ื”ืงืœื™ื“ื• ืืช ืฉืชื™ื”ืŸ.
ื•ืื ืืชื ืžืงืœื™ื“ื™ื ืืช ื”ืžื™ืœื” ื”ื ื›ื•ื ื”
04:32
And if you type the correct word
111
272620
1575
ืขื‘ื•ืจ ื”ืžื™ืœื” ืฉื”ืžืขืจื›ืช ื™ื•ื“ืขืช ืืช ื”ืชืฉื•ื‘ื” ืฉืœื”,
04:34
for the one for which the system knows the answer,
112
274219
2377
ื”ื™ื ืžื ื™ื—ื” ืฉืืชื ื‘ื ื™ ืื“ื,
04:36
it assumes you are human
113
276620
1256
04:37
and it also gets some confidence that you typed the other word correctly.
114
277900
3456
ื•ื’ื ืฉื•ืื‘ืช ื‘ื™ื˜ื—ื•ืŸ ืžืกื•ื™ื™ื ืฉื”ืงืœื“ืชื ื ื›ื•ืŸ ืืช ื”ืžื™ืœื” ื”ืฉื ื™ื™ื”.
ื•ืื ื ื—ื–ื•ืจ ืขืœ ื”ืชื”ืœื™ืš ืœื›10 ืื ืฉื™ื ืฉื•ื ื™ื
04:41
And if we repeat this process to 10 different people
115
281380
2456
ื•ื›ื•ืœื ื™ืกื›ื™ืžื• ืžื” ื”ืžื™ืœื”,
04:43
and they agree on what the new word is,
116
283860
1896
ืื– ื”ืฆืœื—ื ื• ืœื”ืคื•ืš ืžื™ืœื” ื ื•ืกืคืช ืœื“ื™ื’ื™ื˜ืœื™ืช.
04:45
then we get one more word digitized accurately.
117
285780
2216
ื•ื›ืš ื”ืžืขืจื›ืช ืขื•ื‘ื“ืช.
04:48
So this is how the system works.
118
288020
1576
ื•ื‘ืขื™ืงืจื•ืŸ, ืžืื– ืฉืฉื™ื—ืจืจื ื• ืื•ืชื” ืœืคื ื™ ื›ืฉืœื•ืฉ ืื• ืืจื‘ืข ืฉื ื™ื,
04:49
And since we released it about three or four years ago,
119
289620
2616
ื”ืจื‘ื” ืืชืจื™ื ืขื‘ืจื• ืžืฉื™ืžื•ืฉ
04:52
a lot of websites have started switching from the old CAPTCHA,
120
292260
2936
ื‘ืงืคืืฆ'ื” ื”ื™ืฉื ื” ื‘ื” ืื ืฉื™ื ื‘ื–ื‘ื–ื• ืืช ื–ืžื ื
04:55
where people wasted their time,
121
295220
1536
ืœืงืืคืฆ'ื” ื”ื—ื“ืฉื”, ื‘ื” ืื ืฉื™ื ืขื•ื–ืจื™ื ืœื”ืคื•ืš ืกืคืจื™ื ืœื“ื™ื’ื™ื˜ืœื™ื™ื.
04:56
to the new CAPTCHA where people are helping to digitize books.
122
296780
2936
ืื– ืœื“ื•ื’ืžื, ื˜ื™ืงื˜ืžืืกื˜ืจ (Tickermaster).
04:59
So every time you buy tickets on Ticketmaster,
123
299740
2176
ื‘ื›ืœ ืคืขื ืฉืืชื ืงื•ื ื™ื ื›ืจื˜ื™ืกื™ื ื‘ื˜ื™ืงื˜ ืžืืกื˜ืจ ืืชื ืขื•ื–ืจื™ื ืœื”ืคื•ืš ืกืคืจื™ื ืœื“ื™ื’ื™ื˜ืœื™ื™ื.
05:01
you help to digitize a book.
124
301940
1376
ืคื™ื™ืกื‘ื•ืง: ื‘ื›ืœ ืคืขื ืฉืืชื ืžื•ืกื™ืคื™ื ื—ื‘ืจ ืื• ืขื•ืฉื™ื ืคื•ืง ืœืžื™ืฉื”ื•,
05:03
Facebook: Every time you add a friend or poke somebody,
125
303340
2616
ืืชื ืขื•ื–ืจื™ื ืœื”ืคื•ืš ืกืคืจื™ื ืœื“ื™ื’ื™ื˜ืœื™ื™ื.
05:05
you help to digitize a book.
126
305980
1376
ื˜ื•ื•ื™ื˜ืจ ื•ื›350,000 ืืชืจื™ื ืื—ืจื™ื ืžืฉืชืžืฉื™ื ื›ื•ืœื ื‘ืจื™-ืงืืคืฆ'ื”.
05:07
Twitter and about 350,000 other sites are all using reCAPTCHA.
127
307380
2936
ื•ืœืžืขืฉื”, ืžืกืคืจ ื”ืืชืจื™ื ื”ืžืฉืชืžืฉื™ื ื‘ืจื™-ืงืืคืฆ'ื” ื”ื•ื ื›ื” ื’ื‘ื•ื”
05:10
And the number of sites that are using reCAPTCHA is so high
128
310340
2816
ืฉืžืกืคืจ ื”ืžื™ืœื™ื ืฉืื ื—ื ื• ื”ื•ืคื›ื™ื ืœื“ื™ื’ื™ื˜ืœื™ื•ืช ื”ื•ื ืžืžืฉ ืžืžืฉ ื’ื“ื•ืœ.
05:13
that the number of words we're digitizing per day is really large.
129
313180
3136
ื”ื•ื ื›100 ืžืœื™ื•ืŸ ื‘ื™ื•ื,
05:16
It's about 100 million a day,
130
316340
1416
ืฉื–ื” ืฉื•ื•ื” ืขืจืš ืœื›ืฉื ื™ื™ื ื•ื—ืฆื™ ืžืœื™ื•ืŸ ืกืคืจื™ื ื‘ืฉื ื”.
05:17
which is the equivalent of about two and a half million books a year.
131
317780
3496
ื•ื›ืœ ื–ื” ื ืขืฉื” ืžื™ืœื” ืื—ืช ื‘ื›ืœ ืคืขื
05:21
And this is all being done one word at a time
132
321300
2136
ืข"ื™ ืื ืฉื™ื ืฉืคืฉื•ื˜ ืžืงืœื™ื“ื™ื ืงืืคืฆ'ื” ื‘ืื™ื ื˜ืจื ื˜.
05:23
by just people typing CAPTCHAs on the Internet.
133
323460
2216
(ืชืฉื•ืื•ืช)
05:25
(Applause)
134
325700
6880
05:32
Now, of course,
135
332940
1216
ืขื›ืฉื™ื• ื›ืžื•ื‘ืŸ,
05:34
since we're doing so many words per day,
136
334180
3336
ื›ื™ื•ื•ืŸ ืฉืื—ื ื• ืขื•ืฉื™ื ื›"ื› ื”ืจื‘ื” ืžื™ืœื™ื ื‘ื›ืœ ื™ื•ื,
ื“ื‘ืจื™ื ืžืฆื—ื™ืงื™ื ื™ื›ื•ืœื™ื ืœืงืจื•ืช.
05:37
funny things can happen.
137
337540
1256
05:38
This is especially true because now we're giving people
138
338820
2616
ื•ื–ื” ืขื•ื“ ื™ื•ืชืจ ื ื›ื•ืŸ ื›ื™ ืขื›ืฉื™ื• ืื ื—ื ื• ื ื•ืชื ื™ื ืœืื ืฉื™ื
ืฉืชื™ ืžื™ืœื™ื ืืงืจืื™ื•ืช ื‘ืื ื’ืœื™ืช ืื—ืช ืœื™ื“ ื”ืฉื ื™ื”.
05:41
two randomly chosen English words next to each other.
139
341460
2496
ืื– ื“ื‘ืจื™ื ืžืฆื—ื™ืงื™ื ื™ื›ื•ืœื™ื ืœืงืจื•ืช.
05:43
So funny things can happen.
140
343980
1336
ืœืžืฉืœ, ื”ืฆื’ื ื• ืืช ื”ืžื™ืœื” ื”ื–ืืช.
05:45
For example, we presented this word.
141
345340
1736
ื–ืืช ื”ืžื™ืœื” "ื ื•ืฆืจื™ื" ืื™ืŸ ืขื ื–ื” ื‘ืขื™ื”.
05:47
It's the word "Christians"; there's nothing wrong with it.
142
347100
2736
ืื‘ืœ ืื ืžืฆื™ื’ื™ื ืื•ืชื” ื™ื—ื“ ืขื ืžื™ืœื” ืืงืจืื™ืช ืื—ืจืช,
05:49
But if you present it along with another randomly chosen word,
143
349860
2936
ื“ื‘ืจื™ื ืจืขื™ื ื™ื›ื•ืœื™ื ืœืงืจื•ืช.
05:52
bad things can happen.
144
352820
1336
ืื– ืื ื—ื ื• ืžืงื‘ืœื™ื ืืช ื–ื”. (ื˜ืงืกื˜: ื ื•ืฆืจื™ื ืจืขื™ื)
05:54
So we get this.
145
354180
1216
05:55
[bad Christians]
146
355420
1216
ืื‘ืœ ื–ื” ืขื•ื“ ื™ื•ืชืจ ื’ืจื•ืข, ื›ื™ ื”ืืชืจ ื‘ื• ื”ืจืื ื• ืืช ื–ื”
05:56
But it's even worse, because the website where we showed this
147
356660
2896
ื ืงืจื, ื‘ืžืงืจื”, ืฉื’ืจื™ืจื•ืช ืžืžืœื›ืช ื”ืืœื•ื”ื™ื.
05:59
actually happened to be called The Embassy of the Kingdom of God.
148
359580
3056
(ืฆื—ื•ืง)
06:02
(Laughter)
149
362660
1696
ืื•ืคืก.
06:04
Oops.
150
364380
1216
06:05
(Laughter)
151
365620
3856
(ืฆื—ื•ืง)
ื”ื ื” ืขื•ื“ ืžืงืจื” ืจืข.
06:09
Here's another really bad one.
152
369500
1696
ื’'ื•ืŸ ืื“ื•ืืจื“ืก ื“ื•ื˜ ืงื•ื
06:11
JohnEdwards.com
153
371220
1296
06:12
[Damn liberal]
154
372540
1216
(ื˜ืงืกื˜: ืœื™ื‘ืจืœื™ ืืจื•ืจ)
06:13
(Laughter)
155
373780
4496
(ืฆื—ื•ืง)
ืื– ืื ื—ื ื• ืžืžืฉื™ื›ื™ื ื‘ื›ืœ ื™ื•ื ืœื”ืขืœื™ื‘ ืื ืฉื™ื ืขืœ ื™ืžื™ืŸ ื•ืขืœ ืฉืžืืœ.
06:18
So we keep on insulting people left and right everyday.
156
378300
2816
ืขื›ืฉื™ื•, ืื ื—ื ื• ื›ืžื•ื‘ืŸ ืœื ืจืง ืžืขืœื™ื‘ื™ื ืื ืฉื™ื.
06:21
Of course, we're not just insulting people.
157
381140
2016
ื•ื–ื” ื”ืขื ื™ื™ืŸ, ืžืื– ืฉืื ื—ื ื• ืžืฆื™ื’ื™ื ืฉืชื™ ืžื™ืœื™ื ืืงืจืื™ื•ืช,
06:23
Here's the thing. Since we're presenting two randomly chosen words,
158
383180
3176
ื“ื‘ืจื™ื ืžืขื ื™ื™ื ื™ื ื™ื›ื•ืœื™ื ืœืงืจื•ืช.
06:26
interesting things can happen.
159
386380
1456
06:27
So this actually has given rise to a really big Internet meme
160
387860
4616
ืื– ืœืžืขืฉื” ื ื•ืฆืจ ื›ืืŸ
"ืžื" (ืžื ื”ื’ ื—ื‘ืจืชื™) ืื™ื ื˜ืจื ื˜ื™ ืžืžืฉ ื’ื“ื•ืœ
06:32
that tens of thousands of people have participated in,
161
392500
2536
ืฉืขืฉืจื•ืช ืืœืคื™ ืื ืฉื™ื ื”ืฉืชืชืคื• ื‘ื•,
ืฉื ืงืจื ืื•ืžื ื•ืช ืงืืคืฆ'ื”.
06:35
which is called CAPTCHA art.
162
395060
1656
06:36
I'm sure some of you have heard about it.
163
396740
1976
ืื ื™ ื‘ื˜ื•ื— ืฉื—ืœืงื›ื ืฉืžืขืชื ืขืœ ื–ื”.
06:38
Here's how it works.
164
398740
1256
ื›ืš ื–ื” ืขื•ื‘ื“.
06:40
Imagine you're using the Internet and you see a CAPTCHA
165
400020
2616
ื ืืžืจ ืฉืืชื” ื’ื•ืœืฉ ื‘ืื™ื ื˜ืจื ื˜ ื•ืจืื” ืงืืคืฆ'ื”
06:42
that you think is somewhat peculiar,
166
402660
1736
ืฉื ืจืื™ืช ืœืš ืžื•ื–ืจื”,
06:44
like this CAPTCHA.
167
404420
1216
ื›ืžื• ื–ืืช (ื˜ืงืกื˜: ื˜ื•ืกื˜ืจ ื‘ืœืชื™-ื ืจืื”)
06:45
[invisible toaster]
168
405660
1216
06:46
What you're supposed to do is you take a screenshot of it.
169
406900
2736
ืื– ืžื” ืฉืืชื” ืืžื•ืจ ืœืขืฉื•ืช ื–ื” ืœื”ืขืชื™ืง ืืช ืฆื™ืœื•ื ื”ืžืกืš ืฉืœ ื”ื˜ืงืกื˜
ื•ืื– ื›ืžื•ื‘ืŸ ืœืžืœื ืืช ื”ืงืืคืฆ'ื”
06:49
Then of course, you fill out the CAPTCHA because you help us digitize a book.
170
409660
3656
ื›ื™ ืืชื” ืขื•ื–ืจ ืœื”ืคื•ืš ืกืคืจื™ื ืœื“ื™ื’ื™ื˜ืœื™ื™ื.
ืื‘ืœ ืื–, ืจืืฉื™ืช ืืชื” ืžืขืชื™ืง ืืช ืฆื™ืœื•ื ื”ืžืกืš,
06:53
But first you take a screenshot
171
413340
1496
06:54
and then you draw something that is related to it.
172
414860
2376
ื•ืื– ืืชื” ืžืฆื™ื™ืจ ืžืฉื”ื• ืฉืงืฉื•ืจ ืœื–ื”.
(ืฆื—ื•ืง)
06:57
(Laughter)
173
417260
1696
06:58
That's how it works.
174
418980
1216
ื›ื›ื” ื–ื” ืขื•ื‘ื“.
07:00
(Laughter)
175
420220
1336
07:01
There are tens of thousands of these.
176
421580
2656
ื™ืฉ ืขืฉืจื•ืช ืืœืคื™ื ื›ืืœื”.
07:04
Some of them are very cute.
177
424260
2072
ื—ืœืงื ืžืื•ื“ ื—ืžื•ื“ื™ื (ื˜ืงืกื˜: ืชืคืกืชื™ ืืช ื–ื”)
07:06
[clenched it]
178
426356
1213
(ืฆื—ื•ืง)
07:07
(Laughter)
179
427593
1843
ื—ืœืงื™ื ื™ื•ืชืจ ืžืฆื—ื™ืงื™ื.
07:09
Some of them are funnier.
180
429460
1536
(ื˜ืงืกื˜: ืžื™ื™ืกื“ื™ื ืžืกื˜ื•ืœื™ื)
07:11
[stoned Founders]
181
431020
1216
07:12
(Laughter)
182
432260
4376
(ืฆื—ื•ืง)
07:16
And some of them, like paleontological shvisle ...
183
436660
3429
ื•ื—ืœืงื,
ื›ืžื• ืฉื•ื•ื™ื–ืœ ืคืœืื•ื ื˜ื•ืœื•ื’ื™,
07:20
(Laughter)
184
440113
1923
ืžืฆื™ื’ื™ื ืืช ืกื ื•ืค-ื“ื•ื’.
07:22
they contain Snoop Dogg.
185
442060
1216
07:23
(Laughter)
186
443300
3136
(ืฆื—ื•ืง)
07:26
OK, so this is my favorite number of reCAPTCHA.
187
446460
2576
ืื•ืงื™ื™, ื–ื” ื”ืžืกืคืจ ื”ืื”ื•ื‘ ืขืœื™ื™ ื‘ื™ื•ืชืจ ื‘ืจื™-ืงืืคืฆ'ื”.
ื–ื” ื”ื“ื‘ืจ ืฉืื ื™ ื”ื›ื™ ืื•ื”ื‘ ื‘ื›ืœ ื”ืคืจื•ื™ื™ืงื˜.
07:29
So this is the favorite thing that I like about this whole project.
188
449060
3176
ื–ื” ืžืกืคืจ ื”ืื ืฉื™ื ื”ืฉื•ื ื™ื
07:32
This is the number of distinct people
189
452260
1816
ืืฉืจ ืขื–ืจื• ืœื ื• ืœื”ืคื•ืš ืœื—ื•ืช ืžื™ืœื” ืื—ืช ืœื“ื™ื’ื™ื˜ืœื™ืช ื‘ืขื–ืจืช ืจื™-ืงืืคืฆ'ื”:
07:34
that have helped us digitize at least one word out of a book through reCAPTCHA:
190
454100
3736
750 ืžืœื™ื•ืŸ,
07:37
750 million, a little over 10 percent of the world's population,
191
457860
3056
ืฉื–ื” ืงืฆืช ื™ื•ืชืจ ืž 10 ืื—ื•ื– ืžืื•ื›ืœื•ืกื™ื™ืช ื”ืขื•ืœื,
07:40
has helped us digitize human knowledge.
192
460940
1896
ืฉืขื–ืจื• ืœื ื• ืœื”ืคื•ืš ื™ื“ืข ืื ื•ืฉื™ ืœื“ื™ื’ื™ื˜ืœื™.
07:42
And it is numbers like these that motivate my research agenda.
193
462860
3096
ื•ืžืกืคืจื™ื ื›ืืœื” ื”ื ืืฉืจ ืžื“ืจื‘ื ื™ื ืื•ืชื™ ื‘ืžื—ืงืจ ืฉืœื™.
07:45
So the question that motivates my research is the following:
194
465980
3056
ืื– ื”ืฉืืœื” ืืฉืจ ืžื“ืจื‘ื ืช ืืช ื”ืžื—ืงืจ ืฉืœื™ ื”ื™ื:
ืื ื ื‘ื—ืŸ ืืช ื”ื”ื™ืฉื’ื™ื ื”ื’ื“ื•ืœื™ื ืฉืœ ื”ืื ื•ืฉื•ืช,
07:49
If you look at humanity's large-scale achievements,
195
469060
2416
ื”ื“ื‘ืจื™ื ื”ื’ื“ื•ืœื™ื ื”ืืœื”
07:51
these really big things
196
471500
1216
07:52
that humanity has gotten together and done historically --
197
472740
2715
ืฉื”ืื ื•ืฉื•ืช ื”ืชืื—ื“ื” ื•ื‘ื™ืฆืขื” ื™ื—ื“ื™ื• ื‘ืขื‘ืจ --
07:55
like, for example, building the pyramids of Egypt
198
475479
2477
ืœืžืฉืœ, ื‘ื ื™ื™ืŸ ื”ืคื™ืจืžื™ื“ื•ืช ื‘ืžืฆืจื™ื™ื
07:57
or the Panama Canal
199
477980
1576
ืื• ืชืขืœืช ืคื ืžื”
07:59
or putting a man on the Moon --
200
479580
2056
ืื• ืœื”ื ื—ื™ืช ืื™ืฉ ืขืœ ื”ื™ืจื— --
08:01
there is a curious fact about them,
201
481660
1696
ื™ืฉื ื” ืขื•ื‘ื“ื” ืžืขื ื™ื™ื ืช ื‘ืงืฉืจ ืืœื™ื”ื,
08:03
and it is that they were all done with about the same number of people.
202
483380
3336
ื•ื”ื™ื ืฉื”ื ื›ื•ืœื ื‘ื•ืฆืขื• ื‘ืขื–ืจืช ืื•ืชื• ืžืกืคืจ ืื ืฉื™ื.
ื–ื” ืžื•ื–ืจ, ื›ื•ืœื ื‘ื•ืฆืขื• ื‘ืขื–ืจืช ื› 100,000 ืื™ืฉ.
08:06
It's weird; they were all done with about 100,000 people.
203
486740
2696
ื•ื”ืกื™ื‘ื” ืœื›ืš ื”ื™ื ืฉืœืคื ื™ ื”ืื™ื ื˜ืจื ื˜,
08:09
And the reason for that is because, before the Internet,
204
489460
2656
ืœืชืื ืคืขื™ืœื•ืช ืฉืœ ื™ื•ืชืจ ืž 100,000 ืื™ืฉ,
08:12
coordinating more than 100,000 people,
205
492140
1856
ืฉืœื ืœื“ื‘ืจ ืขืœ ืœืฉืœื ืœื”ื, ื”ื™ื” ืœืžืขืฉื” ื‘ืœืชื™ ืืคืฉืจื™.
08:14
let alone paying them, was essentially impossible.
206
494020
3016
ืื‘ืœ ืขื›ืฉื™ื•, ื‘ืขื–ืจืช ื”ืื™ื ื˜ืจื ื˜, ื”ืจืื™ืชื™ ืœื›ื ืคืจื•ื™ื™ืงื˜
08:17
But now with the Internet, I've just shown you a project
207
497060
2656
ื‘ื• ื”ืฆืœื—ื ื• ืœื”ืขื–ืจ ื‘ 750 ืžืœื™ื•ืŸ ืื ืฉื™ื
08:19
where we've gotten 750 million people to help us digitize human knowledge.
208
499740
3496
ืœื”ืคื•ืš ื™ื“ืข ืื ื•ืฉื™ ืœื“ื™ื’ื™ื˜ืœื™.
ืื– ื”ืฉืืœื” ืฉืžื“ืจื‘ื ืช ืื•ืชื™ ื”ื™ื,
08:23
So the question that motivates my research is,
209
503260
2176
ืื ื™ื›ื•ืœื ื• ืœื”ื ื—ื™ืช ืื™ืฉ ืขืœ ื”ื™ืจื— ื‘ืขื–ืจืช 100,000
08:25
if we can put a man on the Moon with 100,000,
210
505460
2136
08:27
what can we do with 100 million?
211
507620
2176
ืžื” ื ื•ื›ืœ ืœืขืฉื•ืช ืขื 100 ืžืœื™ื•ืŸ?
08:29
So based on this question,
212
509820
1256
ื•ื›ืš, ื‘ื”ืชื‘ืกืก ืขืœ ืฉืืœื” ื–ื•,
08:31
we've had a lot of different projects that we've been working on.
213
511100
3056
ืื ื—ื ื• ืขื•ื‘ื“ื™ื ืขืœ ืคืจื•ื™ื™ืงื˜ื™ื ืจื‘ื™ื ื•ืžื’ื•ื•ื ื™ื.
ื”ืจืฉื• ืœื™ ืœืกืคืจ ืœื›ื ืขืœ ืื—ื“ ืžื”ื ืฉืื ื™ ืžืื•ื“ ืžืชืœื”ื‘ ืžืžื ื•.
08:34
Let me tell you about one that I'm most excited about.
214
514180
2536
08:36
This is something that we've been semiquietly working on
215
516740
2656
ื–ื”ื• ืžืฉื”ื• ืฉืื ื—ื ื• ืขื•ื‘ื“ื™ื ืขืœื™ื• ื‘ื—ืฆื™-ื—ืฉืื™ื•ืช
ื›ื‘ืจ ื›ืฉื ื” ื•ื—ืฆื™.
08:39
for the last year and a half or so.
216
519420
1696
ื”ื•ื ืขื•ื“ ืœื ื”ื•ืฉืง. ื”ื•ื ื ืงืจื ื“ื•ืื•ืœื™ื ื’ื• (Duolingo).
08:41
It hasn't yet been launched. It's called Duolingo.
217
521140
2376
ื‘ื’ืœืœ ืฉืขื“ื™ื™ืŸ ืœื ื”ื•ืฉืง, ืฉืฉืฉ!
08:43
Since it hasn't been launched, shhh!
218
523540
1736
(ืฆื—ื•ืง)
08:45
(Laughter)
219
525300
1656
08:46
Yeah, I can trust you'll do that.
220
526980
2256
ื›ืŸ, ืื ื™ ื™ื›ื•ืœ ืœืกืžื•ืš ืขืœื™ื›ื.
ืื– ื”ื ื” ื”ืคืจื•ื™ื™ืงื˜. ื›ืš ื”ื•ื ื”ืชื—ื™ืœ.
08:49
So this is the project. Here's how it started.
221
529260
2216
ื–ื” ื”ืชื—ื™ืœ ื›ืฉืฉืืœืชื™ ืกื˜ื•ื“ื ื˜ ืฉืœื™ ืฉืืœื”,
08:51
It started with me posing a question to my graduate student, Severin Hacker.
222
531500
3576
ืกื•ื•ืจื™ืŸ ื”ืืงืจ.
ืื•ืงื™ื™, ื–ื” ืกื•ื•ืจื™ืŸ ื”ืืงืจ.
08:55
OK, that's Severin Hacker.
223
535100
1280
ืื– ืฉืืœืชื• ืืช ื”ืกื˜ื•ื“ื ื˜ ืฉืœื™ ืฉืืœื”.
08:57
So I posed the question to my graduate student.
224
537299
2217
ืื’ื‘, ืฉืžืขืชื ืื•ืชื™ ื ื›ื•ืŸ,
08:59
By the way, you did hear me correctly; his last name is Hacker.
225
539540
2976
ืฉื ื”ืžืฉืคื—ื” ืฉืœื• ื”ื•ื ื”ืืงืจ.
09:02
(Laughter)
226
542540
1016
ืื– ืฉืืœืชื™ ืื•ืชื•:
09:03
So I posed this question to him: How can we get 100 million people
227
543580
3296
ืื™ืš ืืคืฉืจ ืœื’ืจื•ื ืœ 100 ืžืœื™ื•ืŸ ืื™ืฉ
09:06
translating the web into every major language for free?
228
546900
2960
ืœืชืจื’ื ืืช ื”ืื™ื ื™ื˜ืจื ื˜ ืœื›ืœ ื”ืฉืคื•ืช ื”ืจืืฉื™ื•ืช ื‘ื—ื™ื ื?
ื˜ื•ื‘, ืืคืฉืจ ืœื”ื’ื™ื“ ื”ืจื‘ื” ื“ื‘ืจื™ื ืขืœ ื”ืฉืืœื” ื”ื–ืืช.
09:10
There's a lot of things to say about this question.
229
550500
2416
ืจืืฉื™ืช, ืชืจื’ื•ื ื”ืจืฉืช.
09:12
First of all, translating the web.
230
552940
1656
ืื– ื ื›ื•ืŸ ืœืขื›ืฉื™ื•, ื”ืจืฉืช ืžื—ื•ืœืงืช ืœืฉืคื•ืช ืจื‘ื•ืช.
09:14
Right now, the web is partitioned into multiple languages.
231
554620
2796
ื—ืœืง ื ื™ื›ืจ ืžืžื ื” ื”ื•ื ื‘ืื ื’ืœื™ืช.
09:17
A large fraction of it is in English.
232
557440
1816
ืื ืืชื” ืœื ื™ื•ื“ืข ืื ื’ืœื™ืช ืœื ืชื•ื›ืœ ืœื’ืฉืช ืืœื™ื•.
09:19
If you don't know English, you can't access it.
233
559280
2216
ืื‘ืœ ื™ืฉื ื ื—ืœืงื™ื ื’ื“ื•ืœื™ื ื‘ืฉืคื•ืช ืฉื•ื ื•ืช,
09:21
But there's large fractions in other different languages,
234
561520
2696
ื•ืื ืืชื” ืœื ื™ื•ื“ืข ืื•ืชืŸ ืœื ืชื•ื›ืœ ืœื’ืฉืช ืืœื™ื”ื.
09:24
and if you don't know them, you can't access it.
235
564240
2256
ืื– ื”ื™ื™ืชื™ ืจื•ืฆื” ืœืชืจื’ื ืืช ื›ืœ ื”ืจืฉืช. ืื• ืœืคื—ื•ืช ืืช ืจื•ื‘ื”,
09:26
So I would like to translate all of the web,
236
566520
2096
09:28
or at least most of it, into every major language.
237
568640
2376
ืœื›ืœ ืฉืคื” ืจืืฉื™ืช.
ืื– ื–ื” ืžื” ืฉื”ื™ื™ืชื™ ืจื•ืฆื” ืœืขืฉื•ืช.
09:31
That's what I would like to do.
238
571040
1496
09:32
Now, some of you may say, why can't we use computers to translate?
239
572560
4476
ืื– ื—ืœืงื›ื ื™ื’ื™ื“, ืœืžื” ืœื ืœื”ืฉืชืžืฉ ื‘ืžื—ืฉื‘ื™ื ืœืชืจื’ื•ื?
ืœืžื” ืื™ ืืคืฉืจ ืœื”ืชืžืฉ ื‘ืชืจื’ื•ื ืžืžื•ื—ืฉื‘?
09:37
Machine translation is starting to translate
240
577060
2096
ืชืจื’ื•ื ืžืžื•ื—ืฉื‘ ื‘ื™ืžื ื•, ืžืชื—ื™ืœ ืœืชืจื’ื ื›ืžื” ืžืฉืคื˜ื™ื ืคื” ื•ืฉื.
09:39
some sentences here and there.
241
579180
1456
ืœืžื” ืœื ืœื”ืชืžืฉ ื‘ื• ืœืชืจื’ื•ื ื›ืœ ื”ืจืฉืช?
09:40
Why can't we use it to translate the web?
242
580660
1976
ื•ื‘ื›ืŸ, ื”ื‘ืขื™ื” ืื™ืชื• ื”ื™ื ืฉื”ื•ื ืขื“ื™ื™ืŸ ืื™ื ื• ื˜ื•ื‘ ืžืกืคื™ืง.
09:42
The problem with that is it's not yet good enough
243
582660
2336
ื•ื›ื ืจืื” ืฉืœื ื™ื”ื™ื” ืžืกืคื™ืง ื˜ื•ื‘ ื‘15 ืขื“ 20 ืฉื ื™ื ื”ื‘ืื•ืช.
09:45
and it probably won't be for the next 15 to 20 years.
244
585020
2496
ื”ื•ื ื˜ื•ืขื” ื”ืžื•ืŸ.
09:47
It makes a lot of mistakes. Even when it doesn't,
245
587540
2336
ื•ืืคื™ืœื• ื›ืฉืื™ื ื• ื˜ื•ืขื”,
09:49
since it makes so many mistakes, you don't know whether to trust it or not.
246
589900
3576
ื›ื™ื•ื•ืŸ ืฉื”ื•ื ื˜ื•ืขื” ื›"ื› ื”ืจื‘ื”, ืื™ ืืคืฉืจ ืœื‘ื˜ื•ื— ื‘ื•.
ืื– ื”ืจืฉื• ืœื™ ืœืชืช ืœื›ื ื“ื•ื’ืžื
09:53
So let me show you an example
247
593500
1416
09:54
of something that was translated with a machine.
248
594940
2256
ืœืžืฉื”ื• ืฉืชื•ืจื’ื ืข"ื™ ืžื›ื•ื ื”.
ืœืžืขืฉื” ื–ื” ื”ื™ื” ืคื•ืกื˜ ื‘ืคื•ืจื•ื.
09:57
Actually, it was a forum post.
249
597220
1456
09:58
It was somebody who was trying to ask a question about JavaScript.
250
598700
3176
ืžื™ืฉื”ื• ื ื™ืกื” ืœืฉืื•ืœ ืฉืืœื” ื‘ื’'ืื•ื•ื”-ืกืงืจื™ืคื˜.
10:01
It was translated from Japanese into English.
251
601900
2616
ื–ื” ืชื•ืจื’ื ืžื™ืคื ื™ืช ืœืื ื’ืœื™ืช
10:04
So I'll just let you read.
252
604540
1776
ืื– ืืชืŸ ืœื›ื ืœืงืจื•ื.
10:06
This person starts apologizing
253
606340
1776
ื”ืื“ื ื”ื–ื” ื”ืชื—ื™ืœ ื”ืชื—ื™ืœ ื‘ื”ืชื ืฆืœื•ืช
10:08
for the fact that it's translated with a computer.
254
608140
2456
ืขืœ ื”ืขื•ื‘ื“ื” ืฉื–ื” ืชื•ืจื’ื ื‘ืืžืฆืขื•ืช ืžื—ืฉื‘.
10:10
So the next sentence is going to be the preamble to the question.
255
610620
3776
ืื– ื”ืžืฉืคื˜ ื”ื‘ื ืืžื•ืจ ืœื”ื™ื•ืช ื”ื”ืงื“ืžื” ืœืฉืืœื”.
ืื– ื”ื•ื ืจืง ืžืกื‘ื™ืจ ืžืฉื”ื•.
10:14
So he's just explaining something.
256
614420
1656
ื–ื›ืจื•, ื–ืืช ืฉืืœื” ืขืœ ื’'ืื•ื•ื”-ืกืงืจื™ืคื˜.
10:16
Remember, it's a question about JavaScript.
257
616100
2056
10:18
[At often, the goat-time install a error is vomit.]
258
618180
2616
(ื˜ืงืกื˜: ืœืขื™ืชื™ื ืชื›ื•ืคื•ืช ื–ืžืŸ-ื”ืขื– ื”ืชืงื ื” ื˜ืขื•ืช ื”ื™ื ืงื™ื.)
10:20
(Laughter)
259
620820
5096
(ืฆื—ื•ืง)
10:25
Then comes the first part of the question.
260
625940
3536
ืื– ืžื’ื™ืข ื”ื—ืœืง ื”ืจืืฉื•ืŸ ืฉืœ ื”ืฉืืœื”.
10:29
[How many times like the wind, a pole, and the dragon?]
261
629500
2936
(ื˜ืงืกื˜: ื›ืžื” ืคืขืžื™ื ื›ืžื• ื”ืจื•ื—, ืžื•ื˜ ื•ื”ื“ืจืงื•ืŸ?)
10:32
(Laughter)
262
632460
4656
(ืฆื—ื•ืง)
ื•ืื– ืžื’ื™ืข ื”ื—ืœืง ื”ืื”ื•ื‘ ืขืœื™ื™ ื‘ืฉืืœื”.
10:37
Then comes my favorite part of the question.
263
637140
2056
10:39
[This insult to father's stones?]
264
639220
1936
(ื˜ืงืกื˜: ื–ื” ืขืœื‘ื•ืŸ ืœืืฉื›ื™ ื”ืื‘?)
10:41
(Laughter)
265
641180
3856
(ืฆื—ื•ืง)
ื•ืื– ืžื’ื™ืข ื”ืกื™ื•ื, ืฉื”ื•ื ื”ื—ืœืง ื”ืื”ื•ื‘ ืขืœื™ื™ ื‘ื›ืœ ื”ืขื ื™ื™ืŸ.
10:45
And then comes the ending,
266
645060
1296
10:46
which is my favorite part of the whole thing.
267
646380
2136
(ื˜ืงืกื˜: ื‘ื‘ืงืฉื” ื”ืชื ืฆืœ ืขืœ ื˜ืคืฉื•ืชืš. ื™ืฉ ื”ืจื‘ื” ืชื•ื“ื” ืจื‘ื”.)
10:48
[Please apologize for your stupidity. There are a many thank you.]
268
648540
3136
10:51
(Laughter)
269
651700
2176
(ืฆื—ื•ืง)
10:53
OK, so computer translation, not yet good enough.
270
653900
2936
ืื•ืงื™ื™, ืื– ืชืจื’ื•ื ืžืžื•ื—ืฉื‘, ืขื“ื™ื™ืŸ ืœื ืžืกืคื™ืง ื˜ื•ื‘.
ื‘ื—ื–ืจื” ืœืฉืืœื”.
10:56
So back to the question.
271
656860
1256
ืื– ืื ื—ื ื• ืจื•ืฆื™ื ืฉืื ืฉื™ื ื™ืชืจื’ืžื• ืืช ื›ืœ ื”ืจืฉืช.
10:58
So we need people to translate the whole web.
272
658140
2976
ื•ื”ืฉืืœื” ื”ื‘ืื” ืฉื™ื›ื•ืœื” ืœื”ื™ื•ืช ืœื›ื ื”ื™ื,
11:01
So now the next question you may have is,
273
661140
1976
ืœืžื” ืื™ ืืคืฉืจ ืคืฉื•ื˜ ืœืฉืœื ืœืื ืฉื™ื ืœืขืฉื•ืช ื–ืืช?
11:03
well, why can't we just pay people to do this?
274
663140
2176
ื ื•ื›ืœ ืœืฉืœื ืœืžืชืจื’ืžื™ื ืžืงืฆื•ืขื™ื™ื ืœืชืจื’ื ืืช ื›ืœ ื”ืจืฉืช.
11:05
We could pay professional translators to translate the whole web.
275
665340
3096
ื ื•ื›ืœ ืœืขืฉื•ืช ื–ืืช.
11:08
We could do that.
276
668460
1256
11:09
Unfortunately, it would be extremely expensive.
277
669740
2216
ืœืžืจื‘ื” ื”ืฆืขืจ, ื–ื” ื™ื”ื™ื” ืžืื•ื“ ื™ืงืจ.
11:11
For example, translating a tiny fraction of the whole web, Wikipedia,
278
671980
3256
ืœืžืฉืœ, ืœืชืจื’ื ื—ืœืง ืงื˜ื ื˜ืŸ ืžื›ืœ ื”ืจืฉืช, ื•ื™ืงื™ืคื“ื™ื”,
ืœืฉืคื” ืื—ืจืช ืื—ืช, ืกืคืจื“ื™ืช.
11:15
into one other language, Spanish.
279
675260
2496
11:17
OK? Wikipedia exists in Spanish,
280
677780
1976
ื•ื™ืงื™ืคื“ื™ื” ืงื™ื™ืžืช ื‘ืกืคืจื“ื™ืช,
11:19
but it's very small compared to the size of English.
281
679780
2456
ืื‘ืœ ื”ื™ื ืงื˜ื ื” ืžืื•ื“ ื‘ื™ื—ืก ืœื–ื• ื”ืื ื’ืœื™ืช.
ื”ื™ื ื› 20 ืื—ื•ื– ืžื’ื•ื“ืœ ื”ืื ื’ืœื™ืช.
11:22
It's about 20 percent of the size of English.
282
682260
2176
ืื ื ืจืฆื” ืœืชืจื’ื ืืช ืฉืืจ 80 ื”ืื—ื•ื– ืœืกืคืจื“ื™ืช,
11:24
If we wanted to translate the other 80 percent into Spanish,
283
684460
2856
ื–ื” ื™ืขืœื” ืœืคื—ื•ืช 50 ืžืœื™ื•ืŸ ื“ื•ืœืจ --
11:27
it would cost at least 50 million dollars --
284
687340
2136
ื•ื–ื” ื‘ืขื–ืจืช ื”ืžื“ื™ื ื” ื”ื›ื™ ืžื ื•ืฆืœืช, ื•ื‘ืฉื™ืžื•ืฉ ืžื™ืงื•ืจ ื—ื•ืฅ ืฉืงื™ื™ืžืช.
11:29
and this is even at the most exploited, outsourcing country out there.
285
689500
3656
ืื– ื–ื” ื™ื”ื™ื” ืžืื•ื“ ื™ืงืจ.
11:33
So it would be very expensive.
286
693180
1456
ืื– ืžื” ืฉืื ื—ื™ื ื• ืจื•ืฆื™ื ืœืขืฉื•ืช ื”ื•ื ืœื’ืจื•ื ืœ 100 ืžืœื™ื•ืŸ ืื ืฉื™ื
11:34
So what we want to do is, we want to get 100 million people
287
694660
2762
ืœืชืจื’ื ืืช ื”ืจืฉืช ืœื›ืœ ื”ืฉืคื•ืช ื”ืจืืฉื™ื•ืช
11:37
translating the web into every major language for free.
288
697446
2590
ื‘ื—ื™ื ื.
ืื ื–ื” ืžื” ืฉืืชื ืจื•ืฆื™ื ืœืขืฉื•ืช,
11:40
If this is what you want to do, you quickly realize
289
700060
2416
ืžื”ืจ ืžืื•ื“ ืชืชืงืœื• ื‘ืฉื ื™ ืืชื’ืจื™ื ื“ื™ ื’ื“ื•ืœื™ื,
11:42
you're going to run into two big hurdles, two big obstacles.
290
702500
2936
ืฉื ื™ ืžื›ืฉื•ืœื™ื ื’ื“ื•ืœื™ื.
11:45
The first one is a lack of bilinguals.
291
705460
3296
ื”ืจืืฉื•ืŸ ื”ื•ื ืžื—ืกื•ืจ ื‘ื“ื•-ืœืฉื•ื ื™ื™ื.
11:48
So I don't even know
292
708780
2176
ืื ื™ ืืคื™ืœื• ืœื ื™ื•ื“ืข
11:50
if there exists 100 million people out there using the web
293
710980
2736
ืื ื™ืฉ ื‘ื›ืœืœ 100 ืžืœื™ื•ืŸ ืžืฉืชืžืฉื™ื
11:53
who are bilingual enough to help us translate.
294
713740
2296
ืืฉืจ ื”ื ื ื“ื•-ืœืฉื•ื ื™ื™ื ืžืกืคื™ืง ื›ื“ื™ ืœืขื–ื•ืจ ืœื ื• ื‘ืชืจื’ื•ื.
ื–ืืช ื‘ืขื™ื” ื’ื“ื•ืœื”.
11:56
That's a big problem.
295
716060
1216
11:57
The other problem you're going to run into is a lack of motivation.
296
717300
3176
ื”ื‘ืขื™ื” ื”ืฉื ื™ื” ืฉืชืชืงืœื• ื‘ื” ื”ื™ื ื—ื•ืกืจ ื‘ืžื•ื˜ื™ื‘ืฆื™ื”.
ืื™ืš ื ื“ืจื‘ืŸ ืื ืฉื™ื
12:00
How are we going to motivate people to actually translate the web for free?
297
720500
3536
ืœืขื–ื•ืจ ืœืชืจื’ื ืืช ืจืฉืช ื”ืื™ื ื˜ืจื ื˜ ื‘ื—ื™ื ื
ื‘ื“"ื› ืฆืจื™ืš ืœืชื’ืžืœ ืขื‘ื•ืจ ื“ื‘ืจ ื›ื–ื”
12:04
Normally, you have to pay people to do this.
298
724060
2296
12:06
So how are we going to motivate them to do it for free?
299
726380
2616
ืื– ืื™ืš ื ื“ืจื‘ืŸ ืื•ืชื ืœืขืฉื•ืช ืืช ื–ื” ื‘ื—ื™ื ื?
ื•ื›ืฉื”ืชื—ืœื ื• ืœื—ืฉื•ื‘ ืขืœ ื–ื”, ื ืชืงืขื ื• ื‘ืฉื ื™ ื”ืžื›ืฉื•ืœื™ื ื”ืืœื”.
12:09
When we were starting to think about this, we were blocked by these two things.
300
729020
3736
ืื‘ืœ ืื– ื”ื‘ื ื•, ืœืžืขืฉื” ืืคืฉืจ
12:12
But then we realized, there's a way
301
732780
1696
ืœืคืชื•ืจ ืืช ืฉืชื™ ื”ื‘ืขื™ื•ืช ื‘ืขื–ืจืช ืคืชืจื•ืŸ ืื—ื“.
12:14
to solve both these problems with the same solution.
302
734500
2456
ื™ืฉ ื“ืจืš ืœื”ืจื•ื’ ืฉืชื™ ืฆื™ืคื•ืจื™ื ื‘ืื‘ืŸ ืื—ืช.
12:16
To kill two birds with one stone.
303
736980
1616
ื•ื”ื“ืจืš ื”ื™ื ืœื”ืคื•ืš ืืช ืžืœืื›ืช ื”ืชืจื’ื•ื
12:18
And that is to transform language translation
304
738620
2136
12:20
into something that millions of people want to do
305
740780
2816
ืœืžืฉื”ื• ืฉืžืœื™ื•ื ื™ ืื ืฉื™ื ื™ืจืฆื• ืœืขืฉื•ืช,
12:23
and that also helps with the problem of lack of bilinguals,
306
743620
3136
ื•ืฉื’ื ืขื•ื–ืจืช ืœื”ืชืžื•ื“ื“ ืขื ื‘ืขื™ื™ืช ื”ืžื—ืกื•ืจ ื‘ื“ื•-ืœืฉื•ื ื™ื™ื,
12:26
and that is language education.
307
746780
2376
ื•ื”ื“ืจืš ื”ื™ื ืœื™ืžื•ื“ ืฉืคื•ืช.
12:29
So it turns out that today,
308
749180
1976
ืžืกืชื‘ืจ ืฉื‘ื™ืžื ื•,
12:31
there are over 1.2 billion people learning a foreign language.
309
751180
3400
ื™ื•ืชืจ ืž 1.2 ืžืœื™ืืจื“ ืื ืฉื™ื ืœื•ืžื“ื™ื ืฉืคื” ื–ืจื”.
ืื ืฉื™ื ืžืื•ื“ ืžืื•ื“ ืจื•ืฆื™ื ืœืœืžื•ื“ ืฉืคื” ื–ืจื”.
12:35
People really want to learn a foreign language.
310
755300
2216
ื•ื–ื” ืœื ืจืง ื‘ื’ืœืœ ืฉืžื›ืจื™ื—ื™ื ืื•ืชื ื‘ื‘ื™ื”"ืก.
12:37
And it's not just because they're being forced to do so in school.
311
757540
3136
ืœืžืฉืœ, ื‘ืืจื”"ื‘ ืœื‘ื“ื”,
12:40
In the US alone, there are over five million people
312
760700
2416
ื™ืฉ ื™ื•ืชืจ ืžื—ืžื™ืฉื” ืžืœื™ื•ืŸ ืื ืฉื™ื ืฉืฉื™ืœืžื• ื™ื•ืชืจ ืž 500$
12:43
who have paid over $500 for software to learn a new language.
313
763140
2896
ืขื‘ื•ืจ ืชื•ื›ื ื” ืœืœื™ืžื•ื“ ืฉืคื” ื—ื“ืฉื”.
ืื– ืื ืฉื™ื ืžืื•ื“ ืžืื•ื“ ืจื•ืฆื™ื ืœืœืžื•ื“ ืฉืคื” ื—ื“ืฉื”.
12:46
So people really want to learn a new language.
314
766060
2176
ืื– ืžื” ืฉืขื‘ื“ื ื• ืขืœื™ื• ื‘ืžืฉืš ื—ืžืฉ ื•ื—ืฆื™ ืฉื ื™ื ื”ื•ื ืืชืจ ื—ื“ืฉ --
12:48
So what we've been working on for the last year and a half
315
768260
2736
ืฉื ืงืจื ื“ื•ืื•ืœื™ื ื’ื• --
12:51
is a new website -- it's called Duolingo --
316
771020
2056
ื‘ื• ื”ืจืขื™ื•ืŸ ื”ืขืงืจื•ื ื™ ื”ื•ื ืฉืื ืฉื™ื ืœื•ืžื“ื™ื ืฉืคื” ื—ื“ืฉื” ื‘ื—ื™ื ื
12:53
where the basic idea is people learn a new language for free
317
773100
2856
12:55
while simultaneously translating the web.
318
775980
2056
ื•ื‘ื” ื‘ืขืช ืžืชืจื’ืžื™ื ืืช ื”ืจืฉืช.
ื•ืœืžืขืฉื” ื”ื ืœื•ืžื“ื™ื ื‘ืขื–ืจืช ืขืฉื™ื™ื”.
12:58
And so basically, they're learning by doing.
319
778060
2536
ื•ืื™ืš ืฉื–ื” ืขื•ื‘ื“
13:00
So the way this works
320
780620
1216
13:01
is whenever you're a just a beginner, we give you very simple sentences.
321
781860
3416
ื–ื” ืฉื›ืฉืืชื” ืจืง ืžืชื—ื™ืœ, ืื ื—ื ื• ืžืฆื™ื’ื™ื ืœืš ืžืฉืคื˜ื™ื ืžืื•ื“ ืžืื•ื“ ืคืฉื•ื˜ื™ื.
ื™ืฉ, ื›ืžื•ื‘ืŸ, ื”ืžื•ืŸ ืžืฉืคื˜ื™ื ืžืื•ื“ ืคืฉื•ื˜ื™ื ื‘ืจืฉืช.
13:05
There's a lot of very simple sentences on the web.
322
785300
2376
ืื ื—ื ื• ืžืฆื™ื’ื™ื ืœืš ืžืฉืคื˜ื™ื ืžืื•ื“ ืžืื•ื“ ืคืฉื•ื˜ื™ื
13:07
We give you very simple sentences along with what each word means.
323
787700
3216
ื™ื—ื“ ืขื ืคื™ืจื•ืฉื” ืฉืœ ื›ืœ ืžื™ืœื”.
13:10
And as you translate them
324
790940
1336
ื•ื›ืฉืืชื” ืžืชืจื’ื ืื•ืชื, ื•ื›ืฉืืชื” ืจื•ืื” ืื™ืš ืื—ืจื™ื ืชื™ืจื’ืžื• ืื•ืชื,
13:12
and as you see how other people translate them,
325
792300
2216
ืืชื” ืžืชื—ื™ืœ ืœืœืžื•ื“ ืืช ื”ืฉืคื”.
13:14
you start learning the language.
326
794540
1576
ื•ื›ืฉืืชื” ืžืฉืชืคืจ ื™ื•ืชืจ ื•ื™ื•ืชืจ,
13:16
And as you get more advanced,
327
796140
1416
13:17
we give you more complex sentences to translate.
328
797580
2256
ื ืฆื™ื’ ืœืš ืžืฉืคื˜ื™ื ืžื•ืจื›ื‘ื™ื ื™ื•ืชืจ ื•ื™ื•ืชืจ ืœืชืจื’ื.
13:19
But at all times, you're learning by doing.
329
799860
2016
ืื‘ืœ ื›ืœ ื”ื–ืžืŸ ืืชื” ืœื•ืžื“ ืชื•ืš ื›ื“ื™ ืขืฉื™ื™ื”.
13:21
Now, the crazy thing about this method is that it actually really works.
330
801900
3696
ื•ื”ื“ื‘ืจ ื”ืžืฉื•ื’ืข ื‘ืฉื™ื˜ื” ื”ื–ืืช
ื”ื•ื ืฉื”ื™ื ืœืžืขืฉื” ืขื•ื‘ื“ืช.
13:25
People are really learning a language.
331
805620
1856
ืจืืฉื™ืช ื›ืœ, ืื ืฉื™ื ื‘ืืžืช ื‘ืืžืช ืœื•ืžื“ื™ื ืฉืคื”.
13:27
We're mostly done building it and now we're testing it.
332
807500
2616
ื“ื™ื™ ืกื™ื™ืžื ื• ืœื‘ื ื•ืช ืืช ื”ืžืขืจื›ืช ื•ืขื›ืฉื™ื• ืื ื—ื ื• ื‘ื•ื“ืงื™ื ืื•ืชื”.
ืื ืฉื™ื ืžืžืฉ ื™ื›ื•ืœื™ื ืœืœืžื•ื“ ืฉืคื” ื‘ืขื–ืจืชื”.
13:30
People really can learn a language with it.
333
810140
2056
ื•ื”ื ืœื•ืžื“ื™ื ื‘ืขืจืš ื‘ืื•ืชื” ืจืžื” ืฉืœ ืชื•ื›ื ื•ืช ื”ืœื™ืžื•ื“ ื”ืžื•ื‘ื™ืœื•ืช.
13:32
And they learn it about as well as the leading language learning software.
334
812220
3496
ืื– ืื ืฉื™ื ื‘ืืžืช ืœื•ืžื“ื™ื ืฉืคื” ื—ื“ืฉื”.
13:35
So people really do learn a language.
335
815740
1816
ื•ืœื ืจืง ืฉื”ื ืœื•ืžื“ื™ื ืื•ืชื”,
13:37
And not only do they learn it as well, but actually it's more interesting.
336
817580
3496
ื–ื” ืœืžืขืฉื” ื”ืจื‘ื” ื™ื•ืชืจ ืžืขื ื™ื™ืŸ.
ื›ื™ ื‘ื“ื•ืื•ืœื™ื ื’ื• ืื ืฉื™ื ืœื•ืžื“ื™ื ืขื ืชื•ื›ืŸ ืืžื™ืชื™.
13:41
Because with Duolingo, people are learning with real content.
337
821100
2896
ืœื”ื‘ื“ื™ืœ ืžืœื™ืžื•ื“ ืขื ืžืฉืคื˜ื™ื ืžื•ืžืฆืื™ื,
13:44
As opposed to learning with made-up sentences,
338
824020
2176
ืื ืฉื™ื ืœื•ืžื“ื™ื ืขื ืชื•ื›ืŸ ืืžื™ืชื™, ื“ื‘ืจ ืฉื”ื•ื ื‘ืื•ืคืŸ ืžื•ื‘ื ื” ื™ื•ืชืจ ืžืขื ื™ื™ืŸ.
13:46
people are learning with real content, which is inherently interesting.
339
826220
3336
ืื– ืื ืฉื™ื ืื›ืŸ ืœื•ืžื“ื™ื ืฉืคื”.
13:49
So people really do learn a language.
340
829580
1816
ืื‘ืœ ืื•ืœื™ ื‘ืื•ืคืŸ ืžืคืชื™ืข,
13:51
But perhaps more surprisingly,
341
831420
1616
ื”ืชืจื’ื•ืžื™ื ืฉืื ื—ื ื• ืžืงื‘ืœื™ื ืžื”ืžืฉืชืžืฉื™ื,
13:53
the translations that we get from people using the site,
342
833060
2736
13:55
even though they're just beginners,
343
835820
1776
ืืคื™ืœื• ืฉื”ื ืจืง ื‘ืจืžืช ืžืชื—ื™ืœื™ื,
13:57
the translations that we get
344
837620
1376
ื”ืชืจื’ื•ืžื™ื ืฉืื ื—ื ื• ืžืงื‘ืœื™ื ื”ื ืžื“ื•ื™ื™ืงื™ื ื›ืžื• ืืœื• ืฉืœ ืžืชืจื’ืžื™ื ืžืงืฆื•ืขื™ื™ื,
13:59
are as accurate as those of professional language translators,
345
839020
2936
ืฉื–ื” ื“ื‘ืจ ืžืคืชื™ืข ืžืื•ื“.
14:01
which is very surprising.
346
841980
1216
ื”ืจืฉื• ืœื”ืฆื™ื’ ืœื›ื ื“ื•ื’ืžื”.
14:03
So let me show you one example.
347
843220
1536
14:04
This is a sentence that was translated from German into English.
348
844780
3016
ื”ืžืฉืคื˜ ื”ื–ื” ืฉืชื•ืจื’ื ืžื’ืจืžื ื™ืช ืœืื ื’ืœื™ืช.
ื”ืขืœื™ื•ืŸ ื”ื•ื ื‘ื’ืจืžื ื™ืช.
14:07
The top is the German. The middle is an English translation
349
847820
2776
ื”ืืžืฆืขื™ ื”ื•ื ื”ืชืจื’ื•ื ืœืื ื’ืœื™ืช
14:10
that was done by a professional translator
350
850620
2256
ื›ืคื™ ืฉืชื•ืจื’ื ืข"ื™ ืžืชืจื’ื ืžืงืฆื•ืขื™ ืœืื ื’ืœื™ืช
14:12
who we paid 20 cents a word for this translation.
351
852900
2376
ืฉืงื™ื‘ืœ 20 ืกื ื˜ ืœืžื™ืœื” ืขื‘ื•ืจ ื”ืชืจื’ื•ื.
ื”ืชื—ืชื•ืŸ ื”ื•ื ืชืจื’ื•ื ืฉืœ ืžืฉืชืžืฉื™ ื“ื•ืื•ืœื™ื ื’ื•,
14:15
And the bottom is a translation by users of Duolingo,
352
855300
2696
ืื™ืฉ ืžื”ื ืœื ื™ื“ืข ื’ืจืžื ื™ืช
14:18
none of whom knew any German before they started using the site.
353
858020
3736
ืœืคื ื™ ืฉื”ืชื—ื™ืœื• ืœื”ืฉืชืžืฉ ื‘ืืชืจ.
14:21
If you can see, it's pretty much perfect.
354
861780
1976
ืืชื ื™ื›ื•ืœื™ื ืœืจืื•ืช, ื–ื” ื“ื™ ืžื•ืฉืœื.
14:23
Of course, we play a trick here
355
863780
1536
ื›ืžื•ื‘ืŸ, ืื ื—ื ื• ืžืฉืชืžืฉื™ื ื‘ืชื›ืกื™ืก ื›ืืŸ,
14:25
to make the translations as good as professional language translators.
356
865340
3336
ื›ื“ื™ ืœื’ืจื•ื ืœืชืจื’ื•ืžื™ื ืœื”ื™ื•ืช ื‘ืจืžื” ืžืงืฆื•ืขื™ืช,
ืื ื—ื ื• ืžืื—ื“ื™ื ืชืจื’ื•ืžื™ื ืฉืœ ื”ืจื‘ื” ืžืชืจื’ืžื™ื ืžืชื—ื™ืœื™ื
14:28
We combine the translations of multiple beginners
357
868700
2336
ื›ื“ื™ ืœืงื‘ืœ ืจืžื” ืฉืœ ืžืชืจื’ื ืžืงืฆื•ืขื™ ืื—ื“.
14:31
to get the quality of a single professional translator.
358
871060
2896
14:33
Now, even though we're combining the translations,
359
873980
4536
ื•ืืคื™ืœื• ืฉืื ื—ื ื• ืžืื—ื“ื™ื ืชืจื’ื•ืžื™ื,
14:38
the site actually can translate pretty fast.
360
878540
2776
ื”ืืชืจ ืœืžืขืฉื” ืžืชืจื’ื ื“ื™ ืžื”ืจ.
ื”ืจืฉื• ืœื™ ืœื”ืจืื•ืช ืœื›ื,
14:41
So let me show you,
361
881340
1216
14:42
this is our estimates of how fast we could translate Wikipedia
362
882580
2936
ื–ื•ื”ื™ ื”ื”ืขืจื›ื” ืฉืœื ื• ืœืžืฉืš ื”ื–ืžืŸ ืฉื™ืงื— ืœื ื• ืœืชืจื’ื ืืช ื•ื•ื™ืงื™ืคื“ื™ื”
ืžืื ื’ืœื™ืช ืœืกืคืจื“ื™ืช.
14:45
from English into Spanish.
363
885540
1296
14:46
Remember, this is 50 million dollars' worth of value.
364
886860
2976
ื–ื™ื›ืจื•, ื–ืืช ืขื‘ื•ื“ื” ื‘ืขืจืš ืฉืœ 50 ืžืœื™ื•ืŸ ื“ื•ืœืจ.
14:49
So if we wanted to translate Wikipedia into Spanish,
365
889860
2456
ืื– ืื ื ืจืฆื” ืœืชืจื’ื ืืช ื•ื•ื™ืงื™ืคื“ื™ื” ืœืกืคืจื“ื™ืช,
ื ื•ื›ืœ ืœื”ืฉื™ื’ ื–ืืช ื‘ื—ืžื™ืฉื” ืฉื‘ื•ืขื•ืช ืขื 100,000 ืžืฉืชืžืฉื™ื ืคืขื™ืœื™ื.
14:52
we could do it in five weeks with 100,000 active users.
366
892340
2696
ื•ื ื•ื›ืœ ืœื”ืฉื™ื’ ื–ืืช ื‘ื›80 ืฉืขื•ืช ืขื ืžืœื™ื•ืŸ ืžืฉืชืžืฉื™ื ืคืขื™ืœื™ื.
14:55
And we could do it in about 80 hours with a million active users.
367
895060
3056
ื›ื™ื•ื•ืŸ ืฉื‘ื›ืœ ื”ืคืจื•ื™ื™ืงื˜ื™ื ืฉืœ ื”ืงื‘ื•ืฆื” ืฉืœื™ ื”ืฉืชืชืคื• ืžืœื™ื•ื ื™ ืžืฉืชืžืฉื™ื,
14:58
Since all the projects my group has worked on so far
368
898140
2456
15:00
have gotten millions of users,
369
900620
1456
ืื ื—ื ื• ืžืงื•ื•ื™ื ืœื”ืฆืœื™ื— ืœืชืจื’ื
15:02
we're hopeful that we'll be able to translate extremely fast.
370
902100
2896
ืžืื•ื“ ืžื”ืจ ื‘ืขื–ืจืช ื”ืคืจื•ื™ื™ืงื˜ ื”ื–ื”.
ื•ื”ื“ื‘ืจ ืฉื”ื›ื™ ืžืจื’ืฉ ืื•ืชื™ ื‘ื“ื•ืื•ืœื™ื ื’ื•
15:05
Now, the thing that I'm most excited about with Duolingo
371
905020
2976
ื”ื•ื ืฉืื ื™ ื—ื•ืฉื‘ ืฉื”ื•ื ืžืกืคืง ืžื•ื“ืœ ืขืกืงื™ ื”ื•ื’ืŸ ืœืœื™ืžื•ื“ ืฉืคื•ืช.
15:08
is I think this provides a fair business model for language education.
372
908020
3736
ื•ื”ื ื” ื”ืขื ื™ื™ืŸ:
15:11
So here's the thing:
373
911780
1216
ื”ืžื•ื“ืœ ื”ืขืกืงื™ ื”ื ื•ื›ื—ื™ ืฉืœ ืœื™ืžื•ื“ ืฉืคื•ืช
15:13
The current business model for language education
374
913020
2336
ื”ื•ื ืฉื”ืกื˜ื•ื“ื ื˜ ืžืฉืœื
15:15
is the student pays,
375
915380
1376
15:16
and in particular, the student pays Rosetta Stone 500 dollars.
376
916780
3056
ืกืคืฆื™ืคื™ืช, ื”ื•ื ืžืฉืœื ืœ"ืจื•ื–ื˜ื”-ืกื˜ื•ืŸ" (ืชื•ื›ื ื” ื™ื“ื•ืขื”) $500.
(ืฆื—ื•ืง)
15:19
(Laughter)
377
919860
1816
ื–ื” ื”ืžื•ื“ืœ ื”ืขืกืงื™ ื”ื ื•ื›ื—ื™.
15:21
That's the current business model.
378
921700
1656
ื”ื‘ืขื™ื” ืขื ื”ืžื•ื“ืœ ื”ืขืกืงื™ ื”ื–ื”
15:23
The problem with this business model
379
923380
1736
ื”ื™ื ืฉืœ-95 ืื—ื•ื– ืžืื•ื›ืœื•ืกื™ืช ื”ืขื•ืœื ืื™ืŸ 500 ื“ื•ืœืจ.
15:25
is that 95 percent of the world's population doesn't have 500 dollars.
380
925140
3296
ืื– ื”ื•ื ืžืื•ื“ ืœื ื”ื•ื’ืŸ ื›ืœืคื™ ื”ืขื ื™ื™ื.
15:28
So it's extremely unfair towards the poor.
381
928460
2776
ื•ืœื—ืœื•ื˜ื™ืŸ ืžื•ื˜ื” ื›ืœืคื™ ื”ืขืฉื™ืจื™ื.
15:31
This is totally biased towards the rich.
382
931260
1936
ืขื›ืฉื™ื•, ืจืื•, ื‘ื“ื•ืื•ืœื™ื ื’ื•,
15:33
Now, see, in Duolingo,
383
933220
1616
15:34
because while you learn, you're actually creating value,
384
934860
3656
ื‘ื’ืœืœ ืฉื‘ื–ืžืŸ ืฉืืชื” ืœื•ืžื“
ืืชื” ืœืžืขืฉื” ื™ื•ืฆืจ ืขืจืš, ืืชื” ืžืชืจื’ื ื“ื‘ืจื™ื --
15:38
you're translating stuff --
385
938540
1336
15:39
which, for example, we could charge somebody for translations,
386
939900
2936
ืืฉืจ ืœืžืฉืœ, ื ื•ื›ืœ ืœื’ื‘ื•ืช ืขื‘ื•ืจื ื›ืกืฃ ืžืžื™ืฉื”ื•.
15:42
so this is how we could monetize this.
387
942860
1856
ื•ื›ืš ื ื™ืชืŸ ืœื™ื™ืฆืจ ืžื–ื” ื›ืกืฃ.
15:44
Since people are creating value while they're learning,
388
944740
2616
ื›ื™ื•ื•ืŸ ืฉืื ืฉื™ื ืžื™ื™ืฆืจื™ื ืขืจืš ื‘ื–ืžืŸ ืฉื”ื ืœื•ืžื“ื™ื,
ื”ื ืœื ืฆืจื™ื›ื™ื ืœืฉืœื ื‘ื›ืกืคื, ื”ื ืžืฉืœืžื™ื ื‘ื–ืžื ื.
15:47
they don't have to pay with their money, they pay with their time.
389
947380
3096
ืื‘ืœ ื”ื“ื‘ืจ ื”ืงืกื•ื ื›ืืŸ ื”ื•ื ืฉื”ื ืžืฉืœืžื™ื ื‘ื–ืžื ื,
15:50
But the magical thing here
390
950500
1923
15:52
is that is time that would have had to have been spent anyways
391
952447
2996
ืื‘ืœ ื–ื” ืื•ืชื• ื”ื–ืžืŸ ืฉื”ื™ื• ืžื‘ืœื™ื ื‘ื›ืœ ืžืงืจื”
ื‘ืœืžื™ื“ืช ื”ืฉืคื”.
15:55
learning the language.
392
955467
1209
15:56
So the nice thing about Duolingo
393
956700
1576
ืื– ื”ื“ื‘ืจ ื”ื ื—ืžื“ ื‘ื“ื•ืื•ืœื•ื ื’ื•, ื”ื•ื ืœื“ืขืชื™ ื”ืžื•ื“ืœ ื‘ืขืกืงื™ ื”ื”ื•ื’ืŸ --
15:58
is, I think, it provides a fair business model --
394
958300
2336
ืื™ืŸ ืืคืœื™ื™ื” ื ื’ื“ ืขื ื™ื™ื.
16:00
one that doesn't discriminate against poor people.
395
960660
2376
ืื– ื”ื ื” ื”ืืชืจ. ืชื•ื“ื”.
16:03
So here's the site. Thank you.
396
963060
1456
(ืชืฉื•ืื•ืช)
16:04
(Applause)
397
964540
7000
ืื– ื”ื ื” ื”ืืชืจ.
16:13
We haven't yet launched,
398
973060
2416
ืขื“ื™ื™ืŸ ืœื ื”ืฉืงื ื• ืื•ืชื•,
16:15
but if you go there, you can sign up to be part of our private beta,
399
975500
3296
ืื‘ืœ ืื ืชื’ืœืฉื• ืœืฉื ืชื•ื›ืœื• ืœื”ืจืฉื ืœื”ื™ื•ืช ื—ืœืง ืžื”ื‘ื˜ื ื”ืคืจื˜ื™ืช,
16:18
which is probably going to start in three or four weeks.
400
978820
2656
ืฉื›ื ืจืื” ืชืชื—ื™ืœ ื‘ืขื•ื“ ืฉืœื•ืฉื” ืื• ืืจื‘ืขื” ืฉื‘ื•ืขื•ืช.
ืขื“ื™ื™ืŸ ืœื ื”ืฉืงื ื• ืืช ื“ื•ืื•ืœื•ื ื’ื•.
16:21
We haven't yet launched it.
401
981500
1336
16:22
By the way, I'm the one talking here,
402
982860
1816
ืื’ื‘, ืื ื™ ื–ื” ืฉืžื“ื‘ืจ ื›ืืŸ,
16:24
but Duolingo is the work of a really awesome team,
403
984700
2376
ืื‘ืœ ื“ื•ืื•ืœื™ื ื’ื• ื”ื•ื ืœืžืขืฉื” ืคืจื™ ืขืžืœื” ืฉืœ ืงื‘ื•ืฆื” ืžื“ื”ื™ืžื”, ื—ืœืงื ื›ืืŸ.
16:27
some of whom are here. So thank you.
404
987100
1736
ืื– ืชื•ื“ื” ืœื›ื.
16:28
(Applause)
405
988860
5240
(ืชืฉื•ืื•ืช)
ืขืœ ืืชืจ ื–ื”

ืืชืจ ื–ื” ื™ืฆื™ื’ ื‘ืคื ื™ื›ื ืกืจื˜ื•ื ื™ YouTube ื”ืžื•ืขื™ืœื™ื ืœืœื™ืžื•ื“ ืื ื’ืœื™ืช. ืชื•ื›ืœื• ืœืจืื•ืช ืฉื™ืขื•ืจื™ ืื ื’ืœื™ืช ื”ืžื•ืขื‘ืจื™ื ืขืœ ื™ื“ื™ ืžื•ืจื™ื ืžื”ืฉื•ืจื” ื”ืจืืฉื•ื ื” ืžืจื—ื‘ื™ ื”ืขื•ืœื. ืœื—ืฅ ืคืขืžื™ื™ื ืขืœ ื”ื›ืชื•ื‘ื™ื•ืช ื‘ืื ื’ืœื™ืช ื”ืžื•ืฆื’ื•ืช ื‘ื›ืœ ื“ืฃ ื•ื™ื“ืื• ื›ื“ื™ ืœื”ืคืขื™ืœ ืืช ื”ืกืจื˜ื•ืŸ ืžืฉื. ื”ื›ืชื•ื‘ื™ื•ืช ื’ื•ืœืœื•ืช ื‘ืกื ื›ืจื•ืŸ ืขื ื”ืคืขืœืช ื”ื•ื•ื™ื“ืื•. ืื ื™ืฉ ืœืš ื”ืขืจื•ืช ืื• ื‘ืงืฉื•ืช, ืื ื ืฆื•ืจ ืื™ืชื ื• ืงืฉืจ ื‘ืืžืฆืขื•ืช ื˜ื•ืคืก ื™ืฆื™ืจืช ืงืฉืจ ื–ื”.

https://forms.gle/WvT1wiN1qDtmnspy7