What we learned from 5 million books

236,259 views ใƒป 2011-09-20

TED


ืื ื ืœื—ืฅ ืคืขืžื™ื™ื ืขืœ ื”ื›ืชื•ื‘ื™ื•ืช ื‘ืื ื’ืœื™ืช ืœืžื˜ื” ื›ื“ื™ ืœื”ืคืขื™ืœ ืืช ื”ืกืจื˜ื•ืŸ.

ืžืชืจื’ื: Yubal Masalker ืžื‘ืงืจ: Sigal Tifferet
00:15
Erez Lieberman Aiden: Everyone knows
0
15260
2000
ืืจื– ืœื™ื‘ืจืžืŸ ืื™ื™ื“ืŸ: ื›ื•ืœื ื™ื•ื“ืขื™ื
00:17
that a picture is worth a thousand words.
1
17260
3000
ืฉืชืžื•ื ื” ืฉื•ื•ื” ืืœืฃ ืžื™ืœื™ื.
00:22
But we at Harvard
2
22260
2000
ืื‘ืœ ืื ื—ื ื• ื‘ื”ืจื•ื•ืืจื“
00:24
were wondering if this was really true.
3
24260
3000
ืชื”ื™ื ื• ืื ื–ื” ื‘ืืžืช ื ื›ื•ืŸ.
00:27
(Laughter)
4
27260
2000
(ืฆื—ื•ืง)
00:29
So we assembled a team of experts,
5
29260
4000
ืœื›ืŸ ื”ืจื›ื‘ื ื• ืฆื•ื•ืช ืฉืœ ืžื•ืžื—ื™ื
00:33
spanning Harvard, MIT,
6
33260
2000
ืฉืžื’ื™ืขื™ื ืžื”ืจื•ื•ืืจื“, MIT,
00:35
The American Heritage Dictionary, The Encyclopedia Britannica
7
35260
3000
ื”ืžื™ืœื•ืŸ ืœืžื•ืจืฉืช ืืžืจื™ืงืื™ืช, ืื ืฆื™ืงืœื•ืคื“ื™ื” ื‘ืจื™ื˜ื ื™ืงื”
00:38
and even our proud sponsors,
8
38260
2000
ื•ืืคื™ืœื• ืžื ื•ืชื ื™ ื”ื—ืกื•ืช
00:40
the Google.
9
40260
3000
ื”ื’ืื™ื ืฉืœื ื•, ื’ื•ื’ืœ.
00:43
And we cogitated about this
10
43260
2000
ื”ืจื”ืจื ื• ื‘ื–ื”
00:45
for about four years.
11
45260
2000
ื‘ืžืฉืš ื›ืืจื‘ืข ืฉื ื™ื
00:47
And we came to a startling conclusion.
12
47260
5000
ื•ื”ื’ืขื ื• ืœืžืกืงื ื” ืžื“ื”ื™ืžื”.
00:52
Ladies and gentlemen, a picture is not worth a thousand words.
13
52260
3000
ื’ื‘ื™ืจื•ืชื™ื™ ื•ืจื‘ื•ืชื™ื™, ืชืžื•ื ื” ืื™ื ื” ืฉื•ื•ื” ืืœืฃ ืžื™ืœื™ื.
00:55
In fact, we found some pictures
14
55260
2000
ื‘ืขืฆื, ืžืฆืื ื• ื›ืžื” ืชืžื•ื ื•ืช
00:57
that are worth 500 billion words.
15
57260
5000
ื”ืฉื•ื•ืช 500 ืžื™ืœื™ืืจื“ ืžื™ืœื™ื.
01:02
Jean-Baptiste Michel: So how did we get to this conclusion?
16
62260
2000
ื’'ื™ืŸ-ื‘ืคื˜ื™ืกื˜ ืžื™ืฉืœ: ื›ื™ืฆื“ ื”ื’ืขื ื• ืœืžืกืงื ื” ื–ื•?
01:04
So Erez and I were thinking about ways
17
64260
2000
ืืจื– ื•ืื ื™ ื—ืฉื‘ื ื• ืขืœ ื“ืจื›ื™ื
01:06
to get a big picture of human culture
18
66260
2000
ืœืงื‘ืœืช ืชืžื•ื ื” ื›ื•ืœืœืช ืฉืœ ืชืจื‘ื•ืช
01:08
and human history: change over time.
19
68260
3000
ื•ื”ื™ืกื˜ื•ืจื™ื” ืื ื•ืฉื™ืช: ืฉืœ ืฉื™ื ื•ื™ ืœืื•ืจืš ื–ืžืŸ.
01:11
So many books actually have been written over the years.
20
71260
2000
ื›ืœ-ื›ืš ื”ืจื‘ื” ืกืคืจื™ื ื ื›ืชื‘ื• ื‘ืžื”ืœืš ื”ืฉื ื™ื.
01:13
So we were thinking, well the best way to learn from them
21
73260
2000
ืœื›ืŸ ื—ืฉื‘ื ื• ืฉื”ื“ืจืš ื”ื›ื™ ื˜ื•ื‘ื” ืœืœืžื•ื“ ืžื”ื
01:15
is to read all of these millions of books.
22
75260
2000
ื”ื™ื ืœืงืจื•ื ืืช ื›ืœ ืžื™ืœื™ื•ื ื™ ื”ืกืคืจื™ื.
01:17
Now of course, if there's a scale for how awesome that is,
23
77260
3000
ื›ืžื•ื‘ืŸ ืฉืื ื™ืฉ ืžื“ื“ ืœืขื“ ื›ืžื” ืฉื–ื” ืžืจืฉื™ื,
01:20
that has to rank extremely, extremely high.
24
80260
3000
ื™ืฉ ืœืžืงื ืื•ืชื• ืžืื•ื“, ืžืื•ื“ ื’ื‘ื•ื”.
01:23
Now the problem is there's an X-axis for that,
25
83260
2000
ื”ื‘ืขื™ื” ื”ื™ื ืฉืงื™ื™ื ื’ื ืฆื™ืจ X,
01:25
which is the practical axis.
26
85260
2000
ืฉื”ื•ื ืฆื™ืจ ื”ืชื›ืœื™ืชื™ื•ืช ื‘ืžืงืจื” ื–ื”,
01:27
This is very, very low.
27
87260
2000
ืฉืขืœ-ืคื™ื• ื–ื” ืžืื•ื“, ืžืื•ื“ ื ืžื•ืš.
01:29
(Applause)
28
89260
3000
(ืžื—ื™ืื•ืช ื›ืคื™ื™ื)
01:32
Now people tend to use an alternative approach,
29
92260
3000
ืื ืฉื™ื ื ื•ื˜ื™ื ืœื”ืฉืชืžืฉ ื‘ื’ื™ืฉื” ืืœื˜ืจื ื˜ื™ื‘ื™ืช,
01:35
which is to take a few sources and read them very carefully.
30
95260
2000
ืฉื–ื” ืœืงื—ืช ื›ืžื” ืžืงื•ืจื•ืช ื•ืœืงืจื•ื ืื•ืชื ื‘ืชืฉื•ืžืช ืœื‘.
01:37
This is extremely practical, but not so awesome.
31
97260
2000
ื–ื” ืžืื•ื“ ืžืขืฉื™ ืื‘ืœ ืœื ืžืจืฉื™ื.
01:39
What you really want to do
32
99260
3000
ืžื” ืฉื‘ืืžืช ืฆืจื™ืš ืœืขืฉื•ืช
01:42
is to get to the awesome yet practical part of this space.
33
102260
3000
ื–ื” ืœื”ื™ื›ื ืก ืœื—ืœืง ื”ืžืจืฉื™ื ืื‘ืœ ื’ื ื”ืžืขืฉื™ ืฉืœ ืกื‘ื™ื‘ื” ื–ื•.
01:45
So it turns out there was a company across the river called Google
34
105260
3000
ืžืชื‘ืจืจ ืฉื™ืฉ ื—ื‘ืจื” ืžืขื‘ืจ ืœื ื”ืจ ืฉื ืงืจืืช ื’ื•ื’ืœ
01:48
who had started a digitization project a few years back
35
108260
2000
ืฉื”ื—ืœื” ื‘ืžื™ื–ื ื“ื™ื’ื™ื˜ืœื™ื–ืฆื™ื” ืœืคื ื™ ื›ืžื” ืฉื ื™ื
01:50
that might just enable this approach.
36
110260
2000
ื”ืขืฉื•ื™ ืœืืคืฉืจ ืืช ื™ื™ืฉื•ืžื” ืฉืœ ื’ื™ืฉื” ื–ื•.
01:52
They have digitized millions of books.
37
112260
2000
ื”ื ื‘ื™ืฆืขื• ื“ื™ื’ื™ื˜ืœื™ื–ืฆื™ื” ืœืžื™ืœื™ื•ื ื™ ืกืคืจื™ื.
01:54
So what that means is, one could use computational methods
38
114260
3000
ื–ื” ืื•ืžืจ ืฉื ื™ืชืŸ ืœื ืฆืœ ืฉื™ื˜ื•ืช ืžืžื•ื—ืฉื‘ื•ืช
01:57
to read all of the books in a click of a button.
39
117260
2000
ื›ื“ื™ ืœืงืจื•ื ืืช ื›ืœ ื”ืกืคืจื™ื ื‘ืœื—ื™ืฆืช ื›ืคืชื•ืจ.
01:59
That's very practical and extremely awesome.
40
119260
3000
ื–ื” ืžืื•ื“ ืžืขืฉื™ ื•ื’ื ืžืจืฉื™ื ื‘ื™ื•ืชืจ.
02:03
ELA: Let me tell you a little bit about where books come from.
41
123260
2000
ื.ืœ.ื.: ืืกืคืจ ืœื›ื ืงืฆืช ืžืื™ืคื” ื”ืกืคืจื™ื ืžื’ื™ืขื™ื.
02:05
Since time immemorial, there have been authors.
42
125260
3000
ืžืื– ื–ืžื ื™ื ืงื“ื•ืžื™ื, ื”ื™ื• ื›ื‘ืจ ืžื™ืœื™ื•ื ื™ ืกื•ืคืจื™ื.
02:08
These authors have been striving to write books.
43
128260
3000
ืกื•ืคืจื™ื ืืœื” ืฉืืคื• ืœื›ืชื•ื‘ ืกืคืจื™ื.
02:11
And this became considerably easier
44
131260
2000
ื•ื–ื” ื”ืคืš ืœืงืœ ืžืฉืžืขื•ืชื™ืช
02:13
with the development of the printing press some centuries ago.
45
133260
2000
ืขื ื”ืชืคืชื—ื•ืช ื”ื“ืคื•ืก ืœืคื ื™ ืžืกืคืจ ืžืื•ืช ืฉื ื™ื.
02:15
Since then, the authors have won
46
135260
3000
ืžืื–, ื”ืกื•ืคืจื™ื ื–ื›ื• ืœืคืจืกื ืกืคืจื™ื
02:18
on 129 million distinct occasions,
47
138260
2000
129 ืžื™ืœื™ื•ืŸ
02:20
publishing books.
48
140260
2000
ืคืขืžื™ื.
02:22
Now if those books are not lost to history,
49
142260
2000
ืื ื›ืœ ืื•ืชื ื”ืกืคืจื™ื ืœื ื”ืœื›ื• ืœืื™ื‘ื•ื“
02:24
then they are somewhere in a library,
50
144260
2000
ื‘ืžื”ืœืš ื”ื”ื™ืกื˜ื•ืจื™ื”, ื”ื ื ืžืฆืื™ื ื‘ืกืคืจื™ื•ืช,
02:26
and many of those books have been getting retrieved from the libraries
51
146260
3000
ื•ืจื‘ื™ื ืžื‘ื™ืŸ ื”ืกืคืจื™ื ื”ืืœื” ื ืฉืœืคื™ื ืžื”ืกืคืจื™ื•ืช
02:29
and digitized by Google,
52
149260
2000
ื•ืขื•ื‘ืจื™ื ื“ื™ื’ื™ื˜ืœื™ื–ืฆื™ื” ืืฆืœ ื’ื•ื’ืœ,
02:31
which has scanned 15 million books to date.
53
151260
2000
ืืฉืจ ืกืจืงื” ืขื“ ื›ื” 15 ืžื™ืœื™ื•ืŸ ืกืคืจื™ื.
02:33
Now when Google digitizes a book, they put it into a really nice format.
54
153260
3000
ื›ืืฉืจ ื‘ื’ื•ื’ืœ ืขื•ืฉื™ื ื“ื™ื’ื™ื˜ืœื™ื–ืฆื™ื” ืœืกืคืจ, ื”ื ืžืขื‘ื™ืจื™ื ืื•ืชื• ืœืคื•ืจืžื˜ ื‘ืืžืช ื™ืคื”.
02:36
Now we've got the data, plus we have metadata.
55
156260
2000
ื™ืฉ ืœื ื• ื ืชื•ื ื™ื ื•ื‘ื ื•ืกืฃ ื™ืฉ ืœื ื• ื ืชื•ื ื™ื ืขืœ ืžืืคื™ื™ื ื™ ื”ื ืชื•ื ื™ื.
02:38
We have information about things like where was it published,
56
158260
3000
ื™ืฉ ืœื ื• ืžื™ื“ืข ืขืœ ื“ื‘ืจื™ื ื›ื’ื•ืŸ ื”ื™ื›ืŸ ื–ื” ืคื•ืจืกื,
02:41
who was the author, when was it published.
57
161260
2000
ืžื™ ื”ื™ื” ื”ืžื—ื‘ืจ, ืžืชื™ ื–ื” ืคื•ืจืกื.
02:43
And what we do is go through all of those records
58
163260
3000
ื•ืžื” ืฉืื ื• ืขื•ืฉื™ื ื–ื” ืœืขื‘ื•ืจ ืขืœ ื›ืœ ื”ืจืฉื•ืžื•ืช ื”ืืœื•
02:46
and exclude everything that's not the highest quality data.
59
166260
4000
ื•ืœื•ื•ืชืจ ืขืœ ื›ืœ ื”ื ืชื•ื ื™ื ืฉืื™ื ื ืžื”ืื™ื›ื•ืช ื”ื›ื™ ื’ื‘ื•ื”ื”.
02:50
What we're left with
60
170260
2000
ืžื” ืฉื ืฉืืจ ื–ื”
02:52
is a collection of five million books,
61
172260
3000
ืื•ืกืฃ ืฉืœ 5 ืžื™ืœื™ื•ืŸ ืกืคืจื™ื,
02:55
500 billion words,
62
175260
3000
500 ืžื™ืœื™ืืจื“ ืžื™ืœื™ื,
02:58
a string of characters a thousand times longer
63
178260
2000
ืžื—ืจื•ื–ืช ืฉืœ ืื•ืชื™ื•ืช ื”ืืจื•ื›ื” ืคื™ ืืœืฃ
03:00
than the human genome --
64
180260
3000
ืžื”ื—ื•ืžืจ ื”ืชื•ืจืฉืชื™ ื”ืื ื•ืฉื™ --
03:03
a text which, when written out,
65
183260
2000
ื˜ืงืกื˜ ืฉืื ื™ื™ื›ืชื‘,
03:05
would stretch from here to the Moon and back
66
185260
2000
ื™ื’ื™ืข ืžื›ืืŸ ืœื™ืจื— ื•ื‘ื—ื–ืจื”
03:07
10 times over --
67
187260
2000
10 ืคืขืžื™ื ื•ื™ื•ืชืจ --
03:09
a veritable shard of our cultural genome.
68
189260
4000
ื–ื”ื• ืคืœื— ืžืฉืžืขื•ืชื™ ืžื”ืชื•ืจืฉื” ื”ืชืจื‘ื•ืชื™ืช ืฉืœื ื•.
03:13
Of course what we did
69
193260
2000
ื‘ืจื•ืจ ืฉืžื” ืฉืขืฉื™ื ื•
03:15
when faced with such outrageous hyperbole ...
70
195260
3000
ื›ืืฉืจ ื ืชืงืœื ื• ื‘ื”ื™ืคืจื‘ื•ืœื” ืฉืขืจื•ืจื™ื™ืชื™ืช ื›ื–ื• --
03:18
(Laughter)
71
198260
2000
(ืฆื—ื•ืง)
03:20
was what any self-respecting researchers
72
200260
3000
ื”ื™ื” ืžื” ืฉื›ืœ ื—ื•ืงืจ ื”ืžื›ื‘ื“ ืืช ืขืฆืžื•
03:23
would have done.
73
203260
3000
ื”ื™ื” ืขื•ืฉื”.
03:26
We took a page out of XKCD,
74
206260
2000
ืœืงื—ื ื• ื“ืฃ ืžืชื•ืš ืงื˜ืข ืงื•ืžื™ ื‘ืจืฉืช,
03:28
and we said, "Stand back.
75
208260
2000
ื•ืืžืจื ื•, "ืชืชืจื—ืงื™ ืžืื™ืชื ื•.
03:30
We're going to try science."
76
210260
2000
ืื ื• ื”ื•ืœื›ื™ื ืœื”ื™ืขื–ืจ ื‘ืžื“ืข."
03:32
(Laughter)
77
212260
2000
(ืฆื—ื•ืง)
03:34
JM: Now of course, we were thinking,
78
214260
2000
ื’'.ืž.: ื˜ื•ื‘, ื‘ืจื•ืจ ืฉื—ืฉื‘ื ื•
03:36
well let's just first put the data out there
79
216260
2000
ืฉืื•ืœื™ ื ื—ืฉื•ืฃ ืืช ื”ื ืชื•ื ื™ื ืœืื ืฉื™ื
03:38
for people to do science to it.
80
218260
2000
ื›ื“ื™ ืฉื™ืขืฉื• ืขืœื™ื”ื ืžื—ืงืจื™ื ืžื“ืขื™ื™ื.
03:40
Now we're thinking, what data can we release?
81
220260
2000
ื”ืชื—ืœื ื• ืœื—ืฉื•ื‘ ืื™ื–ื” ื ืชื•ื ื™ื ืœืฉื—ืจืจ.
03:42
Well of course, you want to take the books
82
222260
2000
ื”ืืžืช ืฉื”ื™ื™ื ื• ืจื•ืฆื™ื ืœืฉื—ืจืจ
03:44
and release the full text of these five million books.
83
224260
2000
ืืช ืžืœื•ื ื”ื˜ืงืกื˜ ืฉืœ ื›ืœ 5 ืžื™ืœื™ื•ืŸ ื”ืกืคืจื™ื.
03:46
Now Google, and Jon Orwant in particular,
84
226260
2000
ืื‘ืœ ื’ื•ื’ืœ, ื•ื‘ืขื™ืงืจ ื’'ื•ืŸ ืื•ืจื•ื•ื ื˜,
03:48
told us a little equation that we should learn.
85
228260
2000
ืกื™ืคืจื• ืœื ื• ืฉืขืœื™ื ื• ืœืœืžื•ื“ ืžืฉื•ื•ืื” ืื—ืช.
03:50
So you have five million, that is, five million authors
86
230260
3000
ืื ื™ืฉ ืœื›ื 5 ืžื™ืœื™ื•ืŸ, ื–ื” ืื•ืžืจ 5 ืžื™ืœื™ื•ืŸ ืกื•ืคืจื™ื
03:53
and five million plaintiffs is a massive lawsuit.
87
233260
3000
ื•-5 ืžื™ืœื™ื•ืŸ ืชื‘ื™ืขื•ืช ืžืฉืคื˜ื™ื•ืช ืฉื–ื” ืžืžืฉ ื”ืžื•ืŸ.
03:56
So, although that would be really, really awesome,
88
236260
2000
ื›ืš ืฉื’ื ืื ื–ื” ื™ื”ื™ื” ืžืžืฉ, ืžืžืฉ ืžืจืฉื™ื,
03:58
again, that's extremely, extremely impractical.
89
238260
3000
ืฉื•ื‘, ื–ื” ืžืื•ื“, ืžืื•ื“ ืœื ืžืขืฉื™.
04:01
(Laughter)
90
241260
2000
(ืฆื—ื•ืง)
04:03
Now again, we kind of caved in,
91
243260
2000
ื•ืื ื• ืฉื•ื‘ ื•ื™ืชืจื ื•,
04:05
and we did the very practical approach, which was a bit less awesome.
92
245260
3000
ื•ืื™ืžืฆื ื• ืืช ื”ื’ื™ืฉื” ื”ืžืื•ื“ ืžืขืฉื™ืช, ืฉื”ื™ืชื” ืงืฆืช ืคื—ื•ืช ืžืจืฉื™ืžื”.
04:08
We said, well instead of releasing the full text,
93
248260
2000
ืืžืจื ื•, ื‘ืžืงื•ื ืœืฉื—ืจืจ ืืช ืžืœื•ื ื”ื˜ืงืกื˜,
04:10
we're going to release statistics about the books.
94
250260
2000
ื ืฉื—ืจืจ ืกื˜ื˜ื™ืกื˜ื™ืงื•ืช ืขืœ ื”ืกืคืจื™ื.
04:12
So take for instance "A gleam of happiness."
95
252260
2000
ื ื™ืงื— ืœื“ื•ื’ืžื "ืงื•ืจื˜ื•ื‘ ืฉืœ ืื•ืฉืจ".
04:14
It's four words; we call that a four-gram.
96
254260
2000
ืืœื• ื”ืŸ ืืจื‘ืข ืžื™ืœื™ื (ื‘ืื ื’ืœื™ืช); ื ืงืจื ืœื–ื” ืžืฉืงืœ-ืืจื‘ืข.
04:16
We're going to tell you how many times a particular four-gram
97
256260
2000
ื ืกืคืจ ืœื›ื ื›ืžื” ืคืขืžื™ื ืžืฉืงืœ-ืืจื‘ืข ืžืกื•ื™ื™ื
04:18
appeared in books in 1801, 1802, 1803,
98
258260
2000
ื”ื•ืคื™ืข ื‘ืกืคืจื™ื ื‘-1801, 1802, 1803,
04:20
all the way up to 2008.
99
260260
2000
ืขื“ 2008.
04:22
That gives us a time series
100
262260
2000
ื–ื” ื ื•ืชืŸ ืœื ื• ืžืจื•ื•ื—ื™ ื–ืžืŸ ืฉืœ ืชื“ื™ืจื•ืช ื”ืฉื™ืžื•ืฉ
04:24
of how frequently this particular sentence was used over time.
101
264260
2000
ื‘ื‘ื™ื˜ื•ื™ ืžืกื•ื™ื™ื ื–ื” ืœืื•ืจืš ืชืงื•ืคื”.
04:26
We do that for all the words and phrases that appear in those books,
102
266260
3000
ืื ื• ืขื•ืฉื™ื ื–ืืช ืœื›ืœ ื”ืžื™ืœื™ื ื•ื”ื‘ื™ื˜ื•ื™ื™ื ืืฉืจ ืžื•ืคื™ืขื™ื ื‘ืกืคืจื™ื ื”ืœืœื•,
04:29
and that gives us a big table of two billion lines
103
269260
3000
ื•ื–ื” ื ื•ืชืŸ ืœื ื• ื˜ื‘ืœื” ื’ื“ื•ืœื” ืฉืœ ืฉื ื™ ืžื™ืœื™ืืจื“ ืฉื•ืจื•ืช
04:32
that tell us about the way culture has been changing.
104
272260
2000
ื”ืžืกืคืจื•ืช ืœื ื• ืขืœ ื”ื“ืจืš ื‘ื” ืชืจื‘ื•ืช ืžืฉืชื ื”.
04:34
ELA: So those two billion lines,
105
274260
2000
ื.ืœ.ื.: ืฉื ื™ ืžื™ืœื™ืืจื“ ื”ืฉื•ืจื•ืช ื”ืœืœื•,
04:36
we call them two billion n-grams.
106
276260
2000
ืื ื• ืžื›ื ื™ื ืื•ืชืŸ ืฉื ื™ ืžื™ืœื™ืืจื“ ืžืฉืงืœื™-n.
04:38
What do they tell us?
107
278260
2000
ืžื” ื”ืŸ ืžืกืคืจื•ืช ืœื ื•?
04:40
Well the individual n-grams measure cultural trends.
108
280260
2000
ื›ืœ ืžืฉืงืœ-n ืœื›ืฉืขืฆืžื• ืžื•ื“ื“ ืžื’ืžื•ืช ืชืจื‘ื•ืชื™ื•ืช.
04:42
Let me give you an example.
109
282260
2000
ืืชืŸ ืœื›ื ื“ื•ื’ืžื.
04:44
Let's suppose that I am thriving,
110
284260
2000
ื ื ื™ื— ืฉืื ื™ ืžืฆืœื™ื— ื‘ืžืฉื”ื•,
04:46
then tomorrow I want to tell you about how well I did.
111
286260
2000
ืœื›ืŸ ืžื—ืจ ื‘ืจืฆื•ื ื™ ืœืกืคืจ ืœื›ื ืขืœ ื”ื”ืฆืœื—ื” ืฉืœื™.
04:48
And so I might say, "Yesterday, I throve."
112
288260
3000
ืื ื™ ืขืฉื•ื™ ืœื•ืžืจ, "ืืชืžื•ืœ, ื”ืฆืœื—ืชื™ (I throve)."
04:51
Alternatively, I could say, "Yesterday, I thrived."
113
291260
3000
ืื• ืœื—ื™ืœื•ืคื™ืŸ, ืืชืžื•ืœ, ื”ืฆืœื—ืชื™ (I thrived)".
04:54
Well which one should I use?
114
294260
3000
ื‘ืžื” ืขืœื™ื™ ืœื”ืฉืชืžืฉ?
04:57
How to know?
115
297260
2000
ื›ื™ืฆื“ ื™ื•ื“ืขื™ื?
04:59
As of about six months ago,
116
299260
2000
ืขื“ ืœืคื ื™ 6 ื—ื•ื“ืฉื™ื,
05:01
the state of the art in this field
117
301260
2000
ื”ืžืฆื‘ ื”ืขื“ื›ื ื™ ื‘ืชื—ื•ื ื–ื” ื”ื™ื”
05:03
is that you would, for instance,
118
303260
2000
ืฉื”ื•ืœื›ื™ื, ืœื“ื•ื’ืžื,
05:05
go up to the following psychologist with fabulous hair,
119
305260
2000
ืœืคืกื™ื›ื•ืœื•ื’ ื›ื–ื” ืขื ืฉื™ืขืจ ืžื“ื”ื™ื,
05:07
and you'd say,
120
307260
2000
ื•ืื•ืžืจื™ื,
05:09
"Steve, you're an expert on the irregular verbs.
121
309260
3000
"ืกื˜ื™ื‘, ืืชื” ืžื•ืžื—ื” ื‘ืคืขืœื™ื ื—ืจื™ื’ื™ื.
05:12
What should I do?"
122
312260
2000
ืžื” ืขืœื™ื™ ืœืขืฉื•ืช?"
05:14
And he'd tell you, "Well most people say thrived,
123
314260
2000
ื•ื”ื•ื ื”ื™ื” ืขื•ื ื”, "ืจื•ื‘ ื”ืื ืฉื™ื ืื•ืžืจื™ื thrived,
05:16
but some people say throve."
124
316260
3000
ืื‘ืœ ื›ืžื” ืื•ืžืจื™ื throve"
05:19
And you also knew, more or less,
125
319260
2000
ื’ื ืืชื ื™ื•ื“ืขื™ื, ืคื—ื•ืช ืื• ื™ื•ืชืจ,
05:21
that if you were to go back in time 200 years
126
321260
3000
ืฉืื ื”ื™ื™ืชื ื—ื•ื–ืจื™ื 200 ืฉื ื” ืื—ื•ืจื”
05:24
and ask the following statesman with equally fabulous hair,
127
324260
3000
ื•ืฉื•ืืœื™ื ืืช ื”ืžื“ื™ื ืื™ ื”ื–ื” ืฉื’ื ืœื• ื™ืฉ ืฉื™ืขืจ ืžื“ื”ื™ื,
05:27
(Laughter)
128
327260
3000
(ืฆื—ื•ืง)
05:30
"Tom, what should I say?"
129
330260
2000
"ื˜ื•ื, ืžื” ืขืœื™ื™ ืœื”ื’ื™ื“?"
05:32
He'd say, "Well, in my day, most people throve,
130
332260
2000
ื”ื•ื ื”ื™ื” ืขื•ื ื”, "ื‘ื–ืžื ื™, ืจื•ื‘ ื”ืื ืฉื™ื ื”ืฉืชืžืฉื• ื‘-throve,
05:34
but some thrived."
131
334260
3000
ืื‘ืœ ื›ืžื” ื‘-thrived".
05:37
So now what I'm just going to show you is raw data.
132
337260
2000
ื›ืขืช ืžื” ืฉืืจืื” ืœื›ื ื–ื” ื ืชื•ื ื™ื ื’ื•ืœืžื™ื™ื.
05:39
Two rows from this table of two billion entries.
133
339260
4000
ืฉืชื™ ืฉื•ืจื•ืช ืžื˜ื‘ืœื” ื–ื• ืฉืœ 2 ืžื™ืœื™ืืจื“ ืฉื•ืจื•ืช.
05:43
What you're seeing is year by year frequency
134
343260
2000
ืžื” ืฉืจื•ืื™ื ื–ื• ื”ืชื“ื™ืจื•ืช, ืฉื ื” ืื—ืจ ืฉื ื”,
05:45
of "thrived" and "throve" over time.
135
345260
3000
ืฉืœ "thrived" ืžื•ืœ "throve" ืœืื•ืจืš ื–ืžืŸ.
05:49
Now this is just two
136
349260
2000
ืืœื• ืจืง ืฉืชื™ ืฉื•ืจื•ืช
05:51
out of two billion rows.
137
351260
3000
ืžืชื•ืš 2 ืžื™ืœื™ืืจื“ ืฉื•ืจื•ืช.
05:54
So the entire data set
138
354260
2000
ื›ืš ืฉื›ืœ ืžืขืจืš ื”ื ืชื•ื ื™ื
05:56
is a billion times more awesome than this slide.
139
356260
3000
ืžืจืฉื™ื ืคื™ ืžื™ืœื™ืืจื“ ืžืืฉืจ ืฉืงื•ืคื™ืช ื–ื•.
05:59
(Laughter)
140
359260
2000
(ืฆื—ื•ืง)
06:01
(Applause)
141
361260
4000
(ืžื—ื™ืื•ืช ื›ืคื™ื™ื)
06:05
JM: Now there are many other pictures that are worth 500 billion words.
142
365260
2000
ื’'.ืž: ื™ืฉื ืŸ ื”ืจื‘ื” ืชืžื•ื ื•ืช ืื—ืจื•ืช
06:07
For instance, this one.
143
367260
2000
ื”ืฉื•ื•ืช 500 ืžื™ืœื™ืืจื“ ืžื™ืœื™ื. ืœืžืฉืœ ื–ื•.
06:09
If you just take influenza,
144
369260
2000
ืื ื ื™ืงื— ืืช ืฉืคืขืช,
06:11
you will see peaks at the time where you knew
145
371260
2000
ื ืจืื” ืฉื™ืื™ื ื‘ื–ืžื ื™ื ืฉืื ื• ื™ื•ื“ืขื™ื
06:13
big flu epidemics were killing people around the globe.
146
373260
3000
ืฉื‘ื”ื ืžื’ื™ืคื•ืช ื”ืฉืคืขืช ื—ื™ืกืœื• ืื ืฉื™ื ื‘ื›ืœ ื”ืขื•ืœื.
06:16
ELA: If you were not yet convinced,
147
376260
3000
ื.ืœ.ื.: ืื ืขื“ื™ื™ืŸ ืœื ื”ืฉืชื›ื ืขืชื,
06:19
sea levels are rising,
148
379260
2000
ืคื ื™-ื”ื™ื ืขื•ืœื™ื,
06:21
so is atmospheric CO2 and global temperature.
149
381260
3000
ื›ืš ื’ื ื“ื•-ืชื—ืžื•ืฆืช ื”ืคื—ืžืŸ ื‘ืื•ื™ืจ ื•ื”ื˜ืžืคืจื˜ื•ืจื” ื”ืžืžื•ืฆืขืช.
06:24
JM: You might also want to have a look at this particular n-gram,
150
384260
3000
ื’'.ืž.: ืื•ืœื™ ื’ื ืชืจืฆื• ืœืจืื•ืช ืืช ืžืฉืงืœ-n ื”ืžืกื•ื™ื™ื ื”ื–ื”,
06:27
and that's to tell Nietzsche that God is not dead,
151
387260
3000
ื•ื–ื” ื›ื“ื™ ืœืกืคืจ ืœื ื™ื˜ืฉื” ืฉืืœื•ื”ื™ื ืœื ืžืช,
06:30
although you might agree that he might need a better publicist.
152
390260
3000
ืืฃ ืขืœ-ืคื™ ืฉืชืกื›ื™ืžื• ืื•ืœื™ ืฉื”ื•ื ื–ืงื•ืง ืœื™ื—ืฆ"ืŸ ื™ื•ืชืจ ื˜ื•ื‘.
06:33
(Laughter)
153
393260
2000
(ืฆื—ื•ืง)
06:35
ELA: You can get at some pretty abstract concepts with this sort of thing.
154
395260
3000
ื.ืœ.ื.: ื ื™ืชืŸ ืœื”ื’ื™ืข ืœื›ืžื” ืชืคื™ืกื•ืช ืžื•ืคืฉื˜ื•ืช ืžื“ื‘ืจ ื›ื–ื”.
06:38
For instance, let me tell you the history
155
398260
2000
ืœื“ื•ื’ืžื, ืืกืคืจ ืœื›ื ืขืœ ื”ื”ื™ืกื˜ื•ืจื™ื”
06:40
of the year 1950.
156
400260
2000
ืฉืœ ืฉื ืช 1950.
06:42
Pretty much for the vast majority of history,
157
402260
2000
ืœืื•ืจืš ืจื•ื‘ ื”ื”ื™ืกื˜ื•ืจื™ื” ื‘ืงื™ืจื•ื‘,
06:44
no one gave a damn about 1950.
158
404260
2000
ืืฃ ืื—ื“ ืœื ืฉื ืขืœ ืฉื ืช 1950.
06:46
In 1700, in 1800, in 1900,
159
406260
2000
ื‘-1700, ื‘-1800, ื‘-1900,
06:48
no one cared.
160
408260
3000
ืœืืฃ ืื—ื“ ืœื ื”ื™ื” ืื›ืคืช.
06:52
Through the 30s and 40s,
161
412260
2000
ืœืื•ืจืš ืฉื ื•ืช ื”-30 ื•ื”-40,
06:54
no one cared.
162
414260
2000
ืœืืฃ ืื—ื“ ืœื ื”ื™ื” ืื›ืคืช.
06:56
Suddenly, in the mid-40s,
163
416260
2000
ืคืชืื•ื, ื‘ืืžืฆืข ืฉื ื•ืช ื”-40,
06:58
there started to be a buzz.
164
418260
2000
ื”ืชื—ื™ืœ ื”ื‘ืื–.
07:00
People realized that 1950 was going to happen,
165
420260
2000
ืื ืฉื™ื ื’ื™ืœื• ืฉ-1950 ืขื•ืžื“ืช ืœื”ื’ื™ืข,
07:02
and it could be big.
166
422260
2000
ื•ื”ื™ื ื™ื›ื•ืœื” ืœื”ื™ื•ืช ื“ื‘ืจ ื’ื“ื•ืœ.
07:04
(Laughter)
167
424260
3000
(ืฆื—ื•ืง)
07:07
But nothing got people interested in 1950
168
427260
3000
ืื‘ืœ ืฉื•ื ื“ื‘ืจ ืœื ื’ืจื ืœื”ื ืœื”ืชืขื ื™ื™ืŸ ื‘-1950
07:10
like the year 1950.
169
430260
3000
ื›ืžื• ื”ืฉื ื” 1950 ืขืฆืžื”.
07:13
(Laughter)
170
433260
3000
(ืฆื—ื•ืง)
07:16
People were walking around obsessed.
171
436260
2000
ืื ืฉื™ื ื”ืชื”ืœื›ื• ืขื ืื•ื‘ืกืกื™ื” ื‘ืชื•ื›ื.
07:18
They couldn't stop talking
172
438260
2000
ื”ื ืœื ื™ื›ืœื• ืœื”ืคืกื™ืง ืœื“ื‘ืจ
07:20
about all the things they did in 1950,
173
440260
3000
ืขืœ ื›ืœ ื”ื“ื‘ืจื™ื ืฉื”ื ืขืฉื• ื‘-1950,
07:23
all the things they were planning to do in 1950,
174
443260
3000
ื›ืœ ื”ื“ื‘ืจื™ื ืฉื”ื ืชื™ื›ื ื ื• ืœืขืฉื•ืช ื‘-1950,
07:26
all the dreams of what they wanted to accomplish in 1950.
175
446260
5000
ื›ืœ ื”ื—ืœื•ืžื•ืช ืฉื”ื ืจืฆื• ืœื”ื’ืฉื™ื ื‘-1950.
07:31
In fact, 1950 was so fascinating
176
451260
2000
ืœืžืขืฉื”, 1950 ื”ื™ืชื” ื›ื” ืžืจืชืงืช
07:33
that for years thereafter,
177
453260
2000
ืฉื‘ืฉื ื™ื ืฉืœืื—ืจื™ื”,
07:35
people just kept talking about all the amazing things that happened,
178
455260
3000
ืื ืฉื™ื ืคืฉื•ื˜ ื”ืžืฉื™ื›ื• ืœื“ื‘ืจ ืขืœ ื›ืœ ื”ื“ื‘ืจื™ื ื”ืžื“ื”ื™ืžื™ื ืฉืงืจื•,
07:38
in '51, '52, '53.
179
458260
2000
ื‘-51, 52, 53.
07:40
Finally in 1954,
180
460260
2000
ืœื‘ืกื•ืฃ ื‘-1954,
07:42
someone woke up and realized
181
462260
2000
ืžื™ืฉื”ื• ื”ืชืขื•ืจืจ ื•ืฉื ืœื‘
07:44
that 1950 had gotten somewhat passรฉ.
182
464260
4000
ืฉ-1950 ืื™ื›ืฉื”ื• ืขื‘ืจ ื–ืžื ื”.
07:48
(Laughter)
183
468260
2000
(ืฆื—ื•ืง)
07:50
And just like that, the bubble burst.
184
470260
2000
ื•ื›ื›ื” ืกืชื, ื”ื‘ืœื•ืŸ ื”ืชืคื•ืฆืฅ.
07:52
(Laughter)
185
472260
2000
(ืฆื—ื•ืง)
07:54
And the story of 1950
186
474260
2000
ื•ืกื™ืคื•ืจื” ืฉืœ 1950 ื”ื•ื ื”ืกื™ืคื•ืจ
07:56
is the story of every year that we have on record,
187
476260
2000
ืฉืœ ื›ืœ ืฉื ื” ืฉื™ืฉ ืขืœื™ื” ืจืฉื•ืžื•ืช,
07:58
with a little twist, because now we've got these nice charts.
188
478260
3000
ืขื ืฉื™ื ื•ื™ ืงื˜ืŸ, ื›ื™ ื›ืขืช ื™ืฉ ืœื ื• ืืช ื”ืชืจืฉื™ืžื™ื ื”ื™ืคื™ื ื”ืืœื”.
08:01
And because we have these nice charts, we can measure things.
189
481260
3000
ื•ืžืื—ืจ ื•ื™ืฉ ืœื ื• ืื•ืชื, ืื ื• ื™ื›ื•ืœื™ื ืœืžื“ื•ื“ ื“ื‘ืจื™ื ืฉื•ื ื™ื.
08:04
We can say, "Well how fast does the bubble burst?"
190
484260
2000
ืื ื• ื™ื›ื•ืœื™ื ืœืฉืื•ืœ, "ื›ืžื” ืžื”ืจ ื”ื‘ืœื•ืŸ ืžืชืคื•ืฆืฅ?"
08:06
And it turns out that we can measure that very precisely.
191
486260
3000
ืžืชื‘ืจืจ ืฉื ื™ืชืŸ ืœืžื“ื•ื“ ื–ืืช ื‘ื“ื™ื•ืง ืžืื•ื“ ื’ื‘ื•ื”.
08:09
Equations were derived, graphs were produced,
192
489260
3000
ืžื–ื” ื ื•ืฆืจื• ืžืฉื•ื•ืื•ืช, ื ื•ืฆืจื• ื’ืจืคื™ื,
08:12
and the net result
193
492260
2000
ื•ื”ืชื•ืฆืื” ื”ืกื•ืคื™ืช ื”ื™ื
08:14
is that we find that the bubble bursts faster and faster
194
494260
3000
ืฉืžืฆืื ื• ืฉื”ื‘ืœื•ืŸ ืžืชืคื•ืฆืฅ ื™ื•ืชืจ ื•ื™ื•ืชืจ ืžื”ืจ
08:17
with each passing year.
195
497260
2000
ืขื ื›ืœ ืฉื ื” ืฉืขื•ื‘ืจืช.
08:19
We are losing interest in the past more rapidly.
196
499260
5000
ืื ื• ืžืื‘ื“ื™ื ืขื ื™ื™ืŸ ื‘ืขื‘ืจ ื‘ืงืฆื‘ ื”ื•ืœืš ื•ื’ื•ื‘ืจ.
08:24
JM: Now a little piece of career advice.
197
504260
2000
ื’'.ืž.: ื•ืขื›ืฉื™ื• ืขืฆื” ืงื˜ื ื” ื‘ื ื•ืฉื ืงืจื™ื™ืจื”.
08:26
So for those of you who seek to be famous,
198
506260
2000
ืขื‘ื•ืจ ืืœื” ืžื›ื ืฉืฉื•ืืคื™ื ืœื”ืชืคืจืกื,
08:28
we can learn from the 25 most famous political figures,
199
508260
2000
ื ื™ืชืŸ ืœืœืžื•ื“ ืž-25 ื”ืคื•ืœื™ื˜ื™ืงืื™ื ื”ืžื•ื‘ื™ืœื™ื,
08:30
authors, actors and so on.
200
510260
2000
ื”ืกื•ืคืจื™ื, ื”ืฉื—ืงื ื™ื ื•ืขื•ื“.
08:32
So if you want to become famous early on, you should be an actor,
201
512260
3000
ืื ื‘ืจืฆื•ื ื›ื ืœื”ืชืคืจืกื ืžื•ืงื“ื, ืขืœื™ื›ื ืœื”ื™ื•ืช ืฉื—ืงื ื™ื,
08:35
because then fame starts rising by the end of your 20s --
202
515260
2000
ืžื›ื™ื•ื•ืŸ ืฉื”ืคื™ืจืกื•ื ืžืชื—ื™ืœ ืœื˜ืคืก ื‘ืกื•ืฃ ืฉื ื•ืช ื”-20 ืฉืœื›ื --
08:37
you're still young, it's really great.
203
517260
2000
ืืชื ืขื“ื™ื™ืŸ ืฆืขื™ืจื™ื ื•ื–ื” ื ื”ื“ืจ.
08:39
Now if you can wait a little bit, you should be an author,
204
519260
2000
ืื ืืชื ื™ื›ื•ืœื™ื ืœื”ืžืชื™ืŸ ืžืขื˜, ืขืœื™ื›ื ืœื”ื™ื•ืช ืกื•ืคืจื™ื,
08:41
because then you rise to very great heights,
205
521260
2000
ืžืื—ืจ ื•ืื– ืืชื ืžื˜ืคืกื™ื ืœื’ื‘ื”ื™ื ื’ื“ื•ืœื™ื,
08:43
like Mark Twain, for instance: extremely famous.
206
523260
2000
ื›ืžื• ืžืจืง ื˜ื•ื•ื™ื™ืŸ: ืžืื•ื“ ืžืคื•ืจืกื.
08:45
But if you want to reach the very top,
207
525260
2000
ืื‘ืœ ืื ื‘ืจืฆื•ื ื›ื ืœื”ื’ื™ืข ืžืžืฉ ืœืคื™ืกื’ื”,
08:47
you should delay gratification
208
527260
2000
ืขืœื™ื›ื ืœื“ื—ื•ืช ืกื™ืคื•ืงื™ื
08:49
and, of course, become a politician.
209
529260
2000
ื•ื›ืžื•ื‘ืŸ, ืœื”ื™ื•ืช ืคื•ืœื™ื˜ื™ืงืื™.
08:51
So here you will become famous by the end of your 50s,
210
531260
2000
ื›ืืŸ ืชื”ื™ื• ืžืคื•ืจืกืžื™ื ื‘ืกื•ืฃ ืฉื ื•ืช ื”-50 ืฉืœื›ื,
08:53
and become very, very famous afterward.
211
533260
2000
ื•ืชื”ื™ื• ืžืื•ื“, ืžืื•ื“ ืžืคื•ืจืกืžื™ื ืื—ืจ-ื›ืš.
08:55
So scientists also tend to get famous when they're much older.
212
535260
3000
ื’ื ื”ืžื“ืขื ื™ื ื ื•ื˜ื™ื ืœื”ืชืคืจืกื ื›ืืฉืจ ื”ื ืžื‘ื•ื’ืจื™ื ื‘ื”ืจื‘ื”.
08:58
Like for instance, biologists and physics
213
538260
2000
ื›ืžื• ืœื“ื•ื’ืžื, ื‘ื™ื•ืœื•ื’ื™ื ื•ืคื™ื–ื™ืงืื™ื
09:00
tend to be almost as famous as actors.
214
540260
2000
ืฉืžืงื‘ืœื™ื ืคื™ืจืกื•ื ื›ืžื• ืฉื—ืงื ื™ื.
09:02
One mistake you should not do is become a mathematician.
215
542260
3000
ืฉื’ื™ืื” ืื—ืช ืฉืขืœื™ื›ื ืœื”ื™ืžื ืข ืžืžื ื” ื–ื” ืœื”ื™ื•ืช ืžืชืžื˜ื™ืงืื™.
09:05
(Laughter)
216
545260
2000
(ืฆื—ื•ืง)
09:07
If you do that,
217
547260
2000
ืื ืชืขืฉื• ื–ืืช,
09:09
you might think, "Oh great. I'm going to do my best work when I'm in my 20s."
218
549260
3000
ืื•ืœื™ ืชื—ืฉื‘ื•, "ื ื”ื“ืจ, ืื’ื™ืข ืœืฉื™ืื™ ื‘ืฉื ื•ืช ื”-20 ืฉืœื™."
09:12
But guess what, nobody will really care.
219
552260
2000
ืื‘ืœ ืืชื ื™ื•ื“ืขื™ื ืžื”? ืœืืฃ ืื—ื“ ื–ื” ืœื ื™ื”ื™ื” ืื›ืคืช.
09:14
(Laughter)
220
554260
3000
(ืฆื—ื•ืง)
09:17
ELA: There are more sobering notes
221
557260
2000
ื.ืœ.ื.: ื™ืฉื ืŸ ืชื•ื‘ื ื•ืช ื ื•ืกืคื•ืช ืžืื™ืจื•ืช-ืขื™ื ื™ื™ื
09:19
among the n-grams.
222
559260
2000
ื‘ืชื•ืš ื”ืžืฉืงืœื™-n.
09:21
For instance, here's the trajectory of Marc Chagall,
223
561260
2000
ืœื“ื•ื’ืžื, ื”ื ื” ื”ืžืกืœื•ืœ ืฉืœ ืžืจืง ืฉืื’ืœ,
09:23
an artist born in 1887.
224
563260
2000
ืืžืŸ ื™ืœื™ื“ 1887.
09:25
And this looks like the normal trajectory of a famous person.
225
565260
3000
ื•ื–ื” ื ืจืื” ื›ืžืกืœื•ืœ ืจื’ื™ืœ ืฉืœ ืื“ื ืฉื”ืชืคืจืกื.
09:28
He gets more and more and more famous,
226
568260
4000
ื”ื•ื ื ื”ื™ื” ื™ื•ืชืจ ื•ื™ื•ืชืจ ืžืคื•ืจืกื,
09:32
except if you look in German.
227
572260
2000
ืืœื ืื ื‘ื•ื“ืงื™ื ื‘ืฉืคื” ื”ื’ืจืžื ื™ืช.
09:34
If you look in German, you see something completely bizarre,
228
574260
2000
ืื ื‘ื•ื“ืงื™ื ื‘ื’ืจืžื ื™ืช, ืจื•ืื™ื ืžืฉื”ื• ืœื’ืžืจื™ ืžื•ื–ืจ,
09:36
something you pretty much never see,
229
576260
2000
ืžืฉื”ื• ืฉื›ืžืขื˜ ื•ืœื ืจื•ืื™ื,
09:38
which is he becomes extremely famous
230
578260
2000
ืฉื–ื” ืฉื”ื•ื ื ื”ื™ื” ืžืื•ื“ ืžืคื•ืจืกื
09:40
and then all of a sudden plummets,
231
580260
2000
ื•ืื– ืคืชืื•ื ืฆื•ืœืœ ืœืชื—ืชื™ืช,
09:42
going through a nadir between 1933 and 1945,
232
582260
3000
ืขื•ื‘ืจ ืฉืคืœ ื‘ื™ืŸ 1933 ื•-1945,
09:45
before rebounding afterward.
233
585260
3000
ืœืคื ื™ ืขืœื™ื™ืชื• ืžื—ื“ืฉ.
09:48
And of course, what we're seeing
234
588260
2000
ื•ื‘ืขืฆื, ืžื” ืฉืจื•ืื™ื ื–ื• ื”ืขื•ื‘ื“ื”
09:50
is the fact Marc Chagall was a Jewish artist
235
590260
3000
ืฉืžืจืง ืฉืื’ืืœ ื”ื™ื” ืืžืŸ ื™ื”ื•ื“ื™
09:53
in Nazi Germany.
236
593260
2000
ื‘ื’ืจืžื ื™ื” ื”ื ืืฆื™ืช.
09:55
Now these signals
237
595260
2000
ื”ืกื™ืžื ื™ื ื”ืืœื”
09:57
are actually so strong
238
597260
2000
ื”ื ื›ื” ื—ื–ืงื™ื
09:59
that we don't need to know that someone was censored.
239
599260
3000
ืฉืื™ืŸ ืฆื•ืจืš ืœื“ืขืช ืฉืžื™ืฉื”ื• ืฆื•ื ื–ืจ.
10:02
We can actually figure it out
240
602260
2000
ื ื™ืชืŸ ืคืฉื•ื˜ ืœื”ืกื™ืง ื–ืืช
10:04
using really basic signal processing.
241
604260
2000
ื‘ืขื–ืจืช ืขื™ื‘ื•ื“ ื ืชื•ื ื™ื ื‘ืกื™ืกื™.
10:06
Here's a simple way to do it.
242
606260
2000
ื”ื ื” ื“ืจืš ืคืฉื•ื˜ื” ืœืขืฉื•ืช ืืช ื–ื”.
10:08
Well, a reasonable expectation
243
608260
2000
ืกื‘ื™ืจ ืœืฆืคื•ืช ืฉืžื™ื“ืช ื”ืคื™ืจืกื•ื
10:10
is that somebody's fame in a given period of time
244
610260
2000
ืฉืœ ืคืœื•ื ื™ ื‘ื–ืžืŸ ื ืชื•ืŸ ืชื”ื™ื”
10:12
should be roughly the average of their fame before
245
612260
2000
ื‘ืงื™ืจื•ื‘ ื”ืžืžื•ืฆืข ืฉืœ ืคื™ืจืกื•ืžื•
10:14
and their fame after.
246
614260
2000
ืœืคื ื™ ืื•ืชื• ื–ืžืŸ ื•ืคื™ืจืกื•ืžื• ืื—ืจื™ื•.
10:16
So that's sort of what we expect.
247
616260
2000
ื–ื” ื‘ืขืจืš ืžื” ืฉืื ื• ืžืฆืคื™ื
10:18
And we compare that to the fame that we observe.
248
618260
3000
ื•ืžืฉื•ื•ื™ื ืืช ื”ืชื•ืฆืื” ืœืžื™ื“ืช ื”ืคื™ืจืกื•ื ื‘ืคื•ืขืœ.
10:21
And we just divide one by the other
249
621260
2000
ืžื—ืœืงื™ื ืืช ื”ืื—ื“ ื‘ืฉื ื™
10:23
to produce something we call a suppression index.
250
623260
2000
ื›ื“ื™ ืœืงื‘ืœ ืžื” ืฉื ืงืจื ืžื“ื“ ื“ื™ื›ื•ื™.
10:25
If the suppression index is very, very, very small,
251
625260
3000
ืื ืžื“ื“ ื”ื“ื™ื›ื•ื™ ืžืื•ื“, ืžืื•ื“ ืงื˜ืŸ,
10:28
then you very well might be being suppressed.
252
628260
2000
ื–ื” ืื•ืžืจ ืฉืžื™ืฉื”ื• ื›ื ืจืื” ืกื•ื‘ืœ ืžื“ื™ื›ื•ื™.
10:30
If it's very large, maybe you're benefiting from propaganda.
253
630260
3000
ืื ื”ื•ื ืžืื•ื“ ื’ื“ื•ืœ, ืื•ืœื™ ืžื™ืฉื”ื• ื ื”ื ื” ืžืชืขืžื•ืœื”.
10:34
JM: Now you can actually look at
254
634260
2000
ื’'.ืž.: ื ื™ืชืŸ ื‘ืขืฆื ืœื”ืกืชื›ืœ ืขืœ
10:36
the distribution of suppression indexes over whole populations.
255
636260
3000
ืคื™ืœื•ื’ ืžื“ื“ื™ ื”ื“ื™ื›ื•ื™ ืขืœ-ืคื ื™ ื”ืื•ื›ืœื•ืกื™ื™ื” ื›ื•ืœื”.
10:39
So for instance, here --
256
639260
2000
ืœื“ื•ื’ืžื, ืžื“ื“ ื“ื™ื›ื•ื™ ื–ื”
10:41
this suppression index is for 5,000 people
257
641260
2000
ื”ื•ื ืฉืœ 5,000 ืื ืฉื™ื
10:43
picked in English books where there's no known suppression --
258
643260
2000
ื”ืœืงื•ื— ืžืกืคืจื™ื ื‘ืื ื’ืœื™ื” ืฉืœื ืืžื•ืจ ืœื”ื™ื•ืช ืฉื ื“ื™ื›ื•ื™ --
10:45
it would be like this, basically tightly centered on one.
259
645260
2000
ื”ื•ื ื™ื™ืจืื” ื›ืš, ื‘ื’ื“ื•ืœ ืžืจื•ื›ื– ืกื‘ื™ื‘ 1.
10:47
What you expect is basically what you observe.
260
647260
2000
ืžื” ืฉืžืฆืคื™ื ืœื• ื–ื” ื‘ืขืจืš ืžื” ืฉืจื•ืื™ื ื›ืืŸ.
10:49
This is distribution as seen in Germany --
261
649260
2000
ื–ื” ื”ืคื™ืœื•ื’ ืฉืžืชืงื‘ืœ ื‘ื’ืจืžื ื™ื” --
10:51
very different, it's shifted to the left.
262
651260
2000
ืžืื•ื“ ืฉื•ื ื”, ื”ื•ื ืžื•ืกื˜ ืฉืžืืœื”.
10:53
People talked about it twice less as it should have been.
263
653260
3000
ืื ืฉื™ื ืฉื ื“ื™ื‘ืจื• ื›ืคืœื™ื™ื ืคื—ื•ืช ืžืžื” ืฉื”ื™ื” ื ื™ืชืŸ ืœืฆืคื•ืช.
10:56
But much more importantly, the distribution is much wider.
264
656260
2000
ืื‘ืœ ื™ื•ืชืจ ื—ืฉื•ื‘, ื”ืคื™ืœื•ื’ ื”ืจื‘ื” ื™ื•ืชืจ ืจื—ื‘.
10:58
There are many people who end up on the far left on this distribution
265
658260
3000
ื™ืฉ ื”ืจื‘ื” ืื ืฉื™ื ื”ื ืžืฆืื™ื ื‘ืงืฆื” ื”ืฉืžืืœื™ ืฉืœ ืคื™ืœื•ื’ ื–ื”
11:01
who are talked about 10 times fewer than they should have been.
266
661260
3000
ืืฉืจ ื“ื™ื‘ืจื• ืคื™-10 ืคื—ื•ืช ืžืžื” ืฉื”ื™ื• "ืฆืจื™ื›ื™ื".
11:04
But then also many people on the far right
267
664260
2000
ืื‘ืœ ื™ืฉ ื’ื ื”ืจื‘ื” ืื ืฉื™ื ื‘ืงืฆื” ื”ื™ืžื ื™
11:06
who seem to benefit from propaganda.
268
666260
2000
ืฉื ืจืื” ืฉื”ื ืžืจื•ื™ื—ื™ื ืžืชืขืžื•ืœื”.
11:08
This picture is the hallmark of censorship in the book record.
269
668260
3000
ืชืžื•ื ื” ื–ื• ื”ื™ื ื”ืžืืคื™ื™ืŸ ื”ืžื–ื”ื”, ื‘ืžืกื“ ื ืชื•ื ื™ ื”ืกืคืจื™ื, ืœืฆื ื–ื•ืจื”.
11:11
ELA: So culturomics
270
671260
2000
ื.ืœ.ื.: ืื ื• ืžื›ื ื™ื ืฉื™ื˜ื” ื–ื•
11:13
is what we call this method.
271
673260
2000
culturomics (ื—ืงืจ ืชื•ืจืฉื” ืชืจื‘ื•ืชื™ืช).
11:15
It's kind of like genomics.
272
675260
2000
ื–ื” ื“ื•ืžื” ืœื—ืงืจ ื”ืชื•ืจืฉื” ื‘ื‘ื™ื•ืœื•ื’ื™ื”.
11:17
Except genomics is a lens on biology
273
677260
2000
ื—ืงืจ ื”ืชื•ืจืฉื” ืฉื ืขื“ืฉื” ืขืœ ื‘ื™ื•ืœื•ื’ื™ื”
11:19
through the window of the sequence of bases in the human genome.
274
679260
3000
ื“ืจืš ื”ื—ืœื•ืŸ ืฉืœ ืกื“ืจื•ืช ืฉืœ ืจืฆืคื™ ื‘ืกื™ืก ื‘ื—ื•ืžืจ ื”ืชื•ืจืฉืชื™ ื”ืื ื•ืฉื™.
11:22
Culturomics is similar.
275
682260
2000
ื—ืงืจ ืชื•ืจืฉื” ืชืจื‘ื•ืชื™ืช ื–ื” ืžืฉื”ื• ื“ื•ืžื”.
11:24
It's the application of massive-scale data collection analysis
276
684260
3000
ื–ื” ืฉื™ืžื•ืฉ ื‘ืื ืœื™ื–ื” ืฉืœ ืื™ืกื•ืฃ ื ืชื•ื ื™ื ื‘ืงื ื”-ืžื™ื“ื” ืขื ืงื™
11:27
to the study of human culture.
277
687260
2000
ืœื—ืงืจ ืฉืœ ืชืจื‘ื•ืช ืื ื•ืฉื™ืช.
11:29
Here, instead of through the lens of a genome,
278
689260
2000
ื›ืืŸ, ื‘ืžืงื•ื ืœื”ื‘ื™ื˜ ื“ืจืš ืขื“ืฉืช ื”ื—ื•ืžืจ ื”ืชื•ืจืฉืชื™,
11:31
through the lens of digitized pieces of the historical record.
279
691260
3000
ืžื‘ื™ื˜ื™ื ื“ืจืš ืขื“ืฉื” ืฉืœ ืคื™ืกื•ืช ืฉืœ ืจืฉื•ืžื•ืช ื”ื™ืกื˜ื•ืจื™ื•ืช ืฉืขื‘ืจื• ื“ื™ื’ื™ื˜ืœื™ื–ืฆื™ื”.
11:34
The great thing about culturomics
280
694260
2000
ื”ื“ื‘ืจ ื”ื’ื“ื•ืœ ื‘ื—ืงืจ ืชื•ืจืฉื” ืชืจื‘ื•ืชื™ืช
11:36
is that everyone can do it.
281
696260
2000
ื”ื•ื ืฉื›ืœ ืื—ื“ ื™ื›ื•ืœ ืœืขืฉื•ืช ื–ืืช.
11:38
Why can everyone do it?
282
698260
2000
ืžื“ื•ืข ื›ืœ ืื—ื“ ื™ื›ื•ืœ ืœืขืฉื•ืช ื–ืืช?
11:40
Everyone can do it because three guys,
283
700260
2000
ื–ื” ืžื›ื™ื•ื•ืŸ ืฉืฉืœื•ืฉื” ืื ืฉื™ื,
11:42
Jon Orwant, Matt Gray and Will Brockman over at Google,
284
702260
3000
ื’'ื•ืŸ ืื•ืจื•ื•ื ื˜, ืžืื˜ ื’ืจื™ื™ ื•ื•ื™ืœ ื‘ืจื•ืงืžืŸ ืžื’ื•ื’ืœ,
11:45
saw the prototype of the Ngram Viewer,
285
705260
2000
ืจืื• ืืช ืื‘-ื”ื˜ื™ืคื•ืก ืฉืœ ืžืฆื’ืช ื”ืžืฉืงืœ-n,
11:47
and they said, "This is so fun.
286
707260
2000
ื•ืืžืจื•, "ื–ื” ื›ื–ื” ื›ื™ืฃ.
11:49
We have to make this available for people."
287
709260
3000
ืขืœื™ื ื• ืœื”ืคื›ื” ืœื–ืžื™ื ื” ืœืฆื™ื‘ื•ืจ."
11:52
So in two weeks flat -- the two weeks before our paper came out --
288
712260
2000
ืชื•ืš ืฉื‘ื•ืขื™ื™ื -- ืฉื‘ื•ืขื™ื™ื ืœืคื ื™ ืฉื”ืžืืžืจ ืฉืœื ื• ื”ืชืคืจืกื --
11:54
they coded up a version of the Ngram Viewer for the general public.
289
714260
3000
ื”ื ื›ืชื‘ื• ืชื•ื›ื ื™ืช ืœื’ื™ืจืกืช ืžืฆื’ืช ืžืฉืงืœ-n ื‘ืฉื‘ื™ืœ ื›ืœืœ ื”ืฆื™ื‘ื•ืจ.
11:57
And so you too can type in any word or phrase that you're interested in
290
717260
3000
ื ื™ืชืŸ ืœื”ืงืœื™ื“ ื›ืœ ืžื™ืœื” ืื• ื‘ื™ื˜ื•ื™ ืฉื—ืคืฆื™ื ื‘ื”ื
12:00
and see its n-gram immediately --
291
720260
2000
ื•ืœืจืื•ืช ืืช ืžืฉืงืœ ื”-n ืฉืœื”ื ืžื™ื™ื“ --
12:02
also browse examples of all the various books
292
722260
2000
ื•ื’ื ืœื”ืฆื™ื’ ื“ื•ื’ืžืื•ืช ืฉืœ ื›ืœ ื”ืกืคืจื™ื ื”ืžื’ื•ื•ื ื™ื
12:04
in which your n-gram appears.
293
724260
2000
ืฉื‘ื”ื ืžื•ืคื™ืข ื”ืžืฉืงืœ-n ืฉื‘ื—ืจืช.
12:06
JM: Now this was used over a million times on the first day,
294
726260
2000
ื’'.ืž.: ื ืขืฉื” ื‘ื–ื” ืฉื™ืžื•ืฉ ื™ื•ืชืจ ืžืžื™ืœื™ื•ืŸ ืคืขื ื‘ื™ื•ื ื”ืจืืฉื•ืŸ,
12:08
and this is really the best of all the queries.
295
728260
2000
ื•ื–ื• ื‘ืืžืช ื”ืฉืื™ืœืชื ื”ื˜ื•ื‘ื” ื‘ื™ื•ืชืจ ืžื›ื•ืœืŸ.
12:10
So people want to be their best, put their best foot forward.
296
730260
3000
ืื ืฉื™ื ืจื•ืฆื™ื ืืช ื”ื˜ื•ื‘ ื‘ื™ื•ืชืจ, ืœื”ื ื™ื— ืืช ื”ืจื’ืœ ื”ื™ื•ืชืจ ื˜ื•ื‘ื” ืžืœืคื ื™ื.
12:13
But it turns out in the 18th century, people didn't really care about that at all.
297
733260
3000
ืื‘ืœ ืžืชื‘ืจืจ ืฉื‘ืžืื” ื”-18, ืœืื ืฉื™ื ืœื ื”ื™ื” ืžืžืฉ ืื›ืคืช ืžื›ืœ ื–ื”.
12:16
They didn't want to be their best, they wanted to be their beft.
298
736260
3000
ื”ื ืœื ืจืฆื• ืืช ื”ื˜ื•ื‘ ื‘ื™ื•ืชืจ (best), ืืœื ืืช ื”ื˜ื•ื‘ ื‘ื™ื•ืชืจ (beft).
12:19
So what happened is, of course, this is just a mistake.
299
739260
3000
ืœื›ืŸ ืžื” ืฉืงืจื” ื”ื•ื, ื˜ื•ื‘, ื‘ืจื•ืจ ืฉื–ื• ื˜ืขื•ืช.
12:22
It's not that strove for mediocrity,
300
742260
2000
ื–ื” ืœื ืฉื”ื ืฉืืคื• ืœื‘ื™ื ื•ื ื™ื•ืช,
12:24
it's just that the S used to be written differently, kind of like an F.
301
744260
3000
ืืœื ืฉื”ื™ื• ื ื•ื”ื’ื™ื ืœื›ืชื•ื‘ S ื‘ืฆื•ืจื” ืฉื•ื ื”, ื‘ืขืจืš ื›ืžื• F.
12:27
Now of course, Google didn't pick this up at the time,
302
747260
3000
ื’ื•ื’ืœ ืœื ืชืคืกื• ื–ืืช ื‘ื–ืžื ื•,
12:30
so we reported this in the science article that we wrote.
303
750260
3000
ืœื›ืŸ ื“ื™ื•ื•ื—ื ื• ืขืœ ื›ืš ื‘ืžืืžืจ ื”ืžื“ืขื™ ืฉื›ืชื‘ื ื•.
12:33
But it turns out this is just a reminder
304
753260
2000
ืื‘ืœ ืžืชื‘ืจืจ ืฉื–ื” ืจืง ืžื–ื›ื™ืจ ืœื ื•
12:35
that, although this is a lot of fun,
305
755260
2000
ืฉืœืžืจื•ืช ืฉื›ืœ ื–ื” ื›ื™ืฃ ื’ื“ื•ืœ,
12:37
when you interpret these graphs, you have to be very careful,
306
757260
2000
ื›ืืฉืจ ืžืคืจืฉื™ื ืืช ื”ื’ืจืคื™ื ื”ืœืœื•, ืฆืจื™ืš ืžืื•ื“ ืœื”ื™ื–ื”ืจ,
12:39
and you have to adopt the base standards in the sciences.
307
759260
3000
ื•ืฉืขืœื™ื ื• ืœืืžืฅ ืกื˜ื ื“ืจื˜ื™ื ื‘ืกื™ืกื™ื™ื ืฉืœ ืžื“ืข.
12:42
ELA: People have been using this for all kinds of fun purposes.
308
762260
3000
ื.ืœ.ื: ืื ืฉื™ื ื ื•ื”ื’ื™ื ืœื”ืฉืชืžืฉ ื‘ื–ื” ืœื›ืœ ืžื™ื ื™ ืžื˜ืจื•ืช.
12:45
(Laughter)
309
765260
7000
(ืฆื—ื•ืง)
12:52
Actually, we're not going to have to talk,
310
772260
2000
ื‘ืขืฆื, ืื™ืŸ ืœื ื• ื™ื•ืชืจ ืฆื•ืจืš ืœื“ื‘ืจ,
12:54
we're just going to show you all the slides and remain silent.
311
774260
3000
ืื ื• ืจืง ื ืจืื” ืœื›ื ืืช ื›ืœ ื”ืฉืงื•ืคื™ื•ืช ื•ื ื™ืฉืืจ ื“ื•ืžืžื™ื.
12:57
This person was interested in the history of frustration.
312
777260
3000
ืื“ื ื–ื” ื”ืชืขื ื™ื™ืŸ ื‘ื”ื™ืกื˜ื•ืจื™ื™ืช ื”ืชื™ืกื›ื•ืœ.
13:00
There's various types of frustration.
313
780260
3000
ื™ืฉื ื ืกื•ื’ื™ ืชื™ืกื›ื•ืœ ืฉื•ื ื™ื.
13:03
If you stub your toe, that's a one A "argh."
314
783260
3000
ืื ื ืคื’ืขื™ื ื‘ื‘ื•ื”ืŸ, ื™ืฉ ืื—ื“ "ืืจื’".
13:06
If the planet Earth is annihilated by the Vogons
315
786260
2000
ืื ื›ื“ื•ืจ-ื”ืืจืฅ ืžื•ืฉืžื“ ืขืœ-ื™ื“ื™ ื”ื•ื•ื’ื•ื ื™ื
13:08
to make room for an interstellar bypass,
316
788260
2000
ื›ื“ื™ ืœืคื ื•ืช ื“ืจืš ืœืžืขื‘ืจ ื‘ื™ืŸ-ื›ื•ื›ื‘ื™,
13:10
that's an eight A "aaaaaaaargh."
317
790260
2000
ื–ื” ืฉืžื•ื ื” ื "ืืืืืืืืืจื’".
13:12
This person studies all the "arghs,"
318
792260
2000
ืื“ื ื–ื” ื—ื•ืงืจ ืืช ื›ืœ ื”"ืืจื’ื™ื",
13:14
from one through eight A's.
319
794260
2000
ืžืื—ื“ ืขื“ ืฉืžื•ื ื” ื-ื™ื.
13:16
And it turns out
320
796260
2000
ืžืชื‘ืจืจ
13:18
that the less-frequent "arghs"
321
798260
2000
ืฉื”"ืืจื’ื™ื" ื”ืคื—ื•ืช ื ืคื•ืฆื™ื
13:20
are, of course, the ones that correspond to things that are more frustrating --
322
800260
3000
ื”ื ืืœื” ืืฉืจ ืงืฉื•ืจื™ื ื‘ื“ื‘ืจื™ื ื”ื™ื•ืชืจ ืžืชืกื›ืœื™ื --
13:23
except, oddly, in the early 80s.
323
803260
3000
ืžืœื‘ื“, ื‘ืื•ืคืŸ ืžืฉื•ื ื”, ื‘ืฉื ื•ืช ื”-80 ื”ืžื•ืงื“ืžื•ืช.
13:26
We think that might have something to do with Reagan.
324
806260
2000
ืื ื• ืกื‘ื•ืจื™ื ืฉื–ื” ืขืฉื•ื™ ืœื”ื™ื•ืช ืงืฉื•ืจ ืื™ื›ืฉื”ื• ื‘ืจื™ื™ื’ืŸ.
13:28
(Laughter)
325
808260
2000
(ืฆื—ื•ืง)
13:30
JM: There are many usages of this data,
326
810260
3000
ื’'.ืž.: ื™ืฉื ื ื”ืจื‘ื” ืฉื™ืžื•ืฉื™ื ืœื ืชื•ื ื™ื ืืœื”,
13:33
but the bottom line is that the historical record is being digitized.
327
813260
3000
ืื‘ืœ ื”ืฉื•ืจื” ื”ืชื—ืชื•ื ื” ื”ื™ื ืฉืจืฉื•ืžื•ืช ื”ื™ืกื˜ื•ืจื™ื•ืช ืขื•ื‘ืจื•ืช ื“ื™ื’ื™ื˜ืœื™ื–ืฆื™ื”.
13:36
Google has started to digitize 15 million books.
328
816260
2000
ื’ื•ื’ืœ ื”ื—ืœื” ื‘ื“ื™ื’ื™ื˜ืœื™ื–ืฆื™ื” ืฉืœ 15 ืžื™ืœื™ื•ืŸ ืกืคืจื™ื.
13:38
That's 12 percent of all the books that have ever been published.
329
818260
2000
ื–ื” 12 ืื—ื•ื– ืžื›ืœ ื”ืกืคืจื™ื ืฉื™ืฆืื• ืื™-ืคืขื ืœืื•ืจ.
13:40
It's a sizable chunk of human culture.
330
820260
3000
ื–ื•ื”ื™ ืคื™ืกื” ื’ื“ื•ืœื” ืœืžื“ื™ื™ ืฉืœ ื”ืชืจื‘ื•ืช ื”ืื ื•ืฉื™ืช.
13:43
There's much more in culture: there's manuscripts, there newspapers,
331
823260
3000
ื™ืฉ ื‘ื ื•ืกืฃ ืขื•ื“ ื”ืจื‘ื” ื‘ืชืจื‘ื•ืช: ื™ืฉื ื ื›ืชื‘ื™-ื™ื“, ื™ืฉื ื ืขื™ืชื•ื ื™ื,
13:46
there's things that are not text, like art and paintings.
332
826260
2000
ื™ืฉื ื ื“ื‘ืจื™ื ืฉืื™ื ื ื˜ืงืกื˜ื™ื, ื›ืžื• ืืžื ื•ืช ื•ืฆื™ื•ืจื™ื.
13:48
These all happen to be on our computers,
333
828260
2000
ื›ืœ ื–ื” ืืžื•ืจ ืœื”ื™ื•ืช ื‘ืžื—ืฉื‘ื™ื ืฉืœื ื•,
13:50
on computers across the world.
334
830260
2000
ื‘ืžื—ืฉื‘ื™ื ื‘ื›ืœ ื”ืขื•ืœื.
13:52
And when that happens, that will transform the way we have
335
832260
3000
ื•ื›ืืฉืจ ื–ื” ื™ืงืจื”, ื™ื—ื•ืœ ืฉื™ื ื•ื™ ื‘ืื•ืคืŸ ื‘ื• ืื ื• ืžื‘ื™ื ื™ื ืืช ืขื‘ืจื ื•,
13:55
to understand our past, our present and human culture.
336
835260
2000
ืืช ื”ื”ื•ื•ื” ืฉืœื ื• ื•ืืช ื”ืชืจื‘ื•ืช ื”ืื ื•ืฉื™ืช.
13:57
Thank you very much.
337
837260
2000
ืชื•ื“ื” ืจื‘ื” ืœื›ื.
13:59
(Applause)
338
839260
3000
(ืžื—ื™ืื•ืช ื›ืคื™ื™ื)
ืขืœ ืืชืจ ื–ื”

ืืชืจ ื–ื” ื™ืฆื™ื’ ื‘ืคื ื™ื›ื ืกืจื˜ื•ื ื™ YouTube ื”ืžื•ืขื™ืœื™ื ืœืœื™ืžื•ื“ ืื ื’ืœื™ืช. ืชื•ื›ืœื• ืœืจืื•ืช ืฉื™ืขื•ืจื™ ืื ื’ืœื™ืช ื”ืžื•ืขื‘ืจื™ื ืขืœ ื™ื“ื™ ืžื•ืจื™ื ืžื”ืฉื•ืจื” ื”ืจืืฉื•ื ื” ืžืจื—ื‘ื™ ื”ืขื•ืœื. ืœื—ืฅ ืคืขืžื™ื™ื ืขืœ ื”ื›ืชื•ื‘ื™ื•ืช ื‘ืื ื’ืœื™ืช ื”ืžื•ืฆื’ื•ืช ื‘ื›ืœ ื“ืฃ ื•ื™ื“ืื• ื›ื“ื™ ืœื”ืคืขื™ืœ ืืช ื”ืกืจื˜ื•ืŸ ืžืฉื. ื”ื›ืชื•ื‘ื™ื•ืช ื’ื•ืœืœื•ืช ื‘ืกื ื›ืจื•ืŸ ืขื ื”ืคืขืœืช ื”ื•ื•ื™ื“ืื•. ืื ื™ืฉ ืœืš ื”ืขืจื•ืช ืื• ื‘ืงืฉื•ืช, ืื ื ืฆื•ืจ ืื™ืชื ื• ืงืฉืจ ื‘ืืžืฆืขื•ืช ื˜ื•ืคืก ื™ืฆื™ืจืช ืงืฉืจ ื–ื”.

https://forms.gle/WvT1wiN1qDtmnspy7