Can a computer write poetry? | Oscar Schwartz

89,982 views ใƒป 2016-02-10

TED


ืื ื ืœื—ืฅ ืคืขืžื™ื™ื ืขืœ ื”ื›ืชื•ื‘ื™ื•ืช ื‘ืื ื’ืœื™ืช ืœืžื˜ื” ื›ื“ื™ ืœื”ืคืขื™ืœ ืืช ื”ืกืจื˜ื•ืŸ.

ืžืชืจื’ื: Liblib Fib ืžื‘ืงืจ: Ido Dekkers
00:12
I have a question.
0
12881
1230
ื™ืฉ ืœื™ ืฉืืœื”
00:15
Can a computer write poetry?
1
15422
1943
ื”ืื ืžื—ืฉื‘ ื™ื›ื•ืœ ืœื›ืชื•ื‘ ืฉื™ืจื”?
00:18
This is a provocative question.
2
18959
2077
ื–ื• ืฉืืœื” ืคืจื•ื‘ื•ืงื˜ื™ื‘ื™ืช
00:21
You think about it for a minute,
3
21715
1718
ืชื—ืฉื‘ื• ืขืœ ื–ื” ืœืจื’ืข
00:23
and you suddenly have a bunch of other questions like:
4
23457
2590
ื•ืคืชืื ืฆืฆื•ืช ืœื›ื ืขื•ื“ ื›ืžื” ืฉืืœื•ืช, ื›ืžื•:
00:26
What is a computer?
5
26769
1381
ืžื”ื• ืžื—ืฉื‘?
00:28
What is poetry?
6
28710
1575
ืžื”ื™ ืฉื™ืจื”?
00:30
What is creativity?
7
30707
1689
ืžื” ื–ื• ื™ืฆื™ืจืชื™ื•ืช?
00:33
But these are questions
8
33650
1172
ืื‘ืœ ืืœื• ื”ืŸ ืฉืืœื•ืช
00:34
that people spend their entire lifetime trying to answer,
9
34846
3070
ืฉืื ืฉื™ื ืžื‘ืœื™ื ื—ื™ื™ื ืฉืœืžื™ื ื‘ื ืกื™ื•ืŸ ืœืขื ื•ืช ืขืœื™ื”ืŸ
00:37
not in a single TED Talk.
10
37940
2224
ื•ืœื ื‘ื”ืจืฆืืช TED ืื—ืช
00:40
So we're going to have to try a different approach.
11
40188
2445
ืื– ื ืืœืฅ ืœื”ืฉืชืžืฉ ื‘ื’ื™ืฉื” ืฉื•ื ื”
00:42
So up here, we have two poems.
12
42657
2143
ืœืคื ื™ื ื• ืฉื ื™ ืฉื™ืจื™ื
00:45
One of them is written by a human,
13
45839
2276
ืื—ื“ ืžื”ื ื ื›ืชื‘ ืขืดื™ ืื“ื
00:48
and the other one's written by a computer.
14
48139
2102
ื•ื”ืฉื ื™ ื ื›ืชื‘ ืขืดื™ ืžื—ืฉื‘
00:50
I'm going to ask you to tell me which one's which.
15
50754
2410
ืื ื™ ื”ื•ืœืš ืœื‘ืงืฉ ืžื›ื ืœื”ื’ื™ื“ ืœื™ ืžื™ื”ื• ืžื™
00:53
Have a go:
16
53858
1156
ื‘ื•ืื• ื ื ืกื”:
00:55
Poem 1: Little Fly / Thy summer's play, / My thoughtless hand / Has brush'd away.
17
55038
4056
ืฉื™ืจ 1: ื–ื‘ื•ื‘ ืงื˜ืŸ/ ืœืš ืžื—ื–ื”ื• ืฉืœ ื”ืงื™ืฅ/ ื™ื“ื™ ื—ืกืจืช ื”ืžื—ืฉื‘ื”/ ืกื•ืœืงื” ืžืฉื
00:59
Am I not / A fly like thee? / Or art not thou / A man like me?
18
59118
3394
ื”ืื ืื ื™ ืœื/ ื–ื‘ื•ื‘ ื›ืžื•ืชืš/ ืื• ืื•ืžื ื•ืช ืœื ืืชื”/ ืื“ื ื›ืžื•ืชื™?
01:02
Poem 2: We can feel / Activist through your life's / morning /
19
62536
3299
ืฉื™ืจ 2: ืื ื• ื™ื›ื•ืœื™ื ืœื”ืจื’ื™ืฉ/ ืคืขื™ืœื™ื ื‘ืขืฉื™ื™ืช ื—ื™ื™ื ื•/ ืฉื—ืจ
01:05
Pauses to see, pope I hate the / Non all the night to start a / great otherwise (...)
20
65859
4247
ืขื•ืฆืจื™ื ืœืจืื•ืช, ืืคื™ืคื™ื•ืจ ืื ื™ ืฉื•ื ื ืืช/ ืœื ื›ืœ ื”ืœื™ืœื” ืœื”ืชื—ื™ืœ/ ื ื”ื“ืจ ืื—ืจืช (...)
ืื•ืงื™, ื ื’ืžืจ ื”ื–ืžืŸ
01:10
Alright, time's up.
21
70130
1359
01:11
Hands up if you think Poem 1 was written by a human.
22
71513
4096
ื™ื“ื™ื™ื ืœืžืขืœื” ืื ืืชื ื—ื•ืฉื‘ื™ื ืฉืฉื™ืจ 1 ื ื›ืชื‘ ื‘ื™ื“ื™ ืื“ื
01:17
OK, most of you.
23
77547
1490
ื˜ื•ื‘, ืจื•ื‘ื›ื
01:19
Hands up if you think Poem 2 was written by a human.
24
79061
3023
ื™ื“ื™ื™ื ืœืžืขืœื” ืื ืืชื ื—ื•ืฉื‘ื™ื ืฉืฉื™ืจ 2 ื ื›ืชื‘ ื‘ื™ื“ื™ ืื“ื
01:23
Very brave of you,
25
83172
1190
ืžืื“ ืืžื™ืฅ ืžืฆื™ื“ืš
01:24
because the first one was written by the human poet William Blake.
26
84855
4285
ื‘ื’ืœืœ ืฉื”ืฉื™ืจ ื”ืจืืฉื•ืŸ ื ื›ืชื‘ ื‘ื™ื“ื™ ืื“ื, ื”ืžืฉื•ืจืจ ื•ื™ืœื™ืื ื‘ืœื™ื™ืง
01:29
The second one was written by an algorithm
27
89784
2949
ื”ืฉื ื™ ื ื›ืชื‘ ืขืดื™ ืืœื’ื•ืจื™ืชื
01:32
that took all the language from my Facebook feed on one day
28
92757
3692
ืฉืœืงื— ืืช ื›ืœ ื”ืฉืคื” ืžื“ืฃ ื”ืคื™ื™ืกื‘ื•ืง ืฉืœื™ ื‘ื™ื•ื ืื—ื“
01:36
and then regenerated it algorithmically,
29
96473
2763
ื•ืื– ื™ืฆืจ ืืช ื”ืฉื™ืจ ื‘ืืžืฆืขื•ืช ืืœื’ื•ืจื™ืชื
01:39
according to methods that I'll describe a little bit later on.
30
99260
3590
ื‘ืฉื™ื˜ื•ืช ืฉืืชืืจ ืขื•ื“ ืžืขื˜ ื‘ื”ืžืฉืš.
01:43
So let's try another test.
31
103218
2404
ื‘ื•ืื• ื ื ืกื” ืขื•ื“ ืžื‘ื—ืŸ
01:46
Again, you haven't got ages to read this,
32
106398
2093
ืฉื•ื‘ ืื™ืŸ ืœื›ื ื”ืžื•ืŸ ื–ืžืŸ ืœืงืจื™ืื”
01:48
so just trust your gut.
33
108515
1612
ืื– ืชืกืžื›ื• ืขืœ ืชื—ื•ืฉืช ื”ื‘ื˜ืŸ ืฉืœื›ื
01:50
Poem 1: A lion roars and a dog barks. It is interesting / and fascinating
34
110151
4045
ืฉื™ืจ 1: ื”ืืจื™ื” ืฉื•ืื’ ื•ื”ื›ืœื‘ ื ื•ื‘ื—. ื–ื” ืžืขื ื™ื™ืŸ/ ื•ืžืจืชืง
01:54
that a bird will fly and not / roar or bark. Enthralling stories about animals
35
114220
4303
ืฉืฆื™ืคื•ืจ ืขืคื” ื•ืœื/ ืฉืื’ื” ืื• ื ื‘ื™ื—ื” ืกื™ืคื•ืจื™ื ืžืจืชืงื™ื ืขืœ ื‘ืขืœื™ ื”ื—ื™ื™ื
01:58
are in my dreams and I will sing them all if I / am not exhausted or weary.
36
118547
4060
ื ืžืฆืื™ื ื‘ื—ืœื•ืžื•ืชื™ ื•ืืฉื™ืจ ืืช ื›ื•ืœื ืื ืื ื™/ ืœื ืขื™ื™ืฃ ืื• ืžื•ืชืฉ.
02:02
Poem 2: Oh! kangaroos, sequins, chocolate sodas! / You are really beautiful!
37
122631
3985
ืฉื™ืจ 2: ื”ื•! ืงื ื’ื•ืจื•, ืคืื™ื™ื˜ื™ื, ืฉื•ืงื•ืœื“ ืžืฉืงืื•ืช ืžื•ื’ื–ื™ื! / ืืช ืžืžืฉ ื™ืคื”!
02:06
Pearls, / harmonicas, jujubes, aspirins! All / the stuff they've always talked about (...)
38
126640
4358
ืคื ื™ื ื™ื, / ืžืคื•ื—ื™ื•ืช, ืกื•ื›ืจื™ื•ืช ื’ื•ืžื™, ืืกืคื™ืจื™ืŸ! ื”ื›ืœ / ื”ื“ื‘ืจื™ื ืขืœื™ื”ื ืชืžื™ื“ ื™ื“ื‘ืจื• (...)
ืื•ืงื™, ื”ื–ืžืŸ ืชื.
02:11
Alright, time's up.
39
131022
1158
02:12
So if you think the first poem was written by a human,
40
132204
3137
ืื– ืื ืืชื ื—ื•ืฉื‘ื™ื ืฉื”ืฉื™ืจ ื”ืจืืฉื•ืŸ ื ื›ืชื‘ ืขืดื™ ืื“ื
02:15
put your hand up.
41
135365
1215
ื”ืจื™ืžื• ื™ื“ื™ื™ื
02:17
OK.
42
137687
1154
OK
02:18
And if you think the second poem was written by a human,
43
138865
2675
ื•ืื ืืชื ื—ื•ืฉื‘ื™ื ืฉื”ืฉื™ืจ ื”ืฉื ื™ ื ื›ืชื‘ ืขืดื™ ืื“ื
02:21
put your hand up.
44
141564
1155
ื”ืจื™ืžื• ื™ื“ื™ื™ื
02:23
We have, more or less, a 50/50 split here.
45
143779
3810
ืืชื ืžืชื—ืœืงื™ื ื‘ืขืจืš ื—ืฆื™ ื—ืฆื™
02:28
It was much harder.
46
148157
1436
ื”ื™ื” ื”ืจื‘ื” ื™ื•ืชืจ ืงืฉื”
02:29
The answer is,
47
149617
1712
ื”ืชืฉื•ื‘ื” ื”ื™ื
02:31
the first poem was generated by an algorithm called Racter,
48
151353
3483
ืฉื™ืจ 1 ื ื•ืฆืจ ืขืดื™ ืืœื’ื•ืจื™ืชื ื‘ืฉื ืจืงื˜ืจ
02:34
that was created back in the 1970s,
49
154860
3002
ืฉื ื•ืฆืจ ื‘ืฉื ื•ืช ื” 70
02:37
and the second poem was written by a guy called Frank O'Hara,
50
157886
3189
ื•ื”ืฉื™ืจ ื”ืฉื ื™ ื ื›ืชื‘ ืขืดื™ ื‘ื—ื•ืจ ื‘ืฉื ืคืจื ืง ืื•ื”ืจื”
02:41
who happens to be one of my favorite human poets.
51
161099
2668
ืฉื‘ืžืงืจื” ื”ื•ื ืื—ื“ ืžื”ืžืฉื•ืจืจื™ื ื”ืื”ื•ื‘ื™ื ืขืœื™.
02:44
(Laughter)
52
164631
3058
(ืฆื—ื•ืง)
02:48
So what we've just done now is a Turing test for poetry.
53
168046
3228
ืื– ืžื” ืฉืขืฉื™ื ื• ื›ืจื’ืข ื–ื” ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’ ืœืฉื™ืจื”.
02:52
The Turing test was first proposed by this guy, Alan Turing, in 1950,
54
172018
4547
ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’ ื”ื•ืฆืข ืœืจืืฉื•ื ื” ืขืดื™ ื”ื‘ื—ื•ืจ ื”ื–ื”, ืืœืŸ ื˜ื™ื•ืจื™ื ื’ ื‘ 1950
02:56
in order to answer the question,
55
176589
1564
ืขืœ ืžื ืช ืœืขื ื•ืช ืขืœ ื”ืฉืืœื”
02:58
can computers think?
56
178177
1637
ื”ืื ืžื—ืฉื‘ื™ื ื™ื›ื•ืœื™ื ืœื—ืฉื•ื‘?
03:00
Alan Turing believed that if a computer was able
57
180245
2770
ืืœืŸ ื˜ื™ื•ืจื™ื ื’ ื”ืืžื™ืŸ ืฉืื ืžื—ืฉื‘ ืžืกื•ื’ืœ
03:03
to have a to have a text-based conversation with a human,
58
183039
3078
ืœื ื”ืœ ืฉื™ื—ื” ืžื‘ื•ืกืกืช ื˜ืงืกื˜ ืขื ืื“ื
03:06
with such proficiency such that the human couldn't tell
59
186141
2770
ื‘ืžื™ื•ืžื ื•ืช ื›ื–ื• ืฉื”ืื“ื ืœื ื™ื‘ื—ื™ืŸ
03:08
whether they are talking to a computer or a human,
60
188935
2966
ื”ืื ื”ื•ื ืžื“ื‘ืจ ืขื ืžื—ืฉื‘ ืื• ืžื›ื•ื ื”
03:11
then the computer can be said to have intelligence.
61
191925
2856
ืื– ืืคืฉืจ ืœื•ืžืจ ืฉืœืžื—ืฉื‘ ื™ืฉ ืื™ื ื˜ืœื™ื’ื ืฆื™ื”
03:15
So in 2013, my friend Benjamin Laird and I,
62
195270
3295
ื‘ืฉื ืช 2013, ื—ื‘ืจื™ ื‘ื™ื ื™ืžื™ืŸ ืœื™ืจื“ ื•ืื ื™
03:18
we created a Turing test for poetry online.
63
198589
2988
ื™ืฆืจื ื• ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’ ืœืฉื™ืจื” ืžืงื•ื•ื ืช
03:21
It's called bot or not,
64
201601
1277
ื–ื” ื ืงืจื ืณื‘ื•ื˜ ืื• ืœืืณ
03:22
and you can go and play it for yourselves.
65
202902
2044
ื•ืืชื ื™ื›ื•ืœื™ื ืœื”ืฉืžื™ืข ืืช ื–ื” ืœืขืฆืžื›ื
03:24
But basically, it's the game we just played.
66
204970
2251
ืื‘ืœ ืœืžืขืฉื” ื–ื” ื”ืžืฉื—ืง ืฉื›ืจื’ืข ืฉื™ื—ืงื ื•
03:27
You're presented with a poem,
67
207245
1528
ืžืฆื™ื’ื™ื ืœื›ื ืฉื™ืจ
03:28
you don't know whether it was written by a human or a computer
68
208797
3028
ืืชื ืœื ื™ื•ื“ืขื™ื ืื ื”ื•ื ื ื›ืชื‘ ื‘ื™ื“ื™ ืื“ื ืื• ืžื—ืฉื‘
03:31
and you have to guess.
69
211849
1166
ื•ืืชื ืชื“ืจืฉื• ืœื ื—ืฉ
03:33
So thousands and thousands of people have taken this test online,
70
213039
3191
ื•ื‘ื›ืŸ ืืœืคื™ ืืœืคื™ื ืฉืœ ืื ืฉื™ื ื‘ื™ืฆืขื• ืืช ื”ืžื‘ื—ืŸ ื‘ืื•ืคืŸ ืžืงื•ื•ื ืŸ
03:36
so we have results.
71
216254
1449
ื•ื™ืฉ ืœื ื• ืชื•ืฆืื•ืช
03:37
And what are the results?
72
217727
1428
ื•ืžื” ื”ืชื•ืฆืื•ืช?
03:39
Well, Turing said that if a computer could fool a human
73
219704
2879
ื•ื‘ื›ืŸ, ื˜ื™ื•ืจื™ื ื’ ืืžืจ ืฉืื ืžื—ืฉื‘ ื™ื›ื•ืœ ืœืขื‘ื•ื“ ืขืœ ืื ืฉื™ื
03:42
30 percent of the time that it was a human,
74
222607
3019
30% ืžื”ื–ืžืŸ ืฉื”ื•ื ืื“ื
03:45
then it passes the Turing test for intelligence.
75
225650
2397
ืื– ื”ื•ื ืขื•ื‘ืจ ืืช ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’ ืœืื™ื ื˜ืœื™ื’ื ืฆื™ื”
03:48
We have poems on the bot or not database
76
228625
2438
ื™ืฉ ืœื ื• ืฉื™ืจื™ื ืขืœ ื‘ืกื™ืก ื”ื ืชื•ื ื™ื ืฉืœ ืณื‘ื•ื˜ ืื• ืœืืณ
03:51
that have fooled 65 percent of human readers into thinking
77
231087
2979
ืฉื”ืฆืœื™ื—ื• ืœืขื‘ื•ื“ ืขืœ 65% ืžื”ืื ืฉื™ื ืฉืงืจืื• ื•ื—ืฉื‘ื•
03:54
it was written by a human.
78
234090
1395
ืฉื–ื” ื ื›ืชื‘ ื‘ื™ื“ื™ ืื“ื
03:55
So, I think we have an answer to our question.
79
235959
2817
ืื–, ืื ื™ ื—ื•ืฉื‘ ืฉื™ืฉ ืœื ื• ืชืฉื•ื‘ื” ืœืฉืืœื” ืฉืœื ื•
03:59
According to the logic of the Turing test,
80
239546
2348
ืœืคื™ ื”ื”ื™ื’ื™ื•ืŸ ืฉืœ ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’,
04:01
can a computer write poetry?
81
241918
1928
ื”ืื ืžื—ืฉื‘ ืžืกื•ื’ืœ ืœื›ืชื•ื‘ ืฉื™ืจื”?
04:03
Well, yes, absolutely it can.
82
243870
2351
ื•ื‘ื›ืŸ ื›ืŸ, ื”ื•ื ืœื’ืžืจื™ ื™ื›ื•ืœ.
04:07
But if you're feeling a little bit uncomfortable
83
247782
2346
ืื‘ืœ ืื ืืชื ืžืจื’ื™ืฉื™ื ืงืฆืช ื—ื•ืกืจ ื ื•ื—ื•ืช
04:10
with this answer, that's OK.
84
250152
1927
ืขื ื”ืชืฉื•ื‘ื” ื”ื–ื•, ื–ื” ื‘ืกื“ืจ
04:12
If you're having a bunch of gut reactions to it,
85
252103
2316
ืื ื™ืฉ ืœื›ื ืื•ืกืฃ ืฉืœ ืชื—ื•ืฉื•ืช ื‘ื˜ืŸ ืœื’ื‘ื™ ื–ื”
04:14
that's also OK because this isn't the end of the story.
86
254443
3205
ื–ื” ื’ื ื‘ืกื“ืจ ื‘ื’ืœืœ ืฉื–ื” ืœื ืกื•ืฃ ื”ืกื™ืคื•ืจ
04:18
Let's play our third and final test.
87
258594
2324
ื‘ื•ืื• ื ืฉื—ืง ืืช ื”ืžืฉื—ืง ื”ืฉืœื™ืฉื™ ื•ื”ืื—ืจื•ืŸ
04:22
Again, you're going to have to read
88
262000
1750
ืฉื•ื‘, ืชืฆื˜ืจื›ื• ืœืงืจื
04:23
and tell me which you think is human.
89
263774
1909
ื•ืœื”ื’ื™ื“ ืœื™ ืื™ื–ื” ืืชื ื—ื•ืฉื‘ื™ื ื”ื•ื ื”ืื“ื
04:25
Poem 1: Red flags the reason for pretty flags. / And ribbons.
90
265707
3718
ืฉื™ืจ 1: ื“ื’ืœื™ื ืื“ื•ืžื™ื ื”ืกื™ื‘ื” ืœื“ื’ืœื™ื ื™ืคื™ื/ ื•ืกืจื˜ื™ื
04:29
Ribbons of flags / And wearing material / Reasons for wearing material. (...)
91
269449
4321
ืฉืจืฉืจืช ื“ื’ืœื™ื/ ื•ื—ื•ืžืจ ืžืชื›ืœื”/ ืกื™ื‘ื•ืช ืœื—ื•ืžืจื™ื ืžืชื›ืœื™ื
04:33
Poem 2: A wounded deer leaps highest, / I've heard the daffodil
92
273794
3918
ืฉื™ืจ 2: ืฆื‘ื™ ืคืฆื•ืข ืžื–ื ืง ื”ื›ื™ ื’ื‘ื•ื”ื”/ ืฉืžืขืชื™ ืืช ื”ื ืจืงื™ืก
04:37
I've heard the flag to-day / I've heard the hunter tell; /
93
277736
3446
ืฉืžืขืชื™ ืืช ื“ื’ืœ ื”ื™ื•ื/ ืฉืžืขืชื™ ืืช ื”ืฆื™ื™ื“ ืื•ืžืจ/
04:41
'Tis but the ecstasy of death, / And then the brake is almost done (...)
94
281206
3702
ืจืง ื–ื• ืืงืกื˜ื–ืช ื”ืžื•ื•ืช/ ื•ืื– ื”ืขืฆื™ืจื” ื›ืžืขื˜ ื”ื•ืฉืœืžื”.
04:44
OK, time is up.
95
284932
1599
ืื•ืงื™ ื ื’ืžืจ ื”ื–ืžืŸ.
04:46
So hands up if you think Poem 1 was written by a human.
96
286555
3837
ืื– ื™ื“ื™ื™ื ืœืžืขืœื” ืื ืืชื ื—ื•ืฉื‘ื™ื ืฉืฉื™ืจ 1 ื ื›ืชื‘ ื‘ื™ื“ื™ ืื“ื
04:51
Hands up if you think Poem 2 was written by a human.
97
291973
3038
ื™ื“ื™ื™ื ืœืžืขืœื” ืื ืืชื ื—ื•ืฉื‘ื™ื ืฉืฉื™ืจ 2 ื ื›ืชื‘ ื‘ื™ื“ื™ ืื“ื
04:55
Whoa, that's a lot more people.
98
295035
2331
ื•ื•ืื•, ื–ื” ื”ืจื‘ื” ื™ื•ืชืจ ืื ืฉื™ื
04:58
So you'd be surprised to find that Poem 1
99
298327
2968
ืื– ืชืชืคืœืื• ืœื’ืœื•ืช ืฉืฉื™ืจ 1
05:01
was written by the very human poet Gertrude Stein.
100
301319
3993
ื ื›ืชื‘ ืขืดื™ ื”ืžืฉื•ืจืจืช ื”ืžืื•ื“ ืื ื•ืฉื™ืช ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ.
05:06
And Poem 2 was generated by an algorithm called RKCP.
101
306100
5038
ื•ื”ืฉื ื™ ื—ื•ืœืœ ืขืดื™ ืืœื’ื•ืจื™ืชื ื‘ืฉื RKCP
05:11
Now before we go on, let me describe very quickly and simply,
102
311162
3319
ืขื›ืฉื™ื• ืœืคื ื™ ืฉืื ื• ืžืžืฉื™ื›ื™ื, ืชื ื• ืœื™ ืœืชืืจ ืœื›ื ืžื”ืจ ื•ื‘ืคืฉื˜ื•ืช
05:14
how RKCP works.
103
314505
1781
ืื™ืš RKCP ืขื•ื‘ื“
05:16
So RKCP is an algorithm designed by Ray Kurzweil,
104
316873
3850
ืื– RKCP ื–ื” ืืœื’ื•ืจื™ืชื ืฉื›ืชื‘ ืจื™ื™ ืงืจื•ืฆืœ
05:20
who's a director of engineering at Google
105
320747
2222
ืฉื”ื•ื ืžื ื”ืœ ืคื™ืชื•ื— ื‘ื’ื•ื’ืœ
05:22
and a firm believer in artificial intelligence.
106
322993
2360
ื•ืžืืžื™ืŸ ื’ื“ื•ืœ ื‘ื‘ื™ื ื” ืžืœืื›ื•ืชื™ืช
05:25
So, you give RKCP a source text,
107
325822
3991
ืื– ืžืกืคืงื™ื ืœ RKCP ื˜ืงืกื˜ ืžืงื•ืจ
05:29
it analyzes the source text in order to find out how it uses language,
108
329837
4469
ื”ื•ื ื‘ื•ื—ืŸ ืืช ื˜ืงืกื˜ ื”ืžืงื•ืจ ืขืœ ืžื ืช ืœืžืฆื•ื ืื™ืš ื”ื•ื ืžืฉืชืžืฉ ื‘ืฉืคื”
05:34
and then it regenerates language
109
334330
1948
ื•ืื– ื”ื•ื ืžื—ื•ืœืœ ืžื—ื“ืฉ ืฉืคื”
05:36
that emulates that first text.
110
336302
2528
ืฉืžื“ืžื” ืืช ื”ื˜ืงืกื˜ ื”ืจืืฉื•ืŸ
05:38
So in the poem we just saw before,
111
338854
2113
ืื– ื‘ืฉื™ืจ ืฉื”ืจื’ืข ืจืื™ื ื•
05:40
Poem 2, the one that you all thought was human,
112
340991
2625
ืฉื™ืจ 2, ื–ื” ืฉื›ื•ืœื›ื ื—ืฉื‘ืชื ืฉืื“ื ื›ืชื‘
05:43
it was fed a bunch of poems
113
343640
1550
ื”ื•ื–ื ื• ืœื• ืžืกืคืจ ืฉื™ืจื™ื
05:45
by a poet called Emily Dickinson
114
345214
2035
ืฉืœ ืžืฉื•ืจืจืช ื‘ืฉื ืืžื™ืœื™ ื“ื™ืงื ืกื•ืŸ
05:47
it looked at the way she used language,
115
347273
2189
ื”ื•ื ื‘ื—ืŸ ืืช ื”ืื•ืคืŸ ื‘ื” ื”ืฉืชืžืฉื” ื‘ืฉืคื”
05:49
learned the model,
116
349486
1165
ืœืžื“ ืืช ื”ืชื‘ื ื™ืช
05:50
and then it regenerated a model according to that same structure.
117
350675
4258
ื•ืื– ื—ื•ืœืœ ืžื—ื“ืฉ ืชื‘ื ื™ืช ืœืคื™ ืื•ืชื• ืžื‘ื ื”
05:56
But the important thing to know about RKCP
118
356732
2178
ืื‘ืœ ื”ื“ื‘ืจ ืฉื—ืฉื•ื‘ ืœื“ืขืช ืื•ื“ื•ืช RKCP
05:58
is that it doesn't know the meaning of the words it's using.
119
358934
2838
ืฉื”ื•ื ืœื ืžื‘ื™ืŸ ืืช ืžืฉืžืขื•ืช ื”ืžื™ืœื™ื ื‘ื”ืŸ ื”ื•ื ื‘ื•ื—ืจ ืœื”ืฉืชืžืฉ
06:02
The language is just raw material,
120
362359
2276
ื”ืฉืคื” ื”ื™ื ื” ืจืง ื—ื•ืžืจ ื’ืœื
06:04
it could be Chinese, it could be in Swedish,
121
364659
2160
ื–ื” ื™ื›ื•ืœ ืœื”ื™ื•ืช ืกื™ื ื™ืช, ื–ื” ื™ื›ื•ืœ ืœื”ื™ื•ืช ืฉื•ื•ื“ื™ืช
06:06
it could be the collected language from your Facebook feed for one day.
122
366843
4179
ื–ื” ื™ื›ื•ืœ ืœื”ื™ื•ืช ื”ืฉืคื” ืฉื ืืกืคื” ืžื“ืฃ ื”ืคื™ื™ืกื‘ื•ืง ืฉืœืš
06:11
It's just raw material.
123
371046
1652
ื–ื” ืจืง ื—ื•ืžืจ ื’ืœื.
06:13
And nevertheless, it's able to create a poem
124
373380
2697
ื•ืขื“ื™ื™ืŸ ื–ื” ืžืกื•ื’ืœ ืœื™ื™ืฆืจ ืฉื™ืจ
06:16
that seems more human than Gertrude Stein's poem,
125
376101
3327
ืฉื ืจืื” ื™ื•ืชืจ ืื ื•ืฉื™ ืžื–ื” ืฉืœ ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ.
06:19
and Gertrude Stein is a human.
126
379452
2153
ื•ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ ื”ื™ื ืื“ื
06:22
So what we've done here is, more or less, a reverse Turing test.
127
382846
4072
ืื– ืžื” ืฉืขืฉื™ื ื• ื›ืืŸ ื–ื” ื”ื™ืคื•ืš ืฉืœ ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’.
06:27
So Gertrude Stein, who's a human, is able to write a poem
128
387940
5179
ืื– ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ, ืฉื”ื™ื ืื“ื, ื™ื›ื•ืœื” ืœื›ืชื•ื‘ ืฉื™ืจ
06:33
that fools a majority of human judges into thinking
129
393143
3738
ืฉื”ืคื™ืœ ืืช ืžืจื‘ื™ืช ื”ืฉื•ืคื˜ื™ื ื”ืื ื•ืฉื™ื™ื ืœื—ืฉื•ื‘
06:36
that it was written by a computer.
130
396905
1826
ืฉื ื›ืชื‘ ื‘ื™ื“ื™ ืžื—ืฉื‘
06:39
Therefore, according to the logic of the reverse Turing test,
131
399176
4141
ืœื›ืŸ ืœืคื™ ื”ื”ื’ื™ื•ืŸ ืฉืœ ื”ื™ืคื•ืš ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’
06:43
Gertrude Stein is a computer.
132
403341
1916
ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ ื”ื™ื ืžื—ืฉื‘
06:45
(Laughter)
133
405281
1462
(ืฆื—ื•ืง)
06:47
Feeling confused?
134
407358
1294
ืžืจื’ื™ืฉื™ื ืžื‘ื•ืœื‘ืœื™ื?
06:49
I think that's fair enough.
135
409193
1515
ืื ื™ ื—ื•ืฉื‘ ืฉื–ื” ื”ื•ื’ืŸ.
06:51
So far we've had humans that write like humans,
136
411546
4116
ืขื“ ื›ื” ื”ื™ื• ืœื ื• ืื ืฉื™ื ืฉื›ื•ืชื‘ื™ื ื›ืžื• ืื ืฉื™ื
06:55
we have computers that write like computers,
137
415686
3111
ื™ืฉ ืœื ื• ืžื—ืฉื‘ื™ื ืฉื›ื•ืชื‘ื™ื ื›ืžื• ืžื—ืฉื‘ื™ื
06:58
we have computers that write like humans,
138
418821
3055
ื™ืฉ ืœื ื• ืžื—ืฉื‘ื™ื ืฉื›ื•ืชื‘ื™ื ื›ืžื• ืื ืฉื™ื
07:01
but we also have, perhaps most confusingly,
139
421900
3632
ืื‘ืœ ื’ื ื™ืฉ ืœื ื•, ืื•ืœื™ ื”ืžื‘ืœื‘ืœ ื‘ื™ื•ืชืจ
07:05
humans that write like computers.
140
425556
2375
ืื ืฉื™ื ืฉื›ื•ืชื‘ื™ื ื›ืžื• ืžื—ืฉื‘ื™ื
07:08
So what do we take from all of this?
141
428938
1766
ืื– ืžื” ืื ื—ื ื• ืœื•ืงื—ื™ื ืžื›ืœ ื–ื”?
07:11
Do we take that William Blake is somehow more of a human
142
431611
3157
ื”ืื ืื ื• ืœื•ืงื—ื™ื ืฉื•ื™ืœื™ืื ื‘ืœื™ื™ืง ื”ื•ื ืื™ื›ืฉื”ื• ื™ื•ืชืจ ื‘ืŸ-ืื“ื
07:14
than Gertrude Stein?
143
434792
1249
ืžืืฉืจ ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ?
07:16
Or that Gertrude Stein is more of a computer than William Blake?
144
436065
3046
ืื• ืฉื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ ื”ื™ื ื™ื•ืชืจ ืžื—ืฉื‘ ืžื•ื™ืœื™ืื ื‘ืœื™ื™ืง?
07:19
(Laughter)
145
439135
1552
(ืฆื—ื•ืง)
07:20
These are questions I've been asking myself
146
440711
2323
ืืœื• ืฉืืœื•ืช ืฉืฉืืœืชื™ ืืช ืขืฆืžื™
07:23
for around two years now,
147
443058
1465
ื‘ืขืจืš ืฉื ืชื™ื™ื
07:24
and I don't have any answers.
148
444547
2309
ื•ืื™ืŸ ืœื™ ืชืฉื•ื‘ื•ืช
07:26
But what I do have are a bunch of insights
149
446880
2330
ืื‘ืœ ืžื” ืฉื™ืฉ ืœื™ ื”ืŸ ืžืกืคืจ ืชื•ื‘ื ื•ืช
07:29
about our relationship with technology.
150
449234
2534
ืื•ื“ื•ืช ื”ื™ื—ืกื™ื ืฉืœื ื• ืขื ื”ื˜ื›ื ื•ืœื•ื’ื™ื”
07:32
So my first insight is that, for some reason,
151
452999
3609
ืื– ื”ืชื•ื‘ื ื” ื”ืจืืฉื•ื ื” ืฉืœื™ ื”ื™ื ืฉืžืกื™ื‘ื” ื›ืœืฉื”ื™ื
07:36
we associate poetry with being human.
152
456632
3111
ืื ื• ืžืฉื™ื™ื›ื™ื ืฉื™ืจื” ืœื”ื™ื•ืชื ื• ื‘ื ื™ ืื ื•ืฉ
07:40
So that when we ask, "Can a computer write poetry?"
153
460197
3715
ื›ืš ืฉื›ืฉื ืฉืืœ ืดื”ืื ืžื—ืฉื‘ ื™ื›ื•ืœ ืœื›ืชื•ื‘ ืฉื™ืจื”?ืด
07:43
we're also asking,
154
463936
1193
ืื ื• ื’ื ืฉื•ืืœื™ื
07:45
"What does it mean to be human
155
465153
1798
ืžื” ื”ืžืฉืžืขื•ืช ืœื”ื™ื•ืช ื‘ืŸ-ืื“ื
07:46
and how do we put boundaries around this category?
156
466975
3172
ื•ืื™ืš ืื ื• ืžื’ื“ืจื™ื ื’ื‘ื•ืœื•ืช ืกื‘ื™ื‘ ื”ืงื˜ื’ื•ืจื™ื” ื”ื–ื•?
07:50
How do we say who or what can be part of this category?"
157
470171
3658
ืื™ืš ืื ื• ืงื•ื‘ืขื™ื ืžื™ ืื• ืžื” ืฉื™ื™ื›ื™ื ืืœื™ื”?
07:54
This is an essentially philosophical question, I believe,
158
474376
3351
ื–ื•ื”ื™ ื‘ืขืฆื ืฉืืœื” ืคื™ืœื•ืกื•ืคื™ืช, ืื ื™ ืžืืžื™ืŸ
07:57
and it can't be answered with a yes or no test,
159
477751
2229
ื•ืœื ื ื™ืชืŸ ืœืขื ื•ืช ืขืœื™ื” ื‘ืžื‘ื—ืŸ ื›ืŸ ืื• ืœื,
08:00
like the Turing test.
160
480004
1327
ื›ืžื• ืžื‘ื—ืŸ ื˜ื•ื™ืจื™ื ื’
08:01
I also believe that Alan Turing understood this,
161
481805
3045
ืื ื™ ื’ื ืžืืžื™ืŸ ืฉืืœืŸ ื˜ื™ื•ืจื™ื ื’ ื”ื‘ื™ืŸ ื–ืืช,
08:04
and that when he devised his test back in 1950,
162
484874
3305
ื•ื›ืฉื”ืžืฆื™ื ืืช ื”ืžื‘ื—ืŸ ื‘ืฉื ืช 1950
08:08
he was doing it as a philosophical provocation.
163
488203
2802
ื”ื•ื ืขืฉื” ื–ืืช ื›ืคืจื•ื‘ื•ืงืฆื™ื” ืคื™ืœื•ืกื•ืคื™ืช.
08:13
So my second insight is that, when we take the Turing test for poetry,
164
493124
5541
ืื– ื”ืชื•ื‘ื ื” ื”ืฉื ื™ื” ืฉืœื™ ื”ื™ื, ื›ืฉืื ื• ื‘ื•ื—ื ื™ื ืฉื™ืจื” ื‘ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’
08:18
we're not really testing the capacity of the computers
165
498689
3460
ืื ื• ืœื ื‘ืืžืช ื‘ื•ื—ื ื™ื ืืช ื”ื™ื›ื•ืœืช ืฉืœ ื”ืžื—ืฉื‘ื™ื
08:22
because poetry-generating algorithms,
166
502173
2893
ื›ื™ ืืœื’ื•ืจื™ืชืžื™ื ืžื ื™ื‘ื™ ืฉื™ืจื”,
08:25
they're pretty simple and have existed, more or less, since the 1950s.
167
505090
4563
ื”ื ื“ื™ ืคืฉื•ื˜ื™ื ื•ืงื™ื™ืžื™ื ืคื—ื•ืช ืื• ื™ื•ืชืจ ืžืื– ืฉื ื•ืช ื” 50.
08:31
What we are doing with the Turing test for poetry, rather,
168
511055
3118
ืžื” ืฉืื ื• ืขื•ืฉื™ื ืขื ืžื‘ื—ืŸ ื˜ื™ื•ืจื™ื ื’ ืœืฉื™ืจื”,
08:34
is collecting opinions about what constitutes humanness.
169
514197
4615
ื–ื” ืื™ืกื•ืฃ ื“ืขื•ืช ืขืœ ืžื” ืžื”ื•ื•ื” ืื ื•ืฉื™ื•ืช
08:40
So, what I've figured out,
170
520313
2729
ืื– ืžื” ืฉื’ื™ืœื™ืชื™,
08:43
we've seen this when earlier today,
171
523066
2972
ืจืื™ื ื• ืืช ื–ื” ื›ืืฉืจ ืขื•ื“ ืงื•ื“ื ื”ื™ื•ื
08:46
we say that William Blake is more of a human
172
526062
2478
ืืžืจื ื• ืฉื•ื™ืœื™ืื ื‘ืœื™ื™ืง ื”ื•ื ื™ื•ืชืจ ื‘ืŸ-ืื ื•ืฉ
08:48
than Gertrude Stein.
173
528564
1565
ืžืืฉืจ ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ,
08:50
Of course, this doesn't mean that William Blake
174
530153
2462
ื›ืžื•ื‘ืŸ, ื–ื” ืœื ืื•ืžืจ ืฉื•ื™ืœื™ืื ื‘ืœื™ื™ืง
08:52
was actually more human
175
532639
1828
ื”ื•ื ื‘ืืžืช ื™ื•ืชืจ ืื ื•ืฉื™
08:54
or that Gertrude Stein was more of a computer.
176
534491
2327
ืื• ืฉื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ ื”ื™ื ื™ื•ืชืจ ืžื—ืฉื‘
08:57
It simply means that the category of the human is unstable.
177
537533
4714
ื–ื” ืคืฉื•ื˜ ืื•ืžืจ ืฉื”ืงื˜ื’ื•ืจื™ื” ืฉืœ ื‘ืŸ-ืื“ื ืœื ื™ืฆื™ื‘ื”.
09:03
This has led me to understand
178
543450
2074
ื–ื” ื”ื•ื‘ื™ืœ ืื•ืชื™ ืœื”ื‘ื™ืŸ
09:05
that the human is not a cold, hard fact.
179
545548
2763
ืฉืื“ื ื–ื” ืœื ืขื•ื‘ื“ื” ืžื•ืฆืงื”
09:08
Rather, it is something that's constructed with our opinions
180
548832
3132
ื‘ืžืงื•ื ื–ื”, ื”ื•ื ืžืฉื”ื•ื ืฉืžื•ืจื›ื‘ ืžื“ืขื•ืช
09:11
and something that changes over time.
181
551988
2855
ื•ืžืฉื”ื• ืฉืžืฉืชื ื” ืขื ื”ื–ืžืŸ.
09:16
So my final insight is that the computer, more or less,
182
556671
4479
ืื– ื”ืชื•ื‘ื ื” ื”ืื—ืจื•ื ื” ืฉืœื™ ื”ื™ื ืฉืžื—ืฉื‘ื™ื ืคื—ื•ืช ืื• ื™ื•ืชืจ
09:21
works like a mirror that reflects any idea of a human
183
561174
4006
ืขื•ื‘ื“ื™ื ื›ืžื• ืžืจืื” ืฉืžืฉืงืคืช ื›ืœ ืจืขื™ื•ืŸ ืื ื•ืฉื™
09:25
that we show it.
184
565204
1375
ืฉืื ื• ืžืจืื™ื ืœื•.
09:26
We show it Emily Dickinson,
185
566958
1884
ื”ืจืื ื• ืœื• ืืช ืืžื™ืœื™ ื“ื™ืงืกื•ืŸ
09:28
it gives Emily Dickinson back to us.
186
568866
2321
ื•ื”ื•ื ื”ื—ื–ื™ืจ ืืช ืืžื™ืœื™ ื“ื™ืงืกื•ืŸ
09:31
We show it William Blake,
187
571768
1834
ื”ืจืื ื• ืœื• ืืช ื•ื™ืœื™ืื ื‘ืœื™ื™ืง
09:33
that's what it reflects back to us.
188
573626
2285
ื•ื–ื” ืžื” ืฉื”ืฉืชืงืฃ ืœื ื• ื—ื–ืจื”
09:35
We show it Gertrude Stein,
189
575935
1839
ื”ืจืื ื• ืœื• ืืช ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ,
09:37
what we get back is Gertrude Stein.
190
577798
2470
ื•ืžื” ืฉืงื™ื‘ืœื ื• ื—ื–ืจื” ื–ื” ืืช ื’ืจื˜ืจื•ื“ ืฉื˜ื™ื™ืŸ,
09:41
More than any other bit of technology,
191
581083
2368
ื™ื•ืชืจ ืžื›ืœ ื—ืœืงืช ื˜ื›ื ื•ืœื•ื’ื™ื”
09:43
the computer is a mirror that reflects any idea of the human we teach it.
192
583475
5165
ื”ืžื—ืฉื‘ ื”ื•ื ืžืจืื” ืฉืžืฉืงืคืช ื›ืœ ืจืขื™ื•ืŸ ืฉืœ ื”ืื“ื ืฉืื ื• ืžืœืžื“ื™ื ืื•ืชื•.
09:50
So I'm sure a lot of you have been hearing
193
590061
2287
ืื– ืื ื™ ื‘ื˜ื•ื— ืฉืจื‘ื™ื ืžื›ื ืฉืžืขื•
09:52
a lot about artificial intelligence recently.
194
592372
2862
ื”ืจื‘ื” ืขืœ ื‘ื™ื ื” ืžืœืื›ื•ืชื™ืช ืœืื—ืจื•ื ื”.
09:56
And much of the conversation is,
195
596694
2830
ื•ืจื•ื‘ ื”ืฉื™ื—ื” ื”ื™ื
10:00
can we build it?
196
600292
1189
ื”ืื ืื ื• ื™ื›ื•ืœื™ื ืœื‘ื ื•ืช ืืช ื–ื”?
10:02
Can we build an intelligent computer?
197
602383
3135
ื”ืื ืื ื• ื™ื›ื•ืœื™ื ืœื‘ื ื•ืช ืžื—ืฉื‘ ื—ื›ื?
10:05
Can we build a creative computer?
198
605542
2763
ื”ืื ืื ื• ื™ื›ื•ืœื™ื ืœื‘ื ื•ืช ืžื—ืฉื‘ ื™ืฆื™ืจืชื™?
10:08
What we seem to be asking over and over
199
608329
2113
ื ืจืื” ืฉืžื” ืฉืื ื• ืฉื•ืืœื™ื ืฉื•ื‘ ื•ืฉื•ื‘
10:10
is can we build a human-like computer?
200
610466
2724
ื–ื” ื”ืื ืื ื—ื ื• ื™ื›ื•ืœื™ื ืœื™ื™ืฆืจ ืžื—ืฉื‘ ื“ืžื•ื™ ืื“ื?
10:13
But what we've seen just now
201
613961
1556
ืื‘ืœ ืžื” ืฉืจืื™ื ื• ื›ืจื’ืข
10:15
is that the human is not a scientific fact,
202
615541
3088
ื–ื” ืฉื”ืื“ื ืื™ื ื• ืขื•ื‘ื“ื” ืžื“ืขื™ืช
10:18
that it's an ever-shifting, concatenating idea
203
618653
3530
ืฉื”ื•ื ื›ืœ ื”ื–ืžืŸ ืžืฉืชื ื” ื•ืžืฉืจืฉืจ ืจืขื™ื•ื ื•ืช
10:22
and one that changes over time.
204
622207
2531
ื•ืื—ื“ ืฉืžืฉืชื ื” ืœืื•ืจืš ื”ื–ืžืŸ
10:24
So that when we begin to grapple with the ideas
205
624762
3152
ืื– ื›ืฉืื ื—ื ื• ืžืชื—ื™ืœื™ื ืœื”ืชื—ื‘ื˜ ืขื ืจืขื™ื•ื ื•ืช
10:27
of artificial intelligence in the future,
206
627938
2386
ืฉืœ ื‘ื™ื ื” ืžืœืื›ื•ืชื™ืช ื‘ืขืชื™ื“
10:30
we shouldn't only be asking ourselves,
207
630348
1905
ืื ื—ื ื• ืœื ืจืง ืฆืจื™ื›ื™ื ืœืฉืื•ืœ ืืช ืขืฆืžื ื•,
10:32
"Can we build it?"
208
632277
1368
"ืื ื• ื™ื›ื•ืœื™ื ืœื‘ื ื•ืช ืืช ื–ื”?"
10:33
But we should also be asking ourselves,
209
633669
1894
ืื‘ืœ ืื ื• ื’ื ืฆืจื™ื›ื™ื ืœืฉืื•ืœ ืืช ืขืฆืžื ื•
10:35
"What idea of the human do we want to have reflected back to us?"
210
635587
3713
ืžื”ื• ื”ืจืขื™ื•ืŸ ื”ืื ื•ืฉื™ ืฉืื ื• ืจื•ืฆื™ื ืฉื™ืฉืชืงืฃ ืืœื™ื ื• ื—ื–ืจื”?
10:39
This is an essentially philosophical idea,
211
639820
2693
ื–ื”ื• ืœืžืขืฉื” ืจืขื™ื•ืŸ ืคื™ืœื•ืกื•ืคื™
10:42
and it's one that can't be answered with software alone,
212
642537
2997
ื•ื”ื•ื ืœื ื ื™ืชืŸ ืœืžืขื ื” ืจืง ื‘ืขื–ืจืช ืชื•ื›ื ื”
10:45
but I think requires a moment of species-wide, existential reflection.
213
645558
4977
ืืœื ืื ื™ ื—ื•ืฉื‘, ื“ื•ืจืฉ ืจื’ืข ืฉืœ ื”ืฉืชืงืคื•ืช ืงื™ื•ืžื™ืช ืฉืœ ื›ืœืœ ื”ืžื™ื ื™ื,
10:51
Thank you.
214
651040
1153
ืชื•ื“ื”
10:52
(Applause)
215
652217
2695
(ืžื—ื™ืื•ืช ื›ืคื™ื™ื)
ืขืœ ืืชืจ ื–ื”

ืืชืจ ื–ื” ื™ืฆื™ื’ ื‘ืคื ื™ื›ื ืกืจื˜ื•ื ื™ YouTube ื”ืžื•ืขื™ืœื™ื ืœืœื™ืžื•ื“ ืื ื’ืœื™ืช. ืชื•ื›ืœื• ืœืจืื•ืช ืฉื™ืขื•ืจื™ ืื ื’ืœื™ืช ื”ืžื•ืขื‘ืจื™ื ืขืœ ื™ื“ื™ ืžื•ืจื™ื ืžื”ืฉื•ืจื” ื”ืจืืฉื•ื ื” ืžืจื—ื‘ื™ ื”ืขื•ืœื. ืœื—ืฅ ืคืขืžื™ื™ื ืขืœ ื”ื›ืชื•ื‘ื™ื•ืช ื‘ืื ื’ืœื™ืช ื”ืžื•ืฆื’ื•ืช ื‘ื›ืœ ื“ืฃ ื•ื™ื“ืื• ื›ื“ื™ ืœื”ืคืขื™ืœ ืืช ื”ืกืจื˜ื•ืŸ ืžืฉื. ื”ื›ืชื•ื‘ื™ื•ืช ื’ื•ืœืœื•ืช ื‘ืกื ื›ืจื•ืŸ ืขื ื”ืคืขืœืช ื”ื•ื•ื™ื“ืื•. ืื ื™ืฉ ืœืš ื”ืขืจื•ืช ืื• ื‘ืงืฉื•ืช, ืื ื ืฆื•ืจ ืื™ืชื ื• ืงืฉืจ ื‘ืืžืฆืขื•ืช ื˜ื•ืคืก ื™ืฆื™ืจืช ืงืฉืจ ื–ื”.

https://forms.gle/WvT1wiN1qDtmnspy7