Ralph Langner: Cracking Stuxnet, a 21st-century cyber weapon

311,863 views ใƒป 2011-03-29

TED


ืื ื ืœื—ืฅ ืคืขืžื™ื™ื ืขืœ ื”ื›ืชื•ื‘ื™ื•ืช ื‘ืื ื’ืœื™ืช ืœืžื˜ื” ื›ื“ื™ ืœื”ืคืขื™ืœ ืืช ื”ืกืจื˜ื•ืŸ.

ืžืชืจื’ื: Roi Shpaner ืžื‘ืงืจ: Sigal Tifferet
00:15
The idea behind the Stuxnet computer worm
0
15260
3000
ื”ืจืขื™ื•ืŸ ืžืื—ื•ืจื™ ืชื•ืœืขืช ื”ืžื—ืฉื‘ ืกื˜ืืงืกื ื˜
00:18
is actually quite simple.
1
18260
2000
ื”ื•ื ื‘ืขืฆื ื“ื™ ืคืฉื•ื˜.
00:20
We don't want Iran to get the bomb.
2
20260
3000
ืื ื—ื ื• ืœื ืจื•ืฆื™ื ืฉืื™ืจืŸ ื™ื’ื™ืขื• ืœืคืฆืฆื”.
00:23
Their major asset for developing nuclear weapons
3
23260
3000
ื”ื ื›ืก ื”ืขื™ืงืจื™ ืฉืœื”ื ื‘ืคื™ืชื•ื— ื ืฉืง ื’ืจืขื™ื ื™
00:26
is the Natanz uranium enrichment facility.
4
26260
4000
ื”ื•ื ืžืคืขืœ ื”ืขืฉืจืช ื”ืื•ืจื ื™ื•ื ื‘ื ืชื ื–.
00:30
The gray boxes that you see,
5
30260
2000
ื”ืงื•ืคืกืื•ืช ื”ืืคื•ืจื•ืช ืฉืืชื ืจื•ืื™ื,
00:32
these are real-time control systems.
6
32260
3000
ื”ืŸ ืžืขืจื›ื•ืช ื‘ืงืจืช ื–ืžืŸ-ืืžืช.
00:35
Now if we manage to compromise these systems
7
35260
3000
ืื ืื ื—ื ื• ืžืฆืœื™ื—ื™ื ืœืคืจื•ืฅ ืœืžืขืจื›ื•ืช ื”ืืœื”
00:38
that control drive speeds and valves,
8
38260
3000
ืฉืฉื•ืœื˜ื•ืช ืขืœ ืžื”ื™ืจื•ืช ืจื›ื™ื‘ื™ื ื•ืขืœ ืฉืกืชื•ืžื™ื,
00:41
we can actually cause a lot of problems
9
41260
3000
ืื ื—ื ื• ื™ื›ื•ืœื™ื ืœื’ืจื•ื ืœื”ืจื‘ื” ื‘ืขื™ื•ืช
00:44
with the centrifuge.
10
44260
2000
ืขื ื”ืฆื ื˜ืจื™ืคื•ื’ื”.
00:46
The gray boxes don't run Windows software;
11
46260
2000
ื”ืงื•ืคืกืื•ืช ื”ืืคื•ืจื•ืช ืœื ืžืจื™ืฆื•ืช ืชื•ื›ื ืช Windows;
00:48
they are a completely different technology.
12
48260
3000
ื”ืŸ ืขื•ื‘ื“ื•ืช ืขืœ ื˜ื›ื ื•ืœื•ื’ื™ื” ืื—ืจืช ืœื’ืžืจื™.
00:51
But if we manage
13
51260
2000
ืื‘ืœ ืื ืื ื—ื ื• ืžืฆืœื™ื—ื™ื
00:53
to place a good Windows virus
14
53260
3000
ืœื”ื›ื ื™ืก ื•ื™ืจื•ืก Windows ื˜ื•ื‘
00:56
on a notebook
15
56260
2000
ืœืžื—ืฉื‘ ื ื™ื™ื“
00:58
that is used by a maintenance engineer
16
58260
2000
ืฉืžื”ื ื“ืก ืžื›ื•ื ื•ืช ืžืฉืชืžืฉ ื‘ื•
01:00
to configure this gray box,
17
60260
3000
ื›ื“ื™ ืœืฉื ื•ืช ืืช ืชืฆื•ืจืช ื”ืงื•ืคืกื” ื”ืืคื•ืจื” ื”ื–ืืช,
01:03
then we are in business.
18
63260
2000
ืื– ืื ื—ื ื• ื‘ืขื ื™ื™ื ื™ื.
01:05
And this is the plot behind Stuxnet.
19
65260
3000
ื•ื–ืืช ื”ืžื–ื™ืžื” ืžืื—ื•ืจื™ ืกื˜ืืงืกื ื˜.
01:08
So we start with a Windows dropper.
20
68260
5000
ืื– ืื ื—ื ื• ืžืชื—ื™ืœื™ื ืขื ื“ืจื•ืคืจ(ืžืชืงื™ืŸ ื•ื™ืจื•ืกื™ื) ื‘-Windows.
01:13
The payload goes onto the gray box,
21
73260
3000
ื”ื•ื™ืจื•ืก ื ื›ื ืก ืœืงื•ืคืกื” ื”ืืคื•ืจื”,
01:16
damages the centrifuge,
22
76260
2000
ืคื•ื’ืข ื‘ืฆื ื˜ืจื™ืคื•ื’ื•ืช,
01:18
and the Iranian nuclear program is delayed --
23
78260
2000
ื•ื”ืชื•ื›ื ื™ืช ื”ื’ืจืขื™ื ื™ืช ื”ืื™ืจื ื™ืช ื ื“ื—ื™ืช --
01:20
mission accomplished.
24
80260
2000
ื”ืžืฉื™ืžื” ื”ื•ืฉืœืžื”.
01:22
That's easy, huh?
25
82260
2000
ื–ื” ืงืœ, ื”ื?
01:24
I want to tell you how we found that out.
26
84260
3000
ืื ื™ ืจื•ืฆื” ืœืกืคืจ ืœื›ื ืื™ืš ื’ื™ืœื™ื ื• ืืช ื–ื”.
01:27
When we started our research on Stuxnet six months ago,
27
87260
3000
ื›ืฉื”ืชื—ืœื ื• ืืช ื”ืžื—ืงืจ ืฉืœื ื• ืขืœ ืกื˜ืืงืกื ื˜ ืœืคื ื™ ืฉื™ืฉื” ื—ื•ื“ืฉื™ื,
01:30
it was completely unknown what the purpose of this thing was.
28
90260
3000
ื”ืžื˜ืจื” ืฉืœ ื”ื“ื‘ืจ ื”ื–ื” ื”ื™ืชื” ืœื’ืžืจื™ ืœื ื™ื“ื•ืขื”.
01:33
The only thing that was known
29
93260
2000
ื”ื“ื‘ืจ ื”ื™ื—ื™ื“ ืฉื”ื™ื” ื™ื“ื•ืข
01:35
is it's very, very complex on the Windows part, the dropper part,
30
95260
3000
ื”ื•ื ืžืื•ื“ ืžื•ืจื›ื‘ ื‘ื—ืœืงื• ื‘ืžืขืจื›ืช Windows, ื”ื“ืจื•ืคืจ(ืžืชืงื™ืŸ ื”ื•ื™ืจื•ืกื™ื)
01:38
used multiple zero-day vulnerabilities.
31
98260
3000
ื”ืฉืชืžืฉ ื‘ืžืกืคืจ ืคื™ืจืฆื•ืช ื™ื•ื-ืืคืก(ืคื™ืจืฆื•ืช ืฉืœื ื”ื™ื• ื™ื“ื•ืขื•ืช ืœืคื ื™ ื›ืŸ).
01:41
And it seemed to want to do something
32
101260
3000
ื•ื”ื™ื” ื ื“ืžื” ืฉื”ื•ื ืจื•ืฆื” ืœืขืฉื•ืช ืžืฉื”ื•
01:44
with these gray boxes, these real-time control systems.
33
104260
2000
ืขื ื”ืงื•ืคืกืื•ืช ื”ืืคื•ืจื•ืช ื”ืืœื”, ืžืขืจื›ื•ืช ื‘ืงืจืช ื–ืžืŸ-ืืžืช.
01:46
So that got our attention,
34
106260
2000
ืื– ื–ื” ืชืคืก ืืช ืชืฉื•ืžืช ื”ืœื‘ ืฉืœื ื•,
01:48
and we started a lab project
35
108260
2000
ื•ื”ืชื—ืœื ื• ืคืจื•ื™ืงื˜ ื‘ืžืขื‘ื“ื”
01:50
where we infected our environment with Stuxnet
36
110260
4000
ืฉื‘ื• ื”ื“ื‘ืงื ื• ืืช ืกื‘ื™ื‘ืช ื”ืขื‘ื•ื“ื” ืฉืœื ื• ื‘ืกื˜ืืงืกื ื˜
01:54
and checked this thing out.
37
114260
2000
ื•ื‘ื“ืงื ื• ืืช ื”ื“ื‘ืจ ื”ื–ื”.
01:56
And then some very funny things happened.
38
116260
3000
ื•ืื– ื“ื‘ืจื™ื ืžืื•ื“ ืžื•ื–ืจื™ื ืงืจื•.
01:59
Stuxnet behaved like a lab rat
39
119260
3000
ืกื˜ืืงืกื ื˜ ื”ืชื ื”ื’ ื›ืžื• ืขื›ื‘ืจ ืžืขื‘ื“ื”
02:02
that didn't like our cheese --
40
122260
3000
ืฉืœื ืื”ื‘ ืืช ื”ื’ื‘ื™ื ื” ืฉืœื ื• --
02:05
sniffed, but didn't want to eat.
41
125260
2000
ื”ืจื™ื—, ืื‘ืœ ืœื ืจืฆื” ืœืื›ื•ืœ.
02:07
Didn't make sense to me.
42
127260
2000
ื–ื” ืœื ื”ื™ื” ืœื™ ื”ื’ื™ื•ื ื™.
02:09
And after we experimented with different flavors of cheese,
43
129260
3000
ื•ืื—ืจื™ ืฉืขืจื›ื ื• ื›ืžื” ื ื™ืกื•ื™ื™ื ืขื ื˜ืขืžื™ื ืฉื•ื ื™ื ืฉืœ ื’ื‘ื™ื ื•ืช,
02:12
I realized, well, this is a directed attack.
44
132260
4000
ื”ื‘ื ืชื™, ื˜ื•ื‘, ื–ืืช ื”ืชืงืคื” ืžื›ื•ื•ื ืช ืœืžื˜ืจื” ืกืคืฆื™ืคื™ืช.
02:16
It's completely directed.
45
136260
2000
ื”ื™ื ืœื’ืžืจื™ ืžื›ื•ื•ื ืช.
02:18
The dropper is prowling actively
46
138260
2000
ื”ื“ืจื•ืคืจ ืžืฉื•ื˜ื˜ ื‘ืื•ืคืŸ ืคืขื™ืœ
02:20
on the gray box
47
140260
2000
ื‘ืงื•ืคืกื” ื”ืืคื•ืจื”
02:22
if a specific configuration is found,
48
142260
3000
ืื ื”ื•ื ืžื•ืฆื ืชืฆื•ืจื” ืกืคืฆื™ืคื™ืช,
02:25
and even if the actual program code that it's trying to infect
49
145260
4000
ื•ืืคื™ืœื• ืื ื”ืชื•ื›ื ื” ืฉื”ื•ื ืžื ืกื” ืœื”ื“ื‘ื™ืง
02:29
is actually running on that target.
50
149260
2000
ื›ื‘ืจ ืจืฆื” ืขืœ ื”ืžื˜ืจื” ื”ื–ืืช. (ืื– ื”ื•ื™ืจื•ืก ื™ื•ืชืงืŸ)
02:31
And if not, Stuxnet does nothing.
51
151260
3000
ื•ืื ืœื, ืกื˜ืืงืกื ื˜ ืœื ืขื•ืฉื” ื›ืœื•ื.
02:34
So that really got my attention,
52
154260
2000
ืื– ื–ื” ืžืžืฉ ืขื ื™ื™ืŸ ืื•ืชื™,
02:36
and we started to work on this
53
156260
2000
ื•ื”ืชื—ืœื ื• ืœืขื‘ื•ื“ ืขืœ ื–ื”
02:38
nearly around the clock,
54
158260
2000
ื›ืžืขื˜ ืกื‘ื™ื‘ ื”ืฉืขื•ืŸ,
02:40
because I thought, "Well, we don't know what the target is.
55
160260
3000
ื›ื™ ื—ืฉื‘ืชื™, ื˜ื•ื‘, ืื ื—ื ื• ืœื ื™ื•ื“ืขื™ื ืžื” ื”ื™ื ื”ืžื˜ืจื” ืฉืœ ื”ื•ื™ืจื•ืก.
02:43
It could be, let's say for example,
56
163260
2000
ื”ื™ื ื™ื›ื•ืœื” ืœื”ื™ื•ืช, ื ื ื™ื— ืœื“ื•ื’ืžื,
02:45
a U.S. power plant,
57
165260
2000
ืชื—ื ืช ื—ืฉืžืœ ืืžืจื™ืงืื™ืช,
02:47
or a chemical plant in Germany.
58
167260
2000
ืื• ืžืคืขืœ ื›ื™ืžื™ ื‘ื’ืจืžื ื™ื”.
02:49
So we better find out what the target is soon."
59
169260
3000
ืื– ื›ื“ืื™ ืฉื ื’ืœื” ืžื” ื”ืžื˜ืจื” ื‘ืงืจื•ื‘.
02:52
So we extracted and decompiled
60
172260
2000
ืื– ืคืจืกื ื• ื•ื‘ื™ืฆืขื ื• ื”ื™ื“ื•ืจ ื”ืคื•ืš
02:54
the attack code,
61
174260
2000
ืœืงื•ื“ ื”ืชืงื™ืคื”,
02:56
and we discovered that it's structured in two digital bombs --
62
176260
3000
ื•ื’ื™ืœื™ื ื• ืฉื”ื•ื ืžื•ื‘ื ื” ืžืฉืชื™ ืคืฆืฆื•ืช ื“ื™ื’ื™ื˜ืœื™ื•ืช --
02:59
a smaller one and a bigger one.
63
179260
3000
ืื—ืช ืงื˜ื ื” ื•ืื—ืช ื’ื“ื•ืœื”.
03:02
And we also saw that they are very professionally engineered
64
182260
4000
ื•ืจืื™ื ื• ื’ื ืฉื”ืŸ ืžืชื•ื›ื ื ื•ืช ื‘ืื•ืคืŸ ืžืื•ื“ ืžืงืฆื•ืขื™
03:06
by people who obviously had all insider information.
65
186260
4000
ืขืœ ื™ื“ื™ ืื ืฉื™ื ืฉื”ื™ื” ืœื”ื ื‘ืื•ืคืŸ ื‘ืจื•ืจ ืืช ื›ืœ ื”ืžื™ื“ืข ื”ืคื ื™ืžื™.
03:10
They knew all the bits and bites
66
190260
2000
ื”ื ื”ื›ื™ืจื• ืืช ื›ืœ ื”ื‘ื™ื˜ื™ื ื•ื”ื‘ื™ื™ื˜ื™ื
03:12
that they had to attack.
67
192260
2000
ืฉื”ื ื”ื™ื• ืฆืจื™ื›ื™ื ืœืชืงื•ืฃ.
03:14
They probably even know the shoe size of the operator.
68
194260
3000
ื”ื ื›ื ืจืื” ืืคื™ืœื• ื™ื•ื“ืขื™ื ืืช ืžื™ื“ืช ื”ื ืขืœื™ื™ื ืฉืœ ื”ืžืคืขื™ืœ.
03:17
So they know everything.
69
197260
2000
ืื– ื”ื ื™ื•ื“ืขื™ื ื”ื›ืœ.
03:19
And if you have heard that the dropper of Stuxnet
70
199260
3000
ื•ืื ืฉืžืขืชื ืฉื”ื“ืจื•ืคืจ ืฉืœ ืกื˜ืืงืกื ื˜
03:22
is complex and high-tech,
71
202260
2000
ื”ื•ื ืžื•ืจื›ื‘ ื•ืžืฉื•ื›ืœืœ,
03:24
let me tell you this:
72
204260
2000
ืื ื™ ืื’ื™ื“ ืœื›ื ื™ื•ืชืจ ืžื–ื”:
03:26
the payload is rocket science.
73
206260
2000
ื”ื•ื™ืจื•ืก ืขืฆืžื• ื”ื•ื ืžื“ืข ื˜ื™ืœื™ื.
03:28
It's way above everything
74
208260
2000
ื–ื” ื”ืจื‘ื” ืžืขื‘ืจ ืœื›ืœ ื“ื‘ืจ
03:30
that we have ever seen before.
75
210260
3000
ืฉื ืชืงืœื ื• ื‘ื• ื‘ืขื‘ืจ.
03:33
Here you see a sample of this actual attack code.
76
213260
3000
ื›ืืŸ ืืชื ืจื•ืื™ื ื“ื•ื’ืžื” ืฉืœ ืงื•ื“ ื”ืชืงื™ืคื” ื”ืžืงื•ืจื™.
03:36
We are talking about --
77
216260
2000
ืื ื—ื ื• ืžื“ื‘ืจื™ื ืขืœ --
03:38
around about 15,000 lines of code.
78
218260
3000
ืกื‘ื™ื‘ื•ืช 15,000 ืฉื•ืจื•ืช ืงื•ื“.
03:41
Looks pretty much like old-style assembly language.
79
221260
3000
ื ืจืื” ื›ืžืขื˜ ื›ืžื• ืฉืคืช ืืกืžื‘ืœื™ ืฉืœ ืคืขื.
03:44
And I want to tell you how we were able
80
224260
2000
ื•ืื ื™ ืจื•ืฆื” ืœืกืคืจ ืœื›ื ืื™ืš ื”ืฆืœื—ื ื•
03:46
to make sense out of this code.
81
226260
2000
ืœื”ื‘ื™ืŸ ืืช ื”ืงื•ื“ ื”ื–ื”.
03:48
So what we were looking for is, first of all, system function calls,
82
228260
3000
ืื– ืžื” ืฉื—ื™ืคืฉื ื• ืงื•ื“ื ื›ืœ ื”ืŸ ืคื•ื ืงืฆื™ื•ืช ืงืจื™ืื•ืช ืžืขืจื›ืช,
03:51
because we know what they do.
83
231260
2000
ื›ื™ ืื ื—ื ื• ื™ื•ื“ืขื™ื ืžื” ื”ืŸ ืขื•ืฉื•ืช.
03:53
And then we were looking for timers and data structures
84
233260
4000
ื•ืื– ื—ื™ืคืฉื ื• ื˜ื™ื™ืžืจื™ื ื•ืžื‘ื ื™ ื ืชื•ื ื™ื
03:57
and trying to relate them to the real world --
85
237260
2000
ื•ื ื™ืกื™ื ื• ืœืฉื™ื™ืš ืื•ืชื ืœืขื•ืœื ื”ืืžื™ืชื™ --
03:59
to potential real world targets.
86
239260
2000
ืœืžื˜ืจื•ืช ืืคืฉืจื™ื•ืช ื‘ืขื•ืœื ื”ืืžื™ืชื™.
04:01
So we do need target theories
87
241260
3000
ืื– ืื ื—ื ื• ืฆืจื™ื›ื™ื ืชืื•ืจื™ื•ืช ืขืœ ืžื˜ืจื•ืช
04:04
that we can prove or disprove.
88
244260
3000
ืฉื ื•ื›ืœ ืœื”ื•ื›ื™ื— ืื• ืœื”ืคืจื™ืš.
04:07
In order to get target theories,
89
247260
2000
ื›ื“ื™ ืœื”ื’ื™ืข ืœืชืื•ืจื™ื•ืช ืขืœ ืžื˜ืจื•ืช,
04:09
we remember
90
249260
2000
ืื ื—ื ื• ื–ื•ื›ืจื™ื
04:11
that it's definitely hardcore sabotage,
91
251260
2000
ืฉื–ืืช ื‘ื•ื•ื“ืื•ืช ืคืขื•ืœืช ื—ื‘ืœื” ืงืฉื” ืžืื•ื“,
04:13
it must be a high-value target
92
253260
2000
ื–ืืช ื—ื™ื™ื‘ืช ืœื”ื™ื•ืช ืžืขืจื›ืช ื—ืฉื•ื‘ื” ื‘ื™ื•ืชืจ,
04:15
and it is most likely located in Iran,
93
255260
3000
ื•ืจื•ื‘ ื”ืกื™ื›ื•ื™ื™ื ืฉื”ื™ื ื ืžืฆืืช ื‘ืื™ืจืŸ,
04:18
because that's where most of the infections had been reported.
94
258260
4000
ืžืื—ืจ ื•ืฉื ื“ื•ื•ื—ื• ืจื•ื‘ ืžืงืจื™ ื”ืžื—ืฉื‘ื™ื ืฉื”ื•ื“ื‘ืงื•.
04:22
Now you don't find several thousand targets in that area.
95
262260
3000
ืืชื” ืœื ืžื•ืฆื ื›ืžื” ืืœืคื™ ืžื˜ืจื•ืช ื‘ืื–ื•ืจ ื”ื–ื”.
04:25
It basically boils down
96
265260
2000
ื–ื” ื‘ืขื™ืงืจื•ืŸ ืžืฆื˜ืžืฆื
04:27
to the Bushehr nuclear power plant
97
267260
2000
ืœื›ื•ืจ ื”ื’ืจืขื™ื ื™ ื‘ื‘ื•ืฉื”ืจ
04:29
and to the Natanz fuel enrichment plant.
98
269260
2000
ื•ืœืžืคืขืœ ื”ืขืฉืจืช ื”ื“ืœืง ื”ื’ืจืขื™ื ื™ ื‘ื ืชื ื–.
04:31
So I told my assistant,
99
271260
2000
ืื– ืืžืจืชื™ ืœืขื•ื–ืจ ืฉืœื™,
04:33
"Get me a list of all centrifuge and power plant experts from our client base."
100
273260
3000
"ืชืฉื™ื’ ืœื™ ืจืฉื™ืžื” ืฉืœ ื›ืœ ืžื•ืžื—ื™ ื”ืฆื ื˜ืจื™ืคื•ื’ื•ืช ื•ืชื—ื ื•ืช ื”ื›ื— ืžืชื•ืš ื”ืœืงื•ื—ื•ืช ืฉืœื ื•"
04:36
And I phoned them up and picked their brain
101
276260
2000
ื•ื”ืชืงืฉืจืชื™ ืืœื™ื”ื ื•ืชืฉืืœืชื™ ืื•ืชื
04:38
in an effort to match their expertise
102
278260
2000
ื›ื“ื™ ืœื ืกื•ืช ืœื”ืชืื™ื ืืช ื”ื™ื“ืข ื•ื”ืžื•ืžื—ื™ื•ืช ืฉืœื”ื
04:40
with what we found in code and data.
103
280260
3000
ืœืžื” ืฉืžืฆืื ื• ื‘ืงื•ื“ ื•ื‘ื ืชื•ื ื™ื.
04:43
And that worked pretty well.
104
283260
2000
ื•ื–ื” ืขื‘ื“ ื“ื™ ื˜ื•ื‘.
04:45
So we were able to associate
105
285260
2000
ืื– ื”ืฆืœื—ื ื• ืœืงืฉืจ
04:47
the small digital warhead
106
287260
2000
ืืช ืจืืฉ ื”ื˜ื™ืœ ื”ื“ื™ื’ื™ื˜ืœื™ ื”ืงื˜ืŸ
04:49
with the rotor control.
107
289260
2000
ืขื ืฉืœื™ื˜ื” ืขืœ ื”ืจื•ื˜ื•ืจ.
04:51
The rotor is that moving part within the centrifuge,
108
291260
3000
ื”ืจื•ื˜ื•ืจ ื”ื•ื ื”ื—ืœืง ื”ื ืข ื‘ืชื•ืš ื”ืฆื ื˜ืจื™ืคื•ื’ื”,
04:54
that black object that you see.
109
294260
2000
ื”ื—ืคืฅ ื”ืฉื—ื•ืจ ืฉืืชื ืจื•ืื™ื.
04:56
And if you manipulate the speed of this rotor,
110
296260
3000
ื•ืื ืžืฉื ื™ื ืืช ื”ืžื”ื™ืจื•ืช ืฉืœ ื”ืจื•ื˜ื•ืจ ื”ื–ื”,
04:59
you are actually able to crack the rotor
111
299260
2000
ืืคืฉืจ ื‘ืืžืช ืœืกื“ื•ืง ืืช ื”ืจื•ื˜ื•ืจ
05:01
and eventually even have the centrifuge explode.
112
301260
4000
ื•ื‘ืกื•ืคื• ืฉืœ ื“ื‘ืจ ืืคื™ืœื• ืœื’ืจื•ื ืœืฆื ื˜ืจื™ืคื•ื’ื” ืœื”ืชืคื•ืฆืฅ.
05:05
What we also saw
113
305260
2000
ืžื” ืฉืขื•ื“ ืจืื™ื ื•
05:07
is that the goal of the attack
114
307260
2000
ื”ื•ื ืฉื”ืžื˜ืจื” ืฉืœ ื”ื”ืชืงืคื”
05:09
was really to do it slowly and creepy --
115
309260
3000
ื”ื™ืชื” ืœืขืฉื•ืช ืืช ื–ื” ื‘ืื™ื˜ื™ื•ืช ื•ื‘ื—ืฉืื™ื•ืช --
05:12
obviously in an effort
116
312260
2000
ื‘ืžืืžืฅ ื‘ืจื•ืจ
05:14
to drive maintenance engineers crazy,
117
314260
3000
ืœืฉื’ืข ืืช ืžื”ื ื“ืกื™ ื”ืชื—ื–ื•ืงื”,
05:17
that they would not be able to figure this out quickly.
118
317260
3000
ื›ืš ืฉื”ื ืœื ื™ื•ื›ืœื• ืœืคืขื ื— ืืช ื–ื” ื‘ืžื”ื™ืจื•ืช.
05:20
The big digital warhead -- we had a shot at this
119
320260
3000
ืจืืฉ ื”ื˜ื™ืœ ื”ื“ื™ื’ื™ื˜ืœื™ ื”ื’ื“ื•ืœ -- ื ื™ืกื™ื ื• ืœืคืขื ื— ืื•ืชื•
05:23
by looking very closely
120
323260
2000
ืขืœ ื™ื“ื™ ื”ืชื‘ื•ื ื ื•ืช ืงืจื•ื‘ื”
05:25
at data and data structures.
121
325260
2000
ื‘ื ืชื•ื ื™ื ื•ืžื‘ื ื™ ื ืชื•ื ื™ื.
05:27
So for example, the number 164
122
327260
2000
ืื– ืœื“ื•ื’ืžื”, ื”ืžืกืคืจ 164
05:29
really stands out in that code;
123
329260
2000
ืžืžืฉ ื‘ื•ืœื˜ ื‘ืงื•ื“ ื”ื–ื”;
05:31
you can't overlook it.
124
331260
2000
ืื™ ืืคืฉืจ ืœืคืกืคืก ืื•ืชื•.
05:33
I started to research scientific literature
125
333260
2000
ื”ืชื—ืœืชื™ ืœื—ืงื•ืจ ืกืคืจื•ืช ืžื“ืขื™ืช
05:35
on how these centrifuges
126
335260
2000
ืขืœ ืื™ืš ื”ืฆื ื˜ืจื™ืคื•ื’ื•ืช ื”ืืœื”
05:37
are actually built in Natanz
127
337260
2000
ื‘ื ื•ื™ื•ืช ื‘ื ืชื ื–
05:39
and found they are structured
128
339260
2000
ื•ืžืฆืืชื™ ืฉื”ืŸ ืžื•ื‘ื ื•ืช
05:41
in what is called a cascade,
129
341260
2000
ื‘ืžื” ืฉื ืงืจื” ืกื“ืจื”,
05:43
and each cascade holds 164 centrifuges.
130
343260
4000
ื•ื›ืœ ืกื“ืจื” ืžื›ื™ืœื” 164 ืฆื ื˜ืจื™ืคื•ื’ื•ืช.
05:47
So that made sense, that was a match.
131
347260
2000
ืื– ื–ื” ื”ื™ื” ื”ื’ื™ื•ื ื™, ื–ื” ื”ืชืื™ื.
05:49
And it even got better.
132
349260
2000
ื•ื–ื” ืืคื™ืœื• ื”ืฉืชืคืจ.
05:51
These centrifuges in Iran
133
351260
2000
ื”ืฆื ื˜ืจื™ืคื•ื’ื•ืช ื”ืืœื” ื‘ืื™ืจืŸ
05:53
are subdivided into 15, what is called, stages.
134
353260
4000
ืžื—ื•ืœืงื•ืช ืœ-15 ืžื” ืฉื ืงืจื, ืฉืœื‘ื™ื.
05:57
And guess what we found in the attack code?
135
357260
2000
ื•ื ื—ืฉื• ืžื” ืžืฆืื ื• ื‘ืงื•ื“ ื”ืชืงื™ืคื”?
05:59
An almost identical structure.
136
359260
2000
ืžื‘ื ื” ื›ืžืขื˜ ื–ื”ื”.
06:01
So again, that was a real good match.
137
361260
3000
ืื– ืฉื•ื‘, ื–ืืช ื”ื™ืชื” ื”ืชืืžื” ืžืื•ื“ ื˜ื•ื‘ื”.
06:04
And this gave us very high confidence for what we were looking at.
138
364260
3000
ื•ื–ื” ื ืชืŸ ืœื ื• ื”ืจื‘ื” ื‘ื™ื˜ื—ื•ืŸ ืขืœ ืžื” ืฉืื ื—ื ื• ื‘ื•ื“ืงื™ื.
06:07
Now don't get me wrong here, it didn't go like this.
139
367260
3000
ืืœ ืชื‘ื™ื ื• ืื•ืชื™ ืœื ื ื›ื•ืŸ, ื–ื” ืœื ื”ืœืš ื‘ืงืœื•ืช.
06:10
These results have been obtained
140
370260
3000
ื”ืชื•ืฆืื•ืช ื”ืืœื” ื”ื•ืฉื’ื•
06:13
over several weeks of really hard labor.
141
373260
3000
ืื—ืจื™ ืžืกืคืจ ืฉื‘ื•ืขื•ืช ืฉืœ ืขื‘ื•ื“ื” ืงืฉื” ืžืื•ื“.
06:16
And we often went into just a dead end
142
376260
3000
ื•ื”ืจื‘ื” ืคืขืžื™ื ืคืฉื•ื˜ ื ืชืงืขื ื• ื‘ืžื‘ื•ื ืœืœื ืžื•ืฆื
06:19
and had to recover.
143
379260
2000
ื•ื”ื™ื™ื ื• ืฆืจื™ื›ื™ื ืœื ืกื•ืช ืฉื•ื‘.
06:21
Anyway, so we figured out
144
381260
2000
ื‘ื›ืœ ืžืงืจื”, ืื– ื’ื™ืœื™ื ื•
06:23
that both digital warheads
145
383260
2000
ืฉืฉื ื™ ืจืืฉื™ ื”ื˜ื™ืœ ื”ื“ื™ื’ื™ื˜ืœื™ื™ื
06:25
were actually aiming at one and the same target,
146
385260
2000
ื‘ืขืฆื ื›ื•ื•ื ื• ืืœ ืื•ืชื” ืžื˜ืจื”,
06:27
but from different angles.
147
387260
2000
ืื‘ืœ ืžื–ื•ื•ื™ื•ืช ืฉื•ื ื•ืช.
06:29
The small warhead is taking one cascade,
148
389260
3000
ืจืืฉ ื”ื˜ื™ืœ ื”ืงื˜ืŸ ืœื•ืงื— ืกื“ืจื” ืื—ืช,
06:32
and spinning up the rotors and slowing them down,
149
392260
3000
ื•ืžืกื•ื‘ื‘ ืืช ื”ืจื•ื˜ื•ืจื™ื ื•ืžืื˜ ืื•ืชื,
06:35
and the big warhead
150
395260
2000
ื•ืจืืฉ ื”ื˜ื™ืœ ื”ื’ื“ื•ืœ
06:37
is talking to six cascades
151
397260
2000
ืžื“ื‘ืจ ืขื ืฉืฉ ืกื“ืจื•ืช
06:39
and manipulating valves.
152
399260
2000
ื•ืžืฉื ื” ืชืคืงื•ื“ื™ ืฉืกืชื•ืžื™ื.
06:41
So in all, we are very confident
153
401260
2000
ืื– ื‘ืกืš ื”ื›ืœ, ืื ื—ื ื• ืžืื•ื“ ื‘ื˜ื•ื—ื™ื
06:43
that we have actually determined what the target is.
154
403260
2000
ืฉืžืฆืื ื• ืืช ื”ืžื˜ืจื” ื”ืืžื™ืชื™ืช.
06:45
It is Natanz, and it is only Natanz.
155
405260
3000
ื–ืืช ื ืชื ื–, ื•ื–ืืช ืจืง ื ืชื ื–.
06:48
So we don't have to worry
156
408260
2000
ืื– ืื ื—ื ื• ืœื ืฆืจื™ื›ื™ื ืœื“ืื•ื’
06:50
that other targets
157
410260
2000
ืฉืžื ืžื˜ืจื•ืช ืื—ืจื•ืช
06:52
might be hit by Stuxnet.
158
412260
2000
ื™ื™ืคื’ืขื• ืขืœ ื™ื“ื™ ืกื˜ืืงืกื ื˜.
06:54
Here's some very cool stuff that we saw --
159
414260
3000
ื”ื ื” ื›ืžื” ื“ื‘ืจื™ื ืžืื•ื“ ืžื’ื ื™ื‘ื™ื ืฉืจืื™ื ื• --
06:57
really knocked my socks off.
160
417260
2000
ืฉืžืžืฉ ื”ืคืชื™ืขื• ืื•ืชื™.
06:59
Down there is the gray box,
161
419260
2000
ืฉื ืœืžื˜ื” ื ืžืฆืืช ื”ืงื•ืคืกื” ื”ืืคื•ืจื”,
07:01
and on the top you see the centrifuges.
162
421260
3000
ื•ืœืžืขืœื” ืืชื ืจื•ืื™ื ืืช ื”ืฆื ื˜ืจื™ืคื•ื’ื•ืช.
07:04
Now what this thing does
163
424260
2000
ืขื›ืฉื™ื• ืžื” ืฉื”ื“ื‘ืจ ื”ื–ื” ืขื•ืฉื”
07:06
is it intercepts the input values from sensors --
164
426260
3000
ื”ื•ื ืฉื”ื•ื ืžื™ื™ืจื˜ ืืช ื”ืขืจื›ื™ื ืฉืžืชืงื‘ืœื™ื ื‘ื—ื™ื™ืฉื ื™ื --
07:09
so for example, from pressure sensors
165
429260
2000
ืื– ืœื“ื•ื’ืžื, ืžื—ื™ื™ืฉื ื™ ืœื—ืฅ
07:11
and vibration sensors --
166
431260
2000
ื•ื—ื™ื™ืฉื ื™ ืจื˜ื˜ --
07:13
and it provides legitimate program code,
167
433260
3000
ื•ื‘ื•ื ื” ืงื•ื“ ืœื’ื™ื˜ื™ืžื™,
07:16
which is still running during the attack,
168
436260
2000
ืฉืขื“ื™ื™ืŸ ืจืฅ ื‘ื–ืžืŸ ื”ื”ืชืงืคื”,
07:18
with fake input data.
169
438260
2000
ืขื ืžื™ื“ืข ืงืœื˜ ืžื–ื•ื™ื™ืฃ.
07:20
And as a matter of fact, this fake input data
170
440260
2000
ื•ืœืžืขืฉื”, ืžื™ื“ืข ื”ืงืœื˜ ื”ืžื–ื•ื™ื™ืฃ ื”ื–ื”
07:22
is actually prerecorded by Stuxnet.
171
442260
3000
ืžื•ืงืœื˜ ืžืจืืฉ ืขืœ ื™ื“ื™ ืกื˜ืืงืกื ื˜.
07:25
So it's just like from the Hollywood movies
172
445260
2000
ืื– ื–ื” ื›ืžื• ื‘ืกืจื˜ื™ื ื”ื”ื•ืœื™ื•ื•ื“ื™ื™ื
07:27
where during the heist,
173
447260
2000
ืฉื‘ื–ืžืŸ ื”ืขื•ืงืฅ,
07:29
the observation camera is fed with prerecorded video.
174
449260
3000
ืžืฆืœืžื•ืช ื”ืื‘ื˜ื—ื” ืžื•ื–ื ื•ืช ื‘ืกืจื˜ื•ื ื™ื ืžื•ืงืœื˜ื™ื ืžืจืืฉ.
07:32
That's cool, huh?
175
452260
2000
ื–ื” ืžื’ื ื™ื‘, ืื”?
07:35
The idea here is obviously
176
455260
2000
ื”ืจืขื™ื•ืŸ ื›ืืŸ ื”ื•ื ื›ืžื•ื‘ืŸ
07:37
not only to fool the operators in the control room.
177
457260
3000
ืœื ืจืง ืœืฉื˜ื•ืช ื‘ืžืคืขื™ืœื™ื ื‘ื—ื“ืจ ื”ื‘ืงืจื”.
07:40
It actually is much more dangerous and aggressive.
178
460260
4000
ื”ืืžืช ืฉื–ื” ื”ืจื‘ื” ื™ื•ืชืจ ืžืกื•ื›ืŸ ื•ืื’ืจืกื™ื‘ื™.
07:44
The idea
179
464260
2000
ื”ืจืขื™ื•ืŸ
07:46
is to circumvent a digital safety system.
180
466260
3000
ื”ื•ื ืœืขืงื•ืฃ ืžืขืจื›ืช ื‘ื˜ื™ื—ื•ืช ื“ื™ื’ื™ื˜ืœื™ืช.
07:50
We need digital safety systems
181
470260
2000
ืื ื—ื ื• ืฆืจื™ื›ื™ื ืžืขืจื›ื•ืช ื‘ื˜ื™ื—ื•ืช ื“ื™ื’ื™ื˜ืœื™ื•ืช
07:52
where a human operator could not act quick enough.
182
472260
3000
ื‘ืžืงื•ืžื•ืช ืฉืžืคืขื™ืœ ืื ื•ืฉื™ ืœื ื™ื›ื•ืœ ืœืคืขื•ืœ ืžื”ืจ ืžืกืคื™ืง.
07:55
So for example, in a power plant,
183
475260
2000
ืื– ืœื“ื•ื’ืžื”, ื‘ืชื—ื ืช ื›ื—,
07:57
when your big steam turbine gets too over speed,
184
477260
3000
ื›ืฉื˜ื•ืจื‘ื™ื ืช ื”ืงื™ื˜ื•ืจ ื”ื’ื“ื•ืœื” ืฉืœืš ืฆื•ื‘ืจืช ื™ื•ืชืจ ืžื“ื™ ืžื”ื™ืจื•ืช,
08:00
you must open relief valves within a millisecond.
185
480260
3000
ืืชื” ื—ื™ื™ื‘ ืœืคืชื•ื— ืืช ืฉืกืชื•ืžื™ ื”ืฉื—ืจื•ืจ ื‘ืชื•ืš ืืœืคื™ืช ื”ืฉื ื™ื”.
08:03
Obviously, this cannot be done by a human operator.
186
483260
3000
ื‘ืจื•ืจ ืžืืœื™ื• ืฉื–ื” ืœื ื™ื›ื•ืœ ืœื”ืชื‘ืฆืข ืขืœ ื™ื“ื™ ืžืคืขื™ืœ ืื ื•ืฉื™.
08:06
So this is where we need digital safety systems.
187
486260
2000
ืื– ื‘ืžืงื•ืžื•ืช ื›ืืœื” ืื ื—ื ื• ืฆืจื™ื›ื™ื ืžืขืจื›ื•ืช ื‘ื˜ื™ื—ื•ืช ื“ื™ื’ื™ื˜ืœื™ื•ืช.
08:08
And when they are compromised,
188
488260
2000
ื•ื›ืฉื”ืŸ ื ืคืจืฆื•ืช,
08:10
then real bad things can happen.
189
490260
3000
ืื– ื“ื‘ืจื™ื ืžืื•ื“ ืจืขื™ื ื™ื›ื•ืœื™ื ืœืงืจื•ืช.
08:13
Your plant can blow up.
190
493260
2000
ื”ืžืคืขืœ ืฉืœืš ื™ื›ื•ืœ ืœื”ืชืคื•ืฆืฅ.
08:15
And neither your operators nor your safety system will notice it.
191
495260
3000
ื•ืœื ื”ืžืคืขื™ืœื™ื ืฉืœืš ื•ืœื ืžืขืจื›ืช ื”ื‘ื˜ื™ื—ื•ืช ืฉืœืš ื™ืฉื™ืžื• ืœื‘ ืœื›ืš.
08:18
That's scary.
192
498260
2000
ื–ื” ืžืคื—ื™ื“.
08:20
But it gets worse.
193
500260
2000
ืื‘ืœ ื–ื” ื ื”ื™ื” ื™ื•ืชืจ ื’ืจื•ืข.
08:22
And this is very important, what I'm going to say.
194
502260
3000
ื•ื–ื” ืžืื•ื“ ื—ืฉื•ื‘, ืžื” ืฉืื ื™ ื”ื•ืœืš ืœื”ื’ื™ื“.
08:25
Think about this:
195
505260
2000
ืชื—ืฉื‘ื• ืขืœ ื–ื”.
08:27
this attack is generic.
196
507260
3000
ื”ื”ืชืงืคื” ื”ื–ืืช ื”ื™ื ื›ืœืœื™ืช.
08:30
It doesn't have anything to do, in specifics,
197
510260
4000
ื”ื™ื ืœื ืขื•ืกืงืช ืกืคืฆื™ืคื™ืช,
08:34
with centrifuges,
198
514260
2000
ื‘ืฆื ื˜ืจื™ืคื•ื’ื•ืช,
08:36
with uranium enrichment.
199
516260
3000
ื‘ื”ืขืฉืจืช ืื•ืจื ื™ื•ื.
08:39
So it would work as well, for example,
200
519260
3000
ืื– ื”ื™ื ื™ื›ื•ืœื” ืœืขื‘ื•ื“ ื’ื ืœื“ื•ื’ืžื”,
08:42
in a power plant
201
522260
2000
ื‘ืชื—ื ืช ื›ื—
08:44
or in an automobile factory.
202
524260
3000
ืื• ื‘ืžืคืขืœ ืœื™ื™ืฆื•ืจ ื›ืœื™ ืจื›ื‘.
08:47
It is generic.
203
527260
2000
ื”ื™ื ื›ืœืœื™ืช.
08:49
And you don't have -- as an attacker --
204
529260
2000
ื•ืืชื” ืœื ื—ื™ื™ื‘ -- ื‘ืชื•ืจ ืชื•ืงืฃ --
08:51
you don't have to deliver this payload
205
531260
3000
ืืชื” ืœื ื—ื™ื™ื‘ ืœื”ืคื™ืฅ ืืช ื”ื•ื™ืจื•ืก ื”ื–ื”
08:54
by a USB stick,
206
534260
2000
ื‘ืขื–ืจืช ื›ื•ื ืŸ USB,
08:56
as we saw it in the case of Stuxnet.
207
536260
2000
ื›ืžื• ืฉืจืื™ื ื• ื‘ืžืงืจื” ืฉืœ ืกื˜ืืงืกื ื˜.
08:58
You could also use conventional worm technology for spreading.
208
538260
3000
ืืชื” ื™ื›ื•ืœ ื’ื ืœื”ืฉืชืžืฉ ื‘ืฉื™ื˜ื•ืช ืจื’ื™ืœื•ืช ืœื”ืคืฆืช ืชื•ืœืขืช ืžื—ืฉื‘.
09:01
Just spread it as wide as possible.
209
541260
3000
ืคืฉื•ื˜ ืชืคื™ืฅ ืืช ื–ื” ื›ืžื” ืฉืจืง ืืคืฉืจ.
09:04
And if you do that,
210
544260
2000
ื•ืื ืืชื” ืขื•ืฉื” ืืช ื–ื”,
09:06
what you end up with
211
546260
2000
ืžื” ืฉืืชื” ืžืงื‘ืœ
09:08
is a cyber weapon of mass destruction.
212
548260
5000
ื”ื•ื ื ืฉืง ืงื™ื‘ืจื ื˜ื™ ืœื”ืฉืžื“ื” ื”ืžื•ื ื™ืช.
09:14
That's the consequence
213
554260
2000
ื–ืืช ื”ืชื•ืฆืื”
09:16
that we have to face.
214
556260
3000
ืฉืื™ืชื” ืื ื• ืฆืจื™ื›ื™ื ืœื”ืชืžื•ื“ื“.
09:19
So unfortunately,
215
559260
3000
ืื– ืœืจื•ืข ื”ืžื–ืœ,
09:22
the biggest number of targets for such attacks
216
562260
3000
ื”ืžืกืคืจ ื”ื’ื“ื•ืœ ื‘ื™ื•ืชืจ ืฉืœ ืžื˜ืจื•ืช ืœื”ืชืงืคื•ืช ื›ืืœื”
09:25
are not in the Middle East.
217
565260
2000
ื”ื•ื ืœื ื‘ืžื–ืจื— ื”ืชื™ื›ื•ืŸ.
09:27
They're in the United States and Europe and in Japan.
218
567260
3000
ื”ืŸ ื ืžืฆืื•ืช ื‘ืืจื”"ื‘ ื•ืื™ืจื•ืคื” ื•ื‘ื™ืคืŸ.
09:30
So all of the green areas,
219
570260
2000
ืื– ื›ืœ ื”ืื™ื–ื•ืจื™ื ื”ื™ืจื•ืงื™ื,
09:32
these are your target-rich environments.
220
572260
3000
ืืœื” ื”ืกื‘ื™ื‘ื•ืช ืขื ื”ื›ื™ ื”ืจื‘ื” ืžื˜ืจื•ืช.
09:35
We have to face the consequences,
221
575260
3000
ืื ื—ื ื• ืฆืจื™ื›ื™ื ืœื”ืชืžื•ื“ื“ ืขื ื”ืชื•ืฆืื•ืช,
09:38
and we better start to prepare right now.
222
578260
3000
ื•ื›ื“ืื™ ืฉื ืชื—ื™ืœ ืœื”ืชื›ื•ื ืŸ ืขื›ืฉื™ื•.
09:41
Thanks.
223
581260
2000
ืชื•ื“ื”.
09:43
(Applause)
224
583260
6000
(ืžื—ื™ืื•ืช ื›ืคื™ื™ื)
09:49
Chris Anderson: I've got a question.
225
589260
2000
ื›ืจื™ืก ืื ื“ืจืกื•ืŸ: ื™ืฉ ืœื™ ืฉืืœื”.
09:53
Ralph, it's been quite widely reported
226
593260
2000
ืจืืœืฃ, ื–ื” ื“ื•ื•ื— ื‘ื”ืจื—ื‘ื”
09:55
that people assume that Mossad
227
595260
2000
ืฉืื ืฉื™ื ืžื ื™ื—ื™ื ืฉื”ืžื•ืกื“
09:57
is the main entity behind this.
228
597260
2000
ื”ื•ื ื”ื™ืฉื•ืช ื”ืžืจื›ื–ื™ืช ืžืื—ื•ืจื™ ื–ื”.
09:59
Is that your opinion?
229
599260
3000
ื”ืื ื–ืืช ื“ืขืชืš?
10:02
Ralph Langner: Okay, you really want to hear that?
230
602260
2000
ืจืืœืฃ ืœืื ื’ื ืจ: ืื•ืงื™ื™, ืืชื” ื‘ืืžืช ืจื•ืฆื” ืœืฉืžื•ืข ืืช ื–ื”?
10:04
Yeah. Okay.
231
604260
2000
ื›ืŸ. ืื•ืงื™ื™.
10:06
My opinion is that the Mossad is involved,
232
606260
3000
ื“ืขืชื™ ื”ื™ื ืฉื”ืžื•ืกื“ ืžืขื•ืจื‘,
10:09
but that the leading force is not Israel.
233
609260
3000
ืื‘ืœ ืฉื”ื›ื— ื”ืžื•ื‘ื™ืœ ืื™ื ื• ื™ืฉืจืืœ.
10:12
So the leading force behind that
234
612260
2000
ื”ื›ื— ื”ืžื•ื‘ื™ืœ ืžืื—ื•ืจื™ ื–ื”
10:14
is the cyber superpower.
235
614260
3000
ื”ื•ื ืžืขืฆืžืช ื”ืขืœ ื”ืงื™ื‘ืจื ื˜ื™ืช.
10:17
There is only one,
236
617260
2000
ื•ื™ืฉ ืจืง ืื—ืช,
10:19
and that's the United States --
237
619260
2000
ื•ื–ืืช ืืจื”"ื‘ --
10:21
fortunately, fortunately.
238
621260
2000
ืœืžืจื‘ื” ื”ืžื–ืœ, ืœืžืจื‘ื” ื”ืžื–ืœ.
10:23
Because otherwise,
239
623260
2000
ื›ื™ ืื—ืจืช,
10:25
our problems would even be bigger.
240
625260
3000
ื”ื‘ืขื™ื•ืช ืฉืœื ื• ื”ื™ื• ืืคื™ืœื• ื™ื•ืชืจ ื’ื“ื•ืœื•ืช.
10:28
CA: Thank you for scaring the living daylights out of us. Thank you, Ralph.
241
628260
4000
ื›ืจื™ืก: ืชื•ื“ื” ืœืš ืฉื”ืคื—ื“ืช ืื•ืชื ื• ืขื“ ืขืžืงื™ ื ืฉืžืชื ื•. ืชื•ื“ื” ืœืš ืจืืœืฃ.
10:32
(Applause)
242
632260
2000
(ืžื—ื™ืื•ืช ื›ืคื™ื™ื)
ืขืœ ืืชืจ ื–ื”

ืืชืจ ื–ื” ื™ืฆื™ื’ ื‘ืคื ื™ื›ื ืกืจื˜ื•ื ื™ YouTube ื”ืžื•ืขื™ืœื™ื ืœืœื™ืžื•ื“ ืื ื’ืœื™ืช. ืชื•ื›ืœื• ืœืจืื•ืช ืฉื™ืขื•ืจื™ ืื ื’ืœื™ืช ื”ืžื•ืขื‘ืจื™ื ืขืœ ื™ื“ื™ ืžื•ืจื™ื ืžื”ืฉื•ืจื” ื”ืจืืฉื•ื ื” ืžืจื—ื‘ื™ ื”ืขื•ืœื. ืœื—ืฅ ืคืขืžื™ื™ื ืขืœ ื”ื›ืชื•ื‘ื™ื•ืช ื‘ืื ื’ืœื™ืช ื”ืžื•ืฆื’ื•ืช ื‘ื›ืœ ื“ืฃ ื•ื™ื“ืื• ื›ื“ื™ ืœื”ืคืขื™ืœ ืืช ื”ืกืจื˜ื•ืŸ ืžืฉื. ื”ื›ืชื•ื‘ื™ื•ืช ื’ื•ืœืœื•ืช ื‘ืกื ื›ืจื•ืŸ ืขื ื”ืคืขืœืช ื”ื•ื•ื™ื“ืื•. ืื ื™ืฉ ืœืš ื”ืขืจื•ืช ืื• ื‘ืงืฉื•ืช, ืื ื ืฆื•ืจ ืื™ืชื ื• ืงืฉืจ ื‘ืืžืฆืขื•ืช ื˜ื•ืคืก ื™ืฆื™ืจืช ืงืฉืจ ื–ื”.

https://forms.gle/WvT1wiN1qDtmnspy7