Jared Ficklin: New ways to see music (with color! and fire!)

Translator: Joseph Geni Reviewer: Morton Bast

譯者: Chen-Han Hsiao 審譯者: jimi charlie

My passions

我的熱情

are music, technology and making things.

是音樂,科技,以及創造東西。

And it's the combination of these things

而且就是這些東西的組合

that has led me to the hobby of sound visualization,

促使我對聲音的圖像化產生興趣,

and, on occasion, has led me to play with fire.

而且,有時,促使我去玩火.

This is a Rubens' tube. It's one of many I've made over the years,

這是魯本斯管.近幾些年來我做的其中一個,

and I have one here tonight.

今天晚上呢我也帶來了一個.

It's about an 8-foot-long tube of metal,

這東西是一支大約8英尺長的金屬管,

it's got a hundred or so holes on top,

管上差不多有一百多個小孔,

on that side is the speaker, and here

那邊有個喇叭,而這邊呢

is some lab tubing, and it's connected to this tank

有一些實驗用的管子,而且是和這個裝滿丙烷

of propane.

的容器連接的.

So, let's fire it up and see what it does.

那麼,我們現在就把它點燃看看會發生什麼事.

So let's play a 550-herz frequency

嗯...我們先試試550赫茲的頻率吧

and watch what happens.

看看會有什麼效果.

(Frequency)

(頻率)

Thank you. (Applause)

謝謝.(觀衆鼓掌)

It's okay to applaud the laws of physics,

確實值得讚揚物理定律的美,

but essentially what's happening here

但是實際發生的是

-- (Laughter) --

(笑聲)

is the energy from the sound via the air and gas molecules

聲音的能量透過空氣和氣體分子

is influencing the combustion properties of propane,

來影響丙烷的燃燒性質,

creating a visible waveform,

因而產生一個可見的波形,

and we can see the alternating regions of compression

而且我們能看到這些一下子壓縮

and rarefaction that we call frequency,

又接著膨脹的變化區段,我們把它叫頻率,

and the height is showing us amplitude.

而這個高度告訴我們的是振幅.

So let's change the frequency of the sound,

好吧.那讓我們改變一下聲音的頻率,

and watch what happens to the fire.

然後再看看火會發生什麽變化

(Higher frequency)

(更高的頻率)

So every time we hit a resonant frequency we get a standing wave

所以每當我們調到共鳴頻率時,會有駐波產生

and that emergent sine curve of fire.

而那個像正弦函數的火焰就會出現.

So let's turn that off. We're indoors.

好吧我們還是把這關了.我們在室內呢.

Thank you. (Applause)

謝謝. (觀衆鼓掌)

I also have with me a flame table.

我今天還帶來了一個火焰桌.

It's very similar to a Rubens' tube, and it's also used

它跟魯本斯管挺像的,而且它還能用來

for visualizing the physical properties of sound,

將聲音的物理性質圖像化,

such as eigenmodes, so let's fire it up

比如說特徵模式,那我們把它點燃吧

and see what it does.

然後看看會發生什麽.

Ooh. (Laughter)

喔~(笑聲)

Okay. Now, while the table comes up to pressure,

好吧.現在,當這個桌子受到到壓力,

let me note here that the sound is not traveling

此時我注意到聲音(波形)不是以完美的直線在傳遞

in perfect lines. It's actually traveling in all directions,

實際上是在各個方向傳遞,

and the Rubens' tube's a little like bisecting those waves

而魯本斯管有點像是用一條線將這些波形分兩半

with a line, and the flame table's a little like

而這個火焰桌有點像

bisecting those waves with a plane,

用一個面將這些波分兩半,

and it can show a little more subtle complexity, which is why

而它可以呈現一些微妙的複雜性，這就是爲啥

I like to use it to watch Geoff Farina play guitar.

我喜歡用它去看Geoff Farina彈吉他.

(Music)

(音樂)

All right, so it's a delicate dance.

不錯,這是個曼妙的舞蹈.

If you watch closely — (Applause)

如果你看的近一點--(觀衆鼓掌).

If you watch closely, you may have seen

如果你仔細看,你可以發現

some of the eigenmodes, but also you may have seen

一些特徵模式,不過你還可以發現

that jazz music is better with fire.

爵士樂在火焰的陪襯下就感覺不錯.

Actually, a lot of things are better with fire in my world,

實際上,我的世界裡,好多東西在火焰的陪襯下就不錯,

but the fire's just a foundation.

不過火焰只不過是個陪襯.

It shows very well that eyes can hear,

可以將耳朵能聽見的顯示得很清楚,

and this is interesting to me because

我對這個很感興趣

technology allows us to present sound to the eyes

因爲科技允許我們去用眼睛來感受聲音

in ways that accentuate the strength of the eyes

從而加強我們眼睛接受聲音的敏銳度

for seeing sound, such as the removal of time.

比如時間的轉移.

So here, I'm using a rendering algorithm to paint

那麽這裏呢,我用透視演算法去畫出

the frequencies of the song "Smells Like Teen Spirit"

歌曲"Smell Like Teen Spirit"的頻率

in a way that the eyes can take them in

我們的眼睛因而可以將它們讀入

as a single visual impression, and the technique

成爲一種獨一無二的直覺印象,這技術

will also show the strengths of the visual cortex

也能印證視覺皮質對圖案辨認的能力

for pattern recognition.

也能印證視覺皮質對圖案辨認的能力

So if I show you another song off this album,

如果我給你這首專輯裏面另外一首歌,

and another, your eyes will easily pick out

再另外一首,你的眼睛會輕易地捕捉到

the use of repetition by the band Nirvana,

樂隊Nirvana(在音樂中)所使用的重複性,

and in the frequency distribution, the colors,

以及在頻率分布,顔色方面,

you can see the clean-dirty-clean sound

你可以發現這個一會兒清晰一會兒吵雜的聲音

that they are famous for,

那種讓他們成名的音樂特色,

and here is the entire album as a single visual impression,

這是個以獨特的直覺印象呈現的整張專輯,

and I think this impression is pretty powerful.

我認爲這種印象十分的強大.

At least, it's powerful enough that

至少,強大到

if I show you these four songs,

如果我給你看四首歌,

and I remind you that this is "Smells Like Teen Spirit,"

而且我提醒你這首是"Smells Like Teen Spirit",

you can probably correctly guess, without listening

在不聽音樂的情況下,你可能會猜到這首歌，

to any music at all, that the song

在不聽音樂的情況下,你可能會猜到這首歌，

a die hard Nirvana fan would enjoy is this song,

Nirvana的死忠歌迷所喜愛的這首歌,

"I'll Stick Around" by the Foo Fighters,

由Foo Fighters所製作的“I’will Stick Around”,

whose lead singer is Dave Grohl,

他們的主唱是Dave Grohl,

who was the drummer in Nirvana.

他是Nirvana的鼓手.

The songs are a little similar, but mostly

這些歌曲都有點相似, 不過主要是

I'm just interested in the idea that someday maybe

我覺得有一想法蠻有趣的,就是有一天我們買這首歌

we'll buy a song because we like the way it looks.

是因爲我們喜歡它顯現的方式

All right, now for some more sound data.

好吧, 再多呈現一點聲音數據,

This is data from a skate park,

這是從滑冰公園所擷取的數據,

and this is Mabel Davis skate park

而這是Mabel Davis滑冰公園

in Austin, Texas. (Skateboard sounds)

在德州的奧斯丁.(滑板的響聲)

And the sounds you're hearing came from eight

你聽到的聲音是從八個

microphones attached to obstacles around the park,

在公園附近障礙上的麥克風擷取來的,

and it sounds like chaos, but actually

聽起來有點混亂,不過實際上

all the tricks start with a very distinct slap,

所有的特技從一個明顯拍擊聲開始,

but successful tricks end with a pop,

不過成功的特技最後會砰一聲結束,

whereas unsuccessful tricks

而沒表演好的特技

more of a scratch and a tumble,

還會摻雜更多的刮傷和翻滾的聲音,

and tricks on the rail will ring out like a gong, and

而在欄杆上的特技會發出一個像是敲鑼的聲音,

voices occupy very unique frequencies in the skate park.

而這些聲音在這個公園都有著獨特的頻率,

So if we were to render these sounds visually,

所以,我們要是想把這些聲音圖像化,

we might end up with something like this.

我們應該要得到像這樣的東西.

This is all 40 minutes of the recording,

這是40分鐘的記錄,

and right away the algorithm tells us

從這演算法我們立即發現

a lot more tricks are missed than are made,

沒做成功的特技比成功的特技還要多,

and also a trick on the rails is a lot more likely

而且在這些欄杆上的特技很有可能會

to produce a cheer, and if you look really closely,

帶來更多的歡呼聲,如果你非常仔細看的話,

we can tease out traffic patterns.

我們可以找出關於(滑冰公園內)流量的圖案.

You see the skaters often trick in this direction. The obstacles are easier.

你會看到溜冰的人通常朝這個方向,(這方向的)障礙比較少.

And in the middle of the recording, the mics pick this up,

然後在這個記錄的中央, 麥克風把這個記錄下來,

but later in the recording, this kid shows up,

但是在這記錄的後面,這個小孩出現,

and he starts using a line at the top of the park

他在公園最高處沿著一條新的路線開始溜冰

to do some very advanced tricks on something

做一些難度高的特技

called the tall rail.

稱作高欄特技.

And it's fascinating. At this moment in time,

而這相當有看頭.在這個時候，

all the rest of the skaters turn their lines 90 degrees

其餘所有溜冰的人都把他們的路線轉了90度讓路給他.

to stay out of his way.

其餘所有溜冰的人都把他們的路線轉了90度讓路給他.

You see, there's a subtle etiquette in the skate park,

你可以看到,在這個公園裡有個約定俗成的禮儀

and it's led by key influencers,

是由頗具影響力的人主導的,

and they tend to be the kids who can do the best tricks,

而他們傾向是作高難度動作的小孩,

or wear red pants, and on this day the mics picked that up.

或者是穿紅褲子的,而這天麥克風就記錄下這現象.

All right, from skate physics to theoretical physics.

好吧,從溜冰物理到理論物理.

I'm a big fan of Stephen Hawking,

我是Stephen Hawking的大粉絲,

and I wanted to use all eight hours

我想用他在劍橋八個小時的講座來表示我對他的敬意

of his Cambridge lecture series to create an homage.

我想用他在劍橋八個小時的講座來表示我對他的敬意.

Now, in this series he's speaking with the aid of a computer,

現在呢,在這堂講座裡,他借助於電腦作演講，

which actually makes identifying the ends of sentences

這樣確實比較容易分辨句子是否結束

fairly easy. So I wrote a steering algorithm.

因此我寫了個操縱算法.

It listens to the lecture, and then it uses

它一邊聽講座,然後一邊利用

the amplitude of each word to move a point on the x-axis,

每個詞的振幅在x軸方向上來回移動點,

and it uses the inflection of sentences

它還利用句子的音調變化

to move a same point up and down on the y-axis.

針對同一點在Y軸方向上下移動.

And these trend lines, you can see, there's more questions

然後這些指明趨勢的線,你可以看到在物理世界中

than answers in the laws of physics,

未知比已知多,

and when we reach the end of a sentence,

每當我們聽完一個句子,

we place a star at that location.

我們就在(那個句子對應到的)那個地方放一顆星星.

So there's a lot of sentences, so a lot of stars,

所以呢很多句子就產生很多星星,

and after rendering all of the audio, this is what we get.

播完所有的語音檔後,我們就得到了這個

This is Stephen Hawking's universe.

這就是Stephen Hawking的宇宙.

(Applause)

(觀眾鼓掌)

It's all eight hours of the Cambridge lecture series

全部是在劍橋的八個小時的講座

taken in as a single visual impression,

以一個獨一無二的直覺印象呈現,

and I really like this image,

我相當喜歡這個圖案.

but a lot of people think it's fake.

不過很多人認為它是假的.

So I made a more interactive version,

所以我做了個互動版本,

and the way I did that is I used their position in time

而且我所使用的方式是利用他們在時間軸上出現的位置

in the lecture to place these stars into 3D space,

將這些星星放置在3D空間裡(對應位置),

and with some custom software and a Kinect,

同時利用一些客製化軟體和一個Kinect,

I can walk right into the lecture.

這樣我就能”走進”課堂中.

I'm going to wave through the Kinect here

我要握著這個Kinect 作揮手動作

and take control, and now I'm going to reach out

然後作一些操控,接著我伸手去

and I'm going to touch a star, and when I do,

抓一顆星星,當我這樣做的時候,

it will play the sentence

它會播出(這個星星所對應到的)這句話.

that generated that star.

它會播出(這個星星所對應到的)這句話.

Stephen Hawking: There is one, and only one, arrangement

Stephen Hawking:這裡有一個,獨一無二的(星星位置)分布

in which the pieces make a complete picture.

(星星所在的)各個位置構成一個完整的圖案.

Jared Ficklin: Thank you. (Applause)

Jared Ficklin:謝謝 (觀眾鼓掌)

There are 1,400 stars.

這裡面一共有1400顆星星

It's a really fun way to explore the lecture,

這是個很有意思的方式去探索這個講座,

and, I hope, a fitting homage.

我希望(這是表示)合適敬意的方式.

All right. Let me close with a work in progress.

好的.讓我們以接下來的作品來結束今天的演講.

I think, after 30 years, the opportunity exists

我認為,30年後,我們有可能

to create an enhanced version of closed captioning.

去創造沒字幕的特殊版本.

Now, we've all seen a lot of TEDTalks online,

現在,我們在網上可以看到好多TED演講,

so let's watch one now with the sound turned off

因此呢讓我們將聲音關掉去看一個吧

and the closed captioning turned on.

當然我們把字幕打開

There's no closed captioning for the TED theme song,

TED片頭曲是沒有字幕,

and we're missing it, but if you've watched enough of these,

因此我們經常錯過這部分,不過你已經看得夠多了,

you hear it in your mind's ear,

你可以在你心裡聽到這片頭曲,

and then applause starts.

接著呢開始聽到掌聲.

It usually begins here, and it grows and then it falls.

通常(掌聲)是這樣開始,先漸強,最後漸弱.

Sometimes you get a little star applause,

但有時候你僅會有一點的掌聲,

and then I think even Bill Gates takes a nervous breath,

而且我認為即使Bill Gates也緊張地深吸一口氣,

and the talk begins.

然後開始演講.

All right, so let's watch this clip again.

好吧,那麼就讓我們再看一次這個片段.

This time, I'm not going to talk at all.

這次,我不準備說話了.

There's still going to be no audio,

依然是沒有聲音,

but what I am going to do is I'm going to render the sound

不過這回我要做的就是讓這個聲音圖像化, 並顯示在銀幕的最底下

visually in real time at the bottom of the screen.

不過這回我要做的就是讓這個聲音圖像化, 並顯示在銀幕的最底下

So watch closely and see what your eyes can hear.

所以呢 更靠近一點看,看看你的眼睛能”聽”到什麼.

This is fairly amazing to me.

這個對我來說非常神奇

Even on the first view, your eyes will successfully

即使看第一次,你的眼睛也會成功的

pick out patterns, but on repeated views,

擷取一些圖案,不過在重複看之後,

your brain actually gets better

你的腦子將圖案轉化為信息能力方面會變得更好

at turning these patterns into information.

你的腦子將圖案轉化為信息能力方面會變得更好

You can get the tone and the timbre

你可以感覺到說話人的語氣和音色

and the pace of the speech,

以及演講的速度,

things that you can't get out of closed captioning.

這些是你從字幕中得不到的東西.

That famous scene in horror movies

恐怖電影中的經典畫面

where someone is walking up from behind

也就是當有人從後面走來時

is something you can see,

變成了你可以看到的東西,

and I believe this information would be something

因此我相信這個東西會成為

that is useful at times when the audio is turned off

當聲音被關掉的時候的一個有用的工具

or not heard at all, and I speculate that deaf audiences

或者根本無法聽到的時候,我還認為耳聾的觀眾

might actually even be better

會比看正常的聽眾更擅長看聲音

at seeing sound than hearing audiences.

會比看正常的聽眾更擅長看聲音

I don't know. It's a theory right now.

我也不太清楚.目前為這只是個理論.

Actually, it's all just an idea.

實際上,這只不過是個想法.

And let me end by saying that sound moves in all directions,

我結束前我說這樣一句話吧:聲音會向所有的方向移動,

and so do ideas.

想法也會。

Thank you. (Applause)

謝謝 (觀眾鼓掌)