Paul Debevec animates a photo-real digital face

譯者: Bill Hsiung 審譯者: Jing Yao

One of the biggest challenges in computer graphics

電腦繪圖的巨大挑戰之一，

has been being able to create a photo-real,

是如何製作一個如相片般真實的

digital human face.

數位人物面孔。

And one of the reasons it is so difficult is that, unlike aliens and dinosaurs,

不像外星人和恐龍，它之所以如此困難的原因是，

we look at human faces every day.

我們每天都會看到人的面孔，

They are very important to how we communicate with each other.

它們對我們非常重要，因為我們藉此來彼此溝通，

As a result, we're tuned in to the subtlest things

因此，我們會注意到那些通過電腦渲染出來，

that could possibly be wrong with a computer rendering,

不正確的、非常細微的東西，

in order to believe whether these things are realistic.

並用它們來分辨這些事物是否真實的。

And what I'm going to do in the next five minutes

在接下來的五分鐘裡面，

is take you through a process

我將帶領大家一起看一個過程，

where we tried to create a reasonably photo-realistic computer-generated face,

從我們嘗試去創造一個以電腦制作的幾近相片般真實的面孔，

using some computer graphics technology we've developed,

通過我們

and also some collaborators at a company called Image Metrics.

和一些來自 Image Metrics 的伙伴開發的一系列電腦繪圖技術。

And we're going to try to do a photo-real face

我們將嘗試製作

of an actress named Emily O'Brien, who is right there.

一位名叫 Emily O'Brien 的女演員的面孔，她就在那兒。

And that's actually a completely computer-generated rendering of her face.

那實際上是一個完全依據她的面孔，由電腦渲染產生的結果。

By the end of the talk, we're going to see it move.

在演講的最後，我們將會看到她動起來。

The way that we did this is we tried to start with Emily herself,

我們嘗試的方式是由 Emily 她本身開始，

who was gracious enough to come to our laboratory

她非常親切地來到我們的實驗室。

in Marina Del Rey, and sit for a session in Light Stage 5.

在瑪麗安德爾灣，並坐到了我們的設備「光台五號」(Light Stage 5) 當中。

This is a face-scanning sphere, with 156 white LEDs all around

這是一個臉部掃描球體，有 156 個白色的發光二極管，

that allow us to photograph her

能讓我們在一系列嚴格控制的照明條件下，

in a series of very controlled illumination conditions.

對她進行拍照。

And the lighting that we use these days looks something like this.

我們當時使用的照明方式看起來是這樣的。

We shoot all of these photographs in about three seconds.

我們在三秒鐘之內拍攝出這裡全部的照片。

And we basically capture enough information

我們基本上採集到了足夠的資訊，

with video projector patterns that drape over the contours of her face,

能夠通過圖像投影模擬出她的面部輪廓，

and different principle directions of light from the light stage,

以及在燈光舞台不同角度的照明條件下，

to figure out both the coarse-scale

來計算出她面孔粗略的

and the fine-scale detail of her face.

以及精細的面部細節模型。

If we zoom in on this photograph right here,

如果我們放大這張照片的這個地方，

we can see it's a really nice photograph to have of her,

我們可以很清晰地看到她臉部的細節，

because she is lit from absolutely everywhere at the same time

因為她的臉部同時從各個角度被完全地照亮了，

to get a nice image of her facial texture.

因此得到一份非常優質的她的臉部紋理的相片。

And in addition, we've actually used polarizers on all the lights --

再加上，我們實際上在所有燈光上應用了偏光鏡

just like polarized sunglasses can block

就如同偏光的太陽眼鏡可以阻擋

the glare off of the road,

馬路上的強烈反強光，

polarizers can block the shine off of the skin,

偏光鏡可以阻擋皮膚上出現的光澤，

so we don't get all those specular reflections to take this map.

所以在這張圖中，我們沒有採集到那些球面反射。

Now, if we turn the polarizers around just a little bit,

現在，如果我們稍微調整一下偏光鏡，

we can actually bring that specular reflection

我們就可以將球面反射

of the skin back in,

帶回到皮膚上面，

and you can see she looks kind of shiny and oily at this point.

並且，你會看到她的面部看起來有些閃耀和油光。

If you take the difference between these two images here,

如果你取得這兩張圖像的差別，

you can get an image lit from the entire sphere of light

你可以得到的一個被球形光線照亮的

of just the shine off of Emily's skin.

Emily 的皮膚的相片。光照效果

I don't think any photograph like this had ever been taken

我認為沒有任何人拍過像這樣的照片，

before we had done this.

我們是第一個這麼做的。

And this is very important light to capture,

採集這種光線是非常重要的，

because this is the light that reflects off the first surface of the skin.

因為這是皮膚最表層所反射出來的光線，

It doesn't get underneath the translucent

它不會跑到皮膚的透明層下面去，

layers of the skin and blur out.

然後造成模糊不清。

And, as a result, it's a very good cue

因此，它是一個

to the detailed shape of the skin-pore structure

用來獲得皮膚細緻毛孔結構

and all of the fine wrinkles that all of us have,

以及我們都有的皮膚全部的褶皺細節的很好線索，

the things that actually make us look like real humans.

實際上，是這些東西讓我們看起來像真實人類。

So, if we use information that comes off of this specular reflection,

如果，我們使用來自高光反射的資訊，

we can go from a traditional face scan

我們可以從傳統的臉部掃描開始，

that might have the gross contours of the face and the basic shape,

來得到臉部的粗略輪廓和基本形狀，

and augment it with information

再加上大量

that puts in all of that skin pore structure and fine wrinkles.

皮膚的毛孔結構和紋理特徵的資訊。

And, even more importantly,

更重要的是，

since this is a photometric process that only takes three seconds to capture,

因為這是基於照片建模的流程而我們只採集了三秒鐘的照片資料，

we can shoot Emily

我們可以用一下午的時間

in just part of an afternoon,

來拍愛蜜麗 (Emily)，

in many different facial poses and facial expressions.

拍攝不同的面部姿態和表情變化。

So, here you can see her moving her eyes around, moving her mouth around.

這裡，你可以看到她轉動眼睛，嘴巴的開合。

And these we're actually going to use to create a photo-real digital character.

這些，實際上都會被我們用來創造數位虛擬的寫實人物。

If you take a look at these scans that we have of Emily,

來看一下這些我們對愛蜜麗 (Emily) 進行的掃描，

you can see that the human face does an enormous amount of amazing things

你會看到我們人類臉部表情是千變萬化的，

as it goes into different facial expressions.

在不同的臉部表情變換之中，

You can see things. Not only the face shape changes,

你會看到，不僅僅是臉部形態發生變化，

but all sorts of different skin buckling and skin wrinkling occurs.

各種各樣的皮膚的弧線和皮膚上的褶皺也會出現。

You can see that the skin pore structure changes enormously

你可以看到皮膚毛孔結構也有著巨大的變化，

from stretched skin pores

從被拉伸的皮膚毛孔，

to the regular skin texture.

到普通的皮膚質地。

You can see the furrows in the brow and how the microstructure changes there.

你可以看到前額眉骨那裡細微的變化，

You can see muscles pulling down at flesh to bring her eyebrows down.

你可以看到在她皺眉時候肌肉的收縮。

Her muscles bulging in her forehead when she winces like that.

在她畏懼的時候，她前額的肌肉會這樣地膨脹。

In addition to this kind of high-resolution geometry,

對於這樣高分辨率的幾何體，

since it's all captured with cameras, we've got a great texture map to use for the face.

因為這些都通過照相機採集，我們有她面部表情足夠的貼圖。

And by looking at how the different color channels of the illumination,

通過顏色和亮度不同的頻道，

the red and the green and the blue,

紅綠藍色彩頻道，

diffuse the light differently,

受光照不同的散射，

we can come up with a way of shading the skin on the computer.

我們可以通過電腦總結出為皮膚添加陰影的方式。

Then, instead of looking like a plaster mannequin,

然後，不同於看起來如同石膏像的假人，

it actually looks like it's made out of living human flesh.

她看起來如同人類一樣鮮活。

And this is what we used

這就是我們

to give to the company Image Metrics

為 Image Metrics 公司

to create a rigged, digital version of Emily.

所製作的寫實的數位虛擬人物 Emily。

We're just seeing the coarse-scale geometry here.

這裡，我們看一下粗糙的幾何模型，

But they basically created a digital puppet of her,

但他們已經對她創建了一些基本的數據程式，

where you can pull on these various strings,

你可以拖曳這些不同的座標繩，

and it actually moves her face in ways that are

它實際上運動的方式就會依照

completely consistent with the scans that we took.

我們掃描出的數據那樣。

And, in addition to the coarse-scale geometry,

並且，對於這個基本模型，

they also used all of that detail

他們也使用了全部的細節

to create a set of what are called "displacement maps"

來創建一系列的「置換貼圖」，

that animate as well.

同樣可以對其進行動畫設置。

These are the displacement maps here.

這裡是一些置換貼圖，

And you can see those different wrinkles actually show up as she animates.

你可以看到不同的褶皺出現在她的動畫之中。

So the next process was then to animate her.

所以，接下來的步驟是對她進行動畫設置，

We actually used one of her own performances to provide the source data.

我們實際上應用她自身的表演作為源數據。

So, by analyzing this video with computer vision techniques,

通過計算機視覺技術來分析這些影片，

they were able to drive the facial rig

他們可以使用電腦合成的表演，

with the computer-generated performance.

來驅動表情的變化。

So what you're going to see now, after this,

所以，我們將看到，通過這一切處理，

is a completely photo-real digital face.

是一個完全寫實的數位化臉部。

We can turn the volume up a little bit if that's available.

我們可以把聲音開大一點，如果可以的話。

Emily: Image Metrics is a markerless, performance-driven animation company.

Emily: Image Metrics 是一家追求高品質的動畫公司。

We specialize in high-quality facial animation

我們致力於為電動和電影

for video games and films.

製作高品質的臉部表情動畫。

Image Metrics is a markerless, performance-driven animation company.

Image Metrics 是一家追求高品質動畫的公司。

We specialize in high quality facial animation

我們致力於為電動和電影

for video games and films.

製作高品質的臉部表情動畫。

Paul Debevec: So, if we break that down into layers, here's that diffuse component

保羅•德貝維奇：因此，如果我們把它分解到各個層，這是散射部分，

we saw in the first slide.

我們在第一張投影片裡面看到的。

Here is the specular component animating.

這是高光部分的動畫。

You can see all the wrinkles happening there.

你們可以看到各處皮膚褶皺的出現。

And there is the underlying wireframe mesh.

這是原始的線框結構層。

And that is Emily herself.

那是 Emily 她自己。

Now, where are we going with this here?

現在，我們接下來會做些什麼？

We've gone a little bit beyond Light Stage 5. This is Light Stage 6,

我們已經有了高於「光台五」的裝置設備。這是「光台六」。

and we're looking at taking this technology

我們期望將這一技術

and applying it to whole human bodies.

應用到人的全身。

This is Bruce Lawmen, one of our researchers in the group,

這是布魯斯•勞曼，我們研發團隊中的一位，

who graciously agreed to get captured running in the Light Stage.

非常樂意配合我們，採集他在「光台」中跑步的數據。

And let's take a look at a computer-generated version

接著讓我們看一下電腦合成的，

of Bruce, running in a new environment.

在一個全新的環境中跑步的，布魯斯的影像。

And thank you very much.

感謝大家。

(Applause)

（掌聲）