Rupal Patel: Synthetic voices, as unique as fingerprints

Translator: Aida Musai Reviewer: Helena Bedalli

I'd like to talk today

Sot dua t'ju flas

about a powerful and fundamental aspect

per nje aspekt shume te rendesishem

of who we are: our voice.

te identitetit tone: zerin.

Each one of us has a unique voiceprint

Cdonjeri prej nesh ka nje ze unik

that reflects our age, our size,

qe pasqyron moshen tone, dimensionet tona,

even our lifestyle and personality.

madje edhe stilin e jetes apo personalitetin.

In the words of the poet Longfellow,

Madje sipas poetit Longfellow,

"the human voice is the organ of the soul."

"zeri i njeriut eshte organ i shpirtit te tij".

As a speech scientist, I'm fascinated

Si shkencetare e zerit, mu me mahnit

by how the voice is produced,

menyra se si prodhohet zeri,

and I have an idea for how it can be engineered.

madje kam nje ide se si mund realizoj ate.

That's what I'd like to share with you.

Kjo eshte ajo qe dua te ndaj me ju.

I'm going to start by playing you a sample

Do filloj duke ju dhene te degjoni

of a voice that you may recognize.

nje ze qe ndoshta ju do t'a njihni.

(Recording) Stephen Hawking: "I would have thought

(regjistrim) Stephen Hawking: "Mendoja se

it was fairly obvious what I meant."

ajo qe doja te thoja ishte mjaft e qarte."

Rupal Patel: That was the voice

Rupal Patel: Ky ishte zeri i

of Professor Stephen Hawking.

profesor Stephen Hawking.

What you may not know is that same voice

Ajo qe ndoshta nuk dini eshte se po i njejti ze

may also be used by this little girl

mund t'i jepet kesaj vajze te vogel

who is unable to speak

e cila e ka te pamundur te flase

because of a neurological condition.

per shkak te nje problemi neurologjik.

In fact, all of these individuals

Ne te vertete, secili prej ketyre individeve

may be using the same voice,

mund te perdore te njejtin ze,

and that's because there's only a few options available.

dhe kjo ndodh sepse alternativat jane te pakta.

In the U.S. alone, there are 2.5 million Americans

Vetem ne SHBA ka 2.5 milione amerikane

who are unable to speak,

qe nuk mund te flasin,

and many of whom use computerized devices

shumica prej te cileve perdorin pajisje te kompjuterizuara

to communicate.

per te komunikuar.

Now that's millions of people worldwide

Ne bote ka miliona njerez

who are using generic voices,

qe perdorin zerat e zakonshem,

including Professor Hawking,

duke perfshire ketu profesor Hawking,

who uses an American-accented voice.

i cili perdor nje ze me theks amerikan.

This lack of individuation of the synthetic voice

Mungesa e personalizimit te zerave sintetike

really hit home

me zgjoi interes

when I was at an assistive technology conference

kur mora pjese ne nje konference mbi teknologjite ndihmuese

a few years ago,

disa vite me pare,

and I recall walking into an exhibit hall

dhe mbaj mend se duke ecur ne holl

and seeing a little girl and a grown man

pashe nje vajze te vogel dhe nje burre

having a conversation using their devices,

te cilet po bisedonin duke perdorur pajisjet e tyre,

different devices, but the same voice.

dy pajisje te ndryshme qe leshonin te njejtin ze.

And I looked around and I saw this happening

Pashe perreth dhe kuptova se kjo po ndodhte

all around me, literally hundreds of individuals

kudo, qindra individe

using a handful of voices,

qe perdornin nje numer te vogel zerash,

voices that didn't fit their bodies

zera qe nuk i pershtateshin trupave

or their personalities.

apo personaliteteve te tyre.

We wouldn't dream of fitting a little girl

Ne nuk do te donim ti jepnim nje vajze te vogel

with the prosthetic limb of a grown man.

krahun proteze te nje burri te rritur.

So why then the same prosthetic voice?

Atehere pse duhet te kene ata te njejtin ze sintetik?

It really struck me,

Kjo gje me goditi fort,

and I wanted to do something about this.

dhe doja te beja dicka ne lidhje me kete.

I'm going to play you now a sample

Tani do t'ju tregoj nje rregjistrim

of someone who has, two people actually,

te nje personi, ne fakt jane dy persona,

who have severe speech disorders.

me crregullime te renda ne te folur.

I want you to take a listen to how they sound.

Dua qe te degjoni tingullin e zerit te tyre.

They're saying the same utterance.

Ata thone te njejten gje.

(First voice)

(zeri i pare)

(Second voice)

(zeri i dyte)

You probably didn't understand what they said,

Ndoshta nuk arritet te kuptonit se c'thone,

but I hope that you heard

por shpresoj te keni dalluar

their unique vocal identities.

identitetin unik te zerit te tyre.

So what I wanted to do next is,

Keshtu, ajo qe vendosa te beja

I wanted to find out how we could harness

ishte te kuptoja se si mund t'i perdorja

these residual vocal abilities

keto mbetje te zerit

and build a technology

per te krijuar nje teknologji

that could be customized for them,

qe mund te personalizohej per ta,

voices that could be customized for them.

zera qe mund te personalizoheshin per ta.

So I reached out to my collaborator, Tim Bunnell.

Ja pse iu drejtova bashkepunetorit tim, Tim Bunnell.

Dr. Bunnell is an expert in speech synthesis,

Dr. Bunnell eshte ekspert ne sintetizimin e zerit,

and what he'd been doing is building

dhe ai ka punuar per te krijuar

personalized voices for people

zera te personalizuar

by putting together

duke punuar mbi

pre-recorded samples of their voice

rregjistrime te meparshme te zerit te tyre

and reconstructing a voice for them.

dhe duke ndertuar per ta nje ze te ri.

These are people who had lost their voice

Keta jane njerez qe e kane humbur zerin

later in life.

me vone ne jeten e tyre.

We didn't have the luxury

Por ne nuk e kishim luksin

of pre-recorded samples of speech

e rregjistrimeve ekzistuese

for those born with speech disorder.

per ata qe kishin lindur me probleme ne te follur.

But I thought, there had to be a way

Por une mendova se duhet te gjendej nje menyre

to reverse engineer a voice

per te rindertuar nje ze

from whatever little is left over.

nga ato pak tinguj qe mbeteshin prej tij.

So we decided to do exactly that.

Keshtu, ne vendosem te benim pikerisht kete.

We set out with a little bit of funding from the National Science Foundation,

Me shume pak fonde nga Fondacioni Kombetar i Shkences,

to create custom-crafted voices that captured

ne filluam te krijonim zera te vecante qe percillnin

their unique vocal identities.

identitetin vokal unik te tyre.

We call this project VocaliD, or vocal I.D.,

E kemi quajtur kete projekt VocaliD, ose vocal ID,

for vocal identity.

qe do te thote identitet vokal.

Now before I get into the details of how

Para se te hyj ne detajet mbi

the voice is made and let you listen to it,

menyren se si eshte i ndertuar zeri dhe para se tjua tregoj ate,

I need to give you a real quick speech science lesson. Okay?

Me duhet t'ju jap nje mesim shume te shkurter shkencor mbi zerin. Mire?

So first, we know that the voice is changing

Se pari, ne e dime se zeri ndryshon

dramatically over the course of development.

jashtezakonisht shume gjate rritjes.

Children sound different from teens

Femijet kane ze tjeter nga adoleshentet,

who sound different from adults.

te cilet tingellojne ndryshe nga te rriturit.

We've all experienced this.

Kete e kemi vene re qe te gjithe.

Fact number two is that speech

E dhena numer dy eshte se e folura

is a combination of the source,

vjen si kombinim i burimit,

which is the vibrations generated by your voice box,

pra vibrimeve qe gjenerohen nga kutia juaj e zerit,

which are then pushed through

te cilat me pas shtyhen drejt

the rest of the vocal tract.

pjeses se mbetur te traktit zanor.

These are the chambers of your head and neck

Ato qe dridhen jane dhomat e kokes

that vibrate,

dhe qafes tuaj,

and they actually filter that source sound

dhe ato e filtrojne ate tingull fillestar

to produce consonants and vowels.

per te krijuar bashtingellore dhe zanore.

So the combination of source and filter

Pra eshte fale kombinimit te burimit dhe filtrave

is how we produce speech.

qe prodhohet zeri.

And that happens in one individual.

Kjo ndodh brenda nje individi te vetem.

Now I told you earlier that I'd spent

Ju thashe me pare se une kam kaluar

a good part of my career

pjesen me te madhe te karrieres

understanding and studying

duke kuptuar dhe studiuar

the source characteristics of people

karakteristikat e burimit tek njerez

with severe speech disorder,

me crregullime te medha ne te folur,

and what I've found

dhe ajo qe zbulova ishte se,

is that even though their filters were impaired,

edhe pse filtrat e tyre ishin difektoze,

they were able to modulate their source:

ata ishin ne gjendje te modifikonin burimin:

the pitch, the loudness, the tempo of their voice.

lartesine, fortesine, dhe ritmin e zerit te tyre.

These are called prosody, and I've been documenting for years

Keto quhen prozodi, dhe prej vitesh une kam dokumentuar

that the prosodic abilities of these individuals

se si aftesite prozodike te ketyre njerezve

are preserved.

ruhen gjate kohes.

So when I realized that those same cues

Kur kuptova se po te njejtat elemente

are also important for speaker identity,

jane te rendesishme per identitetin zanor,

I had this idea.

me erdhi nje ide.

Why don't we take the source

Pse nuk marrim burimin

from the person we want the voice to sound like,

nga personi te cilin duam t'i ngjaje zeri,

because it's preserved,

duke qene se nuk ka demtime,

and borrow the filter

dhe marrim hua filtrin

from someone about the same age and size,

e nje personi me te njejten moshe dhe peshe,

because they can articulate speech,

duke qene se mund te artikulojne,

and then mix them?

dhe pastaj t'i perziejme ato?

Because when we mix them,

Sepse kur i perziejme ato,

we can get a voice that's as clear

na jepet nje ze po aq i qarte

as our surrogate talker --

sa ai i folesit surrogat --

that's the person we borrowed the filter from—

ky eshte personi te cilit i morrem filtrat hua --

and is similar in identity to our target talker.

dhe i ngjan identitetit te folesit tone te synuar.

It's that simple.

Kaq e thjeshte.

That's the science behind what we're doing.

Kjo eshte shkenca qe fshihet pas asaj qe po bejme.

So once you have that in mind,

Duke pasur parasysh kete,

how do you go about building this voice?

si ndertohet zeri?

Well, you have to find someone

Epo, si fillim duhet te gjesh dike

who is willing to be a surrogate.

qe eshte i gatshem te behet surrogato.

It's not such an ominous thing.

Nuk eshte aq veprim i mbrapshte.

Being a surrogate donor

Te behesh dhurues surrogato

only requires you to say a few hundred

do te thote thjesht te thuash nga disa qindra

to a few thousand utterances.

ne disa mijera fjale.

The process goes something like this.

Procesi zhvillohet pak a shume keshtu.

(Video) Voice: Things happen in pairs.

(video) Zeri: Gjerat behen ne cift.

I love to sleep.

Mua me pelqen te fle.

The sky is blue without clouds.

Qielli eshte blu dhe pa re.

RP: Now she's going to go on like this

RP: Ajo do te vazhdoje me kete ritem

for about three to four hours,

per tre apo kater ore,

and the idea is not for her to say everything

dhe ideja nuk eshte qe ajo te thote gjithcka

that the target is going to want to say,

qe personi do doje te thoje,

but the idea is to cover all the different combinations

por qe te mbuloje te gjitha kombinimet e mundshme

of the sounds that occur in the language.

te tingullit qe vihen re ne nje gjuhe.

The more speech you have,

Sa me i gjate te jete ky rregjistrim,

the better sounding voice you're going to have.

aq me mire do te tingelloje zeri i personit marres.

Once you have those recordings,

Pasi marrim keto rregjistrime,

what we need to do

ajo qe ne bejme

is we have to parse these recordings

eshte te ndajme keto rregjistrime

into little snippets of speech,

ne fragmente te shkurtra,

one- or two-sound combinations,

ne kombinime me nje apo dy tinguj,

sometimes even whole words

ndonjehere edhe fjale te plota

that start populating a dataset or a database.

me te cilat nisim popullimin e nje baze te dhenash.

We're going to call this database a voice bank.

Kete baze te dhenash do t'a quajme banka e zerave.

Now the power of the voice bank

Fuqia e bankes se zerave

is that from this voice bank,

eshte se nga kjo banke zerash,

we can now say any new utterance,

ne mund te themi fjali te reja,

like, "I love chocolate" --

si per shembull "Mua me pelqejne cokollatat"...

everyone needs to be able to say that—

te gjithe duhet t'a kene mundesine per t'a thene kete--

fish through that database

ne peshkojme ne bazen e te dhenave

and find all the segments necessary

dhe gjejme segmentet e nevojshme per

to say that utterance.

te krijuar ate fjali.

(Video) Voice: I love chocolate.

(Video) Zeri: Mua me pelqejne cokollatat.

RP: So that's speech synthesis.

RP: Pra keshte sintetizohet e folura.

It's called concatenative synthesis, and that's what we're using.

Quhet sintetizim konkatenativ, dhe eshte ai qe ne perdorim.

That's not the novel part.

Por nuk eshte kjo risia.

What's novel is how we make it sound

Risia gjendet ne tingullin qe ne i japim

like this young woman.

si ne rastin e kesaj vajze te re.

This is Samantha.

Quhet Samanta.

I met her when she was nine,

Une e kam takuar ku ishte nente vjec,

and since then, my team and I

dhe qe atehere, skuadra ime dhe une

have been trying to build her a personalized voice.

kemi punuar per t'i ndertuar nje ze te personalizuar.

We first had to find a surrogate donor,

Ne fillim na u desh te gjenim nje dhurues surrogato,

and then we had to have Samantha

dhe pastaj duhet t'i kerkonim Samantes

produce some utterances.

te na jepte disa tinguj.

What she can produce are mostly vowel-like sounds,

Ajo mund te prodhoje vetem tinguj qe permbajne zanore,

but that's enough for us to extract

por kjo ishte e mjaftueshme qe ne te identifikonim

her source characteristics.

karakteristikat e burimit te saj.

What happens next is best described

Ajo qe ndodhi pas kesaj mund te pershkruhet

by my daughter's analogy. She's six.

me ane te analogjise se vajzes sime. Ajo eshte gjashte vjec.

She calls it mixing colors to paint voices.

Ajo kete e quan perzierje ngjyrash per te krijuar zera.

It's beautiful. It's exactly that.

Eshte e mrekullueshme. Ndodh pikerisht keshtu.

Samantha's voice is like a concentrated sample

Zeri i Samantes eshte si nje ekstrakt i perqendruar

of red food dye which we can infuse

i bojes ushqimore te kuqe qe e perziejme

into the recordings of her surrogate

me rregjistrimet e surrogatos se saj

to get a pink voice just like this.

per te arritur tek nje ze i tille.

(Video) Samantha: Aaaaaah.

(video) Samanta: Aaaaaaaa

RP: So now, Samantha can say this.

RP: Tani ajo mund te thoje kete.

(Video) Samantha: This voice is only for me.

(video) Samanta: Ky ze eshte vetem imi.

I can't wait to use my new voice with my friends.

Me zor pres t'a perdor zerin tim te ri me miqte.

RP: Thank you. (Applause)

RP: Faleminderit (duartrokitje)

I'll never forget the gentle smile

Nuk do t'a harroj kurre buzeqeshjen e embel

that spread across her face

qe celi ne fytyren e saj

when she heard that voice for the first time.

kur degjoi kete ze per here te pare.

Now there's millions of people

Ne bote ka miliona njerez

around the world like Samantha, millions,

si Samanta, miliona,

and we've only begun to scratch the surface.

dhe ne sapo kemi filluar te perballemi me problemin.

What we've done so far is we have

Deri tani neve kemi arritur te kemi

a few surrogate talkers from around the U.S.

disa foles surrogato ne SHBA

who have donated their voices,

te cilet kane dhuruar zerat e tyre,

and we have been using those

dhe ne filluam me to

to build our first few personalized voices.

dhe ndertuam vetem disa zera te personalizuar.

But there's so much more work to be done.

Por ka ende shume pune per te bere.

For Samantha, her surrogate

Ne rastin e Samantes surrogati

came from somewhere in the Midwest, a stranger

vinte nga Midwest, nje e huaj

who gave her the gift of voice.

e cila i dhuroi asaj zerin.

And as a scientist, I'm so excited

Si shkencetare, une jam shume e lumtur

to take this work out of the laboratory

qe t'a nxjerr kete pune jashte laboratorit,

and finally into the real world

me ne fund ne boten reale,

so it can have real-world impact.

ne menyre qe te kete nje impakt global.

What I want to share with you next

Dua te ndaj tani me ju

is how I envision taking this work

visionin tim per hapin e ardhshem

to that next level.

te kesaj pune.

I imagine a whole world of surrogate donors

Imagjinoj nje bote te mbushur me dhurues surrogato

from all walks of life, different sizes, different ages,

nga te gjitha fushat e jetes, me pesha dhe mosha te ndryshme,

coming together in this voice drive

te cilet bashkohen

to give people voices

per t'u dhuruar njerezve zera

that are as colorful as their personalities.

qe jane plot ngjyra si edhe personalitetet e tyre.

To do that as a first step,

Per t'a bere kete, hapi i pare

we've put together this website, VocaliD.org,

ishte krijimi i nje siti web VocaliD.org,

as a way to bring together those

si nje menyre per te mbledhur bashke

who want to join us as voice donors,

te gjithe ata qe duan te na bashkohen si dhurues zeri,

as expertise donors,

dhurues ekspertize,

in whatever way to make this vision a reality.

apo ne menyra te tjera qe do t'a shnderrojne kete vizion ne realitet.

They say that giving blood can save lives.

Thone se duke dhuruar gjak shpeton jete.

Well, giving your voice can change lives.

E pra, duke dhuruar zerin ndryshon jete.

All we need is a few hours of speech

Na duhen thjeshte disa ore fjalimi

from our surrogate talker,

nga folesi surrogato,

and as little as a vowel from our target talker,

dhe vetem nje zanore nga folesi i synuar,

to create a unique vocal identity.

ne menyre qe te krijojme nje identitet zanor unik.

So that's the science behind what we're doing.

Kjo pra eshte shkenca e fshehur pas asaj qe po bejme.

I want to end by circling back to the human side

Dua t'a mbyll duke u kthyer tek aspekti njerezor,

that is really the inspiration for this work.

i cili eshte frymezimi i vertete per kete pune.

About five years ago, we built our very first voice

Rreth pese vite me pare ne ndertuam zerin e pare

for a little boy named William.

per nje djale te quajtur William.

When his mom first heard this voice,

Kur nena e tij e degjoi zerin per here te pare,

she said, "This is what William

ajo tha, "Ky do te ishte zeri

would have sounded like

i Williamit nese ai

had he been able to speak."

do te mund te fliste."

And then I saw William typing a message

Pastaj pashe se Williami po shkruante nje mesazh

on his device.

ne aparatin e tij.

I wondered, what was he thinking?

Thashe, c'te jete duke menduar valle?

Imagine carrying around someone else's voice

Imagjinoni sikur te udhetoni me zerin e dikujt tjeter

for nine years

per nente vjet

and finally finding your own voice.

dhe pastaj te gjeni zerin tuaj.

Imagine that.

Imagjinojeni

This is what William said:

Kjo eshte ajo qe Williami tha:

"Never heard me before."

"Nuk e kisha degjuar asnjehere veten."

Thank you.

Faleminderit.

(Applause)

(duartrokitje)