What if You Could Sing in Your Favorite Musician's Voice? | Holly Herndon | TED

(Overlapping voices sing)

(Singing ends)

(Applause)

Hi, my name is Holly, and I’m an artist.

That was my voice,

but I didn’t sing that clip.

I trained an AI on my own voice,

and now she can sing anything in multiple languages.

Her name is Holly+,

and you just heard her perform “El Cant de la Sibilla,”

which is a traditional song arranged by Maria Arnal in Catalan.

Not a language that I speak

and not a vocal tradition that I've trained in.

Those melismatic runs are really difficult to hit.

To show you Holly+'s full range, I'll also play an example in German.

This is "Mack the Knife" by Brecht.

(Singing in German)

(Music ends)

Of course Holly+ can -- thank you.

(Applause)

Of course, Holly+ can also perform my own music,

so here's an excerpt from my own song, "Frontier."

(Singing "Frontier" in overlapping voices)

(Singing ends)

Holly+ uses a process called timbre transfer,

where the timbre, or sound quality of one sound,

may be mapped onto the performance of another.

Timbre transfer is done by creating a machine-learning model of a sound.

In this case, it's my voice.

So I recorded a wide variety of phrases in my entire vocal range.

To be clear, this version of Holly+ is reading notes from a score.

So I'll also play you an example of her singing in Spanish,

“Bésame Mucho” by Velázquez.

(Music: "Bésame Mucho")

(Music ends)

Thank you.

(Applause)

I can't speak Spanish, by the way, so.

Teaching an AI the sonic properties of one sound

in order to generate an entirely new sound

is what I like to call "spawning."

Spawning is what allows Holly+ to create a wide range of vocals

that I didn't sing

from a set of recorded phrases that I did sing.

I like to think of spawning as a kind of 21st-century corollary

to the musical tradition of sampling,

which had a really big impact on both music and intellectual property.

But I think spawning is far more exciting and potentially really weird.

(Laughter)

So, for example, with sampling,

usually you copy and remix a recording by someone else to create something new.

But with spawning, you can perform as someone else

based on trained information about them.

And as an artist, this is making me rethink my own past work

as not only my archive

but potentially also I myself could become reanimated with AI.

This also opens up the question

of how we deal with a collective human archive

if we can reanimate old media.

It opens up really big ethical

and intellectual property questions

that require entirely new conceptual and legal frameworks.

One way that I like to think about intellectual property

I call identity play.

So rather than limiting the use of my voice,

I'm creating instruments to allow as many people as possible

to create music with me, and even as me.

That's why I've made versions of Holly+ freely available

for anyone to use online.

If I can allow people to play with my IP,

my digital identity, my intellectual property,

what might they come up with?

Could someone else go on tour as me, with my permission?

Could I be in a thousand different bands in multiple languages?

And what would that even sound like?

To be clear, musicians have been taught to be really protective over our IP,

and that's for really good reason.

But I'm trying to think of ways to make this new capability mutually beneficial.

So to allow people to use my voice

but still to maintain the ability to approve certain derivative works.

So I invite you to consider, if given the opportunity,

who would you like to perform through?

And can you imagine someone else performing you?

With that in mind,

I'd like to invite the incredible musician Pher to the stage.

(Applause)

So today, Pher will be performing his own song, "Murky."

And with one microphone you'll hear Pher's beautiful, natural voice.

(Pher vocalizing)

(Audience cheers)

And with this microphone,

you'll hear a live version of Holly+ developed with Voctro Labs.

(Pher vocalizing in Holly+ voice)

(Laughter and applause)

Take it away Pher.

(Pher singing "Murky")

It gets so murky

Loving what you know

But I, I keep on swimming,

I'm swimming

Further up the road

See, the problem is,

I don't know what to say

When you come around

Acting this way

Oh oh

And, yes, the truth is

I show you every day

Cause you love to stay

(Applause)

Living in all the pain

And it gets so murky

Loving what you know

But I'll keep on swimming

Further up the road

And I've been calling out

Calling out your name

Deep down in my sleep

And I even memorized your face

So ain't no leaving' me

And I just handle my biz

And keep the train moving on

And you know what it is

Is just you that

I'm choosing

Cause it gets so murky

Loving what you know

And I'll

I'll keep on swimming

Further up the road

And it gets so murky

Loving what you know

And I’ll,

I'll keep on swimming

Further up the road

Swimming

I'm swimming

Further up the road

And I'll keep on swimming

Swimming

Further up the road

(Vocalizing)

(Cheers and applause)

Holly Herndon: Thank you.

Thank you.