John Walker: Re-creating great performances

Ninety-nine percent of us have the dream of listeners.

Not being the musicians -- the listeners, right?

And we crave one thing,

even though we kind of don't know it all the time.

We crave to be in the room with the musician

the day it was recorded, the day it was played.

And we go to live concerts, and we get that as much as we can.

But then we listen to the other 99 percent of our stuff recorded.

And it turns out the further back you go in history,

the little rougher it sounds.

And so we said, there's a solution to this.

Let's separate the performance, as a thing,

out from the recording, which was how it was made.

You know, the thing with microphones in the room and all that day.

But the performance itself was how the musicians worked their fingers,

and what instruments they were using.

And it's the data hidden inside the recording.

In order to do this, it's a lot of hardware and software

that runs in a very high resolution.

And Yamaha makes an incredible thing called the Disklavier Pro

that looks like a nice grand piano there.

And you probably didn't realize it's going to do all these things --

but full of solenoids, and fiber optics, and computers

and all this kind of stuff. The highest resolution out of Japan.

And this just didn't work until we could cross this line that says high-definition.

And we were able to cross this line, called the uncanny valley,

in terms of -- artificial intelligence terms.

We have a process where we, you know,

kind of put it into the computer and digitize it, and then a whole lot of analysis.

And we look at every single note,

and all the attributes of those notes:

how hard they were struck, and how they were held down,

and how you move the fingers.

So we had to develop a whole new science of how you move your fingers.

And, you know, it's a thing your piano teacher teaches you,

but we never had a science behind these kinds of things.

I'm going to start with Glenn Gould.

He died 25 years ago this year, and was born 75 years ago this year.

Was a beloved pianist, maybe the great cult pianist of the twentieth century.

He just got tired of being in front of an audience,

and felt like -- a performing monkey was, in fact, his term.

So he stepped back, and did nothing but the crafting of his work.

And Gould's specialty was playing Bach.

His maybe most famous recording

was something called "The Goldberg Variations."

Bach only wrote themes and variations one time.

He wrote some early pieces,

but late in his life, in his mature period,

he said, "Here's a theme -- 30 variations."

In fact, the theme isn't even the melody, it's the bass line.

And Gould recorded it in two major recordings that you may know about,

one in mono, and one in stereo.

And the one in mono, by the way, he used the pedal,

and as he got older, he said, "No, no, wait a minute.

I'm going to get very scientific about this, and not use the pedal."

What I'd like you to hear live is the 1955 version,

and we'll play the first couple pieces of it.

Glenn Gould, 1955.

(Music)

How about that?

(Applause)

So let me tell you a little bit how this was done.

First of all, let me get you to the end step.

This is -- we have a fairly complex process

that, you know, software and musicians and so on,

but when we're all done, we know that the ear is the final arbiter.

We can play the original in one ear, and a new recording in the other.

So I'm going to do this for you right now, what you just heard.

And in the right speaker is going to be the original recording,

and the left speaker is going to be the new recording,

actually of an instrument just like that one,

and I'm going to play them together at the same time.

(Music)

That's the original. [Unclear] That's the two together.

(Music)

Before "Jurassic Park,"

there was no science for how skin hung off of muscle, right?

So, in the video world,

we've been able to invent, in our lifetimes, natural behavior.

And this is kind of another example

of putting a science behind natural behavior.

And then you heard the original.

Ultimately, I started with the experience.

And the experience is: I want to be in the room and hear the musicians.

Lots of you can afford to buy one of these.

But, if not, there is now high-definition surround sound.

And I got to tell you, if you haven't heard high-definition surround,

go down to your audio dealer, your audiophile dealer.

It's so involving compared to regular stereo.

But if you don't have that, maybe you can listen on your headphones.

And so on the same disk we have five recordings --

Sony has five recordings.

And you could listen in headphones

with this thing called binaural recording.

And it's a dummy head that sits in front of the instrument,

and it's got microphones where the ears are.

And when you put on headphones, and you listen to this,

you're inside of Glenn Gould's body.

And it is a chuckle until, you know, the musicians,

who are musicians who play the piano, listen to this, say,

"I can't believe it! It's just what it's like to play the piano."

Except now you're inside Glenn Gould's body playing the piano,

and it feels like your fingers are making the decisions

and moving through the whole process.

It's a game changer.

Here's now something we know in spectacular quality.

The whole process is very sensitive to temperature and humidity.

What you heard today was not perfect.

It's an amalgam of wood, and cast iron, and felt,

and steel strings, and all these,

and they're all amazingly sensitive to temperature and humidity.

So when you go into the recording session,

you get to stop after every piece and rebuild the piano if you need to.

There's the whole action there, sitting, kind of, on the side,

and the dummy head and our recording engineers

standing around while we rebuild the piano.

Without putting dates next to these things,

step-by-step music will be turned into data,

like every field that's occurred in the past 35 or 40 years.

Audio has come very late to this game --

I'm not talking about digitizing, and bits, and re-mastering.

I'm talking about turn it into the data that it was made from,

which is how it was performed.

And audio came very late because our ears are so hard to fool --

they're high-resolution, and they're wired straight to our emotions,

and you can't trick them very easily.

Your eyes are pretty happy with some color and movement, you know.

All right, there's this episode of "Star Trek."

(Laughter)

I get it -- it was all just laid in for me yesterday there.

The episode of "Star Trek" for me was James Daly played Methuselah --

remember this one?

And at some point he's dancing with his --

and I won't ruin the episode for you, from 1967.

Right, do you know where I'm going?

And Nimoy, I'm sorry, Spock sits down at the piano,

and he starts playing this Brahms waltz, and they all dance to it.

And then Spock turns round, he goes,

"James, I know all of the Brahms waltzes,

and I don't believe this is one of them in the category."

That's where I'm at.

I want to hear the waltzes Brahms didn't write.

I want to hear the pieces that Horowitz didn't play.

But I believe we're on a path now, when we get to data,

that we can distill styles, and templates, and formulas, and all these kinds of things,

again, that you've seen happen in the computer graphics world.

It's now coming in this world.

The transition will be this one.

It says right now, we think music is notes and how they're played.

And I believe this is coming.

Because what you've just heard was a computer playing data --

no Glenn Gould in the room.

But yet, it was human.

And I believe you'll get to the next step, the real dream of listeners.

Every time you listen to a recording today,

every time you take out your iPod and whatever,

every time you listen to it, it's the same thing -- it's frozen.

Wouldn't it be cool if every time you listened, it could be different?

This morning, you're sadder, you want to hear your song,

the same song, played sadder than you did yesterday.

You want to hear it played by different musicians.

You want to hear it in different rooms and whatever.

We've seen all these "Star Treks," and they're all holodeck episodes as well.

Every time I listen to that, I get goose bumps.

It's so amazing, it's so exciting.

Every time I listen to that recording it's like, "Oh my God,

I can't believe I'm in the same room. I can't believe this is happening."

It's a way better experience

than whatever you're used to listening to, in whatever form.

And lastly, I will wrap up with one minute of Art Tatum.

So I've really overshot my budget here.

We made a new recording of him

playing in the Shrine Auditorium in September.

It was a concert he recorded in the Shrine Auditorium in 1949.

And I've got to tell you, we have this lab

where we build and measure everything, back in Raleigh, North Carolina,

and we flew out to Los Angeles.

And as the president of the company,

I didn't feel real comfortable about where we were.

That's a real uncomfortable feeling,

when all the equipment's come out and a whole Sony team,

and people are going to be sitting there in the audience.

And we put the piano on the sweet spot of the stage in the Shrine,

which has not changed since 1949, still seats 6,000 people.

And on the sweet spot on the stage, Tatum starts playing ...

and every note, every beat, every slur, every accent, every pedal

was perfect, because he played it for that room on that day.

And we captured all that data all over again.

And I want you to hear that right now.

And fortunately, it's right in here.

This is an encore he used to do.

It's one minute long.

It's an Irish jig, and I want you to hear his humor.

(Music)

(Applause)

And that's just what the live audience did.

(Applause)

So thank you very much, Michael, thank you for the opportunity.