The AI-Powered Tools Supercharging Your Imagination | Bilawal Sidhu | TED

All right. Good afternoon, y’all.

Let's talk about blending reality and imagination.

But first, let's take a step back in time to 2001.

As an 11-year-old in India,

I became obsessed with computer graphics and visual effects.

Of course, at that age,

it meant making cheesy videos kind of like this.

But therein started a foundational theme in my life,

the quest to blend reality and imagination.

And that quest has stayed with me

and permeated across my decade-long career in tech,

working as a product manager at companies like Google

and as a content creator on platforms like YouTube and TikTok.

So today, let's deconstruct this quest to blend reality and imagination

and explore how it’s getting supercharged -- buzzword alert --

by artificial intelligence.

Let's start with the reality bit.

You probably heard about photogrammetry.

It's the art and science of measuring stuff in the real world

using photos and other sensors.

What required massive data centers and teams of experts in the 2000s

became increasingly democratized by the 2010s.

Then, of course, machine learning came along

and took things to a whole new level

with techniques like neural radiance fields, or NeRFs.

What you're seeing here is an AI model

creating a ground-up volumetric 3D representation

using 2D images alone.

But unlike older techniques for reality capture,

NeRFs do a really good job of encapsulating the sheer complexity

and nuance of reality.

The vibe, if you will.

Twelve months later, you can do all of this stuff

using the iPhone in your pocket,

using apps like Luma.

It's like 3D screenshots for the real world.

Capture anything once and reframe it infinitely in postproduction,

so you can start building that collection of spaces, places and objects

that you truly care about

and conjure them up in your future creations.

So that's the reality bit.

As NeRFs were popping off last year,

the AI summer was also in full effect,

with Midjourney, DALL-E 2, Stable Diffusion

all hitting the market around the same time.

But what I fell in love with was inpainting.

This technique allows you to take existing imagery

and augment it with whatever you like,

and the results are photorealistically fantastic.

It blew my mind

because stuff that would have taken me like three hours in classical workflows

I could pull off in just three minutes.

But I wanted more.

Enter ControlNet,

a game-changing technique by Stanford researchers

that allows you to use various input conditions

to guide and control the AI image generation process.

So in my case, I could take the depth information

and the texture detail from my 3D scans

and use it to literally reskin reality.

Now, this isn't just cool video.

There’s a lot of useful use cases, too.

For example, in this case I'm taking a 3D scan of my parents' drawing room,

as my mother likes to call it,

and reskinning it to different styles of Indian decor

and doing so while respecting the spatial context and the layout

of the interior space.

If you squint, I'm sure you can see

how this is going to transform architecture

and interior design forever.

You could take that 2016 scan of a Buddha statue

and reskin it to be gloriously golden

while pulling off these impossible camera moves

you just couldn't do any other way.

Or you could take that vacation footage from your trip to Tokyo

and bring these cherry blossoms to life in a whole new way.

And let me tell you, cherry blossoms look really good during the day,

but they look even better at night.

Oh, my God. They sure are glowing.

It's almost like this dreamlike quality

where you can use AI to accentuate the best aspects of the real world.

Natural landscapes look just as beautiful.

Like this waterfall that could be on another planet.

But of course, you could go over the hills and far away

to the French Alps from another dimension.

But it's not just static scenes.

You can do this stuff with video, too.

I can't wait till this technology is running at 30 frames per second

because it's going to transform augmented reality and 3D rendering.

I mean, how soon until we're channel-surfing realities

layered on top of the real world?

Of course, just like reality capture got democratized,

all these tools from last year are getting even easier.

So instead of me spending hours

weaving together a bunch of different tools,

tools like Runway and Kaiber let you do exactly the same stuff

with just a couple clicks.

Want to go from day to night? No problemo.

Want to get that retro 90s aesthetic from "Full House"?

You can do that too.

But it goes beyond reality capture.

Companies like Wonder Dynamics

are turning video into this immaculate form of performance capture

so you can embody fantastical creatures using the phone in your pocket.

This is stuff that James Cameron only dreamt about in the 2000s.

And now you could do it with your iPhone? That’s absolutely wild to me.

So when I look back at the past two decades

and this ill-tailored tapestry of tools that I've had to learn,

I feel a sense of optimism for what lies ahead

for the next generation of creators.

The 11-year-olds of today

don't have to worry about all of that crap.

All they need to do is have a creative vision

and a knack for working in concert with these AI models,

these AI models that are truly a distillation

of human knowledge and creativity.

And that's a future I'm excited about,

a future where you can blend reality and imagination

with your trusty AI copilot.

Thank you very much.

(Applause)