The Invisible Technology Behind AI's Biggest Breakthroughs

[cerebral music]

Hi, I'm Paul Rousseau,

vice president at TSMC North America.

TSMC is the company

that manufactures

the world's most advanced chips.

The ones that go into your phone,

the ones that answer your questions on AI,

the ones that might self-drive your car.

I have a PhD in applied physics,

and I spent 25 years at TSMC

watching how the things we do

get smaller and smaller,

while what they accomplish

gets bigger and bigger.

Today, I'm going to explain

how a chip works,

first to a kid, then to a college student,

then to someone who does this for a living.

Let's start small, fittingly.

[charming music]

Good morning, Elena.

How you doing today?

Good.

Yeah? How old are you?

I'm eight, and I just turned eight.

[Paul] You just turned eight. Happy birthday.

Thank you.

So you know what, Elena?

I have a little present for you today.

It's a postage stamp,

but before I give it to you,

you have to answer a riddle.

Okay, I love riddles.

What's smaller than a postage stamp,

but it's the smartest thing

people have ever built?

A robot.

A robot. A robot could be very smart.

Bigger than a stamp though, eh?

Oh yeah.

Elena, what did you play with

this morning?

I played with my tablet.

You played with your tablets.

You were this close

to touching the smartest thing

people have ever built.

Let me show you something, Elena.

This is a board,

and this is a type of board

you might find in your tablet.

And there's a bunch of chips on here.

So is it this one?

[Paul] Uh-huh.

On this one?

The crazy part is

a chip looks a lot like a city.

Have you ever been to a big city?

Yeah, like this one.

I'll show you a model.

This is a city.

Do you see it has streets?

Oh yeah, just like this one.

And buildings.

And the building has libraries

which store all of the information

the chip needs to know.

Have you been to the library before?

Yeah, we pick up cool books.

You pick up cool books, right?

Libraries is where

all the information is stored.

So the chip has its libraries.

It also has its buildings

where it does its computations.

It does all the math.

Oh.

And then it has messengers.

The cars are messenger cars.

They're little zaps of electricities

that go around carrying the messages.

And the streets are like the teeny,

teeny wires in the chip.

They're smaller than a human hair.

Wow.

And so when you hit the screen

on your tablet, it sends a message.

And what would happen, Elena,

if this city didn't have

any traffic lights?

Oh, I think that everything

will keep on going, and then we'll crash,

and then it'll be a little crazy.

It'll be a mess.

So this city has billions

of traffic lights,

and what they do is they control

the messenger car

so they go exactly from the right building

to the other building

carrying the right messages

and doing all the work.

Okay. Wow.

And that happens

billions of times per second.

So it's going from here to there.

So it's very, very busy.

Very, very busy. Yeah.

And that's how a chip works, Elena.

Wow.

So remember our board, Elena?

Yeah.

Take another look at it.

Does it look like a city?

It does, but very, very small.

Very, very small.

I can see the streets,

and they're smaller than a piece of hair.

Right.

Now imagine we take this whole board

and we shrink it to something like this.

That must be hard.

This is already tiny, tiny, tiny.

Really, really small.

It's really, really small.

So have you ever dreamed, Elena,

of creating a machine,

building a machine? Oh, yes.

Yeah?

What kind of machine would you build?

A robot.

[Paul] A robot. What kind of robot?

A robot that is pink,

and it can clean my room,

make me breakfast, and sort my socks.

So I want you to close your eyes

for a second, Elena.

Okay.

You can't see anything now,

right? No.

But you can still hear me

when you're thinking. Yeah.

Where are you thinking?

In my brain.

[Paul] In my brain.

So what does this robot need?

A brain.

A brain. And what's that called?

A chip.

Exactly. You got it.

Well, Elena, you solved my riddle.

I did? Wow.

You did. That was fun.

Good. You get to keep the stamp.

Oh, thank you.

And the chip.

Wow.

That way you can ask other people

the same riddle.

Well, first thing first,

I'm going to tell them

that it's a super chip

for the electronics,

not the chip that they eat.

Yeah, you want to make sure

they don't eat that chip.

Hell yeah.

But then you can teach them

what a real chip is.

Yeah.

It was a pleasure talking to you, Elena.

Thank you so much.

You're welcome.

[intriguing music]

Hey, Joe. Great to meet you.

Nice to meet you too, Paul.

So I hear you're a student

in electrical engineering.

Tell me a little bit

about what you've been studying.

I study analog integrated circuits

with a focus on semiconductors

and MOSFET specifically.

So Joe, I'm curious,

tell me a little bit

what you know about scaling,

how we've been on this trajectory

of scaling transistors

and semiconductors for many years.

I know about Moore's Law,

the idea that every two years, I think,

we basically can double

the amount of transistors

that we can fit into a given area.

So Joe, let's demonstrate.

I'm going to give you a piece of paper,

and I want you to fold it in half.

There you go.

That's one generation of Moore's Law.

That wasn't that hard, was it?

Yeah, it was pretty easy. [both laughing]

Except we've been doing this

for over 30 generations.

Okay.

So keep folding, Joe.

I don't think I'm going to be able

to fold this 30 times,

but I'll give it a shot.

Think about the smartphone I have.

It has the power of a supercomputer

from the mid-90s.

And not only that,

that supercomputer

would have cost me $50 million.

I can get this for about $1,000.

That's the power of Moore's Law.

That's amazing.

It is amazing.

Joe, you're slowing down.

What's going on?

So I'm running out of the strength.

I mean, it's getting too thick

to fold basically. It's getting thick.

As you keep scaling,

it gets harder and harder.

For example, the name of our latest technology

in production is two nanometers.

That's basically 10 atoms across.

Wow.

I want to show you a wafer

that we built right here in the US.

This is a 300-millimeter wafer,

and it's on four nanometers.

And if you look at it carefully,

you can see the individual chips in there.

So what do you think about the thickness

versus the size of this thing?

Well, you said it's 300 millimeters

in diameter. Yeah.

[Joe] It looks to me

about one millimeter thick maybe.

Right.

That's the whole point

is a lot of silicon is two-dimensional.

The actual thickness we use for circuits

is actually less than one.

That one millimeter is there for structure.

Right.

In a real circuit,

it's actually thinner

than this piece of paper.

I'm going to fold this and think about

if I had a big chip before,

what happens when I stack it, right?

So I was trying to communicate

from this end of the paper

to this end of the paper.

It's pretty long distance.

Yeah.

What happens now

when I fold it together?

The distance of that communication,

that interconnect is much smaller.

And so that means it's much faster,

and it's much more power efficient.

Right.

That's another way

to keep scaling the compute of our system

by just adding more and more layers

in the third dimension.

That's amazing.

So when you think

about everything we talked about, Joe,

what excites you the most

about the future?

Well, as an aspiring engineer,

I think the idea that we're really running

up against the laws of physics

is very exciting.

Finding ways around kind of the universe

that we live in,

in order to make better machines

that can enhance our lives

is very interesting,

and I'm really excited

to be entering this field.

Really happy to have you join the club

because we need a lot

of smart people like you.

Thank you, Joe.

Thank you, Paul. Pleasure.

[dynamic music]

Welcome, Stephanie.

How are you doing today?

I'm doing well. And you?

Good, thank you.

So you're a professor at Penn State.

Can you describe what you do there?

Yeah, so I'm a professor

of material science and engineering,

and we make new materials

atomic layer by atomic layer.

I know you do a lot of your work

on 2D materials.

That's continuing the scaling.

Can you talk a little bit about that?

Absolutely.

Right now, your transistors are what?

10 atoms thick of that order?

[Paul] Exactly.

It turns out

for a material like silicon,

there's a limit to how small

you can make a silicon transistor.

And if we want to keep packing

more and more transistors on a chip

to make our chips

more and more powerful,

we need to look at new materials.

And one material class

that we can think about

are 2D materials.

These are materials that are literally

one or two layers of atoms thick.

So you can imagine if you go

from a transistor made of 10 atoms

to a transistor made of only one atom,

now maybe you can pack 10 times

as many transistors on your chip,

and now your chip can be

10 times as powerful.

So there's a lot of great stuff

that's happening on new material research,

but what about packaging?

How do you see things shifting

in packaging these days?

Packaging is an interesting topic.

If you're able to,

instead of making one big chip

that has the compute here

and the memory and whatever,

if you can make

all these different little chiplets,

you can stack them up,

you can package them

inside one package.

Now, in addition to being able

to go in the third dimension,

you can also design each chiplet

to do its own job

as efficiently as possible.

Yeah, I'm so glad you mentioned that

because here's an example.

Here's one of our CoWoS cells latest technology.

So I have my main silicon,

my HBM on the sides,

and then on the top and the bottom,

I have my IO chips

that communicate with the outside world.

So now I can have each piece of silicon

optimized exactly for what it needs to do.

And that also means

that you reduce your energy usage.

And on the consumer side,

it also means that my cell phone battery

lasts much longer.

Yes, exactly.

You can do so much more.

Exactly.

Going back to Penn State

with your students,

what are you going to be working on now?

What are we going to be working on?

I mean, we're going to keep designing

new materials for the future.

We're going to keep working

on 2D materials, trying to understand...

One of the major problems right now

in 2D materials

is actually it's called the doping problem.

So how can we, in one material,

change its properties

just by adding tiny amounts

of doping materials?

So this is actually a real challenge

in 2D materials,

and it's a challenge we need to solve

if we want to use these materials

for advanced transistors.

It sounds like high-tech cooking, right?

How do you have just the right amount

of spice to make it take?

It exactly is like that, right?

So when you want to...

Say you want to make chocolate chip cookies,

what do you do?

You go to the store,

you buy the bag of chocolate chips,

you follow the recipe on the back.

We can do the same thing

with material science.

If we want to make a known material,

we can look in the literature,

find its recipe, follow the recipe,

get the material.

But if you want to bake something new,

what do you do?

Well, you try some things.

Maybe you start from a recipe you know,

you change the thing here,

you change it there,

and eventually maybe you have a new

better recipe for chocolate chip cookies

or a totally different kind of cookie.

And that's exactly what we're doing, right?

We start from a known recipe,

we change things a little bit here and there

and see if we can't make our materials even better.

So what excites you most

about where we're headed?

I think the thing I'm most excited about

is actually the ability to do this

really interdisciplinary work

where people who have different expertise

work together to try to solve a big problem.

And this is the only way

that we're going to be able

to solve the big problems

that we have nowadays.

Yeah, yeah, beautifully said.

We have a name for that.

We call it STCO,

system technology co-optimization,

which basically means that people can't go off

and design everything in their own sandbox.

They have to work together.

I think one of the things

that I'm really excited about

is the ability to use AI

and machine learning

to accelerate materials development.

Right now, we get so much data

from our experiments,

and I'm sure that you have orders

of magnitude more data than we do,

and trying to spot that needle

in the haystack for a human is very difficult.

It's too much data

for one person to think about.

But artificial intelligence,

it doesn't have those same limitations

at looking at huge data sets,

so it can look at all the data

and find the signal

that we're really interested in

to solve some of these problems.

Well, thank you, Stephanie.

It's been a real pleasure talking to you,

and I want to wish you and your students

a lot of luck

and hope you find many needles

in those haystacks.

Thank you so much.

I hope you enjoyed learning about how

some of the teeniest technology

in the world

is powering the biggest advances

in artificial intelligence.

It's a great time to be an engineer

or to become one,

because there's plenty of tough problems

to solve, and we need to solve them together.

It's also a great time to be alive

because with AI,

nothing has to be done

the way it was done before.

All you need is an idea

and an AI platform running on chips

designed by our customers

that we built with atomic precision.

And here's the part I love.

With AI, there's a virtuous cycle at work.

AI is now helping us design

the next generation of chips,

which will power

the next generation of AI.

So somewhere out there,

there's a chip we haven't built yet,

doing things we haven't imagined yet,

maybe designed by a kid

who's holding a postage stamp right now.

[upbeat music]