Original subtitles

[MUSIC PLAYING]

JAMES CAMERON: When the term is a computer-generated character,

there's the impression that the computer's

creating the character.

Computer's not creating anything.

Computer is doing the number crunching to generate an image.

But character is being created by the actor.

It's not animation, it's performance capture.

[MUSIC PLAYING]

We've talked about low-budget filmmaking.

Now we're going to talk about the opposite end

of the spectrum.

In "Avatar," we were-- it was a highly experimental production.

So one of the things that's unique about it

is that we went into it starting years before the release.

We started in 2005, and the film came out in late 2009,

not knowing exactly how we were going

to solve most of the problems.

We had a general idea of how to do things.

We were pretty thin on specifics.

And I had actually written "Avatar" in 1995,

as a challenge to my digital effects company,

Digital Domain.

And they said, we can't do it yet.

We don't have the tools to do the muscle

rigs, and the facial muscle rigs,

and the eyes, and the hands.

And we can't do it.

It's too much.

It's beyond us right now.

I said, all right, I'll just put it--

I'll just put it in a shelf for a few years.

And I waited, and I waited, and I waited.

And then I was watching "The Two Towers."

And there's the scene where Gollum talks to himself.

And he's in two places at once.

And he's having this kind of schizophrenic, out-of-body

experience.

And it's highly expressive, highly emotional.

It goes on for several minutes.

And it's 100% CG character.

And I said to myself, if they can

do that, I can make "Avatar."

And that's what set me down the path of making the movie.

So I was inspired by the work of another filmmaker.

I was building on what Peter Jackson had done

and what WETA Digital had already done.

So I came in with a whole new set of challenges

and a new high bar they had to get through.

And they loved that.

They loved that challenge.

We just studied a lot of close-ups, really

big close-ups of men and women and looked at them

in different kinds of lighting and said, well what do you see?

What do you see about the way they move?

What do you see about the way they take the light?

What do you see about the skin pores?

What do you see about the eye?

You know?

And one of the breakthroughs was the eye is not dry,

the eye is wet.

That's kind of obvious.

But it also means that the upper part of the lower eyelid

is actually a wet surface.

And there's a little meniscus of tear fluid between that eyelid

and between the eyeball.

And it catches the light.

And there's a little glint there, and it's always there.

But if you're not looking for it, you'll never see it.

The second we put the meniscus in, the eye came to life.

I know, it sounds like a little tiny thing, something

you'd never think about.

But when it's missing, you sense the lack.

And I always felt that CG characters fell down

around the movement of the mouth.

It always felt mask-like.

It always felt sort of too thin at the corner of the mouth

and not interacting properly with the teeth.

It didn't feel like those muscles really exist,

like the dimensionality of the mouth

and the face really existed.

So we spent literally a couple of years figuring out how

to do this with WETA Digital.

And WETA Digital did the heavy lifting on this.

And it was iteration after iteration after iteration

and coming up with little things.

So the most critical thing about all visual effects,

and certainly any kind of CG character design,

and especially if you're trying to emulate photo reality,

is to observe.

Observe, observe, observe.

[MUSIC PLAYING]

In a nutshell, the methodology was

that we're going to capture the performance using

a collection of data.

The idea of motion capture where we

capture the body movement of a character had been around--

I used it on "Titanic" for some of the crowd scenes.

So that had been around for a while.

It had been around for at least 10 years.

The idea of how you were going to capture

the specifics of a facial performance, which

is where you see the majority of the emotion in the eyes,

the mouth, all the many, many muscles in the face.

So it's almost it's own separate world

from the full body performance.

People that have tried to do this in the past

tried to do it by using markers that they put on the face

and capturing the face the same way they captured

the body, with a bunch of motion capture cameras and a big array

around it.

But the detail wasn't fine enough.

They could only work in very small volumes.

It really limited the action.

It limited how they were going to-- you

know, the type of action that they could stage.

And it was very--

I mean you had to literally glue like 80 little, tiny glass bead

balls onto the actor's face.

It was horrible.

Actors didn't like it.

The end result wasn't that great.

So we proposed a completely different solution,

which is to take something that looked a lot like a boom mic

for a singer in a concert and put a little tiny video

camera out on the end of it on a pretty wide lens

and shoot the face, film the face, or video the face,

while the actor is doing the scene.

And it's knolled out to them.

It stays with them wherever they go.

And it's getting a 100% recording, a close-up

recording, of their performance, their facial performance.

So now you've got a video data stream coming out of the face,

and we called that image-based facial performance capture.

Right?

And that set a trend that's still in motion to this day.

A lot of other people have used it.

Different people have brought different refinements

to the process.

But it hasn't fundamentally changed since 2005 when

we first launched the concept.

So now you have two data streams that are recording the actor's

performance.

You're getting the body performance

off a standard motion capture system

with a lot of really tricky bells and whistles

on it to make it better.

And you're getting the facial performance

from this little camera that's on this little camera boom.

Here's me with my virtual camera on "Avatar."

And I'm working with the actors in their capture suits.

You can see that they have marker suits.

We're just in a big lit stage.

There's no jungle there whatsoever.

They're running on a log.

So I think these are probably more stunt

doubles than anything.

And they're running on a log.

They're just running basically across the floor.

It's very safe.

And I'm looking--

I'm not looking at them with the camera,

I'm looking at the characters that they're

playing in real-time in the world,

in the CG world all at low resolution.

It almost looks like an old, crappy video game

from the '80s.

JAMES CAMERON (ON VIDEO): (SHOUTING) And 6.

And 7.

And 8.

We're much better at it these days than we were--

this is all, you know, obviously 11 years old.

Here's me.

And, once again, I'm not looking at them.

I'm looking at my image.

But I'm near them, so I can talk to them.

And I'm seeing what they look like in the world.

And then here's me playing back.

And what I'm seeing is they're up

on what we call a vine bridge in between two floating mountains,

and they're 1,000 feet above the ground.

But, of course, because the characters

are very mountain goat-like and can run around

1,000 feet above the ground, it was okay for them

to not be scared and balancing and wobbling along.

And it was completely safe.

And so we're creating a fantasy image

and using relatively simple setups to do it.

And basically, we found out that it worked very well.

You could do just about anything.

You could run, you could jump, you could ride a horse,

you could have a big argument.

You could even, if you were careful,

you could drink a beer.

You could do whatever you needed to do.

The only thing we couldn't shoot using the system

was a kiss, for obvious reasons.

Because the two booms would be in the way.

So we had to use a different kind of harder technique that

was more labor-intensive to do the kiss scene and a couple

of other moments.

But for the most part it worked for I'd

say 99% of what we were trying to shoot.

We had a lot of proof of concept.

We spent a fair bit of the studio's money--

the studio being 20th Century Fox--

to prove that we could do it.

And to do that, we did a test.

We did a 1 and 1/2 minute test.

Boy meets girl in the jungle.

And they talk.

He only come and make problems.

Only.

MAN (AS JAKE): Okay, fine.

Fine.

Fine.

You love your little forest friends.

So why not just let them kill my ass?

What's the thinking?

Why save?

MAN (AS JAKE): Yes.

Why save.

You not fear.

You strong heart.

And it was compelling enough that Fox was willing to pull

the trigger on a $200-plus million project that then went

somewhat over that budget.

And it was three years later that we

had the first shot that really proved that we could do it.

So think about that.

We had our proof of concept done in 2005.

We got the first shot that we could actually

use in the movie in 2008.

And everything in between was a gigantic leap of faith.

And it's easy to look back on it and say, oh, it

was all a no-brainer.

Come on, that world building, those great designs,

all that stuff.

Yeah.

That's true.

World building was a big part of it.

This was the most important part,

because the audience enters the film through the eyes

and hearts of the characters.

So we had to have the characters.

[MUSIC PLAYING]

The key for me is, if the actor did the performance,

we've captured it, it's going to be in the character later.

If we want to run a little splitscreen here

where you can see Zoe Saldana in one

of her big emotional moments in the film with Sam Worthington.

Emotional moment for him, too.

I trusted you!

Trust me now.

Please.

[SPEAKING IN NA'VI] That is why you not Na'vi [INAUDIBLE]..

(ANGRILY) You will never be one of The People!

He tried to stop them--

ZOE SALDANA (AS NEYTIRI): [SPEAKING IN NA'VI]

Neytiri, please!

Please!

JAMES CAMERON: So she's freaking out,

and it's the end of the relationship.

But I think what's interesting here

is to look at how closely we were

able to capture her expression.

It's not perfect, because her character is not exactly

physiologically identical to her.

Her nose is very different.

Her eyes are very different.

Her forehead's very different.

But her mouth is very similar.

And you can see that the performance capture technique

captured what she did.

It's an actor-driven process.

Everything that she does, everything that Sam does,

translates into the character.

And this is what we hope for.

This is what we worked toward.

And, of course, this is the head rig.

And this facial performance is being captured by this camera

and by a video feed, a live video feed,

we broadcast it off the body to a record deck.

So you can see that he blinks, his lips are pulled back

in that exact moment exactly the same way.

It maps through frame by frame by frame.

And we don't take liberties with this.

It took us a long time to figure out

how to create the neuromuscular rig, which we now

call the algorithm for Zoe.

Because Zoe's face did a lot of things

that a lot of people's faces don't do.

The way in which her lips would curl out,

and her-- the tension, and the way her lips

would thin out over her teeth in a very feral way.

It's a very expressive face.

And we needed to figure out how to get the pseudo-muscles that

exist entirely mathematically to do that.

And that was our goal.

And that's what the technical teams at WETA were able to do.

But the hand motion, everything, you

can see how it maps frame by frame.

And there's absolutely zero compromise in that system.

You have to get the performance in the moment.

You can't make it better later.

There are a couple of times where

we tried to alter a moment slightly,

maybe to have a tiny bit more of a smile or a little bit

more fear.

[SCOFFS] It was a horrible failure.

It has to be the actor has to do it.

And then it's just up to us to honor it

in the downstream pipeline, which takes about a year.

From the moment you capture it, you're at least

a year away from seeing a shot, which

is a thing in and of itself.

I don't recommend making a movie like "Avatar,"

but it's good for people to know,

even if they're just starting out, what can be done.

[MUSIC PLAYING]

If you're shooting a scene with camera, a photographic scene,

you're going to shoot a master, some kind of wide shot.

And you're going to probably shoot some kind of a two-shot,

or a closer shot that ties both actors in.

And then you're going to shoot an over-the-shoulder.

And then you're probably going to shoot either a tighter over

or a tight single on this side of it.

Then you're going to turn the lighting all around,

you're going to come around over here,

then you're going to shoot the corresponding tight over,

and then probably a tight single.

That's classic coverage.

A lot of filmmakers don't like to do that,

because it's too classic.

But it works.

And there are reasons why it works,

and people still do it all the time.

But let's just take that as an example,

because it is classic coverage.

So that's six setups.

Across those six setups, maybe you do five takes of each shot,

just for the actors, just to figure out in the master what

you're doing, figure out in the two-shot the timing,

figure out in the close-up the emotion,

the moment that somebody is going

to cry or have some big reaction or whatever it is.

The actors need time.

They need to work their way to it.

And so let's say five-- let's say five takes.

So that's six setups times five takes.

That's 30 times that the actors have to do it the same way,

so that you can cut it together.

On the other hand, I could do--

I think I did seven takes of this scene.

And none of them were the same.

But it didn't matter, because for this scene--

I'm gonna show you something really interesting here.

And this is the fundamental difference

I think with the performance capture.

All right.

So we start in a fairly tight two-shot.

They're both barely in frame with each other,

but it emphasizes the way he's trying to hold on to her,

and she's pushing him away.

All right?

Now we're going to cut to her close-up.

All right?

That's a separate set up, right?

Then we're going to briefly be on that two-shot

again while she slams him away.

Then we're going to continue with her

in a slightly looser close-up.

And then we're going to cut to his reaction

as he tries to explain it.

And then we're going to cut to a medium shot,

so that we could see her hand and her gesture

as she pulls away.

Again, I went wider here to suggest distance.

She's already pulled away from him.

We see a little bit of her mother.

Her mother tries to stop her.

She pushes the mother away.

Guess what?

It's all from the same take.

All those shots are from the same take-- take seven.

I happen to remember that, because I was very happy when

I realized that there was one take that had everything in it

that I wanted, where Sam was great, Zoe was great,

everything was great.

We had good reference coverage.

So all of those shots were extracted

from the same captured take.

Now we had done other takes.

I didn't like them as well.

And so I didn't want to shoot my virtual camera

coverage from those takes.

I did it all from that one take.

We don't always do that.

But it shows what's possible.