All language subtitles for MasterClass - James Cameron Teaches Filmmaking 14.Avatar Creating Technology Eng

[MUSIC PLAYING]

JAMES CAMERON: When the term is
a computer-generated character,

there's the impression
that the computer's

creating the character.

Computer's not
creating anything.

Computer is doing the number
crunching to generate an image.

But character is being
created by the actor.

It's not animation, it's
performance capture.

[MUSIC PLAYING]

We've talked about
low-budget filmmaking.

Now we're going to talk
about the opposite end

of the spectrum.

In "Avatar," we were-- it was a
highly experimental production.

So one of the things
that's unique about it

is that we went into it starting
years before the release.

We started in 2005, and the
film came out in late 2009,

not knowing exactly
how we were going

to solve most of the problems.

We had a general idea
of how to do things.

We were pretty
thin on specifics.

And I had actually
written "Avatar" in 1995,

as a challenge to my
digital effects company,

Digital Domain.

And they said, we
can't do it yet.

We don't have the
tools to do the muscle

rigs, and the
facial muscle rigs,

and the eyes, and the hands.

And we can't do it.

It's too much.

It's beyond us right now.

I said, all right,
I'll just put it--

I'll just put it in a
shelf for a few years.

And I waited, and I
waited, and I waited.

And then I was watching
"The Two Towers."

And there's the scene where
Gollum talks to himself.

And he's in two places at once.

And he's having this kind of
schizophrenic, out-of-body

experience.

And it's highly expressive,
highly emotional.

It goes on for several minutes.

And it's 100% CG character.

And I said to
myself, if they can

do that, I can make "Avatar."

And that's what set me down
the path of making the movie.

So I was inspired by the
work of another filmmaker.

I was building on what
Peter Jackson had done

and what WETA Digital
had already done.

So I came in with a whole
new set of challenges

and a new high bar they
had to get through.

And they loved that.

They loved that challenge.

We just studied a lot
of close-ups, really

big close-ups of men and
women and looked at them

in different kinds of lighting
and said, well what do you see?

What do you see about
the way they move?

What do you see about the
way they take the light?

What do you see
about the skin pores?

What do you see about the eye?

You know?

And one of the breakthroughs
was the eye is not dry,

the eye is wet.

That's kind of obvious.

But it also means that the
upper part of the lower eyelid

is actually a wet surface.

And there's a little meniscus of
tear fluid between that eyelid

and between the eyeball.

And it catches the light.

And there's a little glint
there, and it's always there.

But if you're not looking
for it, you'll never see it.

The second we put the meniscus
in, the eye came to life.

I know, it sounds like a
little tiny thing, something

you'd never think about.

But when it's missing,
you sense the lack.

And I always felt that
CG characters fell down

around the movement
of the mouth.

It always felt mask-like.

It always felt sort of too
thin at the corner of the mouth

and not interacting
properly with the teeth.

It didn't feel like those
muscles really exist,

like the dimensionality
of the mouth

and the face really existed.

So we spent literally a couple
of years figuring out how

to do this with WETA Digital.

And WETA Digital did the
heavy lifting on this.

And it was iteration after
iteration after iteration

and coming up with
little things.

So the most critical thing
about all visual effects,

and certainly any kind
of CG character design,

and especially if you're trying
to emulate photo reality,

is to observe.

Observe, observe, observe.

[MUSIC PLAYING]

In a nutshell, the
methodology was

that we're going to capture
the performance using

a collection of data.

The idea of motion
capture where we

capture the body movement of
a character had been around--

I used it on "Titanic" for
some of the crowd scenes.

So that had been
around for a while.

It had been around
for at least 10 years.

The idea of how you
were going to capture

the specifics of a
facial performance, which

is where you see the majority
of the emotion in the eyes,

the mouth, all the many,
many muscles in the face.

So it's almost it's
own separate world

from the full body performance.

People that have tried
to do this in the past

tried to do it by using markers
that they put on the face

and capturing the face
the same way they captured

the body, with a bunch of motion
capture cameras and a big array

around it.

But the detail
wasn't fine enough.

They could only work
in very small volumes.

It really limited the action.

It limited how they
were going to-- you

know, the type of action
that they could stage.

And it was very--

I mean you had to literally glue
like 80 little, tiny glass bead

balls onto the actor's face.

It was horrible.

Actors didn't like it.

The end result
wasn't that great.

So we proposed a completely
different solution,

which is to take something that
looked a lot like a boom mic

for a singer in a concert
and put a little tiny video

camera out on the end of
it on a pretty wide lens

and shoot the face, film
the face, or video the face,

while the actor is
doing the scene.

And it's knolled out to them.

It stays with them
wherever they go.

And it's getting a 100%
recording, a close-up

recording, of their performance,
their facial performance.

So now you've got a video data
stream coming out of the face,

and we called that image-based
facial performance capture.

Right?

And that set a trend that's
still in motion to this day.

A lot of other
people have used it.

Different people have
brought different refinements

to the process.

But it hasn't fundamentally
changed since 2005 when

we first launched the concept.

So now you have two data streams
that are recording the actor's

performance.

You're getting the
body performance

off a standard
motion capture system

with a lot of really
tricky bells and whistles

on it to make it better.

And you're getting
the facial performance

from this little camera that's
on this little camera boom.

Here's me with my virtual
camera on "Avatar."

And I'm working with the
actors in their capture suits.

You can see that they
have marker suits.

We're just in a big lit stage.

There's no jungle
there whatsoever.

They're running on a log.

So I think these are
probably more stunt

doubles than anything.

And they're running on a log.

They're just running
basically across the floor.

It's very safe.

And I'm looking--

I'm not looking at
them with the camera,

I'm looking at the
characters that they're

playing in real-time
in the world,

in the CG world all
at low resolution.

It almost looks like an
old, crappy video game

from the '80s.

JAMES CAMERON (ON
VIDEO): (SHOUTING) And 6.

And 7.

And 8.

We're much better at it
these days than we were--

this is all, you know,
obviously 11 years old.

Here's me.

And, once again, I'm
not looking at them.

I'm looking at my image.

But I'm near them, so
I can talk to them.

And I'm seeing what they
look like in the world.

And then here's me playing back.

And what I'm seeing
is they're up

on what we call a vine bridge in
between two floating mountains,

and they're 1,000
feet above the ground.

But, of course,
because the characters

are very mountain goat-like
and can run around

1,000 feet above the
ground, it was okay for them

to not be scared and
balancing and wobbling along.

And it was completely safe.

And so we're creating
a fantasy image

and using relatively
simple setups to do it.

And basically, we found out
that it worked very well.

You could do just
about anything.

You could run, you could
jump, you could ride a horse,

you could have a big argument.

You could even, if
you were careful,

you could drink a beer.

You could do whatever
you needed to do.

The only thing we couldn't
shoot using the system

was a kiss, for obvious reasons.

Because the two booms
would be in the way.

So we had to use a different
kind of harder technique that

was more labor-intensive to
do the kiss scene and a couple

of other moments.

But for the most part
it worked for I'd

say 99% of what we
were trying to shoot.

We had a lot of
proof of concept.

We spent a fair bit of
the studio's money--

the studio being
20th Century Fox--

to prove that we could do it.

And to do that, we did a test.

We did a 1 and 1/2 minute test.

Boy meets girl in the jungle.

And they talk.

He only come
and make problems.

Only.

MAN (AS JAKE): Okay, fine.

Fine.

Fine.

You love your little
forest friends.

So why not just let
them kill my ass?

What's the thinking?

Why save?

MAN (AS JAKE): Yes.

Why save.

You not fear.

You strong heart.

And it was compelling enough
that Fox was willing to pull

the trigger on a $200-plus
million project that then went

somewhat over that budget.

And it was three
years later that we

had the first shot that really
proved that we could do it.

So think about that.

We had our proof of
concept done in 2005.

We got the first shot
that we could actually

use in the movie in 2008.

And everything in between
was a gigantic leap of faith.

And it's easy to look
back on it and say, oh, it

was all a no-brainer.

Come on, that world building,
those great designs,

all that stuff.

Yeah.

That's true.

World building was
a big part of it.

This was the most
important part,

because the audience enters
the film through the eyes

and hearts of the characters.

So we had to have
the characters.

[MUSIC PLAYING]

The key for me is, if the
actor did the performance,

we've captured it, it's going
to be in the character later.

If we want to run a
little splitscreen here

where you can see
Zoe Saldana in one

of her big emotional moments in
the film with Sam Worthington.

Emotional moment for him, too.

I trusted you!

Trust me now.

Please.

[SPEAKING IN NA'VI] That is
why you not Na'vi [INAUDIBLE]..

(ANGRILY) You will never
be one of The People!

He tried to stop them--

ZOE SALDANA (AS NEYTIRI):
[SPEAKING IN NA'VI]

Neytiri, please!

Please!

JAMES CAMERON: So
she's freaking out,

and it's the end of
the relationship.

But I think what's
interesting here

is to look at how
closely we were

able to capture her expression.

It's not perfect, because
her character is not exactly

physiologically
identical to her.

Her nose is very different.

Her eyes are very different.

Her forehead's very different.

But her mouth is very similar.

And you can see that the
performance capture technique

captured what she did.

It's an actor-driven process.

Everything that she does,
everything that Sam does,

translates into the character.

And this is what we hope for.

This is what we worked toward.

And, of course, this
is the head rig.

And this facial performance is
being captured by this camera

and by a video feed,
a live video feed,

we broadcast it off the
body to a record deck.

So you can see that he blinks,
his lips are pulled back

in that exact moment
exactly the same way.

It maps through frame
by frame by frame.

And we don't take
liberties with this.

It took us a long
time to figure out

how to create the
neuromuscular rig, which we now

call the algorithm for Zoe.

Because Zoe's face
did a lot of things

that a lot of people's
faces don't do.

The way in which her
lips would curl out,

and her-- the tension,
and the way her lips

would thin out over her
teeth in a very feral way.

It's a very expressive face.

And we needed to figure out how
to get the pseudo-muscles that

exist entirely
mathematically to do that.

And that was our goal.

And that's what the technical
teams at WETA were able to do.

But the hand motion,
everything, you

can see how it maps
frame by frame.

And there's absolutely zero
compromise in that system.

You have to get the
performance in the moment.

You can't make it better later.

There are a couple
of times where

we tried to alter
a moment slightly,

maybe to have a tiny bit more
of a smile or a little bit

more fear.

[SCOFFS] It was a
horrible failure.

It has to be the
actor has to do it.

And then it's just
up to us to honor it

in the downstream pipeline,
which takes about a year.

From the moment you
capture it, you're at least

a year away from
seeing a shot, which

is a thing in and of itself.

I don't recommend making
a movie like "Avatar,"

but it's good for
people to know,

even if they're just starting
out, what can be done.

[MUSIC PLAYING]

If you're shooting a scene with
camera, a photographic scene,

you're going to shoot a
master, some kind of wide shot.

And you're going to probably
shoot some kind of a two-shot,

or a closer shot that
ties both actors in.

And then you're going to
shoot an over-the-shoulder.

And then you're probably going
to shoot either a tighter over

or a tight single
on this side of it.

Then you're going to turn
the lighting all around,

you're going to come
around over here,

then you're going to shoot
the corresponding tight over,

and then probably
a tight single.

That's classic coverage.

A lot of filmmakers
don't like to do that,

because it's too classic.

But it works.

And there are
reasons why it works,

and people still
do it all the time.

But let's just take
that as an example,

because it is classic coverage.

So that's six setups.

Across those six setups, maybe
you do five takes of each shot,

just for the actors, just to
figure out in the master what

you're doing, figure out
in the two-shot the timing,

figure out in the
close-up the emotion,

the moment that
somebody is going

to cry or have some big
reaction or whatever it is.

The actors need time.

They need to work
their way to it.

And so let's say five--
let's say five takes.

So that's six setups
times five takes.

That's 30 times that the actors
have to do it the same way,

so that you can cut it together.

On the other hand, I could do--

I think I did seven
takes of this scene.

And none of them were the same.

But it didn't matter,
because for this scene--

I'm gonna show you something
really interesting here.

And this is the
fundamental difference

I think with the
performance capture.

All right.

So we start in a
fairly tight two-shot.

They're both barely in
frame with each other,

but it emphasizes the way
he's trying to hold on to her,

and she's pushing him away.

All right?

Now we're going to
cut to her close-up.

All right?

That's a separate set up, right?

Then we're going to
briefly be on that two-shot

again while she slams him away.

Then we're going to
continue with her

in a slightly looser close-up.

And then we're going
to cut to his reaction

as he tries to explain it.

And then we're going to
cut to a medium shot,

so that we could see
her hand and her gesture

as she pulls away.

Again, I went wider here
to suggest distance.

She's already pulled
away from him.

We see a little
bit of her mother.

Her mother tries to stop her.

She pushes the mother away.

Guess what?

It's all from the same take.

All those shots are from
the same take-- take seven.

I happen to remember that,
because I was very happy when

I realized that there was one
take that had everything in it

that I wanted, where Sam
was great, Zoe was great,

everything was great.

We had good reference coverage.

So all of those
shots were extracted

from the same captured take.

Now we had done other takes.

I didn't like them as well.

And so I didn't want to
shoot my virtual camera

coverage from those takes.

I did it all from that one take.

We don't always do that.

But it shows what's possible.