All language subtitles for 2. Frame Interpolation When Slowing Down Footage

When we were speeding up footage we were removing frames,

which means we had to blend frames together to recreate motion blur.

When we are slowing down footage we need to create new

frames to extend the length of the clip.

This process is called interpolation.

We can use the same methods to create these frames that

we used to create the motion blur,

but it is harder to create whole new frames from scratch than

it is adding motion blur to existing ones.

We're going to work with this clip of some skateboarders,

and we're going to slow them down by 50%.

I've trimmed the clip down to 35 frames where the

guy jumps over the manhole cover.

This trimmed clip runs from frame 210 to 245, 36 frames long.

If we're slowing this down by a factor of two we'll be doubling the length,

so let's set the project frame range from 210 to 280.

Let's start with the Retime node, so we have a reference.

Let's set the speed to 0.

5.

Now even with the filter set to box we aren't getting any motion interpolation.

You can see that each frame of the source holds for two frames of our timeline,

and gives us a very jumpy playback.

We need to change the shutter setting to enable the frame blending.

If you hover over the shutter control the tip that pops up

says that Nuke should blend a minimum of two frames when

the shutter is set to one, but this doesn't seem to be the case.

If you look in the documentation it says,

increase the shutter parameter to enable frame blending,

so it looks like a shutter of one doesn't do anything

when we are slowing down a clip.

I'm going to duplicate this Retime node,

so we can compare the differences with the frame blending on.

Let's increase the shutter to two.

Now this is telling Nuke to blend two frames together to create a new frame.

When we step through the shot now we will see that every other frame is the

original and matches the version without the frame blending,

but the frame's in between and new.

If you look closely at the new frames you can see that they are made up of

the frame before and the frame after mixed together.

If we look at one of these new frames we can recreate it by using a

FrameHold node to give us the frame before and a frame after,

and combining them with a dissolve set to 0.

5.

We can get the same effect with the TimeWarp node.

To create the slow down effect in the TimeWarp node we need to go to the

key frame that is created at the end of the timeline,

and set it to 245, the final frame of the source.

Let's click the df/dt button.

Strangely,

in this shutter box we have to enter one to give us the same

result as a value of two did in the Retime node.

These frame blended interpolations are definitely much better

than the stuttery results that we had before,

but we can do even better with obstacle flow.

So let's bring in the no flow node.

OFlow defaults to a speed of 0.

5, so we don't have to change that.

Just make sure that the frame range is updated.

If we play this we will see that most of the image looks much better.

If we look at the three skaters on the right in the

background we have a really nice result.

OFlow is using optical flow to analyze the local

movement of each area of the image,

and using that information to create much better in between frames.

It's actually rebuilding all of the frames,

not every other one like the Return node did.

This works really well for subtle, smooth,

consistent movement, like most of the right of the frame,

but it breaks where the movement is more extreme or where

objects appear and disappear in between frames.

When the optical flow loses sight of an object it

can't work out where it goes next.

When it breaks we get these crazy artifacts that you can

see around the left of the action.

The guy's arm is moving too far and too fast for a good rebuild,

the details behind the car breaking as he moves over them,

and the details on the bottom of the skateboard are going a

bit crazy as they rotate in and out of view.

We also have weird stuff happening around the edges of frame where

areas are revealed with the camera movement.

Let's see if we can improve some of these results

with the settings in the OFlow node.

First of all, let's visualize what the optical flow is doing.

At the bottom of the node in the Advanced section there

is a checkbox called Overlay Vectors.

If we enable this we can see the vectors that Nuke is generating.

These vectors relate to the direction that Nuke thinks each pixel is moving.

You can see that in the areas that are working well Nuke has

generated vectors that match the movement,

but the areas that aren't working have crazy vectors.

This view is just so we can see what Nuke is working with.

If we change the vector spacing here we don't actually

change the results of the retime.

We can change the quality of the vectors with these two sliders,

Vector Detail and Smoothness.

The Vector Detail slider increases how many vectors are generated,

with one creating a vector for every pixel of the image.

This gives us more details, but can be much slower.

In this case,

setting the Vector Detail to one improves most of the

areas we are having trouble with, but it doesn't fix any of them completely.

The biggest improvement's around the guy jumping

where the extra detail was important.

It's counterintuitive, but sometimes the extra detail creates a worse result.

If you have something with high detail, like an explosion,

a reduction in Vector Detail can sometimes give you smoother results,

so don't assume that high detail always gives you the best results.

The smoothness slider smooths out the results of the vectors.

The new documentation describes the results like this.

A high smoothness misses lots of local detail,

but is less likely to provide you with the odd spurious vector.

A low smoothness concentrates on detail matching,

even if the result field is jagged.

In this case,

turning the smoothing to one reduced the really

obvious artifacts on the buys arm,

but makes the background behind him much worse.

The high smoothing also reduces the boiling in the road.

If we set the smoothing to 0.

1 the artifacts become more obvious, but the amount of distortion is reduced.

Unfortunately,

it is often the case that what works for one area of the shot

doesn't work for another area of the shot.

One of the worst truths I have learned in my VFX career is that retiming

almost never works like it does in the product demos,

and there is usually a lot of manual cleanup needed to get a perfect result.

In the previous video I said the OFlow and Kronos were similar,

but the Foundry has added some improvements to Kronos in the latest version.

Unfortunately, Kronos is only available in Nuke X,

but the improvements and extra features are definitely worth mentioning,

so let's take a look.

Just with the default settings you can see that Kronos is

doing a much better job than OFlow.

The issues with the brick wall on the left are almost completely gone.

There are still issues with the road and his arm, but they are much better.

They have changed the name of the Smoothness glider in Kronos to Strength.

In this case,

setting the Vector Detail to one and leaving the Strength at the default of 1.

5 gives us a pretty impressive result for most of the image.

One feature that Kronos has that OFlow doesn't is a Matte input.

If you have a matte for the foreground layer you can use it to help

Kronos work out where the different layers are.

In this case, I have Roto for the skateboarder.

If we connect it to Kronos it gives us a much better result.

You can try and tweak the result with a erodes or blurs on

the matte before it is connected to Kronos.

Obviously,

if we slow the footage down more we're going to get more artifacts because Nuke

is having to create more frames with the same initial data.

If we set the speed in this Kronos to 0.

25 we now need to create 3 frames between every original frame.

Kronos still gives us impressive results,

but you can see that the artifacts are more obvious now.

So you can see that optical flow is pretty amazing, but isn't perfect.

Depending on your source footage, and the amount you are slowing it down,

you are going to have some degree of artifacts.

You might be able to reduce these problems just tweaking

the sliders in the OFlow or Kronos nodes,

but the chances are you're going to have to do some manual cleanup work too.

I will show you a few approaches to cleaning up

these artifacts in a later video,

but for now know that if you have access to Kronos it will

give you a much better result than OFlow,

and if you have time to do some Roto that will help too.

So let's summarize this module.

When we speed shots up we have to create motion blurs to simulate the

smoother results we would expect from a camera.

When we slow shots down we have to create new frames to fill in the gaps

left when we increase the time between the original frames.

Optical flow technology usually produces the best

results for both of these operations.

Kronos is the best of the optical flow nodes if you have it available,

but it isn't perfect.

Different settings will work for different areas of your shot,

and you'll probably have to do some manual cleanup work too.