All language subtitles for 4. Control-Loras SDXL - an Introduction

Control Laura's on one of the most important developments in

Stable diffusion over the last

What was the last few months

The based on the idea of control Nets and control Nets basically came about in February

And the idea was

You would use instead of just using words to

Define the image to

What's the prompt

What you would do is you would also use

An image

Switch to define the prompt

So the prompt will be both words and image

And lost control net

Initially

Activate it around March of this year

Control Laura's are very new

They've only appeared since

Really about 3 weeks after sdx

Came into being so they've only been around for less than a week

And they're still failing you we're still getting used to them

But that they are so much better than control Nets and control Nets are

Basically something that

We can use but I think in almost every situation where you would want to use a control net

You probably in the end end up using a control Laura

So this is part of the evolution of stable diffusion from

Ideas just coming through at

Of a train running at full speed

The improvements

That come with control Laura is so significant that I think as we start working with sdx

Launch image Siz

I can't see many people working with control Nets unless it's absolutely necessary to

Not at the moment there are only just a few that are handful of control loras

We don't know exactly how well they will work and we don't know how well they'll compare against the control Nets

Did they

Nearly as well as control net

Because control Nets are extremely slow very very unproductive

They're very good but they take a long time and a lot of system resources

Unless the control Laura's are much worse than controlling

I think the future is going to be probably more the control Laura's

Dim the control

But the two are very closely integrated so

Disappear altogether

Woke up a control Nets

Little bit later on

But it's not really my top priority for this video

So control Laura's allow us to take an image turn it into

Pre-processed I'm

And then use that image to influence the output

We still need a prompt so with this particular

Scheme with this particular workflow we've got the image which is coming in

And control Laura's are really only going to work with sdx at the moment

They're not going to work with SSD 1.5

What I did was that I went into canva and just downloaded a whole bunch of images

Which are all 1024 by 1024 in terms of size

That one megapixel

And we are using these

Fast forward testing out these control lures

Control

So far seem really cool

What I'm going to show you here is basically this image here with

Select white clicking on this option here

You can then select any image you want from your from your hard drive I won't be able to give you these because they are

Play copyrighted so

I'll ask you to use your own images

Chosen the image it basically goes through a process that were fairly familiar with

We've got the stxr base here which is the

1.0 version of

Off the model for sdx then we

Something called the load control net model

What this allows us to do is to load the control net

Particular case I'm going with

Add this guy here this is the

Aurora Kenny

And Laura Carney control Laura Kenny

Give us this type of images of binary image black and white

And it just basically tells

I swear the

Features of the photographer

And you can actually change

How it renders by changing the threshold so we could increase the threshold States 250

And that should

Allow us to

Change slightly how this actually looks

There are other features in which we won't go into too much but one of the important ones here

Is we've got a strength so we can actually say right

What I want you to do is to

Apply the control net with a high degree of strength

Or we can say

I want you to process the images you normally would

20 of the time

And then afterwards

I want you to process the image

With a bit more

With more of the involvement of the control Nets

So we can say start at 10%

I think we need to use

Fractions of 1

We could say let me just click away

At no point one

And this guy will continue until 100

Lights

If I was using a control net

The situation would be completely different

If I was running a control

I would have to stop the recording now

Then wait about 10 or 15 minutes

Until the controller that had completed

Then I'll be able to see what it looks like and then I'll be able to tell you whether or not it was a success but with the

Get it going

And it should finish fairly quickly you can see how quickly it's running

We've got 30 steps we're using a CFC of 8

Oil and ancestral

You can change this to whatever you want basically

And I should be done fairly soon

And then we have our beautiful Cleopatra are beautiful

Egyptian queen

And you can see the The Prompt is basically portray

Queen Cleopatra wearing ancient Egyptian dress braided hair

Hair decorated Cleopatra Style

Background of ancient Egypt

And this is what it gave us I think this is looking

You're pretty nice

I don't think the background is ancient Egypt but the background looks nice

What we might want to do is to reduce the

Off the control net so give it a bit more freedom

What about 0.8

Thereabout

And that we could also change the checkpoints so we can change the checkpoint to maybe

The dream shaper

Excel

Alpha

And then see what that does does it produce a better Cleopatra

I like this one I think it's pretty nice

So this is the result that we got using the dream shaper

End of the slightly lower strength so what's happened is that we've got

What's the result of the

Her hair is a little bit different from the original photo

Is taking a little bit more freedom and how is interpreted the

And we've got slightly different I mean we can take a look at the

What's the current

And that's the previous one

I like them

But I think there's a bit more freedom

With the lowest strength that we gave to the

Do the control

And I think also

Dream dream shaper

I think it has a slightly more Fantastical feel I think with the

St XL it looks good but it does look a little bit like an illustration as well

So I do like that it's a little bit more

A little bit more photorealistic with dream shaper

But that's how we control the control net or the control Laura

And I think one of the difficulties I had with the the first

Course that I did with

A lot of students

Coming from the coming from automatic 11:11

Absolutely convinced that automatically 11:11 is the best thing ever

That's the Way Forward is just to repeat everything

Comfort you I

That was done in automatic 11:11 where is I've always been aware that comfy your eyes capable of more

Then automatic 11:11 in terms of its efficiency

In particular

And there's a different philosophy

Compared to automatic 1111

And I don't know if I'm actually going to demonstrate

How slow the control Nets are they

Deathly slow compared to

Particularly with image sizes of 1024 by 1024 deathly slow

Compared to the control Laur

So I think part of the

Automatic 11:11 users

Getting into the mindset

Configure I is a completely different platform with a completely different philosophy

And just repeating everything

What happens inside of automatic 1111

Is not the way forward you have to look at what comfy you I can do

And then basically Jude

The software on the basis of what he can do

And sometimes you may want to go back to automatic 11:11 because it does things better

But most of the time

Learning configure I thoroughly

Gives you the best grounding to understand how to use it effective

How do we get this control Laura how do we install them

Everything's new I'll give you this file probably

And its finished stage

And then you can decide

To experiment

As much as you want

Once you've got the basic

Groundwork the basic Foundation

I think most of the control Nets or the controllers will work

You're very similar way

The first thing that we need to do is to go to the manager and we need to go to install custom loads

We're going to install an extension since custom loads are basically extensions

I'm a strong believer in installing the minimum number of extensions

Set alarm for your workflow so

For me the installed extensions

If we remove the search

It's just a handful

And I can get by with

What will

Is it going to the search box and

Type in control net

We didn't come up with these These are the installed ones

If we change

Use this to

We can see the available options

The ones that I've installed are just two out of the available ones and is the control net preprocessor

We install this one

And I don't think we necessarily need to install this one so

The control net preprocessors

Find novel 16 number 9

Number 10 I don't think it's absolutely essential I have installed it

I don't know if you actually need it if you can work without it then work without it

Let's click here and see what this is all about

Okay so this is the project and quick correction it's actually says the repository has been archived

So probably what you want to do is install both of them so this one and this one

I

Is there any problems I've experienced so far having both of them

The second one is definitely work in progress

However the one the upper one is actually

Are

It's basically a set of preprocessors for come for UI

There isn't a lot

Useful information here but you have got a couple of

Details that show you what it looks like so we have the source the input the output

We have also another one here the

So you can see what the

What the actual preprocessors do here

But there's a massive amount of information and

I think you'll end up with a headache if you tried to take it all in at the same time

Take a look at the other one

Which I think is the one

That replaces this one

Auxiliary

Continue

Control Netflix auxiliary preprocessors

So these seem to be still work in progress and we can still we can see some of the similar images

Showing you what the controlnet will produce

What the preprocessor will produce here we have the

Black and white

And then we have a slightly softer result

And then we have scrib

Is something

That allows you to tell

Close to you and what's far away from you so

The object is the brighter the area is the closer it is to you

Uses artificial intelligence to figure out what's close to you

And what's far away from you

Inside of the

Inside of the image

Put another depth map with a normal map which is a slightly different way of looking at surfaces

How light works with them

I've got open pose

So these are the

Hopefully at some stage we will have control

Laura's for all of these different features

But at the moment we've got four controllers

Install this one so click install on this one

And click install on this one

Give it time to process between clicking install so it'll

Make a couple seconds of a couple of minutes

Process

Each command once you click on install

And then restart

Comfy UI so that we actually have

The preprocessors installed

Some of this stuff does take a bit of time to to actually install

It can take quite a bit of time

On the terminal on the

On the command prom

Just make sure

That everything has finished installing before you

Before you

Let's go ahead and take a look

At where these control Laura's are actually located

Hugging face

Stability AI have released

A number of controllers let's go to the main page

And there's a bit of a description here

And what we see here is that

There are two types of control lures

256 ones

And the rank 128 on

Now the rank 128 ones you might think to yourself all those are the small

Inferior ones and

The rank 2561

Are the

Juicy professional ones

Experimental

They are not guaranteed to work properly so

Use them but don't necessarily prioritize them prioritize the write 128 ones

The rank 25610 738

Megabytes these ones are very manageable

Add 377

Now the control Nets are actually

64.7 gb 5 gigabytes

They are huge and they're extremely extremely painful to run particularly if you have a less powerful

Now if you've got an 8 GB system you should be able to run the control lowers without too much difficulty these

When it comes to the control Nets you're going to struggle and I won't be able to Jackie to show you let me know because the system

I won't be able to run

We've got a Midas in

Play

Laura control Laura

And this one can be used to produce

Images with some sort of difference between the foreground and background or

Difference between

Object

Where the objects close to you and the object is far away so it's literally a depth

I mean

It's self-explanatory

Then we have Connie Edge which we've actually used

Next and then there's this photograph

And sketch colorizer

So what this one does it's taken this gentleman here

I think this is an AI gentleman I don't think this is a real photograph

And it's taking him and giving him a bit of color

Background color bit of skin color and

His clothing as well

It's all looking very good very intelligent in the way it's it's it's done

And then we have another one which actually gives color to sketches

Black and white sketches

Does the revision Laura this one allows us to take two images

And then to combine them conceptually so

If we click on here we can actually see a little bit clearer

Choose two images and say look

Combine these two images

And let's produce something wonderful

And this is

Conceptually

Similar to this

So it's something similar to what used to be called style transfer

Style transfer basically on steroids or sound transfer on LSD

It's amazing to look at let's see what we can actually do with it

All of this stuff is very new

Now there are other future plans to create a user interface which is going to be

Interface which runs

On comfy you I or other versions of a stable diffusion so this is working progress

It's an early stages and I don't think we'll cover it but

It gives you an idea of what the future is for stable diffusion with stability

Play Tai

Not for this particular

Spell tutorial

We would go to the files and we will actually just get the one to eight I don't think we need to

I worry too much about the 256 ones

And then once we've got those we're good to go

Now inside of comfy you I once you've got these installed

And you've restarted you should be good to actually

Start using the control lower

The process is going to involve

Basically

Taking the image

Which would you load

Recreate the preview image

Preview images is is the one that's created by the preprocessor so that's the stuff that we've

Installed from

From GitHub

And then we apply the control net itself and the control net

Is coming in from this control here so

With using controlnet

The 1281

We can try the 256 ones and we'll see what those look like but the 1281

Will definitely look better

Those are the ones that I think have been refined

So once you're gone actually change the the negative prompt

And run the same thing again

Came up with this really nice image

Little bit more of the background looking nice with the

How about the pyramid

And

The queen looking

Very very Egyptian

So that was just by changing the negative prompt I think this is probably the workflow I'm going to give you for

For this particular lecture

We need to show you where we put the control Nets

So typically when working with control next we would go and put them inside the company you like models

Control net this is where they would end up

And we also put control lures in here so I've got a subject for the control loras

And we don't put the control Laura's

Inside the models

Laura

What's fold

So they would go in the control net folder

And that would allow us to access them from inside of