Original subtitles

Stay Refusion 3 is released. I know you are busy so I'm gonna give you a summary right here.

Yes, you should start using SD3. Can I use it day one? Yes, I'll show you where to download it,

how to get started. Will it give me better results day one? Probably not. Needs fine

tuning. Is it still worth using? Yes. Someone told me this model is bad, is it? Well no, it's a medium

size 2B model compared to the 8B model. You will be fine and you probably won't need the 8B model

until you get a better GPU and I think most fine tunes will be made on the 2B model. Anyway,

why is it better? Text, prompt understanding, 16 channel VAE. If you want no more, keep watching

oh and you know, some more stuff. I'm clearly the superior model. People can't get enough of my

anime art and all that amazing control net setup. Please control net, I got that too and also much

higher resolution. Don't be silly, those control and lowers can match my 1.5 control nets and I can

do higher resolution too with high res fixes and tie up skills just to name a few. Step aside kids,

you're outdated. I can do text, you know, letters that actually go together and make words. Let me

spell it out for you. Bye bye. Who does words anyway? Exactly, can you even animate? Well I'm

not sure yet but I got prompt understanding. Better faces and hands, well at least sometimes.

No one can do hands. Yikes. Are you even fine tuned? Well no, not yet but I mean the community

will surely fix that. I have thousands of models and lowers, you'll never match that. I'm sure

people will fine tune me. I'm also safe to use, unlike some of you guys. What does that even mean?

I'm all about unlimited control. Hi guys, I can do pretty images.

So jokes aside, SD3 will probably outperform both 1.5 and SDXL. Will it do it day one? Well

probably not, maybe not. Maybe. It's a starting point. I think we will need some fine tunes from

the community to see Stable Fusion 3 XL and shine as much as possible. But there are some key

architectural features where Stable Fusion 3 really outshine other models. First let's talk about

the VAE. So VAE used in previous models, use a 4 channel VAE. Now you can go 4 channel, A channel,

16, 32, etc. You get the point. And Stable Fusion 3 uses a 16 channel VAE. You can clearly see

in this comparison image here, which I think is a great reference and showing the difference.

And this is just not on outputting an image. It's actually incorporated when you train the model.

So as you are training the model, you will retain more detail and will be able to output more detail.

Now SD3 is a 1024x1024 pixel model not to be confused with the previous one. So SD1.5 was trained

for 512x512. SDXL was trained 1024x1024. The difference here though is that SD3 works well

by doing 512x512 images. This does not mean that it's a 512 model. This is a 1024 model

that can be used in more sizes than previous. So this is good and especially for people that

are not on a massive GPU machine. I mean I'm on a 4090 so I can generate a lot, not everything,

but a lot. But many people are not able to run SDXL. So having a model, a better model that can

run 512x512 and doing that faster and less resource intensive than SDXL. That's a huge deal

for many people. And one of the reasons for that is that we are getting the medium model or the 2B

model. So there is the full weight on the 8B. For the majority of people, 2B will be fine. It's

going to be great. Requirements are going to be much less than the SDXL. You're going to have a

great time with it on most machines. The diminishing returns you get from the increased quality in

8B compared to the resources you require. I mean the curve goes like and then it flattens out.

There's the diminishing returns, right? So is 8B better? Well, yes. Yes. Yes, it is. Will you need

it? Probably not. At least right now, the majority of users will want to use the 2B1 because it's

going to be faster to run. And if you eventually get the 8B1 and you want to run it, sure go for it.

It's going to be slower. You're going to get a slight increase in quality where some areas are

going to be higher quality than others. So it depends on what your end goal is. But right now,

go for the 2B1. That's the one we have. And start fine-tuning it because it's a great base.

Just jumping back to the V8 and the 16 channel thing. That's going to be a huge improvement.

And if you start digging in the research paper of Stable Fusion 3 under the chapter

improved auto encoders here, you can see here similar to this research paper from 2023,

we find that increasing the number of latent channels significantly boosts

performance. See Table 3. And Table 3 is this one here. Now it's not as pretty as the image we

looked at before. These are, this is a comparison, 4 channel, 8 channel, 16 channel and the FID score

here, which is something called a freshet inception distance. The arrow indicates which is better.

So a lower score is better. 16 channel clearly outperforms the other one's

perceptual similarity. Also lower is better. It's almost twice as good as the 4 channel one.

And in all the other data, it outperforms the other channels. You can keep reading here, but

basically, you know, yadda yadda yadda. Best models with increased capacity should be able to

perform better for a larger yadda yadda, ultimately achieving higher image quality.

We confirm this hypothesis in figure 10. Let's look at figure 10. And here's a channel,

16 channel is the yellow one here. So here are a couple of images that I pulled from the research

paper. I'm going to compare these to just mid-journey, WFusion XL and some doggly three

generations. Now it's not a fair comparison. Okay, I'll give you that because not everyone

handles the prompting in the same way. So using a prompt in mid-journey, the same prompting door

and the same prompting stabilization, it's not a fair comparison. I get it. It's apples, oranges,

and all that. I get it. I get it. I get it. But I think it's, it's, you know, people want to see

that kind of comparison. So I figured I'll do that. And we're going to use the images again,

like in the research paper, we're also going to render them with the actual SD3 model and see

how cherry-picked these are. So first we'll look at this. We have a beautiful pixel art of a wizard

with hovering text achievement unlocked. Diffusion models can spell now. This one looks, well,

looks kind of good. On the right here, we have this frog. Frog sitting in a 1950s diner wearing

a leather jacket and a top hat. On the table is a giant burger and a small sign that says Froggy

Fridays. And I mean, it looks pretty good. We have the frog. It's in a diner. He's wearing a

leather jacket and a top hat. You have the burger and the sign is spelled correctly. Now,

I don't know how many generations it took to get that or if they got it like 90% plus of the time,

but I'm sure we'll see that once we generate it ourselves. And here we have something I think

is kind of cool. This translucent pig inside is a smaller pig. So spoiler, I did this in some of

the other tools and this was a hard prompt to get right. And the last one here, a massive alien

spaceship that is shaped like a pretzel. So all of them kind of cool. And let's look at the first

comparisons. So in this one here, we have old SDXL on the left, we have mid journey in the middle,

and we have Dolly on the right. Clearly in the SDX wall, while it looks kind of good as an image,

the text is not working at all. We get some froggies, some fargoggy, frog roggies, Somali

bird eyes. It just doesn't make any sense. The frog, however, is wearing a leather jacket and a

top hat in all three shoulders of bowling and bowler hat bowling. I don't know. It's kind of

getting it right. Not in all of them. The burger is in front of the frog in three of the images.

You know, we're halfway there. It's okay. The text is not okay. In the mid journey one,

we have this very mid journey cinematic, artsy kind of style. Froggy Fridays is actually working

in most of the images. The first one here is a slight misspelling. That one just says froggy,

but in general, I think it's okay. This one said Fridays here on the little sign down here. This

one's pretty good. It's actually straight and everything. Froggy Fridays. We did get some

Fridays and froggy down here too. In the dolly one, the images look kind of dolly. We got froggy

Fridays, but it's hardly working. In the third image here, we actually get a properly spelled

Fridays. Now bear in mind, I've not cherry picked any of these. It's just the first four generations

out. I can be lucky. I can be unlucky. Again, this is an apples to oranges kind of comparison. We'll

look at the SD3 ones in a bit. Pig here. Translucent pig inside is a smaller pig, SDXL. It's just one

pig. It is sort of translucent mid journey. While you're getting two pigs and some of them,

none is actually inside the other pig. The dolly actually has all of the smaller pigs are inside

the big pig. They're not really translucent. You can like, there's a part where you can see inside.

So as you know, I would say, as far as understanding what the task was, dolly kind of, you know,

solved it in the best way, but also didn't, you know, I think mid journey in general from this

instance had the most most realistic looking images. Looking at the wizard, we clearly have

different styles. SDXL pixel art is very minimalistic. The mid journey one is very artsy and the dolly

is somewhere kind of in between. And in this instance, the mid journey ones actually have the

better text. We have achievement unlock diffusion models now, achievement unlock diffusion can

spell now. It's a long sentence. And it's, you know, models now diffusion can spell. It's getting

there, right? Whereas in the dolly one, the words are kind of in the right place, but everything's

misspelled and all over the place. I should aim the lock diffusion your models can spell now. It's

like, yeah, you can see that it tried the I is not useful for anything. The good part is the words

kind of get, you know, in the proper positions, you could go in after a fact in Photoshop and,

you know, fix that. And SDXL is all over the place. It's this isn't working. So compared to SD3,

you know, that's a huge step. And here again, SDXL on the left mid journey in the center and

dolly on the right. Starting with the SDXL one, we got spaceships. And they are really pretzels,

like they're super pretzels. They're actual pretzels, probably not what we're looking for

in the mid journey ones. They are getting there, like you getting a spaceship and weird kind of

dimensions. And it's trying to like circle them up or, you know, piece them together like a pretzel,

not really getting the shape right, but you know, pretty okay, just as an image. And for the dolly

ones, same thing, right? You're getting a spaceship that goes into the weird shapes. And this one

actually has a pretzel shape, but that's above the spaceship or maybe it's a separate spaceship.

So as far as these examples go, that our chair picked in the research paper, I will see what

we actually get. Okay, guys, we are alive. Hugging face stability AI stable fusion three medium.

We need to agree. This is me. Here's an email. I'm in this country. No, yes, agree. Once we have

agreed to this, we're going to file some versions when you have some options here. You get depending

on what you want to do, you can download the medium one. That doesn't include the clips. If

you're using stable swarm, those are committed to download automatically. So that there are text

encoders, which is the clip G, the clip L, the T five, those to go in the models clips folders,

you can also download medium, including the clips or the including clips with the T five,

which go in your models checkpoint or models stable fusion folder, depending on which you are

use. There's also some example workflows that you can use here. So if you drop one of them in

the basic one here, for example, this is a workflow that you get, you have a loaded checkpoint. So

this is SD three, we have the clips here. If you are using the SD three with including clips,

I would assume that you can just drag that and put that in there instead. And if you journey

generate now, you will get as a female character with long flowing hair. And if we can get an

outment here in a second, that is exactly what we're getting default. We have a resolution of

1024 by 1024. We will note here is as resolution should be around one megapixel and with height

must be a multiple of 64 got a basic problem. So this is interesting. The case sampler is by

default set to 28 steps at a C of 4.5 with DPM plus plus two M, which, well, I like that one.

I've used Keras previously. But let's go with the SGM uniform. And see how that does because

that's what they preset for us. And if you are using stable swarm, if you use selecting the model

here, and when we press generate, we'll get some downloads in the background, which are the text

encoders. And then you get this cute little SD three text encoder thing here. So you can select

clip only t five only or clip plus t five, t five only will get you a slight increase in quality

at a massive resource cost. So clip only will get you, you know, pretty far. Now I don't know what

negatives they used in our general in the comparisons. So we're just going to leave that

blank for now, which is probably again, not a fair comparison. But it is what it is. It is what I have

changes to four images, we're going to generate, we have a 1024 by 1024 resolution, I just starting

using this. So I might be doing something wrong, obviously. So we're getting first, we're actually

not getting any pigs inside of the other pigs, as you can see here. So get generating tell me in

the comments what you think about it so far. If you're still watching this day one, you can use

it on any comfy back end system that doesn't only include comfy into a swarm, I think some of the

focus variants you can use as well. I'm going to keep playing with this in the coming days. So

it picked more videos on the topic, but I'm going to stop it here for now, just so I can get this out

and you can start play with it. Have fun.