Different results after swapping graphics card

[littlelionel]

Hi all,

I just swapped my graphics card from a gtx 1050 to a gtx 1060 6GB card.

The problem I'm having, ever since switching to the 1060 and installing xformers, the speed has greatly increased but I'm having less than desirable results.

Trying to reproduce any of the images I generated earlier while using the 1050 seems impossible.. using the same seed, sampler, steps etc but the results are significantly different. What was once color images are now B&W, poses of people are different, Even faces no longer have a likeness of of the intended person in the prompt.

I have tried using earlier commits, using different command arguments low/medvram etc and it's still producing the same results. I can only reproduce images from the point of changing graphics cards.

Is this normal?

Thanks for your time!

L

[littlelionel]

Here are a few examples using same model, steps, seed, sampler, cfg scale and prompt ..

•1050•

•1060•

•1050•

•1060•

[cgessai]

While I don't share the same exact issues you're dealing with, I came here to explore a somewhat similar issue I'm having. See, I use old 1940's actresses in my color renders and I noticed on more than a few occasions that during my render of a large batch, they'll switch from color to B&W and finish out the run that way. I absent-mindedly chalked it up to seed progression. However, even running a separate prompt afterward will cause the renders to unexpectedly continue in B&W. It doesn't seem like they're 'hard locked' in that mode but it's like it just really wants to do everything in B&W from then on unless I explicitly kick it out using a really forceful color prompt.

It may or may not be connected but I also noticed that after rendering a series of women and using a 1920's starlet's likeness, sometimes subsequent renders of different, unrelated female celebrities will also be rendered with that distinctly period (costume?) hat I saw her wearing on her first render.

Reading this message last night, I started to get the idea that maybe both the unusual effects I was seeing were from residual cache files left over from previous renders. How, exactly, those things could be related, I don't know. It takes enough time to render a bunch of images on my 4GB GTX 1650 that the possibility of testing by rendering hundreds of images on my notebook has caused me to put it off detailed testing. I wanted to include my experiences and that message link as possible clues to help.

Also, if you use IMG2IMG, are these inconsistencies occurring with both TXt2IMG and IMG2IMG?

[littlelionel]

I have also read a few "same prompt/seed different results" threads but none seem to be connected as rolling back to a previous commit fixed the problem for them.

I'm literally stumped I've spent all day trying different revisions in the hopes that it was something broken in a recent update. Alas I figured I'd go back through the images I've generated to see at what point it started happening and figured it was when I swapped graphics cards.

Img2img also seems to be driving the results towards the same style as the 1060 images.

Custom models also seem to be affected as if the seeds have just randomly changed as no picture is being generated in the same style as before.

Will swap cards back tomorrow and see if it fixes the issue.

[ShnitzelKiller]

so you didn't have use xformers with your old graphics card? Xformers does change the results due to the optimizations it applies. Though I'm not sure if they are that drastic.

[littlelionel]

I don't think that's the issue as I've tried multiple fresh installs of previous commits without xformers and with the most recent commit without the --xformers argument but still the same results unfortunately.

[kall666]

have you updated stable-diffusion-webui after you change your card to 1060

[ClashSAN]

Were you using opt-split-attention-v1, results may be different? Also, did you have some setting you did not know about on? (highres-fix, eta noise multipler, etc) Do you still have the old repo setup + graphics card? There are some things that may be updated when you are connected to wifi, such as updates to codeformer or other modules. Turn off wifi before trying to replicate.

[littlelionel]

No just --lowvram with the 1050 all other settings were on default. I don't have the old setup unfortunately but I do have the graphics card.

Going to swap them back around now to see if I can reproduce the old results.

[littlelionel]

Yes I can confirm it's the graphics card. I've just swapped them over and I'm getting the old results again. Same build same settings other than --lowvram

[ClashSAN]

commit version?

[littlelionel]

Tested on both the latest and [04c745e]

[ZeroCool22]

[littlelionel]

Great stuff.

Still quite different than the results I'm getting but it would seem to be going towards the same style as the 1060 generations.

Still don't quite understand what's changed or how the 1050/CPU (noticed hardware acceleration was switched off in windows) was producing better results especially with people/characters and with more vibrant colors as now the colors seem quite flat.

[ctznkn]

Looks like the 1050 generates the color images using less steps than the 1060 for some reason. My GPU generates the same image with CFG:7/Steps 20 as your 1060.

[littlelionel]

That's interesting, maybe it has something to do with the time taken to generate with the 1050 20 steps took around 3-4 minutes whereas with the 1060 only 25 seconds. It seems the longer it takes on the image the better the quality just somewhat strange that the 1050 would be producing better results with fewer steps.

[cgessai]

Holy geeze, is this even a known effect? I.e. has this B&W issue come up anywhere else before? My hat's off to you for running this test. Yet again I'm having renders come back with B&W images for no good reason and this might give me a clue how to get out of it.

I'm just so puzzled by it all, though.

[iioaia]

I also changed video card yesterday. From a amd Rx 580 to a rtx 2060 12gb.

Speed is great but definitely more black and white images than before. Literally same install just removed the Arg for no CUDA.

[littlelionel]

Running some more tests at the minute but like ctznkn tested above it seems the more steps and time taken generating, the more the style and quality of the image comes back from the images generated on older/slower cards although not exactly the same results it's the closest I've been so far.

[0xdevalias]

A friend mentioned that using xformers could make things non-deterministic, and that there were a lot of references to it on the repo issues here. Wanting to understand a bit more about it, and link a bunch of potentially related issues together, I tried to find as many issues as I could that seemed to be related to xformers and the potential for it to be causing non-deterministic / unstable / inconsistent results:

The following may potentially be related (ordered by issue number):

Issues:

• Xformers optimization causes inconsistent generations with the same parameters #1999
• No token limit for prompts broken? Different results with the same prompts and seed #2005
• The same image can no longer be output under the same conditions. #2197
• On the problem that the same resolution can not be generated stably #2293
• Different results after swapping graphics card #2423
• Different images for same seed with xformers and "Highres. fix" enabled when ancestral samplers are used #2528
• Different outputs since WebUI update #2943
• xformers and euler_a variant results for same generation settings #3054
• [Bug]: The prompts do not produce the same results as before the latest updates on 10/22/22 #3476
• [Bug]: xformers always generate different results, seed is the same! #3967
• [Bug]: In img2img, setting denoising strength to 0 does not return original image #4011

Discussions:

• Different results after swapping graphics card #2418
• --xformers CMD line option causes slightly poorer quality images on GTX 1650 (Turing), possibly others? #2592
• Highres.fix composition #2705
• Exactly same prompt no more the same ? #2849
• xformers option has good and bad aspects #2958
• What has changed recently? Different and Worse Results from Same Settings #3461

I also came across this thread in the xformers repo, which while I can't guarantee is related, am wondering if it might be:

• xformers ViT-B ImageNet MAE + Deepnorm training instability facebookresearch/xformers#219

And a question I raised on a PR in the diffusers repo:

• Up to 2x speedup on GPUs using memory efficient attention huggingface/diffusers#532 (comment)

  • A friend just mentioned that apparently using xformers tends to lower the quality of generated images, and apparently even causes StableDiffusion to generate different images for the same seed/settings. I haven't looked too deeply into things to try and validate that, but I was wondering if a) that is something that is currently known/documented, and b) if that's likely to be fixed at any point, or if it's just 'the price we pay' for the benefits it adds. Would any of that be a concern with integrating things here? I did stumble upon this issue on the xformers repo, but not sure if it would be the same/similar root cause as my friend described: xformers ViT-B ImageNet MAE + Deepnorm training instability facebookresearch/xformers#219

[littlelionel]

In this particular issue it was the graphics card,

For some reason the gtx 1050 with 2gb vram seems to produce better images (see above) although these results seemed to be unique to this card and not the intended results (more to what everyone else's results would of been) was the image the gtx 1060 6gb vram was giving which are still not identical to what I should be getting and less than desirable than the ones I was getting previously.

While using the 1060 xformers has no effect on the results for me just has an increase in speed without it I'm producing identical results as I am with it.

The same goes for using old emphasis and kdiffusers settings. Literally no difference.

Rolling back to any previous commit also had no difference in the output using the gtx 1060.

Different command line arguments seems to also have no effect on the final result, just the speed in which it is produced.

The only cause/difference I can see is the GPU and VRAM.

It seems with the 1050 2g vram I was getting good but random results that would not have been reproducible by anyone else.

Using the 1060 6gb vram the results have gotten closer to the image I should be producing but still not identical to what everyone else is producing.

I have tested multiple images using prompts from lexica using same seed, size, cfg, 50 steps, and PLMS sampler which it uses.

Only a handful of results I can produce identically but more often than not I get different results than the author, I can see the same style and can see the same sort of image but some details are different.

It seems the bigger the resolution/size the worse the differences get and can completely change the look of the overall result.

Also, even producing 512x512 has slightly different details but more like the intended picture.
Sometimes trying the exact same prompt again multiple times can give me the intended result identical to the authors picture. which is something that is causing the results to differ with each generation.

After all the testing I have given up on finding a solution and just put it down to using an old gpu with low vram and an old machine bordering on the minimum requirements to run.

I think the issue lies solely with the RAM and VRAM as upgrading to the 1060 6gb has gotten me closer to the intended results but is still different to what everyone else is producing.

L