xformers option has good and bad aspects

[Remowylliams]

On windows I was able to enable the --xformers option and was struck by how the performance increased and vram requirement decreased. So I used it for many days and found a few niggles that in no way is a show stopper. It is something people may wish to keep in mind.

with --xformers enabled, the variation (not to be confused with variations on a image) tend to have a smaller range of differences than without this option enabled.

It also seems that some influence is left from images rendered previously in a batch compared to without --xformers. I noticed this using a batch size of 4 and no matter what I did to reproduce a specific given image it was never the same using the same seed of the image, or the same beginning seed of the batch. It's like it's caching image noise from previous runs. Nothing changed with the running system other than I tried a higher sampling step and then returned after several tries to the original stepping.

Without a doubt Automatic1111 has been a joy to use 98% of the time.
Thank you very kindly for all your efforts.
Cheers.

[cgessai]

It's like it's caching image noise from previous runs.

If you have a discrete series of steps to reproduce exactly what you're talking about, I or someone else might be interested in trying to reproduce the situation. I'm on the fence about this- what I originally thought was a similar type of "leftover noise from previous runs" may have turned out to have been (in my case) having "alphonse mucha" in my prompt, and which led to repeated 'adornment' elements of his art intruding into my renders and me thinking that was related to cache files.

Having said that, I still think there may be weirdness going on involving certain cards or series of cards which makes understanding the issue difficult.

For instance, this issue: A user changes out his NVidia GTX 1050 (GB?) for a GTX 1060 with 6GB and he starts getting back different renders for the same exact prompt, seed, etc. than he did on his old card. Same general theme but the subject (a gravestone) isn't the same shape/orientation and the image is not in color anymore. I look at his old and new images, pull the config info out of his old image and my GTX 1650 spits out a different 'new' image than he got. A different guy also uses the config parameters for the 'old' image and apparently gets a whole 'nother different image- Three different users apparently getting 3 different images run from the same seed!

That same tester awesomely runs a large matrix of images and seems to infer that the old video card actually produced better (color) images at a lower number of steps than his new one!

In this issue everyone was using xformers. Is that the cause of the apparent inconsistency? I dunno. I've used home computers for 39 years and this is a very puzzling. I've posted that xformers appear to cause somewhat inconsistent images (color fidelity, small details) but there obviously may be some bigger, weirder issue here.

I'm not even sure it has to do with AUTOMATIC1111 or xformers. In these inconsistent results, we could all be seeing minor differences in NVidia cards based on cost-cutting/efficiency tweaks in the manufacturing process.

[aeloi]

I'll add that I and others have seen xformers break determinate results - and this is made worse with the AND keyword. I can click generate over and over again without changing a single setting, and get slight differences(more than slight with AND and complex prompts) every single time, seed set to a persistent value

[C43H66N12O12S2]

#2967

[0xdevalias]

A friend mentioned that using xformers could make things non-deterministic, and that there were a lot of references to it on the repo issues here. Wanting to understand a bit more about it, and link a bunch of potentially related issues together, I tried to find as many issues as I could that seemed to be related to xformers and the potential for it to be causing non-deterministic / unstable / inconsistent results:

The following may potentially be related (ordered by issue number):

Issues:

• Xformers optimization causes inconsistent generations with the same parameters #1999
• No token limit for prompts broken? Different results with the same prompts and seed #2005
• The same image can no longer be output under the same conditions. #2197
• On the problem that the same resolution can not be generated stably #2293
• Different results after swapping graphics card #2423
• Different images for same seed with xformers and "Highres. fix" enabled when ancestral samplers are used #2528
• Different outputs since WebUI update #2943
• xformers and euler_a variant results for same generation settings #3054
• [Bug]: The prompts do not produce the same results as before the latest updates on 10/22/22 #3476
• [Bug]: xformers always generate different results, seed is the same! #3967
• [Bug]: In img2img, setting denoising strength to 0 does not return original image #4011

Discussions:

• Different results after swapping graphics card #2418
• --xformers CMD line option causes slightly poorer quality images on GTX 1650 (Turing), possibly others? #2592
• Highres.fix composition #2705
• Exactly same prompt no more the same ? #2849
• xformers option has good and bad aspects #2958
• What has changed recently? Different and Worse Results from Same Settings #3461

I also came across this thread in the xformers repo, which while I can't guarantee is related, am wondering if it might be:

• xformers ViT-B ImageNet MAE + Deepnorm training instability facebookresearch/xformers#219

And a question I raised on a PR in the diffusers repo:

• Up to 2x speedup on GPUs using memory efficient attention huggingface/diffusers#532 (comment)

  • A friend just mentioned that apparently using xformers tends to lower the quality of generated images, and apparently even causes StableDiffusion to generate different images for the same seed/settings. I haven't looked too deeply into things to try and validate that, but I was wondering if a) that is something that is currently known/documented, and b) if that's likely to be fixed at any point, or if it's just 'the price we pay' for the benefits it adds. Would any of that be a concern with integrating things here? I did stumble upon this issue on the xformers repo, but not sure if it would be the same/similar root cause as my friend described: xformers ViT-B ImageNet MAE + Deepnorm training instability facebookresearch/xformers#219