Highres.fix composition

[slavoisard]

I often use Highres. fix, and most of the time, the base picture generated before the hi-res imgtoimg treatment is perfect. Is there's a way to stop at that first step of process ? Or maybe have a Highres.comp checkbox that just does that, doing a fine composition when not using 512x512 ?

[sedetweiler]

I have also noticed that! It is looking amazing and then it hesitates and wrecks the image to apply whatever fix is incoming.

[slavoisard]

May be we can find a solution with the new options (firstpass width & height) and in Extra the Resize options... still working on it ;)

[hunterfrerich]

A checkbox to "save a firstpass copy when using highres.fix" in the settings would be very useful. Especially if you're using settings that prevent perfect seed reproducibility (xformers).

Although, at the same time highres.fix works very well for me. Be sure and try and make the aspect ratio of first/second pass as close as you can. Sometimes it's off by like 15% and that can crop out too much.

[Somasonic]

You can for example run a bunch of 512x512 images and only highres the ones you like by using the same seed and prompt to regenerate the image with the highres fix enabled. This is slower in some ways (extra steps), but saves taking the time to highres images you may not end up wanting to keep and enables you to have both the original image and the highres fix image. Extra tip, don't put your denoising above .5 - .6 or it will start to alter the image more.

[0xdevalias]

A friend mentioned that using xformers could make things non-deterministic, and that there were a lot of references to it on the repo issues here. Wanting to understand a bit more about it, and link a bunch of potentially related issues together, I tried to find as many issues as I could that seemed to be related to xformers and the potential for it to be causing non-deterministic / unstable / inconsistent results:

The following may potentially be related (ordered by issue number):

Issues:

• Xformers optimization causes inconsistent generations with the same parameters #1999
• No token limit for prompts broken? Different results with the same prompts and seed #2005
• The same image can no longer be output under the same conditions. #2197
• On the problem that the same resolution can not be generated stably #2293
• Different results after swapping graphics card #2423
• Different images for same seed with xformers and "Highres. fix" enabled when ancestral samplers are used #2528
• Different outputs since WebUI update #2943
• xformers and euler_a variant results for same generation settings #3054
• [Bug]: The prompts do not produce the same results as before the latest updates on 10/22/22 #3476
• [Bug]: xformers always generate different results, seed is the same! #3967
• [Bug]: In img2img, setting denoising strength to 0 does not return original image #4011

Discussions:

• Different results after swapping graphics card #2418
• --xformers CMD line option causes slightly poorer quality images on GTX 1650 (Turing), possibly others? #2592
• Highres.fix composition #2705
• Exactly same prompt no more the same ? #2849
• xformers option has good and bad aspects #2958
• What has changed recently? Different and Worse Results from Same Settings #3461

I also came across this thread in the xformers repo, which while I can't guarantee is related, am wondering if it might be:

• xformers ViT-B ImageNet MAE + Deepnorm training instability facebookresearch/xformers#219

And a question I raised on a PR in the diffusers repo:

• Up to 2x speedup on GPUs using memory efficient attention huggingface/diffusers#532 (comment)

  • A friend just mentioned that apparently using xformers tends to lower the quality of generated images, and apparently even causes StableDiffusion to generate different images for the same seed/settings. I haven't looked too deeply into things to try and validate that, but I was wondering if a) that is something that is currently known/documented, and b) if that's likely to be fixed at any point, or if it's just 'the price we pay' for the benefits it adds. Would any of that be a concern with integrating things here? I did stumble upon this issue on the xformers repo, but not sure if it would be the same/similar root cause as my friend described: xformers ViT-B ImageNet MAE + Deepnorm training instability facebookresearch/xformers#219

[ClashSAN]

useful big records, we all like this, thanks.
this may be something to include in the wiki.

[fdwr]

Ooh, thanks for the info, as non-determinism will make me steer clear of it (it's already pretty fast, and I prefer the same settings yield the same result).

[Ehplodor]

Nobody mentionned the "scale latent" option. Have you tried with/without this option enabled ? (all other things being equal)