Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

VAE tiling seems to skip the last several tiles during processing #564

Open
lostdisc opened this issue Jan 10, 2025 · 3 comments
Open

VAE tiling seems to skip the last several tiles during processing #564

lostdisc opened this issue Jan 10, 2025 · 3 comments

Comments

@lostdisc
Copy link

With both SD 1.5 and SDXL, running with VAE tiling seems to skip the last several tiles during processing.

With SD 1.5, VAE tiling has 16 tiles, but skips to the end after doing 8. The relevant part of the verbose output looks like this:

[DEBUG] ggml_extend.hpp:549  - tile work buffer size: 0.77 MB
[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
[INFO ] ggml_extend.hpp:563  - processing 16 tiles
  |===>                                              | 1/16 - 0.00it/s[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 1/16 - 2.18s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |======>                                           | 2/16 - 2.20s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=========>                                        | 3/16 - 2.25s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |============>                                     | 4/16 - 2.17s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |===============>                                  | 5/16 - 2.17s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |==================>                               | 6/16 - 2.13s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 7/16 - 2.14s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 8/16 - 2.22s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================================| 16/16 - 2.22s/it
[DEBUG] stable-diffusion.cpp:1080 - computing vae [mode: DECODE] graph completed, taking 21.88s
[INFO ] stable-diffusion.cpp:1487 - latent 1 decoded, taking 21.88s
[INFO ] stable-diffusion.cpp:1491 - decode_first_stage completed, taking 21.88s
[INFO ] stable-diffusion.cpp:1614 - txt2img completed in 57.88s
save result image to 'output.png'

With SDXL, VAE tiling has 64 tiles, but skips to the end after doing 48. The relevant part of the verbose output looks like this:

[DEBUG] ggml_extend.hpp:513  - tile work buffer size: 0.77 MB
[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
[INFO ] ggml_extend.hpp:527  - processing 64 tiles
  |>                                                 | 1/64 - 0.00it/s[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |>                                                 | 1/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=>                                                | 2/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==>                                               | 3/64 - 1.75s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 4/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 5/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====>                                             | 6/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====>                                            | 7/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |======>                                           | 8/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======>                                          | 9/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======>                                          | 10/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |========>                                         | 11/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========>                                        | 12/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========>                                       | 13/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========>                                       | 14/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===========>                                      | 15/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============>                                     | 16/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=============>                                    | 17/64 - 1.72s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============>                                   | 18/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============>                                   | 19/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===============>                                  | 20/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================>                                 | 21/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================>                                | 22/64 - 1.68s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================>                                | 23/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================>                               | 24/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================>                              | 25/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====================>                             | 26/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 27/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 28/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |======================>                           | 29/64 - 1.73s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======================>                          | 30/64 - 1.74s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |========================>                         | 31/64 - 1.73s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 32/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 33/64 - 1.58s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========================>                       | 34/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===========================>                      | 35/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============================>                     | 36/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============================>                     | 37/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=============================>                    | 38/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============================>                   | 39/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===============================>                  | 40/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================================>                 | 41/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================================>                 | 42/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================================>                | 43/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================>               | 44/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================================>              | 45/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================================>              | 46/64 - 1.72s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====================================>             | 47/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================================>            | 48/64 - 1.74s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================================| 64/64 - 1.74s/it
[DEBUG] stable-diffusion.cpp:1045 - computing vae [mode: DECODE] graph completed, taking 83.93s
[INFO ] stable-diffusion.cpp:1416 - latent 1 decoded, taking 83.93s
[INFO ] stable-diffusion.cpp:1420 - decode_first_stage completed, taking 83.93s
[INFO ] stable-diffusion.cpp:1539 - txt2img completed in 346.14s
save result image to 'output.png'

And for visual comparison... First, here is an SD 1.5 image generated with non-tiled VAE:
sd-cpp-vulkan_cat_output

And here is how it comes out with tiled VAE:
sd-cpp-vulkan_vae-tiled_cat_output

The colors look a little different if you flip between the two, but I don't see an obvious boundary in the tiled version, FWIW.
@olivbrau
Copy link

I also noticed that, but have not yet found bad result : my images are correct.
May be it is only a simple error in % calculation ? (because of a bad total ?)

@stduhpf
Copy link
Contributor

stduhpf commented Jan 10, 2025

I'm pretty sure #484 fixes that. This was not the initial goal of the PR, but I can't reproduce this issue with these changes merged in.

@lostdisc
Copy link
Author

Neat, thanks!

@stduhpf stduhpf mentioned this issue Feb 6, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@stduhpf @olivbrau @lostdisc