Some RGBA images cause broken backgrounds due to incorrect RGB conversion

[city96]

Currently, the load_image function in train_utils uses the PIL image.convert("RGB") method for converting the loaded 4 RGBA channel images to the expected 3 channel RGB format.

This is incorrect for some edge cases, and causes the background to become distorted.

As an example, loading this RGBA image results in heavy background artifacts. when used with the test code below:

from PIL import Image
from library.train_util import load_image

img = load_image("test.png")
Image.fromarray(img, 'RGB').save("test_convert.png")

To verify this issue, a LoRA was created using that single image as the dataset, using a caption that includes the words "white background". Seed and everything else was fixed between the two tests.

The top row is the latest code using a fresh VENV. The bottom row is the proposed fix, which I will create a PR for shortly. It uses the Image.alpha_composite function with a new blank white image to handle the alpha channel.

[feffy380]

They're not artifacts. Those colors really exist in the png, they're just fully transparent. Dropping the alpha channel makes them visible again. Most image editors automatically set transparent pixels to black but for some reason the artist disabled that.

Automatically setting them to a solid color is still a good idea, but I just wanted to point out the garbage data comes from the PNG itself

[city96]

@feffy380 Interesting, seems like you're right. I kind-of just assumed it was some funky conversion logic but I guess that explains it.

I've created a quick script to check for hidden/corrupted backgrounds.

Currently testing it against my smaller 300K dataset to see what comes up. The current hit rate is 101 out of ~10000 images, only considering PNGs, so seems pretty rare.

[kohya-ss]

This is an intended behavior as feffy380 wrote. I think a new option may be needed if we change this, because it is a breaking change.

#1223 is still draft, but it will be a good solution for this.

[city96]

@kohya-ss Thank you. That solution is indeed more optimal, I'll close my PR and will follow the one you linked instead.
Leaving the issue open until that PR gets merged in case someone else runs into this same behavior.