Stable Diffusion 3 support #16030

AUTOMATIC1111 · 2024-06-16T05:15:51Z

Description

initial SD3 support
can load sd3_medium.safetensors from https://huggingface.co/stabilityai/stable-diffusion-3-medium
will download CLIP models from huggingface into models/CLIP directory
code for neural network, with minimal changes, taken from https://github.com/mcmonkey4eva/sd3-ref/
added support for live previews with all three methods: cheap, approx-nn and TAESD
uses a lot of VRAM
I impersonated mcmonkey4eva in the commit a7116aa so that the code is marked as contributed as him in git blame
TODO:

add support for infinite length prompts
add support for using optimized attention (currently uses SDP regardless of the setting, with some einsums inside) (do we even need that?)
add support for not loading T5
add support for counting tokens
add support for DDIM and other timesteps profilers
investigate why DPM++ 2M Karras isn't working well
go through generation with profiler
support img2img
support medvram

Screenshots/videos:

…/sd3-ref/

protector131090 · 2024-06-16T06:48:16Z

Error on loading model. AttributeError: 'NoneType' object has no attribute 'is_sdxl'

AUTOMATIC1111 · 2024-06-16T06:49:49Z

protector131090: Share the full stack trace from the console.

protector131090 · 2024-06-16T06:53:42Z

protector131090: Share the full stack trace from the console.

Traceback (most recent call last):
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1429, in process_api
inputs = self.preprocess_data(fn_index, inputs, state)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1222, in preprocess_data
self.validate_inputs(fn_index, inputs)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1209, in validate_inputs
raise ValueError(
ValueError: An event handler (modelmerger) didn't receive enough input values (needed: 16, got: 0).
Check if the event handler calls a Javascript function, and make sure its return value is correct.
Wanted inputs:
[label, dropdown, dropdown, dropdown, radio, slider, checkbox, textbox, radio, radio, dropdown, textbox, checkbox, checkbox, checkbox, textbox]
Received inputs:
[]
Traceback (most recent call last):
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1429, in process_api
inputs = self.preprocess_data(fn_index, inputs, state)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1222, in preprocess_data
self.validate_inputs(fn_index, inputs)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1209, in validate_inputs
raise ValueError(
ValueError: An event handler (modelmerger) didn't receive enough input values (needed: 16, got: 0).
Check if the event handler calls a Javascript function, and make sure its return value is correct.
Wanted inputs:
[label, dropdown, dropdown, dropdown, radio, slider, checkbox, textbox, radio, radio, dropdown, textbox, checkbox, checkbox, checkbox, textbox]
Received inputs:
[]

AUTOMATIC1111 · 2024-06-16T06:57:21Z

Okay, well, you're trying to merge models, not generate images.

protector131090 · 2024-06-16T07:07:34Z

Okay, well, you're trying to merge models, not generate images.

what do you mean? i switch model in UI like usual. Here is switch to epic realism - generate - switch to Realvis 4 - generate and switch to sd3 medium :

Applying attention optimization: xformers... done.
Model loaded in 8.6s (load weights from disk: 0.2s, create model: 0.4s, apply weights to model: 7.4s, move model to device: 0.4s).
Using already loaded model epicrealism_naturalSinRC1VAE.safetensors [84d76a0328]: done in 3.6s (send model to cpu: 2.6s, send model to device: 0.9s)
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.45it/s]
Using already loaded model realvisxlV40_v40Bakedvae.safetensors [912c9dc74f]: done in 3.5s (send model to cpu: 0.7s, send model to device: 2.8s)
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00, 4.13it/s]
Reusing loaded model epicrealism_naturalSinRC1VAE.safetensors [84d76a0328] to load comfy\sd3_medium.safetensors [cc236278d2]
Loading weights [cc236278d2] from C:\sd.webui\copy\models\Stable-diffusion\comfy\sd3_medium.safetensors
Creating model from config: C:\sd.webui\copy\configs\sd3-inference.yaml
mmdit initializing with: input_size=None, patch_size=2, in_channels=16, depth=24, mlp_ratio=4.0, learn_sigma=False, adm_in_channels=2048, context_embedder_config={'target': 'torch.nn.Linear', 'params': {'in_features': 4096, 'out_features': 1536}}, register_length=0, attn_mode='torch', rmsnorm=False, scale_mod_only=False, swiglu=False, out_channels=None, pos_embed_scaling_factor=None, pos_embed_offset=None, pos_embed_max_size=192, num_patches=36864, qk_norm=None, qkv_bias=True, dtype=torch.float16, device='cpu'
creating model quickly: TypeError
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\threading.py", line 973, in _bootstrap
self._bootstrap_inner()
File "C:\Program Files\Python310\lib\threading.py", line 1016, in _bootstrap_inner
self.run()
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "C:\sd.webui\copy\modules\ui_settings.py", line 316, in
fn=lambda value, k=k: self.run_settings_single(value, key=k),
File "C:\sd.webui\copy\modules\ui_settings.py", line 95, in run_settings_single
if value is None or not opts.set(key, value):
File "C:\sd.webui\copy\modules\options.py", line 165, in set
option.onchange()
File "C:\sd.webui\copy\modules\call_queue.py", line 14, in f
res = func(*args, **kwargs)
File "C:\sd.webui\copy\modules\initialize_util.py", line 181, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
File "C:\sd.webui\copy\modules\sd_models.py", line 969, in reload_model_weights
load_model(checkpoint_info, already_loaded_state_dict=state_dict)
File "C:\sd.webui\copy\modules\sd_models.py", line 809, in load_model
sd_model = instantiate_from_config(sd_config.model, state_dict)
File "C:\sd.webui\copy\modules\sd_models.py", line 764, in instantiate_from_config
return constructor(**params)
File "C:\sd.webui\copy\modules\models\sd3\sd3_model.py", line 136, in init
self.cond_stage_model = SD3Cond()
File "C:\sd.webui\copy\modules\models\sd3\sd3_model.py", line 61, in init
self.tokenizer = SD3Tokenizer()
File "C:\sd.webui\copy\modules\models\sd3\other_impls.py", line 207, in init
self.t5xxl = T5XXLTokenizer()
File "C:\sd.webui\copy\modules\models\sd3\other_impls.py", line 305, in init
super().init(pad_with_end=False, tokenizer=T5TokenizerFast.from_pretrained("google/t5-v1_1-xxl"), has_start_token=False, pad_to_max_length=False, max_length=99999999, min_length=77)
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1825, in from_pretrained
return cls._from_pretrained(
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1988, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs)
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\models\t5\tokenization_t5_fast.py", line 133, in init
super().init(
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\tokenization_utils_fast.py", line 114, in init
fast_tokenizer = convert_slow_tokenizer(slow_tokenizer)
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\convert_slow_tokenizer.py", line 1307, in convert_slow_tokenizer
return converter_class(transformer_tokenizer).converted()
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\convert_slow_tokenizer.py", line 445, in init
from .utils import sentencepiece_model_pb2 as model_pb2
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\utils\sentencepiece_model_pb2.py", line 91, in
_descriptor.EnumValueDescriptor(
File "C:\sd.webui\copy\venv\lib\site-packages\google\protobuf\descriptor.py", line 789, in new
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.If you cannot immediately regenerate your protos, some other possible workarounds are:

Downgrade the protobuf package to 3.20.x or lower.
Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Failed to create model quickly; will retry using slow method.
mmdit initializing with: input_size=None, patch_size=2, in_channels=16, depth=24, mlp_ratio=4.0, learn_sigma=False, adm_in_channels=2048, context_embedder_config={'target': 'torch.nn.Linear', 'params': {'in_features': 4096, 'out_features': 1536}}, register_length=0, attn_mode='torch', rmsnorm=False, scale_mod_only=False, swiglu=False, out_channels=None, pos_embed_scaling_factor=None, pos_embed_offset=None, pos_embed_max_size=192, num_patches=36864, qk_norm=None, qkv_bias=True, dtype=torch.float16, device='cpu'
changing setting sd_model_checkpoint to comfy\sd3_medium.safetensors [cc236278d2]: TypeError
Traceback (most recent call last):
File "C:\sd.webui\copy\modules\options.py", line 165, in set
option.onchange()
File "C:\sd.webui\copy\modules\call_queue.py", line 14, in f
res = func(*args, **kwargs)
File "C:\sd.webui\copy\modules\initialize_util.py", line 181, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
File "C:\sd.webui\copy\modules\sd_models.py", line 969, in reload_model_weights
load_model(checkpoint_info, already_loaded_state_dict=state_dict)
File "C:\sd.webui\copy\modules\sd_models.py", line 818, in load_model
sd_model = instantiate_from_config(sd_config.model, state_dict)
File "C:\sd.webui\copy\modules\sd_models.py", line 764, in instantiate_from_config
return constructor(**params)
File "C:\sd.webui\copy\modules\models\sd3\sd3_model.py", line 136, in init
self.cond_stage_model = SD3Cond()
File "C:\sd.webui\copy\modules\models\sd3\sd3_model.py", line 61, in init
self.tokenizer = SD3Tokenizer()
File "C:\sd.webui\copy\modules\models\sd3\other_impls.py", line 207, in init
self.t5xxl = T5XXLTokenizer()
File "C:\sd.webui\copy\modules\models\sd3\other_impls.py", line 305, in init
super().init(pad_with_end=False, tokenizer=T5TokenizerFast.from_pretrained("google/t5-v1_1-xxl"), has_start_token=False, pad_to_max_length=False, max_length=99999999, min_length=77)
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1825, in from_pretrained
return cls._from_pretrained(
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1988, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs)
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\models\t5\tokenization_t5_fast.py", line 133, in init
super().init(
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\tokenization_utils_fast.py", line 114, in init
fast_tokenizer = convert_slow_tokenizer(slow_tokenizer)
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\convert_slow_tokenizer.py", line 1307, in convert_slow_tokenizer
return converter_class(transformer_tokenizer).converted()
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\convert_slow_tokenizer.py", line 445, in init
from .utils import sentencepiece_model_pb2 as model_pb2
File "C:\sd.webui\copy\venv\lib\site-packages\transformers\utils\sentencepiece_model_pb2.py", line 28, in
DESCRIPTOR = _descriptor.FileDescriptor(
File "C:\sd.webui\copy\venv\lib\site-packages\google\protobuf\descriptor.py", line 1072, in new
return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: Couldn't build proto file into descriptor pool: duplicate file name sentencepiece_model.proto

Traceback (most recent call last):
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
result = await self.call_function(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\sd.webui\copy\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 273, in refresh_models
new_model_list = self.get_model_list()
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 269, in get_model_list
return [f for f in os.listdir(model_dir) if f != ".gitkeep" and not any(tag in f for tag in get_sd_rm_tag())]
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 269, in
return [f for f in os.listdir(model_dir) if f != ".gitkeep" and not any(tag in f for tag in get_sd_rm_tag())]
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 261, in get_sd_rm_tag
if shared.sd_model.is_sdxl:
AttributeError: 'NoneType' object has no attribute 'is_sdxl'
Traceback (most recent call last):
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
result = await self.call_function(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\sd.webui\copy\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 273, in refresh_models
new_model_list = self.get_model_list()
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 269, in get_model_list
return [f for f in os.listdir(model_dir) if f != ".gitkeep" and not any(tag in f for tag in get_sd_rm_tag())]
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 269, in
return [f for f in os.listdir(model_dir) if f != ".gitkeep" and not any(tag in f for tag in get_sd_rm_tag())]
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 261, in get_sd_rm_tag
if shared.sd_model.is_sdxl:
AttributeError: 'NoneType' object has no attribute 'is_sdxl'

AUTOMATIC1111 · 2024-06-16T07:14:20Z

Well, that now does look like the right stack trace. Can you try running with environment variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (i e add set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python to webui-user.bat) to see if that changes anything?

protector131090 · 2024-06-16T07:24:36Z

Well, that now does look like the right stack trace. Can you try running with environment variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (i e add set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python to webui-user.bat) to see if that changes anything?

@echo off

set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS=--xformers --ckpt-dir "S:\SD CHECKPOINTS" --disable-safe-unpickle --api
set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python

call webui.bat

venv "C:\sd.webui\copy\venv\Scripts\Python.exe"
Python 3.10.10 (tags/v3.10.10:aad5f6a, Feb 7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)]
Version: v1.9.4-173-g7ee2114c
Commit hash: 7ee2114
[Auto-Photoshop-SD] Attempting auto-update...
[Auto-Photoshop-SD] switch branch to extension branch.
checkout_result: Your branch is up to date with 'origin/master'.

[Auto-Photoshop-SD] Current Branch.
branch_result: * master

[Auto-Photoshop-SD] Fetch upstream.
fetch_result:
[Auto-Photoshop-SD] Pull upstream.
pull_result: Already up to date.
Launching Web UI with arguments: --xformers --ckpt-dir S:\SD CHECKPOINTS --disable-safe-unpickle --api
python_server_full_path: C:\sd.webui\copy\extensions\Auto-Photoshop-StableDiffusion-Plugin\server/python_server
[-] ADetailer initialized. version: 24.5.1, num models: 10
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
ControlNet preprocessor location: C:\sd.webui\webui\extensions\sd-webui-controlnet\annotator\downloads
2024-06-16 10:23:22,252 - ControlNet - INFO - ControlNet v1.1.449
Loading weights [912c9dc74f] from S:\SD CHECKPOINTS\realvisxlV40_v40Bakedvae.safetensors
Creating model from config: C:\sd.webui\copy\repositories\generative-models\configs\inference\sd_xl_base.yaml
Applying attention optimization: xformers... done.
Model loaded in 3.5s (load weights from disk: 0.3s, create model: 0.3s, apply weights to model: 2.5s, calculate empty prompt: 0.1s).
2024-06-16 10:23:26,357 - ControlNet - INFO - ControlNet UI callback registered.
C:\sd.webui\copy\extensions\sd-webui-additional-networks\scripts\metadata_editor.py:399: GradioDeprecationWarning: The style method is deprecated. Please set these arguments in the constructor instead.
with gr.Row().style(equal_height=False):
C:\sd.webui\copy\extensions\sd-webui-additional-networks\scripts\metadata_editor.py:521: GradioDeprecationWarning: The style method is deprecated. Please set these arguments in the constructor instead.
cover_image = gr.Image(
Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
Startup time: 25.6s (prepare environment: 11.0s, import torch: 3.0s, import gradio: 0.8s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.4s, load scripts: 2.7s, create ui: 5.3s, gradio launch: 0.3s, add APIs: 0.9s).
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.59it/s]
Loading model comfy\sd3_medium.safetensors [cc236278d2] (2 out of 2)
Loading weights [cc236278d2] from C:\sd.webui\copy\models\Stable-diffusion\comfy\sd3_medium.safetensors
Creating model from config: C:\sd.webui\copy\configs\sd3-inference.yaml
mmdit initializing with: input_size=None, patch_size=2, in_channels=16, depth=24, mlp_ratio=4.0, learn_sigma=False, adm_in_channels=2048, context_embedder_config={'target': 'torch.nn.Linear', 'params': {'in_features': 4096, 'out_features': 1536}}, register_length=0, attn_mode='torch', rmsnorm=False, scale_mod_only=False, swiglu=False, out_channels=None, pos_embed_scaling_factor=None, pos_embed_offset=None, pos_embed_max_size=192, num_patches=36864, qk_norm=None, qkv_bias=True, dtype=torch.float16, device='cpu'
changing setting sd_model_checkpoint to comfy\sd3_medium.safetensors [cc236278d2]: OutOfMemoryError
Traceback (most recent call last):
File "C:\sd.webui\copy\modules\options.py", line 165, in set
option.onchange()
File "C:\sd.webui\copy\modules\call_queue.py", line 14, in f
res = func(*args, **kwargs)
File "C:\sd.webui\copy\modules\initialize_util.py", line 181, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
File "C:\sd.webui\copy\modules\sd_models.py", line 950, in reload_model_weights
sd_model = reuse_model_from_already_loaded(sd_model, checkpoint_info, timer)
File "C:\sd.webui\copy\modules\sd_models.py", line 916, in reuse_model_from_already_loaded
load_model(checkpoint_info)
File "C:\sd.webui\copy\modules\sd_models.py", line 834, in load_model
load_model_weights(sd_model, checkpoint_info, state_dict, timer)
File "C:\sd.webui\copy\modules\sd_models.py", line 436, in load_model_weights
model.load_state_dict(state_dict, strict=False)
File "C:\sd.webui\copy\modules\sd_disable_initialization.py", line 223, in
module_load_state_dict = self.replace(torch.nn.Module, 'load_state_dict', lambda *args, **kwargs: load_state_dict(module_load_state_dict, *args, **kwargs))
File "C:\sd.webui\copy\modules\sd_disable_initialization.py", line 221, in load_state_dict
original(module, state_dict, strict=strict)
File "C:\sd.webui\copy\venv\lib\site-packages\torch\nn\modules\module.py", line 2138, in load_state_dict
load(self, state_dict)
File "C:\sd.webui\copy\venv\lib\site-packages\torch\nn\modules\module.py", line 2126, in load
load(child, child_state_dict, child_prefix)
File "C:\sd.webui\copy\venv\lib\site-packages\torch\nn\modules\module.py", line 2126, in load
load(child, child_state_dict, child_prefix)
File "C:\sd.webui\copy\venv\lib\site-packages\torch\nn\modules\module.py", line 2126, in load
load(child, child_state_dict, child_prefix)
[Previous line repeated 7 more times]
File "C:\sd.webui\copy\venv\lib\site-packages\torch\nn\modules\module.py", line 2120, in load
module._load_from_state_dict(
File "C:\sd.webui\copy\modules\sd_disable_initialization.py", line 225, in
linear_load_from_state_dict = self.replace(torch.nn.Linear, '_load_from_state_dict', lambda *args, **kwargs: load_from_state_dict(linear_load_from_state_dict, *args, **kwargs))
File "C:\sd.webui\copy\modules\sd_disable_initialization.py", line 191, in load_from_state_dict
module._parameters[name] = torch.nn.parameter.Parameter(torch.zeros_like(param, device=device, dtype=dtype), requires_grad=param.requires_grad)
File "C:\sd.webui\copy\venv\lib\site-packages\torch_meta_registrations.py", line 4507, in zeros_like
res = aten.empty_like.default(
File "C:\sd.webui\copy\venv\lib\site-packages\torch_ops.py", line 448, in call
return self.op(*args, **kwargs or {})
File "C:\sd.webui\copy\venv\lib\site-packages\torch_refs_init.py", line 4681, in empty_like
return torch.empty_permuted(
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 160.00 MiB. GPU 0 has a total capacty of 23.99 GiB of which 0 bytes is free. Of the allocated memory 22.79 GiB is allocated by PyTorch, and 184.33 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Traceback (most recent call last):
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
result = await self.call_function(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\sd.webui\copy\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 273, in refresh_models
new_model_list = self.get_model_list()
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 269, in get_model_list
return [f for f in os.listdir(model_dir) if f != ".gitkeep" and not any(tag in f for tag in get_sd_rm_tag())]
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 269, in
return [f for f in os.listdir(model_dir) if f != ".gitkeep" and not any(tag in f for tag in get_sd_rm_tag())]
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 261, in get_sd_rm_tag
if shared.sd_model.is_sdxl:
AttributeError: 'NoneType' object has no attribute 'is_sdxl'
Traceback (most recent call last):
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
result = await self.call_function(
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\sd.webui\copy\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "C:\sd.webui\copy\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\sd.webui\copy\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 273, in refresh_models
new_model_list = self.get_model_list()
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 269, in get_model_list
return [f for f in os.listdir(model_dir) if f != ".gitkeep" and not any(tag in f for tag in get_sd_rm_tag())]
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 269, in
return [f for f in os.listdir(model_dir) if f != ".gitkeep" and not any(tag in f for tag in get_sd_rm_tag())]
File "C:\sd.webui\copy\extensions\sd-webui-animatediff\scripts\animatediff_ui.py", line 261, in get_sd_rm_tag
if shared.sd_model.is_sdxl:
AttributeError: 'NoneType' object has no attribute 'is_sdxl'

…wants float16

AUTOMATIC1111 · 2024-06-16T08:06:21Z

protector131090: The problem in your stack trace seems to be that you're running out of VRAM.

I found out that there is a hump in VRAM usage when loading the model, going over 24GB. On my machine I have a setting that automatically offloads that to RAM, so it worked for me.

Anyway, the fix for that is very simple, and now VRAM use hovers around 15.3GB during model load and after it.

protector131090 · 2024-06-16T09:22:06Z

protector131090: The problem in your stack trace seems to be that you're running out of VRAM.

I found out that there is a hump in VRAM usage when loading the model, going over 24GB. On my machine I have a setting that automatically offloads that to RAM, so it worked for me.

Anyway, the fix for that is very simple, and now VRAM use hovers around 15.3GB during model load and after it.

model does load now. but i get tgis error:

venv "C:\sd.webui\SD3\webui\venv\Scripts\Python.exe"
Python 3.10.10 (tags/v3.10.10:aad5f6a, Feb 7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)]
Version: v1.9.4-174-gb443fdcf
Commit hash: b443fdc
Launching Web UI with arguments: --xformers --ckpt-dir S:\SD CHECKPOINTS --disable-safe-unpickle --api
Loading weights [cc236278d2] from C:\sd.webui\SD3\webui\models\Stable-diffusion\comfy\sd3_medium.safetensors
Creating model from config: C:\sd.webui\SD3\webui\configs\sd3-inference.yaml
mmdit initializing with: input_size=None, patch_size=2, in_channels=16, depth=24, mlp_ratio=4.0, learn_sigma=False, adm_in_channels=2048, context_embedder_config={'target': 'torch.nn.Linear', 'params': {'in_features': 4096, 'out_features': 1536}}, register_length=0, attn_mode='torch', rmsnorm=False, scale_mod_only=False, swiglu=False, out_channels=None, pos_embed_scaling_factor=None, pos_embed_offset=None, pos_embed_max_size=192, num_patches=36864, qk_norm=None, qkv_bias=True, dtype=torch.float16, device='cpu'
C:\sd.webui\SD3\webui\venv\lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True.
warnings.warn(
Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
Startup time: 10.6s (prepare environment: 2.0s, import torch: 3.6s, import gradio: 0.8s, setup paths: 1.0s, initialize shared: 0.3s, other imports: 0.7s, load scripts: 1.0s, create ui: 0.4s, gradio launch: 0.4s, add APIs: 0.2s).
C:\sd.webui\SD3\webui\venv\lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True.
warnings.warn(
Applying attention optimization: xformers... done.
Model loaded in 9.6s (load weights from disk: 0.1s, create model: 1.9s, apply weights to model: 2.5s, apply half(): 0.1s, load weights from state dict: 4.5s, move model to device: 0.1s, calculate empty prompt: 0.3s).
50%|█████████████████████████████████████████ | 10/20 [00:00<00:00, 14.69it/s]C:\sd.webui\SD3\webui\modules\sd_samplers_common.py:68: RuntimeWarning: invalid value encountered in cast:00, 15.97it/s]
x_sample = x_sample.astype(np.uint8)
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 13.61it/s]
*** Error completing request███████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 13.35it/s]
*** Arguments: ('task(vu43rbt2pr3n4pt)', <gradio.routes.Request object at 0x0000024206328190>, '', '', [], 1, 1, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', 'Use same scheduler', '', '', [], 0, 20, 'DPM++ 2M', 'Automatic', False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False) {}
Traceback (most recent call last):
File "C:\sd.webui\SD3\webui\modules\call_queue.py", line 58, in f
res = list(func(*args, **kwargs))
File "C:\sd.webui\SD3\webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "C:\sd.webui\SD3\webui\modules\txt2img.py", line 109, in txt2img
processed = processing.process_images(p)
File "C:\sd.webui\SD3\webui\modules\processing.py", line 847, in process_images
res = process_images_inner(p)
File "C:\sd.webui\SD3\webui\modules\processing.py", line 995, in process_images_inner
devices.test_for_nans(samples_ddim, "unet")
File "C:\sd.webui\SD3\webui\modules\devices.py", line 265, in test_for_nans
raise NansException(message)
modules.devices.NansException: A tensor with NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.

tritant · 2024-06-16T09:48:42Z

Hello, I get an error when I try to load the model

To create a public link, set share=True in launch().
Startup time: 23.8s (prepare environment: 7.8s, import torch: 3.0s, import gradio: 0.7s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.5s, load scripts: 3.0s, create ui: 1.6s, gradio launch: 2.4s, add APIs: 3.0s, app_started_callback: 0.8s).
Applying attention optimization: xformers... done.
Model loaded in 8.3s (load weights from disk: 0.7s, create model: 0.4s, apply weights to model: 5.4s, apply half(): 1.2s, move model to device: 0.3s, calculate empty prompt: 0.3s).
Reusing loaded model CreaPrompt_Ultimate_Mix2.safetensors [fc814f22af] to load sd3_medium.safetensors [cc236278d2]
Loading weights [cc236278d2] from C:\sd.webui\webui\models\Stable-diffusion\sd3_medium.safetensors
Creating model from config: C:\sd.webui\webui\configs\sd3-inference.yaml
mmdit initializing with: input_size=None, patch_size=2, in_channels=16, depth=24, mlp_ratio=4.0, learn_sigma=False, adm_in_channels=2048, context_embedder_config={'target': 'torch.nn.Linear', 'params': {'in_features': 4096, 'out_features': 1536}}, register_length=0, attn_mode='torch', rmsnorm=False, scale_mod_only=False, swiglu=False, out_channels=None, pos_embed_scaling_factor=None, pos_embed_offset=None, pos_embed_max_size=192, num_patches=36864, qk_norm=None, qkv_bias=True, dtype=torch.float16, device='cpu'
creating model quickly: TypeError
Traceback (most recent call last):
File "C:\Users\jice\AppData\Local\Programs\Python\Python310\lib\threading.py", line 973, in _bootstrap
self._bootstrap_inner()
File "C:\Users\jice\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1016, in _bootstrap_inner
self.run()
File "C:\sd.webui\webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\sd.webui\webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "C:\sd.webui\webui\modules\ui_settings.py", line 316, in
fn=lambda value, k=k: self.run_settings_single(value, key=k),
File "C:\sd.webui\webui\modules\ui_settings.py", line 95, in run_settings_single
if value is None or not opts.set(key, value):
File "C:\sd.webui\webui\modules\options.py", line 165, in set
option.onchange()
File "C:\sd.webui\webui\modules\call_queue.py", line 14, in f
res = func(*args, **kwargs)
File "C:\sd.webui\webui\modules\initialize_util.py", line 181, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
File "C:\sd.webui\webui\modules\sd_models.py", line 970, in reload_model_weights
load_model(checkpoint_info, already_loaded_state_dict=state_dict)
File "C:\sd.webui\webui\modules\sd_models.py", line 810, in load_model
sd_model = instantiate_from_config(sd_config.model, state_dict)
File "C:\sd.webui\webui\modules\sd_models.py", line 765, in instantiate_from_config
return constructor(**params)
File "C:\sd.webui\webui\modules\models\sd3\sd3_model.py", line 136, in init
self.cond_stage_model = SD3Cond()
File "C:\sd.webui\webui\modules\models\sd3\sd3_model.py", line 61, in init
self.tokenizer = SD3Tokenizer()
File "C:\sd.webui\webui\modules\models\sd3\other_impls.py", line 207, in init
self.t5xxl = T5XXLTokenizer()
File "C:\sd.webui\webui\modules\models\sd3\other_impls.py", line 305, in init
super().init(pad_with_end=False, tokenizer=T5TokenizerFast.from_pretrained("google/t5-v1_1-xxl"), has_start_token=False, pad_to_max_length=False, max_length=99999999, min_length=77)
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1825, in from_pretrained
return cls._from_pretrained(
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1988, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\models\t5\tokenization_t5_fast.py", line 133, in init
super().init(
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\tokenization_utils_fast.py", line 114, in init
fast_tokenizer = convert_slow_tokenizer(slow_tokenizer)
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\convert_slow_tokenizer.py", line 1307, in convert_slow_tokenizer
return converter_class(transformer_tokenizer).converted()
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\convert_slow_tokenizer.py", line 445, in init
from .utils import sentencepiece_model_pb2 as model_pb2
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\utils\sentencepiece_model_pb2.py", line 91, in
_descriptor.EnumValueDescriptor(
File "C:\sd.webui\webui\venv\lib\site-packages\google\protobuf\descriptor.py", line 789, in new
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

Downgrade the protobuf package to 3.20.x or lower.
Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Failed to create model quickly; will retry using slow method.
mmdit initializing with: input_size=None, patch_size=2, in_channels=16, depth=24, mlp_ratio=4.0, learn_sigma=False, adm_in_channels=2048, context_embedder_config={'target': 'torch.nn.Linear', 'params': {'in_features': 4096, 'out_features': 1536}}, register_length=0, attn_mode='torch', rmsnorm=False, scale_mod_only=False, swiglu=False, out_channels=None, pos_embed_scaling_factor=None, pos_embed_offset=None, pos_embed_max_size=192, num_patches=36864, qk_norm=None, qkv_bias=True, dtype=torch.float16, device='cpu'
changing setting sd_model_checkpoint to sd3_medium.safetensors [cc236278d2]: TypeError
Traceback (most recent call last):
File "C:\sd.webui\webui\modules\options.py", line 165, in set
option.onchange()
File "C:\sd.webui\webui\modules\call_queue.py", line 14, in f
res = func(*args, **kwargs)
File "C:\sd.webui\webui\modules\initialize_util.py", line 181, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
File "C:\sd.webui\webui\modules\sd_models.py", line 970, in reload_model_weights
load_model(checkpoint_info, already_loaded_state_dict=state_dict)
File "C:\sd.webui\webui\modules\sd_models.py", line 819, in load_model
sd_model = instantiate_from_config(sd_config.model, state_dict)
File "C:\sd.webui\webui\modules\sd_models.py", line 765, in instantiate_from_config
return constructor(**params)
File "C:\sd.webui\webui\modules\models\sd3\sd3_model.py", line 136, in init
self.cond_stage_model = SD3Cond()
File "C:\sd.webui\webui\modules\models\sd3\sd3_model.py", line 61, in init
self.tokenizer = SD3Tokenizer()
File "C:\sd.webui\webui\modules\models\sd3\other_impls.py", line 207, in init
self.t5xxl = T5XXLTokenizer()
File "C:\sd.webui\webui\modules\models\sd3\other_impls.py", line 305, in init
super().init(pad_with_end=False, tokenizer=T5TokenizerFast.from_pretrained("google/t5-v1_1-xxl"), has_start_token=False, pad_to_max_length=False, max_length=99999999, min_length=77)
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1825, in from_pretrained
return cls._from_pretrained(
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1988, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\models\t5\tokenization_t5_fast.py", line 133, in init
super().init(
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\tokenization_utils_fast.py", line 114, in init
fast_tokenizer = convert_slow_tokenizer(slow_tokenizer)
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\convert_slow_tokenizer.py", line 1307, in convert_slow_tokenizer
return converter_class(transformer_tokenizer).converted()
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\convert_slow_tokenizer.py", line 445, in init
from .utils import sentencepiece_model_pb2 as model_pb2
File "C:\sd.webui\webui\venv\lib\site-packages\transformers\utils\sentencepiece_model_pb2.py", line 28, in
DESCRIPTOR = _descriptor.FileDescriptor(
File "C:\sd.webui\webui\venv\lib\site-packages\google\protobuf\descriptor.py", line 1072, in new
return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: Couldn't build proto file into descriptor pool: duplicate file name sentencepiece_model.proto

AUTOMATIC1111 · 2024-06-16T09:52:30Z

I added protobuf==3.20.0 to requirements which hopefully will make the problem go away.

protector131090 · 2024-06-16T10:00:50Z

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 11.80it/s]
*** Error completing request
*** Arguments: ('task(5kbarr62htczn7w)', <gradio.routes.Request object at 0x0000012DF24243D0>, '', '', [], 1, 1, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', 'Use same scheduler', '', '', [], 0, 20, 'DPM++ 2M', 'Automatic', False, '', 0.8, -1, False, -1, 0, 0, 0, False, 'MultiDiffusion', False, True, 1024, 1024, 96, 96, 48, 4, 'None', 2, False, 10, 1, 1, 64, False, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 'DemoFusion', False, 128, 64, 4, 2, False, 10, 1, 1, 64, False, True, 3, 1, 1, True, 0.85, 0.6, 4, False, False, 3072, 192, True, True, True, False, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False) {}
Traceback (most recent call last):
File "C:\sd.webui\SD3\webui\modules\call_queue.py", line 58, in f
res = list(func(*args, **kwargs))
File "C:\sd.webui\SD3\webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "C:\sd.webui\SD3\webui\modules\txt2img.py", line 109, in txt2img
processed = processing.process_images(p)
File "C:\sd.webui\SD3\webui\modules\processing.py", line 847, in process_images
res = process_images_inner(p)
File "C:\sd.webui\SD3\webui\modules\processing.py", line 995, in process_images_inner
devices.test_for_nans(samples_ddim, "unet")
File "C:\sd.webui\SD3\webui\modules\devices.py", line 265, in test_for_nans
raise NansException(message)
modules.devices.NansException: A tensor with NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.

but with --no-half i get OOM ....

protector131090 · 2024-06-16T10:10:01Z

--disable-nan-check --precision half Fixed it for me!

tritant · 2024-06-16T10:17:53Z

I added protobuf==3.20.0 to requirements which hopefully will make the problem go away.

Yes, it works for me now thank you very much

rltgjqmcpgjadyd · 2024-06-16T10:53:27Z

I have same NaNs issue & same solution
--disable-nan-check is not need

silvertuanzi · 2024-06-16T11:02:33Z

Downloading clip from SAI's original repo suffers 401 HTTP error: Unauthorized
So maybe we need a download source which doesn't need logging in.

By the way, consider to use t5xxl_fp8_e4m3fn.safetensors instead of t5xxl_fp16.safetensors ?
I tried to rename t5xxl_fp8_e4m3fn.safetensors to t5xxl_fp16.safetensors, put it into models/CLIP, and it works well.

JasperG · 2024-06-16T11:08:54Z

--disable-nan-check --precision half Fixed it for me!

Note that this must be seen as a work-around; This will make SDXL output black as this makes the use of --no-half-vae impossible (contradicting).

AUTOMATIC1111 · 2024-06-16T11:10:36Z

pushed an update to make it possible to work without --precision half.

ananosleep: well, shit... It worked for me.

rltgjqmcpgjadyd · 2024-06-16T11:38:34Z

ok it worked without --precision half

AUTOMATIC1111 · 2024-06-16T11:40:38Z

ananosleep: Changed download links to a public huggingface repository.

tritant · 2024-06-16T11:41:36Z

It works for me, but on an rtx 4060ti 16GB it's really very slow. To compare A1111(t5fp8) 7 minutes for 1024x1024 and with comfyui 30 seconds for 1024x1024

light-and-ray · 2024-06-28T06:12:41Z

rtx 3060, T5 + medvram: uses 11.4 GB for the first generation with 1 prompt, any further generations with the same prompt use 6.1 GB. So there is a place for optimization

Btw, can you add an option to run T5 only on cpu

protector131090 · 2024-06-28T06:32:37Z

THANK YOU! for fixing img2img ! that is amazing!!!!

protector131090 · 2024-06-28T07:00:03Z

Long prompts give me this error. How do i fix it? thanks.
UPD --precision half fixes it.

100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:10<00:00, 2.73it/s]
*** Error completing request
*** Arguments: ('task(vs0etxqaz3pd71e)', <gradio.routes.Request object at 0x0000026E43BCBD60>, '(documentary photography:1.4) photo of an elf woman, with deep, emerald green eyes, long, braided, dark brown hair, wearing a simple, brown leather tunic and pants, with a worn leather pouch on her hip, gazing intently into the distance, standing amidst a dense, ancient forest, under the soft, dappled light filtering through the leaves, shot at eye level on a Leica M3, with a 50mm lens, (film grain:1.1), in the style of Roberto Ferri.', '', [], 1, 1, 4, 1024, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', 'Use same scheduler', '', '', [], 0, 28, 'DPM++ 2M', 'SGM Uniform', False, '', 0.8, -1, False, -1, 0, 0, 0, True, False, False, False, 'base', False, False, {'ad_model': 'face_yolov8n.pt', 'ad_model_classes': '', 'ad_tab_enable': True, 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M', 'ad_scheduler': 'Use same scheduler', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_tab_enable': True, 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M', 'ad_scheduler': 'Use same scheduler', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_tab_enable': True, 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M', 'ad_scheduler': 'Use same scheduler', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_tab_enable': True, 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M', 'ad_scheduler': 'Use same scheduler', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_tab_enable': True, 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M', 'ad_scheduler': 'Use same scheduler', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_tab_enable': True, 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M', 'ad_scheduler': 'Use same scheduler', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False, 'Not set', True, True, '', '', '', '', '', 1.3, 'Not set', 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', False, 'None') {}
Traceback (most recent call last):
File "C:\sd.webui\SD3\webui\modules\call_queue.py", line 58, in f
res = list(func(*args, **kwargs))
File "C:\sd.webui\SD3\webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "C:\sd.webui\SD3\webui\modules\txt2img.py", line 109, in txt2img
processed = processing.process_images(p)
File "C:\sd.webui\SD3\webui\modules\processing.py", line 847, in process_images
res = process_images_inner(p)
File "C:\sd.webui\SD3\webui\modules\processing.py", line 995, in process_images_inner
devices.test_for_nans(samples_ddim, "unet")
File "C:\sd.webui\SD3\webui\modules\devices.py", line 265, in test_for_nans
raise NansException(message)
modules.devices.NansException: A tensor with NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.

…lated incorrectly

tritant · 2024-06-28T10:43:39Z

Hires.fix not working with sd3
100%|██████████████████████████████████████████████████████████████████████████████████| 28/28 [00:21<00:00, 1.28it/s]
tiled upscale: 100%|███████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
0%| | 0/5 [00:01<?, ?it/s]
*** Error completing request
RuntimeError: The size of tensor a (123) must match the size of tensor b (122) at non-singleton dimension 3

tritant · 2024-06-28T16:31:39Z

when an sd3 checkpoint is selected and the browser page is refreshed, the token counter is set to zero, the prompt must be modified for it to be calculated

tritant · 2024-06-29T06:53:59Z

Since the last update, I can no longer load an SD3 model with --medvram

Loading weights [cc236278d2] from C:\sd.webui\webui\models\Stable-diffusion\sd3_medium.safetensors
Creating model from config: C:\sd.webui\webui\configs\sd3-inference.yaml
changing setting sd_model_checkpoint to sd3_medium.safetensors [cc236278d2]: AttributeError
Traceback (most recent call last):
File "C:\sd.webui\webui\modules\options.py", line 165, in set
option.onchange()
File "C:\sd.webui\webui\modules\call_queue.py", line 14, in f
res = func(*args, **kwargs)
File "C:\sd.webui\webui\modules\initialize_util.py", line 181, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
File "C:\sd.webui\webui\modules\sd_models.py", line 977, in reload_model_weights
load_model(checkpoint_info, already_loaded_state_dict=state_dict)
File "C:\sd.webui\webui\modules\sd_models.py", line 849, in load_model
send_model_to_device(sd_model)
File "C:\sd.webui\webui\modules\sd_models.py", line 756, in send_model_to_device
lowvram.apply(m)
File "C:\sd.webui\webui\modules\lowvram.py", line 29, in apply
setup_for_low_vram(sd_model, not shared.cmd_opts.lowvram)
File "C:\sd.webui\webui\modules\lowvram.py", line 100, in setup_for_low_vram
setattr(obj, field, None)
File "C:\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1754, in setattr
super().setattr(name, value)
AttributeError: can't set attribute 'cond_stage_model'

kevkevinpal · 2024-06-30T15:32:15Z

investigate why DPM++ 2M Karras isn't working well

Trying to debug into this, but unsure what you mean by "isn't working well" I was able to get an image

drhead · 2024-07-02T04:41:10Z

investigate why DPM++ 2M Karras isn't working well

Karras schedule doesn't really make much sense to use on a flow model, it would be best to use Uniform. DPM++ 2M itself should work fine, but as far as the Karras schedule goes you can consider it to be as out of scope as SDE samplers (which will not work well, and you can't make them work well).

ch-zgh-1993 · 2024-07-04T12:37:41Z

Hello, When I try to load the model, meet this error.

Stable diffusion model failed to load
Loading weights [cc236278d2] from /mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/models/Stable-diffusion/sd3_medium.safetensors
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/ui_extra_networks.py", line 392, in pages_html
    return refresh()
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/ui_extra_networks.py", line 398, in refresh
    pg.refresh()
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/ui_extra_networks_textual_inversion.py", line 13, in refresh
    sd_hijack.model_hijack.embedding_db.load_textual_inversion_embeddings(force_reload=True)
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 255, in load_textual_inversion_embeddings
    self.expected_shape = self.get_expected_shape()
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 154, in get_expected_shape
    vec = shared.sd_model.cond_stage_model.encode_embedding_init_text(",", 1)
AttributeError: 'NoneType' object has no attribute 'cond_stage_model'
Creating model from config: /mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/configs/v1-inference.yaml
loading stable diffusion model: RuntimeError
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/ui.py", line 1298, in <lambda>
    update_image_cfg_scale_visibility = lambda: gr.update(visible=shared.sd_model and shared.sd_model.cond_stage_key == "edit")
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/shared_items.py", line 110, in sd_model
    return modules.sd_models.model_data.get_sd_model()
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/sd_models.py", line 499, in get_sd_model
    load_model()
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/sd_models.py", line 626, in load_model
    load_model_weights(sd_model, checkpoint_info, state_dict, timer)
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/sd_models.py", line 353, in load_model_weights
    model.load_state_dict(state_dict, strict=False)
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/sd_disable_initialization.py", line 223, in <lambda>
    module_load_state_dict = self.replace(torch.nn.Module, 'load_state_dict', lambda *args, **kwargs: load_state_dict(module_load_state_dict, *args, **kwargs))
  File "/mnt/workspace/demos/stable_diffusion_webui/stable-diffusion-webui/modules/sd_disable_initialization.py", line 221, in load_state_dict
    original(module, state_dict, strict=strict)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
	size mismatch for first_stage_model.encoder.conv_out.weight: copying a param with shape torch.Size([32, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([8, 512, 3, 3]).
	size mismatch for first_stage_model.encoder.conv_out.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([8]).
	size mismatch for first_stage_model.decoder.conv_in.weight: copying a param with shape torch.Size([512, 16, 3, 3]) from checkpoint, the shape in current model is torch.Size([512, 4, 3, 3]).


Stable diffusion model failed to load

zhenhao-huang · 2024-07-05T07:47:22Z

venv "X:\sd.webui\webui\venv\Scripts\Python.exe"
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Version: v1.9.4
Commit hash: feee37d75f1b168768014e4634dcb156ee649c05
removing nvidia-cudnn-cu11
Launching Web UI with arguments: --api --listen --enable-insecure-extension-access
2024-06-16 21:48:03.357201: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-06-16 21:48:04.056059: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
WARNING:xformers:WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 1.13.1+cu117 with CUDA 1107 (you have 2.1.2+cu121)
    Python  3.10.9 (you have 3.10.9)
  Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details
No module 'xformers'. Proceeding without it.
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
Using sqlite file: X:\sd.webui\webui\extensions\sd-webui-agent-scheduler\task_scheduler.sqlite3
== WD14 tagger /gpu:0, uname_result(system='Windows', node='DESKTOP-464LIJO', release='10', version='10.0.22631', machine='AMD64') ==
Loading weights [ac17f32d24] from X:\sd.webui\webui\models\Stable-diffusion\XL\tPonynai3_v51WeightOptimized.safetensors
Creating model from config: X:\sd.webui\webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 91.1s (prepare environment: 30.4s, import torch: 4.1s, import gradio: 1.1s, setup paths: 3.1s, initialize shared: 0.3s, other imports: 0.9s, list SD models: 0.1s, load scripts: 38.3s, scripts before_ui_callback: 0.3s, create ui: 6.4s, gradio launch: 4.3s, add APIs: 0.5s, app_started_callback: 1.1s).
Applying attention optimization: sdp... done.
Model loaded in 30.0s (load weights from disk: 1.2s, create model: 0.7s, apply weights to model: 26.9s, move model to device: 0.2s, load textual inversion embeddings: 0.3s, calculate empty prompt: 0.6s).
Reusing loaded model XL\tPonynai3_v51WeightOptimized.safetensors [ac17f32d24] to load SD3\stableDiffusion3SD3_sd3Medium.safetensors [cc236278d2]
Loading weights [cc236278d2] from X:\sd.webui\webui\models\Stable-diffusion\SD3\stableDiffusion3SD3_sd3Medium.safetensors
Creating model from config: X:\sd.webui\webui\configs\v1-inference.yaml
changing setting sd_model_checkpoint to SD3\stableDiffusion3SD3_sd3Medium.safetensors [cc236278d2]: RuntimeError
Traceback (most recent call last):
  File "X:\sd.webui\webui\modules\options.py", line 165, in set
    option.onchange()
  File "X:\sd.webui\webui\modules\call_queue.py", line 13, in f
    res = func(*args, **kwargs)
  File "X:\sd.webui\webui\modules\initialize_util.py", line 181, in <lambda>
    shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
  File "X:\sd.webui\webui\modules\sd_models.py", line 879, in reload_model_weights
    load_model(checkpoint_info, already_loaded_state_dict=state_dict)
  File "X:\sd.webui\webui\modules\sd_models.py", line 748, in load_model
    load_model_weights(sd_model, checkpoint_info, state_dict, timer)
  File "X:\sd.webui\webui\modules\sd_models.py", line 393, in load_model_weights
    model.load_state_dict(state_dict, strict=False)
  File "X:\sd.webui\webui\modules\sd_disable_initialization.py", line 223, in <lambda>
    module_load_state_dict = self.replace(torch.nn.Module, 'load_state_dict', lambda *args, **kwargs: load_state_dict(module_load_state_dict, *args, **kwargs))
  File "X:\sd.webui\webui\modules\sd_disable_initialization.py", line 221, in load_state_dict
    original(module, state_dict, strict=strict)
  File "X:\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
        size mismatch for first_stage_model.encoder.conv_out.weight: copying a param with shape torch.Size([32, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([8, 512, 3, 3]).
        size mismatch for first_stage_model.encoder.conv_out.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([8]).
        size mismatch for first_stage_model.decoder.conv_in.weight: copying a param with shape torch.Size([512, 16, 3, 3]) from checkpoint, the shape in current model is torch.Size([512, 4, 3, 3]).

The model does not load.

I also have it, how to solve it

tritant · 2024-07-06T08:49:23Z

https://stability.ai/news/license-update

steve84wien · 2024-07-10T16:15:08Z

Hey, It would be amazing to add SD3 LoRA support

light-and-ray · 2024-07-20T10:44:02Z

It has been added in last RC commits @steve84wien

protector131090 · 2024-07-27T15:47:36Z

*** Error running postprocess: C:\sd.webui\webui\extensions\sd-webui-kohya-hiresfix\scripts\khrfix.py
Traceback (most recent call last):
File "C:\sd.webui\webui\modules\scripts.py", line 872, in postprocess
script.postprocess(p, processed, *script_args)
File "C:\sd.webui\webui\extensions\sd-webui-kohya-hiresfix\scripts\khrfix.py", line 135, in postprocess
for i, b in enumerate(p.sd_model.model.diffusion_model.input_blocks):
File "C:\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1695, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'MMDiT' object has no attribute 'input_blocks'

lijiajun1997 · 2024-07-29T17:38:09Z

Failed to create model quickly; will retry using slow method.
changing setting sd_model_checkpoint to sd3_medium.safetensors [cc236278d2]: OSError
Traceback (most recent call last):
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\options.py", line 165, in set
option.onchange()
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\call_queue.py", line 14, in f
res = func(*args, **kwargs)
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\initialize_util.py", line 181, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: sd_models.reload_model_weights()), call=False)
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\sd_models.py", line 977, in reload_model_weights
load_model(checkpoint_info, already_loaded_state_dict=state_dict)
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\sd_models.py", line 829, in load_model
sd_model = instantiate_from_config(sd_config.model, state_dict)
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\sd_models.py", line 775, in instantiate_from_config
return constructor(**params)
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\models\sd3\sd3_model.py", line 34, in init
self.text_encoders = SD3Cond()
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\models\sd3\sd3_cond.py", line 164, in init
self.tokenizer = SD3Tokenizer()
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\models\sd3\other_impls.py", line 221, in init
self.t5xxl = T5XXLTokenizer()
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\modules\models\sd3\other_impls.py", line 317, in init
super().init(pad_with_end=False, tokenizer=T5TokenizerFast.from_pretrained("google/t5-v1_1-xxl"), has_start_token=False, pad_to_max_length=False, max_length=99999999, min_length=77)
File "D:\SD\sd-webui-aki-v4(1)\sd-webui-aki-v4\py310\lib\site-packages\transformers\tokenization_utils_base.py", line 1809, in from_pretrained
raise EnvironmentError(
OSError: Can't load tokenizer for 'google/t5-v1_1-xxl'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'google/t5-v1_1-xxl' is the correct path to a directory containing all relevant files for a T5TokenizerFast tokenizer.

Zac12138 · 2024-08-01T10:01:01Z

@lijiajun1997 Ensure that you can access Hugging Face or download file locally.

mswcmps · 2024-08-08T09:12:15Z

When I do a manual upscale in the extras tab, I get this error in the console

*** Error running postprocess: C:\sd.webui\webui\scripts\khrfix.py Traceback (most recent call last): File "C:\sd.webui\webui\modules\scripts.py", line 872, in postprocess script.postprocess(p, processed, *script_args) File "C:\sd.webui\webui\scripts\khrfix.py", line 101, in postprocess for i, b in enumerate(p.sd_model.model.diffusion_model.input_blocks): File "C:\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1695, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: 'MMDiT' object has no attribute 'input_blocks'

same error

mcmonkey4eva and others added 4 commits June 16, 2024 07:13

add SD3 reference implementation from https://github.com/mcmonkey4eva…

a7116aa

…/sd3-ref/

initial SD3 support

5b2a60b

linter

79de09c

typo

7ee2114

prevent accidental creation of CLIP models in float32 type when user …

b443fdc

…wants float16

add protobuf==3.20.0 to requirements

80f618e

fix NaN issue when running without --precision half

06d0a5a

change CLIP links to allow anonymous downloading

58dc35a

change t5xxl checkpoint to fp8

d4b814a

AUTOMATIC1111 added 3 commits June 28, 2024 07:51

get deepbooru to run with --precision-half

06fe174

get T5 to work both with and without --precision half

fc8b126

add the missing get_first_stage_encoding function

0c7bdcc

fix img2img

0b64633

fix the problem with infinite prompts where empty cond would be calcu…

179ae47

…lated incorrectly

allow generation to be started with any dimensions specified

d67348a

AUTOMATIC1111 added 3 commits June 29, 2024 00:38

support loading clip/t5 from the main model checkpoint

7e4b06f

do sampler calculations on CPU

1394eca

remove AutocastLinear from SD3's MLP

ebe8be9

fix --medvram

9e404c3

AUTOMATIC1111 merged commit c02e3a5 into dev Jul 6, 2024
6 checks passed

AUTOMATIC1111 deleted the sd3 branch July 6, 2024 06:39

lawchingman mentioned this pull request Oct 5, 2024

### Features: #16535

Closed

Stable Diffusion 3 support #16030

Stable Diffusion 3 support #16030

Conversation

AUTOMATIC1111 commented Jun 16, 2024 • edited Loading

Description

Screenshots/videos:

protector131090 commented Jun 16, 2024 • edited Loading

AUTOMATIC1111 commented Jun 16, 2024

protector131090 commented Jun 16, 2024

AUTOMATIC1111 commented Jun 16, 2024

protector131090 commented Jun 16, 2024 • edited Loading

AUTOMATIC1111 commented Jun 16, 2024

protector131090 commented Jun 16, 2024 • edited Loading

AUTOMATIC1111 commented Jun 16, 2024 • edited Loading

protector131090 commented Jun 16, 2024

tritant commented Jun 16, 2024

AUTOMATIC1111 commented Jun 16, 2024

protector131090 commented Jun 16, 2024

protector131090 commented Jun 16, 2024

tritant commented Jun 16, 2024

rltgjqmcpgjadyd commented Jun 16, 2024 • edited Loading

silvertuanzi commented Jun 16, 2024 • edited Loading

JasperG commented Jun 16, 2024 • edited Loading

AUTOMATIC1111 commented Jun 16, 2024

rltgjqmcpgjadyd commented Jun 16, 2024

AUTOMATIC1111 commented Jun 16, 2024

tritant commented Jun 16, 2024 • edited Loading

light-and-ray commented Jun 28, 2024

protector131090 commented Jun 28, 2024

protector131090 commented Jun 28, 2024 • edited Loading

tritant commented Jun 28, 2024

tritant commented Jun 28, 2024

tritant commented Jun 29, 2024 • edited Loading

kevkevinpal commented Jun 30, 2024

drhead commented Jul 2, 2024

ch-zgh-1993 commented Jul 4, 2024

zhenhao-huang commented Jul 5, 2024

tritant commented Jul 6, 2024

steve84wien commented Jul 10, 2024

light-and-ray commented Jul 20, 2024

protector131090 commented Jul 27, 2024

lijiajun1997 commented Jul 29, 2024

Zac12138 commented Aug 1, 2024

mswcmps commented Aug 8, 2024

AUTOMATIC1111 commented Jun 16, 2024 •

edited

Loading

protector131090 commented Jun 16, 2024 •

edited

Loading

protector131090 commented Jun 16, 2024 •

edited

Loading

protector131090 commented Jun 16, 2024 •

edited

Loading

AUTOMATIC1111 commented Jun 16, 2024 •

edited

Loading

rltgjqmcpgjadyd commented Jun 16, 2024 •

edited

Loading

silvertuanzi commented Jun 16, 2024 •

edited

Loading

JasperG commented Jun 16, 2024 •

edited

Loading

tritant commented Jun 16, 2024 •

edited

Loading

protector131090 commented Jun 28, 2024 •

edited

Loading

tritant commented Jun 29, 2024 •

edited

Loading