TGATE v0.1.1 encounter ValueError when performing multiple forward inferences

[colorjam]

Hi! Thank you for the amazing work.

I encounter the ValueError when performing multiple forward inferences:

Here's the testing code I used:

pipe = TgateSDLoader(
            pipe,
            gate_step=gate_step,
            num_inference_steps=inference_step
       ).to("cuda")
start_time = time.time()
for _ in range(infer_times):
    tagate_image = pipe.tgate(
          prompt,
          gate_step=gate_step,
          num_inference_steps=inference_step
      ).images    
    latency = (time.time() - start_time) / infer_times
    logging.info("T-GATE: {:.2f} seconds".format(latency))

Hope you can resolve this issue.

[greasebig]

same.
ValueError: not enough values to unpack (expected 2, got 1)

[greasebig]

1731 if ip_adapter_image is not None or ip_adapter_image_embeds is not None:
1732 added_cond_kwargs["image_embeds"] = image_embeds
...
--> 184 hidden_uncond, hidden_pred_text = hidden_states.chunk(2)
185 cache = (hidden_uncond + hidden_pred_text ) / 2
187 if input_ndim == 4:

ValueError: not enough values to unpack (expected 2, got 1)

[greasebig]

your T-GATE can only be used in sd1.5 sd2.1

[HaozheLiu-ST]

@colorjam which model and which pipeline do you use? We will shortly have a try in our local node.

[HaozheLiu-ST]

your T-GATE can only be used in sd1.5 sd2.1

There might be some customized pipelines based on the same model. However, they may also revise the __call__(); we encourage the community to contribute TGATE to these methods. We cannot support all methods by ourselves.
This is also why we provide simple code, where developers can easily integrate it into their code base.

We sincerely thank you for your attention, recognition, understanding, and potential contribution.

[colorjam]

@HaozheLiu-ST StableDiffusionPipeline and https://huggingface.co/runwayml/stable-diffusion-v1-5, the entire code is as follows:

pipe = StableDiffusionPipeline.from_pretrained(model, torch_dtype=torch.float16).to("cuda:0")
gate_step = 8
n_steps = 25
pipe = TgateSDLoader(
        pipe,
        gate_step=gate_step,
        num_inference_steps=n_steps
    ).to("cuda")

for _ in range(infer_times):
    tagate_image = pipe.tgate(
        prompt,
        gate_step=gate_step,
        num_inference_steps=n_steps
    ).images    

[colorjam]

@HaozheLiu-ST I found some issues with the current code:

1. cur_step will be set to 0 from the second inference, but by default it starts from 1.
   

   T-GATE/SD_1_5/customize_attention.py

   Line 47 in 0c88804

   self.cur_step = (self.cur_step + 1) % inference_num_per_image

2. cur_step is divided by inference_num_per_image, which may not equal to the actual inference steps _num_timesteps

   T-GATE/SD_1_5/stable_diffusion.py

   Line 1038 in 0c88804

   self._num_timesteps = len(timesteps)

[WentianZhang-ML]

Hi @colorjam and @greasebig, thanks again for your attention. We have addressed this issue.

Please update the tgate to version 0.1.2 by pip install -U tgate and try again.

[WentianZhang-ML]

your T-GATE can only be used in sd1.5 sd2.1

Hi @greasebig, if you still have problems with these pipelines, please give me your completed code and log files.

BTW, if you want to contribute TGATE to other pipelines, please feel free to open a PR. We welcome all contributions from the community.