Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flux.1 image generation has a relatively unfinished image output that inferenced on iGPU than CPU #28445

Closed
JamieVC opened this issue Jan 15, 2025 · 5 comments · Fixed by #28828
Assignees
Labels
bug Something isn't working category: GPU OpenVINO GPU plugin
Milestone

Comments

@JamieVC
Copy link

JamieVC commented Jan 15, 2025

Describe the bug
Flux.1 image generation has a relatively unfinished image output that inferenced on iGPU than CPU.

Config for iGPU inference and the output:

from optimum.intel.openvino import OVDiffusionPipeline
model_dir = model_base_dir / "INT4" if use_quantized_models.value else model_base_dir / "FP16"
ov_config = {"INFERENCE_PRECISION_HINT" : "f16", "ACTIVATIONS_SCALE_FACTOR": "8.0", "CACHE_DIR" : ""}
ov_pipe = OVDiffusionPipeline.from_pretrained(model_dir, device=device.value, ov_config=ov_config)

Image

Config for CPU inference and the output:

from optimum.intel.openvino import OVDiffusionPipeline
model_dir = model_base_dir / "INT4" if use_quantized_models.value else model_base_dir / "FP16" 
ov_pipe = OVDiffusionPipeline.from_pretrained(model_dir, device=device.value)
Image

Expected behavior
Output that inferenced on iGPU should also complete a sign with the whole texts.

Installation instructions (Please mark the checkbox)
[ yes ] I followed the installation guide at https://github.com/openvinotoolkit/openvino_notebooks#-installation-guide to install the notebooks.

@eaidova eaidova transferred this issue from openvinotoolkit/openvino_notebooks Jan 15, 2025
@eaidova eaidova added the category: GPU OpenVINO GPU plugin label Jan 15, 2025
@andrei-kochin andrei-kochin added the bug Something isn't working label Jan 15, 2025
@dnkurek
Copy link
Contributor

dnkurek commented Jan 15, 2025

What version of OpenVINO are you using?

@geunhwan
Copy link

geunhwan commented Jan 16, 2025

This is known issue to be fixed. We seem to have an issue in sdpa. (cc. @e-ddykim)

@JamieVC
Copy link
Author

JamieVC commented Jan 16, 2025

onnx 1.17.0
openai 1.57.1
opencv-python 4.10.0.84
openvino 2025.0.0 (local Build with the PR #27265 )
openvino-genai 2024.6.0.0rc3
openvino-telemetry 2024.1.0
openvino-tokenizers 2024.6.0.0
optimum 1.24.0.dev0
optimum-intel 1.22.0.dev0+58aec63

@JamieVC
Copy link
Author

JamieVC commented Feb 4, 2025

sdpa

hi @geunhwan
Can I know what issue# in internal JIRA system?

@e-ddykim
Copy link
Contributor

e-ddykim commented Feb 5, 2025

Hi @JamieVC, #28828 is ready for review.

github-merge-queue bot pushed a commit that referenced this issue Feb 5, 2025
### Details:
- Add support of per-head attention mask for sdpa_micro kernel, as some
models have different mask configurations for each head

### Tickets:
 - CVS-161777
 - Closes #28445
@mlukasze mlukasze added this to the 2025.1 milestone Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working category: GPU OpenVINO GPU plugin
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants