[GSOC] Add Image-to-Image and Image-to-Text generation with GUI support for OpenVINO GenAI #740
Conversation
- add convert script for model conversion
- part of post process logic - python implmentation of visualization
using the same latent from numpy, and the results are just the same
- support random seed - add remove button for input image
There was a problem hiding this comment.
Could we add the guide for OpenCV installation?
I downloaded opencv 4.9.0 and got cmake error.
log_cmake_error_opencv.txt
There was a problem hiding this comment.
sure, will add guide for opencv.
and I just checked opencv 4.9.0, it works fine.
I'm using "x64 Native Tools Command Prompt for VS 2022" instead of the default cmd.exe to do the build.
and opencv related outputs should like
-- Found Threads: TRUE
-- OpenCV ARCH: x64
-- OpenCV RUNTIME: vc16
-- OpenCV STATIC: OFF
-- Found OpenCV: C:/Users/chuxd/Downloads/opencv/build (found suitable version "4.9.0", minimum required is "4.9.0")
-- Found OpenCV 4.9.0 in C:/Users/chuxd/Downloads/opencv/build/x64/vc16/lib
-- You might need to add C:\Users\chuxd\Downloads\opencv\build\x64\vc16\bin to your PATH to be able to run your applications.
seems OpenCV RUNTIME not found correctlly in your setup.
There was a problem hiding this comment.
add x64\vc16\lib for -DOpenCV_DIR:
cmake -S . -B build -DOpenCV_DIR="C:\Users\S590\Downloads\opencv\build\x64\vc16\lib" -DOpenVINO_DIR="C:\llm\LG\w_openvino_toolkit_windows_2024.3.0.16041.1e3b88e4e3f_x86_64\w_openvino_toolkit_windows_2024.3.0.16041.1e3b88e4e3f_x86_64\runtime\cmake"
There was a problem hiding this comment.
Currently this verification within scripts works on Linux.
Please also provide windows verification.
For windows verification, some modifications are required:
- add requirements.txt for python script and notebook.
- Copy from “stable_diffusion_1_5_controlnet\cpp\scripts\pose.png” to detectors\scripts
- add opencv path for cmake: cmake -S . -B build -DOpenCV_DIR="C:\Users\S590\Downloads\opencv\build\x64\vc16\lib"
- add "set(CMAKE_CXX_STANDARD 17)" into cmakelist
- debug the "GoogleTestAddTests" on Windows
- upload images into ./media
- modify the windows path of exe in verify.ipynb: ../build/Release/detectors_bridge
There was a problem hiding this comment.
To more precisely validate your detector implementation, add more quantitative comparisons of images (pixel level).
|
This blog Stable Diffusion ControlNet Pipeline with OpenVINO™ In C++ has analyzed the differences between full pipelines in C++ and Python with pixel-level comparison and visualization. |
Hi all,
Here is the draft PR for the first part of the GSOC project. This PR includes:
The OpenPose detector implemented in pure C++
A new image-to-image pipeline using ControlNet-OpenPose
I will create new PRs for the remaining tasks, which include the GUI client and the CLIP model in OpenVINO.