Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local inference image support #836

Merged
merged 10 commits into from
Nov 4, 2024
Merged

Local inference image support #836

merged 10 commits into from
Nov 4, 2024

Conversation

joein
Copy link
Member

@joein joein commented Oct 30, 2024

  • refactor and extend tests
  • define INFERENCE_OBJECT_TYPES as a constant in embed

Copy link

netlify bot commented Oct 30, 2024

Deploy Preview for poetic-froyo-8baba7 ready!

Name Link
🔨 Latest commit aa81592
🔍 Latest deploy log https://app.netlify.com/sites/poetic-froyo-8baba7/deploys/672548e5b01dfc00085b22fe
😎 Deploy Preview https://deploy-preview-836--poetic-froyo-8baba7.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@joein joein force-pushed the local-inference-image-support branch from 3c30b10 to 39ea3f8 Compare October 31, 2024 16:11
@joein
Copy link
Member Author

joein commented Oct 31, 2024

#829

embedding_model_inst = self._get_or_init_image_model(
model_name=model_name, **(image.options or {})
)
image_data = base64.b64decode(image.image)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does seem like a lot of overhead and not really a user-friendly interface, but I understand why it is done like this (I assume, cause we want local remove work the same)

Copy link
Member

@generall generall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for this PR, but we would need to discuss how to simplify image selection, as asking users to read image from file and encode it info base64 is not going to fly well

@joein
Copy link
Member Author

joein commented Nov 1, 2024

Not for this PR, but we would need to discuss how to simplify image selection, as asking users to read image from file and encode it info base64 is not going to fly well

We can convert to base64 on our own, but we would either need PIL for that, or find some other ways
We would also need an extended image model (because models.Image only accepts strings)

* new: add inference object support

* new: add inference object support

* fix: remove redundant import

* refactor: return newline

* fix: fix propagate options test
@joein joein merged commit d2771af into dev Nov 4, 2024
9 of 14 checks passed
joein added a commit that referenced this pull request Jan 16, 2025
* new: add backbone for image support

* new: convert b64 to pil, embed images, add test

* tests: add test file

* refactor: replace 3 different inference object vars with a common one

* fix: fix type hints

* fix: fix type hints

* tests: add tests

* fix: remove redundant imports

* new: propagate image options

* Custom inference object (#837)

* new: add inference object support

* new: add inference object support

* fix: remove redundant import

* refactor: return newline

* fix: fix propagate options test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants