Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send Accept: image/png as header for image tasks #1567

Merged
merged 3 commits into from
Jul 26, 2023

Conversation

Wauplin
Copy link
Contributor

@Wauplin Wauplin commented Jul 21, 2023

cc @philschmid

Tasks that expect an image to be returned by the server (i.e. image-to-image and text-to-image) must send Accept: image/png as header by default. This value can be overwritten by the user if needed. This PR fixes a bug happening with Inference Endpoints which returns a base64-encoded image in case Accept header is not set. This doesn't change anything for InferenceAPI calls.

EDIT: also fixes a bug in AsyncInferenceClient where headers set by the user where not taken into account.

NOTE: failing tests are unrelated

@Wauplin Wauplin requested a review from philschmid July 21, 2023 09:37
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jul 21, 2023

The documentation is not available anymore as the PR was closed or merged.

@StephenHodgson
Copy link

I also have not been able to get this to work myself. I think some tasks needed to be updated with the changes I made last month in huggingface/api-inference-community#287.

@Wauplin
Copy link
Contributor Author

Wauplin commented Jul 24, 2023

@StephenHodgson If I understand correctly, this PR and the one you contributed last month are not exactly related. Here it is to fix InferenceClient to make it work with Inference Endpoints by force a default image mime type (so not related to InferenceAPI which doesn't run on the same infra). But agree it could be safer to for a mime type for audio as well (though it can be done in another PR)

@StephenHodgson
Copy link

@Wauplin correct me if I'm wrong, but I'm pretty sure it uses the same infra, last I checked
That's the main reason why I ended up making the PR there because I was trying to prove it out in my Unity client.

@Wauplin
Copy link
Contributor Author

Wauplin commented Jul 26, 2023

@StephenHodgson To clarify things:

  • InferenceAPI is a service to quickly test and evaluate models hosted on the Hub. It's free of charge but only meant for quick prototyping. Docs are here. Part of the source code is public under api-inference-community.
  • InferenceEndpoint is a product to deploy models for production. This is a paid service to setup by the client (1 endpoint = 1 model = 1 GPU to pay). The entire implementation is private.
  • InferenceClient is a client in huggingface_hub that can run inference with both InferenceAPI and InferenceEndpoint. The goal is to be compatible with both, and also endpoints hosted on other providers (AWS Sagemaker, Azure Endpoints,...). At the moment, we've more focused our attention on the HF solutions.

@Wauplin Wauplin merged commit aa58090 into main Jul 26, 2023
@Wauplin Wauplin deleted the fix-header-accept-inference-client branch July 26, 2023 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants