Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix wrong endpoint for tei embedding gaudi wrapper #1043

Merged
merged 5 commits into from
Dec 18, 2024
Merged

Fix wrong endpoint for tei embedding gaudi wrapper #1043

merged 5 commits into from
Dec 18, 2024

Conversation

Spycsh
Copy link
Member

@Spycsh Spycsh commented Dec 18, 2024

Description

access to embedding wrapper should use /v1/embeddings with {"input", "xxx"}

access to / should use {"inputs": "xxx"} but return are not openai compatible

>>> TEI_EMBEDDING_ENDPOINT
'http://localhost:8090/v1/embeddings'
>>> client = AsyncInferenceClient(model=TEI_EMBEDDING_ENDPOINT, token=HUGGINGFACEHUB_API_TOKEN)
>>> rr=asyncio.run(client.post(json={"input": "Explain the OPEA project"}))
>>> rr
b'{"object":"list","data":[{"object":"embedding","embedding":[-0.0006848863,-0.015310476,-0.037823148,0.011144044,0.042748522,0.0222726,-0.0041277106,0.052072655,-0.011957196,-0.010230217,0.006164461,-0.004530414,-0.07267249,0.036800902,-0.00455364

Issues

opea-project/GenAIExamples#1263

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

None

Tests

UT

@Spycsh Spycsh requested a review from lkk12014402 December 18, 2024 06:59
@Spycsh Spycsh changed the title fix doc index retriever embed issue on gaudi fix wrong endpoint for tei embedding gaudi wrapper Dec 18, 2024
@Spycsh Spycsh changed the title fix wrong endpoint for tei embedding gaudi wrapper Fix wrong endpoint for tei embedding gaudi wrapper Dec 18, 2024
@Spycsh Spycsh requested a review from chensuyue December 18, 2024 07:17
@Spycsh
Copy link
Member Author

Spycsh commented Dec 18, 2024

@chensuyue this pr should fix the two failures in opea-project/GenAIExamples#1263.

@lkk12014402
Copy link
Collaborator

Description

access to embedding wrapper should use /v1/embeddings with {"input", "xxx"}

access to / should use {"inputs": "xxx"} but return are not openai compatible

>>> TEI_EMBEDDING_ENDPOINT
'http://localhost:8090/v1/embeddings'
>>> client = AsyncInferenceClient(model=TEI_EMBEDDING_ENDPOINT, token=HUGGINGFACEHUB_API_TOKEN)
>>> rr=asyncio.run(client.post(json={"input": "Explain the OPEA project"}))
>>> rr
b'{"object":"list","data":[{"object":"embedding","embedding":[-0.0006848863,-0.015310476,-0.037823148,0.011144044,0.042748522,0.0222726,-0.0041277106,0.052072655,-0.011957196,-0.010230217,0.006164461,-0.004530414,-0.07267249,0.036800902,-0.00455364

Issues

opea-project/GenAIExamples#1263

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

None

Tests

UT

yes, we should use tei /v1/embeddings api, I add this usage at the readme

image

lkk12014402
lkk12014402 approved these changes Dec 18, 2024
View reviewed changes
Copy link
Collaborator

@lkk12014402 lkk12014402 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lkk12014402 lkk12014402 merged commit 70c151d into opea-project:main Dec 18, 2024
13 checks passed
@joshuayao joshuayao linked an issue Dec 20, 2024 that may be closed by this pull request
[Bug] 2 examples failed with aiohttp.client_exceptions.ContentTypeError in nightly test opea-project/GenAIExamples#1263
Closed
6 tasks
smguggen pushed a commit to opea-aws-proserve/GenAIComps that referenced this pull request Jan 23, 2025
@Spycsh @pre-commit-ci @smguggen
Fix wrong endpoint for tei embedding gaudi wrapper (opea-project#1043)
9696f08 
* fix doc index retriever embed issue on gaudi

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* align test router with examples

* align readme

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lkk12014402 lkk12014402 lkk12014402 approved these changes

@letonghan letonghan letonghan approved these changes

@lvliang-intel lvliang-intel Awaiting requested review from lvliang-intel lvliang-intel is a code owner

@chensuyue chensuyue Awaiting requested review from chensuyue chensuyue is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug] 2 examples failed with aiohttp.client_exceptions.ContentTypeError in nightly test
3 participants
@Spycsh @lkk12014402 @letonghan