Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix wrong endpoint for tei embedding gaudi wrapper #1043

Merged
merged 5 commits into from
Dec 18, 2024
Merged

Fix wrong endpoint for tei embedding gaudi wrapper #1043

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6a77762
fix doc index retriever embed issue on gaudi
Spycsh Dec 18, 2024
64d91a7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 18, 2024
bc672c1
align test router with examples
Spycsh Dec 18, 2024
9e80a5d
Merge branch 'fix_doc_index_retriever' of https://github.com/Spycsh/G…
Spycsh Dec 18, 2024
cd059f4
align readme
Spycsh Dec 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions comps/embeddings/tei/langchain/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ curl localhost:$your_port/v1/embeddings \
Start the embedding service with the TEI_EMBEDDING_ENDPOINT.

```bash
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport/v1/embeddings"
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport"
export TEI_EMBEDDING_MODEL_NAME="BAAI/bge-large-en-v1.5"
python embedding_tei.py
```
Expand Down Expand Up @@ -71,7 +71,7 @@ curl localhost:$your_port/embed/v1/embeddings \
Export the `TEI_EMBEDDING_ENDPOINT` for later usage:

```bash
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport/v1/embeddings"
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport"
export TEI_EMBEDDING_MODEL_NAME="BAAI/bge-large-en-v1.5"
```

Expand Down
4 changes: 3 additions & 1 deletion comps/embeddings/tei/langchain/embedding_tei.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,9 @@ async def aembed_query(request: Dict, async_client: AsyncInferenceClient) -> Uni

def get_async_inference_client(access_token: str) -> AsyncInferenceClient:
headers = {"Authorization": f"Bearer {access_token}"} if access_token else {}
return AsyncInferenceClient(model=TEI_EMBEDDING_ENDPOINT, token=HUGGINGFACEHUB_API_TOKEN, headers=headers)
return AsyncInferenceClient(
model=f"{TEI_EMBEDDING_ENDPOINT}/v1/embeddings", token=HUGGINGFACEHUB_API_TOKEN, headers=headers
)


if __name__ == "__main__":
Expand Down
2 changes: 1 addition & 1 deletion tests/embeddings/test_embeddings_tei_langchain.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ function start_service() {
model="BAAI/bge-base-en-v1.5"
unset http_proxy
docker run -d --name="test-comps-embedding-tei-endpoint" -p $tei_endpoint:80 -v ./data:/data --pull always ghcr.io/huggingface/text-embeddings-inference:cpu-1.5 --model-id $model
export TEI_EMBEDDING_ENDPOINT="http://${ip_address}:${tei_endpoint}/v1/embeddings"
export TEI_EMBEDDING_ENDPOINT="http://${ip_address}:${tei_endpoint}"
tei_service_port=5002
docker run -d --name="test-comps-embedding-tei-server" -e LOGFLAG=True -e http_proxy=$http_proxy -e https_proxy=$https_proxy -p ${tei_service_port}:6000 --ipc=host -e TEI_EMBEDDING_ENDPOINT=$TEI_EMBEDDING_ENDPOINT opea/embedding-tei:comps
sleep 3m
Expand Down
Loading