Patient App - TTS Issue

[monkeydust]

Tried using Groq (default) TTS - using same API key for LLM_API_KEY and GROQ_SST_API_KEY but keep getting error below.

I tried changing STT_PROVIDER = "deepgram" and set DG_API_KEY but it always uses Groq and throws same error.

---

rx-orchestrator | initial response sent to STT
2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] INFO: incoming request
2024-09-24 09:35:49 xrx-orchestrator | reqId: "req-1"
2024-09-24 09:35:49 xrx-orchestrator | req: {
2024-09-24 09:35:49 xrx-orchestrator | "method": "GET",
2024-09-24 09:35:49 xrx-orchestrator | "url": "/api/v1/ws",
2024-09-24 09:35:49 xrx-orchestrator | "hostname": "localhost:8000",
2024-09-24 09:35:49 xrx-orchestrator | "remoteAddress": "172.18.0.1",
2024-09-24 09:35:49 xrx-orchestrator | "remotePort": 35752
2024-09-24 09:35:49 xrx-orchestrator | }
2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: preValidation hook
2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: Connection opened
2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: {"host":"localhost:8000","connection":"Upgrade","pragma":"no-cache","cache-control":"no-cache","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0","upgrade":"websocket","origin":"http://localhost:3000","sec-websocket-version":"13","accept-encoding":"gzip, deflate, br, zstd","accept-language":"en-GB,en;q=0.9,en-US;q=0.8","sec-websocket-key":"O5SiDIMkdDvR7cKJQLuidg==","sec-websocket-extensions":"permessage-deflate; client_max_window_bits"}
2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: STT Opening WebSocket:ws://xrx-stt:8001/api/v1/ws
2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: TTS Opening WebSocket:ws://xrx-tts:8002/api/v1/ws
2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: STT WebSocket opened
2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: TTS WebSocket opened
2024-09-24 09:35:51 xrx-client | ⚠ The "images.domains" configuration is deprecated. Please use "images.remotePatterns" configuration instead.
2024-09-24 09:35:51 xrx-client | ⨯ Error: 'sharp' is required to be installed in standalone mode for the image optimization to function correctly. Read more at: https://nextjs.org/docs/messages/sharp-missing-in-production
2024-09-24 09:35:51 xrx-client | ⨯ Failed to write image to cache 8bP9JR6UNW0uOjfukrlwvbAvrsrDZUbVnRb-NJWyjWk= [Error: EACCES: permission denied, mkdir '/nextjs-client/.next/cache'] {
2024-09-24 09:35:51 xrx-client | errno: -13,
2024-09-24 09:35:51 xrx-client | code: 'EACCES',
2024-09-24 09:35:51 xrx-client | syscall: 'mkdir',
2024-09-24 09:35:51 xrx-client | path: '/nextjs-client/.next/cache'
2024-09-24 09:35:51 xrx-client | }
2024-09-24 09:35:57 xrx-stt | INFO:groq_stt:Transcribing audio using Groq API...
2024-09-24 09:35:57 xrx-stt | INFO:groq_stt:Groq model initialized.
2024-09-24 09:35:58 xrx-stt | INFO:httpx:HTTP Request: POST https://api.groq.com/openai/v1/audio/transcriptions "HTTP/1.1 401 Unauthorized"
2024-09-24 09:35:58 xrx-stt | ERROR: Exception in ASGI application
2024-09-24 09:35:58 xrx-stt | Traceback (most recent call last):
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi
2024-09-24 09:35:58 xrx-stt | result = await self.app(self.scope, self.asgi_receive, self.asgi_send) # type: ignore[func-returns-value]
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call
2024-09-24 09:35:58 xrx-stt | return await self.app(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call
2024-09-24 09:35:58 xrx-stt | await super().call(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 123, in call
2024-09-24 09:35:58 xrx-stt | await self.middleware_stack(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 151, in call
2024-09-24 09:35:58 xrx-stt | await self.app(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in call
2024-09-24 09:35:58 xrx-stt | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
2024-09-24 09:35:58 xrx-stt | raise exc
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-09-24 09:35:58 xrx-stt | await app(scope, receive, sender)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 756, in call
2024-09-24 09:35:58 xrx-stt | await self.middleware_stack(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
2024-09-24 09:35:58 xrx-stt | await route.handle(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 373, in handle
2024-09-24 09:35:58 xrx-stt | await self.app(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 96, in app
2024-09-24 09:35:58 xrx-stt | await wrap_app_handling_exceptions(app, session)(scope, receive, send)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
2024-09-24 09:35:58 xrx-stt | raise exc
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-09-24 09:35:58 xrx-stt | await app(scope, receive, sender)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 94, in app
2024-09-24 09:35:58 xrx-stt | await func(session)
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 348, in app
2024-09-24 09:35:58 xrx-stt | await dependant.call(**values)
2024-09-24 09:35:58 xrx-stt | File "/stt/main.py", line 45, in websocket_endpoint
2024-09-24 09:35:58 xrx-stt | result = await stt_model.transcribe(data)
2024-09-24 09:35:58 xrx-stt | File "/stt/groq_stt.py", line 72, in transcribe
2024-09-24 09:35:58 xrx-stt | transcription = model.audio.transcriptions.create(
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/groq/resources/audio/transcriptions.py", line 118, in create
2024-09-24 09:35:58 xrx-stt | return self._post(
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/groq/_base_client.py", line 1225, in post
2024-09-24 09:35:58 xrx-stt | return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/groq/_base_client.py", line 920, in request
2024-09-24 09:35:58 xrx-stt | return self._request(
2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/groq/_base_client.py", line 1018, in _request
2024-09-24 09:35:58 xrx-stt | raise self._make_status_error_from_response(err.response) from None
2024-09-24 09:35:58 xrx-stt | groq.AuthenticationError: Error code: 401 - {'error': {'message': 'Invalid API Key', 'type': 'invalid_request_error', 'code': 'invalid_api_key'}}
2024-09-24 09:35:58 xrx-stt | INFO: connection closed

[chrislott]

Thanks monkeydust, we'll look at this today.

[chrislott]

@monkeydust , can you look in your .env file to see if there is a duplicate DG_API_KEY under the TTS section? That is probably overwriting the Deepgram API key, which is why you're seeing a 401. We will fix this by leaving it commented out by default.

[monkeydust]

Its referenced twice so I put in the key twice, either way it never pointed to Deepgram, per log trace it keeps using Groq and I cant understand why thats throwing an error. I tried re-creating the key as well.

[chrislott]

Were you able to get it working with a Groq API key?

[alessandro-neri]

Hi @monkeydust, thank you for reporting the issue. Could you please confirm that you are changing the STT module and not the TTS module?

There should be 2 .env variables: TTS_PROVIDER="elevenlabs" and STT_PROVIDER="groq". Please ensure that you change the model in the appropriate variable.

[monkeydust]

Hi @alessandro-neri below is the .env redacted - I gave up with Groq so created a Deepgram account but as I said it never seems to initiate Deepgram and still tries to do the TTS using Groq.

# ============== Configuration ==============
# Minimal configuration to get started.
# Get your API keys from the following URLs:
# - Groq: https://console.groq.com/keys/
# - ElevenLabs: https://elevenlabs.io/app/speech-synthesis/text-to-speech
# =============================================

# === LLM Configuration ===
LLM_API_KEY=<redacted>
LLM_BASE_URL="https://api.groq.com/openai/v1"
LLM_MODEL_ID="llama3-70b-8192"

# === Speech-to-Text (STT) Configuration ===
STT_PROVIDER="deepgram"     # Alternative providers are  "deepgram", "faster_whisper"
#GROQ_STT_API_KEY=<redacted>
#GROQ_STT_API_KEY=<redacted>

# === Text-to-Speech (TTS) Configuration ===
TTS_PROVIDER="elevenlabs"       # Alternative providers: "deepgram", "openai", "cartesia"
ELEVENLABS_API_KEY=<redacted>
ELEVENLABS_VOICE_ID="your_elevenlabs_voice_id"


# =============================================
# Customize your configuration below
# =============================================

# === Reasoning Configuration ===
INITIAL_RESPONSE="Hello! How can I help you?"

# === Speech-to-Text (STT) Configuration ===
#DG_API_KEY=  # required if you want to use Deepgram
# STT_SAMPLE_RATE="16000"

# === Text-to-Speech (TTS) Configuration ===
# Get your API keys from the following URLs:
# - Deepgram: https://deepgram.com/
# - OpenAI: https://openai.com/
TTS_SAMPLE_RATE="24000"
# Deepgram configuration
DG_TTS_MODEL_VOICE="aura-asteria-en"
DG_API_KEY=<redacted>
# OpenAI configuration
OPENAI_API_KEY=<redacted>
OPENAI_TTS_MODEL="tts-1"
OPENAI_TTS_VOICE="alloy"
# CACHE_DIR="your_cache_directory_path"


# === Guardrails Proxy Configuration ===
# AGENT_HOST="xrx-guardrails"
# AGENT_PORT="8004"
# GUARDRAILS_AGENT_HOST="xrx-reasoning"
# GUARDRAILS_AGENT_PORT="8003"

# === LLM Observability Configuration ===
LLM_OBSERVABILITY_LIBRARY="none"
# Alternatives: 
# LLM_OBSERVABILITY_LIBRARY="langsmith"
# LLM_OBSERVABILITY_LIBRARY="langfuse"

# Langfuse configuration
LANGFUSE_SECRET_KEY="your_langfuse_secret_key"
LANGFUSE_PUBLIC_KEY="your_langfuse_public_key"
LANGFUSE_HOST="http://host.docker.internal:3001"
# For localhost: LANGFUSE_HOST="http://localhost:3001"
# For cloud-hosted Langfuse: LANGFUSE_HOST="https://us.cloud.langfuse.com"

# LangSmith configuration
LANGCHAIN_TRACING_V2="true"
LANGCHAIN_API_KEY="your_langsmith_api_key"
LANGCHAIN_PROJECT="your_langsmith_project_name"
LANGCHAIN_ENDPOINT="https://api.smith.langchain.com"

# === UI ===
NEXT_PUBLIC_UI_DEBUG_MODE="false"

# === Redis Configuration ===
REDIS_HOST="localhost"

# === LLM Configuration ===
# JSON fixing model (if needed)
LLM_MODEL_ID_JSON_FIXER="llama3-70b-8192"

# === Orchestrator Configuration ===
# AGENT_WAIT_MS="your_agent_wait_time"
# SAMPLE_RATE="your_sample_rate"
# STT_WAIT_MS="your_stt_wait_time"

# STT_HOST="xrx-stt"
# STT_PORT="8001"
# STT_PATH="/api/v1/ws"

# TTS_HOST="xrx-tts"
# TTS_PORT="8002"
# TTS_PATH="/api/v1/ws"

# AGENT_PATH="/run-reasoning-agent"

[alessandro-neri]

Hi @monkeydust

I've tried using your .env file by replacing the API keys with mine, and it works perfectly with Deepgram. Are you running the app using Docker? If so, you should also update REDIS_HOST="xrx-redis" as indicated in the example env file. We strongly advise running the app in a Docker container rather than deploying each part locally.
Let me know if the error persists. If it does, I will wait for more information to reproduce the error.

[chrislott]

Also, please try switching to the develop branch. There is some fixes for the patient intake demo.

[monkeydust]

@chrislott hi getting a bit further but not there yet...

1. On windows 11 and yes using docker container.

2. After cloning dev branch, using same keys and setting REDIS_HOST="xrx-redis" I run this:

C:\Users\admin\xrx-sample-apps\patient-information-app>docker-compose up --build

3. Then I am using windows docker desktop to manage the container (start/stop)

4. The app does load but on localhost:8000 where it asks me for questions. When I answer it does seem to transcribe now on Groq but throws error (at very bottom ).

5. If I run localhost:8003 I get this error:

---

{
"detail": "Not Found"
}

---

6. If I run per the readme the command given to launch container I get this error:

C:\Users\admin\xrx-sample-apps\patient-information-app>docker run -p 8003:8003 --env-file .env patient-information-agent
Unable to find image 'patient-information-agent:latest' locally
docker: Error response from daemon: pull access denied for patient-information-agent, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

Hopefully this helps diagnose what the issue is. Thanks

**

Error:
991446, 'compression_ratio': 0.82857144, 'no_speech_prob': 0.10233396}], x_groq={'id': 'req_01j8jqtcscek2sbnts97t9z78q'})
2024-09-24 20:23:36 xrx-tts | INFO:main:Received action: cancel
2024-09-24 20:23:36 xrx-tts | INFO:main:Received cancel action
2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 INFO:Received request body: {'messages': [{'role': 'user', 'content': 'Yes, go ahead.'}, {'role': 'user', 'content': 'Hello.'}, {'role': 'user', 'content': 'Yes, can you?'}, {'role': 'user', 'content': 'You count.'}, {'role': 'user', 'content': 'questions.'}, {'role': 'user', 'content': 'please ask me some questions.'}], 'session': {'guid': '76443260-3726-45c2-b564-361179f4d37f'}, 'action': {}}
2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 INFO:Request headers: Headers({'host': 'xrx-reasoning:8003', 'connection': 'keep-alive', 'content-type': 'application/json', 'accept': 'text/event-stream', 'accept-language': '*', 'sec-fetch-mode': 'cors', 'user-agent': 'node', 'accept-encoding': 'gzip, deflate', 'content-length': '328'})
2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 INFO:Request client: Address(host='172.18.0.7', port=36068)
2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 INFO:Created task with task ID: 534c591b-3b75-4382-87a5-eae27cc908cd
2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 ERROR:An error occurred while processing the request: Error 111 connecting to localhost:6379. 111.
2024-09-24 20:23:36 xrx-reasoning | INFO: 172.18.0.7:36068 - "POST /run-reasoning-agent HTTP/1.1" 500 Internal Server Error
2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Received from STT: please ask me some questions.
2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Received from STT: please ask me some questions.
2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Cancelling all agent activity
2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Sending to agent:please ask me some questions.
2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: send chat history to agent
2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Sending to agent xrx-reasoning:8003/run-reasoning-agent: [{"role":"user","content":"Yes, go ahead."},{"role":"user","content":"Hello."},{"role":"user","content":"Yes, can you?"},{"role":"user","content":"You count."},{"role":"user","content":"questions."},{"role":"user","content":"please ask me some questions."}]
2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Sending to agent: {"guid":"76443260-3726-45c2-b564-361179f4d37f"}
2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Raw output: {"detail":"Error 111 connecting to localhost:6379. 111."}

[chrislott]

We just updated main -- can you do a fresh clone and copy your env file into the patient app and try docker compose up again?

[monkeydust]

@chrislott painfully close now!

So its asks me for questions, I can see its transcribing my voice but its not asking the questions (no sound) looking at logs its not resolving the voice_id for elevenlabs, I have tried the actual voice_id and name neither works.

E.g.

voice_id = CwhRBWXzGAHq8TQ4Fs17 for Roger

from https://api.elevenlabs.io/v1/voices

---

2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Received from STT:  What are your questions?
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Cancelling all agent activity
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Sending to agent:What are your questions?
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: send chat history to agent
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Sending to agent xrx-reasoning:8003/run-reasoning-agent: [{"role":"user","content":"Yes, go for it."},{"role":"assistant","content":"{\n   \"information-received\": {},\n   \"response\": \"Can you please confirm your first and last name?\"\n}"},{"role":"user","content":"What are your questions?"}]
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Sending to agent: {"guid":"35e54838-97b0-4bf2-9f19-7db166fac528","information-received":"{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Raw output: data: {"messages": [{"role": "assistant", "content": "{\n   \"information-received\": {},\n   \"response\": \"To start, could you please tell me your full name?\"\n}"}], "node": "Widget", "output": {"type": "patient-information", "details": "{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}, "session": {"guid": "35e54838-97b0-4bf2-9f19-7db166fac528", "information-received": "{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}}
2024-09-25 18:12:08 xrx-orchestrator  | 
2024-09-25 18:12:08 xrx-orchestrator  | data: {"messages": [{"role": "assistant", "content": "{\n   \"information-received\": {},\n   \"response\": \"To start, could you please tell me your full name?\"\n}"}], "node": "CustomerResponse", "output": "To start, could you please tell me your full name?", "session": {"guid": "35e54838-97b0-4bf2-9f19-7db166fac528", "information-received": "{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}}
2024-09-25 18:12:08 xrx-orchestrator  | 
2024-09-25 18:12:08 xrx-orchestrator  | 
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Agent response: {"messages":[{"role":"assistant","content":"{\n   \"information-received\": {},\n   \"response\": \"To start, could you please tell me your full name?\"\n}"}],"node":"Widget","output":{"type":"patient-information","details":"{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"},"session":{"guid":"35e54838-97b0-4bf2-9f19-7db166fac528","information-received":"{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}}
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Handling agent response of type Widget: [object Object], modality: audio
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Received from Agent:[object Object]
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Not sending to TTS
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Agent response: {"messages":[{"role":"assistant","content":"{\n   \"information-received\": {},\n   \"response\": \"To start, could you please tell me your full name?\"\n}"}],"node":"CustomerResponse","output":"To start, could you please tell me your full name?","session":{"guid":"35e54838-97b0-4bf2-9f19-7db166fac528","information-received":"{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}}
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Handling agent response of type CustomerResponse: To start, could you please tell me your full name?, modality: audio
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Received from Agent:To start, could you please tell me your full name?
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Sending to TTS:To start, could you please tell me your full name?
2024-09-25 18:12:08 xrx-tts           | INFO:elevenlabs_tts:Connected to 11labs websocket
2024-09-25 18:12:08 xrx-tts           | INFO:elevenlabs_tts:Sent initial message to 11labs
2024-09-25 18:12:08 xrx-tts           | INFO:elevenlabs_tts:Sent EOS message to 11labs
2024-09-25 18:12:08 xrx-tts           | ERROR:elevenlabs_tts:Error from 11labs: voice_id_does_not_exist
2024-09-25 18:12:08 xrx-tts           | ERROR:elevenlabs_tts:An error occurred while receiving data from 11labs
2024-09-25 18:12:08 xrx-tts           | Traceback (most recent call last):
2024-09-25 18:12:08 xrx-tts           |   File "/tts/elevenlabs_tts.py", line 81, in synthesize
2024-09-25 18:12:08 xrx-tts           |     raise Exception(f"ElevenLabs API error: {data['error']}")
2024-09-25 18:12:08 xrx-tts           | Exception: ElevenLabs API error: voice_id_does_not_exist
2024-09-25 18:12:09 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Received from TTS: {"action":"done"}
2024-09-25 18:12:09 xrx-orchestrator  | [17:12:08 UTC] DEBUG: TTS done, sending cached agent responses

[alessandro-neri]

Hi @monkeydust .

You need to add the voice to your "voice library". If you don't, you'll encounter the cryptic "voice_id _does_not_exist" error. To do this, paste the ID from the .env file onto the elevenlabs dashboard and click add.

https://www.reddit.com/r/ElevenLabs/comments/16f22rd/texttospeech_api_for_generated_voices_not_working/

[monkeydust]

@alessandro-neri sorry for delay, so something not right here. I did above using a voice id that works when calling it through their docs with my own key

https://elevenlabs.io/docs/api-reference/text-to-speech#/voices/Get_voices_v1_voices_get

yet through the app it shows errors with same id per below* - still would process text and fill out the form, that was cool, where it fell down a bit was when it entered things incorrectly and I tried to fix it e.g. spelling of my name, it kept forcing me to the next question but appreciate its a poc.

rx-orchestrator | [09:03:14 UTC] DEBUG: Sending to TTS:Let me try that again. Can you please tell me your date of birth, like the month, day, and year?
2024-10-02 10:03:14 xrx-tts | ERROR:elevenlabs_tts:Error from 11labs: voice_id_does_not_exist
2024-10-02 10:03:14 xrx-tts | ERROR:elevenlabs_tts:An error occurred while receiving data from 11labs
2024-10-02 10:03:14 xrx-tts | Traceback (most recent call last):
2024-10-02 10:03:14 xrx-tts | File "/tts/elevenlabs_tts.py", line 81, in synthesize
2024-10-02 10:03:14 xrx-tts | raise Exception(f"ElevenLabs API error: {data['error']}")
2024-10-02 10:03:14 xrx-tts | Exception: ElevenLabs API error: voice_id_does_not_exist
2024-10-02 10:03:15 xrx-orchestrator | [09:03:14 UTC] DEBUG: Received from TTS: {"action":"done"}
2024-10-02 10:03:15 xrx-orchestrator | [09:03:14 UTC] DEBUG: TTS done, sending cached agent responses
2024-10-02 10:03:28 xrx-stt | INFO:groq_stt:Transcribing audio using Groq API...
2024-10-02 10:03:28 xrx-stt | INFO:httpx:HTTP Request: POST https://api.groq.com/openai/v1/audio/transcriptions "HTTP/1.1 200 OK"
2024-10-02 10:03:28 xrx-stt | INFO:groq_stt:-----
2024-10-02 10:03:28 xrx-stt | INFO:groq_stt:Transcription(text=' Thank you.', task='transcribe', language='English', duration=0.51, segments=[{'id': 0, 'seek': 0, 'start': 0, 'end': 29.98, 'text': ' Thank you.', 'tokens': [50365, 1044, 291, 13, 51864], 'temperature': 0, 'avg_logprob': -

[monkeydust]

@alessandro-neri sorry for delay, so something not right here. I did above using a voice id that works when calling it through their docs with my own key

https://elevenlabs.io/docs/api-reference/text-to-speech#/voices/Get_voices_v1_voices_get

yet through the app it shows errors with same id per below* - still would process text and fill out the form, that was cool, where it fell down a bit was when it entered things incorrectly and I tried to fix it e.g. spelling of my name, it kept forcing me to the next question but apperciate its a poc.

rx-orchestrator | [09:03:14 UTC] DEBUG: Sending to TTS:Let me try that again. Can you please tell me your date of birth, like the month, day, and year?
2024-10-02 10:03:14 xrx-tts | ERROR:elevenlabs_tts:Error from 11labs: voice_id_does_not_exist
2024-10-02 10:03:14 xrx-tts | ERROR:elevenlabs_tts:An error occurred while receiving data from 11labs
2024-10-02 10:03:14 xrx-tts | Traceback (most recent call last):
2024-10-02 10:03:14 xrx-tts | File "/tts/elevenlabs_tts.py", line 81, in synthesize
2024-10-02 10:03:14 xrx-tts | raise Exception(f"ElevenLabs API error: {data['error']}")
2024-10-02 10:03:14 xrx-tts | Exception: ElevenLabs API error: voice_id_does_not_exist
2024-10-02 10:03:15 xrx-orchestrator | [09:03:14 UTC] DEBUG: Received from TTS: {"action":"done"}
2024-10-02 10:03:15 xrx-orchestrator | [09:03:14 UTC] DEBUG: TTS done, sending cached agent responses
2024-10-02 10:03:28 xrx-stt | INFO:groq_stt:Transcribing audio using Groq API...
2024-10-02 10:03:28 xrx-stt | INFO:httpx:HTTP Request: POST https://api.groq.com/openai/v1/audio/transcriptions "HTTP/1.1 200 OK"
2024-10-02 10:03:28 xrx-stt | INFO:groq_stt:-----
2024-10-02 10:03:28 xrx-stt | INFO:groq_stt:Transcription(text=' Thank you.', task='transcribe', language='English', duration=0.51, segments=[{'id': 0, 'seek': 0, 'start': 0, 'end': 29.98, 'text': ' Thank you.', 'tokens': [50365, 1044, 291, 13, 51864], 'temperature': 0, 'avg_logprob': -