[Azure] Support Realtime API - Standalone client #1283
Conversation
There was a problem hiding this comment.
nit: I think these examples should mention realtime in the path somewhere, e.g.
examples/realtime/azure/websocket.tsexamples/azure/realtime/websocket.tsexamples/azure/realtime-websocket.ts
(applies to thews.tsexample as well)
| @@ -2,6 +2,7 @@ | |||
|
|
|||
| import { AzureOpenAI } from 'openai'; | |||
| import { getBearerTokenProvider, DefaultAzureCredential } from '@azure/identity'; | |||
| import 'dotenv/config'; | |||
There was a problem hiding this comment.
nit: I think it'd be better to reduce the amount of deps a user would need to add if they copy-paste this example file.
I don't feel super strongly here though, and dotenv is very standard... maybe @kwhinnery-openai has thoughts?
| const rt = new AzureOpenAIRealtimeWebSocket(client); | ||
| await rt.open(); | ||
|
|
||
| // access the underlying `ws.WebSocket` instance |
There was a problem hiding this comment.
| // access the underlying `ws.WebSocket` instance | |
| // access the underlying `WebSocket` instance |
| } | ||
|
|
||
| async open(): Promise<void> { | ||
| async function getUrl({ |
There was a problem hiding this comment.
nit: can you move this function outside of this class? would be much easier to read
| if (model !== undefined && !this.baseURL.includes('/deployments')) { | ||
| options.path = `/deployments/${model}${options.path}`; | ||
| } | ||
| } | ||
| return super.buildRequest(options, props); | ||
| } | ||
|
|
||
| private async _getAzureADToken(): Promise<string | undefined> { | ||
| async getAzureADToken(): Promise<string | undefined> { |
There was a problem hiding this comment.
question: do you think this is helpful to expose to users and something we should make part of the public API?
If not and this is only public for usage in the realtime client implementation then we should at least keep the _ prefix or keep it private and just ts-ignore at the call-site
| @@ -95,3 +95,106 @@ export class OpenAIRealtimeWebSocket extends OpenAIRealtimeEmitter { | |||
| } | |||
| } | |||
| } | |||
|
|
|||
| export class AzureOpenAIRealtimeWebSocket extends OpenAIRealtimeEmitter { | |||
There was a problem hiding this comment.
question: does the Azure API support ephemeral session tokens? Asking as the OpenAI API client requires dangerouslyAllowBrowser: true if you're not using an ephemeral session token
There was a problem hiding this comment.
@kwhinnery-openai curious if you have any opinions on the location of the azure classes, is it fine / good for them to be in the same file as the OpenAI ones?
It does feel a bit verbose to have to do import { AzureOpenAIRealtimeWebSocket } from 'openai/beta/realtime/azure/websocket';, but there are a couple advantages I can see to splitting them up is that:
- very small reduction in bundle size for non-azure users
- it would be easier to add optional peer dependencies for azure specific things down the line if needed
|
Based on feedback in the channel, I opened #1287 instead. |
Support Azure Realtime API
In order to do so, this PR adds new
AzureOpenAIRealtimeWebSocketandAzureOpenAIRealtimeWSclientsKey Changes
1. Introduce New
openMethodopenmethod enables asynchronous fetching of the Azure AD token and establishes the WebSocket connection afterward.2. New
getAzureADTokenMethod inAzureOpenAIgetAzureADTokenmethod in theAzureOpenAIclass to provide refreshed Azure AD tokens.