Runtime Switchable Deployment Model Name

[mattahearn]

Hello, I am attempting to make a toggle for the Azure OpenAI Deployment Name within User Settings so that a user can, at runtime, choose a different deployment i.e. gpt-35-turbo-deployment vs gpt4-deployment. The issue I am running into is the Semantic Kernel/Provider are both created at app startup. What would be the recommended approach to "update" the Configuration i.e. Deployment value for an existing dependency service? Thank you for your help.

The "Deployment" config resides within the AzureOpenAIText section of the appsettings.json file for the web api project.

[matthewbolanos]

Within Semantic Kernel, you can author a custom service selector. I believe this can be used to dynamically alter which model is used during runtime. We have a sample here: https://github.com/microsoft/semantic-kernel/blob/9a4450622021ce003234863bcf4def9613ae1153/dotnet/samples/Concepts/Kernel/CustomAIServiceSelector.cs#L11

Do you think this addresses your challenges?

[aaronba]

Hey @matthewbolanos , Great sample.

@mattahearn , you might want to consider mapping the sample concept approach for the CustomAIServiceSelector to the following code in the chat-copilot repo below:

https://github.com/microsoft/chat-copilot/blob/main/webapi/Services/SemanticKernelProvider.cs

[mattahearn]

[like] Matt Ahearn reacted to your message:

…

________________________________ From: Aaron Barth ***@***.***> Sent: Friday, May 3, 2024 12:45:50 AM To: microsoft/chat-copilot ***@***.***> Cc: Matt Ahearn ***@***.***>; Mention ***@***.***> Subject: Re: [microsoft/chat-copilot] Runtime Switchable Deployment Model Name (Discussion #957) Hey @matthewbolanos<https://github.com/matthewbolanos> , Great sample. @mattahearn<https://github.com/mattahearn> , you might want to consider mapping the sample concept approach for the CustomAIServiceSelector to the following code in the chat-copilot repo below: https://github.com/microsoft/chat-copilot/blob/main/webapi/Services/SemanticKernelProvider.cs — Reply to this email directly, view it on GitHub<#957 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AOJJMJXYBP5PJ4JPL5FMZ2TZALM35AVCNFSM6AAAAABG66Q5ZWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGMBRGA4TI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[mattahearn]

[like] Matt Ahearn reacted to your message:

…

________________________________ From: Matthew Bolaños ***@***.***> Sent: Thursday, May 2, 2024 11:56:11 PM To: microsoft/chat-copilot ***@***.***> Cc: Matt Ahearn ***@***.***>; Author ***@***.***> Subject: Re: [microsoft/chat-copilot] Runtime Switchable Deployment Model Name (Discussion #957) Within Semantic Kernel, you can author a custom service selector. I believe this can be used to dynamically alter which model is used during runtime. We have a sample here: https://github.com/microsoft/semantic-kernel/blob/9a4450622021ce003234863bcf4def9613ae1153/dotnet/samples/Concepts/Kernel/CustomAIServiceSelector.cs#L11 Do you think this addresses your challenges? — Reply to this email directly, view it on GitHub<#957 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AOJJMJWATSEQWRHATRGZSK3ZALHBXAVCNFSM6AAAAABG66Q5ZWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGMBQHA4TI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[mattahearn]

@matthewbolanos your sample was helpful. To extend this to the solution we need, how would you pass a parameter to your GptAIServiceSelector() from the Client in your example?

I'm struggling to understand how I can send a runtime parameter from the Client to a DI service that is already added via builder.Services.AddSingleton(). For example, the user on the client-side will chose "gpt4" or "gpt-35-turbo", then how do I pass this value down into the service collection so the Kernel uses the correct deployment i.e. "gpt4" or "gpt-35-turbo".

Thanks again for your help on this.

[sam0044]

@mattahearn did you end finding a solution for this?