Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for azure openai new version API (2023-07-01-preview) #450

Closed
wants to merge 1 commit into from
Closed

Add support for azure openai new version API (2023-07-01-preview) #450

wants to merge 1 commit into from

Conversation

ZeroDeng01
Copy link
Contributor

@ZeroDeng01 ZeroDeng01 commented Jul 20, 2023 •
edited
Loading

Describe the change
Added support for Azure openai 2023-07-01-preview API. This version of the API mainly adds data related to content security in the returned content. Such as prompt_annotations and content_filter_results

Describe your solution
It is mainly to supplement the return parameters added by the new version API in the corresponding structs in the chat_stream.go and chat.go files. And these parameters are marked with omitempty to be compatible with the old version of api and openai's api

Tests
Test with common test cases

Additional context

CURL

Stream:true
curl "https://TEST.openai.azure.com/openai/deployments/gpt-35-turbo-16k/chat/completions?api-version=2023-07-01-preview"   -H "Content-Type: application/json"   -H "api-key: MY_KEY"   -d "{
>   \"messages\": [{\"role\":\"system\",\"content\":\"You are an AI assistant that helps people find information.\"},{\"role\":\"user\",\"content\":\"hi\"},{\"role\":\"assistant\",\"content\":\"Hello! How can I assist you today?\"}],
>   \"max_tokens\": 800,
>   \"temperature\": 0.7,
>   \"frequency_penalty\": 0,
>   \"presence_penalty\": 0,
>   \"top_p\": 0.95,
>   \"stop\": null,
>   \"stream\": true
> }" -v
*   Trying 20.232.91.180:443...
* TCP_NODELAY set
* Connected to TEST.openai.azure.com (20.232.91.180) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=WA; L=Redmond; O=Microsoft Corporation; CN=eastus.api.cognitive.microsoft.com
*  start date: Jun 16 14:44:59 2023 GMT
*  expire date: Jun 10 14:44:59 2024 GMT
*  subjectAltName: host "TEST.openai.azure.com" matched cert's "*.openai.azure.com"
*  issuer: C=US; O=Microsoft Corporation; CN=Microsoft Azure TLS Issuing CA 06
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5612c69057c0)
> POST /openai/deployments/gpt-35-turbo-16k/chat/completions?api-version=2023-07-01-preview HTTP/2
> Host: TEST.openai.azure.com
> user-agent: curl/7.68.0
> accept: */*
> content-type: application/json
> api-key: MY_KEY
> content-length: 353
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
* We are completely uploaded and fine
< HTTP/2 200
< cache-control: no-cache, must-revalidate
< content-type: text/event-stream
< access-control-allow-origin: *
< apim-request-id: 2295a78f-8995-4280-81ed-a853253a25a0
< x-content-type-options: nosniff
< openai-processing-ms: 70.0725
< x-ms-region: East US
< x-accel-buffering: no
< x-request-id: ee707c92-2588-4aa6-a43e-2575e4449a04
< x-ms-client-request-id: 2295a78f-8995-4280-81ed-a853253a25a0
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< azureml-model-session: turbo-659513-0
< azureml-model-group: online
< date: Thu, 20 Jul 2023 03:42:05 GMT
<
data: {"id":"","object":"","created":0,"model":"","prompt_annotations":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"choices":[],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"role":"assistant"},"content_filter_results":{}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":"Hi"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":" there"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":"!"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":" How"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":" can"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":" I"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":" assist"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":" you"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":" today"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":null,"delta":{"content":"?"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":null}

data: {"id":"chatcmpl-7eEqDlKQQKa0RoWG6HvlrFEiNYqBq","object":"chat.completion.chunk","created":1689824525,"model":"gpt-35-turbo-16k","choices":[{"index":0,"finish_reason":"stop","delta":{},"content_filter_results":{}}],"usage":null}

data: [DONE]
* Connection #0 to host TEST.openai.azure.com left intact
Stream:false
curl "https://TEST.openai.azure.com/openai/deployments/gpt-35-turbo-16k/chat/completions?api-version=2023-07-01-preview"   -H "Content-Type: application/json"   -H "api-key: MY_KEY"   -d "{
"messag>   \"messages\": [{\"role\":\"system\",\"content\":\"You are an AI assistant that helps people find information.\"},{\"role\":\"user\",\"content\":\"hi\"},{\"role\":\"assistant\",\"content\":\"Hello! How can I assist you today?\"}],
max_to>   \"max_tokens\": 800,
>   \"temperature\": 0.7,
>   \"frequency_penalty\": 0,
>   \"presence_penalty\": 0,
>   \"top_p\": 0.95,
>   \"stop\": null,
>   \"stream\": false
 -v> }" -v
*   Trying 20.232.91.180:443...
* TCP_NODELAY set
* Connected to TEST.openai.azure.com (20.232.91.180) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=WA; L=Redmond; O=Microsoft Corporation; CN=eastus.api.cognitive.microsoft.com
*  start date: Jun 16 14:44:59 2023 GMT
*  expire date: Jun 10 14:44:59 2024 GMT
*  subjectAltName: host "TEST.openai.azure.com" matched cert's "*.openai.azure.com"
*  issuer: C=US; O=Microsoft Corporation; CN=Microsoft Azure TLS Issuing CA 06
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x563e2bdc67c0)
> POST /openai/deployments/gpt-35-turbo-16k/chat/completions?api-version=2023-07-01-preview HTTP/2
> Host: TEST.openai.azure.com
> user-agent: curl/7.68.0
> accept: */*
> content-type: application/json
> api-key: MY_KEY
> content-length: 354
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
* We are completely uploaded and fine
< HTTP/2 200
< cache-control: no-cache, must-revalidate
< content-length: 789
< content-type: application/json
< access-control-allow-origin: *
< apim-request-id: 4771dda4-5a70-4941-a0fb-14cb42535966
< openai-model: gpt-35-turbo-16k
< x-content-type-options: nosniff
< openai-processing-ms: 358.3437
< x-ms-region: East US
< x-accel-buffering: no
< x-request-id: 58c95a99-9d7f-47af-9f30-ab6243c16d56
< x-ms-client-request-id: 4771dda4-5a70-4941-a0fb-14cb42535966
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< azureml-model-session: turbo-659513-1
< azureml-model-group: online
< date: Thu, 20 Jul 2023 03:44:26 GMT
<
{"id":"chatcmpl-7eEsVPPnScxxmeysuc471qvwOhR5V","object":"chat.completion","created":1689824667,"model":"gpt-35-turbo-16k","prompt_annotations":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Hi there! How can I help you today?"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":{"completion_tokens":10,"prompt_tokens":36,"total_tokens":46}}
* Connection #0 to host TEST.openai.azure.com left intact

Issue: #444

@codecov
Copy link

codecov bot commented Jul 20, 2023

Codecov Report

Merging #450 (79e34b3) into master (1876e0c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #450   +/-   ##
=======================================
  Coverage   97.03%   97.03%           
=======================================
  Files          17       17           
  Lines         709      709           
=======================================
  Hits          688      688           
  Misses         15       15           
  Partials        6        6
Impacted Files Coverage Δ
chat.go 100.00% <ø> (ø)
chat_stream.go 100.00% <ø> (ø)

@ZeroDeng01
Copy link
Contributor Author

ZeroDeng01 commented Jul 20, 2023 •
edited
Loading

@sashabaranov @vvatanabe Hello, due to my limited technology, but I urgently need to use this version of the api, so I submitted this PR myself, I hope you can help review!THANKS!

@sashabaranov
Copy link
Owner

Thank you for the PR!

@sashabaranov
Copy link
Owner

Doc link for reference: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter

sashabaranov
sashabaranov reviewed Jul 20, 2023
View reviewed changes
@ZeroDeng01 ZeroDeng01 closed this Jul 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sashabaranov sashabaranov sashabaranov left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ZeroDeng01 @sashabaranov