[AI] [Inference] Create AI inference SDK (Azure#41482)

* initial auto-genned code * use camel case for Json in class name * initial work on setting up a test * Fix some compile errors * test actually compiles (and fails) * add to azure-sdks-all * fix json typo causing defaultCompleteOptions to throw * add completeStreaming convenience method and InferenceSSE class * add chat samples (sync), streaming convenience fields * simplify convenience method for defaultCompleteOptions * got test working in intellij * add streaming chat test * add streaming chat async sample and support code to async client * add convenience method for async complete, basic async chat sample * change options class name * merge deletes * add async tests for complete and completeStreaming * remove refs to jackson package * add chat completions code to README (links needed) * add troubleshooting file, links in README * add readmeSamples class for readme code sample auto gen * update genned code with embeddingsClient, imageEmbeddingsClient * revert parameters to Object type * add embedding sample * add Embeddings test classes * add async embeddings test * add text embeddings async sample * make embed method public * cleaner embeddings sample output * fix javadoc gen * fix dep versions * phrasing * add AAD sample with scope * better explanation of auth scope setting in sample * allow mvn compile * create and allow playback for test recordings * change display name in ci.yml * add codeowners * kick off new build * add tests.yml * remove EnableBatchRelease field * update to pass codegen test * use existing label * add ubinary to ignored words * use proper streaming types, remove delta from ChatChoice * rename getEmbedding to getEmbeddingList, add auto-genned method * make getModelInfo public for java * use Json instead of JSON in class/file name * sync ChatChoice with TS output * set partial-update to true * codegen more closely synced * use latest typespec * add customization code * updated typespec * App Config Correlation Context update + yaml hint fix (Azure#41607) * Updating correlation context to use pipeline context * Fixing yaml hints * Updating property docs from yaml * removing unused imports * Use code customizations to map internal Storage error in generated code (Azure#41624) Use code customizations to map internal Storage error in generated code * Add namespaces (packages) to packageInfo (Azure#41639) * Add namespaces (packages) to packageInfo * Actually name the script correctly * Fix incorrect parameter name * correct fail the step if no namespaces can be determined for one of the libraries being processed * Fix the pom file to remove the duplicate org.apache.maven.plugins:maven-jar-plugin definition * turn off docs for spring-cloud-azure-appconfiguration-config-web until it actually generates docs * Updates for feedback * Fix dependency version issue * Publish code coverage results from 1 to2 (Azure#41656) * Update PublishCodeCoverageResults task version from 1 to 2 * Add prerequisite task * last commit missed a file needing a save * Fix typo (Azure#41658) * prepare for 1.2.27 bom release (Azure#41659) * Move logging token acquisition to VERBOSE (Azure#41648) * Sync eng/common directory with azure-sdk-tools for PR 8875 (Azure#41660) * chore(): add playwright-testing to product slugs * Update eng/common/scripts/Test-SampleMetadata.ps1 --------- Co-authored-by: Siddharth Singha Roy <[email protected]> Co-authored-by: Wes Haggard <[email protected]> * [App Configuration] Add test case for feature management lib (Azure#41498) * add feature management common test cases * add test case for no filters * initialize feature manager and call `featureManager.isEnabled()` to compare the result * update the schema of test case * get sample file and tests file by listFiles api * update the todo comment. Throw exception and use little endian are breaking change, will have both when update to 6.xx version. * use `getContextClassLoader().getResource` to get the resource folder path * address comment: typo wording error fix, add "@SuppressWarnings" * address comment: typo wording error fix * address comment: update member name * add running log * use contains to filter out the "TargetingFilter.sample.json" * add some info log * sort the file list * move to another the package to avoid making feature manager construction as public * address comment: use logging, remove else * address comment: throw exception when empty "feature_management" section. * address comment: rename symbol * Update ValidationsTest.java * fixing linting * address comment: ignore both targeting filter test case --------- Co-authored-by: Matt Metcalf <[email protected]> * Retrieve TokenCredential from ConfigurableBootstrapContext for env post processor (Azure#41580) * get TokenCredential from ConfigurableBootstrapContext --------- Co-authored-by: Muyao <[email protected]> Co-authored-by: Muyao Feng <[email protected]> * [Automation] Generate SDK based on TypeSpec 0.20.0 (Azure#41666) * [Automation] Generate SDK based on TypeSpec 0.20.0 * update assets.json * revert healthinsights changes --------- Co-authored-by: Haoling Dong <[email protected]> * add customization to unfinalize base classes * more typespec syncs * correct doc comment * make extra parameters public * fix bad gen * remove dupe imports * sync with typespec * add lint exception for PagedFlux * lint fixes * App Config Correlation Context update + yaml hint fix (Azure#41607) * Updating correlation context to use pipeline context * Fixing yaml hints * Updating property docs from yaml * removing unused imports * [OpenAI-Assistant] Added support for 2024-07-01 (Azure#41464) * Added support 2024-07-01 service API version (Azure#41473) * Default to sub config file paths/svc connection for public + sovereign live tests (Azure#41267) * remove azure.xml dependency * make completeWithResponse public * make complete with options public * add string constructor for ChatRequestUserMessage * add scopes to ClientBuilders, change parameters in FunctionDefinition to BinaryData type * update comment * correct string constructor for ChatRequestUserMessage * formatting * add streaming tool call sample * add Mono<Response<>> completeWithResponse API to AsyncClient * better implementation of completeWithResponse * remove unused import * add embedWithResponse API to embeddings clients * add tests for complete(options) and completeWithResponse APIs * more sync tests * add stream response test * add completeStreamWithResponse test, change completeStreaming to completeStream * new test recordings * changes for TypeSpec sync * fix pom error * add deleted file * restore deleted files * restore more deleted files * even more deleted files * pull in openai changes from main * more deleted files * unmod openai file * update HTTP client link * spacing for typespec gen * update dependencies * spacing * spacing * spacing * update min jacoco settings * trigger CI build * Naive implementations of new APIs fromContentItems and file constructor for ChatMessageImageContentItem * spacing * add image read sample and correct API implementations * sanity check, cleaner syntax, and samples style fixes * Add ImageUrlChatSample * update dependency version * typespec sync * review feedback * Update sdk/ai/azure-ai-inference/README.md Co-authored-by: Bill Wert <[email protected]> * Update sdk/ai/azure-ai-inference/README.md Co-authored-by: Bill Wert <[email protected]> * fix TypeSpec crash * remove unneeded paragraph * make TypeSpec sync happy * add access helper to ChatCompletionsOptions * typespec sync * review feedback --------- Co-authored-by: Matthew Metcalf <[email protected]> Co-authored-by: Alan Zimmer <[email protected]> Co-authored-by: James Suplizio <[email protected]> Co-authored-by: Shawn Fang <[email protected]> Co-authored-by: Bill Wert <[email protected]> Co-authored-by: Azure SDK Bot <[email protected]> Co-authored-by: Siddharth Singha Roy <[email protected]> Co-authored-by: Wes Haggard <[email protected]> Co-authored-by: ivywei0125 <[email protected]> Co-authored-by: Xiaolu Dai <[email protected]> Co-authored-by: Muyao <[email protected]> Co-authored-by: Muyao Feng <[email protected]> Co-authored-by: Haoling Dong <[email protected]> Co-authored-by: Ben Broderick Phillips <[email protected]>
mssfang · Oct 21, 2024 · b7b5541 · b7b5541
1 parent e95dd46
commit b7b5541
Show file tree

Hide file tree

Showing 104 changed files with 13,141 additions and 0 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -64,6 +64,9 @@
 # ServiceLabel: %Advisor
 # ServiceOwners:                                     @mojayara @Prasanna-Padmanabhan
 
+# PRLabel: %AI Model Inference
+/sdk/ai/azure-ai-inference/                          @dargilco @jhakulin @glharper
+
 # ServiceLabel: %AKS
 # ServiceOwners:                                     @Azure/aks-pm
 

diff --git a/.vscode/cspell.json b/.vscode/cspell.json
@@ -837,6 +837,13 @@
         "NDVI"
       ]
     },
+    {
+      "filename": "/sdk/ai/**",
+      "words": [
+        "ubinary",
+        "UBINARY"
+      ]
+    },
     {
       "filename": "/sdk/datalakeanalytics/**",
       "words": [

diff --git a/eng/code-quality-reports/src/main/resources/checkstyle/checkstyle-suppressions.xml b/eng/code-quality-reports/src/main/resources/checkstyle/checkstyle-suppressions.xml
@@ -286,6 +286,8 @@ the main ServiceBusClientBuilder. -->
   <suppress checks="com.azure.tools.checkstyle.checks.ServiceClientCheck" files="com.azure.ai.openai.(OpenAIClient|OpenAIAsyncClient).java"/>
   <!-- Checkstyle suppression for OpenAI Assistants client APIs that use Flux instead of PagedFlux for methods that return a collection -->
   <suppress checks="com.azure.tools.checkstyle.checks.ServiceClientCheck" files="com.azure.ai.openai.assistants.(AssistantsClient|AssistantsAsyncClient).java"/>
+  <!-- Checkstyle suppression for Inference client APIs that use Flux instead of PagedFlux for methods that return a collection -->
+  <suppress checks="com.azure.tools.checkstyle.checks.ServiceClientCheck" files="com.azure.ai.inference.(ChatCompletionsClient|ChatCompletionsAsyncClient).java"/>
 
   <!-- jdbc sdk suppression -->
   <suppress checks="com.azure.tools.checkstyle.checks.ExternalDependencyExposedCheck"

diff --git a/eng/versioning/version_client.txt b/eng/versioning/version_client.txt
@@ -42,6 +42,7 @@ com.azure:azure-ai-documentintelligence;1.0.0-beta.4;1.0.0-beta.5
 com.azure:azure-ai-documenttranslator;1.0.0-beta.1;1.0.0-beta.2
 com.azure:azure-ai-formrecognizer;4.1.10;4.1.10
 com.azure:azure-ai-formrecognizer-perf;1.0.0-beta.1;1.0.0-beta.1
+com.azure:azure-ai-inference;1.0.0-beta.1;1.0.0-beta.1
 com.azure:azure-ai-metricsadvisor;1.2.1;1.3.0-beta.1
 com.azure:azure-ai-metricsadvisor-perf;1.0.0-beta.1;1.0.0-beta.1
 com.azure:azure-ai-openai;1.0.0-beta.11;1.0.0-beta.12

diff --git a/pom.xml b/pom.xml
@@ -13,6 +13,7 @@
     <module>eng/code-quality-reports</module>
     <module>sdk/advisor</module>
     <module>sdk/agrifood</module>
+    <module>sdk/ai</module>
     <module>sdk/alertsmanagement</module>
     <module>sdk/anomalydetector</module>
     <module>sdk/aot</module>

diff --git a/sdk/ai/azure-ai-inference/CHANGELOG.md b/sdk/ai/azure-ai-inference/CHANGELOG.md
@@ -0,0 +1,13 @@
+# Release History
+
+## 1.0.0-beta.1 (Unreleased)
+
+- Azure AI Inference client library for Java. This package contains Microsoft Azure AI Inference client library.
+
+### Features Added
+
+### Breaking Changes
+
+### Bugs Fixed
+
+### Other Changes
diff --git a/sdk/ai/azure-ai-inference/README.md b/sdk/ai/azure-ai-inference/README.md
@@ -0,0 +1,225 @@
+# Azure AI Inference client library for Java
+
+Azure AI Inference client library for Java.
+
+This package contains the Azure AI Inference client library.
+
+## Documentation
+
+Various documentation is available to help you get started
+
+- [API reference documentation][docs]
+- [Product documentation][product_documentation]
+
+## Getting started
+
+### Prerequisites
+
+- [Java Development Kit (JDK)][jdk] with version 8 or above
+- [Azure Subscription][azure_subscription]
+
+### Adding the package to your product
+
+[//]: # ({x-version-update-start;com.azure:azure-ai-inference;current})
+```xml
+<dependency>
+    <groupId>com.azure</groupId>
+    <artifactId>azure-ai-inference</artifactId>
+    <version>1.0.0-beta.1</version>
+</dependency>
+```
+[//]: # ({x-version-update-end})
+
+### Authentication
+
+In order to interact with the Azure AI Inference Service you'll need to create an instance of client class,
+[ChatCompletionsAsyncClient][chat_completions_client_async] or [ChatCompletionsClient][chat_completions_client_sync] by using
+[ChatCompletionsClientBuilder][chat_completions_client_builder]. To configure a client for use with
+Azure Inference, provide a valid endpoint URI to an Azure Model resource along with a corresponding key credential,
+token credential, or [Azure Identity][azure_identity] credential that's authorized to use the Azure Model resource.
+
+#### Create a Chat Completions client with key credential
+Get Azure Model `key` credential from the Azure Portal.
+
+```java readme-sample-createSyncClientKeyCredential
+ChatCompletionsClient client = new ChatCompletionsClientBuilder()
+    .credential(new AzureKeyCredential("{key}"))
+    .endpoint("{endpoint}")
+    .buildClient();
+```
+or
+```java readme-sample-createAsyncClientKeyCredential
+ChatCompletionsAsyncClient client = new ChatCompletionsClientBuilder()
+    .credential(new AzureKeyCredential("{key}"))
+    .endpoint("{endpoint}")
+    .buildAsyncClient();
+```
+
+#### Create a client with Azure Active Directory credential
+Azure SDK for Java supports an Azure Identity package, making it easy to get credentials from Microsoft identity
+platform.
+
+Authentication with AAD requires some initial setup:
+* Add the Azure Identity package
+
+[//]: # ({x-version-update-start;com.azure:azure-identity;dependency})
+```xml
+<dependency>
+    <groupId>com.azure</groupId>
+    <artifactId>azure-identity</artifactId>
+    <version>1.13.1</version>
+</dependency>
+```
+[//]: # ({x-version-update-end})
+
+Authorization is easiest using [DefaultAzureCredential][wiki_identity]. It finds the best credential to use in its
+running environment. For more information about using Azure Active Directory authorization with OpenAI service, please
+refer to [the associated documentation][aad_authorization].
+
+```java readme-sample-createChatCompletionsClientWithAAD
+TokenCredential defaultCredential = new DefaultAzureCredentialBuilder().build();
+ChatCompletionsClient client = new ChatCompletionsClientBuilder()
+    .credential(defaultCredential)
+    .endpoint("{endpoint}")
+    .buildClient();
+```
+
+## Key concepts
+
+## Examples
+The following sections provide several code snippets covering some of the most common OpenAI service tasks, including:
+
+* [Chat completions sample](#chat-completions "Chat completions")
+* [Streaming chat completions sample](#streaming-chat-completions "Streaming chat completions")
+<!--
+* [Embeddings sample](#text-embeddings "Text Embeddings")
+-->
+
+## Examples
+
+### Chat completions
+
+```java readme-sample-getChatCompletions
+List<ChatRequestMessage> chatMessages = new ArrayList<>();
+chatMessages.add(new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."));
+chatMessages.add(new ChatRequestUserMessage("Can you help me?"));
+chatMessages.add(new ChatRequestAssistantMessage("Of course, me hearty! What can I do for ye?"));
+chatMessages.add(new ChatRequestUserMessage("What's the best way to train a parrot?"));
+
+ChatCompletions chatCompletions = client.complete(new ChatCompletionsOptions(chatMessages));
+
+System.out.printf("Model ID=%s is created at %s.%n", chatCompletions.getId(), chatCompletions.getCreated());
+for (ChatChoice choice : chatCompletions.getChoices()) {
+    ChatResponseMessage message = choice.getMessage();
+    System.out.printf("Index: %d, Chat Role: %s.%n", choice.getIndex(), message.getRole());
+    System.out.println("Message:");
+    System.out.println(message.getContent());
+}
+```
+For a complete sample example, see sample [Chat Completions][sample_get_chat_completions].
+
+Please refer to the service documentation for a conceptual discussion of [text completion][microsoft_docs_openai_completion].
+
+### Streaming chat completions
+
+```java readme-sample-getChatCompletionsStream
+List<ChatRequestMessage> chatMessages = new ArrayList<>();
+chatMessages.add(new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."));
+chatMessages.add(new ChatRequestUserMessage("Can you help me?"));
+chatMessages.add(new ChatRequestAssistantMessage("Of course, me hearty! What can I do for ye?"));
+chatMessages.add(new ChatRequestUserMessage("What's the best way to train a parrot?"));
+
+client.completeStream(new ChatCompletionsOptions(chatMessages))
+    .forEach(chatCompletions -> {
+        if (CoreUtils.isNullOrEmpty(chatCompletions.getChoices())) {
+            return;
+        }
+        StreamingChatResponseMessageUpdate delta = chatCompletions.getChoices().get(0).getDelta();
+        if (delta.getRole() != null) {
+            System.out.println("Role = " + delta.getRole());
+        }
+        if (delta.getContent() != null) {
+            String content = delta.getContent();
+            System.out.print(content);
+        }
+    });
+```
+
+To compute tokens in streaming chat completions, see sample [Streaming Chat Completions][sample_get_chat_completions_streaming].
+
+<!--
+### Text embeddings
+
+```java readme-sample-getEmbedding
+```
+For a complete sample example, see sample [Embedding][sample_get_embedding].
+
+Please refer to the service documentation for a conceptual discussion of [openAI embedding][microsoft_docs_openai_embedding].
+-->
+
+### Service API versions
+
+The client library targets the latest service API version by default.
+The service client builder accepts an optional service API version parameter to specify which API version to communicate.
+
+#### Select a service API version
+
+You have the flexibility to explicitly select a supported service API version when initializing a service client via the service client builder.
+This ensures that the client can communicate with services using the specified API version.
+
+When selecting an API version, it is important to verify that there are no breaking changes compared to the latest API version.
+If there are significant differences, API calls may fail due to incompatibility.
+
+Always ensure that the chosen API version is fully supported and operational for your specific use case and that it aligns with the service's versioning policy.
+
+## Troubleshooting
+### Enable client logging
+You can set the `AZURE_LOG_LEVEL` environment variable to view logging statements made in the client library. For
+example, setting `AZURE_LOG_LEVEL=2` would show all informational, warning, and error log messages. The log levels can
+be found here: [log levels][logLevels].
+
+### Default HTTP Client
+All client libraries by default use the Netty HTTP client. Adding the above dependency will automatically configure
+the client library to use the Netty HTTP client. Configuring or changing the HTTP client is detailed in the
+[HTTP clients wiki](https://github.com/Azure/azure-sdk-for-java/wiki/Configure-HTTP-Clients).
+
+### Default SSL library
+All client libraries, by default, use the Tomcat-native Boring SSL library to enable native-level performance for SSL
+operations. The Boring SSL library is an uber jar containing native libraries for Linux / macOS / Windows, and provides
+better performance compared to the default SSL implementation within the JDK. For more information, including how to
+reduce the dependency size, refer to the [performance tuning][performance_tuning] section of the wiki.
+
+For more details, see [TROUBLESHOOTING][troubleshooting] guideline.
+
+## Next steps
+
+## Contributing
+
+For details on contributing to this repository, see the [contributing guide](https://github.com/Azure/azure-sdk-for-java/blob/main/CONTRIBUTING.md).
+
+1. Fork it
+1. Create your feature branch (`git checkout -b my-new-feature`)
+1. Commit your changes (`git commit -am 'Add some feature'`)
+1. Push to the branch (`git push origin my-new-feature`)
+1. Create new Pull Request
+
+<!-- LINKS -->
+[product_documentation]: https://azure.microsoft.com/services/
+[docs]: https://azure.github.io/azure-sdk-for-java/
+[jdk]: https://learn.microsoft.com/azure/developer/java/fundamentals/
+[aad_authorization]: https://docs.microsoft.com/azure/cognitive-services/authentication#authenticate-with-azure-active-directory
+[azure_subscription]: https://azure.microsoft.com/free/
+[azure_identity]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity
+[sample_get_chat_completions]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetChatCompletionsSample.java
+[sample_get_chat_completions_streaming]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetChatCompletionsStreamSample.java
+[microsoft_docs_openai_completion]: https://learn.microsoft.com/azure/cognitive-services/openai/how-to/completions
+[microsoft_docs_openai_embedding]: https://learn.microsoft.com/azure/cognitive-services/openai/concepts/understand-embeddings
+[chat_completions_client_async]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIAsyncClient.java
+[chat_completions_client_builder]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIClientBuilder.java
+[chat_completions_client_sync]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIClient.java
+[logLevels]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/core/azure-core/src/main/java/com/azure/core/util/logging/ClientLogger.java
+[performance_tuning]: https://github.com/Azure/azure-sdk-for-java/wiki/Performance-Tuning
+[troubleshooting]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/TROUBLESHOOTING.md
+[wiki_identity]: https://learn.microsoft.com/azure/developer/java/sdk/identity
+
+![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-java%2Fsdk%2Fai%2Fazure-ai-inference%2FREADME.png)