Discussion on Rethinking Design and Responsibility Scope

[vvatanabe]

Overview

Currently, Go OpenAI supports both OpenAI API and Azure OpenAI Service. Many contributors have been actively supporting both, but over time, differences in specifications between the OpenAI API and Azure OpenAI have been emerging.
For instance, discrepancies can be observed in the following:

• When the azure service is unavailable, the error message returned does not contain the error node #465
• Compatible with the 2023-07-01-preview API interface of Azure Openai, when content interception is triggered, the error message will contain innererror #460
• Add support for azure openai new version API (2023-07-01-preview) #451
• Implement Azure DALL-E 2 CreateImage Support (#252) #391
• fix json marshaling error response of azure openai (#343) #345
• Fix OpenAI or Azure returns an undefined error message return #283

The code abstracting both OpenAI API and Azure OpenAI Service is becoming increasingly complex, compromising its maintainability as a result. Moreover, it becomes challenging for users to determine which specific fields in the requests or responses are valid for the OpenAI API versus the Azure OpenAI Service.

Given this, there's a perceived need to discuss the future design and responsibility scope of Go OpenAI.

Proposal

I propose the following two approaches:

1. Separate the code for OpenAI API and Azure OpenAI Service, placing each in different repositories.
2. Keep the code for both OpenAI API and Azure OpenAI Service in the same repository but separate them into distinct packages.

Each of these approaches has its own merits and demerits. I have summarized each below.

1. Separate the code for OpenAI API and Azure OpenAI Service, placing each in different repositories

Pros:

• Clear Responsibilities: Each repository can focus on a specific service, making its structure and purpose clear.
• Independent Releases: When updating or making changes to a specific service, there's no need to consider the other service.
• Independent Issue Tracking: Problems with each API/service can be tracked separately.

Cons:

• Code Duplication: If there are common functionalities or utilities needed for both services, there might be a need to duplicate such code across the two repositories.
• Lack of Consistency: If unified updates or improvements are necessary, they would have to be implemented across both repositories.

Note: As proposed by Alex, there's also an option to maintain separate libraries for OpenAI API and Azure OpenAI Service within the https://github.com/go-llm organization.

2. Keep the code for both OpenAI API and Azure OpenAI Service in the same repository but separate them into distinct packages

Pros:

• Code Reusability: Common functionalities and utilities can be maintained in a single repository and reused as necessary.
• Unified Documentation: Documentation for both the API/service can be consolidated in one place.
• Unified Issue Tracking: Issues and feedback can be managed within a single tracking system.
• Consistent Codebase: Applying a common style guide or best practices becomes easier.

Cons:

• Complexity: Maintaining both the API and service within a single repository might make the structure more complex.
• Ambiguity of Scope: For new contributors, it might be challenging to determine which packages or code sections are related to which API/service.

I would love to discuss this with all of you in this open forum.

[zjy282]

I suggest option 2. Although OpenAI API and Azure OpenAI Service have gradually produced some differences, I think the same parts of the two are still overwhelmingly

[vvatanabe]

FYI: I found azure-sdk-for-go as the official SDK for Azure. This SDK supports Azure OpenAI Service.
https://github.com/Azure/azure-sdk-for-go

I'm asking out of curiosity, but what could be the reason for using Go OpenAI instead of the official SDK for Azure OpenAI Service?

[zjy282]

FYI: I found azure-sdk-for-go as the official SDK for Azure. This SDK supports Azure OpenAI Service. https://github.com/Azure/azure-sdk-for-go

I'm asking out of curiosity, but what could be the reason for using Go OpenAI instead of the official SDK for Azure OpenAI Service?

I use Go OpenAI for the following reasons:

1. Start using it when there is no Azure version sdk
2. The project supports both Azure's request and OpenAI's request

[ZeroDeng01]

The reason why I use go-openai is mainly the following 3 points:

• Azure did not provide Go SDK when my project was created, and it is a waste of development resources to replace the SDK now;
  • See: https://github.com/Azure/azure-sdk-for-go/commits/main/sdk/ai/azopenai
• Some of our projects cannot be accessed because of the policy of the implementation site Openai's api, so we are compatible with azure's services, but azure's api update has been several versions slower than the official openai, so our project has to support both openai and azure openai according to the region, so we chose to support both azure openai and openai
• Azure’s official sdk update is sometimes not very timely, and it is usually troublesome for us to submit a PR. When there are some API new features updated, we can smoothly upgrade by submitting a PR to go-openai.

About Discussion on Rethinking Design and Responsibility Scope

At present, go-openai is compatible with azure openai and openai through the unified entrance, which is very convenient to use. The caller usually does not need to make too many code modifications to be compatible with the APIs of these two service providers. However, as the differentiation between azure and openai increases in the future, the maintenance of the sdk will be very troublesome, so I personally suggest that the sdk of the subsequent two service providers should be placed in a code warehouse and divided into two packages, but the entry method is unified, through Adding a tag parameter or a function pointer allows the entry method to distinguish whether to use the azure or openai package to analyze the return parameters, because there will not be much difference in the difference between azure and openai in terms of requests, mainly because the difference in the returned content is relatively large.

[ealvar3z]

I recommend a clean split and consolidating it all under the go-llm organization

[coggsflod]

@ealvar3z the go-llm project is super interesting, however it's a project that I'm sure many folks would be wary of running in an enterprise production environment given the number of experimental features in the code (ie. arbitrary BASH code execution on the host machine).