Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add namespaces (packages) to packageInfo #41639

Merged
merged 8 commits into from
Aug 27, 2024

Conversation

JimSuplizio
Copy link
Member

@JimSuplizio JimSuplizio commented Aug 26, 2024

For the Spring people on this PR: I'm opportunistically fixing sdk/spring/spring-cloud-azure-starter-appconfiguration-config/pom.xml which has been causing a build warning because of the duplicate org.apache.maven.plugins:maven-jar-plugin definition.

The namespaces (packages) are added to the packageInfo json files prior to being processed by MSDocs. The problems with this are as follows:

  1. This doesn't happen until the Docs.MS Release task at release time, which is a bit late if there's a problem.
  2. The process that pulls namespaces does so from the javadoc and instead of using the one that's built, it downloads the javadoc jar, effectively making a needless network call.

The solution is to add the namespaces to the packageInfo files earlier in the process. Because I'm using the staged artifacts, the place to put this is just after the artifacts have been staged. The Fetch-Namespaces-From-Javadoc function in Docs-Toc.ps1 was updated to take the location of the javadoc jar. If the location was set, it'll use that instead of trying to download. This fixes #2 above.

Get the namespaces to update the packageInfo during the Build job, just after artifacts are copied to staging. This fixes #1. The reason for doing this here is that there were issues with people not having skipPublishDocMs set, meaning that it's expecting to produce docs, but the library was creating an empty javadoc jar with just the README so no namespaces could be determined.

Testing was done against the java - core - ci, which has libraries that publish docs, and java - template - ci whose libraries do not publish docs.

This fixes the Java part of Azure/azure-sdk-tools#7990

Note: I'm setting skipPublishDocMs and skipPublishDocGithubIo to true for com.azure.spring:spring-cloud-azure-appconfiguration-config-web because its POM file is producing an empty javadoc jar so we can't determine namespaces. This failure is known and needs to be fixed.

Q) Why use the Javadoc Jar instead of the actual library's jar?
A) The maven-javadoc-plugin definition has exclusions, things we don't want in docs. The main plugin definition, defined in azure-client-sdk-parent, has common exclusions with additional exclusions able to be defined in a library's individual pom file. This means that these classes are not part of the documentation and the exclusion is done at build time for the javadoc jar file. The library's jar would still have these classes meaning we'd have to duplicate the exclusions in some manner which would be janky, at best.

@azure-sdk
Copy link
Collaborator

API change check

API changes are not detected in this pull request.

Copy link
Member

@weshaggard weshaggard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest move the filtering into the script.

@JimSuplizio JimSuplizio force-pushed the NamespacesForPackageInfo branch from 4640047 to 8c10388 Compare August 27, 2024 17:27
@JimSuplizio
Copy link
Member Author

This is already past the point where all of my changes are being tested and core is having a random failure. Since my change to sdk/spring/spring-cloud-azure-appconfiguration-config/pom.xml fixes a version SNAFU affecting every pipeline I'm going to override check-enforcer and merge

@JimSuplizio JimSuplizio merged commit fad1f7e into Azure:main Aug 27, 2024
22 of 41 checks passed
@JimSuplizio JimSuplizio deleted the NamespacesForPackageInfo branch August 27, 2024 18:00
glharper pushed a commit that referenced this pull request Aug 28, 2024
* Add namespaces (packages) to packageInfo

* Actually name the script correctly

* Fix incorrect parameter name

* correct fail the step if no namespaces can be determined for one of the libraries being processed

* Fix the pom file to remove the duplicate org.apache.maven.plugins:maven-jar-plugin definition

* turn off docs for spring-cloud-azure-appconfiguration-config-web until it actually generates docs

* Updates for feedback

* Fix dependency version issue
glharper added a commit that referenced this pull request Sep 18, 2024
* initial auto-genned code

* use camel case for Json in class name

* initial work on setting up a test

* Fix some compile errors

* test actually compiles (and fails)

* add to azure-sdks-all

* fix json typo causing defaultCompleteOptions to throw

* add completeStreaming convenience method and InferenceSSE class

* add chat samples (sync), streaming convenience fields

* simplify convenience method for defaultCompleteOptions

* got test working in intellij

* add streaming chat test

* add streaming chat async sample and support code to async client

* add convenience method for async complete, basic async chat sample

* change options class name

* merge deletes

* add async tests for complete and completeStreaming

* remove refs to jackson package

* add chat completions code to README (links needed)

* add troubleshooting file, links in README

* add readmeSamples class for readme code sample auto gen

* update genned code with embeddingsClient, imageEmbeddingsClient

* revert parameters to Object type

* add embedding sample

* add Embeddings test classes

* add async embeddings test

* add text embeddings async sample

* make embed method public

* cleaner embeddings sample output

* fix javadoc gen

* fix dep versions

* phrasing

* add AAD sample with scope

* better explanation of auth scope setting in sample

* allow mvn compile

* create and allow playback for test recordings

* change display name in ci.yml

* add codeowners

* kick off new build

* add tests.yml

* remove EnableBatchRelease field

* update to pass codegen test

* use existing label

* add ubinary to ignored words

* use proper streaming types, remove delta from ChatChoice

* rename getEmbedding to getEmbeddingList, add auto-genned method

* make getModelInfo public for java

* use Json instead of JSON in class/file name

* sync ChatChoice with TS output

* set partial-update to true

* codegen more closely synced

* use latest typespec

* add customization code

* updated typespec

* App Config Correlation Context update + yaml hint fix (#41607)

* Updating correlation context to use pipeline context

* Fixing yaml hints

* Updating property docs from yaml

* removing unused imports

* Use code customizations to map internal Storage error in generated code (#41624)

Use code customizations to map internal Storage error in generated code

* Add namespaces (packages) to packageInfo (#41639)

* Add namespaces (packages) to packageInfo

* Actually name the script correctly

* Fix incorrect parameter name

* correct fail the step if no namespaces can be determined for one of the libraries being processed

* Fix the pom file to remove the duplicate org.apache.maven.plugins:maven-jar-plugin definition

* turn off docs for spring-cloud-azure-appconfiguration-config-web until it actually generates docs

* Updates for feedback

* Fix dependency version issue

* Publish code coverage results from 1 to2 (#41656)

* Update PublishCodeCoverageResults task version from 1 to 2

* Add prerequisite task

* last commit missed a file needing a save

* Fix typo (#41658)

* prepare for 1.2.27 bom release (#41659)

* Move logging token acquisition to VERBOSE (#41648)

* Sync eng/common directory with azure-sdk-tools for PR 8875 (#41660)

* chore(): add playwright-testing to product slugs

* Update eng/common/scripts/Test-SampleMetadata.ps1

---------

Co-authored-by: Siddharth Singha Roy <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>

* [App Configuration] Add test case for feature management lib (#41498)

* add feature management common test cases

* add test case for no filters

* initialize feature manager and call `featureManager.isEnabled()` to compare the result

* update the schema of test case

* get sample file and tests file by listFiles api

* update the todo comment.
Throw exception and use little endian are breaking change, will have both when update to 6.xx version.

* use `getContextClassLoader().getResource` to get the resource folder path

* address comment: typo wording error fix, add "@SuppressWarnings"

* address comment: typo wording error fix

* address comment: update member name

* add running log

* use contains to filter out the "TargetingFilter.sample.json"

* add some info log

* sort the file list

* move to another the package to avoid making feature manager construction as public

* address comment: use logging, remove else

* address comment: throw exception when empty "feature_management" section.

* address comment: rename symbol

* Update ValidationsTest.java

* fixing linting

* address comment: ignore both targeting filter test case

---------

Co-authored-by: Matt Metcalf <[email protected]>

* Retrieve TokenCredential from ConfigurableBootstrapContext for env post processor (#41580)

* get TokenCredential from ConfigurableBootstrapContext

---------

Co-authored-by: Muyao <[email protected]>
Co-authored-by: Muyao Feng <[email protected]>

* [Automation] Generate SDK based on TypeSpec 0.20.0 (#41666)

* [Automation] Generate SDK based on TypeSpec 0.20.0

* update assets.json

* revert healthinsights changes

---------

Co-authored-by: Haoling Dong <[email protected]>

* add customization to unfinalize base classes

* more typespec syncs

* correct doc comment

* make extra parameters public

* fix bad gen

* remove dupe imports

* sync with typespec

* add lint exception for PagedFlux

* lint fixes

* App Config Correlation Context update + yaml hint fix (#41607)

* Updating correlation context to use pipeline context

* Fixing yaml hints

* Updating property docs from yaml

* removing unused imports

* [OpenAI-Assistant] Added support for 2024-07-01 (#41464)

* Added support 2024-07-01 service API version (#41473)

* Default to sub config file paths/svc connection for public + sovereign live tests (#41267)

* remove azure.xml dependency

* make completeWithResponse public

* make complete with options public

* add string constructor for ChatRequestUserMessage

* add scopes to ClientBuilders, change parameters in FunctionDefinition to BinaryData type

* update comment

* correct string constructor for ChatRequestUserMessage

* formatting

* add streaming tool call sample

* add Mono<Response<>> completeWithResponse API to AsyncClient

* better implementation of completeWithResponse

* remove unused import

* add embedWithResponse API to embeddings clients

* add tests for complete(options) and completeWithResponse APIs

* more sync tests

* add stream response test

* add completeStreamWithResponse test, change completeStreaming to completeStream

* new test recordings

* changes for TypeSpec sync

* fix pom error

* add deleted file

* restore deleted files

* restore more deleted files

* even more deleted files

* pull in openai changes from main

* more deleted files

* unmod openai file

* update HTTP client link

* spacing for typespec gen

* update dependencies

* spacing

* spacing

* spacing

* update min jacoco settings

* trigger CI build

* Naive implementations of new APIs fromContentItems and file constructor for ChatMessageImageContentItem

* spacing

* add image read sample and correct API implementations

* sanity check, cleaner syntax, and samples style fixes

* Add ImageUrlChatSample

* update dependency version

* typespec sync

* review feedback

* Update sdk/ai/azure-ai-inference/README.md

Co-authored-by: Bill Wert <[email protected]>

* Update sdk/ai/azure-ai-inference/README.md

Co-authored-by: Bill Wert <[email protected]>

* fix TypeSpec crash

* remove unneeded paragraph

* make TypeSpec sync happy

* add access helper to ChatCompletionsOptions

* typespec sync

* review feedback

---------

Co-authored-by: Matthew Metcalf <[email protected]>
Co-authored-by: Alan Zimmer <[email protected]>
Co-authored-by: James Suplizio <[email protected]>
Co-authored-by: Shawn Fang <[email protected]>
Co-authored-by: Bill Wert <[email protected]>
Co-authored-by: Azure SDK Bot <[email protected]>
Co-authored-by: Siddharth Singha Roy <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>
Co-authored-by: ivywei0125 <[email protected]>
Co-authored-by: Xiaolu Dai <[email protected]>
Co-authored-by: Muyao <[email protected]>
Co-authored-by: Muyao Feng <[email protected]>
Co-authored-by: Haoling Dong <[email protected]>
Co-authored-by: Ben Broderick Phillips <[email protected]>
jairmyree pushed a commit to jairmyree/azure-sdk-for-java that referenced this pull request Sep 23, 2024
* Add namespaces (packages) to packageInfo

* Actually name the script correctly

* Fix incorrect parameter name

* correct fail the step if no namespaces can be determined for one of the libraries being processed

* Fix the pom file to remove the duplicate org.apache.maven.plugins:maven-jar-plugin definition

* turn off docs for spring-cloud-azure-appconfiguration-config-web until it actually generates docs

* Updates for feedback

* Fix dependency version issue
mssfang added a commit to mssfang/azure-sdk-for-java that referenced this pull request Oct 21, 2024
* initial auto-genned code

* use camel case for Json in class name

* initial work on setting up a test

* Fix some compile errors

* test actually compiles (and fails)

* add to azure-sdks-all

* fix json typo causing defaultCompleteOptions to throw

* add completeStreaming convenience method and InferenceSSE class

* add chat samples (sync), streaming convenience fields

* simplify convenience method for defaultCompleteOptions

* got test working in intellij

* add streaming chat test

* add streaming chat async sample and support code to async client

* add convenience method for async complete, basic async chat sample

* change options class name

* merge deletes

* add async tests for complete and completeStreaming

* remove refs to jackson package

* add chat completions code to README (links needed)

* add troubleshooting file, links in README

* add readmeSamples class for readme code sample auto gen

* update genned code with embeddingsClient, imageEmbeddingsClient

* revert parameters to Object type

* add embedding sample

* add Embeddings test classes

* add async embeddings test

* add text embeddings async sample

* make embed method public

* cleaner embeddings sample output

* fix javadoc gen

* fix dep versions

* phrasing

* add AAD sample with scope

* better explanation of auth scope setting in sample

* allow mvn compile

* create and allow playback for test recordings

* change display name in ci.yml

* add codeowners

* kick off new build

* add tests.yml

* remove EnableBatchRelease field

* update to pass codegen test

* use existing label

* add ubinary to ignored words

* use proper streaming types, remove delta from ChatChoice

* rename getEmbedding to getEmbeddingList, add auto-genned method

* make getModelInfo public for java

* use Json instead of JSON in class/file name

* sync ChatChoice with TS output

* set partial-update to true

* codegen more closely synced

* use latest typespec

* add customization code

* updated typespec

* App Config Correlation Context update + yaml hint fix (Azure#41607)

* Updating correlation context to use pipeline context

* Fixing yaml hints

* Updating property docs from yaml

* removing unused imports

* Use code customizations to map internal Storage error in generated code (Azure#41624)

Use code customizations to map internal Storage error in generated code

* Add namespaces (packages) to packageInfo (Azure#41639)

* Add namespaces (packages) to packageInfo

* Actually name the script correctly

* Fix incorrect parameter name

* correct fail the step if no namespaces can be determined for one of the libraries being processed

* Fix the pom file to remove the duplicate org.apache.maven.plugins:maven-jar-plugin definition

* turn off docs for spring-cloud-azure-appconfiguration-config-web until it actually generates docs

* Updates for feedback

* Fix dependency version issue

* Publish code coverage results from 1 to2 (Azure#41656)

* Update PublishCodeCoverageResults task version from 1 to 2

* Add prerequisite task

* last commit missed a file needing a save

* Fix typo (Azure#41658)

* prepare for 1.2.27 bom release (Azure#41659)

* Move logging token acquisition to VERBOSE (Azure#41648)

* Sync eng/common directory with azure-sdk-tools for PR 8875 (Azure#41660)

* chore(): add playwright-testing to product slugs

* Update eng/common/scripts/Test-SampleMetadata.ps1

---------

Co-authored-by: Siddharth Singha Roy <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>

* [App Configuration] Add test case for feature management lib (Azure#41498)

* add feature management common test cases

* add test case for no filters

* initialize feature manager and call `featureManager.isEnabled()` to compare the result

* update the schema of test case

* get sample file and tests file by listFiles api

* update the todo comment.
Throw exception and use little endian are breaking change, will have both when update to 6.xx version.

* use `getContextClassLoader().getResource` to get the resource folder path

* address comment: typo wording error fix, add "@SuppressWarnings"

* address comment: typo wording error fix

* address comment: update member name

* add running log

* use contains to filter out the "TargetingFilter.sample.json"

* add some info log

* sort the file list

* move to another the package to avoid making feature manager construction as public

* address comment: use logging, remove else

* address comment: throw exception when empty "feature_management" section.

* address comment: rename symbol

* Update ValidationsTest.java

* fixing linting

* address comment: ignore both targeting filter test case

---------

Co-authored-by: Matt Metcalf <[email protected]>

* Retrieve TokenCredential from ConfigurableBootstrapContext for env post processor (Azure#41580)

* get TokenCredential from ConfigurableBootstrapContext

---------

Co-authored-by: Muyao <[email protected]>
Co-authored-by: Muyao Feng <[email protected]>

* [Automation] Generate SDK based on TypeSpec 0.20.0 (Azure#41666)

* [Automation] Generate SDK based on TypeSpec 0.20.0

* update assets.json

* revert healthinsights changes

---------

Co-authored-by: Haoling Dong <[email protected]>

* add customization to unfinalize base classes

* more typespec syncs

* correct doc comment

* make extra parameters public

* fix bad gen

* remove dupe imports

* sync with typespec

* add lint exception for PagedFlux

* lint fixes

* App Config Correlation Context update + yaml hint fix (Azure#41607)

* Updating correlation context to use pipeline context

* Fixing yaml hints

* Updating property docs from yaml

* removing unused imports

* [OpenAI-Assistant] Added support for 2024-07-01 (Azure#41464)

* Added support 2024-07-01 service API version (Azure#41473)

* Default to sub config file paths/svc connection for public + sovereign live tests (Azure#41267)

* remove azure.xml dependency

* make completeWithResponse public

* make complete with options public

* add string constructor for ChatRequestUserMessage

* add scopes to ClientBuilders, change parameters in FunctionDefinition to BinaryData type

* update comment

* correct string constructor for ChatRequestUserMessage

* formatting

* add streaming tool call sample

* add Mono<Response<>> completeWithResponse API to AsyncClient

* better implementation of completeWithResponse

* remove unused import

* add embedWithResponse API to embeddings clients

* add tests for complete(options) and completeWithResponse APIs

* more sync tests

* add stream response test

* add completeStreamWithResponse test, change completeStreaming to completeStream

* new test recordings

* changes for TypeSpec sync

* fix pom error

* add deleted file

* restore deleted files

* restore more deleted files

* even more deleted files

* pull in openai changes from main

* more deleted files

* unmod openai file

* update HTTP client link

* spacing for typespec gen

* update dependencies

* spacing

* spacing

* spacing

* update min jacoco settings

* trigger CI build

* Naive implementations of new APIs fromContentItems and file constructor for ChatMessageImageContentItem

* spacing

* add image read sample and correct API implementations

* sanity check, cleaner syntax, and samples style fixes

* Add ImageUrlChatSample

* update dependency version

* typespec sync

* review feedback

* Update sdk/ai/azure-ai-inference/README.md

Co-authored-by: Bill Wert <[email protected]>

* Update sdk/ai/azure-ai-inference/README.md

Co-authored-by: Bill Wert <[email protected]>

* fix TypeSpec crash

* remove unneeded paragraph

* make TypeSpec sync happy

* add access helper to ChatCompletionsOptions

* typespec sync

* review feedback

---------

Co-authored-by: Matthew Metcalf <[email protected]>
Co-authored-by: Alan Zimmer <[email protected]>
Co-authored-by: James Suplizio <[email protected]>
Co-authored-by: Shawn Fang <[email protected]>
Co-authored-by: Bill Wert <[email protected]>
Co-authored-by: Azure SDK Bot <[email protected]>
Co-authored-by: Siddharth Singha Roy <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>
Co-authored-by: ivywei0125 <[email protected]>
Co-authored-by: Xiaolu Dai <[email protected]>
Co-authored-by: Muyao <[email protected]>
Co-authored-by: Muyao Feng <[email protected]>
Co-authored-by: Haoling Dong <[email protected]>
Co-authored-by: Ben Broderick Phillips <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Include CHANGELOG Blob storage hangs for files > about 3500 kb
4 participants