Skip to content

Commit

Permalink
Storage/feature branch merge with master (#7276)
Browse files Browse the repository at this point in the history
* Update Tracing README versions (#6642)

* Changed Datalake to beta instead of preview (#6644)

* Add maven profile to verify readme (#6625)

* Add maven profile to verify readme

* Revert changes to readme to pass the CI pipeline

* Separate profiles for Windows

* Workaround template behavior change. (#6652)

This commit should only impact live tests which are currently broken. Checking this in as an emergency work around. Might become permanent if this behavior change sticks.

* Profile activation for verify readme (#6650)

* Add maven profile to verify readme

* Revert changes to readme to pass the CI pipeline

* Separate profiles for Windows

* Profile activation for verify readme

* Revert test changes

* Track 1 data plane dev packages (#6655)

* Add dev package generation to data plane.

* Add test skip condition.

* Cosmos Issue #6240: Port read-my-writes fix from V4 (#6242)

* Port from v4

* Corrected package misspelling in log4j.properties and removed System.exit from Main.java

* Responded to code review comments

* Cosmos Issue #6253: Port RntbdRequestTimer memory leak fix from V4 (#6254)

* Port from v4

* Corrected package misspelling in log4j.properties and removed System.exit from Main.java

* Port from V4

* Responded to code review comments

* Reverting tests.yml file back to master

* Prevent HttpLoggingPolicy Consuming Body (#6638)

* Prevent HttpLoggingPolicy from consuming request and response bodies

* Updated tests to use named ClientLogger and updated tests to use a string converted to bytes

* Fixing linting issues

* Remove hard-coded version and name from clients (#6557)

* Remove hard-coded version and name from clients

* Excluding beanutils library from commons validator because of security vulnerability (#6649)

* Exposed default time to live API on CosmosContainerProperties (#6651)

* Kv keys updates (#6657)

Kv Keys update

* Kv keys version updates (#6682)

* API updates

* samples update

* API changes

* sample updates + checkstyle fixes

* feedback updates

* compile fix

* feedback update + sans update + checkstyle fixes

* api updates

* feedback updates

* feedback updates

* javadoc update

* checkstyle fix

* compile error fix

* code updates

* remove package-info file

* feedback + tests update

* feedback updates

* remove unwanted jacoco file

* secret version update

* version file update

* pom version updates

* add tests

* tests update

* drop codecs dependency

* update base 64 usage

* spotbugs fix

* feedback update

* code updates

* build ifx

* changelog + tests update

* keys version updates

* feedback updates

* Update package index links in README.md (#6672)

Package index is now controlled centrally in the azure-sdk repo
https://github.com/Azure/azure-sdk/blob/master/_data/releases/latest/java-packages.csv

* Revert "Remove hard-coded version and name from clients (#6557)" (#6684)

This reverts commit 5e37d79.

* Update versions of released libraries (#6685)

* Fixed issue where account SAS does not work on clients (#6601)

* Introducing Revapi into build system (#6381)

* Introducing Revapi into build system. Currently it is configured to not fail the build (because it is comparing the 1.0.0-preview.N releases rather than 1.0.0). Once this versioning issue is fixed, we should flip this to fail the build.

* Add revapi versions to external-dependencies.txt

* Ignore packages that are under implementation.

* Fail build if there are API inconsistencies.

* Add versionFormat string to compare.

* Skip revapi in azure-core-test

* Use resource file in sdk-build-tools instead of having to resolve relative path based on pom.

* Revapi not needed for aggregate report.

* Storage: regenerate package-2019-06 (#6597)

* CosmosDB: regenerate package-2019-08-preview (#6628)

* CosmosDB: regenerate package-2019-08-preview

* CosmosDB: regenerate package-2019-08-preview

* Post ga version updates dec2019 (#6706)

* Update com.azure:azure-security-keyvault-keys version post release

* Update file versions through the update script

* Blob Sas (#6409)

* Add Custom Header - In AddHeaderPolicy and RequestIDPolicy  (#6602)

New AddHeadersFromContextPolicy to add multiple headers and values for those headers.
Allowing override header name in RequestIdPolicy.

* Add Live TestMode and Skip Recording Concepts (#6671)

* Added TestMode Live, DoNotRecord annotation, and TestContextManager

* Fix linting issue

* Updated new constructors to take TestContextManager instead of splayed parameters

* Updated unit tests based on code changes

* Add missing dependency comment

* Fixing linting issues

* Remove Queue Cryptography Module (#6715)

* Removed Queue Cryptography references since the module is unused

* Delete Queue Cryptography POM file

* Queue, File and Datalake Sas (#6709)

* Fixed bug where eTag would not deserialize in listPaths (#6723)

* Fixed parsing of consistency level string value from property bag (#6708)

* README for samples in Event Hubs (#6710)

* README for samples in Event Hubs

* Add impressions

* Add missing readme sections

* Update root readme with reference to sample readme

* init CI pipeline (#6731)

* init CI pipeline

* correct module and artifact name

* Update ci.yml

* HDInsight: regenerate package-2018-06-preview (#6705)

* Add missing HTTP request methods to HttpMethod enum (#6729)

* Add OPTIONS to HttpMethod enum

* Add all missing HTTP methods

* Clients fetches module name and version from property file (#6734)

* Added null value holder change (#6737)

* [ServiceBus] Minor change to teardown in tests so teatdown doesn't throw exception if setup fails for any reason (#6732)

Minor change to teardown in tests so teatdown doesn't throw exception if setup fails for any reason

* Changing dependencies with current version of core, since we added new functionality in core for AddHeadersFromContextPolicy (#6745)

* Enhance Blob Test Reliability (#6566)

* Updating tests to enhance reliability

* Cosmos ChangeFeedProcessor fixes (#6744)

- fix to terminate the current processing threads when the lease renewal worker thread has failed.
- fix to out of bounds exception when the leases load balancer thread is trying to aquire a lease.

* Update root readme for December releases (#6749)

* AutoRest: update autorest.java cmd (#6735)

* AutoRest: update autorest.java cmd

* AutoRest: update by review comments

* AutoRest: restrict version number validation

* Run live tests on all operating systems (#6668)

* live tests for all OS

* Use values from global for Maven options

* Changes for Cosmos

* Correctly set the DisplayName

* set DisplayName part 2

* adjust spacing in yml

* set display name

* Remove the setting of display name to empty

* Added compiler args to suppress unused warnings in samples and test classes when compiling with Groovy (#6724)

* Added some tests for making up the test coverage (#6754)

* Added some tests for make up the logging and FluxUtils coverage

* Update version tags on tracing readme (#6766)

* update samples and package versions

* Revert "update samples and package versions"

This reverts commit 9e9951b.

* update tracing readme

* update to include md files in version update:

* Added support for exists method on PathClient (#6767)

* Ignore null headers and allow full url paths (#6760)

* Add changelog to artifacts if it exists (#6746)

* AzConfig: User provided Customized Headers Policy  (#6544)

Added support for user-provided customized headers

* Publish test results even if build is canceled (#6780)

- If build is cancelled due to agent timeout, some tests results may have already been generated and should still be published

* More tests on fluxUtil and Base64Util (#6770)

* Added response decoder configuration to fix max header size issue (#6690)

* Add eventhub properties to processing spans (#6768)

* Add "eng" folder to core path triggers (#6803)

- Aligns Java with other languages

* Added support for ADLS Error Mapping (#6753)

* Added changeLog for azure-core (#6805)

* Added changeLog for azure-core

* Added overloads that do not overwrite by default (#6774)

* Add connection string null or empty message error (#6811)

* 'connectionString' cannot be null or empty.

* Setting selector filter  (#6693)

Allowed user to define KeyFilter and LabelFilter

* Fixes wrong property file name with unknown name and unknown version (#6813)

* fixes wrong property file name and update headers for all json files

* Shorten tracing span names (#6810)

* This will snapshot the repo state for analysis if the build fails. (#6826)

* Capture repo state during build. (#6828)

* Remove repo capture logic (#6836)

* Removes the repo capture logic since it is relatively expensive.

* Move to condition for repo capture so we can do it easily in the future.

* update service mapper for missing dirs (#6847)

* Fixed storage flaky live tests (#6677)

* Change live tests to live mode

* Fixed the queue and file sas tests

* Updated release versions to 3.5.0 (#6852)

* Add managed identity live tests in new e2e module (#6833)

* Add e2e module

* Add managed identity e2e test

* Add e2e module to pom.client.xml

* Try to fix assume

* Work around global configuration issue

* Remove pfx

* Fix configuration usage

* Fix dependency version tag

* Fix dependency version script

* Fix again

* Skip key vault tests when env var is missing

* Check key vault everywhere

* Fix Event Hubs READMEs (#6849)

* Move old README into correct directory.

* Update READMEs for current released versions.

* Fix readme for track 1

* Update qpid-proton-j-extensions dependency (#6743)

* Update to qpid-proton-j-extensions 1.2.2

* Update module info.

* Add links on batch send operation in eventhubs (#6773)

* Add Migration Guide for Event Hubs (#6854)

* Adding .vscode and .factorypath to gitignore.

*  Fixing typo in Azure storage migration guide.

* Adding exception handling and event processor client sample.

* Adding link to additional samples.

* Fixed storage flaky live tests Part 2 (#6850)

* Remove surefire failed test reruns (#6883)

* Doc index updates (#6871)

* Set core dep version to 1.1.0 (#6928)

* Set core dep version to 1.1.0

* More libs needing unreleased dep core version

* Prepare Repo for DocWarden Updates  (#6662)

* Update, docsettings.yml, packages.md, update readme and changelog case.

* Update docsettings

* Fix azure-data-appconfigurationchangelog (#6932)

* KeyVault Certificate client API review consistency update (#6823)

* Update Certificates library version in README  (#6939)

* Add readme exclusion for path (#6940)

* Add exclusion fir missing changelog (#6941)

* Updated versions to next beta version (#6926)

* Test against all built (#6927)

* Test run with everything built from source

* add missing paren

* use python not python3 for from source

* Set return code from pwsh

* Verify succeeded before running tests

* Rev up azure-core-test beta after release (#6938)

* Rev up azure-core-test beta after release
* Release  new  azure-core-test beta
* storage  depend on unreleased version of  azure core test temporarily. will change it back today

* Update certs version post preview release (#6948)

* Fix changelog (#6951)

* Updated storage versions post release (#6953)

* Fix Java Links in the root README (#6925)

* Fix Java Links in the root README

* Update core readme.md link

* run rel link replace before creating doc index

* Update for python args quotes problem

* Update storage-blob changelog (#6974)

* Configure single shot upload (#6950)

* Added maximumSingleUploadSize option to ParallelTransferOptions. Started tests

* Changelog

* Added tests

* Changelog update

* More Robust BlobBatch Implementation (#6882)

* Added unit tests for single operation failing and submitting batch multiple times

* More robust Blob Batch implementation

* Additional comments and change to use monoError instead of Mono.error

* Text analytics Preview 1  (#6161)

Have a preview version of Text Analytics

* Policy: generate package-2019-09 (#6630)

* Policy: generate package-2019-09

* Policy: add new module for pom.mgmt.xml

* increase and verify version functions (#6980)

* increase and verify version functions

* Update for feedback

* actually remove the unnamed anchored regex this time

* Updates Event Hubs samples (#6986)

* Update ConsumeEvents sample.

* Add producer samples for web sockets and proxy.

* Add sample for publishing streams of events.

* Rename EventProcessor -> EventProcessorClient.

* Adding EPH sample.

* Update READMEs. Remove dead links.

* Update error handling logic and add tests for EPC (#6978)

* Update error handling logic and add tests for EPC

* Fix unit tests

* Add message to exceptions and make initial position exclusive

* Revert changes to make event position exclusive

* Correct documentation of `EventPosition.FromOffset` (#6990)

In #6986 we changed the semantics of `EventPosition.FromOffset` to
return an exclusive instead of inclusive offset. However, we missed
updating the documentation for the method to match the new semantics.

This corrects that issue.

* Remove isInclusive testcase for offset (#6991)

* Changes for unreleased dependencies (#6989)

* Changes for unreleased dependencies

* Forward slashes in the use case commands in set_version

* [Event Hubs] Use code snippets from .java files in README (#6993)

* Use code snippets from .java files in README

* fix path

* Fix checkstyle

* Media Services: generate 2018 07 01 (#7053)

* Media Services: generate 2018 07 01

* feat: change version

* remove duplicate version (#7055)

* Add JUnit engine dependency to run unit tests for Text Analytics (#7052)

* Update for Disabled tests in Text Analytics (#7061)

* readme updates (#7019)

* readme uodates

* update correct link for API doc

* key concept

* update sample links

* sample fixes

* Added ability to overwrite for download to file (#6994)

* Changing the version of azure client authentication library dependency (#6987)

* Changing the version of azure client authentication library dependency. Need this to solve a compile warning.

* Correcting version change in external dependencies.

* Split Text Analytics to function based package private clients (#7050)

* ChangeLog updates for core modules (#7066)

* Net App: generate 2019 10 01 (#7088)

* Net App: generate 2019 10 01

* fix: add module in pom.mgmt.xml

* Unit tests for user error handler (#7048)

* User error tests

* Update unit tes

* Use jdk 8 methods

* Add unit tests

* Move version and product name from amqp to event hubs library (#7092)

* Move version and product name from amqp to event hubs library

* Update AMQP library pass user agent info from client library

* Fix checkstyle

* January release versions for azure-core - getting ready (#7040)

* January release versions update for azure-core only. Also have README, CHANGELOG updates.

* draft to update contributing for updated versioning (#7093)

* draft to update contributing for updated versioning

* update a couple of missed write calls to output errors in red

* Update for feedback

* Update for feedback, part 2

* Added async samples and refactored sync samples (#7073)

* add async samples and refactor sync samples

* Initial update for January 2020 release (#7098)

Update old November and December references to January.

A second PR will be needed to update all links to the latest README.md Tags next week

* AAD sample and README revised, and correct indentation (#7074)

* revised aad sample and readme

* indentation

* add ReadmeSamples and update readme automatically

* fixes spotbugs

* try catch final revised

* Doc Index Generation Clean up (#7102)

* BlobOutputStream using BufferedUpload (#7067)

* core interdependencies should be using current (#7110)

* Added a package for Java nio implementation (#7105)

* Added a package for Java nio implementation

* Corrected version number

* cleanup

* PR feedback

* docsettings and empty class for compiling

* Javadocs

* Port from v2.6 | Default Direct TCP Options feature to v3 (#7123)

* Reactor netty connection pooling (#7130)

* Added some logging around open connections

* Removed extra logging

* Removed dispose calls to re-use connection

* Removed empty close method

* fix parent tag validation (#7141)

* KV-Keys code update (#7075)

KV keys code update.

* changing previously missed readme (#7114)

* connectionString() method should throw IllegalArgurmentException (#7121)

* added throw IllegalArgException

* add release info for 2020 January (#7116)

* add release info for 2020 January

* connection string

* connection string changelog

* Use standard scheduler and dispose of Integration test one (#7127)

* Use standard scheduler and dispose of it.

* Using elastic scheduler.

* Update Event Hubs and azure-core-amqp current versions and CHANGELOG to GA (#7100)

* Remove version_client_java_files.

* Update current-version to GA for Event Hubs.

* Update README files with versions.

* Updating azure-core-amqp CHANGELOG

* Update Event Hubs changelog.

* Fix error in update_version script since files.txt is removed.

* Fix dependency versions for azure-core-amqp.

* Update azure-messaging-eventhubs to use correct pom.

* Print out test name before running test (#7109)

* Cosmosdb release 3.6.0-beta.1 (#7146)

* Updated release versions to 3.6.0-SNAPSHOT

* Updated snapshot to beta.1 suggested by azure core team

* Updated versions to correct dependency and current version. Added helpful comments in version_data.txt file

* Added canLogAtLevel API in logging. (#6886)

* Added canLogAtLevel API in logging.

* Inject codesnippets for readme, update changelog Text Analytics (#7118)

* Update AzConfig release version to 1.0.1 (#7151)

* update release version

* Code Owners for Text Analytics (#7160)

* Code owners for Text Analytics

* Fix spacing

* Additional spacing fixes

* Last of the spacing fixes

* release update and minor fix (#7161)

* [eventhubs] Add a note about Roles for AAD Auth (#7157)

Add a small note to the `README.md` which notes that when using AAD that
you need to assign a specific role to your principal in order to access
Event Hubs.

* Updated changelog and versions post release (#7164)

* Updated changelog and versions post release

* Updated comment in version_data.txt file

* Added basic CI file for spring boot. (#7196)

* Policy Insights: generate 2019 10 01 (#6793)

* policyinsights: generate 2019 10 01

* fix: stage name error

* fix: add module in pom.mgmt.xml

* IoT Central: generate 2018 09 01 (#6792)

* IoT Central: generate 2018 09 01

* change version in pom to match release

* feat: change version to 1.0.0 according to service team

* Storage Release Version Update (#7155)

* Version up after azure core release.  (#7163)

* Version up after azure core release. Validate via pom_file_version_scanner.ps1

* reming version up in README,  since they are not released yet.

* up core test ver

* review comments

* update_versions.py fix for update-type all (#7231)

* update_versions.py fix for update-type all
* fix spacing in .yml file

* update release date to current date (#7235)

* KV version updates Jan 2020 (#7144)

Update KV versions.

* Improve logging and update consumer close logic (#7175)

* Improve logging and update consumer close logic

* Update localization text message

* Fix unit test

* Fix checkpointstore tests

* Update identity for 1.0.2 (#7104)

* Update identity change log for 1.0.2

* Update identity pom version to 1.0.2

* Recreating connections on transient failures (#7090)

* Adding EventHubConnectionProcessor.

* Add tests for fetching new connection.

* Add tests to ensure on error conditions.

* Remove circuar dependency on EventHubConnection from EventHubConnectionProcessor

* Update AmqpConnection interface to create send and receive links.

* Fixing errors in ConnectionProcessor and update ReactorAmqpConnection to keep track of send links.

* Removing EventHubConnection.

* Update clients to use ConnectionProcessor.

* Fixing EHubClientBuilder to repeat the mono, so we have a potentially infinite stream of connections.

* Add test to verify that on non-transient errors, the connection is not revived.

* Delay retrying EventHubConnection creation on a transient error.

* Consolidate request from upstream for new connection.

* Not requesting new connection unless needed.

* No, really fix the script this time (#7239)

* README Fixes : Fixing links to sdk libraries  (#7236)

* Fixing links to sdk libraries, moving them to githubio release page

* Update identity to 1.0.2 everywhere (#7244)

* KV changelog updates (#7176)

KV changelog updates.

* Fixes NullReferenceException when using long running producer (#7166)

* Fixing potential NPE when token managers are created multiple times.

* Fix test assertions.

* Add test for provider.

Co-authored-by: Alan Zimmer <[email protected]>
Co-authored-by: Srikanta <[email protected]>
Co-authored-by: Mitch Denny <[email protected]>
Co-authored-by: David Noble <[email protected]>
Co-authored-by: Sima Zhu <[email protected]>
Co-authored-by: Kushagra Thapar <[email protected]>
Co-authored-by: Vinay Gera <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>
Co-authored-by: Jonathan Giles <[email protected]>
Co-authored-by: Chuang <[email protected]>
Co-authored-by: James Suplizio <[email protected]>
Co-authored-by: Hemant Tanwar <[email protected]>
Co-authored-by: Shawn Fang <[email protected]>
Co-authored-by: Vijaya Gopal Yarramneni <[email protected]>
Co-authored-by: Milis <[email protected]>
Co-authored-by: Sameeksha Vaity <[email protected]>
Co-authored-by: Jianghao Lu <[email protected]>
Co-authored-by: Mike Harder <[email protected]>
Co-authored-by: Connie Yau <[email protected]>
Co-authored-by: Chidozie Ononiwu <[email protected]>
Co-authored-by: Rick Ley <[email protected]>
Co-authored-by: Matt Ellis <[email protected]>
Co-authored-by: Tanyi Chen <[email protected]>
Co-authored-by: Josh Free <[email protected]>
  • Loading branch information
1 parent b03044f commit 303bb3a
Show file tree
Hide file tree
Showing 481 changed files with 39,479 additions and 3,396 deletions.
41 changes: 21 additions & 20 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -6,33 +6,34 @@
###########

# Catch all
/sdk/ @joshfree @jonathangiles
/sdk/ @joshfree @jonathangiles

# Service teams
/sdk/appconfiguration/ @mssfang @alzimmermsft @conniey @sima-zhu
/sdk/batch/ @xingwu1 @bgklein @matthchr
/sdk/core/ @alzimmermsft @jianghaolu @srnagar @hemanttanwar
/sdk/cosmos/ @moderakh @christopheranderson @kushagraThapar
/sdk/eventhubs/ @conniey @srnagar @mssfang
/sdk/identity/ @jianghaolu @g2vinay @hemanttanwar
/sdk/keyvault/ @g2vinay @samvaity
/sdk/servicebus/ @yvgopal @nemakam
/sdk/tracing/ @samvaity @alzimmermsft
/sdk/appconfiguration/ @mssfang @alzimmermsft @conniey @sima-zhu
/sdk/batch/ @xingwu1 @bgklein @matthchr
/sdk/core/ @alzimmermsft @jianghaolu @srnagar @hemanttanwar
/sdk/cosmos/ @moderakh @christopheranderson @kushagraThapar
/sdk/eventhubs/ @conniey @srnagar @mssfang
/sdk/identity/ @jianghaolu @g2vinay @hemanttanwar
/sdk/keyvault/ @g2vinay @samvaity
/sdk/servicebus/ @yvgopal @nemakam
/sdk/textanalytics/ @samvaity @mssfang
/sdk/tracing/ @samvaity @alzimmermsft

/sdk/storage/ @rickle-msft @jaschrep-msft @gapra-msft @alzimmermsft @sima-zhu
/sdk/storage/ @rickle-msft @jaschrep-msft @gapra-msft @alzimmermsft @sima-zhu

# Management Plane
/**/resource-manager/ @yaohaizh
/sdk/**/azure-mgmt-*/ @yaohaizh
/**/resource-manager/ @yaohaizh
/sdk/**/azure-mgmt-*/ @yaohaizh

###########
# Eng Sys
###########
/eng/ @JimSuplizio @mitchdenny @weshaggard @danieljurek
/eng/code-quality-reports/ @mssfang @JonathanGiles
/eng/jacoco-test-coverage/ @srnagar @JonathanGiles
/eng/spotbugs-aggregate-report/ @srnagar @JonathanGiles
/parent/ @JimSuplizio
/eng/ @JimSuplizio @mitchdenny @weshaggard @danieljurek
/eng/code-quality-reports/ @mssfang @JonathanGiles
/eng/jacoco-test-coverage/ @srnagar @JonathanGiles
/eng/spotbugs-aggregate-report/ @srnagar @JonathanGiles
/parent/ @JimSuplizio

/**/tests.yml @danieljurek
/**/ci.yml @mitchdenny
/**/tests.yml @danieljurek
/**/ci.yml @mitchdenny
69 changes: 22 additions & 47 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,31 +87,31 @@ The dependency-version should be set to the most recent released version and the

`com.azure:azure-identity;1.0.0-preview.4;1.0.0-preview.5`

Note: In the case of a new or unreleased artifact both versions will be the same.
Note: In the case of a new artifact both versions will be the same. In the case of a released artifact, the dependecny version should be the latest released version.

### Libraries vs External Dependencies

Libraries refer to things that are built and released as part of the Azure SDK. Libraries have a current version and a dependency version.

External Dependencies refer to dependencies for things that are not built and released as part of the Azure SDK regardless of the source. External Dependencies will only ever have one version.
External Dependencies refer to dependencies for things that are not built and released as part of the Azure SDK regardless of the source. External Dependencies will only ever have a dependency version.

### Current version vs Dependency version
### Current version, Dependency version and Unreleased Dependency version

Current version - This is the version we should be using when defining a component in its POM file and also when dependent components are built within the same pipeline. The current version is the version currently in development.
Dependency version - This is the version we should be using when a given library is a dependency outside of a particular area. This should be the latest released version of the package.
Dependency version - This is the version we should be using when a given library is a dependency outside of a particular area. This should be the latest released version of the package whenever possible.
Unreleased Dependency version – Whenever possible, libraries should be using the latest released version for dependencies but there is the case where active development in one library is going to be needed by another library or libraries that are built in separate pipelines. These types of changes are specifically additive and not breaking. Once a library has GA’d, nothing short of breaking changes should ever force the dependency versions across the repo to an unreleased version. The reason for this is that it would prevent other libraries, that don’t need this change, from releasing.

For example: `com.azure:azure-storage-blob-batch` has dependencies on `com.azure:azure-core`, `com.azure:azure-core-http-netty` and `com.azure:azure-storage-blob`. Because `com.azure:azure-core` and `com.azure:azure-core-http-netty` are both built outside of azure-storage pipeline we should be using the released or *Dependency* versions of these when they're dependencies of another library. Similarly, libraries built as part of the same pipeline, that have interdependencies, should be using the Current version. Since `com.azure:azure-storage-blob-batch` and `com.azure:azure-storage-blob` are both built part of the azure-batch pipeline when `com.azure:azure-storage-blob` is declared as a dependency of `com.azure:azure-storage-blob-batch` it should be the *Current* version.
An example of Current vs Dependency versions: `com.azure:azure-storage-blob-batch` has dependencies on `com.azure:azure-core`, `com.azure:azure-core-http-netty` and `com.azure:azure-storage-blob`. Because `com.azure:azure-core` and `com.azure:azure-core-http-netty` are both built outside of azure-storage pipeline we should be using the released or *Dependency* versions of these when they're dependencies of another library. Similarly, libraries built as part of the same pipeline, that have interdependencies, should be using the Current version. Since `com.azure:azure-storage-blob-batch` and `com.azure:azure-storage-blob` are both built part of the azure-batch pipeline when `com.azure:azure-storage-blob` is declared as a dependency of `com.azure:azure-storage-blob-batch` it should be the *Current* version.

This is going to be especially important after GA when releases aren't going to be the entire Azure SDK every time. If we're releasing a patch for a targeted azure-storage fix then we shouldn't need to build and release azure-core, we should be targeting the released versions and only building/releasing that update to azure-storage. It's worth noting that right now, in the version_client.txt, the dependency/current versions are the same. This will change once we GA, at which point the current version should be ahead of the dependency version.

What about README files? Right now the README files in the repo end up getting into an odd state since things like samples and versions can get updated during the development process. We're in the process of versioning documentation with the releases which means that the docs are snapshot at the time of the release and then versioned and stored. This will allow the README files in the repo to have updated samples and versions that are setup for the next release.
An example of an Unreleased Dependency version: Additive, not breaking, API changes have been made to `com.azure:azure-core`. `com.azure:azure-storage-blob` has a dependency on `com.azure:azure-core` and requires the additive API change that has not yet been released. An unreleased entry needs to be created in [version_client.txt](./eng/versioning/version_client.txt), under the unreleased section, with the following format: `unreleased_<groupId>:<artifactId>;dependency-version`, in this example that would be `unreleased_com.azure:azure-core;1.2.0` (this should match the 'current' version of core). The dependency update tags in the pom files that required this dependency would now reference `{x-version-update;unreleased_com.azure:azure-core;dependency}`. Once the updated library has been released the unreleased dependency version should be removed and the POM file update tags should be referencing the released version.

### Tooling, version files and marker tags

All of the tooling lives under the **eng\versioning** directory.

- version_client.txt - Contains the Client library and versions
- version_data.txt - Contains Data library and versions
- external_dependencies.txt - Contains the external dependency versions
- update_versions.py - This is just a basic python script that will climb through the source tree and update POM and README files. The script utilizes tags within the files to do replacements and the tags are slightly different between the POM and README files.
- set_versions.py - This script should only be used by the build system when we start producing nightly ops builds.

Expand All @@ -135,56 +135,31 @@ In README files this ends up being slightly different. Because the version tag i
```
[//]: # ({x-version-update-end})

What if I've got something that, for whatever reason, shoudln't be updated? There's a tag for that.

`<!-- {x-version-exempt;<groupId>:<artifactId>;reason for excemption} -->`

In theory, absence of an x-version-update tag would do the same thing but the tooling is still being developed and eventually there will be checkin blockers if xml has a version element with no tag.

### What does the process look like?

Let's say we've GA'd and I need to tick up the version of azure-storage libraries how would I do it? Guidelines for incrementing versions after release can be found [here](https://github.com/Azure/azure-sdk/blob/master/docs/policies/releases.md#incrementing-after-release).

1. I'd open up eng\versioning\version_client.txt and update the current-versions of the libraries that are built and released as part of the azure storage pipeline. This list can be found in pom.service.xml under the sdk/storage directory. It's worth noting that any module entry starting with "../" are external module dependencies and not something that's released as part of the pipeline. Once we GA, these build dependencies for library components outside of a given area should go away and be replaced with downloading the appropriate dependency from Maven like we do for external dependencies.
1. I'd open up eng\versioning\version_client.txt and update the current-versions of the libraries that are built and released as part of the azure storage pipeline. This list can be found in pom.service.xml under the sdk/storage directory. It's worth noting that any module entry starting with "../" are external module dependencies and not something that's released as part of the pipeline. Dependencies for library components outside of a given area would be downloading the appropriate dependency from Maven like we do for external dependencies.
2. Execute the update_versions python script from the root of the enlistment
`python eng/versioning/update_versions.py --ut libary --bt client`
This will go through the entire source tree and update all of the references in the POM and README files with the updated versions. Git status will show all of the modified files.
3. Review and submit a PR with the modified files.

### Next steps: External dependencies, Management plane and service pipeline changes
### Next steps: Management plane

- External dependencies. Right now there are only version files for client and data (eng\versioning\version_\[client|data\].txt) which only encompass the built binaries for their respective tracks. External dependencies for both client and data are next on the list which should allow modification of the parent/pom.xml to remove the list of version properties and dependency management sections which brings things one step closer to not having to publish the parent pom.
- Management plane. Management is in the process of being moved to service pipeline builds. The versioning work needs to wait until that work is finished.
- Service pipeline changes. The service pipelines currently have to build not only the libraries that are part of that pipeline but also the Azure SDK libraries that are dependencies. Once we GA and can start targeting the released version of those packages and pulling them from Maven instead of building them. An good example of this would be in sdk/appconfiguration/pom.service.xml where to build azure-data-appconfiguration we end up building azure-core, azure-core-test and azure-core-http-netty along with azure-data-appconfiguration instead of just building azure-data-appconfiguration.

### How are versioning and dependencies going to impact development after GA?

As mentioned above, in the service pipeline changes, the plan after we GA is to start targeting the released version of the packages and pulling them from Maven. This is going to fundamentally change some aspects of the development process especially when work needs to be done on given library that requires dependency changes in one or more libraries.

- **Scenario 1: Making changes to a single library which is not a dependency of any other libraries:** This ends up being the most straightforward scenario and really isn't much different than it is today.
- [ ] Appropriately increase the version
- [ ] Make the code changes
- [ ] Submit the PR
- [ ] Merge the PR
- [ ] Publish the new version

- **Scenario 2: Making changes to a library that also requires dependency changes:** Right now things are in a state where dependency changes can be made along with libraries that depend on them because of the project dependencies in the service pom files. Local development isn't going to change that much except when changing the version of a library and its dependency or dependencies means that the service poms are going to have to be built, and installed, in the appropriate order. This is because these new versions of the library dependencies won't yet be released and Maven will need to find these in the local cache. The biggest change to the process is going to be around PRs and publishing. Separate PRs are going to have to be submitted in order, with dependencies being submitted first. This is necessary because the dependencies need to be published in order to allow things that depend on them to continue using the published version. Trying to submit everything in one PR would cause build breaks since the dependency being referenced is a version not yet published. An example of this would be making changes to `com.azure:azure-storage-common` that also required dependency changes to `com.azure:azure-identity`.
Changes are going to have to be made to `com.azure:azure-identity` first.
- [ ] Appropriately increase the version of `com.azure:azure-identity`
- [ ] Make the code changes
- [ ] Build and optionally install locally
This isn't completely necessary other than to install the updated version of the dependency into the local cache on the machine. The alternative to this would be to publish (DevOps or otherwise) and reference that version of the dependency after the release. Either one would allow `com.azure:azure-storage-common` to use the updated version of `com.azure:azure-identity`
- [ ] Submit the PR for the `com.azure:azure-identity`
- [ ] Merge the PR for the `com.azure:azure-identity`
- [ ] Publish the `com.azure:azure-identity` with the updated version.

Only after the dependency `com.azure:azure-identity` has been published can the PR for `com.azure:azure-storage-common` be created.
- [ ] Appropriately increase the version of `com.azure:azure-storage-common` and the dependency version of `com.azure:azure-identity` in its pom file.
- [ ] Make the code changes, if any
- [ ] Build/Test or whatever
- [ ] Submit the PR for `com.azure:azure-storage-common`
- [ ] Merge the PR for `com.azure:azure-storage-common`
- [ ] Publish the PR for `com.azure:azure-storage-common`

### Making changes to an already GA'd library that require other libraries to depend on the unreleased version

This is where the `unreleased_` dependency tags come into play. Using the Unreleased Dependency example above, where `com.azure:azure-storage-blob` has a dependency on an unreleased `com.azure:azure-core`:

- [ ] Make the additive changes to `com.azure:azure-core`
- [ ] In [version_client.txt](./eng/versioning/version_client.txt) add the entry for the unreleased azure core in the unreleased section at the bottom of the file. The entry would look like `unreleased_com.azure:azure-core;<version>`.
Note: The version of the library referenced in the unreleased version tag should match the current version of that library.
- [ ] In the pom.xml file for `com.azure:azure-storage-blob`, the dependency tag for `com.azure:azure-core` which was originally `{x-version-update;com.azure:azure-core-test;dependency}` would now become `{x-version-update;unreleased_com.azure:azure-core-test;dependency}`
After the unreleased version of `com.azure:azure-core` was released but before `com.azure:azure-storage-blob` has been released.
- [ ] In [version_client.txt](./eng/versioning/version_client.txt) the the dependency version of `com.azure:azure-core` would become the released version and the "unreleased_" entry, at this time, would be removed.
- [ ] In the pom.xml file for `com.azure:azure-storage-blob`, the dependency tag for `com.azure:azure-core` would get changed back to `{x-version-update;com.azure:azure-core-test;dependency}`

### Nightly package builds

Expand Down
Loading

0 comments on commit 303bb3a

Please sign in to comment.