Add Worker Versioning doc (#1533)

assembly/guide-configs/app-dev/golang/versioning.json

@@ -14,6 +14,10 @@
         {
             "type": "p",
             "id": "go/versioning"
+        },
+        {
+            "type": "h2",
+            "id": "go/how-to-use-worker-versioning-in-go"
         }
     ]
-}
+}

assembly/guide-configs/app-dev/java/versioning.json

@@ -14,6 +14,10 @@
         {
             "type": "p",
             "id": "java/versioning"
+        },
+        {
+            "type": "h2",
+            "id": "java/how-to-use-worker-versioning-in-java"
         }
     ]
 }

assembly/guide-configs/concepts/workers.json

@@ -62,6 +62,10 @@
     {
       "type": "h3",
       "id": "concepts/what-is-a-worker-session"
+    },
+    {
+      "type": "h2",
+      "id": "concepts/what-is-worker-versioning"
     }
   ]
 }

docs-src/concepts/what-is-a-workflow-definition.md

@@ -44,7 +44,7 @@ The following are the two reasons why a Command might be generated out of sequen
 1. Code changes are made to a Workflow Definition that is in use by a running Workflow Execution.
 2. There is intrinsic non-deterministic logic (such as inline random branching).
 
-### Code changes can cause non-deterministic behavior
+### Code changes can cause non-deterministic behavior {#non-deterministic-change}
 
 The Workflow Definition can change in very limited ways once there is a Workflow Execution depending on it.
 To alleviate non-deterministic issues that arise from code changes, we recommend using [Workflow Versioning](#workflow-versioning).
@@ -102,7 +102,7 @@ When those APIs are used, the results are stored as part of the Event History, w
 
 In other words, all operations that do not purely mutate the Workflow Execution's state should occur through a Temporal SDK API.
 
-### Workflow Versioning
+### Workflow Versioning {#workflow-versioning}
 
 The Workflow Versioning feature enables the creation of logical branching inside a Workflow Definition based on a developer specified version identifier.
 This feature is useful for Workflow Definition logic needs to be updated, but there are running Workflow Executions that currently depends on it.

docs-src/concepts/what-is-worker-versioning.md

@@ -0,0 +1,223 @@
+---
+id: what-is-worker-versioning
+title: What is Worker Versioning?
+sidebar_label: Worker Versioning
+description: Worker Versioning lets you more easily deploy changes to Workflow Definitions.
+tags:
+  - explanation
+  - versioning
+---
+
+Worker Versioning simplifies the process of deploying changes to [Workflow Definitions](/workflows/#workflow-definition).
+It does this by letting you define sets of versions that are compatible with each other, and then assigning a Build ID to the code that defines a Worker.
+The Temporal Server uses the Build ID to determine which versions of a Workflow Definition a Worker can process.
+
+We recommend that you read about Workflow Definitions before proceeding, because Workflow Versioning is largely concerned with helping to manage nondeterministic changes to those definitions.
+
+Worker Versioning helps manage nondeterministic changes by providing a convenient way to ensure that [Workers](#workers) with different Workflow and Activity Definitions operating on the same [Task Queue](#task-queue) don't attempt to process [Workflow Tasks](#workflow-task) and [Activity Tasks](#activity-task-execution) that they can't successfully process, according to sets of versions associated with that Task Queue that you've defined.
+
+Accomplish this goal by assigning a Build ID (a free-form string) to the code that defines a Worker, and specifying which Build IDs are compatible with each other by updating the version sets associated with the Task Queue, stored by the Temporal Server.
+
+### When and why you should use Worker Versioning
+
+The main reason to use this feature is to deploy incompatible changes to short-lived [Workflows](/workflows).
+On Task Queues using this feature, the Workflow starter doesn't have to know about the introduction of new versions.
+
+The new code in the newly deployed Workers executes new [Workflow Executions](#workflow-execution), while only Workers with an appropriate version process old Workflow Executions.
+
+#### Decommision old Workers
+
+You can decommission old Workers after you archive all open Workflows using their version.
+If you have no need to query closed Workflows, you can decommission them when no open Workflows remain at that version.
+
+For example, if you have a Workflow that completes within a day, a good strategy is to assign a new Build ID to every new Worker build and add it as the new overall default in the version sets.
+
+Because your Workflow completes in a day, you know that you won't need to keep older Workers running for more than a day after you deploy the new version (assuming availability).
+
+Removing old versions from the sets isn't necessary.
+Leaving them doesn't cause any harm.
+
+You can apply this technique to longer-lived Workflows too; however, you might need to run multiple Worker versions simultaneously while open Workflows complete.
+
+#### Deploy code changes to Workers
+
+The feature also lets you implement compatible changes to or prevent a buggy code path from executing on currently open Workflows.
+You can achieve this by adding a new version to an existing set and defining it as _compatible_ with an existing version, which shouldn't execute any future Workflow Tasks.
+Because the new version processes existing [Event Histories](/workflows/#event-history), it must adhere to the usual [deterministic constraints](/workflows/#deterministic-constraints), and you might need to use one of the [versioning APIs](/workflows/#workflow-versioning).
+
+Moreover, this feature let you make incompatible changes to Activity Definitions in conjunction with incompatible changes to Workflow Definitions that use those Activities.
+This functionality works because any Activity that a Workflow schedules on the same Task Queue gets dispatched only to Workers compatible with the Workflow that scheduled it.
+If you want to change an Activity Definition's type signature while creating a new incompatible Build ID for a Worker, you can do so without worrying about the Activity failing to execute on some other Worker with an incompatible definition.
+The same principle applies to Child Workflows.
+
+:::tip
+
+Public-facing Workflows on a versioned Task Queue shouldn't change their signatures because doing so contradicts the purpose of Workflow-launching Clients remaining unaware of changes in the Workflow Definition.
+If you need to change a Workflow's signature, use a different Workflow Type or a completely new Task Queue.
+
+:::
+
+:::note
+
+If you schedule an Activity or a Child Workflow on _a different_ Task Queue from the one the Workflow runs on, the system doesn't assign a specific version.
+This means if the target queue is versioned, they run on the latest default, and if it's unversioned, they operate as they would have without this feature.
+
+:::
+
+**Continue-As-New and Worker Versioning**
+
+By default, a versioned Task Queue's Continue-as-New function starts the continued Workflow on the same compatible set as the original Workflow.
+
+If you continue-as-new onto a different Task Queue, the system doesn't assign any particular version.
+You also have the option to specify that the continued Workflow should start using the Task Queue's latest default version.
+
+### How to use Worker Versioning
+
+To use Worker Versioning, follow these steps:
+
+1. Define Worker build-identifier version sets for the Task Queue.
+   You can use either the `temporal` CLI or your choice of SDK.
+2. Enable the feature on your Worker by specifying a Build ID.
+
+#### Defining the version sets
+
+Whether you use [Temporal CLI](/cli/) or an SDK, updating the version sets feels the same.
+You specify the Task Queue that you're targeting, the Build ID that you're adding (or promoting), whether it becomes the new default version, and any existing versions it should be considered compatible with.
+
+The rest of this section uses updates to one Task Queue's version sets as examples.
+
+By default, both Task Queues and Workers are in an unversioned state.
+[Unversioned Worker](#unversioned-workers) can poll unversioned Task Queues and receive tasks.
+To use this feature, both the Task Queue and the Worker must be associated with Build IDs.
+
+If you run a Worker using versioning against a Task Queue that has not been set up to use versioning (or is missing that Worker's Build ID), it won't get any tasks.
+Likewise, a unversioned Worker polling a Task Queue with versioning won't work either.
+
+:::note Versions don't need to follow semver or any other semantic versioning scheme!
+
+The versions in the following examples look like semver versions for clarity, but they don't need to be.
+Versions can be any arbitrary string.
+
+:::
+
+First, add a version `1.0` to the Task Queue as the new default.
+Your version sets now look like this:
+
+| set 1 (default) |
+| --------------- |
+| 1.0 (default)   |
+
+All new Workflows started on the Task Queue have their first tasks assigned to version `1.0`.
+Workers with their Build ID set to `1.0` receive these Tasks.
+
+If Workflows that don't have an assigned version are still running on the Task Queue, Workers without a version take those tasks.
+So ensure that such Workers are still operational if any Workflows were open when you added the first version.
+If you deployed any Workers with a _different_ version, those Workers receive no Tasks.
+
+Now, imagine you need change the Workflow for some reason.
+
+Add `2.0` to the sets as the new default:
+
+| set 1         | set 2 (default) |
+| ------------- | --------------- |
+| 1.0 (default) | 2.0 (default)   |
+
+All new Workflows started on the Task Queue have their first tasks assigned to version `2.0`.
+Existing `1.0` Workflows keep generating tasks targeting `1.0`.
+Each deployment of Workers receives their respective Tasks.
+This same concept carries forward for each new incompatible version.
+
+Maybe you have a bug in `2.0`, and you want to make sure all open `2.0` Workflows switch to some new code as fast as possible.
+So, you add `2.1` to the sets, marking it as compatible with `2.0`.
+Now your sets look like this:
+
+| set 1         | set 2 (default) |
+| ------------- | --------------- |
+| 1.0 (default) | 2.0             |
+|               | 2.1 (default)   |
+
+All new Workflow Tasks that are generated for Workflows whose last Workflow Task completion was on version `2.0` are now assigned to version `2.1`.
+Because you specified that `2.1` is compatible with `2.0`, Temporal Server assumes that Workers with this version can process the existing Event Histories successfully.
+
+Continue with your normal development cycle, adding a `3.0` version.
+Nothing new here:
+
+| set 1         | set 2         | set 3 (default) |
+| ------------- | ------------- | --------------- |
+| 1.0 (default) | 2.0           | 3.0 (default)   |
+|               | 2.1 (default) |                 |
+
+Now imagine that version `3.0` doesn't have an explicit bug, but something about the business logic
+is less than ideal.
+You are okay with existing `3.0` Workflows running to completion, but you want new Workflows to use the old `2.x` branch.
+This operation is supported by performing an update targeting `2.1` (or `2.0`) and setting its set as the current default, which results in these sets:
+
+| set 1         | set 3         | set 2 (default) |
+| ------------- | ------------- | --------------- |
+| 1.0 (default) | 3.0 (default) | 2.0             |
+|               |               | 2.1 (default)   |
+
+Now new Workflows start on `2.1`.
+
+#### Permitted and forbidden operations on version sets
+
+A request to change the sets can do one of the following:
+
+- Add a version to the sets as the new default version in a new overall-default compatible set.
+- Add a version to an existing set that's compatible with an existing version.
+  - Optionally making it the default for that set.
+  - Optionally making that set the overall-default set.
+- Promote a version within an existing set to become the default for that set.
+- Promote a set to become the overall-default set.
+
+You can't explicitly delete versions.This helps you avoid the situation in which Workflows accidentally become stuck with no means of making progress because the version they're associated with no longer exists.
+
+However, sometimes you might want to do this intentionally.
+If you _want_ to make sure that all Workflows currently being processed by, say, `2.0` stop (even if you don't yet have a new version ready), you can add a new version `2.1` to the sets marked as compatible with `2.0`.
+New tasks will target `2.1`, but because you haven't deployed any `2.1` Workers, they won't make any progress.
+
+#### Set constraints
+
+The sets have a maximum size limit, which defaults to 100 build IDs across all sets.
+This limit is configurable on Temporal Server via the `limit.versionBuildIdLimitPerQueue` dynamic config property.
+Operations to add new Build IDs to the sets fail if the limit would be exceeded.
+
+There is also a limit on the number of sets, which defaults to 10.
+This limit is configurable via the `limit.versionCompatibleSetLimitPerQueue` dynamic config property.
+
+In practice, these limits should rarely be a concern because a version is no longer needed after no open Workflows are using that version, and a background process will delete IDs and sets that are no longer needed.
+
+There is also a limit on the size of each Build ID or version string, which defaults to 255 characters.
+This limit is configurable on the server via the `limit.workerBuildIdSize` dynamic config property.
+
+### Build ID reachability
+
+Eventually, you'll want to know whether you can retire the old Worker versions.
+Temporal provides functionality to help you determine whether a version is still in use by open or closed Workflows.
+You can use the Temporal CLI to do this with the following command:
+
+```command
+temporal task-queue get-build-id-reachability
+```
+
+The command determines, for each Task Queue, whether the Build ID in question is unreachable, only reachable by closed Workflows, or reachable by open and new Workfloww.
+For example, this "2.0" Build ID is shown here by the CLI to be reachable by both new Workflows and some existing Workflows:
+
+```command
+temporal task-queue get-build-id-reachability --build-id "2.0"
+```
+
+```output
+BuildId                         TaskQueue                                   Reachability
+    2.0  build-id-versioning-dc0068f6-0426-428f-b0b2-703a7e409a97  [NewWorkflows
+                                                                   ExistingWorkflows]
+```
+
+For more information, see the [CLI documentation](/cli/) or help output.
+
+You can also use this API `GetWorkerTaskReachability` directly from within language SDKs.
+
+### Unversioned Workers
+
+Unversioned Workers refer to Workers that have not opted into the Worker Versioning feature in their configuration.
+They receive tasks only from Task Queues that do not have any version sets defined on them, or that have open workflows that began executing before versions were added to the queue.