diff --git a/docs/config.json b/docs/config.json index c30f72cee000f..16b6e809e2dea 100644 --- a/docs/config.json +++ b/docs/config.json @@ -29,13 +29,17 @@ "slug": "/upgrading/overview/" }, { - "title": "Set up Automatic Agent Updates", - "slug": "/upgrading/automatic-agent-updates/" + "title": "Cluster Upgrading Reference", + "slug": "/upgrading/upgrading-reference/" }, { - "title": "Upgrading Reference", - "slug": "/upgrading/upgrading-reference/" - }, + "title": "Managed Updates for Agent (v1)", + "slug": "/upgrading/agent-managed-updates-v1/" + }, + { + "title": "Managed Updates for Agents (v2)", + "slug": "/upgrading/agent-managed-updates/" + }, { "title": "Client Tool Automatic Updates", "slug": "/upgrading/client-tools-autoupdate/" diff --git a/docs/pages/installation.mdx b/docs/pages/installation.mdx index 502c8e01aac74..92a1eaac20ddf 100644 --- a/docs/pages/installation.mdx +++ b/docs/pages/installation.mdx @@ -9,6 +9,7 @@ tocDepth: 3 This guide shows you how to install Teleport binaries on your platform, including: - `teleport` +- `teleport-update` - `tsh` - `tctl` - `tbot` @@ -57,7 +58,7 @@ Teleport maintains DEB and RPM package repositories for different operating systems, platforms, and Teleport versions. A server that installs Teleport from a DEB or RPM package must have systemd installed. You can also download TAR archives containing Teleport binaries. All installations include `teleport`, -`tsh`, `tctl`, `fdpass-teleport` and `tbot`. +`teleport-update`, `tsh`, `tctl`, `fdpass-teleport` and `tbot`. ### Recommended installation steps @@ -319,17 +320,16 @@ repositories. -### Downloading packages and TAR archives (self-hosted only) +### Downloading packages and TAR archives Teleport maintains TAR archives as well as DEB and RPM packages for Linux-compatible binaries at `https://cdn.teleport.dev`. This section explains how to install Teleport by manually downloading a release. -It is not possible to install the automatic agent updater using this method, so -using packages and TAR archives is only available for users who self-hosted -Teleport. Teleport Cloud customers must use the [one-line installation -script](#one-line-installation-script) or manually install Teleport from a -[package repository](#package-repositories) in order to install the updater. +The original (v1) Teleport updater is not compatible with this method. +If you use cloud-hosted Teleport, you must use `teleport-update` to manage +your Teleport installation. We recommend using the [one-line installation +script](#one-line-installation-script). 1. In your terminal, assign environment variables that you will use to download your intended archive. @@ -375,7 +375,7 @@ script](#one-line-installation-script) or manually install Teleport from a $ shasum --check -a 256 ${TELEPORT_PKG?}-v${TELEPORT_VERSION?}-linux-${SYSTEM_ARCH?}-bin.tar.gz $ tar -xvf ${TELEPORT_PKG?}-v${TELEPORT_VERSION?}-linux-${SYSTEM_ARCH?}-bin.tar.gz $ cd ${TELEPORT_PKG?} - $ sudo ./install + $ sudo ./teleport-update enable --proxy example.teleport.sh # or sudo ./install for static installation ``` @@ -386,6 +386,7 @@ script](#one-line-installation-script) or manually install Teleport from a $ curl -O https://cdn.teleport.dev/${TELEPORT_PKG?}_${TELEPORT_VERSION?}_${SYSTEM_ARCH?}.deb $ shasum --check -a 256 ${TELEPORT_PKG?}_${TELEPORT_VERSION?}_${SYSTEM_ARCH?}.deb $ sudo dpkg -i ${TELEPORT_PKG?}_${TELEPORT_VERSION?}_${SYSTEM_ARCH?}.deb + $ sudo teleport-update enable --proxy example.teleport.sh # enable managed updates ``` @@ -398,6 +399,7 @@ script](#one-line-installation-script) or manually install Teleport from a $ shasum --check -a 256 https://cdn.teleport.dev/${TELEPORT_PKG?}-${TELEPORT_VERSION?}-1.${SYSTEM_ARCH?}.rpm # Or use yum localinstall, dnf localinstall etc. $ sudo rpm -i https://cdn.teleport.dev/${TELEPORT_PKG?}-${TELEPORT_VERSION?}-1.${SYSTEM_ARCH?}.rpm + $ sudo teleport-update enable --proxy example.teleport.sh # enable managed updates ``` @@ -416,7 +418,7 @@ script](#one-line-installation-script) or manually install Teleport from a $ shasum --check -a 256 teleport-ent-v${TELEPORT_VERSION?}-linux-${SYSTEM_ARCH?}-fips-bin.tar.gz $ tar -xvf teleport-ent-v${TELEPORT_VERSION?}-linux-${SYSTEM_ARCH?}-fips-bin.tar.gz $ cd teleport-ent - $ sudo ./install + $ sudo ./teleport-update enable --proxy example.teleport.sh # or sudo ./install for static installation ``` @@ -430,6 +432,7 @@ script](#one-line-installation-script) or manually install Teleport from a $ curl -O https://cdn.teleport.dev/teleport-ent_${TELEPORT_VERSION}-fips_${SYSTEM_ARCH}.deb $ shasum --check -a 256 teleport-ent_${TELEPORT_VERSION}-fips_${SYSTEM_ARCH}.deb $ sudo dpkg -i teleport-ent_${TELEPORT_VERSION}-fips_${SYSTEM_ARCH}.deb + $ sudo teleport-update enable --proxy example.teleport.sh # enable managed updates ``` @@ -445,6 +448,7 @@ script](#one-line-installation-script) or manually install Teleport from a $ shasum --check -a 256 https://cdn.teleport.dev/teleport-ent-${TELEPORT_VERSION?}-1-fips.${SYSTEM_ARCH?}.rpm # Or use yum localinstall, dnf localinstall etc. $ sudo rpm -i https://cdn.teleport.dev/teleport-ent-${TELEPORT_VERSION?}-1-fips.${SYSTEM_ARCH?}.rpm + $ sudo teleport-update enable --proxy example.teleport.sh # enable managed updates ``` diff --git a/docs/pages/reference/cli/teleport-update.mdx b/docs/pages/reference/cli/teleport-update.mdx new file mode 100644 index 0000000000000..f6a0db52c259c --- /dev/null +++ b/docs/pages/reference/cli/teleport-update.mdx @@ -0,0 +1,283 @@ +--- +title: teleport-update CLI reference +description: Comprehensive reference of subcommands, flags, and arguments for the teleport-update CLI tool. +--- + +`teleport-update` is a CLI tool that is used to update Teleport agents installed on Linux servers. + +See [Teleport Agent Managed Updates](../upgrading/agent-managed-updates.mdx) for more details. + +The primary commands for `teleport-update` are as follows: + +| Command | Description | +|----------------------------------|----------------------------------------------------------------------------------------| +| `teleport-update help` | Output guidance for using commands with `teleport-update`. | +| `teleport-update version` | Output the current version of the `teleport-update` binary. | +| `teleport-update enable` | Install the version of Teleport advertised by the cluster and enable managed updates. | +| `teleport-update disable` | Disable auto-updates without removing Teleport. Disables all requests. | +| `teleport-update pin` | Install the version of Teleport advertised by the cluster and stay on that version. | +| `teleport-update unpin` | Allow the installed version of Teleport to be updated, if managed updates are enabled. | | +| `teleport-update update` | Update Teleport to the version advertised by the cluster. | | +| `teleport-update link-package` | Restore the system-packaged (apt/yum) version of Teleport to /usr/local/bin. | | +| `teleport-update unlink-package` | Remove the system-packaged (apt/yum) version of Teleport from /usr/local/bin. | | +| `teleport-update status` | Output the status of the installed version of Teleport. | | +| `teleport-update uninstall` | Remove both Teleport and the updater. | | + +## teleport-update enable + +Enables agent managed updates and performs an initial installation of the Teleport agent. +This command also creates a systemd timer that periodically runs the update subcommand. + +If Teleport is already installed, `enable` will update to the cluster-advertised version +and ensure managed updates are enabled. + +Existing package-based installations will be converted to managed updates automatically. +Existing tarball-based, static installations may require `--overwrite`. +(A clear error will let you know if this is the case.) + +Files are installed to the following paths (with the default `install-suffix`): + +- `/usr/local/bin/{teleport,tsh,...}` - Symbolic links into `/opt/teleport/default/versions/X.Y.Z/bin/` +- `/lib/systemd/system/teleport.service` - Teleport SystemD service +- `/opt/teleport/default` - Storage for Teleport versions and updater configuration +- `/etc/systemd/system/teleport-update.{service,timer}` - Updater SystemD timer and service +- `/etc/systemd/system/teleport.service.d/teleport-update.conf` - Environment variables that configure Teleport +- `/etc/needrestart/conf.d/teleport-update.conf` - Needrestart defaults for Teleport + +Most flags passed to `enable` are persisted for `update`. +To change these flags, run `enable` again with the new flags. + +### Flags + +| Flag | Description | +|----------------------|------------------------------------------------------------------------------------------------------------------------| +| -d, --[no-]debug | Verbose logging to stdout. | +| --log-format | Controls the format of output logs. Can be `json` or `text`. Defaults to `text`. | +| -i, --install-suffix | Suffix for creating an agent installation outside of the default $PATH. Note: this changes the default data directory. | +| --[no-]insecure | Insecure mode disables certificate verification. Do not use in production. | +| -p, --proxy | Address of the Teleport Proxy. | +| -g, --group | Update group for this agent installation. | +| -b, --base-url | Base URL used to override the Teleport download URL. | +| -o, --[no-]overwrite | Allow existing installed Teleport binaries to be overwritten. | + +### Examples + +**Example for a new installation.** + +Install Teleport with managed updates enabled on a fresh system. + +```code +# create /etc/teleport.yaml +$ sudo teleport-update enable +$ sudo systemctl enable teleport --now +``` + +**Example for an existing installation.** + +Install Teleport with managed updates enabled on a system with a running Teleport version. +Package-based installations will be converted to managed updates automatically. + +```code +$ teleport-update enable +``` + +**Example with a custom installation suffix.** + +Install Teleport with managed updates enabled to an isolated path. +This will not interfere with package-based installations. +Multiple installations with separate suffixes may operate individually and independently against separate clusters. + +```code +# create /etc/teleport_mycluster.yaml +$ sudo teleport-update --install-suffix mycluster enable +$ sudo systemctl enable teleport_mycluster --now +$ export PATH=/opt/teleport/mycluster/bin:$PATH +``` + +## teleport-update disable + +Disable managed updates for the installed agent. +This command does not remove or change the active installation of Teleport. + +Unlike `pin`, this command will not touch the current installation, and version lookup requests will stop entirely. + +### Flags + +| Flag | Description | +|----------------------|------------------------------------------------------------------------------------------------------------------------| +| -d, --[no-]debug | Verbose logging to stdout. | +| --log-format | Controls the format of output logs. Can be `json` or `text`. Defaults to `text`. | +| -i, --install-suffix | Suffix for creating an agent installation outside of the default $PATH. Note: this changes the default data directory. | +| --[no-]insecure | Insecure mode disables certificate verification. Do not use in production. | + +### Examples + +**Example for disabling managed updates.** + +Disable managed updates for the current agent. + +```code +$ sudo teleport-update disable +``` + +## teleport-update pin + +Pin the installed agent to a specific version of Teleport. +This command updates Teleport to latest version (or a version specified with `--force-version`), and ensures the local +installation of Teleport remains at that version. +New versions will continue to be reported in SystemD `teleport-update.service` logs, they but will not be installed. + +`pin` is similar to `enable`, but the resulting version will not change until `unpin` is run. + +### Flags + +| Flag | Description | +|----------------------|------------------------------------------------------------------------------------------------------------------------| +| -d, --[no-]debug | Verbose logging to stdout. | +| --log-format | Controls the format of output logs. Can be `json` or `text`. Defaults to `text`. | +| -i, --install-suffix | Suffix for creating an agent installation outside of the default $PATH. Note: this changes the default data directory. | +| --[no-]insecure | Insecure mode disables certificate verification. Do not use in production. | +| -p, --proxy | Address of the Teleport Proxy. | +| -g, --group | Update group for this agent installation. | +| -b, --base-url | Base URL used to override the Teleport download URL. | +| -o, --[no-]overwrite | Allow existing installed Teleport binaries to be overwritten. | +| -f, --force-version | Force the provided version instead of using the version provided by the Teleport cluster. | | + +### Examples + +**Example for a new installation at the cluster-advertised version.** + +Install the cluster-advertised Teleport version on a fresh system, but lock the version in-place. + +```code +# create /etc/teleport.yaml +$ sudo teleport-update pin +$ sudo systemctl enable teleport --now +``` + +**Example for a new installation at a custom version.** + +Install the specified Teleport version on a fresh system, but lock the version in-place. + +```code +# create /etc/teleport.yaml +$ sudo teleport-update pin --force-version v17.1.2 +$ sudo systemctl enable teleport --now +``` + +**Example for an existing installation.** + +Install the cluster-advertised Teleport version on a system with a running Teleport agent, but lock the version +in-place. +Package-based installations will be converted to managed updates automatically. + +```code +$ sudo teleport-update pin +``` + +## teleport-update unpin + +Unpin the version of Teleport, so that the cluster-advertised version will be installed on the next update. +This command does not immediately remove or change the active installation of Teleport. + +### Flags + +| Flag | Description | +|----------------------|------------------------------------------------------------------------------------------------------------------------| +| -d, --[no-]debug | Verbose logging to stdout. | +| --log-format | Controls the format of output logs. Can be `json` or `text`. Defaults to `text`. | +| -i, --install-suffix | Suffix for creating an agent installation outside of the default $PATH. Note: this changes the default data directory. | +| --[no-]insecure | Insecure mode disables certificate verification. Do not use in production. | + +### Examples + +**Example for unpinning managed updates to allow the cluster-advertised version to be installed.** + +Unpin managed updates for the current agent. + +```code +$ sudo teleport-update unpin +$ sudo teleport-update update --now # force immediate update, if desired +``` + +## teleport-update update + +Update the version of Teleport to either the pinned version, or the version advertised by the cluster. + +If `--now` is not specified, `update` will only update if the current time is in the update window for the agent's group. + +This command is used by the `teleport-update` SystemD service/timer, and does not need to be manually executed. + +### Flags + +| Flag | Description | +|----------------------|------------------------------------------------------------------------------------------------------------------------| +| -d, --[no-]debug | Verbose logging to stdout. | +| --log-format | Controls the format of output logs. Can be `json` or `text`. Defaults to `text`. | +| -i, --install-suffix | Suffix for creating an agent installation outside of the default $PATH. Note: this changes the default data directory. | +| --[no-]insecure | Insecure mode disables certificate verification. Do not use in production. | +| -n, --[no-]now | Force immediate update even if update window is not active. | + +### Examples + +**Example for updating immediately.** + +Force update outside of update window with `--now`. + +```code +$ sudo teleport-update update --now +``` + +## teleport-update link-package + +Link the system installation of Teleport from the Teleport package, if managed updates is disabled. + +This command is used to link the system package installation by: +- Creating symbolic links from `/opt/teleport/system/bin/*` into `/usr/local/bin/`. +- Copying the Teleport systemd service file from `/opt/teleport/system/lib/systemd/system/teleport.service` into `/ib/systemd/system/teleport.service`. + +This command is executed automatically when the Teleport package is installed, and does not need to be manually executed. + +Managed updates must be disabled, and the active version of Teleport must be removed, for this command to work. +Note that `uninstall` will automatically remove the active version of Teleport and link the system package if the package is installed. + +### Flags + +| Flag | Description | +|----------------------|------------------------------------------------------------------------------------------------------------------------| +| -d, --[no-]debug | Verbose logging to stdout. | +| --log-format | Controls the format of output logs. Can be `json` or `text`. Defaults to `text`. | +| -i, --install-suffix | Suffix for creating an agent installation outside of the default $PATH. Note: this changes the default data directory. | +| --[no-]insecure | Insecure mode disables certificate verification. Do not use in production. | + +### Examples + +**Example for restoring the system package installation of Teleport.** + +Re-link Teleport apt/yum package. + +```code +$ sudo teleport-update link-package +``` + +## teleport-update unlink-package + +Unlink the system installation of Teleport from the Teleport package. + +This command is used to unlink the system package installation by: +- Removing symbolic links from `/opt/teleport/system/bin/*` into `/usr/local/bin/`. +- Removing the Teleport systemd service file from `/ib/systemd/system/teleport.service` (if symlinks are removed). + +This command is executed automatically when the Teleport package is removed, and should not be manually executed. + +**Note that this command will unlink a running version of Teleport, potentially causing loss of access to the system via Teleport.** + +### Flags + +| Flag | Description | +|----------------------|------------------------------------------------------------------------------------------------------------------------| +| -d, --[no-]debug | Verbose logging to stdout. | +| --log-format | Controls the format of output logs. Can be `json` or `text`. Defaults to `text`. | +| -i, --install-suffix | Suffix for creating an agent installation outside of the default $PATH. Note: this changes the default data directory. | +| --[no-]insecure | Insecure mode disables certificate verification. Do not use in production. | + diff --git a/docs/pages/reference/resources.mdx b/docs/pages/reference/resources.mdx index 8dc55641b2202..921073708cb79 100644 --- a/docs/pages/reference/resources.mdx +++ b/docs/pages/reference/resources.mdx @@ -212,7 +212,14 @@ spec: ## Cluster maintenance config -Global configuration options for the agents enrolled into automatic updates. +Global configuration options for the agents enrolled into automatic updates (v1). + + +`cluster_maintenance_config` configures the original (v1) Teleport Agent updater, +which is currently supported but will be deprecated after the new updater is released. +The `autoupdate_config` and `autoupdate_version` resources further down configure the +new (v2) Teleport updater. + (!docs/pages/includes/cluster-maintenance-config-spec.mdx!) @@ -331,34 +338,231 @@ Find out more on the ## Auto-update config -Configuration options for managed updates for Teleport client tools. +Configuration options for Teleport agent and client tools Managed Updates (v2). + + +The `autoupdate_config` and `autoupdate_version` resources configure the both the +new (v2) Teleport Agent updater and original (v1) Teleport Agent updater. +`cluster_maintenance_config` above configures the original (v1) Teleport Agent updater, +which is currently supported but will be deprecated after the new updater is released. + ```yaml kind: autoupdate_config metadata: name: autoupdate-config spec: + agents: + # mode allows users to enable, disable, or suspend agent updates at the + # cluster level. Disable agent automatic updates only if self-managed + # updates are in place. This value may also be set in autoupdate_version. + # If set in both places, disabled overrides suspended, which overrides enabled. + # Possible values: "enabled", "disabled", "suspended" + # Default: "disabled" (unless specified in autoupdate_version) + mode: enabled + + # strategy used to roll out updates to agents. + # The halt-on-error strategy ensures that groups earlier in the schedule are + # given the opportunity to update to the target_version before groups that are + # later in the schedule. (Currently, the schedule must be stopped manually by + # setting the mode to "suspended" or "disabled". In the future, errors will be + # detected automatically). + # The time-based strategy ensure that each group updates within a defined + # time window, with no dependence between groups. + # Possible values: "halt-on-error" or "time-based" + # Default: "halt-on-error" + strategy: halt-on-error + + # maintenance_window_duration configures the duration after the start_hour + # when updates may occur. Only valid for the time-based strategy. + # maintenance_window_duration: 1h + + # schedules define groups of agents with different update times. + # Currently, only the regular schedule is configurable. + schedules: + regular: + + # name of each group, configured locally via "teleport-update enable --group" + - name: staging + + # start_hour of the update, in UTC + start_hour: 4 + + # days that the update may occur on + # Days are not configurable for most Enterprise cloud-hosted users. + # Possible values: "Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun", and "*" + # Default: [ "Mon", "Tue", "Wed", "Thu" ] + days: [ "Mon", "Tue", "Wed", "Thu" ] + + - name: production + start_hour: 5 + + # wait_hours ensures that the group executes at least a specific number of hours + # after the previous group. Only valid for the halt-on-error schedule. + # Default: 0 + wait_hours: 24 + tools: - # tools mode allows users to enable or disable client tool updates at the + # mode allows users to enable or disable client tool updates at the # cluster level. Disable client tool automatic updates only if self-managed # updates are in place. - mode: enabled|disabled + # Possible values: "enabled" or "disabled" + # Default: "disabled" + mode: enabled ``` -See [Teleport Client Tool Automatic Updates](../upgrading/client-tools-autoupdate.mdx) for more details. +See [Teleport Client Tool Managed Updates](../upgrading/client-tools-autoupdate.mdx) and +[Teleport Agent Managed Updates](../upgrading/agent-managed-updates.mdx) for more details. ## Auto-update version -Allows cluster administrators to manage the version of client tools that must be installed after logging into the cluster. +Allows cluster administrators to manage versions used for agent and client tools Managed Updates. + + +The `autoupdate_config` and `autoupdate_version` resources configure the both the +new (v2) Teleport Agent updater and original (v1) Teleport Agent updater. +`cluster_maintenance_config` above configures the original (v1) Teleport Agent updater, +which is currently supported but will be deprecated after the new updater is released. + + +This resource is not editable for cloud-managed Teleport Enterprise to ensure that all of +your clients receive security patches and remain compatible with your cluster. ```yaml kind: autoupdate_version metadata: name: autoupdate-version spec: + agents: + # start_version is the version used to install new agents before their + # group's scheduled update time. Agents never update to the start_version + # automatically, but may be required to via "teleport-update update --now". + start_version: v17.2.0 + + # target_version is the version that agents update to during their group's + # scheduled update time. New agents also use this version after their group's + # scheduled update time. + target_version: v17.2.1 + + # schedule used to roll out updates. + # The regular schedule is defined in the autoupdate_config resource. + # The immediate schedule updates all agents to target_version immediately. + # Possible values: "regular" or "immediate" + # Default: "regular" + schedule: regular + + # mode allows users to enable, disable, or suspend agent updates at the + # cluster level. Disable agent automatic updates only if self-managed + # updates are in place. This value may also be set in autoupdate_config. + # If set in both places, disabled overrides suspended, which overrides enabled. + # Possible values: "enabled", "disabled", "suspended" + # Default: "disabled" (unless specified in autoupdate_config) + mode: enabled + tools: # target_version is the semver version of client tools the cluster will advertise. - target_version: X.Y.Z + # Client tools such as tsh and tctl will update to this version at login if the + # mode set in autoupdate_config is set to enabled. + # Default: cluster version + target_version: v17.2.1 +``` + +See [Teleport Client Tool Managed Updates](../upgrading/client-tools-autoupdate.mdx) and +[Teleport Agent Managed Updates](../upgrading/agent-managed-updates.mdx) for more details. + +## Auto-update agent rollout + +Allows cluster administrators to view the current agent rollout for Teleport Agent Managed Updates (v2). + +This resource should not be edited by humans. + +```yaml +kind: autoupdate_agent_rollout +metadata: + name: autoupdate-agent-rollout +spec: + # start_version is the version used to install new agents before their + # group's scheduled update time. Agents never update to the start_version + # automatically, but may be required to via "teleport-update update --now". + start_version: v17.2.0 + + # target_version is the version that agents update to during their group's + # scheduled update time. New agents also use this version after their group's + # scheduled update time. + target_version: v17.2.1 + + # schedule used to roll out updates. + # The regular schedule is defined in the autoupdate_config resource. + # The immediate schedule updates all agents to target_version immediately. + # Possible values: "regular" or "immediate" + schedule: regular + + # autoupdate_mode allows users to enable, disable, or suspend agent updates at the + # cluster level. Disable agent automatic updates only if self-managed + # updates are in place. This value may also be set in autoupdate_config. + # If set in both places, disabled overrides suspended, which overrides enabled. + # Possible values: "enabled", "disabled", "suspended" + autoupdate_mode: enabled + + # strategy used to roll out updates to agents. + # The halt-on-error strategy ensures that groups earlier in the schedule are + # given the opportunity to update to the target_version before groups that are + # later in the schedule. (Currently, the schedule must be stopped manually by + # setting the mode to "suspended" or "disabled". In the future, errors will be + # detected automatically). + # The time-based strategy ensure that each group updates within a defined + # time window, with no dependence between groups. + # Possible values: "halt-on-error" or "time-based" + # Default: "halt-on-error" + strategy: halt-on-error + + # maintenance_window_duration configures the duration after the start_hour + # when updates may occur. Only valid for the time-based strategy. + # maintenance_window_duration: 1h + +status: + + # groups contains the status for each group in the currently executing schedule. + groups: + + # name of each group, configured locally via "teleport-update enable --group" + - name: staging + + # start_time of the group + start_time: 0001-01-01T00:00:00Z + + # state of the group + # Possible values: unstarted, active, done, rolledback + state: active + + # last_update_time of this group's status + last_update_time: 0001-01-01T00:00:00Z + + # last_update_reason of this group's status + last_update_reason: "new version" + + # days that the update may occur on, from autoupdate_config + # Possible values: "Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun", and "*" + config_days: [ "Mon", "Tue", "Wed", "Thu" ] + + # start_hour of the update, in UTC, from autoupdate_config + config_start_hour: 4 + + - name: production + + # ... + + # config_wait_hours is specific number of hours after the previous group that this + # group may execute after, from autoupdate_config. + config_wait_hours: 24 + + # start_time of the rollout + start_time: 0001-01-01T00:00:00Z + + # state of the entire rollout + # Possible values: unstarted, active, done, rolledback + state: active + ``` -See [Teleport Client Tool Automatic Updates](../upgrading/client-tools-autoupdate.mdx) for more details. \ No newline at end of file +See [Teleport Agent Managed Updates](../upgrading/agent-managed-updates.mdx) for more details. diff --git a/docs/pages/upgrading/automatic-agent-updates.mdx b/docs/pages/upgrading/agent-managed-updates-v1.mdx similarity index 82% rename from docs/pages/upgrading/automatic-agent-updates.mdx rename to docs/pages/upgrading/agent-managed-updates-v1.mdx index 00bb8e7ac5328..ef83cbc76ee90 100644 --- a/docs/pages/upgrading/automatic-agent-updates.mdx +++ b/docs/pages/upgrading/agent-managed-updates-v1.mdx @@ -1,9 +1,36 @@ --- -title: Set up Automatic Agent Updates -description: Describes how to set up automatic agent updates. +title: Teleport Managed Agent Updates (v1) +description: Describes how to set up managed agent updates using the original updater. --- -On cloud-hosted Teleport Enterprise accounts, users must set up automatic agent + +This document describes the original (v1) Teleport Agent updater, which is +currently supported but will be deprecated after the new (v2) updater is released. + + +The original updater uses a script called `teleport-upgrade` that is provided by +the `teleport-ent-updater` package and configured by the `cluster_maintenance_config` +Teleport resource. The new updater uses a binary called `teleport-update` that is +provided by the `teleport` package and configured by the `autoupdate_version` and +`autoupdate_config`. The original updater and resource are described in this document. + +Please consider using the new updater, which will be distributed with +all versions of Teleport and provides a safer, simpler, more flexible, and more reliable +update experience compared to the original updater. + +The new `teleport-update` updater is backwards compatible with the +`cluster_maintenance_config` resource, and the old `teleport-upgrade` script +is forwards compatible with the `autoupdate_` resources. +Agents connected to the same cluster may use either resource. +If the `autoupdate_ ` resources are present, `cluster_maintenance_config` is ignored. +This allows for a safe, non-breaking, incremental migration to the new update system. +Users of cloud-hosted Teleport Enterprise will be migrated to the new set of resources +in the first half of 2025 and should plan to migrate their agents to `teleport-update`. + +For details about the new updater, see +[Managed Updates for Agents (v2)](./agent-managed-updates.mdx). + +On cloud-hosted Teleport Enterprise accounts, users must set up managed agent updates to ensure that the version of Teleport running on agents remains compatible with the version running on the Auth Service and Proxy Service. If an agent does not maintain [version compatibility](./overview.mdx) with your @@ -14,17 +41,17 @@ updates are performed every 4 months. You can monitor and subscribe to the [Teleport Status](https://status.teleport.sh/) page to be notified of scheduled updates. -Teleport supports automatic agent updates for systemd-based Linux distributions +Teleport supports managed agent updates for SystemD-based Linux distributions using `apt`, `yum`, and `zypper` package managers, as well as Kubernetes clusters. -This guide explains how to enable automatic updates for Teleport agents on +This guide explains how to enable managed updates for Teleport agents on Teleport Enterprise clusters, including both self-hosted and cloud-hosted clusters. ## How it works -When automatic updates are enabled, a Teleport updater is installed alongside +When managed updates are enabled, a Teleport updater is installed alongside each Teleport agent. The updater communicates with the Teleport Proxy Service to determine when an update is available. When an update is available, the updater will update the Teleport agent during the next maintenance window. However, if a @@ -38,22 +65,22 @@ regular maintenance window. Reference](upgrading-reference.mdx) to read about manual updates. - Familiarity with the [Upgrading Compatibility Overview](./overview.mdx) guide, which describes the sequence in which to upgrade components in your cluster. -- Teleport agents that are not yet enrolled in automatic updates. +- Teleport agents that are not yet enrolled in managed updates. - (!docs/pages/includes/tctl-tsh-prerequisite.mdx!) - (!docs/pages/includes/tctl.mdx!) -## Step 1/4. Enable automatic agent upgrades +## Step 1/4. Enable managed agent upgrades If you are running a cloud-hosted Teleport Enterprise cluster, skip to [Step 2](#step-24-find-agents-to-enroll-in-automatic-updates). -Before enabling automatic upgrades in your self-hosted Teleport cluster, you +Before enabling managed upgrades in your self-hosted Teleport cluster, you must enable a **version server**. This section shows you how to enable a version server in your cluster. ### Configure a maintenance schedule -To enable automatic upgrades in your cluster, you must create a cluster +To enable managed upgrades in your cluster, you must create a cluster maintenance configuration. This configures a maintenance schedule for the Teleport cluster that agents use to determine when to check for upgrades. @@ -137,10 +164,10 @@ $ curl "https:///v1/webapi/automaticupgra (=teleport.version=) ``` -## Step 2/4. Find agents to enroll in automatic updates +## Step 2/4. Find agents to enroll in managed updates Use the `tctl inventory ls` command to list connected agents along with their current -version. Use the `--upgrader=none` flag to list agents that are not enrolled in automatic +version. Use the `--upgrader=none` flag to list agents that are not enrolled in managed updates. ```code @@ -151,7 +178,7 @@ Server ID Hostname Services Version Upgrader ... ``` -## Step 3/4. Enroll agents on Linux servers in automatic updates +## Step 3/4. Enroll agents on Linux servers in managed updates 1. For each agent ID returned by the `tctl inventory ls` command, copy the ID and run the following `tctl` command to access the host via `tsh`: @@ -177,7 +204,7 @@ Server ID Hostname Services Version Upgrader 1. Ensure that the Teleport repository is properly configured to use the channel, and install the `teleport-ent-updater` package. You must install `teleport-ent-updater` on each agent you would like - to enroll into automatic updates: + to enroll into managed updates: @@ -246,7 +273,7 @@ $ sudo chown /etc/teleport-upgrade.d/schedule schedule. This is expected immediately after install as the maintenance schedule might not be exported yet. -## Step 4/4. Enroll Kubernetes agents in automatic updates +## Step 4/4. Enroll Kubernetes agents in managed updates This section assumes that the name of your `teleport-kube-agent` release is `teleport-agent`, and that you have installed it in the `teleport` namespace. @@ -334,7 +361,7 @@ can consult the updater logs to help troubleshoot the problem: $ journalctl -u teleport-upgrade ``` -### Troubleshooting automatic agent upgrades on Kubernetes +### Troubleshooting managed agent upgrades on Kubernetes The updater is a controller that periodically reconciles expected Kubernetes resources with those in the cluster. The updater executes a reconciliation loop @@ -348,12 +375,12 @@ until the next reconciliation, you can trigger an event. $ kubectl -n annotate statefulset/ 'debug.teleport.dev/trigger-event=1' ``` -1. To suspend automatic updates for an agent, annotate the agent deployment +1. To suspend managed updates for an agent, annotate the agent deployment with `teleport.dev/skipreconcile: "true"`, either by setting the `annotations.deployment` value in Helm, or by patching the deployment directly with `kubectl`. -### Troubleshooting automatic agent upgrades on Linux +### Troubleshooting managed agent upgrades on Linux 1. If an agent is not automatically upgraded, you can invoke the upgrader manually and look at its logs: @@ -362,7 +389,7 @@ until the next reconciliation, you can trigger an event. $ sudo teleport-upgrade run ``` -1. To suspend automatic updates, disable the systemd timer: +1. To suspend managed updates, disable the systemd timer: ```code $ sudo systemctl disable --now teleport-upgrade.timer diff --git a/docs/pages/upgrading/agent-managed-updates.mdx b/docs/pages/upgrading/agent-managed-updates.mdx new file mode 100644 index 0000000000000..1ae06ea7fbde5 --- /dev/null +++ b/docs/pages/upgrading/agent-managed-updates.mdx @@ -0,0 +1,465 @@ +--- +title: Teleport Managed Agent Updates (v2) +description: Describes how to set up managed agent updates using the next-generation updater. +--- + + +This document describes the new (v2) Teleport Agent updater, which is currently +in beta. + + +The original updater uses a script called `teleport-upgrade` that is provided by +the `teleport-ent-updater` package and configured by the `cluster_maintenance_config` +Teleport resource. The new updater uses a binary called `teleport-update` that is +provided by the `teleport` package and configured by the `autoupdate_version` and +`autoupdate_config`. The new updater and resources are described in this document. + +Please consider using the new updater, which will be distributed with +all versions of Teleport and provides a safer, simpler, more flexible, and more reliable +update experience compared to the original updater. + +The new `teleport-update` updater is backwards compatible with the +`cluster_maintenance_config` resource, and the old `teleport-upgrade` script +is forwards compatible with the `autoupdate_` resources. +Agents connected to the same cluster may use either resource. +If the `autoupdate_ ` resources are present, `cluster_maintenance_config` is ignored. +This allows for a safe, non-breaking, incremental migration to the new update system. +Users of cloud-hosted Teleport Enterprise will be migrated to the new set of resources +in the first half of 2025 and should plan to migrate their agents to `teleport-update`. + +For details about the original updater, see +[Managed Updates for Agents (v1)](./agent-managed-updates-v1.mdx). + +On cloud-hosted Teleport Enterprise accounts, users must set up managed agent +updates to ensure that the version of Teleport running on agents remains +compatible with the version running on the Auth Service and Proxy Service. If an +agent does not maintain [version compatibility](./overview.mdx) with your +Teleport cluster, connections to those agents will become degraded or lost. + +Cloud-hosted Teleport clusters are updated on a weekly basis. Major version +updates are performed every 4 months. You can monitor and subscribe to the +[Teleport Status](https://status.teleport.sh/) page to be notified of scheduled +updates. + +Teleport supports managed agent updates for SystemD-based Linux distributions +as well as Kubernetes clusters. + +This guide explains how to enable managed updates for Teleport agents on +Teleport Enterprise clusters, including both self-hosted and cloud-hosted +clusters. + +## How it works + +When managed updates are enabled, a Teleport updater runs alongside +each Teleport agent. The updater communicates with the Teleport Proxy Service to +determine when an update is available. When an update is available, the updater +will update the Teleport agent. Each agent belongs to an update group, and the +time each group is updated can be configured using `tctl`. + +For Linux server-based installations, `teleport-update` command configures +managed updates locally on the server. + +For Kubernetes-based installations, the `teleport-kube-agent` Helm chart +is configured to automatically update the Teleport container. + +## Prerequisites + +- A Teleport Enterprise cluster. If you do not have one, [sign + up](https://goteleport.com/signup) for a free trial or consult the [Update + Reference](upgrading-reference.mdx) to read about manual updates. +- Familiarity with the [Upgrading Compatibility Overview](./overview.mdx) guide, + which describes the sequence in which to upgrade components in your cluster. +- Teleport agents that are not yet enrolled in managed updates. +- (!docs/pages/includes/tctl-tsh-prerequisite.mdx!) +- (!docs/pages/includes/tctl.mdx!) + +## Quick setup for existing connected Linux servers + +Users of cloud-hosted Teleport Enterprise can enable managed updates +on Linux servers that are already connected with a single command: + +```code +$ sudo teleport-update enable +``` + + +If this command is not available, update the `teleport` package +to the latest version that is supported by your cluster. + + +The `teleport-update enable` command will disable (but not remove) +the original (v1) updater if present. No other action is necessary. + +If everything is working, the deprecated updater can be removed: + +```code +$ sudo apt remove teleport-ent-updater +``` + +If the new updater does not work, your installation can be reverted +back to manual updates or the original updater (if it has not been removed): + +```code +$ sudo teleport-update uninstall +``` + +If Teleport was installed via the apt or yum package, this will revert the +running version of Teleport back to the version provided by the package. + +For cloud-hosted Teleport Enterprise, updates will roll out automatically +during the first chosen maintenance window that is at least 36 hours after +the cluster version is updated. Keep reading to learn how to customize the +schedule. + +## Quick setup for new Linux servers + +The [Install Script](installation.mdx#one-line-installation-script) is the +fastest way to onboard new Linux servers. However, you may also use +`teleport-update` by itself to setup a Teleport agent manually. + +Users can create a new installation of Teleport using any version of the +`teleport-update` binary. First, download copy of the [Teleport tarball] from +the downloads page. Next, invoke `teleport-update` to install the correct version +for your cluster. + +```code +$ tar xf teleport-[version].tgz +$ cd teleport-[version] +$ sudo ./teleport-update enable --proxy example.teleport.sh +``` + +After Teleport is installed, you can create `/etc/teleport.yaml`, either manually +or using `teleport configure`. After, the Teleport agent can be enabled and +started via the `systemctl` command: + +```code +$ sudo systemctl enable teleport --now +``` + +## Configuring managed agent updates + +To configure managed updates in your cluster, you must have access to +the `autoupdate_config` and `autoupdate_version` resources. By default, +the `editor` role can modify both resources (self-hosted) or just +`autoupdate_config` (cloud-hosted). + +### Configuring the schedule + +For both cloud-hosted and self-hosted editions of Teleport, an update schedule +may be set with the `autoupdate_config` resource. The default resource looks +like this: + +```yaml +kind: autoupdate_config +spec: + agents: + schedules: + regular: + - name: default + days: [ "Mon", "Tue", "Wed", "Thu" ] + start_hour: 16 +``` + +For cloud-hosted Teleport Enterprise, the `days` are not configurable for most +customers, and the `start_hour` is defaulted to your selected maintenance window. + +For example, a user of cloud-hosted Teleport with a staging environment and a +production environment might create a custom schedule that looks like this: + +```yaml +kind: autoupdate_config +spec: + agents: + schedules: + regular: + - name: staging + start_hour: 4 + - name: production + start_hour: 5 + wait_hours: 24 +``` + +This schedule would update agents in the `staging` group at 4 UTC, and then update +the `production` group at 5 UTC the next day. The `production` group will not execute +update until the staging group has updated. The `wait_hours` field sets a minimum +duration between groups, ensuring that `production` happens the day after `staging`, +and not one hour after. + + +While failed installations will revert automatically on the client-side, +server-side healthchecks are still in development. To prevent the `production` +group above from updating after `staging` has failed, you must manually suspend +the schedule by setting the `spec.agents.mode` to `suspended`. + + +You may wish to schedule groups of agents to update without any dependence between +them. For example, groups may represent geographic areas and not environments. +To accomplish this, you can change the default `halt-on-error` strategy to the +`time-based` strategy: + +```yaml +kind: autoupdate_config +spec: + agents: + strategy: time-based + maintenance_window_duration: 1h + schedules: + regular: + - name: nyc + start_hour: 4 + - name: sjc + start_hour: 20 +``` + +With this strategy, updates to `sjc` may occur before `nyc`, depending on when +new versions become available. The `maintenance_window_duration` restricts +updates to the specified duration after the `start_hour`. This ensures that +disruptions do not occur outside a known window. + +The time-based strategy does not support the `wait_days` option. + +To add agents to groups, run `teleport-update enable --group group-name`. +You may execute `teleport-update enable` repeatedly to change the group +(or other managed update settings). + +### Setting the version + +For cloud-hosted Teleport Enterprise, managed updates are enabled by default. +The `autoupdate_version` resource is managed for you and cannot be edited. + +For self-hosted Teleport Enterprise, the `autoupdate_version` resource must be +created and edited to enabled managed updates. + +Create a file called `autoupdate_version.yaml` containing: + +```yaml +kind: autoupdate_version +spec: + agents: + start_version: v17.2.0 + target_version: v17.2.1 + schedule: regular + mode: enabled +``` + +This resource is used to deploy new versions of Teleport to your agents. +The `start_version` is the version used to install new agents before their +scheduled update time, while `target_version` is the version that agents +will be updated to according to the configured schedule. + +The `schedule` may be changed from `regular` to `immediate` to force all +agents to update to the `target_version` immediately. + +The `mode` is used to enable, disable, or suspend managed updates. +The `mode` may be set in both `autoupdate_version` or `autoupdate_config`, +such that `disabled` overrides `suspended`, which overrides `enabled` on either side. + +Run the following command to create or update the resource: + +```code +$ tctl create autoupdate_version.yaml +``` +## Migrating agents on Linux servers to managed updates + +### Finding unmanaged agents + +Use the `tctl inventory ls` command to list connected agents along with their current +version. Use the `--upgrader=none` flag to list agents that are not enrolled in managed +updates. + +```code +$ tctl inventory ls --upgrader=none +Server ID Hostname Services Version Upgrader +------------------------------------ ------------- -------- ------- -------- +00000000-0000-0000-0000-000000000000 ip-10-1-6-130 Node v14.4.5 none +... +``` + +Use the `--upgrader=unit` flag to list agents that are enrolled in the original +updater and need to be configured to the new updater (`teleport-update`): + +```code +$ tctl inventory ls --upgrader=unit +Server ID Hostname Services Version Upgrader +------------------------------------ ------------- -------- ------- -------- +00000000-0000-0000-0000-000000000000 ip-10-1-6-131 Node v14.4.5 unit +... +``` + +Agents using the new updater can be queried with the `--upgrader=binary` flag. + +### Enrolling unmanaged agents + +1. For each agent ID returned by the `tctl inventory ls` command, copy the ID + and run the following `tctl` command to access the host via `tsh`: + + ```code + $ HOST=00000000-0000-0000-0000-000000000000 + $ USER=root + $ tsh ssh "${USER?}@${HOST?}" + ``` + +1. Run `teleport-update enable` on each agent you would like + to enroll into managed updates: + + ```code + $ sudo teleport-update enable + ``` + +1. Confirm that the version of the `teleport` binary is the one you expect: + + ```code + $ teleport version + ``` + +1. Remove the original updater if present: + + + + + ```code + $ sudo apt remove teleport-ent-updater + ``` + + + + + ```code + $ sudo yum remove teleport-ent-updater + ``` + + + + +
+ +If you changed the agent user to run as non-root, create +`/etc/teleport-upgrade.d/schedule` and grant ownership to your Teleport user: + +```code +$ sudo mkdir -p /etc/teleport-upgrade.d/ +$ sudo touch /etc/teleport-upgrade.d/schedule +$ sudo chown /etc/teleport-upgrade.d/schedule +``` + +While `teleport-update` does not read this file, `teleport` will warn if it +cannot disable the original updater using this file. + +
+ +## Enroll Kubernetes agents in managed updates + +This section assumes that the name of your `teleport-kube-agent` release is +`teleport-agent`, and that you have installed it in the `teleport` namespace. + +1. Confirm that you are using the Teleport Enterprise edition of the + `teleport-kube-agent` chart. You should see the following when you query your + `teleport-kube-agent` release: + + ```code + $ helm -n `teleport` get values `teleport-agent` -o json | jq '.enterprise' + true + ``` + + If another value such as `null` is returned, update your existing agent + `values.yaml` to use the Enterprise version: + + ```yaml + enterprise: true + ``` + +1. Add the following chart values to the values file for the + `teleport-kube-agent` chart: + + ```yaml + updater: + enabled: true + ``` + +1. Update the Teleport Helm repository to include any new versions of the + `teleport-kube-agent` chart: + + ```code + $ helm repo update teleport + ``` + +1. Update the Helm chart release with the new values: + + + + + ```code + $ helm -n upgrade teleport/teleport-kube-agent \ + --values=values.yaml \ + --version="(=cloud.version=)" + ``` + + + + ```code + $ helm -n upgrade teleport/teleport-kube-agent \ + --values=values.yaml \ + --version="(=teleport.version=)" + ``` + + + +1. You can validate the updater is running properly by checking if its pod is + ready: + + ```code + $ kubectl -n teleport-agent get pods + NAME READY STATUS RESTARTS AGE + -0 1/1 Running 0 14m + -1 1/1 Running 0 14m + -2 1/1 Running 0 14m + -updater-d9f97f5dd-v57g9 1/1 Running 0 16m + ``` + +1. Check for any deployment issues by checking the updater logs: + + ```code + $ kubectl -n logs deployment/-updater + 2023-04-28T13:13:30Z INFO StatefulSet is already up-to-date, not updating. {"controller": "statefulset", "controllerGroup": "apps", "controllerKind": "StatefulSet", "StatefulSet": {"name":"my-agent","namespace":"agent"}, "namespace": "agent", "name": "my-agent", "reconcileID": "10419f20-a4c9-45d4-a16f-406866b7fc05", "namespacedname": "agent/my-agent", "kind": "StatefulSet", "err": "no new version (current: \"v12.2.3\", next: \"v12.2.3\")"} + ``` + +## Troubleshooting + +Teleport agents are not updated immediately when a new version of Teleport is +released, and agent updates can lag behind the cluster by a few days. + +If the Teleport agent has not been automatically updating for several weeks, you +can consult the updater logs to help troubleshoot the problem: + +```code +$ journalctl -u teleport-update +``` + +### Troubleshooting managed agent upgrades on Kubernetes + +The updater is a controller that periodically reconciles expected Kubernetes +resources with those in the cluster. The updater executes a reconciliation loop +every 30 minutes or in response to a Kubernetes event. If you don't want to wait +until the next reconciliation, you can trigger an event. + +1. Any deployment update will send an event, so you can trigger the upgrader by + annotating the resource: + + ```code + $ kubectl -n annotate statefulset/ 'debug.teleport.dev/trigger-event=1' + ``` + +1. To suspend managed updates for an agent, annotate the agent deployment + with `teleport.dev/skipreconcile: "true"`, either by setting the + `annotations.deployment` value in Helm, or by patching the deployment + directly with `kubectl`. + +### Troubleshooting managed agent upgrades on Linux + +1. If an agent is not automatically updated, you can invoke the updater + manually and look at its logs: + + ```code + $ sudo teleport-update update --now + ``` +