Skip to content

Latest commit

 

History

History
194 lines (141 loc) · 6.13 KB

README.md

File metadata and controls

194 lines (141 loc) · 6.13 KB

Upgrade to v0.35.x

Warning

Upgrade only supported from v0.34.x.

Prerequisites

  • Read through the changelog to check if there are any changes you need to be aware of. Read through the release notes, Platform Administrator notices, Application Developer notices, and Security notice.

  • Notify the users (if any) before the upgrade starts;

  • Check if there are any pending changes to the environment;

  • Check the state of the environment, pods, nodes and backup jobs:

    ./bin/ck8s test sc|wc
./bin/ck8s ops kubectl sc|wc get pods -A -o custom-columns=NAMESPACE:metadata.namespace,POD:metadata.name,READY-false:status.containerStatuses[*].ready,REASON:status.containerStatuses[*].state.terminated.reason | grep false | grep -v Completed
./bin/ck8s ops kubectl sc|wc get nodes
./bin/ck8s ops kubectl sc|wc get jobs -A
./bin/ck8s ops helm sc|wc list -A --all
velero get backup

  • Silence the notifications for the alerts. e.g you can use alertmanager silences;

Automatic method

  1. Pull the latest changes and switch to the correct branch:

    git pull
git switch -d v0.35.x

  2. Prepare upgrade - non-disruptive

    Done before maintenance window.

    ./bin/ck8s upgrade both v0.35 prepare

# check if the netpol IPs need to be updated
./bin/ck8s update-ips both dry-run
# if you agree with the changes apply
./bin/ck8s update-ips both apply

    [!NOTE] It is possible to upgrade wc and sc clusters separately by replacing both when running the upgrade command, e.g. the following will only upgrade the workload cluster:

    ./bin/ck8s upgrade wc v0.35 prepare
./bin/ck8s upgrade wc v0.35 apply

  3. Apply upgrade - disruptive

    Done during maintenance window.

    ./bin/ck8s upgrade both v0.35 apply

Manual method

Prepare upgrade - non-disruptive

Done before maintenance window.

  1. Pull the latest changes and switch to the correct branch:

    git pull
git switch -d v0.35.x

  2. Set whether or not upgrade should be prepared for both clusters or for one of sc or wc:

    export CK8S_CLUSTER=<wc|sc|both>

  3. Remove deprecated Harbor Notary config:

./migration/v0.35/prepare/40-harbor.sh

  1. Update apps configuration:

    This will take a backup into backups/ before modifying any files.

    ./bin/ck8s init ${CK8S_CLUSTER}
# or
./migration/v0.35/prepare/50-init.sh

# check if the netpol IPs need to be updated
./bin/ck8s update-ips ${CK8S_CLUSTER} dry-run
# if you agree with the changes apply
./bin/ck8s update-ips ${CK8S_CLUSTER} apply

Apply upgrade - disruptive

Done during maintenance window.

  1. Set whether or not upgrade should be applied for both clusters or for one of sc or wc:

    export CK8S_CLUSTER=<wc|sc|both>

  2. Apply the trivy-operator CRDs

    ./migration/v0.35/apply/11-trivy-operator-crds.sh execute

  3. Rerun bootstrap:

    ./bin/ck8s bootstrap {sc|wc}
# or
./migration/v0.35/apply/20-bootstrap.sh execute

  4. Upgrade harbor:

    ./migration/v0.35/apply/50-harbor.sh execute

  5. Upgrade Velero

    ./migration/v0.35/apply/60-velero-crds-upgrade.sh execute

  6. Upgrade applications:

    ./bin/ck8s apply {sc|wc}
# or
./migration/v0.35/apply/80-apply.sh execute

Postrequisite:

  • Check the state of the environment, pods and nodes:

    ./bin/ck8s test sc|wc
./bin/ck8s ops kubectl sc|wc get pods -A -o custom-columns=NAMESPACE:metadata.namespace,POD:metadata.name,READY-false:status.containerStatuses[*].ready,REASON:status.containerStatuses[*].state.terminated.reason | grep false | grep -v Completed
./bin/ck8s ops kubectl sc|wc get nodes
./bin/ck8s ops helm sc|wc list -A --all

  • Enable the notifications for the alerts;

  • Notify the users (if any) when the upgrade is complete;

Note

Additionally it is good to check:

  • if any alerts generated by the upgrade didn't close;
  • if you can login to Grafana, Opensearch or Harbor;
  • you can see fresh metrics and logs.