diff --git a/.changelog/13023.txt b/.changelog/13023.txt new file mode 100644 index 000000000000..cadf7bb93849 --- /dev/null +++ b/.changelog/13023.txt @@ -0,0 +1,3 @@ +```release-note:improvement +ui: the topology view now properly displays services with mixed connect and non-connect instances. +``` diff --git a/.changelog/17075.txt b/.changelog/17075.txt new file mode 100644 index 000000000000..1c882fabbddd --- /dev/null +++ b/.changelog/17075.txt @@ -0,0 +1,3 @@ +```release-note:improvement +agent: remove agent cache dependency from service mesh leaf certificate management +``` diff --git a/.changelog/17160.txt b/.changelog/17160.txt new file mode 100644 index 000000000000..666a6e8f252c --- /dev/null +++ b/.changelog/17160.txt @@ -0,0 +1,3 @@ +```release-note:bug +Fix a bug that wrongly trims domains when there is an overlap with DC name. +``` diff --git a/.changelog/17483.txt b/.changelog/17483.txt new file mode 100644 index 000000000000..26c81dbe4cdf --- /dev/null +++ b/.changelog/17483.txt @@ -0,0 +1,3 @@ +```release-note:bug +peering: Fix a bug that caused server agents to continue cleaning up peering resources even after loss of leadership. +``` diff --git a/.changelog/17546.txt b/.changelog/17546.txt new file mode 100644 index 000000000000..6f4025553986 --- /dev/null +++ b/.changelog/17546.txt @@ -0,0 +1,3 @@ +```release-note:improvement +connect: update supported envoy versions to 1.23.10, 1.24.8, 1.25.7, 1.26.2 +``` diff --git a/.changelog/17565.txt b/.changelog/17565.txt new file mode 100644 index 000000000000..f7cf46c38954 --- /dev/null +++ b/.changelog/17565.txt @@ -0,0 +1,3 @@ +```release-note:feature +reloadable config: Made enable_debug config reloadable and enable pprof command to work when config toggles to true +``` \ No newline at end of file diff --git a/.changelog/17582.txt b/.changelog/17582.txt new file mode 100644 index 000000000000..122b9df98116 --- /dev/null +++ b/.changelog/17582.txt @@ -0,0 +1,3 @@ +```release-note:feature +cli: `consul operator raft list-peers` command shows the number of commits each follower is trailing the leader by to aid in troubleshooting. +``` diff --git a/.changelog/17596.txt b/.changelog/17596.txt new file mode 100644 index 000000000000..1058df1ea3ab --- /dev/null +++ b/.changelog/17596.txt @@ -0,0 +1,3 @@ +```release-note:improvement + debug: change default setting of consul debug command. now default duration is 5ms and default log level is 'TRACE' + ``` \ No newline at end of file diff --git a/.changelog/17609.txt b/.changelog/17609.txt new file mode 100644 index 000000000000..cbace1f8c7d2 --- /dev/null +++ b/.changelog/17609.txt @@ -0,0 +1,4 @@ +```release-note:bug +gateways: Fixed a bug in API gateways where binding a route that only targets a service imported from a peer results +in the programmed gateway having no routes. +``` diff --git a/.changelog/17631.txt b/.changelog/17631.txt new file mode 100644 index 000000000000..b24b7461ec6e --- /dev/null +++ b/.changelog/17631.txt @@ -0,0 +1,3 @@ +```release-note:bug +gateways: Fixed a bug where API gateways were not being taken into account in determining xDS rate limits. +``` diff --git a/.changelog/17719.txt b/.changelog/17719.txt new file mode 100644 index 000000000000..f45370b3f7dc --- /dev/null +++ b/.changelog/17719.txt @@ -0,0 +1,3 @@ +```release-note:security +Bump Dockerfile base image to `alpine:3.18`. + ``` diff --git a/.changelog/17739.txt b/.changelog/17739.txt new file mode 100644 index 000000000000..14bbceeaa084 --- /dev/null +++ b/.changelog/17739.txt @@ -0,0 +1,3 @@ +```release-note:bug +http: fixed API endpoint `PUT /acl/token/:AccessorID` (update token), no longer requires `AccessorID` in the request body. Web UI can now update tokens. + ``` diff --git a/.changelog/17754.txt b/.changelog/17754.txt new file mode 100644 index 000000000000..56ab20dc213e --- /dev/null +++ b/.changelog/17754.txt @@ -0,0 +1,3 @@ +```release-note:feature +ui: consul version is displayed in nodes list with filtering and sorting based on versions +``` diff --git a/.changelog/17755.txt b/.changelog/17755.txt new file mode 100644 index 000000000000..7edf7b26e159 --- /dev/null +++ b/.changelog/17755.txt @@ -0,0 +1,3 @@ +```release-note:improvement +mesh: Stop jwt providers referenced by intentions from being deleted. +``` \ No newline at end of file diff --git a/.changelog/17757.txt b/.changelog/17757.txt new file mode 100644 index 000000000000..e207438cf843 --- /dev/null +++ b/.changelog/17757.txt @@ -0,0 +1,3 @@ +```release-note:improvement +connect: Improve transparent proxy support for virtual services and failovers. +``` diff --git a/.changelog/17759.txt b/.changelog/17759.txt new file mode 100644 index 000000000000..0836608ae1f2 --- /dev/null +++ b/.changelog/17759.txt @@ -0,0 +1,3 @@ +```release-note:improvement +extensions: Improve validation and error feedback for `property-override` builtin Envoy extension +``` diff --git a/.changelog/17775.txt b/.changelog/17775.txt new file mode 100644 index 000000000000..8060cfa128ce --- /dev/null +++ b/.changelog/17775.txt @@ -0,0 +1,3 @@ +```release-note:bug +connect: Fix issue where changes to service exports were not reflected in proxies. +``` diff --git a/.changelog/17780.txt b/.changelog/17780.txt new file mode 100644 index 000000000000..b90925a8b9fd --- /dev/null +++ b/.changelog/17780.txt @@ -0,0 +1,3 @@ +```release-note:feature +cli: `consul watch` command uses `-filter` expression to filter response from checks, services, nodes, and service. +``` diff --git a/.changelog/17846.txt b/.changelog/17846.txt new file mode 100644 index 000000000000..bd5a052f851f --- /dev/null +++ b/.changelog/17846.txt @@ -0,0 +1,3 @@ +```release-note:bug +connect/ca: Fixes a bug preventing CA configuration updates in secondary datacenters +``` diff --git a/.changelog/17885.txt b/.changelog/17885.txt new file mode 100644 index 000000000000..2cd690488d92 --- /dev/null +++ b/.changelog/17885.txt @@ -0,0 +1,2 @@ +```release-note:bug +ca: Fixed a bug where the Vault provider was not passing the configured role param for AWS auth diff --git a/.changelog/17888.txt b/.changelog/17888.txt new file mode 100644 index 000000000000..f50fcae09b07 --- /dev/null +++ b/.changelog/17888.txt @@ -0,0 +1,3 @@ +```release-note:improvement +connect: Add capture group labels from Envoy cluster FQDNs to Envoy exported metric labels +``` \ No newline at end of file diff --git a/.changelog/17894.txt b/.changelog/17894.txt new file mode 100644 index 000000000000..5749f995f71a --- /dev/null +++ b/.changelog/17894.txt @@ -0,0 +1,3 @@ +```release-note:bug +connect: Fix incorrect protocol config merging for transparent proxy implicit upstreams. +``` diff --git a/.changelog/17911.txt b/.changelog/17911.txt new file mode 100644 index 000000000000..a17cd6d2bdf2 --- /dev/null +++ b/.changelog/17911.txt @@ -0,0 +1,4 @@ +```release-note:bug +gateway: Fixes a bug where envoy would silently reject RSA keys that are smaller than 2048 bits, +we now reject those earlier in the process when we validate the certificate. +``` diff --git a/.changelog/17939.txt b/.changelog/17939.txt new file mode 100644 index 000000000000..069ac53ee7be --- /dev/null +++ b/.changelog/17939.txt @@ -0,0 +1,4 @@ +```release-note:improvement +http: GET API `operator/usage` endpoint now returns node count +cli: `consul operator usage` command now returns node count +``` \ No newline at end of file diff --git a/.changelog/17978.txt b/.changelog/17978.txt new file mode 100644 index 000000000000..81d5a648da1f --- /dev/null +++ b/.changelog/17978.txt @@ -0,0 +1,3 @@ +```release-note:improvement +mesh: Expose remote jwks cluster configuration through jwt-provider config entry +``` \ No newline at end of file diff --git a/.changelog/18011.txt b/.changelog/18011.txt new file mode 100644 index 000000000000..d6c989f00e9f --- /dev/null +++ b/.changelog/18011.txt @@ -0,0 +1,4 @@ +```release-note:bug +connect: Removes the default health check from the `consul connect envoy` command when starting an API Gateway. +This health check would always fail. +``` diff --git a/.changelog/18024.txt b/.changelog/18024.txt new file mode 100644 index 000000000000..a661e7304c62 --- /dev/null +++ b/.changelog/18024.txt @@ -0,0 +1,3 @@ +```release-note:bug +connect: fix a bug with Envoy potentially starting with incomplete configuration by not waiting enough for initial xDS configuration. +``` \ No newline at end of file diff --git a/.changelog/18068.txt b/.changelog/18068.txt new file mode 100644 index 000000000000..be55ad365877 --- /dev/null +++ b/.changelog/18068.txt @@ -0,0 +1,3 @@ +```release-note:bug +xds: Prevent partial application of non-Required Envoy extensions in the case of failure. +``` \ No newline at end of file diff --git a/.changelog/18080.txt b/.changelog/18080.txt new file mode 100644 index 000000000000..9826b249eb31 --- /dev/null +++ b/.changelog/18080.txt @@ -0,0 +1,3 @@ +```release-note:improvement +Fix some typos in metrics docs +``` \ No newline at end of file diff --git a/.changelog/18112.txt b/.changelog/18112.txt new file mode 100644 index 000000000000..ddd37786f552 --- /dev/null +++ b/.changelog/18112.txt @@ -0,0 +1,3 @@ +```release-note:bug +ca: Fixes a Vault CA provider bug where updating RootPKIPath but not IntermediatePKIPath would not renew leaf signing certificates +``` diff --git a/.changelog/18140.txt b/.changelog/18140.txt new file mode 100644 index 000000000000..fabd9fc2916b --- /dev/null +++ b/.changelog/18140.txt @@ -0,0 +1,3 @@ +```release-note:improvement +hcp: Removes requirement for HCP to provide a management token +``` diff --git a/.changelog/18150.txt b/.changelog/18150.txt new file mode 100644 index 000000000000..492e7ad1b9ff --- /dev/null +++ b/.changelog/18150.txt @@ -0,0 +1,3 @@ +```release-note:improvement +xds: Explicitly enable WebSocket connection upgrades in HTTP connection manager +``` diff --git a/.changelog/18184.txt b/.changelog/18184.txt new file mode 100644 index 000000000000..594546ca3259 --- /dev/null +++ b/.changelog/18184.txt @@ -0,0 +1,3 @@ +```release-note:bug +api: Fix client deserialization errors by marking new Enterprise-only prepared query fields as omit empty +``` diff --git a/.changelog/18186.txt b/.changelog/18186.txt new file mode 100644 index 000000000000..dcc75b57653b --- /dev/null +++ b/.changelog/18186.txt @@ -0,0 +1,3 @@ +```release-note:security +Upgrade golang.org/x/net to address [CVE-2023-29406](https://nvd.nist.gov/vuln/detail/CVE-2023-29406) +``` diff --git a/.changelog/18190.txt b/.changelog/18190.txt new file mode 100644 index 000000000000..3468442e2161 --- /dev/null +++ b/.changelog/18190.txt @@ -0,0 +1,5 @@ +```release-note:security +Upgrade to use Go 1.20.6. +This resolves [CVE-2023-29406](https://github.com/advisories/GHSA-f8f7-69v5-w4vx)(`net/http`) for uses of the standard library. +A separate change updates dependencies on `golang.org/x/net` to use `0.12.0`. +``` diff --git a/.changelog/18223.txt b/.changelog/18223.txt new file mode 100644 index 000000000000..067ca64f48e8 --- /dev/null +++ b/.changelog/18223.txt @@ -0,0 +1,3 @@ +```release-note:feature +cli: `consul members` command uses `-filter` expression to filter members based on bexpr. +``` diff --git a/.changelog/18291.txt b/.changelog/18291.txt new file mode 100644 index 000000000000..bb0ec6f89295 --- /dev/null +++ b/.changelog/18291.txt @@ -0,0 +1,3 @@ +```release-note:bug +api-gateway: fix race condition in proxy config generation when Consul is notified of the bound-api-gateway config entry before it is notified of the api-gateway config entry. +``` diff --git a/.changelog/18303.txt b/.changelog/18303.txt new file mode 100644 index 000000000000..4afc4473b7c9 --- /dev/null +++ b/.changelog/18303.txt @@ -0,0 +1,3 @@ +```release-note:improvement +connect: update supported envoy versions to 1.23.12, 1.24.10, 1.25.9, 1.26.4 +``` diff --git a/.changelog/18319.txt b/.changelog/18319.txt new file mode 100644 index 000000000000..bb9c8cdf2c72 --- /dev/null +++ b/.changelog/18319.txt @@ -0,0 +1,6 @@ +```release-note:improvement +acl: added builtin ACL policy that provides global read-only access (builtin/global-read-only) +``` +```release-note:improvement +acl: allow for a single slash character in policy names +``` diff --git a/.changelog/18325.txt b/.changelog/18325.txt new file mode 100644 index 000000000000..b2870352b6c0 --- /dev/null +++ b/.changelog/18325.txt @@ -0,0 +1,3 @@ +```release-note:bug +mesh: **(Enterprise Only)** Require that `jwt-provider` config entries are created in the `default` namespace. +``` \ No newline at end of file diff --git a/.changelog/_5517.txt b/.changelog/_5517.txt new file mode 100644 index 000000000000..5152a6ff78f7 --- /dev/null +++ b/.changelog/_5517.txt @@ -0,0 +1,3 @@ +```release-note:bug +namespaces: **(Enterprise only)** fixes a bug where agent health checks stop syncing for all services on a node if the namespace of any service has been removed from the server. +``` diff --git a/.changelog/_5614.txt b/.changelog/_5614.txt new file mode 100644 index 000000000000..9951b9111875 --- /dev/null +++ b/.changelog/_5614.txt @@ -0,0 +1,4 @@ +```release-note:bug +namespaces: **(Enterprise only)** fixes a bug where namespaces are stuck in a deferred deletion state indefinitely under some conditions. +Also fixes the Consul query metadata present in the HTTP headers of the namespace read and list endpoints. +``` diff --git a/.changelog/_5669.txt b/.changelog/_5669.txt new file mode 100644 index 000000000000..6528460e69bd --- /dev/null +++ b/.changelog/_5669.txt @@ -0,0 +1,3 @@ +```release-note:improvement +audit-logging: **(Enterprise only)** enable error response and request body logging +``` \ No newline at end of file diff --git a/.changelog/_5740.txt b/.changelog/_5740.txt new file mode 100644 index 000000000000..4f1d6f6448f3 --- /dev/null +++ b/.changelog/_5740.txt @@ -0,0 +1,3 @@ +```release-note:feature +api: (Enterprise only) Add `POST /v1/operator/audit-hash` endpoint to calculate the hash of the data used by the audit log hash function and salt. +``` \ No newline at end of file diff --git a/.changelog/_5750.txt b/.changelog/_5750.txt new file mode 100644 index 000000000000..027753c72156 --- /dev/null +++ b/.changelog/_5750.txt @@ -0,0 +1,3 @@ +```release-note:feature +cli: (Enterprise only) Add a new `consul operator audit hash` command to retrieve and compare the hash of the data used by the audit log hash function and salt. +``` \ No newline at end of file diff --git a/.changelog/_5805.txt b/.changelog/_5805.txt new file mode 100644 index 000000000000..786effc31025 --- /dev/null +++ b/.changelog/_5805.txt @@ -0,0 +1,3 @@ +```release-note:security +audit-logging: **(Enterprise only)** limit `v1/operator/audit-hash` endpoint to ACL token with `operator:read` privileges. +``` \ No newline at end of file diff --git a/.github/workflows/backport-assistant.yml b/.github/workflows/backport-assistant.yml index fffa774d32a5..17462f811261 100644 --- a/.github/workflows/backport-assistant.yml +++ b/.github/workflows/backport-assistant.yml @@ -40,4 +40,4 @@ jobs: curl -s -H "Authorization: token ${{ secrets.PR_COMMENT_TOKEN }}" \ -X POST \ -d "{ \"body\": \"${github_message}\"}" \ - "https://api.github.com/repos/${GITHUB_REPOSITORY}/pull/${{ github.event.pull_request.number }}/comments" + "https://api.github.com/repos/${GITHUB_REPOSITORY}/issues/${{ github.event.pull_request.number }}/comments" diff --git a/.github/workflows/backport-reminder.yml b/.github/workflows/backport-reminder.yml deleted file mode 100644 index b358122dbf0c..000000000000 --- a/.github/workflows/backport-reminder.yml +++ /dev/null @@ -1,29 +0,0 @@ -# Copyright (c) HashiCorp, Inc. -# SPDX-License-Identifier: MPL-2.0 - -# This workflow sends a reminder comment to PRs that have labels starting with -# `backport/` to check that the backport has run successfully. - -name: Backport Assistant Reminder - -on: - pull_request: - types: [ labeled ] - # Runs on PRs to main and all release branches - branches: - - main - - release/* - -jobs: - backport-label-check: - if: "startsWith(github.event.label.name, 'backport/')" - runs-on: ubuntu-latest - - steps: - - name: Comment on PR - run: | - github_message="After merging, confirm that you see linked PRs AND check them for CI errors." - curl -s -H "Authorization: token ${{ secrets.PR_COMMENT_TOKEN }}" \ - -X POST \ - -d "{ \"body\": \"${github_message}\"}" \ - "https://api.github.com/repos/${GITHUB_REPOSITORY}/issues/${{ github.event.pull_request.number }}/comments" diff --git a/.github/workflows/build-distros.yml b/.github/workflows/build-distros.yml index 6f5722a82ab9..10c520893341 100644 --- a/.github/workflows/build-distros.yml +++ b/.github/workflows/build-distros.yml @@ -2,7 +2,7 @@ # It is aimed at checking new commits don't introduce any breaking build changes. name: build-distros -on: +on: pull_request: push: branches: @@ -15,6 +15,7 @@ permissions: env: GOTAGS: ${{ endsWith(github.repository, '-enterprise') && 'consulent' || '' }} + GOPRIVATE: github.com/hashicorp # Required for enterprise deps jobs: setup: @@ -26,13 +27,13 @@ jobs: compute-large: ${{ steps.setup-outputs.outputs.compute-large }} compute-xl: ${{ steps.setup-outputs.outputs.compute-xl }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - id: setup-outputs name: Setup outputs run: ./.github/scripts/get_runner_classes.sh check-go-mod: - needs: + needs: - setup uses: ./.github/workflows/reusable-check-go-mod.yml with: @@ -42,21 +43,21 @@ jobs: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} build-386: - needs: - - setup + needs: + - setup - check-go-mod env: XC_OS: "freebsd linux windows" runs-on: ${{ fromJSON(needs.setup.outputs.compute-xl) }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git if: ${{ endsWith(github.repository, '-enterprise') }} run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" - - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - name: Build @@ -67,20 +68,20 @@ jobs: build-amd64: needs: - - setup + - setup - check-go-mod env: XC_OS: "darwin freebsd linux solaris windows" runs-on: ${{ fromJSON(needs.setup.outputs.compute-xl) }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git if: ${{ endsWith(github.repository, '-enterprise') }} run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" - - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - name: Build @@ -91,22 +92,22 @@ jobs: build-arm: needs: - - setup + - setup - check-go-mod runs-on: ${{ fromJSON(needs.setup.outputs.compute-xl) }} env: CGO_ENABLED: 1 GOOS: linux steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git if: ${{ endsWith(github.repository, '-enterprise') }} run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" - - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - run: | @@ -116,6 +117,26 @@ jobs: - run: CC=arm-linux-gnueabihf-gcc GOARCH=arm GOARM=6 go build -tags "${{ env.GOTAGS }}" - run: CC=aarch64-linux-gnu-gcc GOARCH=arm64 go build -tags "${{ env.GOTAGS }}" + + build-s390x: + if: ${{ endsWith(github.repository, '-enterprise') }} + needs: + - setup + - check-go-mod + runs-on: ${{ fromJSON(needs.setup.outputs.compute-xl) }} + steps: + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 + + # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. + - name: Setup Git + run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" + + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 + with: + go-version-file: 'go.mod' + - name: Build + run: GOOS=linux GOARCH=s390x CGO_ENABLED=0 go build -tags "${{ env.GOTAGS }}" + # This is job is required for branch protection as a required gihub check # because GitHub actions show up as checks at the job level and not the # workflow level. This is currently a feature request: @@ -125,18 +146,18 @@ jobs: # - be placed after the fanout of a workflow so that everything fans back in # to this job. # - "need" any job that is part of the fan out / fan in - # - implement the if logic because we have conditional jobs - # (go-test-enteprise) that this job needs and this would potentially get - # skipped if a previous job got skipped. So we use the if clause to make + # - implement the if logic because we have conditional jobs + # (go-test-enteprise) that this job needs and this would potentially get + # skipped if a previous job got skipped. So we use the if clause to make # sure it does not get skipped. - build-distros-success: - needs: + needs: - setup - check-go-mod - build-386 - build-amd64 - build-arm + - build-s390x runs-on: ${{ fromJSON(needs.setup.outputs.compute-small) }} if: ${{ always() }} steps: diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 66b5a14a739c..eb5fc674f628 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -14,6 +14,7 @@ on: env: PKG_NAME: consul METADATA: oss + GOPRIVATE: github.com/hashicorp # Required for enterprise deps jobs: set-product-version: @@ -25,7 +26,7 @@ jobs: pre-version: ${{ steps.set-product-version.outputs.prerelease-product-version }} shared-ldflags: ${{ steps.shared-ldflags.outputs.shared-ldflags }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - name: set product version id: set-product-version uses: hashicorp/actions-set-product-version@v1 @@ -56,6 +57,7 @@ jobs: echo "Product Date: ${{ needs.set-product-version.outputs.product-date }}" echo "Prerelease Version: ${{ needs.set-product-version.outputs.pre-version }}" echo "Ldflags: ${{ needs.set-product-version.outputs.shared-ldflags }}" + generate-metadata-file: needs: set-product-version runs-on: ubuntu-latest @@ -63,7 +65,7 @@ jobs: filepath: ${{ steps.generate-metadata-file.outputs.filepath }} steps: - name: 'Checkout directory' - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - name: Generate metadata file id: generate-metadata-file uses: hashicorp/actions-generate-metadata@v1 @@ -71,7 +73,7 @@ jobs: version: ${{ needs.set-product-version.outputs.product-version }} product: ${{ env.PKG_NAME }} - - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # pin@3.1.2 + - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2 with: name: metadata.json path: ${{ steps.generate-metadata-file.outputs.filepath }} @@ -82,23 +84,23 @@ jobs: strategy: matrix: include: - - {go: "1.20.4", goos: "linux", goarch: "386"} - - {go: "1.20.4", goos: "linux", goarch: "amd64"} - - {go: "1.20.4", goos: "linux", goarch: "arm"} - - {go: "1.20.4", goos: "linux", goarch: "arm64"} - - {go: "1.20.4", goos: "freebsd", goarch: "386"} - - {go: "1.20.4", goos: "freebsd", goarch: "amd64"} - - {go: "1.20.4", goos: "windows", goarch: "386"} - - {go: "1.20.4", goos: "windows", goarch: "amd64"} - - {go: "1.20.4", goos: "solaris", goarch: "amd64"} + - {go: "1.20.6", goos: "linux", goarch: "386"} + - {go: "1.20.6", goos: "linux", goarch: "amd64"} + - {go: "1.20.6", goos: "linux", goarch: "arm"} + - {go: "1.20.6", goos: "linux", goarch: "arm64"} + - {go: "1.20.6", goos: "freebsd", goarch: "386"} + - {go: "1.20.6", goos: "freebsd", goarch: "amd64"} + - {go: "1.20.6", goos: "windows", goarch: "386"} + - {go: "1.20.6", goos: "windows", goarch: "amd64"} + - {go: "1.20.6", goos: "solaris", goarch: "amd64"} fail-fast: true name: Go ${{ matrix.go }} ${{ matrix.goos }} ${{ matrix.goarch }} build steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - name: Setup with node and yarn - uses: actions/setup-node@64ed1c7eab4cce3362f8c340dee64e5eaeef8f7c # pin@v3.6.0 + uses: actions/setup-node@64ed1c7eab4cce3362f8c340dee64e5eaeef8f7c # v3.6.0 with: node-version: '14' cache: 'yarn' @@ -160,18 +162,69 @@ jobs: echo "RPM_PACKAGE=$(basename out/*.rpm)" >> $GITHUB_ENV echo "DEB_PACKAGE=$(basename out/*.deb)" >> $GITHUB_ENV - - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # pin@3.1.2 + - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2 if: ${{ matrix.goos == 'linux' }} with: name: ${{ env.RPM_PACKAGE }} path: out/${{ env.RPM_PACKAGE }} - - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # pin@3.1.2 + - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2 if: ${{ matrix.goos == 'linux' }} with: name: ${{ env.DEB_PACKAGE }} path: out/${{ env.DEB_PACKAGE }} + build-s390x: + needs: set-product-version + if: ${{ endsWith(github.repository, '-enterprise') }} + runs-on: ubuntu-latest + strategy: + matrix: + include: + - {go: "1.20.6", goos: "linux", goarch: "s390x"} + fail-fast: true + + name: Go ${{ matrix.go }} ${{ matrix.goos }} ${{ matrix.goarch }} build + steps: + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 + + - name: Setup with node and yarn + uses: actions/setup-node@64ed1c7eab4cce3362f8c340dee64e5eaeef8f7c # v3.6.0 + with: + node-version: '14' + cache: 'yarn' + cache-dependency-path: 'ui/yarn.lock' + + - name: Build UI + run: | + CONSUL_VERSION=${{ needs.set-product-version.outputs.product-version }} + CONSUL_DATE=${{ needs.set-product-version.outputs.product-date }} + CONSUL_BINARY_TYPE=${CONSUL_BINARY_TYPE} + CONSUL_COPYRIGHT_YEAR=$(git show -s --format=%cd --date=format:%Y HEAD) + echo "consul_version is ${CONSUL_VERSION}" + echo "consul_date is ${CONSUL_DATE}" + echo "consul binary type is ${CONSUL_BINARY_TYPE}" + echo "consul copyright year is ${CONSUL_COPYRIGHT_YEAR}" + cd ui && make && cd .. + rm -rf agent/uiserver/dist + mv ui/packages/consul-ui/dist agent/uiserver/ + - name: Go Build + env: + PRODUCT_VERSION: ${{ needs.set-product-version.outputs.product-version }} + PRERELEASE_VERSION: ${{ needs.set-product-version.outputs.pre-version }} + CGO_ENABLED: "0" + GOLDFLAGS: "${{needs.set-product-version.outputs.shared-ldflags}}" + uses: hashicorp/actions-go-build@v0.1.7 + with: + product_name: ${{ env.PKG_NAME }} + product_version: ${{ needs.set-product-version.outputs.product-version }} + go_version: ${{ matrix.go }} + os: ${{ matrix.goos }} + arch: ${{ matrix.goarch }} + reproducible: report + instructions: |- + go build -ldflags="$GOLDFLAGS" -o "$BIN_PATH" -trimpath -buildvcs=false + build-darwin: needs: set-product-version runs-on: macos-latest @@ -179,15 +232,15 @@ jobs: matrix: goos: [ darwin ] goarch: [ "amd64", "arm64" ] - go: [ "1.20.4" ] + go: [ "1.20.6" ] fail-fast: true name: Go ${{ matrix.go }} ${{ matrix.goos }} ${{ matrix.goarch }} build steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 - + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 + - name: Setup with node and yarn - uses: actions/setup-node@64ed1c7eab4cce3362f8c340dee64e5eaeef8f7c # pin@v3.6.0 + uses: actions/setup-node@64ed1c7eab4cce3362f8c340dee64e5eaeef8f7c # v3.6.0 with: node-version: '14' cache: 'yarn' @@ -229,13 +282,17 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - arch: ["386", "amd64", "arm", "arm64"] + include: + - { arch: "386" } + - { arch: "arm" } + - { arch: "amd64" } + - { arch: "arm64" } env: repo: ${{github.event.repository.name}} version: ${{needs.set-product-version.outputs.product-version}} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # Strip everything but MAJOR.MINOR from the version string and add a `-dev` suffix # This naming convention will be used ONLY for per-commit dev images @@ -269,7 +326,7 @@ jobs: version: ${{needs.set-product-version.outputs.product-version}} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - uses: hashicorp/actions-docker-build@v1 with: version: ${{env.version}} @@ -289,7 +346,7 @@ jobs: version: ${{needs.set-product-version.outputs.product-version}} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # Strip everything but MAJOR.MINOR from the version string and add a `-dev` suffix # This naming convention will be used ONLY for per-commit dev images @@ -318,7 +375,11 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - arch: ["386", "amd64", "arm", "arm64"] + include: + - { arch: "386" } + - { arch: "arm" } + - { arch: "amd64" } + - { arch: "arm64" } fail-fast: true env: version: ${{ needs.set-product-version.outputs.product-version }} @@ -326,21 +387,24 @@ jobs: name: Verify ${{ matrix.arch }} linux binary steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 + if: ${{ endsWith(github.repository, '-enterprise') || matrix.arch != 's390x' }} - name: Download ${{ matrix.arch }} zip - uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # pin@v3.0.2 + if: ${{ endsWith(github.repository, '-enterprise') || matrix.arch != 's390x' }} + uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # v3.0.2 with: name: ${{ env.zip_name }} - name: Set up QEMU - uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # pin@v2.1.0 + uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # v2.1.0 if: ${{ matrix.arch == 'arm' || matrix.arch == 'arm64' }} with: # this should be a comma-separated string as opposed to an array platforms: arm,arm64 - name: Run verification for ${{ matrix.arch }} binary + if: ${{ endsWith(github.repository, '-enterprise') || matrix.arch != 's390x' }} run: .github/scripts/verify_artifact.sh ${{ env.zip_name }} v${{ env.version }} verify-darwin: @@ -356,10 +420,10 @@ jobs: name: Verify amd64 darwin binary steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - name: Download amd64 darwin zip - uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # pin@v3.0.2 + uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # v3.0.2 with: name: ${{ env.zip_name }} @@ -376,14 +440,18 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - arch: ["i386", "amd64", "armhf", "arm64"] + include: + - { arch: "i386" } + - { arch: "armhf" } + - { arch: "amd64" } + - { arch: "arm64" } # fail-fast: true env: version: ${{ needs.set-product-version.outputs.product-version }} name: Verify ${{ matrix.arch }} debian package steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - name: Set package version run: | @@ -394,12 +462,12 @@ jobs: echo "pkg_name=consul_${{ env.pkg_version }}-1_${{ matrix.arch }}.deb" >> $GITHUB_ENV - name: Download workflow artifacts - uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # pin@v3.0.2 + uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # v3.0.2 with: name: ${{ env.pkg_name }} - name: Set up QEMU - uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # pin@v2.1.0 + uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # v2.1.0 with: platforms: all @@ -413,14 +481,18 @@ jobs: runs-on: ubuntu-latest strategy: matrix: + include: + - { arch: "i386" } + - { arch: "x86_64" } # TODO(eculver): re-enable when there is a smaller verification container available - arch: ["i386", "x86_64"] #, "armv7hl", "aarch64"] + # - { arch: "armv7hl" } + # - { arch: "aarch64" } env: version: ${{ needs.set-product-version.outputs.product-version }} name: Verify ${{ matrix.arch }} rpm steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - name: Set package version run: | @@ -431,12 +503,12 @@ jobs: echo "pkg_name=consul-${{ env.pkg_version }}-1.${{ matrix.arch }}.rpm" >> $GITHUB_ENV - name: Download workflow artifacts - uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # pin@v3.0.2 + uses: actions/download-artifact@9bc31d5ccc31df68ecc42ccf4149144866c47d8a # v3.0.2 with: name: ${{ env.pkg_name }} - name: Set up QEMU - uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # pin@v2.1.0 + uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # v2.1.0 with: platforms: all diff --git a/.github/workflows/check-legacy-links-format.yml b/.github/workflows/check-legacy-links-format.yml deleted file mode 100644 index 85dbb0e70f2e..000000000000 --- a/.github/workflows/check-legacy-links-format.yml +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright (c) HashiCorp, Inc. -# SPDX-License-Identifier: MPL-2.0 - -name: Legacy Link Format Checker - -on: - push: - paths: - - "website/content/**/*.mdx" - - "website/data/*-nav-data.json" - -jobs: - check-links: - uses: hashicorp/dev-portal/.github/workflows/docs-content-check-legacy-links-format.yml@475289345d312552b745224b46895f51cc5fc490 - with: - repo-owner: "hashicorp" - repo-name: "consul" - commit-sha: ${{ github.sha }} - mdx-directory: "website/content" - nav-data-directory: "website/data" diff --git a/.github/workflows/go-tests.yml b/.github/workflows/go-tests.yml index 787f92560ec1..58ef29ba0957 100644 --- a/.github/workflows/go-tests.yml +++ b/.github/workflows/go-tests.yml @@ -1,6 +1,6 @@ name: go-tests -on: +on: pull_request: branches-ignore: - stable-website @@ -15,12 +15,13 @@ on: # Push events on the main branch - main - release/** - + permissions: contents: read env: TEST_RESULTS: /tmp/test-results + GOPRIVATE: github.com/hashicorp # Required for enterprise deps jobs: setup: @@ -32,13 +33,13 @@ jobs: compute-large: ${{ steps.setup-outputs.outputs.compute-large }} compute-xl: ${{ steps.setup-outputs.outputs.compute-xl }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 - id: setup-outputs name: Setup outputs run: ./.github/scripts/get_runner_classes.sh check-go-mod: - needs: + needs: - setup uses: ./.github/workflows/reusable-check-go-mod.yml with: @@ -48,16 +49,16 @@ jobs: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} check-generated-protobuf: - needs: - - setup - runs-on: ${{ fromJSON(needs.setup.outputs.compute-small) }} + needs: + - setup + runs-on: ${{ fromJSON(needs.setup.outputs.compute-medium) }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git if: ${{ endsWith(github.repository, '-enterprise') }} run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - run: make proto-tools @@ -74,18 +75,18 @@ jobs: name: "Protobuf Lint" - name: Notify Slack if: ${{ failure() }} - run: .github/scripts/notify_slack.sh + run: .github/scripts/notify_slack.sh check-generated-deep-copy: - needs: - - setup + needs: + - setup runs-on: ${{ fromJSON(needs.setup.outputs.compute-large) }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git if: ${{ endsWith(github.repository, '-enterprise') }} run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - run: make --always-make deep-copy @@ -99,19 +100,19 @@ jobs: run: .github/scripts/notify_slack.sh lint-enums: - needs: - - setup + needs: + - setup runs-on: ${{ fromJSON(needs.setup.outputs.compute-large) }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git if: ${{ endsWith(github.repository, '-enterprise') }} run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - - run: go install github.com/reillywatson/enumcover/cmd/enumcover@master && enumcover ./... + - run: go install github.com/reillywatson/enumcover/cmd/enumcover@master && enumcover ./... - name: Notify Slack if: ${{ failure() }} run: .github/scripts/notify_slack.sh @@ -121,11 +122,11 @@ jobs: - setup runs-on: ${{ fromJSON(needs.setup.outputs.compute-small) }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - run: make lint-container-test-deps @@ -134,16 +135,16 @@ jobs: run: .github/scripts/notify_slack.sh lint-consul-retry: - needs: - - setup + needs: + - setup runs-on: ${{ fromJSON(needs.setup.outputs.compute-small) }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git if: ${{ endsWith(github.repository, '-enterprise') }} run: git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}:@github.com".insteadOf "https://github.com" - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - run: go install github.com/hashicorp/lint-consul-retry@master && lint-consul-retry @@ -152,8 +153,8 @@ jobs: run: .github/scripts/notify_slack.sh lint: - needs: - - setup + needs: + - setup uses: ./.github/workflows/reusable-lint.yml with: runs-on: ${{ needs.setup.outputs.compute-xl }} @@ -162,8 +163,8 @@ jobs: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} lint-32bit: - needs: - - setup + needs: + - setup uses: ./.github/workflows/reusable-lint.yml with: go-arch: "386" @@ -174,20 +175,33 @@ jobs: # create a development build dev-build: - needs: - - setup + needs: + - setup uses: ./.github/workflows/reusable-dev-build.yml - with: + with: runs-on: ${{ needs.setup.outputs.compute-xl }} repository-name: ${{ github.repository }} secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} + # dev-build-s390x: + # if: ${{ endsWith(github.repository, '-enterprise') }} + # needs: + # - setup + # uses: ./.github/workflows/reusable-dev-build.yml + # with: + # uploaded-binary-name: 'consul-bin-s390x' + # runs-on: ${{ needs.setup.outputs.compute-xl }} + # go-arch: "s390x" + # repository-name: ${{ github.repository }} + # secrets: + # elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} + # dev-build-arm64: # # only run on enterprise because GHA does not have arm64 runners in OSS # if: ${{ endsWith(github.repository, '-enterprise') }} - # needs: - # - setup + # needs: + # - setup # uses: ./.github/workflows/reusable-dev-build.yml # with: # uploaded-binary-name: 'consul-bin-arm64' @@ -200,8 +214,8 @@ jobs: # go-test-arm64: # # only run on enterprise because GHA does not have arm64 runners in OSS # if: ${{ endsWith(github.repository, '-enterprise') }} - # needs: - # - setup + # needs: + # - setup # - dev-build-arm64 # uses: ./.github/workflows/reusable-unit-split.yml # with: @@ -217,8 +231,8 @@ jobs: # datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" go-test-oss: - needs: - - setup + needs: + - setup - dev-build uses: ./.github/workflows/reusable-unit-split.yml with: @@ -229,7 +243,7 @@ jobs: go-tags: "" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} @@ -237,8 +251,8 @@ jobs: go-test-enterprise: if: ${{ endsWith(github.repository, '-enterprise') }} - needs: - - setup + needs: + - setup - dev-build uses: ./.github/workflows/reusable-unit-split.yml with: @@ -249,15 +263,15 @@ jobs: go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" go-test-race: - needs: - - setup + needs: + - setup - dev-build uses: ./.github/workflows/reusable-unit.yml with: @@ -269,15 +283,15 @@ jobs: go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" go-test-32bit: - needs: - - setup + needs: + - setup - dev-build uses: ./.github/workflows/reusable-unit.yml with: @@ -289,12 +303,33 @@ jobs: go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" + # go-test-s390x: + # if: ${{ endsWith(github.repository, '-enterprise') }} + # needs: + # - setup + # - dev-build-s390x + # uses: ./.github/workflows/reusable-unit.yml + # with: + # uploaded-binary-name: 'consul-bin-s390x' + # directory: . + # go-test-flags: 'export GO_TEST_FLAGS="-short"' + # runs-on: ${{ needs.setup.outputs.compute-xl }} + # repository-name: ${{ github.repository }} + # go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" + # permissions: + # id-token: write # NOTE: this permission is explicitly required for Vault auth. + # contents: read + # secrets: + # elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} + # consul-license: ${{secrets.CONSUL_LICENSE}} + # datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" + go-test-envoyextensions: needs: - setup @@ -307,7 +342,7 @@ jobs: go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} @@ -325,15 +360,15 @@ jobs: go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" go-test-api-1-19: - needs: - - setup + needs: + - setup - dev-build uses: ./.github/workflows/reusable-unit.yml with: @@ -341,17 +376,18 @@ jobs: runs-on: ${{ needs.setup.outputs.compute-xl }} repository-name: ${{ github.repository }} go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" + go-version: "1.19" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" go-test-api-1-20: - needs: - - setup + needs: + - setup - dev-build uses: ./.github/workflows/reusable-unit.yml with: @@ -359,17 +395,18 @@ jobs: runs-on: ${{ needs.setup.outputs.compute-xl }} repository-name: ${{ github.repository }} go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" + go-version: "1.20" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" go-test-sdk-1-19: - needs: - - setup + needs: + - setup - dev-build uses: ./.github/workflows/reusable-unit.yml with: @@ -377,17 +414,18 @@ jobs: runs-on: ${{ needs.setup.outputs.compute-xl }} repository-name: ${{ github.repository }} go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" + go-version: "1.19" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} datadog-api-key: "${{ !endsWith(github.repository, '-enterprise') && secrets.DATADOG_API_KEY || '' }}" go-test-sdk-1-20: - needs: - - setup + needs: + - setup - dev-build uses: ./.github/workflows/reusable-unit.yml with: @@ -395,9 +433,10 @@ jobs: runs-on: ${{ needs.setup.outputs.compute-xl }} repository-name: ${{ github.repository }} go-tags: "${{ github.event.repository.name == 'consul-enterprise' && 'consulent consulprem consuldev' || '' }}" + go-version: "1.20" permissions: id-token: write # NOTE: this permission is explicitly required for Vault auth. - contents: read + contents: read secrets: elevated-github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} consul-license: ${{secrets.CONSUL_LICENSE}} @@ -417,13 +456,13 @@ jobs: # - be placed after the fanout of a workflow so that everything fans back in # to this job. # - "need" any job that is part of the fan out / fan in - # - implement the if logic because we have conditional jobs - # (go-test-enteprise) that this job needs and this would potentially get - # skipped if a previous job got skipped. So we use the if clause to make + # - implement the if logic because we have conditional jobs + # (go-test-enteprise) that this job needs and this would potentially get + # skipped if a previous job got skipped. So we use the if clause to make # sure it does not get skipped. go-tests-success: - needs: + needs: - setup - check-generated-deep-copy - check-generated-protobuf @@ -435,7 +474,7 @@ jobs: - lint-32bit # - go-test-arm64 - go-test-enterprise - - go-test-oss + - go-test-oss - go-test-race - go-test-envoyextensions - go-test-troubleshoot @@ -444,6 +483,7 @@ jobs: - go-test-sdk-1-19 - go-test-sdk-1-20 - go-test-32bit + # - go-test-s390x runs-on: ${{ fromJSON(needs.setup.outputs.compute-small) }} if: ${{ always() }} steps: diff --git a/.github/workflows/reusable-dev-build.yml b/.github/workflows/reusable-dev-build.yml index d134508584c0..2db9670655e4 100644 --- a/.github/workflows/reusable-dev-build.yml +++ b/.github/workflows/reusable-dev-build.yml @@ -1,4 +1,4 @@ -name: reusable-dev-build +name: reusable-dev-build on: workflow_call: @@ -25,12 +25,12 @@ jobs: build: runs-on: ${{ fromJSON(inputs.runs-on) }} steps: - - uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # pin@v3.3.0 + - uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2 # NOTE: This step is specifically needed for ENT. It allows us to access the required private HashiCorp repos. - name: Setup Git if: ${{ endsWith(inputs.repository-name, '-enterprise') }} run: git config --global url."https://${{ secrets.elevated-github-token }}:@github.com".insteadOf "https://github.com" - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 with: go-version-file: 'go.mod' - name: Build @@ -38,7 +38,7 @@ jobs: GOARCH: ${{ inputs.goarch }} run: make dev # save dev build to pass to downstream jobs - - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # pin@v3.1.2 + - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2 with: name: ${{inputs.uploaded-binary-name}} path: ./bin/consul diff --git a/.github/workflows/reusable-unit-split.yml b/.github/workflows/reusable-unit-split.yml index 3b9eb489590a..6c13670e742e 100644 --- a/.github/workflows/reusable-unit-split.yml +++ b/.github/workflows/reusable-unit-split.yml @@ -46,7 +46,7 @@ on: required: true env: TEST_RESULTS: /tmp/test-results - GOTESTSUM_VERSION: 1.8.2 + GOTESTSUM_VERSION: "1.10.1" GOARCH: ${{inputs.go-arch}} TOTAL_RUNNERS: ${{inputs.runner-count}} CONSUL_LICENSE: ${{secrets.consul-license}} diff --git a/.github/workflows/reusable-unit.yml b/.github/workflows/reusable-unit.yml index e7caaae8d6a1..c49a6291fa2e 100644 --- a/.github/workflows/reusable-unit.yml +++ b/.github/workflows/reusable-unit.yml @@ -33,6 +33,10 @@ on: required: false type: string default: "" + go-version: + required: false + type: string + default: "" secrets: elevated-github-token: required: true @@ -42,7 +46,7 @@ on: required: true env: TEST_RESULTS: /tmp/test-results - GOTESTSUM_VERSION: 1.8.2 + GOTESTSUM_VERSION: "1.10.1" GOARCH: ${{inputs.go-arch}} CONSUL_LICENSE: ${{secrets.consul-license}} GOTAGS: ${{ inputs.go-tags}} @@ -57,7 +61,13 @@ jobs: - name: Setup Git if: ${{ endsWith(inputs.repository-name, '-enterprise') }} run: git config --global url."https://${{ secrets.elevated-github-token }}:@github.com".insteadOf "https://github.com" - - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # pin@v3.5.0 + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 + if: ${{ inputs.go-version != '' }} + with: + go-version: ${{ inputs.go-version }} + cache: true + - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 + if: ${{ inputs.go-version == '' }} with: go-version-file: 'go.mod' cache: true diff --git a/.github/workflows/test-integrations.yml b/.github/workflows/test-integrations.yml index 260044aee8bb..867925453f7e 100644 --- a/.github/workflows/test-integrations.yml +++ b/.github/workflows/test-integrations.yml @@ -19,10 +19,11 @@ env: TEST_RESULTS_ARTIFACT_NAME: test-results CONSUL_LICENSE: ${{ secrets.CONSUL_LICENSE }} GOTAGS: ${{ endsWith(github.repository, '-enterprise') && 'consulent' || '' }} - GOTESTSUM_VERSION: "1.9.0" + GOTESTSUM_VERSION: "1.10.1" CONSUL_BINARY_UPLOAD_NAME: consul-bin # strip the hashicorp/ off the front of github.repository for consul - CONSUL_LATEST_IMAGE_NAME: ${{ endsWith(github.repository, '-enterprise') && github.repository || 'consul' }} + CONSUL_LATEST_IMAGE_NAME: ${{ endsWith(github.repository, '-enterprise') && github.repository || 'hashicorp/consul' }} + GOPRIVATE: github.com/hashicorp # Required for enterprise deps jobs: setup: @@ -239,7 +240,7 @@ jobs: # this is further going to multiplied in envoy-integration tests by the # other dimensions in the matrix. Currently TOTAL_RUNNERS would be # multiplied by 8 based on these values: - # envoy-version: ["1.23.8", "1.24.6", "1.25.4", "1.26.0"] + # envoy-version: ["1.23.12", "1.24.10", "1.25.9", "1.26.4"] # xds-target: ["server", "client"] TOTAL_RUNNERS: 4 JQ_SLICER: '[ inputs ] | [_nwise(length / $runnercount | floor)]' @@ -273,7 +274,7 @@ jobs: strategy: fail-fast: false matrix: - envoy-version: ["1.23.8", "1.24.6", "1.25.4", "1.26.0"] + envoy-version: ["1.23.12", "1.24.10", "1.25.9", "1.26.4"] xds-target: ["server", "client"] test-cases: ${{ fromJSON(needs.generate-envoy-job-matrices.outputs.envoy-matrix) }} env: @@ -367,7 +368,11 @@ jobs: - uses: actions/checkout@24cb9080177205b6e8c946b17badbe402adc938f # v3.4.0 - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # v3.5.0 with: - go-version-file: 'go.mod' + # pinning this to 1.20.5 because this issue in go-testcontainers occurs + # in 1.20.6 with the error "http: invalid Host header, host port waiting failed" + # https://github.com/testcontainers/testcontainers-go/issues/1359 + # go-version-file: 'go.mod' + go-version: '1.20.5' - run: go env - name: docker env run: | @@ -470,7 +475,7 @@ jobs: strategy: fail-fast: false matrix: - consul-version: [ "1.14", "1.15"] + consul-version: ["1.14", "1.15", "1.16"] env: CONSUL_LATEST_VERSION: ${{ matrix.consul-version }} ENVOY_VERSION: "1.24.6" @@ -478,7 +483,11 @@ jobs: - uses: actions/checkout@24cb9080177205b6e8c946b17badbe402adc938f # v3.4.0 - uses: actions/setup-go@6edd4406fa81c3da01a34fa6f6343087c207a568 # v3.5.0 with: - go-version-file: 'go.mod' + # pinning this to 1.20.5 because this issue in go-testcontainers occurs + # in 1.20.6 with the error "http: invalid Host header, host port waiting failed" + # https://github.com/testcontainers/testcontainers-go/issues/1359 + # go-version-file: 'go.mod' + go-version: '1.20.5' - run: go env # Get go binary from workspace diff --git a/CHANGELOG.md b/CHANGELOG.md index ee1dccacc139..ff307a926983 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,143 @@ +## 1.16.0 (June 26, 2023) + +BREAKING CHANGES: + +* api: The `/v1/health/connect/` and `/v1/health/ingress/` endpoints now immediately return 403 "Permission Denied" errors whenever a token with insufficient `service:read` permissions is provided. Prior to this change, the endpoints returned a success code with an empty result list when a token with insufficient permissions was provided. [[GH-17424](https://github.com/hashicorp/consul/issues/17424)] +* peering: Removed deprecated backward-compatibility behavior. + Upstream overrides in service-defaults will now only apply to peer upstreams when the `peer` field is provided. + Visit the 1.16.x [upgrade instructions](https://developer.hashicorp.com/consul/docs/upgrading/upgrade-specific) for more information. [[GH-16957](https://github.com/hashicorp/consul/issues/16957)] + +SECURITY: + +* Bump Dockerfile base image to `alpine:3.18`. [[GH-17719](https://github.com/hashicorp/consul/issues/17719)] +* audit-logging: **(Enterprise only)** limit `v1/operator/audit-hash` endpoint to ACL token with `operator:read` privileges. + +FEATURES: + +* api: (Enterprise only) Add `POST /v1/operator/audit-hash` endpoint to calculate the hash of the data used by the audit log hash function and salt. +* cli: (Enterprise only) Add a new `consul operator audit hash` command to retrieve and compare the hash of the data used by the audit log hash function and salt. +* cli: Adds new command - `consul services export` - for exporting a service to a peer or partition [[GH-15654](https://github.com/hashicorp/consul/issues/15654)] +* connect: **(Consul Enterprise only)** Implement order-by-locality failover. +* mesh: Add new permissive mTLS mode that allows sidecar proxies to forward incoming traffic unmodified to the application. This adds `AllowEnablingPermissiveMutualTLS` setting to the mesh config entry and the `MutualTLSMode` setting to proxy-defaults and service-defaults. [[GH-17035](https://github.com/hashicorp/consul/issues/17035)] +* mesh: Support configuring JWT authentication in Envoy. [[GH-17452](https://github.com/hashicorp/consul/issues/17452)] +* server: **(Enterprise Only)** added server side RPC requests IP based read/write rate-limiter. [[GH-4633](https://github.com/hashicorp/consul/issues/4633)] +* server: **(Enterprise Only)** allow automatic license utilization reporting. [[GH-5102](https://github.com/hashicorp/consul/issues/5102)] +* server: added server side RPC requests global read/write rate-limiter. [[GH-16292](https://github.com/hashicorp/consul/issues/16292)] +* xds: Add `property-override` built-in Envoy extension that directly patches Envoy resources. [[GH-17487](https://github.com/hashicorp/consul/issues/17487)] +* xds: Add a built-in Envoy extension that inserts External Authorization (ext_authz) network and HTTP filters. [[GH-17495](https://github.com/hashicorp/consul/issues/17495)] +* xds: Add a built-in Envoy extension that inserts Wasm HTTP filters. [[GH-16877](https://github.com/hashicorp/consul/issues/16877)] +* xds: Add a built-in Envoy extension that inserts Wasm network filters. [[GH-17505](https://github.com/hashicorp/consul/issues/17505)] + +IMPROVEMENTS: + +* * api: Support filtering for config entries. [[GH-17183](https://github.com/hashicorp/consul/issues/17183)] +* * cli: Add `-filter` option to `consul config list` for filtering config entries. [[GH-17183](https://github.com/hashicorp/consul/issues/17183)] +* agent: remove agent cache dependency from service mesh leaf certificate management [[GH-17075](https://github.com/hashicorp/consul/issues/17075)] +* api: Enable setting query options on agent force-leave endpoint. [[GH-15987](https://github.com/hashicorp/consul/issues/15987)] +* audit-logging: **(Enterprise only)** enable error response and request body logging +* ca: automatically set up Vault's auto-tidy setting for tidy_expired_issuers when using Vault as a CA provider. [[GH-17138](https://github.com/hashicorp/consul/issues/17138)] +* ca: support Vault agent auto-auth config for Vault CA provider using AliCloud authentication. [[GH-16224](https://github.com/hashicorp/consul/issues/16224)] +* ca: support Vault agent auto-auth config for Vault CA provider using AppRole authentication. [[GH-16259](https://github.com/hashicorp/consul/issues/16259)] +* ca: support Vault agent auto-auth config for Vault CA provider using Azure MSI authentication. [[GH-16298](https://github.com/hashicorp/consul/issues/16298)] +* ca: support Vault agent auto-auth config for Vault CA provider using JWT authentication. [[GH-16266](https://github.com/hashicorp/consul/issues/16266)] +* ca: support Vault agent auto-auth config for Vault CA provider using Kubernetes authentication. [[GH-16262](https://github.com/hashicorp/consul/issues/16262)] +* command: Adds ACL enabled to status output on agent startup. [[GH-17086](https://github.com/hashicorp/consul/issues/17086)] +* command: Allow creating ACL Token TTL with greater than 24 hours with the -expires-ttl flag. [[GH-17066](https://github.com/hashicorp/consul/issues/17066)] +* connect: **(Enterprise Only)** Add support for specifying "Partition" and "Namespace" in Prepared Queries failover rules. +* connect: update supported envoy versions to 1.23.10, 1.24.8, 1.25.7, 1.26.2 [[GH-17546](https://github.com/hashicorp/consul/issues/17546)] +* connect: update supported envoy versions to 1.23.8, 1.24.6, 1.25.4, 1.26.0 [[GH-5200](https://github.com/hashicorp/consul/issues/5200)] +* fix metric names in /docs/agent/telemetry [[GH-17577](https://github.com/hashicorp/consul/issues/17577)] +* gateway: Change status condition reason for invalid certificate on a listener from "Accepted" to "ResolvedRefs". [[GH-17115](https://github.com/hashicorp/consul/issues/17115)] +* http: accept query parameters `datacenter`, `ap` (enterprise-only), and `namespace` (enterprise-only). Both short-hand and long-hand forms of these query params are now supported via the HTTP API (dc/datacenter, ap/partition, ns/namespace). [[GH-17525](https://github.com/hashicorp/consul/issues/17525)] +* systemd: set service type to notify. [[GH-16845](https://github.com/hashicorp/consul/issues/16845)] +* ui: Update alerts to Hds::Alert component [[GH-16412](https://github.com/hashicorp/consul/issues/16412)] +* ui: Update to use Hds::Toast component to show notifications [[GH-16519](https://github.com/hashicorp/consul/issues/16519)] +* ui: update from