Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor TeamCity Services Diff Check to run as one job #10158

Merged
merged 19 commits into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions .github/actions/build-downstream/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
name: build
description: Generates Google/Google-Beta Provider
BBBmau marked this conversation as resolved.
Show resolved Hide resolved
inputs:
repo:
description: "provider repo"
required: true
token:
description: "github token"
required: true

runs:
using: "composite"
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Ruby
uses: ruby/setup-ruby@036ef458ddccddb148a2b9fb67e95a22fdbf728b # v1.160.0
with:
ruby-version: '3.1'

- name: Cache Bundler gems
uses: actions/cache@v3
with:
path: mmv1/vendor/bundle
key: ${{ runner.os }}-gems-${{ hashFiles('mmv1/**/Gemfile.lock') }}
restore-keys: |
${{ runner.os }}-gems-

- name: Install Ruby dependencies
shell: bash
run: |
bundle config path mmv1/vendor/bundle
bundle install
working-directory: mmv1

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '^1.20'

# Cache Go modules
- name: Cache Go modules
uses: actions/cache@v3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-

- run: go install golang.org/x/tools/cmd/goimports@latest
shell: bash
- name: Build ${{ inputs.repo }}
shell: bash
env:
BASE_BRANCH: ${{ github.event.pull_request.base.ref || github.ref_name }}
GITHUB_TOKEN: ${{ inputs.token }}
run: |
set -e
set -x
# Set GOPATH to a directory the runner user has access to
export GOPATH=~/go

function clone_repo() {
export OUTPUT_PATH=$GOPATH/src/github.com/$UPSTREAM_OWNER/$GH_REPO
GITHUB_PATH=https://x-access-token:[email protected]/$UPSTREAM_OWNER/$GH_REPO
mkdir -p "$(dirname $OUTPUT_PATH)"
git clone $GITHUB_PATH $OUTPUT_PATH --branch $BASE_BRANCH
}

GH_REPO="${{ inputs.repo }}"
if [ "$GH_REPO" == "docs-examples" ] && [ "$BASE_BRANCH" == "main" ]; then
BASE_BRANCH="master"
fi

GITHUB_PATH=https://x-access-token:[email protected]/$UPSTREAM_OWNER/$GH_REPO

if [[ "$GH_REPO" == terraform-provider-google* ]]; then
UPSTREAM_OWNER=hashicorp
clone_repo
if [ "$GH_REPO" == "terraform-provider-google" ]; then
export VERSION=ga
else
export VERSION=beta
fi
make clean-provider
make provider
elif [ "$GH_REPO" == "terraform-google-conversion" ]; then
UPSTREAM_OWNER=GoogleCloudPlatform
clone_repo
make clean-tgc
make tgc
elif [ "$GH_REPO" == "docs-examples" ]; then
UPSTREAM_OWNER=terraform-google-modules
clone_repo
make tf-oics
else
echo "case not supported"
exit 1
fi

(current_dir=$(pwd) && cd $OUTPUT_PATH && zip -r "$current_dir/output.zip" .)

- name: Upload built artifacts
uses: actions/upload-artifact@3cea5372237819ed00197afe530f5a7ea3e805c8 # v3.1.0
with:
name: artifact-${{ inputs.repo }}
path: output.zip
BBBmau marked this conversation as resolved.
Show resolved Hide resolved
109 changes: 56 additions & 53 deletions .github/workflows/teamcity-services-diff-check-weekly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,57 +11,60 @@ on:
- cron: '0 4 * * 1-2'

jobs:
terraform-provider-google:
uses: ./.github/workflows/build-downstream.yml
with:
repo: 'terraform-provider-google'

terraform-provider-google-beta:
uses: ./.github/workflows/build-downstream.yml
with:
repo: 'terraform-provider-google-beta'

teamcity-services-diff-check:
needs: [terraform-provider-google, terraform-provider-google-beta]
runs-on: ubuntu-22.04
steps:
- name: Checkout Repository
uses: actions/checkout@v4

- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: '^1.20'

- name: Download built artifacts - GA provider
uses: actions/download-artifact@v2
with:
name: artifact-terraform-provider-google
path: artifacts

- name: Unzip the artifacts and delete the zip
run: |
unzip -o artifacts/output.zip -d ./provider
rm artifacts/output.zip

- name: Download built artifacts - Beta provider
uses: actions/download-artifact@v2
with:
name: artifact-terraform-provider-google-beta
path: artifacts

- name: Unzip the artifacts and delete the zip
run: |
unzip -o artifacts/output.zip -d ./provider
rm artifacts/output.zip

- name: Check that new services have been added to the TeamCity configuration code
run: |
# Create lists of service packages in providers
ls provider/google/services > tools/teamcity-diff-check/services_ga.txt
ls provider/google-beta/services > tools/teamcity-diff-check/services_beta.txt

# Run tool to compare service packages in the providers vs those listed in TeamCity config files
cd tools/teamcity-diff-check
go run main.go -service_file=services_ga
go run main.go -service_file=services_beta
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: ubuntu-22.04
steps:
- name: TeamCity Google Provider Generate
uses: ./.github/actions/build-downstream
with:
repo: 'terraform-provider-google'
token: '$GITHUB_TOKEN'

- name: TeamCity Google Beta Provider Generate
uses: ./.github/actions/build-downstream
with:
repo: 'terraform-provider-google-beta'
token: '$GITHUB_TOKEN'

- name: Checkout Repository
uses: actions/checkout@v4

- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: '^1.20'

- name: Download built artifacts - GA provider
uses: actions/download-artifact@v2
with:
name: artifact-terraform-provider-google
path: artifacts

- name: Unzip the artifacts and delete the zip
run: |
unzip -o artifacts/output.zip -d ./provider
rm artifacts/output.zip

- name: Download built artifacts - Beta provider
uses: actions/download-artifact@v2
with:
name: artifact-terraform-provider-google-beta
path: artifacts

- name: Unzip the artifacts and delete the zip
run: |
unzip -o artifacts/output.zip -d ./provider
rm artifacts/output.zip
BBBmau marked this conversation as resolved.
Show resolved Hide resolved

- name: Check that new services have been added to the TeamCity configuration code
run: |
# Create lists of service packages in providers
ls provider/google/services > tools/teamcity-diff-check/services_ga.txt
ls provider/google-beta/services > tools/teamcity-diff-check/services_beta.txt

# Run tool to compare service packages in the providers vs those listed in TeamCity config files
cd tools/teamcity-diff-check
go run main.go -service_file=services_ga
go run main.go -service_file=services_beta
44 changes: 24 additions & 20 deletions .github/workflows/teamcity-services-diff-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ on:
- 'mmv1/third_party/terraform/.teamcity/components/inputs/services_beta.kt'
- 'mmv1/products/**'
jobs:
check-pr:
teamcity-services-diff-check:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: ubuntu-22.04
outputs:
services: ${{steps.services.outputs.services}}
steps:
- name: Checkout Repository
uses: actions/checkout@v4
Expand All @@ -24,57 +24,61 @@ jobs:
newServices=$(($(git diff --name-only --diff-filter=A origin/main HEAD | grep -P "mmv1/products/.*/product.yaml" | wc -l)))
echo "services=$newServices" >> "${GITHUB_OUTPUT}"
if [ "$newServices" = "0" ];then
echo "No new service found."
echo "No new service found."
fi
terraform-provider-google:
if: ${{needs.check-pr.outputs.services != '0'}}
needs: check-pr
uses: ./.github/workflows/build-downstream.yml
with:
repo: 'terraform-provider-google'
- name: TeamCity Google Provider Generate
id: generate
if: ${{steps.services.outputs.services != '0'}}
uses: ./.github/actions/build-downstream
with:
repo: 'terraform-provider-google'
token: '$GITHUB_TOKEN'

terraform-provider-google-beta:
if: ${{needs.check-pr.outputs.services != '0'}}
needs: check-pr
uses: ./.github/workflows/build-downstream.yml
with:
repo: 'terraform-provider-google-beta'
- name: TeamCity Google Beta Provider Generate
if: steps.generate.outcome == 'success'
uses: ./.github/actions/build-downstream
with:
repo: 'terraform-provider-google-beta'
token: '$GITHUB_TOKEN'

teamcity-services-diff-check:
needs: [terraform-provider-google, terraform-provider-google-beta]
runs-on: ubuntu-22.04
steps:
- name: Checkout Repository
if: steps.generate.outcome == 'success'
uses: actions/checkout@v4

- name: Setup Go
if: steps.generate.outcome == 'success'
uses: actions/setup-go@v4
with:
go-version: '^1.20'

- name: Download built artifacts - GA provider
if: steps.generate.outcome == 'success'
uses: actions/download-artifact@v2
with:
name: artifact-terraform-provider-google
path: artifacts

- name: Unzip the artifacts and delete the zip
if: steps.generate.outcome == 'success'
run: |
unzip -o artifacts/output.zip -d ./provider
rm artifacts/output.zip

- name: Download built artifacts - Beta provider
if: steps.generate.outcome == 'success'
uses: actions/download-artifact@v2
with:
name: artifact-terraform-provider-google-beta
path: artifacts

- name: Unzip the artifacts and delete the zip
if: steps.generate.outcome == 'success'
run: |
unzip -o artifacts/output.zip -d ./provider
rm artifacts/output.zip

- name: Check that new services have been added to the TeamCity configuration code
if: steps.generate.outcome == 'success'
run: |
# Create lists of service packages in providers
ls provider/google/services > tools/teamcity-diff-check/services_ga.txt
Expand Down