Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix GitHub Actions to pass authentication #4963

Merged
merged 55 commits into from
Jan 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
f347200
fix: Update workflows to also use workflow_call
franciscojavierarceo Jan 20, 2025
7f16143
fix: Also adding workflow_call to release version
franciscojavierarceo Jan 20, 2025
5efd815
fix: Adding type to release version input
franciscojavierarceo Jan 20, 2025
79a915a
chore: Fix operator release workflow (v0.43-branch) (#4937)
tchughesiv Jan 20, 2025
e67106b
fix: Updated type in yaml
franciscojavierarceo Jan 20, 2025
84e4858
fix: Updating publish.yml
franciscojavierarceo Jan 20, 2025
6891c58
fix: Update type in publish_images
franciscojavierarceo Jan 20, 2025
5300dd6
fix: Updating publish helm charts
franciscojavierarceo Jan 20, 2025
1c5c3d8
fix: Updating publish java sdk
franciscojavierarceo Jan 20, 2025
aacb68e
fix: Updating environment variable in the build_wheels
franciscojavierarceo Jan 20, 2025
31559b7
updating input parameters to build wheels
franciscojavierarceo Jan 20, 2025
4b60b1e
formatting issue
franciscojavierarceo Jan 20, 2025
1ebf243
chore: Fix operator release for 0.43.0 (v0.43-branch) (#4939)
tchughesiv Jan 20, 2025
ccaf71d
renaming args and adding more
franciscojavierarceo Jan 20, 2025
d9435b6
fix: Adding outputs to build_wheels workflow
franciscojavierarceo Jan 20, 2025
b95c0d3
reverting changes
franciscojavierarceo Jan 20, 2025
7379db3
chore: Add ability to change default feature-server image during Oper…
tchughesiv Jan 21, 2025
9e1b4db
chore: CR image no longer needed for e2e (v0.43-branch) (#4943)
tchughesiv Jan 21, 2025
c8378a3
chore: Fix publish, versions issue (#4944)
tchughesiv Jan 21, 2025
2fd6d1a
chore: Fix publish, versions issue (#4945)
tchughesiv Jan 21, 2025
002c075
chore: Fix publish, versions issue (#4946)
tchughesiv Jan 21, 2025
c5b2085
chore: Fix publish, versions issue (#4947)
tchughesiv Jan 21, 2025
ca18b6e
fix publish, versions issue (#4948)
tchughesiv Jan 21, 2025
3ee27ee
fix: adding checkout to publish sdk
franciscojavierarceo Jan 21, 2025
8196d78
chore: Change get version workflow to an action (#4951)
tchughesiv Jan 21, 2025
301ef9a
fix: Release Fix - Added the workflow_call element in missing workflo…
lokeshrangineni Jan 22, 2025
44ec146
chore: Build wheels fix (#4957)
tchughesiv Jan 22, 2025
e93d164
chore: Fix python-sdk workflow (#4958)
tchughesiv Jan 22, 2025
922bd65
fix: adding type and workflow_call to build_wheels.yml
franciscojavierarceo Jan 22, 2025
b502f46
chore: Remove checkout from get-version action (#4959)
tchughesiv Jan 22, 2025
33725c3
chore: Fixing build wheels pre-release image build (#4960)
tchughesiv Jan 22, 2025
f392c73
fix: Updating pypi upload to use legacy repo url (#4962)
franciscojavierarceo Jan 22, 2025
6e80c26
fix: Updating repostiry to repostory url
franciscojavierarceo Jan 22, 2025
daf2236
converting to dash to see if that works
franciscojavierarceo Jan 22, 2025
cf5d4ab
fix: Updating pypi publish action and adding name
franciscojavierarceo Jan 22, 2025
6081fd2
fix: moving back to kebab case
franciscojavierarceo Jan 22, 2025
cfd345f
fix: Trying to hardcode gh-action version
franciscojavierarceo Jan 22, 2025
4b83bd7
trying new version with oidc false
franciscojavierarceo Jan 22, 2025
62f4c52
trying without attestations
franciscojavierarceo Jan 22, 2025
308b864
reverting again to old action and removing checkout action
franciscojavierarceo Jan 22, 2025
420ffbb
reverting code to existing master without actions checkout
franciscojavierarceo Jan 22, 2025
c8c4312
logging token length to validate it and adding quotes to password
franciscojavierarceo Jan 23, 2025
3aa873b
adding logging to the build wheels to see if secret is present there
franciscojavierarceo Jan 23, 2025
5927dd4
adding secrets to jobs keyword
franciscojavierarceo Jan 23, 2025
8eaa8ec
adding input parameters to build wheels and publish python sdk
franciscojavierarceo Jan 23, 2025
c83e705
addding inherit
franciscojavierarceo Jan 23, 2025
2be247d
commenting out the publish python sdk and adding inherit
franciscojavierarceo Jan 23, 2025
82c7415
commenting out the needs
franciscojavierarceo Jan 23, 2025
242ad7a
removing commented out code and logging to incorporate code that worked
franciscojavierarceo Jan 23, 2025
8d43d32
adding one more change to remove logging
franciscojavierarceo Jan 23, 2025
9152d60
removing changes outside of actions
franciscojavierarceo Jan 23, 2025
d7d9a5b
removing makefile change
franciscojavierarceo Jan 23, 2025
5b5b833
removing yamls
franciscojavierarceo Jan 23, 2025
51b330f
removing changes files and reverting move of file
franciscojavierarceo Jan 23, 2025
db4a1e7
reverting changed files
franciscojavierarceo Jan 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions .github/actions/get-semantic-release-version/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
name: Get semantic release version
description: ""
inputs:
custom_version: # Optional input for a custom version
description: "Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing"
required: false
token:
description: "Personal Access Token"
required: true
default: ""
outputs:
release_version:
description: "The release version to use (e.g., v1.2.3)"
value: ${{ steps.get_release_version.outputs.release_version }}
version_without_prefix:
description: "The release version to use without 'v' (e.g., 1.2.3)"
value: ${{ steps.get_release_version_without_prefix.outputs.version_without_prefix }}
highest_semver_tag:
description: "The highest semantic version tag without the 'v' prefix (e.g., 1.2.3)"
value: ${{ steps.get_highest_semver.outputs.highest_semver_tag }}
runs:
using: composite
steps:
- name: Get release version
id: get_release_version
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.token }}
GIT_AUTHOR_NAME: feast-ci-bot
GIT_AUTHOR_EMAIL: [email protected]
GIT_COMMITTER_NAME: feast-ci-bot
GIT_COMMITTER_EMAIL: [email protected]
run: |
if [[ -n "${{ inputs.custom_version }}" ]]; then
VERSION_REGEX="^v[0-9]+\.[0-9]+\.[0-9]+$"
echo "Using custom version: ${{ inputs.custom_version }}"
if [[ ! "${{ inputs.custom_version }}" =~ $VERSION_REGEX ]]; then
echo "Error: custom_version must match semantic versioning (e.g., v1.2.3)."
exit 1
fi
echo "::set-output name=release_version::${{ inputs.custom_version }}"
elif [[ "${GITHUB_REF}" == refs/tags/* ]]; then
echo "Using tag reference: ${GITHUB_REF#refs/tags/}"
echo "::set-output name=release_version::${GITHUB_REF#refs/tags/}"
else
echo "Defaulting to branch name: ${GITHUB_REF#refs/heads/}"
echo "::set-output name=release_version::${GITHUB_REF#refs/heads/}"
fi
- name: Get release version without prefix
id: get_release_version_without_prefix
shell: bash
env:
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
run: |
if [[ "${RELEASE_VERSION}" == v* ]]; then
echo "::set-output name=version_without_prefix::${RELEASE_VERSION:1}"
else
echo "::set-output name=version_without_prefix::${RELEASE_VERSION}"
fi
- name: Get highest semver
id: get_highest_semver
shell: bash
env:
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
run: |
if [[ -n "${{ inputs.custom_version }}" ]]; then
HIGHEST_SEMVER_TAG="${{ inputs.custom_version }}"
echo "::set-output name=highest_semver_tag::$HIGHEST_SEMVER_TAG"
echo "Using custom version as highest semantic version: $HIGHEST_SEMVER_TAG"
else
source infra/scripts/setup-common-functions.sh
SEMVER_REGEX='^v[0-9]+\.[0-9]+\.[0-9]+(-([0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*))?$'
if echo "${RELEASE_VERSION}" | grep -P "$SEMVER_REGEX" &>/dev/null ; then
echo ::set-output name=highest_semver_tag::$(get_tag_release -m)
echo "Using infra/scripts/setup-common-functions.sh to generate highest semantic version: $HIGHEST_SEMVER_TAG"
fi
fi
- name: Check output
shell: bash
env:
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
VERSION_WITHOUT_PREFIX: ${{ steps.get_release_version_without_prefix.outputs.version_without_prefix }}
HIGHEST_SEMVER_TAG: ${{ steps.get_highest_semver.outputs.highest_semver_tag }}
run: |
echo $RELEASE_VERSION
echo $VERSION_WITHOUT_PREFIX
echo $HIGHEST_SEMVER_TAG
80 changes: 57 additions & 23 deletions .github/workflows/build_wheels.yml
Original file line number Diff line number Diff line change
@@ -1,30 +1,33 @@
name: build_wheels
name: build wheels

# Call this workflow from other workflows in the repository by specifying "uses: ./.github/workflows/build_wheels.yml"
# Developers who are starting a new release should use this workflow to ensure wheels will be built correctly.
# Devs should check out their fork, add a tag to the last master commit on their fork, and run the release off of their fork on the added tag to ensure wheels will be built correctly.
on:
workflow_dispatch:
tags:
- 'v*.*.*'
workflow_call:
workflow_dispatch: # Allows manual trigger of the workflow
inputs:
release_version:
description: 'The release version to use (e.g., v1.2.3)'
custom_version: # Optional input for a custom version
description: 'Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing'
required: false
type: string
token:
description: 'Personal Access Token'
required: true
default: ""
type: string
workflow_call:
inputs:
custom_version: # Optional input for a custom version
description: 'Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing'
required: false
type: string
highest_semver_tag:
description: 'The highest semantic version tag without the "v" prefix (e.g., 1.2.3)'
token:
description: 'Personal Access Token'
required: true
default: ""
type: string

jobs:
get-version:
uses: ./.github/workflows/get_semantic_release_version.yml
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}

build-python-wheel:
name: Build wheels
runs-on: ubuntu-latest
Expand All @@ -42,10 +45,18 @@ jobs:
registry-url: 'https://registry.npmjs.org'
- name: Build UI
run: make build-ui
- id: get-version
uses: ./.github/actions/get-semantic-release-version
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}
- name: Build wheels
env:
VERSION: ${{ steps.get-version.outputs.release_version }}
PYPI_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
git fetch --tags
git checkout v0.42.0
git checkout ${VERSION}
python -m pip install build
python -m build --wheel --outdir wheelhouse/
- uses: actions/upload-artifact@v4
Expand All @@ -69,11 +80,18 @@ jobs:
with:
node-version-file: './ui/.nvmrc'
registry-url: 'https://registry.npmjs.org'
- id: get-version
uses: ./.github/actions/get-semantic-release-version
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}
- name: Build and install dependencies
env:
VERSION: ${{ steps.get-version.outputs.release_version }}
# There's a `git restore` in here because `make install-go-ci-dependencies` is actually messing up go.mod & go.sum.
run: |
git fetch --tags
git checkout v0.42.0
git checkout ${VERSION}
pip install -U pip setuptools wheel twine
make build-ui
git status
Expand All @@ -89,11 +107,12 @@ jobs:

# We add this step so the docker images can be built as part of the pre-release verification steps.
build-docker-images:
name: Build Docker images
runs-on: ubuntu-latest
needs: get-version
needs: [ build-python-wheel, build-source-distribution ]
strategy:
matrix:
component: [ feature-server, feature-server-java, feature-transformation-server, feast-operator ]
component: [ feature-server-dev, feature-server-java, feature-transformation-server, feast-operator ]
env:
REGISTRY: feastdev
steps:
Expand All @@ -102,15 +121,23 @@ jobs:
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- id: get-version
uses: ./.github/actions/get-semantic-release-version
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}
- name: Build image
env:
VERSION_WITHOUT_PREFIX: ${{ steps.get-version.outputs.version_without_prefix }}
RELEASE_VERSION: ${{ steps.get-version.outputs.release_version }}
run: |
echo "Building docker image for ${{ matrix.component }} with version $VERSION_WITHOUT_PREFIX and release version $RELEASE_VERSION"
make build-${{ matrix.component }}-docker REGISTRY=${REGISTRY} VERSION=${VERSION_WITHOUT_PREFIX}
env:
VERSION_WITHOUT_PREFIX: ${{ needs.get-version.outputs.version_without_prefix }}

verify-python-wheels:
name: Verify Python wheels
runs-on: ${{ matrix.os }}
needs: [ build-python-wheel, build-source-distribution, get-version ]
needs: [ build-python-wheel, build-source-distribution ]
strategy:
matrix:
os: [ ubuntu-latest, macos-13 ]
Expand All @@ -129,8 +156,8 @@ jobs:
else
echo "Succeeded!"
fi
VERSION_WITHOUT_PREFIX: ${{ needs.get-version.outputs.version_without_prefix }}
steps:
- uses: actions/checkout@v4
- name: Setup Python
id: setup-python
uses: actions/setup-python@v5
Expand Down Expand Up @@ -160,7 +187,14 @@ jobs:
if: ${{ matrix.from-source }}
run: pip install dist/*tar.gz
# Validate that the feast version installed is not development and is the correct version of the tag we ran it off of.
- id: get-version
uses: ./.github/actions/get-semantic-release-version
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}
- name: Validate Feast Version
env:
VERSION_WITHOUT_PREFIX: ${{ steps.get-version.outputs.version_without_prefix }}
run: |
feast version
if ! VERSION_OUTPUT=$(feast version); then
Expand Down
84 changes: 0 additions & 84 deletions .github/workflows/get_semantic_release_version.yml

This file was deleted.

28 changes: 19 additions & 9 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,42 +9,52 @@ on:
custom_version: # Optional input for a custom version
description: 'Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing'
required: false
type: string
token:
description: 'Personal Access Token'
required: true
default: ""
type: string
workflow_call: # Allows trigger the workflow from other workflow
inputs:
custom_version: # Optional input for a custom version
description: 'Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing'
required: false
type: string
token:
description: 'Personal Access Token'
required: true
default: ""
type: string

jobs:
get-version:
uses: ./.github/workflows/get_semantic_release_version.yml
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}

publish-python-sdk:
uses: ./.github/workflows/publish_python_sdk.yml
secrets: inherit
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}

build-publish-docker-images:
uses: ./.github/workflows/publish_images.yml
needs: [ get-version, publish-python-sdk ]
needs: [ publish-python-sdk ]
secrets: inherit
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}

publish-helm-charts:
uses: ./.github/workflows/publish_helm_charts.yml
needs: [ get-version, publish-python-sdk ]
needs: [ publish-python-sdk ]
secrets: inherit
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}

publish-java-sdk:
uses: ./.github/workflows/publish_java_sdk.yml
needs: [ get-version, publish-python-sdk ]
needs: [ publish-python-sdk ]
secrets: inherit
with:
custom_version: ${{ github.event.inputs.custom_version }}
token: ${{ github.event.inputs.token }}
Loading
Loading