Skip to content

Commit 45bc864

Browse files
rashidnhmringo-but-quantumAmintorDuskomaliasadi
authored
Updated CI workflows to avoid race conditions on PRs with concurrency label (#778)
**Context:** Currently there exists a race condition within the CI due to misconfiguration of the concurrency label. This PR updates the affected workflows to fix the concurrency labels. **Description of the Change:** `${{ github.event }}` is not a string, but rather an object, when you have that in the concurrency label you get: ```yaml # Label group: my-workflow-${{ github.event }}-${{ github.ref }} # Renders as group: my-workflow-[object Object]-refs/heads/my-branch ``` This was causing workflows that had `pull_request` and `workflow_call` triggers to overstep on each other's runtime as they both rendered with the same concurrency label. This PR changes the attribute to use the correct string name `event_name`. ```yaml # Label group: my-workflow-${{ github.event_name }}-${{ github.ref }} # Renders as group: my-workflow-pull_request-refs/heads/my-branch ``` **Benefits:** Race conditions averted. No random job cancellations especially when PR needs wheels built. **Possible Drawbacks:** None. **Related GitHub Issues:** None. [sc-66955](https://app.shortcut.com/xanaduai/story/66955/fix-bug-in-lightning-ci-causing-race-conditions-of-parallel-job-runs) --------- Co-authored-by: ringo-but-quantum <[email protected]> Co-authored-by: AmintorDusko <[email protected]> Co-authored-by: Ali Asadi <[email protected]>
1 parent 3d6db7a commit 45bc864

16 files changed

+109
-17
lines changed

‎.github/CHANGELOG.md

+3
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@
3535

3636
### Improvements
3737

38+
* Updated concurrency labels for tests ensuring different workflows do not run under the same concurrency group label
39+
[(#778)](https://github.com/PennyLaneAI/pennylane-lightning/pull/778)
40+
3841
* Release candidate branches are automatically switched over to using the new large GitHub runner pool.
3942
[(#769)](https://github.com/PennyLaneAI/pennylane-lightning/pull/769)
4043

‎.github/workflows/tests_lgpu_cpp.yml

+9-4
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ env:
3030
TORCH_VERSION: 1.11.0+cpu
3131

3232
concurrency:
33-
group: tests_lgpu_cpp-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
33+
group: tests_lgpu_cpp-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3434
cancel-in-progress: true
3535

3636
jobs:
@@ -108,6 +108,11 @@ jobs:
108108
109109
# Adding venv name as an output for subsequent steps to reference if needed
110110
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
111+
echo "Python_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
112+
echo "Python3_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
113+
114+
# Adding venv site-packages to output for subsequent step to referecen if needed
115+
echo "site_packages_dir=$(${{ env.VENV_NAME }}/bin/python -c 'import sysconfig; print(sysconfig.get_path("platlib"))')" >> $GITHUB_OUTPUT
111116
112117
- name: Display Python-Path
113118
id: python_path
@@ -116,14 +121,14 @@ jobs:
116121
echo "Python Interpreter Path => $py_path"
117122
echo "python=$py_path" >> $GITHUB_OUTPUT
118123
119-
pip_path=$(which python)
124+
pip_path=$(which pip)
120125
echo "PIP Path => $pip_path"
121126
echo "pip=$pip_path" >> $GITHUB_OUTPUT
122127
123128
- name: Install required packages
124129
run: |
125-
python -m pip install ninja cmake custatevec-cu${{ matrix.cuda_version }}
126-
sudo apt-get -y -q install liblapack-dev
130+
python -m pip install ninja cmake custatevec-cu${{ matrix.cuda_version }} scipy
131+
sudo apt-get -y -q install liblapack-dev black
127132
128133
- name: Build and run unit tests
129134
run: |

‎.github/workflows/tests_lgpu_python.yml

+8-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ env:
3434
TORCH_VERSION: 1.11.0+cpu
3535

3636
concurrency:
37-
group: tests_lgpu_python-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
37+
group: tests_lgpu_python-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3838
cancel-in-progress: true
3939

4040
jobs:
@@ -128,6 +128,13 @@ jobs:
128128
# Adding venv name as an output for subsequent steps to reference if needed
129129
source ${{ env.VENV_NAME }}/bin/activate
130130
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
131+
echo "Python_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
132+
echo "Python3_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
133+
134+
135+
# Adding venv site-packages to output for subsequent step to referecen if needed
136+
echo "site_packages_dir=$(${{ env.VENV_NAME }}/bin/python -c 'import sysconfig; print(sysconfig.get_path("platlib"))')" >> $GITHUB_OUTPUT
137+
131138
132139
- name: Display Python-Path
133140
id: python_path

‎.github/workflows/tests_lgpumpi_cpp.yml

+3-1
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ env:
2929
CI_CUDA_ARCH: 86
3030

3131
concurrency:
32-
group: tests_lgpumpi_cpp-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
32+
group: tests_lgpumpi_cpp-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3333
cancel-in-progress: true
3434

3535
jobs:
@@ -85,6 +85,8 @@ jobs:
8585
8686
# Adding venv name as an output for subsequent steps to reference if needed
8787
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
88+
echo "Python_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
89+
echo "Python3_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
8890
8991
- name: Display Python-Path
9092
id: python_path

‎.github/workflows/tests_lgpumpi_python.yml

+3-1
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ env:
3030
CI_CUDA_ARCH: 86
3131

3232
concurrency:
33-
group: tests_lgpumpi_python-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
33+
group: tests_lgpumpi_python-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3434
cancel-in-progress: true
3535

3636
jobs:
@@ -90,6 +90,8 @@ jobs:
9090
# Adding venv name as an output for subsequent steps to reference if needed
9191
source ${{ env.VENV_NAME }}/bin/activate
9292
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
93+
echo "Python_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
94+
echo "Python3_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
9395
9496
9597
- name: Display Python-Path

‎.github/workflows/tests_linux_cpp.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ env:
3030
OMP_PROC_BIND: "false"
3131

3232
concurrency:
33-
group: tests_linux_cpp-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
33+
group: tests_linux_cpp-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3434
cancel-in-progress: true
3535

3636
jobs:

‎.github/workflows/tests_lkcpu_python.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ env:
3434
OMP_PROC_BIND: "false"
3535

3636
concurrency:
37-
group: tests_lkcpu_python-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
37+
group: tests_lkcpu_python-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3838
cancel-in-progress: true
3939

4040
jobs:

‎.github/workflows/tests_lkcuda_cpp.yml

+28-1
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ env:
2929
TORCH_VERSION: 1.11.0+cpu
3030

3131
concurrency:
32-
group: tests_lkcuda_cpp-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
32+
group: tests_lkcuda_cpp-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3333
cancel-in-progress: true
3434

3535
jobs:
@@ -153,6 +153,33 @@ jobs:
153153
with:
154154
python-version: '3.9'
155155

156+
# Since the self-hosted runner can be re-used. It is best to set up all package
157+
# installations in a virtual environment that gets cleaned at the end of each workflow run
158+
- name: Setup Python virtual environment
159+
id: setup_venv
160+
env:
161+
VENV_NAME: ${{ github.workspace }}/venv_${{ steps.setup_python.outputs.python-version }}_${{ github.sha }}
162+
run: |
163+
# Clear any pre-existing venvs
164+
rm -rf venv_*
165+
166+
# Create new venv for this workflow_run
167+
python --version
168+
python -m venv ${{ env.VENV_NAME }}
169+
170+
# Add the venv to PATH for subsequent steps
171+
echo ${{ env.VENV_NAME }}/bin >> $GITHUB_PATH
172+
173+
# Adding venv name as an output for subsequent steps to reference if needed
174+
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
175+
echo "Python_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
176+
echo "Python3_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
177+
178+
179+
# Adding venv site-packages to output for subsequent step to referecen if needed
180+
echo "site_packages_dir=$(${{ env.VENV_NAME }}/bin/python -c 'import sysconfig; print(sysconfig.get_path("platlib"))')" >> $GITHUB_OUTPUT
181+
182+
156183
- name: Install dependencies
157184
run: sudo apt-get update && sudo apt-get -y -q install cmake gcc-$GCC_VERSION g++-$GCC_VERSION ninja-build gcovr lcov
158185

‎.github/workflows/tests_lkcuda_python.yml

+28-1
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ env:
3333
TORCH_VERSION: 1.11.0+cpu
3434

3535
concurrency:
36-
group: tests_lkcuda_python-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
36+
group: tests_lkcuda_python-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3737
cancel-in-progress: true
3838

3939
jobs:
@@ -173,6 +173,33 @@ jobs:
173173
with:
174174
python-version: '3.9'
175175

176+
# Since the self-hosted runner can be re-used. It is best to set up all package
177+
# installations in a virtual environment that gets cleaned at the end of each workflow run
178+
- name: Setup Python virtual environment
179+
id: setup_venv
180+
env:
181+
VENV_NAME: ${{ github.workspace }}/venv_${{ steps.setup_python.outputs.python-version }}_${{ github.sha }}
182+
run: |
183+
# Clear any pre-existing venvs
184+
rm -rf venv_*
185+
186+
# Create new venv for this workflow_run
187+
python --version
188+
python -m venv ${{ env.VENV_NAME }}
189+
190+
# Add the venv to PATH for subsequent steps
191+
echo ${{ env.VENV_NAME }}/bin >> $GITHUB_PATH
192+
193+
# Adding venv name as an output for subsequent steps to reference if needed
194+
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
195+
echo "Python_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
196+
echo "Python3_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
197+
198+
199+
# Adding venv site-packages to output for subsequent step to referecen if needed
200+
echo "site_packages_dir=$(${{ env.VENV_NAME }}/bin/python -c 'import sysconfig; print(sysconfig.get_path("platlib"))')" >> $GITHUB_OUTPUT
201+
202+
176203
- name: Install dependencies
177204
run: sudo apt-get update && sudo apt-get -y -q install cmake gcc-$GCC_VERSION g++-$GCC_VERSION
178205

‎.github/workflows/tests_lmps_tncuda_cpp.yml

+9-1
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,8 @@ env:
3131
GCC_VERSION: 11
3232

3333
concurrency:
34-
group: tests_lmps_tncuda_cpp-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
34+
# TODO: Revert back to using event_name
35+
group: tests_lmps_tncuda_cpp-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3536
cancel-in-progress: true
3637

3738
jobs:
@@ -115,6 +116,13 @@ jobs:
115116
116117
# Adding venv name as an output for subsequent steps to reference if needed
117118
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
119+
echo "Python_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
120+
echo "Python3_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
121+
122+
123+
# Adding venv site-packages to output for subsequent step to referecen if needed
124+
echo "site_packages_dir=$(${{ env.VENV_NAME }}/bin/python -c 'import sysconfig; print(sysconfig.get_path("platlib"))')" >> $GITHUB_OUTPUT
125+
118126
119127
- name: Display Python-Path
120128
id: python_path

‎.github/workflows/tests_lmps_tncuda_python.yml

+7-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ env:
3434
GCC_VERSION: 11
3535

3636
concurrency:
37-
group: tests_lmps_tncuda_python-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
37+
group: tests_lmps_tncuda_python-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
3838
cancel-in-progress: true
3939

4040
jobs:
@@ -121,6 +121,12 @@ jobs:
121121
# Adding venv name as an output for subsequent steps to reference if needed
122122
source ${{ env.VENV_NAME }}/bin/activate
123123
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
124+
echo "Python_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
125+
echo "Python3_ROOT_DIR=${{ env.VENV_NAME }}" >> $GITHUB_ENV
126+
127+
# Adding venv site-packages to output for subsequent step to referecen if needed
128+
echo "site_packages_dir=$(${{ env.VENV_NAME }}/bin/python -c 'import sysconfig; print(sysconfig.get_path("platlib"))')" >> $GITHUB_OUTPUT
129+
124130
125131
- name: Display Python-Path
126132
id: python_path

‎.github/workflows/tests_lqcpu_python.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ env:
2424
OMP_PROC_BIND: "false"
2525

2626
concurrency:
27-
group: tests_lqcpu_python-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
27+
group: tests_lqcpu_python-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
2828
cancel-in-progress: true
2929

3030
jobs:

‎.github/workflows/tests_windows_cpp.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ on:
2020
- '!pennylane_lightning/core/src/simulators/lightning_gpu/**'
2121

2222
concurrency:
23-
group: tests_windows_cpp-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
23+
group: tests_windows_cpp-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
2424
cancel-in-progress: true
2525

2626
jobs:

‎.github/workflows/tests_without_binary.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ env:
2121
COVERAGE_FLAGS: "--cov=pennylane_lightning --cov-report=term-missing --cov-report=xml:./coverage.xml --no-flaky-report -p no:warnings --tb=native"
2222

2323
concurrency:
24-
group: tests_without_binary-${{ github.ref }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
24+
group: tests_without_binary-${{ github.ref }}-${{ github.event_name }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
2525
cancel-in-progress: true
2626

2727
jobs:

‎.github/workflows/update_dev_version.yml

+5
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@ on:
44
branches:
55
- master
66

7+
concurrency:
8+
group: ${{ github.workflow }}-${{ github.ref }}
9+
cancel-in-progress: true
10+
11+
712
jobs:
813
update-dev-version:
914
name: Update version for development branches

‎pennylane_lightning/core/_version.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,4 @@
1616
Version number (major.minor.patch[-label])
1717
"""
1818

19-
__version__ = "0.37.0-dev50"
19+
__version__ = "0.37.0-dev51"

0 commit comments

Comments
 (0)