Skip to content

Commit 01187c1

Browse files
vincentmrAmintorDuskogithub-actions[bot]ringo-but-quantumrashidnhm
authored andcommitted
Accelerate CI parallelizing over OMP configs (#712)
* Update Changelog * update release candidate version * run agains pennylane rc * update relese cadidate version * Increase tolerance for a flaky test (test_single_return_value) * Update `LightningBaseFallBack` to raise an error instead of falling back to default.qubit (#689) * Initial commit. * Update changelog * Auto update version * trigger ci * Unused import. * Auto update version * trigger ci * Fix imports. * Auto update version * Fix LightningException * make format * Auto update version from '0.36.0-dev34' to '0.36.0-dev35' * Auto update version from '0.36.0-dev35' to '0.36.0-dev36' * Auto update version from '0.36.0-dev36' to '0.36.0-dev37' * Make python -m pip install consistent. * Fix ImportError * Auto update version from '0.36.0-dev37' to '0.36.0-dev38' * --force-reinstall * Auto update version from '0.36.0-dev38' to '0.36.0-dev39' * Remove --force-reinstall * Auto update version from '0.36.0-dev39' to '0.36.0-dev40' * Auto update version from '0.36.0-dev42' to '0.36.0-dev43' * Auto update version from '0.36.0-dev43' to '0.36.0-dev44' * Spec CMAKE_CXX_COMPILER * Fix quotes. * Remove quotes * Hardcode g++-11 * use env.COMPILER * which g++ * Revert * Downgrade pip * Pin libomp. * Revert wheel_linux * Upload artifacts. * Comment upload trigger * Install pip==23 * Revert libomp * try testing outside cibuildwheel * yum install devtools * yum install devtools * Do not install python * Revert changes * Don't editable install. * Not edit install * Not edit install * Fix cov path * Explicit mcm tests path * Fix qml.QuantumFunctionError error messages. --------- Co-authored-by: Dev version update bot <github-actions[bot]@users.noreply.github.com> Co-authored-by: ringo-but-quantum <[email protected]> Co-authored-by: AmintorDusko <[email protected]> * trigger CIs * Introduce pytest-splits (#696) * Try update lightning CI paths * Auto update version * Update kokkos tests to trigger on changes to pl core * Auto update version * Split tests_linux.yml into cpp/python. * Auto update version from '0.36.0-dev36' to '0.36.0-dev38' * Split GPU tests into cpp/python. * Split MPI-GPU tests into cpp/python. * Fix LGPU triggers. * Remove python tests from LGPU-C++. * Remove upload-to-codecov-linux-cpp from LGPUMPI tests. * Rename test workflows. * Update changelog. * Change group names. * Use paths-ignore to simplify workflow paths * Auto update version from '0.36.0-dev38' to '0.36.0-dev39' * Fix test names. * you may only define one of `paths` and `paths-ignore` for a single event * Fix formatting * Auto update version from '0.36.0-dev39' to '0.36.0-dev40' * Change to paths-ignore in no-bin tests. * Auto update version from '0.36.0-dev40' to '0.36.0-dev41' * trigger ci * Change error message. * Split build/test jobs. * Revert pennylane_lightning/lightning_kokkos/lightning_kokkos.py * Fix setup.py command * Comment workflows * Comment pull_request: * Copy wheels to github.workspace /wheels * SKIP_COMPILATION=True * Revert SKIP_COMPILATION=True * Remove duplicate python actions. * Split LQ-test into 5 groups. * Update requirements-dev.txt * 2 groups * 1 group * 7 groups * Refactor LKokos tests * Auto update version from '0.36.0-dev42' to '0.36.0-dev43' * Fix cache id * Fix needs * Remove all builds * put back CMAKE_PREFIX_PATH * install fake lightning * Fix compat paths * Auto update version from '0.36.0-dev42' to '0.36.0-dev43' * Revert changes. * Update changelog * trigger ci * build_and_cache_Kokkos_linux.yml not concurrent. * Remove C++ tests from compat tests (skipped anyways). * Use ${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }} for concurrency groups. * PL_BACKEND: ${{ matrix.pl_backend }} * Update changelog * Install wheel package. * Auto update version from '0.36.0-dev43' to '0.36.0-dev44' * Install scipy wheel in one place. * Comment job triggers. * Add LK[cpu] est durations. * trigger ci * Use sha independent keys. * trigger ci * build_and_cache_Kokkos * Change LK keys * ls -l * Install version nonce * Do not skip comp. * Trade ref for sha in key * Install after PL * ref ==> sha * Reorder * Try LQ workflows. * Use actions/setup-python * Revert triggers. * Provide correct cov path. * Revert pennylane_lightning/core/src/utils/config.h * LK uses 5 groups. * Restore wheels to main * main/wheels * Revert cache paths * Fix ls -l * fail-on-cache-miss: true * Introduce env.GITHUB_WORKSPACE * Use artifacts * Fix paths * No ls -l * Download to github.workspace * fix wheel name * WHEEL_NAME * Add blas. * Use outputs. * Send wheel name * Fix artifact name * Fix name * Use WHEEL_NAME var * artifacts kokkos * fix wheel name * Restore bins. * Clean up * Split Linux CPU python tests into 2. * Fix compat workflows. * Split format and tidy workflows. * fix lkcpu * fix concurrency group * Do not use editable installs. * Split full kokkos tests except unitary_correct * Upload LK test durations. * duration-paths * Unique test_durations * Unique test_durations * Unique test_durations * Fix path * Update python_lightning_kokkos_test_durations * Remove obsolete all and SERIAL tests. * Update changelog * Creating the `LightningTensor` device class based on the new API (#671) * empty commit (triggering CI) * Auto update version * Definition of the two front-end classes * adding the `lightning_tensor` string to the supported backends in `setup.py` * adding `__init__.py` file to directly import the `lightning_tensor` device class * re-naming file * Auto update version * Creating the first prototype of initial MPS state tensor using `quimb` * providing the `backend`, `method` parameters and making `wires` optional * Changing names and structure * Auto update version * adding method required by the new device API design * Auto update version * using the `kwargs` parameter in `LightningTensor` and `CircuitMPS` in `quimb` * taking some further inputs from the new device API * Perhaps decided the overall structure of `LIghtningTensor` * Auto update version * adding docs to methods * temporary changes so that `pylint` does not complain at this stage * running `isort` * re-running formatter after `isort` * re-running formatter after `isort` * Applying suggested formatting change from CI * adding tmp unit tests * Adding `quimb` in `requirements.txt` * runing `isort` on mps test * removing `quimb` from requirement and deleting unit tests for `lightning.tensor` * Auto update version * re-inserting unit tests with an additional `yml` file * running isort on quimb test * changing name of yml file * preventing error in import * updating yml file * inserting `quimb` package in requirements-dev * strange error with `quimb` * strange error with `quimb` * specifying scipy version * removing installation of scipy from yml file * removing the new `yml` file * testing if tests are tested * Covering all lines in tests * forgot final line for formatter * Python formatter on CI complaints * covering missing lines * formatter on CI complaints * Trying not to skip test if Cpp is enabled * skipping tests if Cpp is enabled * removing the only line not covered by tests so far * Auto update version * Applying suggestions from code review and making the `state` attribute private (new API design) * Python formatter * removing params from `QuimbMPS` * Auto update version * removing `**kwargs` from `QuimbMPS` * removing unnecessary param at this stage * covering test line * formatter... * removing param description * Making `pylint` happy * forgot new arg in test * Updating base class and `preprocess` function * Updating `LightningTensor` class with new names from more advanced PR * Auto update version * Auto update version * Triggering CI * Auto update version * Trying to remove pin from `quimb` in `requirements.dev` * Auto update version * Auto update version * Removing infos on derivatives and using config options to pass parameters to interface * Usual `pylint` failures * Trying to solve formatting errors * typo in docstring * Sunday update: improved docstrings and structure * Removing method that was supposed to be in next PR * removing old TODO comment * Removing changes from the `setup.py` file * restoring previous format to `setup.py` * Auto update version from '0.36.0-dev34' to '0.36.0-dev41' * Auto update version from '0.36.0-dev40' to '0.36.0-dev41' * Removing kwargs as suggested from code review * Addressing comments from CR * Skipping tests if CPP binary is available * Auto update version from '0.36.0-dev42' to '0.36.0-dev43' * Auto update version from '0.36.0-dev43' to '0.36.0-dev44' * Restoring name in changelog (?) * Increasing time limit for Python tests * Applying suggestions from code review --------- Co-authored-by: Dev version update bot <github-actions[bot]@users.noreply.github.com> Co-authored-by: ringo-but-quantum <[email protected]> * Increase tolerance for a flaky test (test_single_return_value) (#703) * update dev version * increase tolerance * update changelog * trigger CIs * Auto update version from '0.36.0-dev44' to '0.36.0-dev45' --------- Co-authored-by: ringo-but-quantum <[email protected]> * Auto update version from '0.36.0-dev45' to '0.36.0-dev46' * Remove LTensor stuff * Update changelog * Revert requirements-dev.txt * trigger ci * Update python_lightning_kokkos_test_durations * Restore 'all' tests in tests_lkcuda_python.yml * Fix durations paths. * Try --splitting-algorithm=least_duration * Try no -n auto lkcpu tests * One last test. --------- Co-authored-by: Rashid N H M <[email protected]> Co-authored-by: Dev version update bot <github-actions[bot]@users.noreply.github.com> Co-authored-by: ringo-but-quantum <[email protected]> Co-authored-by: Pietropaolo Frisoni <[email protected]> Co-authored-by: Amintor Dusko <[email protected]> * Update changelog (#706) * Use wire order specified on instantiation (#705) * use map wires instead of map_to_standard_wires * oops * Update .github/CHANGELOG.md * black * Trigger CIs * add test * black on tests * Update .github/CHANGELOG.md --------- Co-authored-by: AmintorDusko <[email protected]> Co-authored-by: Ali Asadi <[email protected]> * Update wire order support for probabilities (#707) * Begin expansion of probs tests * Update tests to exhaustively compute probs with wires * Update all tests in LQ Meas to use CHECK instead of REQUIRES for better signal of failures * Add Jet Permuter * Add noisy functionality * Fix permutation ordering * Update changelog * Fix wrong ordering in probs tests * Add recommendation from apache flink to purge azure dirs * Forbid out of order indices for probs with LK and LG in CPP layer * Remove MSFT tools from GH Actions * Just try to remove dotnet * List opt hostedtoolcache * Remove packages from /opt/hostedtoolcache * Fix missing test hit * Forbid and remove broken tests from LK probs ordering * Fix clang-tidy isms * Ensure libopenblas is not broken from failed apt pull with dev * Appease GH Action * Full remove GH action modifier step due to unexplained failures * Add bidreictional wire validation for LGPU CPP layer * Clang-tidy warning fix * Appease CT * Update JET license agreement * Sync version * pacify CodeFactor * make CodeFactor ignore all JET code * trigger ci * trigger ci * Split OMP builds using matrix. * trigger ci * Ignore .github but not self. * Fix cd.. * Fix path * Update changelog [skip ci] * Revert changes to reqs.txt * Auto update version from '0.37.0-dev0' to '0.37.0-dev2' * Auto update version from '0.37.0-dev1' to '0.37.0-dev2' * Update .github/CHANGELOG.md Co-authored-by: Amintor Dusko <[email protected]> * Update .github/CHANGELOG.md Co-authored-by: Ali Asadi <[email protected]> --------- Co-authored-by: AmintorDusko <[email protected]> Co-authored-by: Dev version update bot <github-actions[bot]@users.noreply.github.com> Co-authored-by: ringo-but-quantum <[email protected]> Co-authored-by: Rashid N H M <[email protected]> Co-authored-by: Pietropaolo Frisoni <[email protected]> Co-authored-by: Amintor Dusko <[email protected]> Co-authored-by: Ali Asadi <[email protected]> Co-authored-by: Christina Lee <[email protected]> Co-authored-by: Lee James O'Riordan <[email protected]>
1 parent 6c2fbb4 commit 01187c1

9 files changed

+72
-504
lines changed

.github/CHANGELOG.md

+14-3
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,9 @@ Amintor Dusko, Shuli Shu
3232

3333
### Improvements
3434

35+
* The various OpenMP configurations of Lightning-Qubit are tested in parallel on different Github Actions runners.
36+
[(#712)](https://github.com/PennyLaneAI/pennylane-lightning/pull/712)
37+
3538
* Update Linux wheels to use manylinux_2_28 images.
3639
[(#667)](https://github.com/PennyLaneAI/pennylane-lightning/pull/667)
3740

@@ -43,7 +46,7 @@ Amintor Dusko, Shuli Shu
4346

4447
This release contains contributions from (in alphabetical order):
4548

46-
Amintor Dusko
49+
Amintor Dusko, Vincent Michaud-Rioux
4750

4851
---
4952

@@ -95,7 +98,6 @@ Amintor Dusko
9598

9699
* Split Lightning-Qubit and Lightning-Kokkos CPU Python tests with `pytest-split`. Remove `SERIAL` from Kokkos' `exec_model` matrix. Remove `all` from Lightning-Kokkos' `pl_backend` matrix. Move `clang-tidy` checks to `tidy.yml`. Avoid editable `pip` installations.
97100
[(#696)](https://github.com/PennyLaneAI/pennylane-lightning/pull/696)
98-
99101
* Update `lightning.gpu` and `lightning.kokkos` to raise an error instead of falling back to `default.qubit`.
100102
[(#689)](https://github.com/PennyLaneAI/pennylane-lightning/pull/689)
101103

@@ -193,11 +195,20 @@ Amintor Dusko
193195
* Update the version of `codecov-action` to v4 and fix the CodeCov issue with the PL-Lightning check-compatibility actions.
194196
[(#682)](https://github.com/PennyLaneAI/pennylane-lightning/pull/682)
195197

198+
* Refactor of dev prerelease auto-update-version workflow.
199+
[(#685)](https://github.com/PennyLaneAI/pennylane-lightning/pull/685)
200+
201+
* Remove gates unsupported by catalyst from toml file.
202+
[(#698)](https://github.com/PennyLaneAI/pennylane-lightning/pull/698)
203+
204+
* Increase tolerance for a flaky test.
205+
[(#703)](https://github.com/PennyLaneAI/pennylane-lightning/pull/703)
206+
196207
### Contributors
197208

198209
This release contains contributions from (in alphabetical order):
199210

200-
Ali Asadi, Amintor Dusko, Christina Lee, Vincent Michaud-Rioux, Lee James O'Riordan, Mudit Pandey, Shuli Shu
211+
Ali Asadi, Amintor Dusko, Thomas Germain, Christina Lee, Erick Ochoa Lopez, Vincent Michaud-Rioux, Rashid N H M, Lee James O'Riordan, Mudit Pandey, Shuli Shu
201212

202213
---
203214

.github/workflows/tests_lgpu_python.yml

+2
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ on:
1313
release:
1414
pull_request:
1515
paths-ignore:
16+
- .github
17+
- '!.github/workflows/tests_lgpu_python.yml'
1618
- pennylane_lightning/core/src/simulators/lightning_kokkos/**
1719
- pennylane_lightning/core/src/simulators/lightning_qubit/**
1820
- pennylane_lightning/lightning_kokkos/**

.github/workflows/tests_lgpumpi_python.yml

+2
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ on:
1616
- main
1717
pull_request:
1818
paths-ignore:
19+
- .github
20+
- '!.github/workflows/tests_lgpumpi_python.yml'
1921
- pennylane_lightning/core/src/simulators/lightning_kokkos/**
2022
- pennylane_lightning/core/src/simulators/lightning_qubit/**
2123
- pennylane_lightning/lightning_kokkos/**

.github/workflows/tests_linux_cpp.yml

+23-46
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,11 @@ jobs:
4545
strategy:
4646
matrix:
4747
pl_backend: ["lightning_qubit"]
48+
enable_kernel_omp: ["OFF", "ON"]
49+
enable_kernel_avx_stream: ["OFF", "ON"]
50+
exclude:
51+
- enable_kernel_omp: OFF
52+
enable_kernel_avx_stream: ON
4853
timeout-minutes: 60
4954
name: C++ tests
5055
runs-on: ${{ needs.determine_runner.outputs.runner_group }}
@@ -66,65 +71,39 @@ jobs:
6671
- name: Build and run unit tests
6772
run: |
6873
cmake . -BBuild -G Ninja \
69-
-DCMAKE_BUILD_TYPE=Debug \
70-
-DBUILD_TESTS=ON \
71-
-DENABLE_PYTHON=OFF \
72-
-DPL_BACKEND=${{ matrix.pl_backend }} \
73-
-DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION) \
74-
-DENABLE_COVERAGE=ON
75-
76-
cmake . -BBuildKernelOMP -G Ninja \
77-
-DCMAKE_BUILD_TYPE=Debug \
78-
-DBUILD_TESTS=ON \
79-
-DENABLE_PYTHON=OFF \
80-
-DPL_BACKEND=${{ matrix.pl_backend }} \
81-
-DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION) \
82-
-DENABLE_COVERAGE=ON \
83-
-DLQ_ENABLE_KERNEL_OMP=ON
84-
85-
cmake . -BBuildKernelAVXStream -G Ninja \
8674
-DCMAKE_BUILD_TYPE=Debug \
8775
-DBUILD_TESTS=ON \
8876
-DENABLE_PYTHON=OFF \
8977
-DPL_BACKEND=${{ matrix.pl_backend }} \
9078
-DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION) \
9179
-DENABLE_COVERAGE=ON \
92-
-DLQ_ENABLE_KERNEL_AVX_STREAM=ON \
93-
-DLQ_ENABLE_KERNEL_OMP=ON
80+
-DLQ_ENABLE_KERNEL_AVX_STREAM=${{ matrix.enable_kernel_avx_stream }} \
81+
-DLQ_ENABLE_KERNEL_OMP=${{ matrix.enable_kernel_omp }}
9482
9583
cmake --build ./Build
96-
cmake --build ./BuildKernelOMP
97-
cmake --build ./BuildKernelAVXStream
98-
99-
for d in Build BuildKernelOMP BuildKernelAVXStream; do
100-
cd ./$d
101-
mkdir -p ./tests/results
102-
for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results/report_$file.xml; done;
103-
lcov --directory . -b ../pennylane_lightning/core/src --capture --output-file coverage.info
104-
lcov --remove coverage.info '/usr/*' --output-file coverage.info
105-
cd ..
106-
done
107-
lcov --add-tracefile ./Build/coverage.info -a ./BuildKernelOMP/coverage.info \
108-
--add-tracefile ./BuildKernelAVXStream/coverage.info -o coverage.info
109-
mv coverage.info coverage-${{ github.job }}-${{ matrix.pl_backend }}.info
84+
85+
cd ./Build
86+
mkdir -p ./tests/results
87+
for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results/report_$file.xml; done;
88+
lcov --directory . -b ../pennylane_lightning/core/src --capture --output-file coverage.info
89+
lcov --remove coverage.info '/usr/*' --output-file coverage.info
90+
mv coverage.info coverage-${{ github.job }}-${{ matrix.pl_backend }}-${{ matrix.enable_kernel_avx_stream }}-${{ matrix.enable_kernel_omp }}.info
11091
11192
- name: Upload test results
11293
uses: actions/upload-artifact@v3
11394
if: always()
11495
with:
115-
name: ubuntu-tests-reports-${{ github.job }}-${{ matrix.pl_backend }}
96+
name: ubuntu-tests-reports-${{ github.job }}-${{ matrix.pl_backend }}-${{ matrix.enable_kernel_avx_stream }}-${{ matrix.enable_kernel_omp }}
11697
path: |
11798
./Build/tests/results/
118-
./BuildKernelOMP/tests/results/
119-
./BuildKernelAVXStream/tests/results/
12099
121100
if-no-files-found: error
122101

123102
- name: Upload code coverage results
124103
uses: actions/upload-artifact@v3
125104
with:
126-
name: ubuntu-codecov-results-cpp
127-
path: ./coverage-${{ github.job }}-${{ matrix.pl_backend }}.info
105+
name: ubuntu-codecov-results-${{ matrix.pl_backend }}-${{ matrix.enable_kernel_avx_stream }}-${{ matrix.enable_kernel_omp }}
106+
path: ./Build/coverage-${{ github.job }}-${{ matrix.pl_backend }}-${{ matrix.enable_kernel_avx_stream }}-${{ matrix.enable_kernel_omp }}.info
128107
if-no-files-found: error
129108

130109
cpptestswithOpenBLAS:
@@ -173,14 +152,14 @@ jobs:
173152
uses: actions/upload-artifact@v3
174153
if: always()
175154
with:
176-
name: ubuntu-tests-reports-${{ github.job }}-${{ matrix.pl_backend }}
155+
name: ubuntu-tests-reports-blas-${{ github.job }}-${{ matrix.pl_backend }}
177156
path: ./Build/tests/results/
178157
if-no-files-found: error
179158

180159
- name: Upload code coverage results
181160
uses: actions/upload-artifact@v3
182161
with:
183-
name: ubuntu-codecov-results-cpp
162+
name: ubuntu-codecov-results-blas-${{ matrix.pl_backend }}
184163
path: ./Build/coverage-${{ github.job }}-${{ matrix.pl_backend }}.info
185164
if-no-files-found: error
186165

@@ -245,8 +224,8 @@ jobs:
245224
-DENABLE_COVERAGE=ON
246225
cmake --build ./Build
247226
cd ./Build
248-
mkdir -p ./tests/results_${{ github.job }}_${{ matrix.pl_backend }}
249-
for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results_${{ github.job }}_${{ matrix.pl_backend }}/report_$file.xml; done;
227+
mkdir -p ./tests/results-${{ github.job }}-${{ matrix.pl_backend }}
228+
for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results-${{ github.job }}-${{ matrix.pl_backend }}/report_$file.xml; done;
250229
lcov --directory . -b ../pennylane_lightning/core/src --capture --output-file coverage.info
251230
lcov --remove coverage.info '/usr/*' --output-file coverage.info
252231
mv coverage.info coverage-${{ github.job }}-${{ matrix.pl_backend }}.info
@@ -255,8 +234,8 @@ jobs:
255234
uses: actions/upload-artifact@v3
256235
if: always()
257236
with:
258-
name: ubuntu-tests-reports-${{ github.job }}_${{ matrix.pl_backend }}
259-
path: ./Build/tests/results_${{ github.job }}_${{ matrix.pl_backend }}
237+
name: ubuntu-tests-reports-${{ github.job }}-${{ matrix.pl_backend }}
238+
path: ./Build/tests/results-${{ github.job }}-${{ matrix.pl_backend }}
260239
if-no-files-found: error
261240

262241
- name: Upload code coverage results
@@ -329,7 +308,6 @@ jobs:
329308
330309
- name: Build and run unit tests
331310
run: |
332-
set -x
333311
cmake . -BBuild -G Ninja \
334312
-DCMAKE_BUILD_TYPE=Debug \
335313
-DBUILD_TESTS=ON \
@@ -341,7 +319,6 @@ jobs:
341319
cd ./Build
342320
mkdir -p ./tests/results_multiple_backends
343321
for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results_multiple_backends/report_$file.xml; done;
344-
set +x
345322
346323
- name: Upload test results
347324
uses: actions/upload-artifact@v3

0 commit comments

Comments
 (0)