Skip to content

Commit 3527765

Browse files
mlxdgithub-actions[bot]
mlxd
and
github-actions[bot]
authored
Add build and test support for CUDA 12 (#606)
* Add build and test support for CUDA 12 * Auto update version * Fix name * Avoid overstepping with CUDA 11 on 12 * Fix wheel output naming * Remove unneeded strings * Add missing 0 to CUDA 11 install * Trigger CI * Update to allow CUDA 11 wheels to build also * Update fromJSON usage * Update int to str in json map * Aim to keep cuda 11.5 wheels for now * Remove CUDA 11 * Migrate windows tests to clangcl to avoid failures * Ensure VS generator used * Update for auditwheel 6.0 changes * Ensure devtoolset values are used in CUDA 12 build * Remove unneeded deref * Retry installing cuda 12.0 * Fix auditwheel arch check * Fix auditwheel arch check again * Revert windows tests * Lower overhead of Windows CI tests (#610) * Cache vcpkg libs and reuse * Auto update version * Trigger CI * Fix matrix tests for windows * Add excluded modules for OpenCPPCoverage * Convert dash to underscore * Use optimized build for Windows coverage * Retrigger CI --------- Co-authored-by: Dev version update bot <github-actions[bot]@users.noreply.github.com> * Update changelog * Remove trainability from stateprep in test * Update dev reqs to use cu12 * Fix formatting * Update the MPI modules in CI * Add ls to module dirs * Remove additional cu11 deps * Use mpirun from PATH * Update to cu12 in missing locations * Remove opt path specifics * Ensure cuda version env vars are inlcuded on Python tests for MPI --------- Co-authored-by: Dev version update bot <github-actions[bot]@users.noreply.github.com>
1 parent f3beabc commit 3527765

17 files changed

+198
-143
lines changed

‎.github/CHANGELOG.md

+6
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,14 @@
77

88
### Breaking changes
99

10+
* Migrate `lightning.gpu` to CUDA 12.
11+
[(#606)](https://github.com/PennyLaneAI/pennylane-lightning/pull/606)
12+
1013
### Improvements
1114

15+
* Lower the overheads of Windows CI tests.
16+
[(#610)](https://github.com/PennyLaneAI/pennylane-lightning/pull/610)
17+
1218
* Decouple LightningQubit memory ownership from numpy and migrate it to Lightning-Qubit managed state-vector class.
1319
[(#601)](https://github.com/PennyLaneAI/pennylane-lightning/pull/601)
1420

‎.github/workflows/compat-check-latest-latest.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
pennylane-version: latest
2121
tests_lgpu_gpu:
2222
name: Lightning Compatibility test (tests_lgpu_gpu) - latest/latest
23-
uses: ./.github/workflows/tests_gpu_cu11.yml
23+
uses: ./.github/workflows/tests_gpu_cuda.yml
2424
with:
2525
lightning-version: latest
2626
pennylane-version: latest

‎.github/workflows/compat-check-latest-stable.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
pennylane-version: stable
2121
tests_lgpu_gpu:
2222
name: Lightning Compatibility test (tests_lgpu_gpu) - latest/stable
23-
uses: ./.github/workflows/tests_gpu_cu11.yml
23+
uses: ./.github/workflows/tests_gpu_cuda.yml
2424
with:
2525
lightning-version: latest
2626
pennylane-version: stable

‎.github/workflows/compat-check-release-release.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
pennylane-version: release
2121
tests_lgpu_gpu:
2222
name: Lightning Compatibility test (tests_lgpu_gpu) - release/release
23-
uses: ./.github/workflows/tests_gpu_cu11.yml
23+
uses: ./.github/workflows/tests_gpu_cuda.yml
2424
with:
2525
lightning-version: release
2626
pennylane-version: release

‎.github/workflows/compat-check-stable-latest.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
pennylane-version: latest
2121
tests_lgpu_gpu:
2222
name: Lightning Compatibility test (tests_lgpu_gpu) - stable/latest
23-
uses: ./.github/workflows/tests_gpu_cu11.yml
23+
uses: ./.github/workflows/tests_gpu_cuda.yml
2424
with:
2525
lightning-version: stable
2626
pennylane-version: latest

‎.github/workflows/compat-check-stable-stable.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
pennylane-version: stable
2121
tests_lgpu_gpu:
2222
name: Lightning Compatibility test (tests_lgpu_gpu) - stable/stable
23-
uses: ./.github/workflows/tests_gpu_cu11.yml
23+
uses: ./.github/workflows/tests_gpu_cuda.yml
2424
with:
2525
lightning-version: stable
2626
pennylane-version: stable

‎.github/workflows/tests_gpu_cu11.yml ‎.github/workflows/tests_gpu_cuda.yml

+10-7
Original file line numberDiff line numberDiff line change
@@ -39,25 +39,27 @@ jobs:
3939
matrix:
4040
os: [ubuntu-22.04]
4141
pl_backend: ["lightning_gpu"]
42+
cuda_version: ["12"]
4243

4344
steps:
4445
- name: Validate GPU version and installed compiler
4546
run: |
4647
source /etc/profile.d/modules.sh
4748
module use /opt/modules
48-
module load cuda/11.8
49+
module load cuda/${{ matrix.cuda_version }}
4950
echo "${PATH}" >> $GITHUB_PATH
5051
echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" >> $GITHUB_ENV
5152
nvcc --version
5253
nvidia-smi
5354
54-
cpptestswithLGPU_cu11:
55+
cpptestswithLGPU:
5556
if: ${{ !contains(fromJSON('["schedule", "workflow_dispatch"]'), github.event_name) }}
5657
needs: [builddeps]
5758
strategy:
5859
matrix:
5960
os: [ubuntu-22.04]
6061
pl_backend: ["lightning_gpu"]
62+
cuda_version: ["12"]
6163

6264
name: C++ tests (Lightning-GPU)
6365
runs-on:
@@ -70,7 +72,7 @@ jobs:
7072
run: |
7173
source /etc/profile.d/modules.sh
7274
module use /opt/modules
73-
module load cuda/11.8
75+
module load cuda/${{ matrix.cuda_version }}
7476
echo "${PATH}" >> $GITHUB_PATH
7577
echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" >> $GITHUB_ENV
7678
nvcc --version
@@ -117,7 +119,7 @@ jobs:
117119
118120
- name: Install required packages
119121
run: |
120-
python -m pip install ninja cmake custatevec-cu11
122+
python -m pip install ninja cmake custatevec-cu${{ matrix.cuda_version }}
121123
sudo apt-get -y -q install liblapack-dev
122124
123125
- name: Build and run unit tests
@@ -161,6 +163,7 @@ jobs:
161163
os: [ubuntu-22.04]
162164
pl_backend: ["lightning_gpu"]
163165
default_backend: ["lightning_qubit"]
166+
cuda_version: ["12"]
164167

165168
name: Python tests with LGPU
166169
runs-on:
@@ -173,7 +176,7 @@ jobs:
173176
run: |
174177
source /etc/profile.d/modules.sh
175178
module use /opt/modules
176-
module load cuda/11.8
179+
module load cuda/${{ matrix.cuda_version }}
177180
echo "${PATH}" >> $GITHUB_PATH
178181
echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" >> $GITHUB_ENV
179182
nvcc --version
@@ -238,7 +241,7 @@ jobs:
238241
run: |
239242
cd main
240243
python -m pip install -r requirements-dev.txt
241-
python -m pip install cmake custatevec-cu11 openfermionpyscf
244+
python -m pip install cmake custatevec-cu${{ matrix.cuda_version }} openfermionpyscf
242245
243246
- name: Checkout PennyLane for release build
244247
if: inputs.pennylane-version == 'release'
@@ -344,7 +347,7 @@ jobs:
344347
token: ${{ secrets.CODECOV_TOKEN }}
345348

346349
upload-to-codecov-linux-cpp:
347-
needs: [cpptestswithLGPU_cu11]
350+
needs: [cpptestswithLGPU]
348351
name: Upload coverage data to codecov
349352
runs-on: ubuntu-latest
350353
steps:

‎.github/workflows/tests_linux_x86_mpi_gpu.yml

+21-16
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ jobs:
3939
max-parallel: 1
4040
matrix:
4141
mpilib: ["mpich", "openmpi"]
42+
cuda_version_maj: ["12"]
43+
cuda_version_min: ["2"]
4244
timeout-minutes: 30
4345

4446
steps:
@@ -92,28 +94,29 @@ jobs:
9294
- name: Install required packages
9395
run: |
9496
python -m pip install -r requirements-dev.txt
95-
python -m pip install cmake custatevec-cu11
97+
python -m pip install cmake custatevec-cu12
9698
sudo apt-get -y -q install liblapack-dev
9799
98-
- name: Validate GPU version and installed compiler
100+
- name: Validate GPU version and installed compiler and modules
99101
run: |
100-
source /etc/profile.d/modules.sh && module use /opt/modules && module load cuda/11.8
102+
source /etc/profile.d/modules.sh && module use /opt/modules && module load cuda/${{ matrix.cuda_version_maj }}
101103
which -a nvcc
102104
nvcc --version
105+
ls -R /opt/modules
103106
104107
- name: Validate Multi-GPU packages
105108
run: |
106-
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}
109+
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}/cuda-${{ matrix.cuda_version_maj }}.${{ matrix.cuda_version_min }}
107110
echo 'Checking for ${{ matrix.mpilib }}'
108111
which -a mpirun
109112
mpirun --version
110113
which -a mpicxx
111114
mpicxx --version
112-
module unload ${{ matrix.mpilib }}
115+
module unload ${{ matrix.mpilib }}/cuda-${{ matrix.cuda_version_maj }}.${{ matrix.cuda_version_min }}
113116
114117
- name: Build and run unit tests
115118
run: |
116-
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}
119+
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}/cuda-${{ matrix.cuda_version_maj }}.${{ matrix.cuda_version_min }}
117120
export CUQUANTUM_SDK=$(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
118121
cmake . -BBuild \
119122
-DPL_BACKEND=lightning_gpu \
@@ -123,15 +126,15 @@ jobs:
123126
-DBUILD_TESTS=ON \
124127
-DENABLE_LAPACK=ON \
125128
-DCMAKE_CXX_COMPILER=mpicxx \
126-
-DCMAKE_CUDA_COMPILER="/usr/local/cuda/bin/nvcc" \
129+
-DCMAKE_CUDA_COMPILER=$(which nvcc) \
127130
-DCMAKE_CUDA_ARCHITECTURES="86" \
128131
-DPython_EXECUTABLE:FILE="${{ steps.python_path.outputs.python }}" \
129132
-G Ninja
130133
cmake --build ./Build
131134
cd ./Build
132135
mkdir -p ./tests/results
133136
for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results/report_$file.xml; done;
134-
for file in *runner_mpi ; do /opt/mpi/${{ matrix.mpilib }}/bin/mpirun -np 2 ./$file --order lex --reporter junit --out ./tests/results/report_$file.xml; done;
137+
for file in *runner_mpi ; do mpirun -np 2 ./$file --order lex --reporter junit --out ./tests/results/report_$file.xml; done;
135138
lcov --directory . -b ../pennylane_lightning/src --capture --output-file coverage.info
136139
lcov --remove coverage.info '/usr/*' --output-file coverage.info
137140
mv coverage.info coverage-${{ github.job }}-lightning_gpu_${{ matrix.mpilib }}.info
@@ -171,6 +174,8 @@ jobs:
171174
max-parallel: 1
172175
matrix:
173176
mpilib: ["mpich", "openmpi"]
177+
cuda_version_maj: ["12"]
178+
cuda_version_min: ["2"]
174179
timeout-minutes: 30
175180

176181
steps:
@@ -232,9 +237,9 @@ jobs:
232237
233238
- name: Install required packages
234239
run: |
235-
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}
240+
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}/cuda-${{ matrix.cuda_version_maj }}.${{ matrix.cuda_version_min }}
236241
python -m pip install -r requirements-dev.txt
237-
python -m pip install custatevec-cu11 mpi4py openfermionpyscf
242+
python -m pip install custatevec-cu${{ matrix.cuda_version_maj }} mpi4py openfermionpyscf
238243
SKIP_COMPILATION=True PL_BACKEND=lightning_qubit python -m pip install -e . -vv
239244
240245
- name: Checkout PennyLane for release build
@@ -256,25 +261,25 @@ jobs:
256261
env:
257262
CUQUANTUM_SDK: $(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
258263
run: |
259-
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}
260-
CMAKE_ARGS="-DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx -DENABLE_MPI=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc -DCMAKE_CUDA_ARCHITECTURES=${{ env.CI_CUDA_ARCH }} -DPython_EXECUTABLE=${{ steps.python_path.outputs.python }}" \
264+
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}/cuda-${{ matrix.cuda_version_maj }}.${{ matrix.cuda_version_min }}
265+
CMAKE_ARGS="-DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx -DENABLE_MPI=ON -DCMAKE_CUDA_COMPILER=$(which nvcc) -DCMAKE_CUDA_ARCHITECTURES=${{ env.CI_CUDA_ARCH }} -DPython_EXECUTABLE=${{ steps.python_path.outputs.python }}" \
261266
PL_BACKEND=lightning_gpu python -m pip install -e . --verbose
262267
263268
# There are issues running py-cov with MPI. A solution is to use coverage as reported
264269
# [here](https://github.com/pytest-dev/pytest-cov/issues/237#issuecomment-544824228)
265270
- name: Run unit tests for MPI-enabled lightning.gpu device
266271
run: |
267-
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}
268-
PL_DEVICE=lightning.gpu /opt/mpi/${{ matrix.mpilib }}/bin/mpirun -np 2 \
272+
source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}/cuda-${{ matrix.cuda_version_maj }}.${{ matrix.cuda_version_min }}
273+
PL_DEVICE=lightning.gpu mpirun -np 2 \
269274
coverage run --rcfile=.coveragerc --source=pennylane_lightning -p -m mpi4py -m pytest ./mpitests --tb=native
270275
coverage combine
271-
coverage xml -o coverage-${{ github.job }}-lightning_gpu_${{ matrix.mpilib }}-main.xml
276+
coverage xml -o coverage-${{ github.job }}-lightning_gpu_${{ matrix.mpilib }}_cu${{ matrix.cuda_version_maj }}-main.xml
272277
273278
- name: Upload code coverage results
274279
uses: actions/upload-artifact@v3
275280
with:
276281
name: ubuntu-codecov-results-python
277-
path: coverage-${{ github.job }}-lightning_gpu_${{ matrix.mpilib }}-*.xml
282+
path: coverage-${{ github.job }}-lightning_gpu_${{ matrix.mpilib }}_cu${{ matrix.cuda_version_maj }}-*.xml
278283
if-no-files-found: error
279284

280285
- name: Cleanup

0 commit comments

Comments
 (0)