Introduce pytest-splits (#696)

* Try update lightning CI paths

* Auto update version

* Update kokkos tests to trigger on changes to pl core

* Auto update version

* Split tests_linux.yml into cpp/python.

* Auto update version from '0.36.0-dev36' to '0.36.0-dev38'

* Split GPU tests into cpp/python.

* Split MPI-GPU tests into cpp/python.

* Fix LGPU triggers.

* Remove python tests from LGPU-C++.

* Remove upload-to-codecov-linux-cpp from LGPUMPI tests.

* Rename test workflows.

* Update changelog.

* Change group names.

* Use paths-ignore to simplify workflow paths

* Auto update version from '0.36.0-dev38' to '0.36.0-dev39'

* Fix test names.

* you may only define one of `paths` and `paths-ignore` for a single event

* Fix formatting

* Auto update version from '0.36.0-dev39' to '0.36.0-dev40'

* Change to paths-ignore in no-bin tests.

* Auto update version from '0.36.0-dev40' to '0.36.0-dev41'

* trigger ci

* Change error message.

* Split build/test jobs.

* Revert pennylane_lightning/lightning_kokkos/lightning_kokkos.py

* Fix setup.py command

* Comment workflows

* Comment pull_request:

* Copy wheels to github.workspace /wheels

* SKIP_COMPILATION=True

* Revert SKIP_COMPILATION=True

* Remove duplicate python actions.

* Split LQ-test into 5 groups.

* Update requirements-dev.txt

* 2 groups

* 1 group

* 7 groups

* Refactor LKokos tests

* Auto update version from '0.36.0-dev42' to '0.36.0-dev43'

* Fix cache id

* Fix needs

* Remove all builds

* put back CMAKE_PREFIX_PATH

* install fake lightning

* Fix compat paths

* Auto update version from '0.36.0-dev42' to '0.36.0-dev43'

* Revert changes.

* Update changelog

* trigger ci

* build_and_cache_Kokkos_linux.yml not concurrent.

* Remove C++ tests from compat tests (skipped anyways).

* Use ${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }} for concurrency groups.

* PL_BACKEND: ${{ matrix.pl_backend }}

* Update changelog

* Install wheel package.

* Auto update version from '0.36.0-dev43' to '0.36.0-dev44'

* Install scipy wheel in one place.

* Comment job triggers.

* Add LK[cpu] est durations.

* trigger ci

* Use sha independent keys.

* trigger ci

* build_and_cache_Kokkos

* Change LK keys

* ls -l

* Install version nonce

* Do not skip comp.

* Trade ref for sha in key

* Install after PL

* ref ==> sha

* Reorder

* Try LQ workflows.

* Use actions/setup-python

* Revert triggers.

* Provide correct cov path.

* Revert pennylane_lightning/core/src/utils/config.h

* LK uses 5 groups.

* Restore wheels to main

* main/wheels

* Revert cache paths

* Fix ls -l

* fail-on-cache-miss: true

* Introduce env.GITHUB_WORKSPACE

* Use artifacts

* Fix paths

* No ls -l

* Download to github.workspace

* fix wheel name

* WHEEL_NAME

* Add blas.

* Use outputs.

* Send wheel name

* Fix artifact name

* Fix name

* Use WHEEL_NAME var

* artifacts kokkos

* fix wheel name

* Restore bins.

* Clean up

* Split Linux CPU python tests into 2.

* Fix compat workflows.

* Split format and tidy workflows.

* fix lkcpu

* fix concurrency group

* Do not use editable installs.

* Split full kokkos tests except unitary_correct

* Upload LK test durations.

* duration-paths

* Unique test_durations

* Unique test_durations

* Unique test_durations

* Fix path

* Update python_lightning_kokkos_test_durations

* Remove obsolete all and SERIAL tests.

* Update changelog

* Creating the `LightningTensor` device class based on the new API (#671)

* empty commit (triggering CI)

* Auto update version

* Definition of the two front-end classes

* adding the `lightning_tensor` string to the supported backends in `setup.py`

* adding `__init__.py` file to directly import the `lightning_tensor` device class

* re-naming file

* Auto update version

* Creating the first prototype of initial MPS state tensor using `quimb`

* providing the `backend`, `method` parameters and making `wires` optional

* Changing names and structure

* Auto update version

* adding method required by the new device API design

* Auto update version

* using the `kwargs` parameter in `LightningTensor` and `CircuitMPS` in `quimb`

* taking some further inputs from the new device API

* Perhaps decided the overall structure of `LIghtningTensor`

* Auto update version

* adding docs to methods

* temporary changes so that `pylint` does not complain at this stage

* running `isort`

* re-running formatter after `isort`

* re-running formatter after `isort`

* Applying suggested formatting change from CI

* adding tmp unit tests

* Adding `quimb` in `requirements.txt`

* runing `isort` on mps test

* removing `quimb` from requirement and deleting unit tests for `lightning.tensor`

* Auto update version

* re-inserting unit tests with an additional `yml` file

* running isort on quimb test

* changing name of yml file

* preventing error in import

* updating yml file

* inserting `quimb` package in requirements-dev

* strange error with `quimb`

* strange error with `quimb`

* specifying scipy version

* removing installation of scipy from yml file

* removing the new `yml` file

* testing if tests are tested

* Covering all lines in tests

* forgot final line for formatter

* Python formatter on CI complaints

* covering missing lines

* formatter on CI complaints

* Trying not to skip test if Cpp is enabled

* skipping tests if Cpp is enabled

* removing the only line not covered by tests so far

* Auto update version

* Applying suggestions from code review and making the `state` attribute private (new API design)

* Python formatter

* removing params from `QuimbMPS`

* Auto update version

* removing `**kwargs` from `QuimbMPS`

* removing unnecessary param at this stage

* covering test line

* formatter...

* removing param description

* Making `pylint` happy

* forgot new arg in test

* Updating base class and `preprocess` function

* Updating `LightningTensor` class with new names from more advanced PR

* Auto update version

* Auto update version

* Triggering CI

* Auto update version

* Trying to remove pin from `quimb` in `requirements.dev`

* Auto update version

* Auto update version

* Removing infos on derivatives and using config options to pass parameters to interface

* Usual `pylint` failures

* Trying to solve formatting errors

* typo in docstring

* Sunday update: improved docstrings and structure

* Removing method that was supposed to be in next PR

* removing old TODO comment

* Removing changes from the `setup.py` file

* restoring previous format to `setup.py`

* Auto update version from '0.36.0-dev34' to '0.36.0-dev41'

* Auto update version from '0.36.0-dev40' to '0.36.0-dev41'

* Removing kwargs as suggested from code review

* Addressing comments from CR

* Skipping tests if CPP binary is available

* Auto update version from '0.36.0-dev42' to '0.36.0-dev43'

* Auto update version from '0.36.0-dev43' to '0.36.0-dev44'

* Restoring name in changelog (?)

* Increasing time limit for Python tests

* Applying suggestions from code review

---------

Co-authored-by: Dev version update bot <github-actions[bot]@users.noreply.github.com>
Co-authored-by: ringo-but-quantum <[email protected]>

* Increase tolerance for a flaky test (test_single_return_value) (#703)

* update dev version

* increase tolerance

* update changelog

* trigger CIs

* Auto update version from '0.36.0-dev44' to '0.36.0-dev45'

---------

Co-authored-by: ringo-but-quantum <[email protected]>

* Auto update version from '0.36.0-dev45' to '0.36.0-dev46'

* Remove LTensor stuff

* Update changelog

* Revert requirements-dev.txt

* trigger ci

* Update python_lightning_kokkos_test_durations

* Restore 'all' tests in tests_lkcuda_python.yml

* Fix durations paths.

* Try --splitting-algorithm=least_duration

* Try no -n auto lkcpu tests

* One last test.

---------

Co-authored-by: Rashid N H M <[email protected]>
Co-authored-by: Dev version update bot <github-actions[bot]@users.noreply.github.com>
Co-authored-by: ringo-but-quantum <[email protected]>
Co-authored-by: Pietropaolo Frisoni <[email protected]>
Co-authored-by: Amintor Dusko <[email protected]>

Author: 6 people

.github/CHANGELOG.md

@@ -41,6 +41,9 @@
 
 ### Breaking changes
 
+* Split Lightning-Qubit and Lightning-Kokkos CPU Python tests with `pytest-split`. Remove `SERIAL` from Kokkos' `exec_model` matrix. Remove `all` from Lightning-Kokkos' `pl_backend` matrix. Move `clang-tidy` checks to `tidy.yml`. Avoid editable `pip` installations.
+  [(#696)](https://github.com/PennyLaneAI/pennylane-lightning/pull/696)
+  
 * Update `lightning.gpu` and `lightning.kokkos` to raise an error instead of falling back to `default.qubit`.
   [(#689)](https://github.com/PennyLaneAI/pennylane-lightning/pull/689)
 

.github/workflows/build_and_cache_Kokkos_linux.yml

@@ -25,7 +25,7 @@ jobs:
     steps:
       - name: Kokkos execution strategy
         id: exec_model
-        run: echo "exec_model=[\"SERIAL\", \"OPENMP\"]" >> $GITHUB_OUTPUT
+        run: echo "exec_model=[\"OPENMP\"]" >> $GITHUB_OUTPUT
 
       - name: Kokkos version
         id: kokkos_version

.github/workflows/compat-check-latest-latest.yml

@@ -20,9 +20,16 @@ jobs:
       lightning-version: latest
       pennylane-version: latest
     secrets: inherit
-  tests_linux_python:
-    name: Lightning Compatibility test (tests_linux_python) - latest/latest
-    uses: ./.github/workflows/tests_linux_python.yml
+  tests_lqcpu_python:
+    name: Lightning Compatibility test (tests_lqcpu_python) - latest/latest
+    uses: ./.github/workflows/tests_lqcpu_python.yml
+    with:
+      lightning-version: latest
+      pennylane-version: latest
+    secrets: inherit
+  tests_lkcpu_python:
+    name: Lightning Compatibility test (tests_lkcpu_python) - latest/latest
+    uses: ./.github/workflows/tests_lkcpu_python.yml
     with:
       lightning-version: latest
       pennylane-version: latest

.github/workflows/compat-check-latest-stable.yml

@@ -20,9 +20,16 @@ jobs:
       lightning-version: latest
       pennylane-version: stable
     secrets: inherit
-  tests_linux_python:
-    name: Lightning Compatibility test (tests_linux_python) - latest/stable
-    uses: ./.github/workflows/tests_linux_python.yml
+  tests_lqcpu_python:
+    name: Lightning Compatibility test (tests_lqcpu_python) - latest/stable
+    uses: ./.github/workflows/tests_lqcpu_python.yml
+    with:
+      lightning-version: latest
+      pennylane-version: stable
+    secrets: inherit
+  tests_lkcpu_python:
+    name: Lightning Compatibility test (tests_lkcpu_python) - latest/stable
+    uses: ./.github/workflows/tests_lkcpu_python.yml
     with:
       lightning-version: latest
       pennylane-version: stable

.github/workflows/compat-check-release-release.yml

@@ -20,9 +20,16 @@ jobs:
       lightning-version: release
       pennylane-version: release
     secrets: inherit
-  tests_linux_python:
-    name: Lightning Compatibility test (tests_linux_python) - release/release
-    uses: ./.github/workflows/tests_linux_python.yml
+  tests_lqcpu_python:
+    name: Lightning Compatibility test (tests_lqcpu_python) - release/release
+    uses: ./.github/workflows/tests_lqcpu_python.yml
+    with:
+      lightning-version: release
+      pennylane-version: release
+    secrets: inherit
+  tests_lkcpu_python:
+    name: Lightning Compatibility test (tests_lkcpu_python) - release/release
+    uses: ./.github/workflows/tests_lkcpu_python.yml
     with:
       lightning-version: release
       pennylane-version: release

.github/workflows/compat-check-stable-latest.yml

@@ -20,9 +20,16 @@ jobs:
       lightning-version: stable
       pennylane-version: latest
     secrets: inherit
-  tests_linux_python:
-    name: Lightning Compatibility test (tests_linux_python) - stable/latest
-    uses: ./.github/workflows/tests_linux_python.yml
+  tests_lqcpu_python:
+    name: Lightning Compatibility test (tests_lqcpu_python) - stable/latest
+    uses: ./.github/workflows/tests_lqcpu_python.yml
+    with:
+      lightning-version: stable
+      pennylane-version: latest
+    secrets: inherit
+  tests_lkcpu_python:
+    name: Lightning Compatibility test (tests_lkcpu_python) - stable/latest
+    uses: ./.github/workflows/tests_lkcpu_python.yml
     with:
       lightning-version: stable
       pennylane-version: latest

.github/workflows/compat-check-stable-stable.yml

@@ -20,9 +20,16 @@ jobs:
       lightning-version: stable
       pennylane-version: stable
     secrets: inherit
-  tests_linux_python:
-    name: Lightning Compatibility test (tests_linux_python) - stable/stable
-    uses: ./.github/workflows/tests_linux_python.yml
+  tests_lqcpu_python:
+    name: Lightning Compatibility test (tests_lqcpu_python) - stable/stable
+    uses: ./.github/workflows/tests_lqcpu_python.yml
+    with:
+      lightning-version: stable
+      pennylane-version: stable
+    secrets: inherit
+  tests_lkcpu_python:
+    name: Lightning Compatibility test (tests_lkcpu_python) - stable/stable
+    uses: ./.github/workflows/tests_lkcpu_python.yml
     with:
       lightning-version: stable
       pennylane-version: stable

.github/workflows/format.yml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  black:
+  format-python:
     name: Format (Python)
     runs-on: ubuntu-22.04
     steps:
@@ -42,38 +42,3 @@ jobs:
 
       - name: Run formatter
         run: PATH=$PATH:/home/ubuntu/.local/bin/:$(dirname $(which python3))/ ./bin/format --check ./pennylane_lightning/core/src
-
-  tidy-cpp:
-    strategy:
-      matrix:
-        os: [ubuntu-22.04]
-        pl_backend: ["lightning_qubit"]
-
-    name: Tidy (C++)
-    runs-on: ${{ matrix.os }}
-
-    steps:
-      - name: Checkout PennyLane-Lightning
-        uses: actions/checkout@v4
-
-      - name: Set up Python
-        uses: actions/setup-python@v5
-        with:
-          python-version: '3.11'
-
-      - name: Install dependencies
-        run: sudo apt update && sudo apt -y install cmake gcc-11 g++-11 ninja-build libomp-14-dev && python -m pip install -r requirements-dev.txt
-        env:
-          DEBIAN_FRONTEND: noninteractive
-
-      - name: Run clang-tidy compilation
-        run: |
-          cmake -BBuild -G Ninja . \
-            -DENABLE_CLANG_TIDY=ON \
-            -DCLANG_TIDY_BINARY=$(dirname $(which python))/clang-tidy \
-            -DBUILD_TESTS=ON \
-            -DENABLE_WARNINGS=ON \
-            -DPL_BACKEND=${{ matrix.pl_backend }} \
-            -DCMAKE_CXX_COMPILER="$(which g++-11)" \
-            -DCMAKE_C_COMPILER="$(which gcc-11)"
-          cmake --build ./Build