Skip to content

Commit

Permalink
Add seamless support of pytest-xdist:
Browse files Browse the repository at this point in the history
- Set randm seed when smoke-random option is given
- Add INI option to distribute workloads by the smoke scope using the custom distribution algorithm
  • Loading branch information
yugokato committed Dec 20, 2024
1 parent 0978089 commit 1bc4da8
Show file tree
Hide file tree
Showing 13 changed files with 342 additions and 141 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,4 @@ jobs:
- name: Test
shell: bash
run: |
tox -e py --installpkg `find dist/*.tar.gz`
tox -e py,xdist --installpkg `find dist/*.tar.gz`
22 changes: 14 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ pip install pytest-smoke

## Usage

The plugin provides the following options, allowing you to limit the amount of tests to run (`N`) and optionally specify
the scope at which `N` is applied:
The plugin provides the following options to limit the amount of tests to run (`N`, default=`1`) and optionally specify
the scope at which `N` is applied.
If provided, the value of `N` can be either a number (e.g., `5`) or a percentage (e.g., `10%`).
```
$ pytest -h
Expand All @@ -44,10 +45,9 @@ Smoke testing:
NOTE: You can also implement your own custom scopes using a hook
```

> - The value of `N` can be a number (e.g. `5`) or a percentage (e.g. `10%`)
> - If `N` is not explicitly specified, the default value of `1` will be used
> - The `--smoke-scope` option supports any custom values, as long as they are handled in the hook
> - You can overwrite the plugin's default value for `N` and `SCOPE` using INI options. See the "INI Options" section below
> - The `--smoke-scope` option also supports any custom values, as long as they are handled in the hook
> - You can override the plugin's default value for `N` and `SCOPE` using INI options. See the "INI Options" section below
> - When using the `pytest-xdist` plugin for parallel testing, you can configure the `pytest-smoke` plugin to enable a custom distribution algorithm that distributes tests based on the smoke scope

## Examples
Expand Down Expand Up @@ -234,7 +234,7 @@ tests/test_something.py::test_something3[17] PASSED [100%]
The plugin provides the following hooks to customize or extend the plugin's capabilities:

### `pytest_smoke_generate_group_id(item, scope)`
This hook allows you to implement your own custom scopes for the `--scope-smoke` option, or overwrite the logic of the
This hook allows you to implement your own custom scopes for the `--smoke-scope` option, or override the logic of the
predefined scopes. Items with the same group ID are grouped together and are considered to be in the same scope,
at which `N` is applied. Any custom values passed to the `--smoke-scope` option must be handled in this hook.

Expand All @@ -250,7 +250,7 @@ selected.

## INI Options

You can overwrite the plugin's default values by setting the following options in a configuration
You can override the plugin's default values by setting the following options in a configuration
file (pytest.ini, pyproject.toml, etc.).

### `smoke_default_n`
Expand All @@ -260,3 +260,9 @@ Plugin default: `1`
### `smoke_default_scope`
The default smoke scope to be applied when not explicitly specified with the `--smoke-scope` option.
Plugin default: `function`

### `smoke_xdist_dist_by_scope`
When using the `pytest-xdist` plugin (>=2.3.0) for parallel testing, a custom distribution algorithm that
distributes tests based on the smoke scope can be enabled. The custom scheduler will be automatically used when
the `-n`/`--numprocesses` option is used.
Plugin default: `false`
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ dev = [
"pytest-xdist[psutil]>=2.5.0,<4",
"ruff==0.8.3",
"tox>=4.0.0,<5",
"tox-uv>=1.0.0,<2"
"tox-uv>=1.0.0,<2",
]

[project.urls]
Expand Down
22 changes: 21 additions & 1 deletion src/pytest_smoke/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,25 @@
try:
__version__ = version("pytest-smoke")
except PackageNotFoundError:
# package is not installed
pass


class PytestSmoke:
def __init__(self):
try:
from xdist import __version__ as __xdist_version__

self._is_xdist_installed = __xdist_version__ >= "2.3.0"
except ImportError:
self._is_xdist_installed = False

@property
def is_xdist_installed(self) -> bool:
return self._is_xdist_installed

@is_xdist_installed.setter
def is_xdist_installed(self, v: bool):
self._is_xdist_installed = v


smoke = PytestSmoke()
Empty file.
39 changes: 39 additions & 0 deletions src/pytest_smoke/extensions/xdist.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
from pytest import Config, Item, Session, hookimpl

from pytest_smoke import smoke
from pytest_smoke.types import SmokeIniOption
from pytest_smoke.utils import generate_group_id, get_scope, parse_ini_option

if smoke.is_xdist_installed:
from xdist import is_xdist_controller
from xdist.scheduler import LoadScopeScheduling

class PytestSmokeXdist:
"""A plugin that extends pytest-smoke to seamlesslly support pytest-xdist"""

name = "smoke-xdist"

def __init__(self):
self._nodes = None

@hookimpl(tryfirst=True)
def pytest_collection(self, session: Session):
if is_xdist_controller(session):
self._nodes = {item.nodeid: item for item in session.perform_collect()}
return True

def pytest_xdist_make_scheduler(self, config: Config, log):
if parse_ini_option(config, SmokeIniOption.SMOKE_XDIST_DIST_BY_SCOPE):
return SmokeScopeScheduling(config, log, nodes=self._nodes)

class SmokeScopeScheduling(LoadScopeScheduling):
"""A custom pytest-xdist scheduler that distributes workloads by smoke scope groups"""

def __init__(self, config: Config, log, *, nodes: dict[str, Item]):
super().__init__(config, log)
self._scope = get_scope(self.config)
self._nodes = nodes

def _split_scope(self, nodeid: str) -> str:
item = self._nodes[nodeid]
return generate_group_id(item, self._scope) or super()._split_scope(nodeid)
2 changes: 1 addition & 1 deletion src/pytest_smoke/hooks.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
def pytest_smoke_generate_group_id(item: Item, scope: str):
"""Return a smoke scope group ID for the predefined or custom scopes
Use this hook to either overwrite the logic of the predefined scopes or to implement logic for your own scopes
Use this hook to either override the logic of the predefined scopes or to implement logic for your own scopes
NOTE: Any custom scopes given to the --smoke-scope option must be handled in this hook
"""

Expand Down
Loading

0 comments on commit 1bc4da8

Please sign in to comment.