How to layout test-files and -folders in a project using multiple import packages?

[buhtz]

Hello,
I want to ask you for your experience based opinion.

I have a Distribution Package (named bitdemo) with two Import Packages (named bitcli and bitgui) inside. The project do use the src-Layout. The linked repo is just a demonstrator.

My question is about how should I organize/layout the test files.
This is not only because of clearness but about separate unit- and integration-tests because IMHO the latter should not count for coverage calculation.

Here are some variants. Which one would you prefer or would you propose a different one?

Variant A

bit_demo
├── pyproject.toml
├── src
│   ├── bitcli
│   │   ├── __init__.py
│   │   └── __main__.py
│   └── bitgui
│       ├── __init__.py
│       └── __main__.py
└── tests
     ├── integration
     └── unit

Variant B

tests
├── bitcli
│   ├── integration
│   └── unit
└── bitgui
     ├── integration
     └── unit

Variant C

bit_demo
├── pyproject.toml
├── src
├── tests_cli
│   ├── integration
│   └── unit
└── tests_gui
     ├── integration
     └── unit

Variant D

The unit-tests related to bitcli don't need bitgui.
But bitgui do need sometimes bitcli because the GUI do use functionality from the its command line version with its basic functionality.
And I also assume that most of the integration tests to involve bitcli and bitgui at the same time. So on the level of integration-tests a separation doesn't make much sense.

tests
├── integration
├── unit_cli
└── unit_gui

[igortg]

Variant D seems the best option, for the reasons you just gave. It also sticks better with the "flat is better than nested" practice.

There's an alternative pattern (that IMHO is particularly good for large code-bases) where each folder has its own tests dir (and that goes recursively). pytest could handle that just fine:

├── src
│   ├── bitcli
│   │   ├── tests  # (unit)
│   │   ├── __init__.py
│   │   └── __main__.py
│   └── bitgui
│       ├── tests  # (unit)
│       ├── __init__.py
│       └── __main__.py
└── tests
     └── integration

[buhtz]

Thanks for that extra pattern.

This IMHO violates the src-Layout which intention is to separate tests from package code. But this is just in a perfect world and of course can differ in the real world. ;)