Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better integration of doctest with unittest and libregrtest #108885

Open
serhiy-storchaka opened this issue Sep 4, 2023 · 0 comments
Open

Better integration of doctest with unittest and libregrtest #108885

serhiy-storchaka opened this issue Sep 4, 2023 · 0 comments
Assignees
Labels
stdlib Python modules in the Lib dir tests Tests in the Lib/test dir type-feature A feature request or enhancement

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Sep 4, 2023

Feature or enhancement

A side effect of migration to the unittest test runner (a long process finished in #89392) was the change in the output in verbose mode. In older version, when test.support.run_doctest() was used to run doctests, it outputted all doctest details when regrtests are run with -v. Now doctests are wrapped in doctest.DocTestClass and only test name is shown when regrtests are run with -v.

The former behavior was too verbose, the current behavior is too terse. I want the following behavior:

  • When regrtests are run without -v, do not output anything.
  • When regrtests are run with single -v, output only doctest name, as for other test cases.
  • When regrtests are run with -vv, output detailed doctest output.
  • Every example count as a subtest.
  • When some example fails, count it as failing subtest, and output detailed reports for every failed example. It should not include details of successful examples (except when run with -vv).
  • It would be nice to integrate it also with the unittest CLI, but taking into account that it needs two -vs to show test case names and will need three -vs to show doctest details.

It means that we should found a way to pass verbosity level to doctests and run every example within TestCase.subTest().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir tests Tests in the Lib/test dir type-feature A feature request or enhancement
Projects
Status: In Progress
Development

No branches or pull requests

2 participants
@serhiy-storchaka and others