Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-117294: Report DocTestCase as skipped if all examples in the doctest are skipped #117297

Merged
merged 3 commits into from
Mar 28, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions Doc/library/doctest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1021,7 +1021,8 @@ from text files and modules with doctests:
and runs the interactive examples in each file. If an example in any file
fails, then the synthesized unit test fails, and a :exc:`failureException`
exception is raised showing the name of the file containing the test and a
(sometimes approximate) line number.
(sometimes approximate) line number. If all the examples in a file are
skipped, then the synthesized unit test is also marked as skipped.

Pass one or more paths (as strings) to text files to be examined.

Expand Down Expand Up @@ -1087,7 +1088,8 @@ from text files and modules with doctests:
and runs each doctest in the module. If any of the doctests fail, then the
synthesized unit test fails, and a :exc:`failureException` exception is raised
showing the name of the file containing the test and a (sometimes approximate)
line number.
line number. If all the examples in a docstring are skipped, then the
synthesized unit test is also marked as skipped.

Optional argument *module* provides the module to be tested. It can be a module
object or a (possibly dotted) module name. If not specified, the module calling
Expand Down
7 changes: 4 additions & 3 deletions Lib/doctest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2262,12 +2262,13 @@ def runTest(self):

try:
runner.DIVIDER = "-"*70
failures, tries = runner.run(
test, out=new.write, clear_globs=False)
results = runner.run(test, out=new.write, clear_globs=False)
if results.skipped == results.attempted:
raise unittest.SkipTest(f"all examples were skipped")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f-string is not needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

finally:
sys.stdout = old

if failures:
if results.failed:
raise self.failureException(self.format_failure(new.getvalue()))

def format_failure(self, err):
Expand Down
49 changes: 49 additions & 0 deletions Lib/test/test_doctest/sample_doctest_skip.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
"""This is a sample module used for testing doctest.

This module includes various scenarios involving skips.
"""

def no_skip_pass():
"""
>>> 2 + 2
4
"""

def no_skip_fail():
"""
>>> 2 + 2
5
"""

def single_skip():
"""
>>> 2 + 2 # doctest: +SKIP
4
"""

def double_skip():
"""
>>> 2 + 2 # doctest: +SKIP
4
>>> 3 + 3 # doctest: +SKIP
6
"""

def partial_skip_pass():
"""
>>> 2 + 2 # doctest: +SKIP
4
>>> 3 + 3
6
"""

def partial_skip_fail():
"""
>>> 2 + 2 # doctest: +SKIP
4
>>> 2 + 2
5
"""

def no_examples():
"""A docstring with no examples should not be counted as run or skipped."""
22 changes: 22 additions & 0 deletions Lib/test/test_doctest/test_doctest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2247,6 +2247,16 @@ def test_DocTestSuite():
>>> suite.run(unittest.TestResult())
<unittest.result.TestResult run=0 errors=0 failures=0>

If all examples in a docstring are skipped, unittest will report it as a
skipped test:

>>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest_skip')
>>> result = suite.run(unittest.TestResult())
>>> result
<unittest.result.TestResult run=6 errors=0 failures=2>
>>> len(result.skipped)
2

We can use the current module:

>>> suite = test.test_doctest.sample_doctest.test_suite()
Expand Down Expand Up @@ -2418,6 +2428,18 @@ def test_DocFileSuite():
Traceback (most recent call last):
ValueError: Package may only be specified for module-relative paths.

If all examples in a file are skipped, unittest will report it as a
skipped test:

>>> suite = doctest.DocFileSuite('test_doctest.txt',
... 'test_doctest4.txt',
... 'test_doctest_skip.txt')
>>> result = suite.run(unittest.TestResult())
>>> result
<unittest.result.TestResult run=3 errors=0 failures=1>
>>> len(result.skipped)
1

You can specify initial global variables:

>>> suite = doctest.DocFileSuite('test_doctest.txt',
Expand Down
4 changes: 4 additions & 0 deletions Lib/test/test_doctest/test_doctest_skip.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
This is a sample doctest in a text file, in which all examples are skipped.

>>> 2 + 2 # doctest: +SKIP
5
4 changes: 2 additions & 2 deletions Lib/test/test_zipimport_support.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
# Retrieve some helpers from other test cases
from test.test_doctest import (test_doctest,
sample_doctest, sample_doctest_no_doctests,
sample_doctest_no_docstrings)
sample_doctest_no_docstrings, sample_doctest_skip)


def _run_object_doctest(obj, module):
Expand Down Expand Up @@ -110,7 +110,7 @@ def test_doctest_issue4197(self):
# The sample doctest files rewritten to include in the zipped version.
sample_sources = {}
for mod in [sample_doctest, sample_doctest_no_doctests,
sample_doctest_no_docstrings]:
sample_doctest_no_docstrings, sample_doctest_skip]:
src = inspect.getsource(mod)
src = src.replace("test.test_doctest.test_doctest", "test_zipped_doctest")
# Rewrite the module name so that, for example,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
A ``DocTestCase`` now reports as skipped if all examples in the doctest are
skipped.
Loading