Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TEST.md Refresh #2521

Merged
merged 3 commits into from
Sep 1, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 38 additions & 52 deletions docs/TESTS.md
Original file line number Diff line number Diff line change
@@ -1,53 +1,58 @@
# Tests

## Installing `pytest`
## Tools to install

We recommend you install [pytest](http://pytest.org/en/latest/) and
[pytest-cache](http://pythonhosted.org/pytest-cache/). `pytest` is a testing
tool that will give you more flexibility over running your unit tests.
We recommend you install [pytest](http://pytest.org/en/latest/) with the following plugins:

To install `pytest`, run the following command:
- [pytest-cache](http://pythonhosted.org/pytest-cache/)
- [pytest-subtests](https://github.com/pytest-dev/pytest-subtests)
- [pytest-pylint](https://github.com/carsongee/pytest-pylint)

The PyTest [Getting Started Guide](https://docs.pytest.org/en/latest/getting-started.html) has quick general instructions, although they do not cover installing the plugins.
Continue reading below for plugin installation.

We also recommend [pylint](https://pylint.pycqa.org/en/latest/user_guide/), as it is part of our automated feedback on the website, and can be a very useful (if noisy!) code analysis tool.

Pylint can be a bit much, so this [tutorial](https://pylint.pycqa.org/en/latest/tutorial.html) can be helpful for getting started, as can this overview of [Code Quality: Tools and Best Practices](https://realpython.com/python-code-quality/) from Real Python.
Finally, [this site])(https://pycodequ.al/docs/pylint-messages.html) is a great place to look up more human-readable descriptions of Pylint linting messages.


### Installing `pytest`

To install `pytest`, run the following command in the environment (_active `venv`, `conda env`, `docker container`, `vm` or other project space with Python 3.8 installed_):

```bash
pip3 install pytest pytest-cache
pip3 install pytest pytest-cache pytest-subtests pytest-pylint
```

If you get a `command not found` response from your system, you can find a
tutorial on how to install `pip`
[here](https://pip.pypa.io/en/stable/installing/).

If you want to check what the default version of `pytest` being used is, run the following:
Once the installation has completed, you can check what the default version of `pytest` is by running the following:

```bash
pytest --version
```

If you choose not to install `pytest`, you can still run tests individually and
skip the rest of this tutorial:
## Testing

```bash
cd exercism/python/bob
python bob_test.py
```

## Running the Tests
### Run All Tests for an Exercise

### Run All Tests

To run all tests for a specific exercise (we will take the `bob.py` exercise as
an example here), place yourself in the directory where that exercise has been
fetched and run:
To run all tests for a specific exercise (_we will take the `bob.py` exercise as
an example here_), navigate to the directory where that exercise has been
downloaded and run the following in the terminal:

```bash
pytest bob_test.py
```

**Note:** To run the tests you need to pass the name of the testsuite file to
`pytest` (generally, the file ending with `_test.py`), **NOT** the file you
created to solve the problem (which is your _implementation_). This is because
in the latter case, since there are no defined test cases in your
implementation, `pytest` will just return a positive result, specifying that
it ran zero tests. Like this:
`pytest` (_generally, this will be the file ending with `_test.py`_), **NOT** the file that was
created to solve the exercsim problem (which is the _solution implementation_).
`PyTest` needs the test definitions in the `_test.py` file in order to know how to call, run, and exercise the _solution implementation_ code.
Since there are no test cases defined in the _solution implementation_, `pytest` will just return a positive result, specifying that it found and ran zero tests. Like this:


```
============================= bob.py ==============================
Expand All @@ -59,8 +64,13 @@ Ran 0 tests in 0.000s
OK
```


### More `pytest` Examples

Below are some additional examples and details for getting up and running quickly with Pytest.
[How to invoke pytest](https://docs.pytest.org/en/latest/how-to/usage.html#usage) and [pytest command-line flags](https://docs.pytest.org/en/latest/reference/reference.html#command-line-flags) offer full details on all of pytests run options.


#### Stop After First Failure
The above will run all the tests, whether they fail or not. If you'd rather stop
the process and exit on the first failure, run:
Expand All @@ -86,7 +96,7 @@ pytest

## Recommended Workflow

We recommend you run this command while working on exercises.
Try this command while working on exercises:

```bash
cd exercism/python/bob
Expand All @@ -95,34 +105,10 @@ pytest -x --ff bob_test.py

## PDB

Will drop you into the python debugger when a test fails. To learn how to use
pdb, check out the
Typing pdb on the command line will drop you into the python debugger when a test fails.
To learn how to usepdb, check out the
[documentation](https://docs.python.org/3/library/pdb.html#debugger-commands).

```bash
pytest --pdb bob_test.py
```

You may also be interested in watching [Clayton Parker's "So you think you can
pdb?" PyCon 2015 talk](https://www.youtube.com/watch?v=P0pIW5tJrRM).

## PEP8

PEP8 is the [Style Guide for Python
Code](https://www.python.org/dev/peps/pep-0008/). If you would like to test for
compliance to the style guide, install
[pytest-pep8](https://pypi.python.org/pypi/pytest-pep8).

```bash
pip install pytest-pep8
```

Then, just add the `--pep8` flag to your command

```bash
pytest --pep8 bob_test.py
```

Read the [pytest documentation](https://docs.pytest.org/en/latest/contents.html) and
[pytest-cache](http://pythonhosted.org/pytest-cache/) documentation to learn
more.
10 changes: 6 additions & 4 deletions exercises/shared/.docs/help.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@

Below are some resources for getting help if you run into trouble:

- [Exercism on Gitter](https://gitter.im/exercism/home) join the Python room for Python-related questions or problems.
- [The PSF](https://www.python.org) hosts Python downloads, documentation, and community resources.
- [Python Community on Discord](https://pythondiscord.com/) is a very helpful and active community.
- [#python on Libera.chat](https://www.python.org/community/irc/)
- [#python on Libera.chat](https://www.python.org/community/irc/) this is where the cored developers for the language hang out and get work done.
- [Exercism on Gitter](https://gitter.im/exercism/home) join the Python room for Python-related questions or problems.
- [/r/learnpython/](https://www.reddit.com/r/learnpython/) is a subreddit designed for Python learners.
- [Python Community Forums](https://discuss.python.org/)
- [Pythontutor](http://pythontutor.com/) for stepping through small code snippets visually.
- [The PSF Website](https://www.python.org) hosts Python downloads, documentation, and community resources.

Additionally, [StackOverflow](http://stackoverflow.com/questions/tagged/python) is a good spot to search for your problem/question to see if it has been answered already. If not - you can always [ask](https://stackoverflow.com/help/how-to-ask) or [answer](https://stackoverflow.com/help/how-to-answer) someone else's question.

Additionally, [StackOverflow](http://stackoverflow.com/questions/tagged/python) is a good spot to search for your problem/question to see if it has been answered already.
If not - you can always [ask](https://stackoverflow.com/help/how-to-ask) or [answer](https://stackoverflow.com/help/how-to-answer) someone else's question.
18 changes: 9 additions & 9 deletions exercises/shared/.docs/tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,21 @@ You can also tell Python to run the pytest module on the command line from eithe

`python -m pytest /fully/qualified/path/to/<exercisename>/` OR `python -m pytest realtive/path/to/<exercisename>` from a non-exercise directory.

Many IDE's and code editors also have built-in spport for using PyTest to run tests.
Many IDE's and code editors also have built-in support for using PyTest to run tests.

[Visual Studio Code](https://code.visualstudio.com/docs/python/testing)
[PyCharm Professional & Community Editions](https://www.jetbrains.com/help/pycharm/pytest.html#create-pytest-test)
[Atom](https://atom.io/packages/atom-python-test)
[Spyder](https://www.spyder-ide.org/blog/introducing-unittest-plugin/)
[Sublime](https://github.com/kaste/PyTest)
[vim-test](https://github.com/vim-test/vim-test)
- [Visual Studio Code](https://code.visualstudio.com/docs/python/testing)
- [PyCharm Professional & Community Editions](https://www.jetbrains.com/help/pycharm/pytest.html#create-pytest-test)
- [Atom](https://atom.io/packages/atom-python-test)
- [Spyder](https://www.spyder-ide.org/blog/introducing-unittest-plugin/)
- [Sublime](https://github.com/kaste/PyTest)
- [vim-test](https://github.com/vim-test/vim-test)

See the [Python tests page](https://exercism.io/tracks/python/tests) for more information.
See the [Python tests page](https://github.com/exercism/python/blob/main/docs/TESTS.md) for more information.

### Common `pytest` options

- `-v` : enable verbose output.
- `-x` : stop running tests on first failure.
- `--ff` : run failures from previous test before running other test cases.

For other options, see `python -m pytest -h`. PyTest documentation can be found [here](https://docs.pytest.org/en/6.2.x/getting-started.html).
For other options, see `python -m pytest -h`. PyTest documentation can be found [here](https://docs.pytest.org/en/latest/getting-started.html).