jeans/developers.home-assistant
jeans/developers.home-assistant
1
0
Fork 0
mirror of https://github.com/home-assistant/developers.home-assistant.git synced 2025-07-20 07:46:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

Not to Tox (#1542)

Browse Source
This commit is contained in:
Franck Nijhof 2022-11-23 14:40:48 +01:00 committed by GitHub
parent 8b5d5c4212
commit 500457acd9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 43 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/config_entries_config_flow_handler.md
View File

@ -322,4 +322,4 @@ Automated tests should verify that the reauth flow updates the existing config e
## Testing your config flow
Integrations with a config flow require full test coverage of all code in `config_flow.py` to be accepted into core. [Test your code](development_testing.md#testing-outside-of-tox) includes more details on how to generate a coverage report.
Integrations with a config flow require full test coverage of all code in `config_flow.py` to be accepted into core. [Test your code](development_testing.md#running-a-limited-test-suite) includes more details on how to generate a coverage report.

84
docs/development_testing.md
View File

@ -7,10 +7,18 @@ As it states in the [Style guidelines section](development_guidelines.md) all co
- All the unit tests pass
- All code passes the checks from the linting tools
Local testing is done using [Tox](https://tox.readthedocs.io), which has been installed as part of running `script/setup` in the [virtual environment](development_environment.mdx). To start the tests, activate the virtual environment and simply run the command:
Local testing is done using [pytest](https://docs.pytest.org/) and using [pre-commit](https://pre-commit.com/) for running out linters, which has been installed as part of running `script/setup` in the [virtual environment](development_environment.mdx).
To run our linters, on the full code base, run the following command:
```shell
tox
pre-commit run --all-files
```
To start the tests, and run the full test suite, activate the virtual environment and run the command:
```shell
pytest tests
```
It might be required that you install additional packages depending on your distribution/operating system:
@ -19,62 +27,45 @@ It might be required that you install additional packages depending on your dist
- Ubuntu: `sudo apt-get install libudev-dev`
:::info Important
Run `tox` before you create your pull request to avoid annoying fixes.
Run `pytest` & `pre-commit` before you create your pull request to avoid annoying fixes.
`pre-commit` will will be invoked automatically by git when commiting changes.
:::
:::note
Running the full `tox` test suite will take quite some time, so as the minimal requirement for pull requests, run at least the tests that are related to your code changes (see details below on how to). The full test suite will anyway be run by the CI once you created your pull request and before it can be merged.
Running the full `ptyest` test suite will take quite some time, so as the minimal requirement for pull requests, run at least the tests that are related to your code changes (see details below on how to). The full test suite will anyway be run by the CI once you created your pull request and before it can be merged.
:::
Running `tox` will run unit tests against the locally available Python releases, as well as validate the code and document style using `pycodestyle`, `pydocstyle` and `pylint`. You can run tests on only one `tox` target -- just use `-e` to select an environment. For example, `tox -e lint` runs the linters only, and `tox -e py39` runs unit tests only on Python 3.9.
`tox` uses virtual environments under the hood to create isolated testing environments. The `tox` virtual environments will get out-of-date when requirements change, causing test errors. Run `tox -r` to tell `tox` to recreate the virtual environments.
macOS users may see an `Error creating virtualenv` when running `tox`. If this occurs, install the [tox-venv](https://pypi.org/project/tox-venv/) package using the command `pip install tox-venv` and try again.
Running `pytest` will run unit tests against the locally available Python version. We run our tests in our CI against all our supported Python versions.
### Adding new dependencies to test environment
If you are working on tests for an integration and you need the dependencies available inside the `tox` environment, update the list inside `script/gen_requirements_all.py`. Then run the script and then run `tox -r` to recreate the virtual environments.
### Running single tests using `tox`
You can pass arguments via `tox` to `pytest` to be able to run single test suites or test files. Replace `py39` with the Python version that you use.
```shell
# Stop after the first test fails
$ tox -e py39 -- tests/test_core.py -x
# Run test with specified name
$ tox -e py39 -- tests/test_core.py -k test_split_entity_id
# Fail a test after it runs for 2 seconds
$ tox -e py39 -- tests/test_core.py --timeout 2
# Show the 10 slowest tests
$ tox -e py39 -- tests/test_core.py --duration=10
```
### Testing outside of Tox
Running `tox` will invoke the full test suite. Even if you specify which tox target to run, you still run all tests inside that target. That's not very convenient to quickly iterate on your code! To be able to run the specific test suites without `tox`, you'll need to install the test dependencies into your Python environment:
If you are working on tests for an integration and you changed the dependencies, then run the `script/gen_requirements_all.py` script to update all requirement files.
Next you can update all dependencies in your development environment by running:
```shell
pip3 install --use-deprecated=legacy-resolver -r requirements_test_all.txt -c homeassistant/package_constraints.txt
```
### Running a limited test suite
Now that you have all test dependencies installed, you can run tests on individual files:
You can pass arguments to `pytest` to be able to run single test suites or test files.
Here are some helpful commands:
```shell
flake8 homeassistant/core.py
pylint homeassistant/core.py
pydocstyle homeassistant/core.py
pytest tests/test_core.py
# Stop after the first test fails
$ pytest tests/test_core.py -x
# Run test with specified name
$ pytest tests/test_core.py -k test_split_entity_id
# Fail a test after it runs for 2 seconds
$ pytest tests/test_core.py --timeout 2
# Show the 10 slowest tests
$ pytest tests/test_core.py --duration=10
```
You can also run linting tests against all changed files, as reported by `git diff upstream/dev... --diff-filter=d --name-only`, using the `lint` script:
```shell
script/lint
```
In case you want to check the code coverage for your new component, run the following from the root of the repository:
If you want to test just your integration, and include a test coverage report,
the following command is recommended:
```shell
pytest ./tests/components/<your_component>/ --cov=homeassistant.components.<your_component> --cov-report term-missing -vv
@ -84,12 +75,21 @@ pytest ./tests/components/<your_component>/ --cov=homeassistant.components.<your
Several linters are setup to run automatically when you try to commit as part of running `script/setup` in the [virtual environment](development_environment.mdx).
You can also run these linters manually:
You can also run these linters manually :
```shell
pre-commit run --show-diff-on-failure
```
The linters are also available directly, you can run tests on individual files:
```shell
flake8 homeassistant/core.py
pylint homeassistant/core.py
black homeassistant/core.py
isort homeassistant/core.py
```
### Notes on PyLint and PEP8 validation
If you can't avoid a PyLint warning, add a comment to disable the PyLint check for that line with `# pylint: disable=YOUR-ERROR-NAME`. Example of an unavoidable one is if PyLint incorrectly reports that a certain object doesn't have a certain member.