mirror of
https://github.com/home-assistant/developers.home-assistant.git
synced 2025-07-16 05:46:30 +00:00
Add snapshot testing (#1690)
This commit is contained in:
parent
e9a1489279
commit
234fd01d65
59
blog/2023-02-20-snapshot-testing.md
Normal file
59
blog/2023-02-20-snapshot-testing.md
Normal file
@ -0,0 +1,59 @@
|
||||
---
|
||||
author: Franck Nijhof
|
||||
authorURL: https://twitter.com/frenck
|
||||
authorImageURL: /img/profile/frenck.png
|
||||
authorTwitter: frenck
|
||||
title: Added support for snapshot testing
|
||||
---
|
||||
|
||||
Home Assistant [now supports snapshot testing](https://github.com/home-assistant/core/pull/88323)
|
||||
for our Python codebase.
|
||||
|
||||
Snapshot testing (also known as approval tests) are tests that assert values
|
||||
against a stored reference value (the snapshot); ensuring the output of code
|
||||
remains the same over time.
|
||||
|
||||
Snapshot tests are different from regular (functional) tests and do not replace
|
||||
functional tests, but they can be very useful for testing larger test outputs.
|
||||
Within Home Assistant they could, for example, be used to test entity states, device
|
||||
or entity registry items, or the output of a diagnostic dump.
|
||||
|
||||
Take, for example, this diagnostic test, which uses a snapshot to assert the
|
||||
output of a diagnostic dump:
|
||||
|
||||
```python
|
||||
# tests/components/example/test_diagnostics.py
|
||||
async def test_diagnostics(
|
||||
hass: HomeAssistant,
|
||||
hass_client: ClientSessionGenerator,
|
||||
init_integration: MockConfigEntry,
|
||||
snapshot: SnapshotAssertion,
|
||||
) -> None:
|
||||
"""Test diagnostics."""
|
||||
assert (
|
||||
await get_diagnostics_for_config_entry(hass, hass_client, init_integration)
|
||||
== snapshot
|
||||
)
|
||||
```
|
||||
|
||||
When this test is run for the first time, it will fail, as no snapshot exists.
|
||||
To create (or update) a snapshot, run pytest with the `--snapshot-update` flag,
|
||||
which will create a snapshot file in the `snapshots` directory of this component.
|
||||
|
||||
The snapshot file is named after the test file, in this case:
|
||||
`tests/components/example/snapshots/test_diagnostics.ambr`. The snapshot files
|
||||
are human-readable and must be committed to the repository.
|
||||
|
||||
Any sequential runs of the tests will then compare the results against the
|
||||
snapshot. If the results are different, the test will fail.
|
||||
|
||||
Snapshots are an easy way to ensure the output of code remains the same over
|
||||
time and can greatly reduce the amount of testing code needed (while providing)
|
||||
a full assert against a complete output.
|
||||
|
||||
Snapshot testing in Home Assistant is build on top of [Syrupy](https://github.com/tophat/syrupy),
|
||||
which has been extended to handle Home Assistant-specific data structures.
|
||||
|
||||
More information on testing integrations,
|
||||
can be found [in our documentation](/docs/development_testing).
|
||||
|
@ -105,3 +105,69 @@ If you can't avoid a PyLint warning, add a comment to disable the PyLint check f
|
||||
- Modify a `ConfigEntry` via the config entries interface [`hass.config_entries`](https://github.com/home-assistant/core/blob/4cce724473233d4fb32c08bd251940b1ce2ba570/homeassistant/config_entries.py#L570).
|
||||
- Assert the state of a config entry via the [`ConfigEntry.state`](https://github.com/home-assistant/core/blob/4cce724473233d4fb32c08bd251940b1ce2ba570/homeassistant/config_entries.py#L169) attribute.
|
||||
- Mock a config entry via the `MockConfigEntry` class in [`tests/common.py`](https://github.com/home-assistant/core/blob/4cce724473233d4fb32c08bd251940b1ce2ba570/tests/common.py#L658)
|
||||
|
||||
### Snapshot testing
|
||||
|
||||
Home Assistant supports a testing concept called snapshot testing (also known
|
||||
as approval tests), which are tests that assert values against a stored
|
||||
reference value (the snapshot).
|
||||
|
||||
Snapshot tests are different from regular (functional) tests and do not replace
|
||||
functional tests, but they can be very useful for testing larger test outputs.
|
||||
Within Home Assistant they could, for example, be used to:
|
||||
|
||||
- Ensure the output of an entity state is and remains as expected.
|
||||
- Ensure an area, config, device, entity, or issue entry in the registry is and
|
||||
remains as expected.
|
||||
- Ensure the output of a diagnostic dump is and remains as expected.
|
||||
- Ensure a FlowResult is and remains as expected.
|
||||
|
||||
And many more cases that have large output, like JSON, YAML, or XML results.
|
||||
|
||||
The big difference between snapshot tests and regular tests is that the results
|
||||
are captured by running the tests in a special mode that creates the snapshots.
|
||||
Any sequential runs of the tests will then compare the results against the
|
||||
snapshot. If the results are different, the test will fail.
|
||||
|
||||
Snapshot testing in Home Assistant is built on top of [Syrupy](https://github.com/tophat/syrupy),
|
||||
their documentation can thus be applied when writing Home Assistant tests.
|
||||
This is a snapshot test that asserts the output of an entity state:
|
||||
|
||||
```python
|
||||
# tests/components/example/test_sensor.py
|
||||
async def test_sensor(
|
||||
hass: HomeAssistant,
|
||||
snapshot: SnapshotAssertion,
|
||||
) -> None:
|
||||
"""Test the sensor state."""
|
||||
state = hass.states.get("sensor.whatever")
|
||||
assert state == snapshot
|
||||
```
|
||||
|
||||
When this test is run for the first time, it will fail, as no snapshot exists.
|
||||
To create (or update) a snapshot, run the test with
|
||||
the `--snapshot-update` flag:
|
||||
|
||||
```shell
|
||||
pytest tests/components/example/test_sensor.py --snapshot-update
|
||||
```
|
||||
|
||||
This will create a snapshot file in the `tests/components/example/snapshots`.
|
||||
The snapshot file is named after the test file, in this case `test_sensor.ambr`,
|
||||
and is human-readable. The snapshot files must be committed to the repository.
|
||||
|
||||
When the test is run again (without the update flag), it will compare the
|
||||
results against the stored snapshot and everything should pass.
|
||||
|
||||
When the test results change, the test will fail and the snapshot needs to be
|
||||
updated again.
|
||||
|
||||
Use snapshot testing with care! As it is very easy to create a snapshot,
|
||||
it can be tempting to assert everything against a snapshot. However, remember,
|
||||
it is not a replacement for functional tests.
|
||||
|
||||
As an example, when testing if an entity would go unavailable when the device
|
||||
returns an error, it is better to assert the specific change you expected:
|
||||
Assert the state of the entity became `unavailable`. This functional test is a
|
||||
better approach than asserting the full state of such an entity using a
|
||||
snapshot, as it assumes it worked as expected (when taking the snapshot).
|
||||
|
Loading…
x
Reference in New Issue
Block a user