* Rename repository fixture to test_repository
Also don't remove the built-in repositories. The list was incomplete,
and tests don't seem to require that anymore.
* Get rid of StoreType
The type doesn't have much value, we have constant strings anyways.
* Introduce types.py
* Use slug to determine which repository urls to return
* Simplify BuiltinRepository enum
* Mock GitRepo load
* Improve URL handling and repository creation logic
* Refactor update_repositories
* Get rid of get_from_url
It is no longer used in production code.
* More refactoring
* Address pylint
* Introduce is_git_based property to Repository class
Return all git based URLs, including the Core repository.
* Revert "Introduce is_git_based property to Repository class"
This reverts commit dfd5ad79bf.
* Fold type.py into const.py
Align more with how Supervisor code is typically structured.
* Update supervisor/store/__init__.py
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Apply repository remove suggestion
* Fix tests
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Use Docker BuildKit to build addons
* Improve error message as suggested by CodeRabbit
* Fix container.remove() tests missing v=True
* Ignore squash rather than falling back to legacy builder
* Use version rather than tag to avoid confusion in run_command()
* Fix tests differently
* Use PropertyMock like other tests
* Restore position of fix_label fn
* Exempt addon builder image from unsupported checks
* Refactor tests
* Fix tests expecting wrong builder image
* Remove harcoded paths
* Fix tests
* Remove get_addon_host_path() function
* Use docker buildx build rather than docker build
Co-authored-by: Stefan Agner <stefan@agner.ch>
---------
Co-authored-by: Stefan Agner <stefan@agner.ch>
This reverts commit 63fde3b410.
This change introduced another more severe regression, causing all
add-ons that haven't been started since Supervisor startup to cause
errors during their backup. More sophisticated check would have to be
implemented to address edge cases during backups for non-existing
add-ons (or their config actually).
Fixes#5924
* Handle non-existing addon config dir
Since users have access to the root of all add-on config directories,
they can delete the directory of an add-ons at any time. Hence we need
to handle gracefully if it doesn't exist anymore.
* Add pytest
* Recreate aiohttp ClientSession after DNS plug-in load
Create a temporary ClientSession early in case we need to load version
information from the internet. This doesn't use the final DNS setup
and hence might fail to load in certain situations since we don't have
the fallback mechanims in place yet. But if the DNS container image
is present, we'll continue the setup and load the DNS plug-in. We then
can recreate the ClientSession such that it uses the DNS plug-in.
This works around an issue with aiodns, which today doesn't reload
`resolv.conf` automatically when it changes. This lead to Supervisor
using the initial `resolv.conf` as created by Docker. It meant that
we did not use the DNS plug-in (and its fallback capabilities) in
Supervisor. Also it meant that changes to the DNS setup at runtime
did not propagate to the aiohttp ClientSession (as observed in #5332).
* Mock aiohttp.ClientSession for all tests
Currently in several places pytest actually uses the aiohttp
ClientSession and reaches out to the internet. This is not ideal
for unit tests and should be avoided.
This creates several new fixtures to aid this effort: The `websession`
fixture simply returns a mocked aiohttp.ClientSession, which can be
used whenever a function is tested which needs the global websession.
A separate new fixture to mock the connectivity check named
`supervisor_internet` since this is often used through the Job
decorator which require INTERNET_SYSTEM.
And the `mock_update_data` uses the already existing update json
test data from the fixture directory instead of loading the data
from the internet.
* Log ClientSession nameserver information
When recreating the aiohttp ClientSession, log information what
nameservers exactly are going to be used.
* Refuse ClientSession initialization when API is available
Previous attempts to reinitialize the ClientSession have shown
use of the ClientSession after it was closed due to API requets
being handled in parallel to the reinitialization (see #5851).
Make sure this is not possible by refusing to reinitialize the
ClientSession when the API is available.
* Fix pytests
Also sure we don't create aiohttp ClientSession objects unnecessarily.
* Apply suggestions from code review
Co-authored-by: Jan Čermák <sairon@users.noreply.github.com>
---------
Co-authored-by: Jan Čermák <sairon@users.noreply.github.com>
* Check local store repository for changes
Instead of simply assume that the local store repository got changed,
use mtime to check if there have been any changes to the local store.
This mimics a similar behavior to the git repository store updates.
Before this change, we end up in the updated repo code path, which
caused a re-read of all add-ons on every store reload, even though
nothing changed at all. Store reloads are triggered by Home Assistant
Core every 5 minutes.
* Fix pytest failure
Now that we actually only reload metadata if the local store changed
we have to fake the change as well to fix the store manager tests.
* Fix path cache update test for local store repository
* Take root directory into account/add pytest
* Rename utils/__init__.py tests to test_utils_init.py
* Add blockbuster library and find I/O from unit tests
* Fix lint and test issue
* Fixes from feedback
* Avoid modifying webapp object in executor
* Split su options validation and only validate timezone on change
* Move read_text to executor
* switch to async_capture_exception
* Finish moving read_text to executor
* Cover read_bytes and some write_text calls as well
* Fix await issues
* Fix format_message
* Test stub for keeping shared images after update
* Keep shared images on addon update
* ImageNotFound should only skip the one image not all
* Fix tests and nonetype error
* Normalize logic between two cleanup methods
* Add manual_forced option to addon boot config
* Include client library in pull request template
* Add boot_config to api output so frontend can use it
* `manual_forced` to `manual_only`
* Allow adoption of existing data disk
* Fix existing tests
* Add test cases and fix image issues
* Fix addon build test
* Run checks during setup not startup
* Addon load mimics plugin and HA load for docker part
* Default image accessible in except
* Cache existence of addon paths
* Always update submodules
* Switch to an always cached model
* Cache on store addon only
* Fix tests
* refresh_cache to refresh_path_cache
* Fix name change in test
* Move logic into StoreManager
* Pass architecture of installed add-on on update
When using multi-architecture container images, the architecture of the
add-on is not passed to Docker in all cases. This causes the
architecture of the Supervisor container to be used, which potentially
is not supported by the add-on in question.
This commit passes the architecture of the add-on to Docker, so that
the correct image is pulled.
* Call update with architecture
* Also pass architecture on add-on restore
* Fix pytest
* Bad message error marks system as unhealthy
* Finish adding test cases for changes
* Rename test file for uniqueness
* bad_message to oserror_bad_message
* Omit some checks and check for network mounts
Instead of setting the ingress port on install, make sure to set
the port when the add-on gets loaded (on Supervisor startup and
before installation). This is necessary since the dynamic ingress
ports are not stored as part of the add-on data storage themself
but in the ingress data store. So on every Supervisor start the
port needs to be transferred to the add-on model.
Note that we still need to check the port on add-on update since
the add-on potentially added (dynamic) ingress on update. Same
applies to add-on restore (the restored version might use a dynamic
ingress port).
* Remove race with watchdog during backup, restore and update
* Fix pylint issues and test
* Stop after image pull during update
* Add test for max failed attempts for plugin watchdog
* Backup and restore track progress in job
* Change to stage only updates and fix tests
* Leave HA alone if it wasn't restored
* skip check HA stage message when we don't check
* Change to helper to get current job
* Fix tests
* Mark jobs as internal to skip notifying HA
* Extend container image name validator
The current validator allows certain invalid names (e.g. upper
case), but disallows valid cases (such as ttl.sh/myimage).
Improve the container image validator to support more valid
options and at the same time disallow some of the invalid
options.
Note that this is not a complete/perfect validation still. A much
much more sophisticated regex would be necessary to be 100% accurate.
Also we format the string and replace {machine}/{arch} using Python
format strings. In that regard the image format in Supervisor deviates
from the Docker/OCI container image name format.
* Use an actual invalid image name in config validation
* Add job group execution limit option
* Fix pylint issues
* Assign variable before usage
* Cleanup jobs when done
* Remove isinstance check for performance
* Explicitly raise from None
* Add some more documentation info
* Reduce executor code for docker
* Fix pylint errors and move import/export image
* Fix test and a couple other risky executor calls
* Fix dataclass and return
* Fix test case and add one for corrupt docker
* Add some coverage
* Undo changes to docker manager startup
* Addon startup waits for healthy
* fix import for pylint
* wait_for to 5 in tests
* Adjust tests to simplify async tasks
* Remove wait_boot time from addons.boot tests
* Eliminate async task race conditions in tests
* Add host UTS namespace support for Add-Ons
Using the UTS host namespace is useful when running a mDNS responder
which learns the hostname from the gethostname syscall. This way the
add-on can use the system's hostname without further doing.
* Check host_uts default
* Adjust Security rating if host UTS mode and CAP_ADMIN is set
* Don't add hostname to DNS server if UTS namespace is disabled
* Simplify hostname logic
* Update supervisor/docker/addon.py
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
