* Store and persist OS upgrade map to fix update path evaluation
The existing logic calculated OS upgrade paths inline during fetch_data,
which will not get reevaluted when the current OS is unsupported
(JobCondition.OS_SUPPORTED). E.g. after updating from 11.4 to 11.5, the
system wouldn't offer the next available update (15.2) because the
upgrade path calculation relied on fresh data from the blocked fetch
operation.
Changes:
- Add ATTR_HASSOS_UPGRADE constant and schema validation
- Store hassos-upgrade map from version JSON in updater data
- Refactor version_hassos property to use stored upgrade map instead of
inline calculation during fetch_data
- Maintain upgrade path logic: upgrade within major version first, then
jump to next major version when at the latest in current major
- Add type safety checks for version.major access
This ensures upgrade paths work correctly even when update data refresh
is blocked due to unsupported OS versions, fixing the scenario where
HAOS 11.5 wouldn't show 15.2 as the next available update.
* Update supervisor/updater.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Address mypy issue
* Fix pytest
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add availability API for addons
* Add cast back and test for latest version of installed addon
* Make error responses more translation/client library friendly
* Add test cases for install/update APIs
Running tests in UTC+2 timezone makes some of the tests fail because the
mocked time in the future is actually in the past, as UTC is used as the
new reference point. Adjust the tests to mock also the time when the
first execution of function happens.
Instances where the second execution happened "immediately" were mocked
to happen 1ms later. The 1ms delta is also needed to be added when
mocking time 1h in the future, otherwise it will be throttled too.
* Add background option to update/install APIs
* Refactor to use common background_task utility in backups too
* Use a validation_complete event rather then looking for bus events
* Handle missing type attribute in add-on map config
Handle missing type attribute in the add-on `map` configuration key.
* Make sure wrong volumes are cleared in any case
Also add warning when string mapping is rejected.
* Add unit tests
* Improve test coverage
* Stop refreshing the update information on outdated OS versions
Add `JobCondition.OS_SUPPORTED` to the updater job to avoid
refreshing update information when the OS version is unsupported.
This effectively freezes installations on unsupported OS versions
and blocks Supervisor updates. Once deployed, this ensures that any
Supervisor will always run on at least the minimum supported OS
version.
This requires to move the OS version check before Supervisor updater
initialization to allow the `JobCondition.OS_SUPPORTED` to work
correctly.
* Run only OS version check in setup loads
Instead of running a full system evaluation, only run the OS version
check right after the OS manager is loaded. This allows the
updater job condition to work correctly without running the full
system evaluation, which is not needed at this point.
* Prevent Core and Add-on updates on unsupported OS versions
Also prevent Home Assistant Core and Add-on updates on unsupported OS
versions. We could imply `JobCondition.SUPERVISOR_UPDATED` whenever
OS is outdated, but this would also prevent the OS update itself. So
we need this separate condition everywhere where
`JobCondition.SUPERVISOR_UPDATED` is used except for OS updates.
It should also be safe to let the add-on store update, we simply
don't allow the add-on to be installed or updated if the OS is
outdated.
* Remove unnecessary Host info update
It seems that the CPE information are already loaded in the HostInfo
object. Remove the unnecessary update call.
* Fix pytest
* Delay refreshing of update data
Delay refreshing of update data until after setup phase. This allows to
use the JobCondition.OS_SUPPORTED safely. We still have to fetch the
updater data in case OS information is outdated. This typically happens
on device wipe.
Note also that plug-ins will automatically refresh updater data in case
it is missing the latest version information.
This will reverse the order of updates when there are new plug-in and
Supervisor update information available (e.g. on first startup):
Previously the updater data got refreshed before the plug-in started,
which caused them to update first. Then the Supervisor got update in
startup phase. Now the updater data gets refreshed in startup phase,
which then causes the Supervisor to update first before the plug-ins
get updated after Supervisor restart.
* Fix pytest
* Fix updater tests
* Add new tests to verify that updater reload is skipped
* Fix pylint
* Apply suggestions from code review
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Add debug message when we delay version fetch
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Fix NetworkManager connection name for VLANs
The connection name for VLANs should include the parent interface name
for better identification. This was originally the intention, but the
interface object's name property was used which appears empty at that
point.
* Disallow creating multiple connections for the same VLAN id
Only allow a single connection per interface and VLAN id. The regular
network commands can be used to alter the configuration.
* Fix pytest
* Simply connection id name generation
Always rely on the Supervisor interface representation's name attribute
to generate the NetworkManager connection id. Make sure that the name
is correctly set when creating VLAN interfaces as well.
* Special case VLAN configuration
We can't use the match information when comparing Supervisor interface
representation with D-Bus representations. Special case VLAN and
compare using VLAN ID and parent interface.
Note that this currently compares connection UUID of the parent
interface.
* Fix pytest
* Separate VLAN creation logic from apply_changes
Apply changes is really all about updating the NetworkManager settings
of a particular network interface. The base in apply_changes() is
NetworkInterface class, which is the NetworkManager Device abstraction.
All physical interfaces have such a Device hence it is always present.
The only exception is when creating a VLAN: Since it is a virtual
device, there is no device when creating a VLAN.
This separate the two cases. This makes it much easier to reason if
a VLAN already exists or not, and to handle the case where a VLAN
needs to be created.
For all other network interfaces, the apply_changes() method can
now rely on the presence of the NetworkInterface Device abstraction.
* Add VLAN test interface and VLAN exists test
Add a test which checks that an error gets raised when a VLAN for a
particular interface/id combination already exists.
* Address pylint
* Fix test_ignore_veth_only_changes pytest
* Make VLAN interface disabled to avoid test issues
* Reference setting 38 in mocked connection
* Make sure interface type matches
Require a interface type match before doing any comparision.
* Add Supervisor host network configuration tests
* Fix device type checking
* Fix pytest
* Fix tests by taking VLAN interface into account
* Fix test_load_with_network_connection_issues
This seems like a hack, but it turns out that the additional active
connection caused coresys.host.network.update() to be called, which
implicitly "fake" activated the connection. Now it seems that our
mocking causes IPv4 gateway to be set.
So in a way, the test checked a particular mock behavior instead of
actual intention.
The crucial part of this test is that we make sure the settings remain
unchanged. This is done by ensuring that the the method is still auto.
* Fix test_check_network_interface_ipv4.py
Now that we have the VLAN interface active too it will raise an issue
as well.
* Apply suggestions from code review
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Fix ruff check issue
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Send progress updates during image pull for install/update
* Add extra to tests about job APIs
* Sent out of date progress to sentry and combine done event
* Pulling container image layer
* Fix docker_config check to ignore Docker VOLUME mounts
Only validate /media and /share mounts that are explicitly configured
in add-on map_volumes, not those created by Docker VOLUME statements.
* Check and test with custom map targets
* Optimize directory_missing_or_empty function
Replace inefficient os.listdir() with os.scandir() and next() to check
if directory is empty. This avoids reading entire directory contents
into memory when we only need to know if any entry exists.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add unit tests for directory_missing_or_empty function
Add comprehensive test coverage for the optimized directory_missing_or_empty
function, testing empty directories, directories with content, non-existent
paths, and files (non-directories).
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Apply suggestions from code review
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Propagate timezone setting to host in OS 16.2 and newer
With home-assistant/operating-system#4224, timezone setting in OS can be
peristently set in HAOS as well. Propagate the timezone configured in
Supervisor config (which can be changed through general system settings
in HA Core) through the DBus API for setting the timezone.
* Persist timezone also when it's been obtained from Whoami
* Suppress pylint fixme error
* Storage space usage API
* Move to host API
* add tests
* fix test url
* more tests
* fix tests
* fix test
* PR comments
* update test
* tweak format and url
* add .DS_Store to .gitignore
* update tests
* test coverage
* update to new struct
* update test
Not all disks have all SMART attributes available, e.g. Sentry showed
devices with missing "wctemp". In practice, any SMART attribute could
be missing. Make sure we handle this gracefully.
* Use context manager for Job concurrency control
* Allow to release lock outside of Job running context
* Improve JobGroup locking with external ownership tracking
Track lock ownership by job UUID instead of execution context. This
allows external lock release via job parameter.
* Fix acquire lock in nested Jobs
* Simplify nested lock tracking
* Simplify Job group lock acquisition logic
* Simplify by using helper methods
* Allow throttling with group concurrency
* Use Lock instead of Semaphore for job concurrency control
Use the same synchronization primitive (Lock) for job concurrency
control as used in job groups.
* Go back to lock ownership tracking with references
* Drop unused property `active_job_id`
* Drop unused property `can_acquire`
* Replace assert with cast
* Add unsupported reason os_version and evaluation
* Order enum and add tests
* Apply suggestions from code review
* Apply suggestions from code review
---------
Co-authored-by: Stefan Agner <stefan@agner.ch>
This avoids that we display a 10% life time use for a brand new
eMMC storage. The values are estimates anyways, and there is a
separate value which represents life time completely used (100%).
* Block OS updates when the system is unhealthy
In #6024 we mark a system as unhealthy when multiple OS installations
were found. The idea was to block OS updates in this case. However, it
turns out that the OS update job was not checking the system health
and thus allowed updates even when the system was marked as unhealthy.
This commit adds the `JobCondition.HEALTHY` condition to the OS update
job, ensuring that OS updates are only performed when the system is
healthy.
Users can force an OS update still by using
`ha jobs options --ignore-conditions healthy`.
* Add test for update of unhealthy system
---------
Co-authored-by: Jan Čermák <sairon@sairon.cz>
* Split execution limit in concurrency and throttle parameters
Currently the execution limit combines two ortogonal features: Limit
concurrency and throttle execution. This change separates the two
features, allowing for more flexible configuration of job execution.
Ultimately I want to get rid of the old limit parameter. But for ease
of review and migration, I'd like to do this in two steps: First
introduce the new parameters, and map the old limit parameters to the
new parameters. Then, in a second step, remove the old limit parameter
and migrate all users to the new concurrency and throttle parameters
as needed.
* Introduce common lock release method
* Fix THROTTLE_WAIT behavior
The concurrency QUEUE does not really QUEUE throttle limits.
* Add documentation for new concurrency/throttle Job options
* Handle group options for concurrency and throttle separately
* Fix GROUP_THROTTLE_WAIT concurrency setting
We need to use the QUEUE concurrency setting instead of GROUP_QUEUE
for the GROUP_THROTTLE_WAIT execution limit. Otherwise the
test_jobs_decorator.py::test_execution_limit_group_throttle_wait
test deadlocks.
The reason this deadlocks is because GROUP_QUEUE concurrency doesn't
really work because we only can release a group lock if the job is
actually running.
Or put differently, throttling isn't supported with GROUP_*
concurrency options.
* Prevent using any throttling with group concurrency
The group concurrency modes (reject and queue) are not compatible with
any throttling, since we currently can't unlock the group lock when
a job doesn't get started (which is the case when throttling is
applied).
* Fix commit in group rate limit
* Explain the deadlock issue with group locks in code
* Handle locking correctly on throttle limit exceptions
* Introduce pytest for new job decorator combinations
* Enable IPv6 by default for new installations
Enable IPv6 by default for new Supervisor installations. Let's also
make the `enable_ipv6` attribute nullable, so we can distinguish
between "not set" and "set to false".
* Add pytest
* Add log message that system restart is required for IPv6 changes
* Fix API pytest
* Create resolution center issue when reboot is required
* Order log after actual setter call
* Add resolution check for duplicate OS installations
* Only create single issue/use separate unhealthy type
* Check MBR partition UUIDs as well
* Use partlabel
* Use generator to avoid code duplication
* Add list of devices, avoid unnecessary exception handling
* Run check only on HAOS
* Fix message formatting
* Fix and simplify pytests
* Fix UnhealthyReason sort order
* Check for duplicate data disks only when the OS is available
Supervised installations do not have a specific data disk, so only
check for duplicate data disks on Home Assistant OS.
* Enable OS for multiple data disks check test
* Drop ensure_builtin_repositories
With the new Repository classes we have the is_builtin property, so we
can easily make sure that built-ins are not removed. This allows us to
further cleanup the code by removing the ensure_builtin_repositories
function and the ALL_BUILTIN_REPOSITORIES constant.
* Make sure we add built-ins on load
* Reuse default set and avoid unnecessary copy
Reuse default set and avoid unnecessary copying during validation if
the default is not being used.
* Add Supervisor connectivity check after DNS restart
When the DNS plug-in got restarted, check Supervisor connectivity
in case the DNS plug-in configuration change influenced Supervisor
connectivity. This is helpful when a DHCP server gets started after
Home Assistant is up. In that case the network provided DNS server
(local DNS server) becomes available after the DNS plug-in restart.
Without this change, the Supervisor connectivity will remain false
until the a Job triggers a connectivity check, for example the
periodic update check (which causes a updater and store reload) by
Core.
* Fix pytest and add coverage for new functionality
* Rename repository fixture to test_repository
Also don't remove the built-in repositories. The list was incomplete,
and tests don't seem to require that anymore.
* Get rid of StoreType
The type doesn't have much value, we have constant strings anyways.
* Introduce types.py
* Use slug to determine which repository urls to return
* Simplify BuiltinRepository enum
* Mock GitRepo load
* Improve URL handling and repository creation logic
* Refactor update_repositories
* Get rid of get_from_url
It is no longer used in production code.
* More refactoring
* Address pylint
* Introduce is_git_based property to Repository class
Return all git based URLs, including the Core repository.
* Revert "Introduce is_git_based property to Repository class"
This reverts commit dfd5ad79bf.
* Fold type.py into const.py
Align more with how Supervisor code is typically structured.
* Update supervisor/store/__init__.py
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Apply repository remove suggestion
* Fix tests
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
When authentication using JSON payload or URL encoded payload fails,
use the generic HTTP response code 401 Unauthorized instead of 400
Bad Request.
This is a more appropriate response code for authentication errors
and is consistent with the behavior of other authentication methods.
* Improve DNS plug-in restart
Instead of simply go by PrimaryConnectioon change, use the DnsManager
Configuration property. This property is ultimately used to write the
DNS plug-in configuration, so it is really the relevant information
we pass on to the plug-in.
* Check for changes and restart DNS plugin
* Check for changes in plug-in DNS
Cache last local (NetworkManager) provided DNS servers. Check against
this DNS server list when deciding when to restart the DNS plug-in.
* Check connectivity unthrottled in certain situations
* Fix pytest
* Fix pytest
* Improve test coverage for DNS plugins restart functionality
* Apply suggestions from code review
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Debounce local DNS changes and event based connectivity checks
* Remove connection check logic
* Remove unthrottled connectivity check
* Fix delayed call
* Store restart task and cancel in case a restart is running
* Improve DNS configuration change tests
* Remove stale code
* Improve DNS plug-in tests, less mocking
* Cover multiple private functions at once
Improve tests around notify_locals_changed() to cover multiple
functions at once.
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Use Docker BuildKit to build addons
* Improve error message as suggested by CodeRabbit
* Fix container.remove() tests missing v=True
* Ignore squash rather than falling back to legacy builder
* Use version rather than tag to avoid confusion in run_command()
* Fix tests differently
* Use PropertyMock like other tests
* Restore position of fix_label fn
* Exempt addon builder image from unsupported checks
* Refactor tests
* Fix tests expecting wrong builder image
* Remove harcoded paths
* Fix tests
* Remove get_addon_host_path() function
* Use docker buildx build rather than docker build
Co-authored-by: Stefan Agner <stefan@agner.ch>
---------
Co-authored-by: Stefan Agner <stefan@agner.ch>
* Rename detect-blocking-io API value to match other APIs
For the new detect-blocking-io option, use dashes instead of
underscores in `on-at-startup` for consistency with other API
endpoints.
This is a breaking change, but since the API is really new and not
really used yet, it is fairly safe to do so.
* Fix pytest
