* Avoid IO in event loop when removing backup
* Refactor backup size calculation
Currently size is lazy loaded when required via properties. This
however is blocking the async event loop.
Backup sizes don't change. Instead of lazy loading the size of a backup
simply determine it on loading/after creation.
* Fix tests for backup size change
* Avoid IO in event loop when loading backups
* Avoid IO in event loop when importing a backup
* Validate Backup always before restoring
Since #5519 we check the encryption password early in restore case.
This has the side effect that we check the file existance early too.
However, in the non-encryption case, the file is not checked early.
This PR changes the behavior to always validate the backup file before
restoring, ensuring both encryption and non-encryption cases are
handled consistently.
In particular, the last case of test_restore_immediate_errors actually
validates that behavior. That test should actually have failed so far.
But it seems that because we validate the backup shortly after freeze
anyways, the exception still got raised early enough.
A simply `await asyncio.sleep(10)` right after the freeze makes the
test case fail. With this change, the test works consistently.
* Address pylint
* Fix backup_manager tests
* Drop warning message
* Fix restoring unencrypted backup in corner case
If a backup has a encrypted and unencrypted location, and the encrypted
location is beeing restored first, the encryption key is still cached.
When the user restores the unencrypted backup next, it will fail because
the Supervisor tries to use encryption key still.
* Add integration test for restoring backups with and without encryption
* Rename _validate_location_password to _set_location_password
* Reload backup metadata from restore location
* Revert "Reload backup metadata from restore location"
This reverts commit 9b47a1cfe9a2682a0908e08cd143373744084fb7.
* Make pytest work/punt the ball on docker config restore issue
* Address pylint error
* Handle non-existing file in Backup password check too
Make sure we handle a non-existing backup file also when validating
the password.
* Update supervisor/backups/manager.py
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Add test case and fix password check when multiple locations
* Mock default backup unprotected by default
Instead of setting the protected property which we might not use
everywhere, simply mock the default backup to be unprotected.
* Fix mock of protected backup
* Introduce test for validate_password
Testing showed that validate_password doesn't return anything. Extend
tests to cover this case and fix the actual code.
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Make the API return 404 for non-existing backup files
* Introduce BackupFileNotFoundError exception
* Return 404 on full restore as well
* Fix remaining API tests
* Improve error handling in delete
* Fix pytest
* Fix tests and change error handling to agreed logic
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Avoid test failure by not checking exact size of backup
This is a workaround for the fact that the backup size is not exactly
the same every time. This is due to the fact that the inner gziped tar
file can vary in size due to difference in json file (key order) and
potentially also different field values (UUID, backup slug).
It seems that sorting the keys makes the actual difference today, but
this has runtime overhead and might not catch all cases.
Simply check if size property is there and a number bigger than 0
instead.
* Fix pytest
* Extend backup upload API with file name parameter
Add a query parameter which allows to specify the file name on upload.
All locations will store the backup with the same file name.
* ruff format
* Update tests to cover bad filename
* Fix ruff check error
* Drop unnecessary logging
* Use version which is treated CalVer by AwesomeVersion
The current dev version `99.9.9dev` is treated as unkown version type
by AwesomeVersion. This prevents the version from comparing with
actual Supervisor versions, e.g. from an exsiting backup file.
Make the development version a valid CalVer version so development
versions can handle non-development backups.
* Bump to year 9999
* Backup protected status can vary per location
* Fix test_backup_remove_error test
* Update supervisor/backups/backup.py
* Add Docker registry configuration to backup metadata
* Make use of backup location fixture
* Address pylint
---------
Co-authored-by: Stefan Agner <stefan@agner.ch>
* Bump Supervisor to Python 3.13
* Update ruff configuration to 0.9.1
Adjust pyproject.toml for ruff 0.9.1. Also make sure that latest version
of ruff is used in pre-commit.
* Set default configuration for pytest-asyncio
* Run ruff check
* Drop deprecated decorator no_type_check_decorator
The upstream PR (https://github.com/python/cpython/issues/106309) says
this never got really implemented by type checkers.
* Bump devcontainer to latest release
Introduce a validate password method which only peaks into the archive
to validate the password before starting the actual restore process.
This makes sure that a wrong password returns an error even when
restoring the backup in background.
* Fix and extend cloud backup support
* Clean up task for cloud backup and remove by location
* Args to kwargs on backup methods
* Fix backup remove error test and typing clean up
When an error occurs when streaming Supervisor logs, the fallback method
receives the follow kwarg as well, which is invalid for the Docker log
handler:
TypeError: APISupervisor.logs() got an unexpected keyword argument 'follow'
The exception is still printed to the logs but with all the extra noise
caused by this error. Removing the argument makes the stack trace more
comprehensible and the fallback actually works as desired.
If no cursor is specified and negative num_skip is used, we're pointing
one record back from the last one, so host logs always returned 101
lines as the default. This was also the case of the lines query argument
that used the number directly as num_skip. Instead of doing that, point
N-1 records to the back and then get N records. Handle 1 record and
invalid numbers silently to avoid the need for error handling in
unpractical edge cases.
* Improve WiFi settings error handling
Currently, the frontend potentially provides no WiFi settings dictionary
but still tries to update other (IP address) settings on the interface.
This leads to a stack trace since network manager is not able to fetch
the WiFi settings from the settings dictionary. Simply fill out what
we can and let NetworkManager provide an error.
Also allow to disable a network interface which has no configuration.
This avoids an error when switching to auto and back to disabled then
press save on a new wireless network interface.
* Add debug message when already disabled
* Add pytest for incomplete WiFi settings as psoted by frontend
Simulate the frontend posting no WiFi settings. Make sure the Supervisor
handles this gracefully.
* Allow to set user DNS through API with auto mode
Currently it is only possible to set DNS servers when in static mode.
However, there are use cases to set DNS servers when in auto mode as
well, e.g. if no local DNS server is provided by the DHCP, or the provided
DNS turns out to be non-working.
* Fix use separate data structure for IP configuration fallout
Make sure gateway is correctly converted to the internal IP
representation. Fix type info.
* Overwrite WiFi settings completely too
* Add test for DNS configuration
* Run ruff format
* ruff format
* Use schema validation as source for API defaults
Instead of using replace() simply set the API defaults in the API
schema.
* Revert "Use schema validation as source for API defaults"
This reverts commit 885506fd37395eb6cea9c787ee23349dac780b75.
* Use explicit dataclass initialization
This avoid the unnecessary replaces from before. It also makes it more
obvious that this part of the API doesn't patch existing settings.
Since headers are clumsy considering the Core proxy between the frontend
and Supervisor, add a way to adjust number of lines and verbose log
format using query parameters as well. If both query parameters and
headers are supplied, prefer the former, as it's more prominent when
reading through the request logs.
* Make IPv4 and IPv6 parse errors raise an API error
Currently, IP address parsing errors lead to an execption which is not
handled by the `api_validate()` call. By using concrete IPv4 and IPv6
types and `vol.Coerce()` parsing errors are properly handled.
* ruff format
* ruff check
* Improve connection settings fixture
Make the connection settings fixture behave more closely to the actual
NetworkManager. The behavior has been tested with NetworkManager 1.42.4
(Debian 12) and 1.44.2 (HAOS 13.1). This likely behaves similar in older
versions too.
* Introduce separate skeleton and settings for wireless
Instead of having a combined network settings object which has
Ethernet and Wirless settings, create a separate settings object for
wireless.
* Handle addresses/address-data property like NetworkManager
* Address ruff check
* Improve network API test
Add a test which changes from "static" to "auto". Validate that settings
are updated accordingly. Specifically, today this does clear the DNS
setting (by not providing the property).
* ruff format
* ruff check
* Complete TEST_INTERFACE rename
* Add partial network update as test case
* Add manual_forced option to addon boot config
* Include client library in pull request template
* Add boot_config to api output so frontend can use it
* `manual_forced` to `manual_only`
* Stop backup if pre backup failed in Core
* Fix API tests
* Partial backup in ci since there's no Home assistant
* Add ssl folder to partial backup
* Allow backups when Home Assistant is not running
* Undo change to skip db test
* Fix no changelog API response
* Add comment reasoning HTTP 200 for no changelog
Co-authored-by: Stefan Agner <stefan@agner.ch>
* Apply suggestions from code review
Co-authored-by: Jan Čermák <sairon@users.noreply.github.com>
---------
Co-authored-by: Jan Čermák <sairon@users.noreply.github.com>
Co-authored-by: Stefan Agner <stefan@agner.ch>
We do not need to capture HostNotSupported errors to Sentry. The only
possible code path this error might come from is where the Journal
Gateway Daemon socket is unavailable, which is already reported as an
"Unsupported system" repair.
Supervisor logs fallback in get_supervisor_logs didn't work properly
because the exception was caught in api_process_raw instead. This was
not discovered in tests because the side effect raised OSError, which
isn't handled there.
To address that, I split the advanced_logs to two functions, one being a
wrapped API handler, one being plain function returning response without
any additional error handling. The tests now check for both cases of
errors (HassioError and random generic Python error).
Refs #5021
* Allow adoption of existing data disk
* Fix existing tests
* Add test cases and fix image issues
* Fix addon build test
* Run checks during setup not startup
* Addon load mimics plugin and HA load for docker part
* Default image accessible in except
* Use Systemd Journal API for all logs endpoints in API
Replace all logs endpoints using container logs with wrapped
advanced_logs function, adding possibility to get logs from previous
boots and following the logs. Supervisor logs are an excetion where
Docker logs are still used - in case an exception is raised while
accessing the Systemd logs, they're used as fallback - otherwise we
wouldn't have an easy way to see what went wrong.
* Refactor testing of advanced logs endpoints to a common method
* Send error while fetching Supervisor logs to Sentry; minor cleanup
* Properly handle errors and use consistent content type in logs endpoints
* Replace api_process_custom with reworked api_process_raw per @mdegat01 suggestion
* Unsupported if wrong image used on virtualization
* Add generic-aarch64 as supported image
* Add virtualization field to API
* Change startup to setup in check
* Use Journal Export Format for host (advanced) logs
Add methods for handling Journal Export Format and use it for fetching
of host logs. This is foundation for colored streaming logs for other
endpoints as well.
* Make pylint happier - remove extra pass statement
* Rewrite journal gateway tests to mock ClientResponse's StreamReader
* Handle connection refused error when connecting to journal-gatewayd
* Use SYSTEMD_JOURNAL_GATEWAYD_SOCKET global path also for connection
* Use parsing algorithm suggested by @agners in review
* Fix timestamps in formatting, always use UTC for now
* Add tests for Accept header in host logs
* Apply suggestions from @agners
Co-authored-by: Stefan Agner <stefan@agner.ch>
* Bail out of parsing earlier if field is not in required fields
* Fix parsing issue discovered in the wild and add test case
* Make verbose formatter more tolerant
* Use some bytes' native functions for some minor optimizations
* Move MalformedBinaryEntryError to exceptions module, add test for it
---------
Co-authored-by: Stefan Agner <stefan@agner.ch>
