84 Commits

Author SHA1 Message Date
Mike Degatano
9a7d547394
Allow all characters in mount credentials (#4399)
* Allow all characters in mount credentials

* Fix permissions on credential files

* Fix pylint issue
2023-06-22 15:55:13 -04:00
Mike Degatano
254ec2d1af
Addon startup waits for healthy (#4376)
* Addon startup waits for healthy

* fix import for pylint

* wait_for to 5 in tests

* Adjust tests to simplify async tasks

* Remove wait_boot time from addons.boot tests

* Eliminate async task race conditions in tests
2023-06-20 10:13:15 -04:00
Mike Degatano
841f68c175
Make issue for problem with config for containers (#4317)
* Make issue for problem with config for containers

* Mount propagation in tests

* Fixes from rebase and feedback
2023-05-30 13:25:38 -04:00
Mike Degatano
e984797f3c
Support share mounts (#4318) 2023-05-29 11:40:03 +02:00
Mike Degatano
a7c1693911
Set bind propagation mode for media (#4308)
* Set bind propagation mode for media

* Add some test cases
2023-05-24 09:12:35 -04:00
Mike Degatano
f6c3bdb6a8
Add mount to supported features (#4301)
* Add mount to supported features

* Typo in enable

* Fix places mocking os available without version

* Increase resilence of problematic repeat task test
2023-05-23 14:00:15 +02:00
Mike Degatano
b4fd5b28f6
Use backup mounts (#4289)
* Add support for backup mounts

* Fix tests

* Allow backups to local when there's a default location
2023-05-16 14:08:22 -04:00
Mike Degatano
34c394c3d1
Add support for network mounts (#4269)
* Add support for network mounts

* Handle backups and save data

* fix pylint issues
2023-05-01 08:45:52 +02:00
Mike Degatano
fce0d2aaed
Prevent multiple data disks with name hassos-data-external (#4222)
* Prevent multiple data disks with name hassos-data-external

* Fix pylint issues

* Do rename before format
2023-04-01 08:40:19 +02:00
Mike Degatano
c0b75edfb7
Format data disk in Supervisor instead of OS Agent (#4212)
* Supervisor formats data disk instead of os agent

* Fix issues occurring during tests

* Can't migrate if target is too small
2023-03-30 14:15:07 -04:00
Mike Degatano
a6caccd845
Use session dbus mocks for all tests (#4198)
* Use session dbus mocks for all tests

* func instead of fn for pylint
2023-03-21 07:30:31 +01:00
Mike Degatano
d9b86fa2ab
Use session dbus for udisks2 tests (#4194)
* Use session dbus for udisks2 tests

* Don't redefine type
2023-03-17 20:56:34 +01:00
Mike Degatano
0ddce4d9bc
Use session dbus for network tests (#4191)
* Use session dbus for network tests

* Don't use session dbus for coresys yet
2023-03-17 14:26:41 +01:00
Mike Degatano
daeec266cc
Mock services on session dbus in tests (#4160)
* Mock services on session dbus in tests

* methods emit prop changes for testing
2023-03-08 11:10:24 +01:00
Mike Degatano
3887fcfc93
Test discovery message deleted on uninstall (#4174)
* Test discovery message deleted on uninstall

* Test data is correct since core looks at it
2023-03-08 11:08:55 +01:00
Raman Gupta
4c2d729646
Add udisks2 dbus support (#3848)
* Add udisks2 dbus support

* assert mountpoints

* Comment

* Add reference links

* docstring

* fix type

* fix type

* add typing extensions as import

* isort

* additional changes

* Simplify classes and conversions, fix bugs

* More simplification

* Fix imports

* fix pip

* Add additional properties and fix requirements

* fix tests maybe

* Handle optionality of certain configuration details

* black

* connect to devices before returning them

* Refactor for latest dbus work

* Not .items

* fix mountpoints logic

* use variants

* Use variants for options too

* isort

* Switch to dbus fast

* Move import to parent

* Add some fixture data

* Add another fixture and reduce the block devices list

* Implement changes discussed with mike

* Add property fixtures

* update object path

* Fix get_block_devices call

* Tests and refactor to minimize dbus reconnects

* Call super init in DBusInterfaceProxy

* Fix permissions on introspection files

---------

Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
2023-02-15 08:17:29 +01:00
Pascal Vizeli
2809f23391
Make dbus-fast calls more robust (#4005)
* Make dbus-fast calls more robust

* Handle all exceptions and add test

* DBus minimal can't return commands list

Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
2022-11-15 22:19:38 +01:00
Mike Degatano
14fcda5d78
Sentry only loaded when diagnostics on (#3993)
* Sentry only loaded when diagnostics on

* Logging when sentry is closed
2022-11-13 21:23:52 +01:00
Mike Degatano
bde5c938a7
Ignore veth changes (#3955)
* Reduce log noise for unmanaged interfaces

* Ignore signals with veth changes only

* Fix test and add one
2022-10-16 11:06:35 +02:00
Stefan Agner
2ebb405871
Add enhanced logging REST endpoints using systemd-journal-gatewayd (#3291)
* Add enhanced logging REST endpoints using systemd-journal-gatewayd

Add /host/logs/entries and /host/logs/{identifier}/entries to expose log
entries from systemd-journald running on the host. Use
systemd-journal-gatewayd which exposes the logs to the Supervisor via
Unix socket.

Current two query string parameters are allowed: "boot" and "follow".
The first will only return logs since last boot. The second will keep
the HTTP request open and send new log entries as they get added to the
systemd-journal.

* Allow Range header

Forward the Range header to systemd-journal-gatewayd. This allows to
select only a certain amount of log data. The Range header is a standard
header to select only partial amount of data. However, the "entries="
prefix is custom for systemd-journal-gatewayd, denoting that the numbers
following represent log entries (as opposed to bytes or other metrics).

* Avoid connecting if systemd-journal-gatewayd is not available

* Use path for all options

* Add pytests

* Address pylint issues

* Boot ID offsets and slug to identifier

* Fix tests

* API refactor from feedback

* fix tests and add identifiers

* stop isort and pylint fighting

* fix tests

* Update default log identifiers

* Only modify /host/logs endpoints

* Fix bad import

* Load log caches asynchronously at startup

* Allow task to complete in fixture

* Boot IDs and identifiers loaded on demand

* Add suggested identifiers

* Fix tests around boot ids

Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
2022-10-13 11:40:11 -04:00
Mike Degatano
c24b811180
Use dbus-fast unpack_variants option (#3885)
* Use dbus-fast unpack_variants option

* More readable log on signals
2022-09-21 20:52:06 +02:00
Mike Degatano
0958cd0c06
Switch from dbus-next to dbus-fast (#3882)
* Switch to dbus-fast

* dbus-fast==1.6.0
2022-09-21 09:24:04 +02:00
Mike Degatano
2072370ccc
Revert "Switch to dbus-fast (#3877)" (#3878)
This reverts commit 615758a1dfde1f91ff1dc7e3e4ecc826a0ed2b58.
2022-09-19 17:18:25 -04:00
Mike Degatano
615758a1df
Switch to dbus-fast (#3877) 2022-09-19 21:55:22 +02:00
Mike Degatano
99bc201688
Listen for dbus property changes (#3872)
* Listen for dbus property changes

* Avoid remaking dbus proxy objects

* proper snake case for pylint

* some cleanup and more tests
2022-09-17 09:55:41 +02:00
Mike Degatano
d195f19fa8
Refactor to dbus-next proxy interfaces (#3862)
* Refactor to dbus-next proxy interfaces

* Fix tests mocking dbus methods

* Fix call dbus
2022-09-13 13:45:28 -04:00
Mike Degatano
b71c6c60da
Share MessageBus across dbus objects (#3860)
* Share MessageBus across dbus objects

* Fix connect calls in tests
2022-09-12 20:10:12 +02:00
Mike Degatano
4f272ad4fd
Dbus refactor and tests (#3854)
* Dbus refactor and tests

* Link to timedate introspection
2022-09-12 11:59:56 -04:00
Stefan Agner
01e24a3e74
Replace loop test fixture with event_loop (#3847)
This addresses a deprecation warning when calling tests with Python
3.10:

```
../python3.10/site-packages/pytest_aiohttp/plugin.py:33: DeprecationWarning: 'loop' fixture is deprecated and scheduled for removal, please use 'event_loop' instead
```
2022-09-09 09:54:57 -04:00
Mike Degatano
ebae1e70ee
Add tests for backups check (#3845) 2022-09-07 10:33:46 +02:00
Mike Degatano
bf48d48c51
Create issue+suggestion when no recent backup (#3814)
* Automatic full backup option

* Fix test for change in free space check

* Suggestions only, no automation

* Remove extra in backup config schema
2022-09-03 09:50:23 +02:00
Mike Degatano
fc646db95f
Reduce connectivity checks (#3836)
* Reduce connectivity checks

* Fix/remove connectivity tests

* Remove throttle from prior connectivity tests

* Use dbus_property wrapper

* Allow variable throttle period with lambda

* Add evaluation for connectivity check disabled
2022-09-03 09:48:30 +02:00
Mike Degatano
d19166bb86
Docker events based watchdog and docker healthchecks (#3725)
* Docker events based watchdog

* Separate monitor from DockerAPI since it needs coresys

* Move monitor into dockerAPI

* Fix properties on coresys

* Add watchdog tests

* Added tests

* pylint issue

* Current state failures test

* Thread-safe event processing

* Use labels property
2022-07-15 09:21:59 +02:00
Joakim Sørensen
b82dbc0cac
Extended store addon info (#3678)
* Add extended store addon info

* Add missing hassio_api

* Fix test
2022-06-09 13:47:15 +02:00
Mike Degatano
7a74d77d43
Move addon repo migration to bootstrap (#3672)
* Move addon repo migration to bootstrap

* Save data after migrating it
2022-06-08 15:21:34 -04:00
Mike Degatano
ccd2c31390
Move repository urls to store settings file (#3665)
* Move repository urls to store settings file

* Remove default repos from supervisor config

* Fix clone at initial store load

* Mock git load in repository fixture
2022-06-07 10:02:21 +02:00
Mike Degatano
138fd7eec9
APIs for adding/removing an addon repository (#3649)
* APIs for adding/removing an addon repository

* Misunderstood addons.store, fixed usage
2022-05-23 09:16:42 +02:00
Mike Degatano
32d1296da1
Merge updates with existing network settings (#3570)
* Preserve network customizations over updates

* Fix issues from testing

* Test case to ensure updates generated correctly
2022-04-18 09:49:30 +02:00
Mike Degatano
a553ba5d24
Update resolved information with host info (#3560) 2022-04-12 09:21:35 +02:00
Mike Degatano
e2ac5042d8
Apply network settings on start to fix defaults (#3528)
* Apply network settings on start to fix defaults

* Use job check and add tests
2022-03-31 12:14:40 +02:00
Stefan Agner
eadc629cd9
De-duplicate Backup/Restore logic (#3311)
* De-duplicate Backup/Restore logic

Create internal _do_backup()/_do_restore() method which de-duplicates
some of the backup/restore logic previously part of full/partial backup/restore.

* Add Backup/Restore test coverage
2021-12-14 17:21:52 +00:00
Joakim Sørensen
88490140af
Add /supervisor/available_updates endpoint (#3300) 2021-11-17 09:28:46 +01:00
Stefan Agner
e5817e9445
D-Bus signal handling improvements (#3248)
* Allow to update connection information

* Introduce Signal wrapper class using async context manager

This allows to start monitoring signals and execute code before
processing signals. With that it is possible to check for state changes
in a race free manor.

* Fix unit tests
2021-10-20 14:40:28 +02:00
Stefan Agner
435f479984
D-Bus NetworkManager improvements (#3243)
* Introduce enum for network connectivity

* Differentiate between Bus Name and Interface consts

The Bus names and interfaces look quite similar in D-Bus: Both use dots
to separate words. Usually all interfaces available below a certan Bus
name start with the Bus name. Quite often the Bus name itself is also
available as an interface.

However, those are different things. To avoid confusion, add the type of
const to the const name.

* Remove unused const

* Disconnect D-Bus when not used

Make sure Python disconnects from D-Bus when objects get destroyed. This
avoids exhausting D-Bus connection limit which causes the following
error message:
[system] The maximum number of active connections for UID 0 has been reached (max_connections_per_user=256)

* Filter signals by object as well

Make sure we only listen to signals on that particular object. Also
support filtering messages via message filter callback.

* Explicitly wait until Connection is activated

Wait for activated or raise an error. This avoids too early/errornous
updates when state of the connection changes to "activating" or similar
intermediate signal states.

Fixes: #2639

* Fix VLAN configuration

* Add link to D-Bus object documentation

* Fix network settings update test

* Make MessageBus object optional
2021-10-19 16:38:52 +02:00
Stefan Agner
7a6663ba80
Use Python dbus-next D-Bus library (#3234)
* Use the correct interface name to get properties of systemd

It seems that gdbus (or systemd) automatically pick the correct
interface and return the properties. However, dbussy requires the
correct interface name to get all properties.

* Don't expect array from Strength property

The property returns a type "y" which equates to "guchar":
https://developer-old.gnome.org/NetworkManager/stable/gdbus-org.freedesktop.NetworkManager.AccessPoint.html#gdbus-property-org-freedesktop-NetworkManager-AccessPoint.Strength

It seems that the old D-Bus implementation returned an array. With
dbus-next a integer is returned, so no list indexing required.

* Support signals and remove no longer used tests and code

* Pass rauc update file path as string

That is what the interface is expecting, otherwise the new lib chocks on
the Pathlib type.

* Support Network configuration with dbus-next

Assemble Python native objects and pass them to dbus-next. Use dbus-next
specific Variant class where necessary.

* Use org.freedesktop.NetworkManager.Connection.Active.StateChanged

org.freedesktop.NetworkManager.Connection.Active.PropertyChanged is
depricated. Also it seems that StateChanged leads to fewer and more
accurate signals.

* Pass correct data type to RequestScan.

RequestScan expects an option dictionary. Pass an empty option
dictionary to it.

* Update unit tests

Replace gdbus specific fixtures with json files representing the return
values. Those can be easily converted into native Python objects.

* Rename D-Bus utils module gdbus to dbus
2021-10-18 23:06:44 +02:00
Pascal Vizeli
288d2e5bdb
Use deepmerge for options (#3162)
* Use deepmerge for options

* fix issue

* Update supervisor/addons/addon.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

* Add tests

* Fix merge schema

* Make save for overwrites

* drop community

* more cleanup

* Fix tests

* Fix lists

* revert strategy

* protect overwritten lists

* Update tests/api/test_store.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2021-09-29 09:37:04 +02:00
Joakim Sørensen
81e7fac848
Fix flaky tests when CI did not have access to /run/supervisor (#3054) 2021-08-09 19:26:37 +02:00
Pascal Vizeli
ece40008c7
Logging in local timezone (#2971)
* Logging in local timezone

* fix convert

* Apply suggestions from code review

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2021-06-21 13:42:39 +02:00
Pascal Vizeli
a4a0b43d91
Using ghcr.io/home-assistant + fallback (#2959) 2021-06-14 20:07:44 +02:00
Joakim Sørensen
b70ed9a60d
Rename fixtures (#2946)
* Rename fixtures

* Use tilde
2021-06-10 12:45:39 +02:00