66 Commits

Author SHA1 Message Date
Mike Degatano
bde5c938a7
Ignore veth changes (#3955)
* Reduce log noise for unmanaged interfaces

* Ignore signals with veth changes only

* Fix test and add one
2022-10-16 11:06:35 +02:00
Stefan Agner
2ebb405871
Add enhanced logging REST endpoints using systemd-journal-gatewayd (#3291)
* Add enhanced logging REST endpoints using systemd-journal-gatewayd

Add /host/logs/entries and /host/logs/{identifier}/entries to expose log
entries from systemd-journald running on the host. Use
systemd-journal-gatewayd which exposes the logs to the Supervisor via
Unix socket.

Current two query string parameters are allowed: "boot" and "follow".
The first will only return logs since last boot. The second will keep
the HTTP request open and send new log entries as they get added to the
systemd-journal.

* Allow Range header

Forward the Range header to systemd-journal-gatewayd. This allows to
select only a certain amount of log data. The Range header is a standard
header to select only partial amount of data. However, the "entries="
prefix is custom for systemd-journal-gatewayd, denoting that the numbers
following represent log entries (as opposed to bytes or other metrics).

* Avoid connecting if systemd-journal-gatewayd is not available

* Use path for all options

* Add pytests

* Address pylint issues

* Boot ID offsets and slug to identifier

* Fix tests

* API refactor from feedback

* fix tests and add identifiers

* stop isort and pylint fighting

* fix tests

* Update default log identifiers

* Only modify /host/logs endpoints

* Fix bad import

* Load log caches asynchronously at startup

* Allow task to complete in fixture

* Boot IDs and identifiers loaded on demand

* Add suggested identifiers

* Fix tests around boot ids

Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
2022-10-13 11:40:11 -04:00
Mike Degatano
c24b811180
Use dbus-fast unpack_variants option (#3885)
* Use dbus-fast unpack_variants option

* More readable log on signals
2022-09-21 20:52:06 +02:00
Mike Degatano
0958cd0c06
Switch from dbus-next to dbus-fast (#3882)
* Switch to dbus-fast

* dbus-fast==1.6.0
2022-09-21 09:24:04 +02:00
Mike Degatano
2072370ccc
Revert "Switch to dbus-fast (#3877)" (#3878)
This reverts commit 615758a1dfde1f91ff1dc7e3e4ecc826a0ed2b58.
2022-09-19 17:18:25 -04:00
Mike Degatano
615758a1df
Switch to dbus-fast (#3877) 2022-09-19 21:55:22 +02:00
Mike Degatano
99bc201688
Listen for dbus property changes (#3872)
* Listen for dbus property changes

* Avoid remaking dbus proxy objects

* proper snake case for pylint

* some cleanup and more tests
2022-09-17 09:55:41 +02:00
Mike Degatano
d195f19fa8
Refactor to dbus-next proxy interfaces (#3862)
* Refactor to dbus-next proxy interfaces

* Fix tests mocking dbus methods

* Fix call dbus
2022-09-13 13:45:28 -04:00
Mike Degatano
b71c6c60da
Share MessageBus across dbus objects (#3860)
* Share MessageBus across dbus objects

* Fix connect calls in tests
2022-09-12 20:10:12 +02:00
Mike Degatano
4f272ad4fd
Dbus refactor and tests (#3854)
* Dbus refactor and tests

* Link to timedate introspection
2022-09-12 11:59:56 -04:00
Stefan Agner
01e24a3e74
Replace loop test fixture with event_loop (#3847)
This addresses a deprecation warning when calling tests with Python
3.10:

```
../python3.10/site-packages/pytest_aiohttp/plugin.py:33: DeprecationWarning: 'loop' fixture is deprecated and scheduled for removal, please use 'event_loop' instead
```
2022-09-09 09:54:57 -04:00
Mike Degatano
ebae1e70ee
Add tests for backups check (#3845) 2022-09-07 10:33:46 +02:00
Mike Degatano
bf48d48c51
Create issue+suggestion when no recent backup (#3814)
* Automatic full backup option

* Fix test for change in free space check

* Suggestions only, no automation

* Remove extra in backup config schema
2022-09-03 09:50:23 +02:00
Mike Degatano
fc646db95f
Reduce connectivity checks (#3836)
* Reduce connectivity checks

* Fix/remove connectivity tests

* Remove throttle from prior connectivity tests

* Use dbus_property wrapper

* Allow variable throttle period with lambda

* Add evaluation for connectivity check disabled
2022-09-03 09:48:30 +02:00
Mike Degatano
d19166bb86
Docker events based watchdog and docker healthchecks (#3725)
* Docker events based watchdog

* Separate monitor from DockerAPI since it needs coresys

* Move monitor into dockerAPI

* Fix properties on coresys

* Add watchdog tests

* Added tests

* pylint issue

* Current state failures test

* Thread-safe event processing

* Use labels property
2022-07-15 09:21:59 +02:00
Joakim Sørensen
b82dbc0cac
Extended store addon info (#3678)
* Add extended store addon info

* Add missing hassio_api

* Fix test
2022-06-09 13:47:15 +02:00
Mike Degatano
7a74d77d43
Move addon repo migration to bootstrap (#3672)
* Move addon repo migration to bootstrap

* Save data after migrating it
2022-06-08 15:21:34 -04:00
Mike Degatano
ccd2c31390
Move repository urls to store settings file (#3665)
* Move repository urls to store settings file

* Remove default repos from supervisor config

* Fix clone at initial store load

* Mock git load in repository fixture
2022-06-07 10:02:21 +02:00
Mike Degatano
138fd7eec9
APIs for adding/removing an addon repository (#3649)
* APIs for adding/removing an addon repository

* Misunderstood addons.store, fixed usage
2022-05-23 09:16:42 +02:00
Mike Degatano
32d1296da1
Merge updates with existing network settings (#3570)
* Preserve network customizations over updates

* Fix issues from testing

* Test case to ensure updates generated correctly
2022-04-18 09:49:30 +02:00
Mike Degatano
a553ba5d24
Update resolved information with host info (#3560) 2022-04-12 09:21:35 +02:00
Mike Degatano
e2ac5042d8
Apply network settings on start to fix defaults (#3528)
* Apply network settings on start to fix defaults

* Use job check and add tests
2022-03-31 12:14:40 +02:00
Stefan Agner
eadc629cd9
De-duplicate Backup/Restore logic (#3311)
* De-duplicate Backup/Restore logic

Create internal _do_backup()/_do_restore() method which de-duplicates
some of the backup/restore logic previously part of full/partial backup/restore.

* Add Backup/Restore test coverage
2021-12-14 17:21:52 +00:00
Joakim Sørensen
88490140af
Add /supervisor/available_updates endpoint (#3300) 2021-11-17 09:28:46 +01:00
Stefan Agner
e5817e9445
D-Bus signal handling improvements (#3248)
* Allow to update connection information

* Introduce Signal wrapper class using async context manager

This allows to start monitoring signals and execute code before
processing signals. With that it is possible to check for state changes
in a race free manor.

* Fix unit tests
2021-10-20 14:40:28 +02:00
Stefan Agner
435f479984
D-Bus NetworkManager improvements (#3243)
* Introduce enum for network connectivity

* Differentiate between Bus Name and Interface consts

The Bus names and interfaces look quite similar in D-Bus: Both use dots
to separate words. Usually all interfaces available below a certan Bus
name start with the Bus name. Quite often the Bus name itself is also
available as an interface.

However, those are different things. To avoid confusion, add the type of
const to the const name.

* Remove unused const

* Disconnect D-Bus when not used

Make sure Python disconnects from D-Bus when objects get destroyed. This
avoids exhausting D-Bus connection limit which causes the following
error message:
[system] The maximum number of active connections for UID 0 has been reached (max_connections_per_user=256)

* Filter signals by object as well

Make sure we only listen to signals on that particular object. Also
support filtering messages via message filter callback.

* Explicitly wait until Connection is activated

Wait for activated or raise an error. This avoids too early/errornous
updates when state of the connection changes to "activating" or similar
intermediate signal states.

Fixes: #2639

* Fix VLAN configuration

* Add link to D-Bus object documentation

* Fix network settings update test

* Make MessageBus object optional
2021-10-19 16:38:52 +02:00
Stefan Agner
7a6663ba80
Use Python dbus-next D-Bus library (#3234)
* Use the correct interface name to get properties of systemd

It seems that gdbus (or systemd) automatically pick the correct
interface and return the properties. However, dbussy requires the
correct interface name to get all properties.

* Don't expect array from Strength property

The property returns a type "y" which equates to "guchar":
https://developer-old.gnome.org/NetworkManager/stable/gdbus-org.freedesktop.NetworkManager.AccessPoint.html#gdbus-property-org-freedesktop-NetworkManager-AccessPoint.Strength

It seems that the old D-Bus implementation returned an array. With
dbus-next a integer is returned, so no list indexing required.

* Support signals and remove no longer used tests and code

* Pass rauc update file path as string

That is what the interface is expecting, otherwise the new lib chocks on
the Pathlib type.

* Support Network configuration with dbus-next

Assemble Python native objects and pass them to dbus-next. Use dbus-next
specific Variant class where necessary.

* Use org.freedesktop.NetworkManager.Connection.Active.StateChanged

org.freedesktop.NetworkManager.Connection.Active.PropertyChanged is
depricated. Also it seems that StateChanged leads to fewer and more
accurate signals.

* Pass correct data type to RequestScan.

RequestScan expects an option dictionary. Pass an empty option
dictionary to it.

* Update unit tests

Replace gdbus specific fixtures with json files representing the return
values. Those can be easily converted into native Python objects.

* Rename D-Bus utils module gdbus to dbus
2021-10-18 23:06:44 +02:00
Pascal Vizeli
288d2e5bdb
Use deepmerge for options (#3162)
* Use deepmerge for options

* fix issue

* Update supervisor/addons/addon.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

* Add tests

* Fix merge schema

* Make save for overwrites

* drop community

* more cleanup

* Fix tests

* Fix lists

* revert strategy

* protect overwritten lists

* Update tests/api/test_store.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2021-09-29 09:37:04 +02:00
Joakim Sørensen
81e7fac848
Fix flaky tests when CI did not have access to /run/supervisor (#3054) 2021-08-09 19:26:37 +02:00
Pascal Vizeli
ece40008c7
Logging in local timezone (#2971)
* Logging in local timezone

* fix convert

* Apply suggestions from code review

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2021-06-21 13:42:39 +02:00
Pascal Vizeli
a4a0b43d91
Using ghcr.io/home-assistant + fallback (#2959) 2021-06-14 20:07:44 +02:00
Joakim Sørensen
b70ed9a60d
Rename fixtures (#2946)
* Rename fixtures

* Use tilde
2021-06-10 12:45:39 +02:00
Pascal Vizeli
bcef34012d
Time handling (#2901)
* Add new time handling

* migrate date for python3.9

* add timedate

* add tests & simplify it

* better testing

* use ssl

* use hostname with new interface

* expose to API

* update data

* add base handler

* new timezone handling

* improve handling

* Improve handling

* Add tests

* Time adjustment function

* Fix logging

* tweak condition

* don't adjust synchronized time

* Guard

* ignore UTC

* small cleanup

* like that, we can leaf it

* add URL

* add comment

* Apply suggestions from code review

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2021-06-09 09:38:32 +02:00
Stefan Agner
a99bfa2926
Update OS on intel-nuc to generic-x86-64 image (#2909)
* Add architectures supported by generic-x86-64 board

* Follow intel-nuc to generic-x86-64 rename when updating

Home Assistant OS renamed the intel-nuc board to generic-x86-64. Make
sure to download the generic-x86-64 raucb OS update when updating a
intel-nuc machine.

* Don't explicit use section, rely on AwesomeVersion's comparision

* Remove unecessary global variable TEST_URL

* Fix version comparision

* Remove rate limiter on updater.fetch_data()

* Update tests/test_hassos.py

Co-authored-by: Pascal Vizeli <pascal.vizeli@syshack.ch>

* Fix black issue

Co-authored-by: Pascal Vizeli <pascal.vizeli@syshack.ch>
2021-05-27 14:25:16 +02:00
Pascal Vizeli
a5ed68b641
Cleanup websession with new aiohttp ssl (#2861) 2021-05-14 10:07:00 +02:00
Pascal Vizeli
cb3e2dab71
Full content trust support (#2735)
* Full content trust support

* expose content-trust for info

* fix log message

* Add system test

* Fix name

* Add tests

* ditch wrong disable

* fix partition

* remove untrust image
2021-03-20 00:10:35 +01:00
Joakim Sørensen
73849b7468
Check management (#2703)
* Check management

* Add test

* Don't allow disable core_security

* options and decorator

* streamline config handling

* streamline v2

* fix logging

* Add tests

* Fix test

* cleanup v1

* fix api

* Add more test

* Expose option also for cli

* address comments from Paulus

* Address second comment

* Update supervisor/resolution/checks/base.py

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* fix lint

* Fix black

Co-authored-by: Pascal Vizeli <pvizeli@syshack.ch>
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2021-03-12 11:32:56 +01:00
Joakim Sørensen
752068bb56
Add /store API (#2626) 2021-02-25 16:15:51 +01:00
Joakim Sørensen
b31ecfefcd
Initial WS support (#2439)
* Initial WS support

* test

* Update frontend to fc7c4af2

* Fix issue with closing states

* log error

* make data optional

* limit stopping states

* Move wrappers to HomeAssistantWebSocket

* use info

* Use call_soon

* Use lookuptable for WS commands

* Fix tests
2021-02-19 11:57:31 +01:00
Pascal Vizeli
aa5297026f
Handle Store suggestion (#2306)
* Handle Store suggestion

* Add fixup

* Add more fixup & list

* Enable fixups

* Add tests

* fix index

* fix break

* fix import

* Load it anyway

* Run suFix ccestion on load too

* fix error message

* fix error message

* Fix remove

* Finishing

* Add tests

* Fix error

* fix cleanup stale stuff

* Fix source

* use source as url

* add test for url

* Apply suggestions from code review

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2020-11-28 15:03:44 +01:00
Pascal Vizeli
845c935b39
Add JobManager API ignore (#2290)
* Disable job condition for unhealth & unsupported systems

* Add JobManager API ignore

* Apply suggestions from code review

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* Update tests/resolution/evaluation/test_evaluate_job_conditions.py

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* fix names

* address comments

* Update decorator.py

* adjust security

* add reset

* Apply suggestions from code review

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2020-11-24 10:54:57 +01:00
Joakim Sørensen
e607d4feeb
Fix missing primary interface after change (#2274)
* Fix missing primary interface after change

* Fix test
2020-11-20 12:01:59 +01:00
Joakim Sørensen
784c5d3b7c
Use bool for host internet (#2247)
* Use bool for host internet

* Ignore host check if no network manager

* Update supervisor/host/network.py

Co-authored-by: Pascal Vizeli <pvizeli@syshack.ch>

* Check dbus connection isntead of supported features

Co-authored-by: Pascal Vizeli <pvizeli@syshack.ch>
2020-11-13 00:07:05 +01:00
Joakim Sørensen
a18b706f99
Add JobManager and Job decorator (#2225)
* Adds condition decorator to block execution that require internet

* Fix exsisting tests

* Add internet state to network info

* Add healthy condition

* Add tests

* It's all changed

* rename
2020-11-12 22:57:28 +01:00
Pascal Vizeli
bd786811a3
Network: abstract dbus and supervisor - ipv6/wifi/vlan (#2217)
* Abstract code between dbus - supervisor

* cleanup v2

* fix address vs interface

* fix API calls

* Fix methodnames

* add vlan type

* add vlan support

* Fix tests

* Add wifi support

* more OOO

* fix typing import

* typing part 2

* Fix profile

* fix test payload

* ignore powersafe

* support privancy

* fix property

* Fix tests

* full support all API

* Fix all

* more robust

* Update supervisor/dbus/network/connection.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

* Fix gateway

* fix empty gateway

* Allow no ipv6 or ipv4 kernel support

* Exclude device drivers

* Add wifi

* Use loop on api

* refactory p1

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p2

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p3

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p4

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p5

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p6

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p7

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p8

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* Fix lint

* update sup p1

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* update sup p2

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* fix tests

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* fix logging

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* improve mock handling

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* add fixtures

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* fix tests

* better testing

* Add more tests

* Fix API test

* Add test for vlan payload

* Support variation

* Fix doc string

* support remove & wifi scan

* make sure we ignore local-link on ipv6

* remove privancy - add vlan

* Fix tests

* fix isort

* Fixture dbus by commands

* Add dnsmanager fixture

* expose commands called by dbus

* Add wifi tests

* Update supervisor/plugins/dns.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

* Address comments & fix tests

* change url to be closer on others

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2020-11-09 08:56:42 +01:00
Pascal Vizeli
042bdcdf37
Make startup more stable (#2167) 2020-10-23 16:48:03 +02:00
Pascal Vizeli
9c53caae80
Speedup HA core auth (#2144)
* Speedup HA core auth

* Add reset API call

* use delete

* Add complexe cache logic

* Allow manage api to handle auth reset/cache

* revert to only cache

* add tests

* ignore protected-access for this tests

* fix comment
2020-10-19 16:38:28 +02:00
Pascal Vizeli
0f60fdd20b
Make api soon available (#2119)
* Make api soon available

* add more tests
2020-10-12 15:56:29 +02:00
Joakim Sørensen
38db375fea
Add docker/info API (#2095) 2020-10-06 11:26:56 +02:00
Joakim Sørensen
1af90721cc
Addon repositories (#2071)
* stash

* Add test

* Use executor

* Make remove a coroutine

* Change logging and return
2020-09-22 23:40:36 +02:00