37 Commits

Author SHA1 Message Date
Stefan Agner
2ebb405871
Add enhanced logging REST endpoints using systemd-journal-gatewayd (#3291)
* Add enhanced logging REST endpoints using systemd-journal-gatewayd

Add /host/logs/entries and /host/logs/{identifier}/entries to expose log
entries from systemd-journald running on the host. Use
systemd-journal-gatewayd which exposes the logs to the Supervisor via
Unix socket.

Current two query string parameters are allowed: "boot" and "follow".
The first will only return logs since last boot. The second will keep
the HTTP request open and send new log entries as they get added to the
systemd-journal.

* Allow Range header

Forward the Range header to systemd-journal-gatewayd. This allows to
select only a certain amount of log data. The Range header is a standard
header to select only partial amount of data. However, the "entries="
prefix is custom for systemd-journal-gatewayd, denoting that the numbers
following represent log entries (as opposed to bytes or other metrics).

* Avoid connecting if systemd-journal-gatewayd is not available

* Use path for all options

* Add pytests

* Address pylint issues

* Boot ID offsets and slug to identifier

* Fix tests

* API refactor from feedback

* fix tests and add identifiers

* stop isort and pylint fighting

* fix tests

* Update default log identifiers

* Only modify /host/logs endpoints

* Fix bad import

* Load log caches asynchronously at startup

* Allow task to complete in fixture

* Boot IDs and identifiers loaded on demand

* Add suggested identifiers

* Fix tests around boot ids

Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
2022-10-13 11:40:11 -04:00
Mike Degatano
618c89c4d8
Don't sync unmanaged interfaces (#3925) 2022-10-04 20:17:30 +02:00
Mike Degatano
48e666e1fc
Bad introspection causes DBus error not KeyError (#3898)
* Bad introspection causes DBus error not KeyError

* Fix none error on disconnect race
2022-09-26 18:27:08 +02:00
Mike Degatano
a5103cc329
Add restart policy evaluation (#3886)
* Add restart policy evaluation

* No container meta does not fail evaluation
2022-09-22 09:16:33 +02:00
Mike Degatano
99bc201688
Listen for dbus property changes (#3872)
* Listen for dbus property changes

* Avoid remaking dbus proxy objects

* proper snake case for pylint

* some cleanup and more tests
2022-09-17 09:55:41 +02:00
Mike Degatano
d195f19fa8
Refactor to dbus-next proxy interfaces (#3862)
* Refactor to dbus-next proxy interfaces

* Fix tests mocking dbus methods

* Fix call dbus
2022-09-13 13:45:28 -04:00
Mike Degatano
4f272ad4fd
Dbus refactor and tests (#3854)
* Dbus refactor and tests

* Link to timedate introspection
2022-09-12 11:59:56 -04:00
Mike Degatano
b4e1e3e853
Identify and handle dhcp issues (#3806)
* Identify and handle dhcp issues

* Change test from DHCP to Connection Problem
2022-08-23 13:57:16 +02:00
Mike Degatano
c3f7a45d61
Fix memory calculation for cgroupv2 (#3802) 2022-08-17 16:50:08 -04:00
Franck Nijhof
3f88236495
Support for Docker manifests base images add-on build (#3724)
* Support for Docker manifests base images add-on build

* Set platform for build and tests

* Remove empty test

Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
2022-08-16 14:34:32 +02:00
Mike Degatano
ccd2c31390
Move repository urls to store settings file (#3665)
* Move repository urls to store settings file

* Remove default repos from supervisor config

* Fix clone at initial store load

* Mock git load in repository fixture
2022-06-07 10:02:21 +02:00
Mike Degatano
32d1296da1
Merge updates with existing network settings (#3570)
* Preserve network customizations over updates

* Fix issues from testing

* Test case to ensure updates generated correctly
2022-04-18 09:49:30 +02:00
Mike Degatano
e2ac5042d8
Apply network settings on start to fix defaults (#3528)
* Apply network settings on start to fix defaults

* Use job check and add tests
2022-03-31 12:14:40 +02:00
Mike Degatano
dd3a4a1f47
Add unsupported message if no systemd-resolved (#3487)
* Require resolved for supported systems

* Added properties for dbus resolved
2022-03-07 10:38:24 +01:00
Joakim Sørensen
5503f93a75
Don't remove add-on repos if add-on is installed (#3364) 2021-12-14 21:04:31 +01:00
Stefan Agner
435f479984
D-Bus NetworkManager improvements (#3243)
* Introduce enum for network connectivity

* Differentiate between Bus Name and Interface consts

The Bus names and interfaces look quite similar in D-Bus: Both use dots
to separate words. Usually all interfaces available below a certan Bus
name start with the Bus name. Quite often the Bus name itself is also
available as an interface.

However, those are different things. To avoid confusion, add the type of
const to the const name.

* Remove unused const

* Disconnect D-Bus when not used

Make sure Python disconnects from D-Bus when objects get destroyed. This
avoids exhausting D-Bus connection limit which causes the following
error message:
[system] The maximum number of active connections for UID 0 has been reached (max_connections_per_user=256)

* Filter signals by object as well

Make sure we only listen to signals on that particular object. Also
support filtering messages via message filter callback.

* Explicitly wait until Connection is activated

Wait for activated or raise an error. This avoids too early/errornous
updates when state of the connection changes to "activating" or similar
intermediate signal states.

Fixes: #2639

* Fix VLAN configuration

* Add link to D-Bus object documentation

* Fix network settings update test

* Make MessageBus object optional
2021-10-19 16:38:52 +02:00
Stefan Agner
7a6663ba80
Use Python dbus-next D-Bus library (#3234)
* Use the correct interface name to get properties of systemd

It seems that gdbus (or systemd) automatically pick the correct
interface and return the properties. However, dbussy requires the
correct interface name to get all properties.

* Don't expect array from Strength property

The property returns a type "y" which equates to "guchar":
https://developer-old.gnome.org/NetworkManager/stable/gdbus-org.freedesktop.NetworkManager.AccessPoint.html#gdbus-property-org-freedesktop-NetworkManager-AccessPoint.Strength

It seems that the old D-Bus implementation returned an array. With
dbus-next a integer is returned, so no list indexing required.

* Support signals and remove no longer used tests and code

* Pass rauc update file path as string

That is what the interface is expecting, otherwise the new lib chocks on
the Pathlib type.

* Support Network configuration with dbus-next

Assemble Python native objects and pass them to dbus-next. Use dbus-next
specific Variant class where necessary.

* Use org.freedesktop.NetworkManager.Connection.Active.StateChanged

org.freedesktop.NetworkManager.Connection.Active.PropertyChanged is
depricated. Also it seems that StateChanged leads to fewer and more
accurate signals.

* Pass correct data type to RequestScan.

RequestScan expects an option dictionary. Pass an empty option
dictionary to it.

* Update unit tests

Replace gdbus specific fixtures with json files representing the return
values. Those can be easily converted into native Python objects.

* Rename D-Bus utils module gdbus to dbus
2021-10-18 23:06:44 +02:00
Pascal Vizeli
288d2e5bdb
Use deepmerge for options (#3162)
* Use deepmerge for options

* fix issue

* Update supervisor/addons/addon.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

* Add tests

* Fix merge schema

* Make save for overwrites

* drop community

* more cleanup

* Fix tests

* Fix lists

* revert strategy

* protect overwritten lists

* Update tests/api/test_store.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2021-09-29 09:37:04 +02:00
Joakim Sørensen
40bcee38f3
Add startup time and boot_timestamp (#3136)
* Add startup time

* Add boot_timestamp

* Fix type

* patch and rename properties fixture
2021-09-22 11:15:38 +02:00
Pascal Vizeli
53eae96a98
DataDisk reload devices (#3129)
* DataDisk reload devices

* improve loading

* simplify

* validate input device

* add comments

* Add agent version to API

* more tests

* fix test lint
2021-09-20 12:52:51 +02:00
Pascal Vizeli
2b6829a786
OS-Agent support (#2811)
* OS-Agent support

* add agent to host feature

* Add support for os-agent on devcontainer

* Rename core

* fix tests

* add setter

* add cgroup / apparmor

* all interfaces added

* fix import

* Add tests

* More tests

* Finish tests

* reformating xml files

* fix doc string

* address comments

* change return value

* fix tests

* Update supervisor/dbus/agent/__init__.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

* Update scripts/supervisor.sh

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2021-06-30 11:55:53 +02:00
Pascal Vizeli
a4a0b43d91
Using ghcr.io/home-assistant + fallback (#2959) 2021-06-14 20:07:44 +02:00
Joakim Sørensen
b70ed9a60d
Rename fixtures (#2946)
* Rename fixtures

* Use tilde
2021-06-10 12:45:39 +02:00
Pascal Vizeli
bcef34012d
Time handling (#2901)
* Add new time handling

* migrate date for python3.9

* add timedate

* add tests & simplify it

* better testing

* use ssl

* use hostname with new interface

* expose to API

* update data

* add base handler

* new timezone handling

* improve handling

* Improve handling

* Add tests

* Time adjustment function

* Fix logging

* tweak condition

* don't adjust synchronized time

* Guard

* ignore UTC

* small cleanup

* like that, we can leaf it

* add URL

* add comment

* Apply suggestions from code review

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2021-06-09 09:38:32 +02:00
mdegat01
667672a20b
Addons can access systemd journal logs (#2722)
* Added journald access to addons

* Name change to journald and added tests
2021-03-17 12:26:23 +01:00
Joakim Sørensen
752068bb56
Add /store API (#2626) 2021-02-25 16:15:51 +01:00
Joakim Sørensen
7ee5737f75
Activate new network settings after update (#2253) 2020-11-13 18:15:46 +01:00
Pascal Vizeli
5552b1da49
Optimize flow / reduce call time (#2250)
* Optimize flow / reduce call time

* rename

* freeze too

* add connectivity task

* use newstyle timeout

* Fix tests

Co-authored-by: Ludeeus <ludeeus@ludeeus.dev>
2020-11-13 12:19:10 +01:00
Pascal Vizeli
c4847ad10d
Handle dbus single byte (#2237) 2020-11-11 13:09:13 +01:00
Pascal Vizeli
9264d437b1
Fix network API default value & tests (#2234) 2020-11-10 15:08:35 +01:00
Pascal Vizeli
bd786811a3
Network: abstract dbus and supervisor - ipv6/wifi/vlan (#2217)
* Abstract code between dbus - supervisor

* cleanup v2

* fix address vs interface

* fix API calls

* Fix methodnames

* add vlan type

* add vlan support

* Fix tests

* Add wifi support

* more OOO

* fix typing import

* typing part 2

* Fix profile

* fix test payload

* ignore powersafe

* support privancy

* fix property

* Fix tests

* full support all API

* Fix all

* more robust

* Update supervisor/dbus/network/connection.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

* Fix gateway

* fix empty gateway

* Allow no ipv6 or ipv4 kernel support

* Exclude device drivers

* Add wifi

* Use loop on api

* refactory p1

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p2

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p3

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p4

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p5

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p6

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p7

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* refactory p8

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* Fix lint

* update sup p1

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* update sup p2

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* fix tests

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* fix logging

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* improve mock handling

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* add fixtures

Signed-off-by: Pascal Vizeli <pvizeli@syshack.ch>

* fix tests

* better testing

* Add more tests

* Fix API test

* Add test for vlan payload

* Support variation

* Fix doc string

* support remove & wifi scan

* make sure we ignore local-link on ipv6

* remove privancy - add vlan

* Fix tests

* fix isort

* Fixture dbus by commands

* Add dnsmanager fixture

* expose commands called by dbus

* Add wifi tests

* Update supervisor/plugins/dns.py

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>

* Address comments & fix tests

* change url to be closer on others

Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
2020-11-09 08:56:42 +01:00
Joakim Sørensen
b513512551
Normalize CPU stats (#2154)
* Normalize CPU stats

* Change fixture to make it clearer

* guard for 0

* Update supervisor/docker/stats.py

Co-authored-by: Pascal Vizeli <pvizeli@syshack.ch>

* Update stats.py

* Update stats.py

Co-authored-by: Pascal Vizeli <pvizeli@syshack.ch>
2020-10-20 22:26:09 +02:00
Joakim Sørensen
25ea80e169
Fixes issues with managing wireless interfaces (#2015)
* Fixes issues with managing wireless interfaces

* Add test

* Add back fixture
2020-09-04 20:58:37 +02:00
Joakim Sørensen
9bcb15dbc0
MVP: Add Network Manager context (#1937)
Co-authored-by: Pascal Vizeli <pvizeli@syshack.ch>
2020-08-24 16:58:02 +02:00
Pascal Vizeli
bb64dca6e6
Fix symlink snapshot (#1833)
* Fix symbolic link issue with snapshot

* Add tests

* fix symlink

* add encrypted test

* Modify x
2020-07-14 15:39:03 +02:00
Pascal Vizeli
35aae69f23
Support armv7 and allow support of multible arch types per CPU (#892)
* Support armv7 and first abstraction

* Change layout

* Add more type hints

* Fix imports

* Update

* move forward

* add tests

* fix type

* fix lint & tests

* fix tests

* Fix unittests

* Fix create folder

* cleanup

* Fix import order

* cleanup loop parameter

* cleanup init function

* Allow changeable image name

* fix setup

* Fix load of arch

* Fix lint

* Add typing

* fix init

* fix hassos cli problem & stick on supervisor arch

* address comments

* cleanup

* Fix image selfheal

* Add comment

* update uvloop

* remove uvloop

* fix tagging

* Fix install name

* Fix validate build config

* Abstract image_name from system cache
2019-01-31 18:47:44 +01:00
Pascal Vizeli
05b58d76b9
Add tests for hass.io (#817)
* Add tests for hass.io

* Fix folder

* Fix test command
2018-11-18 12:08:46 +01:00