* Fix mypy issues in store module
* Fix mypy issues in utils module
* Fix mypy issues in all remaining source files
* Fix ingress user typeddict
* Fixes from feedback
* Fix mypy issues after installing docker-types
* Fix mypy issues in backups module
* Fix mypy issues in dbus module
* Fix mypy issues in api after rebase
* TypedDict to dataclass and other small fixes
* Finish fixing mypy errors in dbus
* local_where must exist
* Fix references to name in tests
It seems that some systems continously run into D-Bus errors
overwhelming the system itself but also generating lots of errors on
Sentry. Rate limit D-Bus errors to 3 for every message every 30s.
* Move read_text to executor
* Fix issues found by coderabbit
* formated to formatted
* switch to async_capture_exception
* Find and replace got one too many
* Update patch mock to async_capture_exception
* Drop Sentry capture from format_message
The error handling got introduced in #2052, however, #2100 essentially
makes sure there will never be a byte object passed to this function.
And even if, the Sentry aiohttp plug-in will properly catch such an
exception.
---------
Co-authored-by: Stefan Agner <stefan@agner.ch>
* Improve D-Bus timeout error handling
Typically D-Bus timeouts are related to systemd activation timing out
after 25s. The current dbus-fast timeout of 10s is well below that
so we never get the actual D-Bus error. This increases the dbus-fast
timeout to 30s, which will make sure we wait long enought to get the
actual D-Bus error from the broker.
Note that this should not slow down a typical system, since we tried
three times each waiting for 10s. With the new error handling typically
we'll end up waiting 25s and then receive the actual D-Bus error. There
is no point in waiting for multiple D-Bus/systemd caused timeouts.
* Create D-Bus TimedOut exception
* Improve and extend error handling on D-Bus connect
Avoid initializing the Supervisor board since it does not support the
Properties interface (see https://github.com/home-assistant/os-agent/issues/206).
This prevents the following somewhat confusing warning:
No OS-Agent support on the host. Some Host functions have been disabled.
The OS Agent is actually installed on the host, it is just a single
object which caused issues. No functionalty was actually lost, as the
Supervisor board object has no features currently, and all other
interfaces got properly initialized still (thanks to gather()).
Print warnings more fine graned so it is clear which object exactly
causes an issue. Also print a log message on the lowest layer when an
error occures on calling D-Bus. This allows to easier track the actual
D-Bus error source.
Fixes: #5241
* Fix tests
* Use local variable
* Avoid stack trace when board support fails to load
* Fix tests
* Use override mechanism to disable Properties support
Instead of disable loading of Supervised entirly override initialization
to prevent loading the Properties interface.
* Revert "Fix tests"
This reverts commit 1e3c491ace.
* Use signals to recognize new disks immediately
* Add test for disabled data disk issue
* Add mock of UDisks2 base service to test
* Apply suggestions from code review
* Shutdown manager first to avoid potential race conditions
* Update tests/dbus_service_mocks/udisks2.py
Co-authored-by: Jan Čermák <sairon@users.noreply.github.com>
---------
Co-authored-by: Stefan Agner <stefan@agner.ch>
Co-authored-by: Jan Čermák <sairon@users.noreply.github.com>
* Migrate to Ruff for lint and format
* Fix pylint issues
* DBus property sets into normal awaitable methods
* Fix tests relying on separate tasks in connect
* Fixes from feedback
* Improve D-Bus error handling for NetworkManager
There are quite some errors captured which are related by seemingly a
suddenly missing NetworkManager. Errors appear as:
23-11-21 17:42:50 ERROR (MainThread) [supervisor.dbus.network] Error while processing /org/freedesktop/NetworkManager/Devices/10: Remote peer disconnected
...
23-11-21 17:42:50 ERROR (MainThread) [supervisor.dbus.network] Error while processing /org/freedesktop/NetworkManager/Devices/35: The name is not activatable
Both errors seem to already happen at introspection time, however
the current code doesn't converts these errors to Supervisor issues.
This PR uses the already existing `DBus.from_dbus_error()`.
Furthermore this adds a new Exception `DBusNoReplyError` for the
`ErrorType.NO_REPLY` (or `org.freedesktop.DBus.Error.NoReply` in
D-Bus terms, which is the type of the first of the two issues above).
And finally it separates the `ErrorType.SERVICE_UNKNOWN` (or
`org.freedesktop.DBus.Error.ServiceUnknown` in D-Bus terms, which is
the second of the above issue) from `DBusInterfaceError` into a new
`DBusServiceUnkownError`.
This allows to handle errors more specifically.
To avoid too much churn, all instances where `DBusInterfaceError`
got handled, we are now also handling `DBusServiceUnkownError`.
The `DBusNoReplyError` and `DBusServiceUnkownError` appear when
the NetworkManager service stops or crashes. Instead of retrying
every interface we know, just give up if one of these issues appear.
This should significantly lower error messages users are seeing
and Sentry events.
* Remove unnecessary statement
* Fix pytests
* Make sure error strings are compared correctly
* Fix typo/remove unnecessary pylint exception
* Fix DBusError typing
* Add pytest for from_dbus_error
* Revert "Make sure error strings are compared correctly"
This reverts commit 10dc2e4c3887532921414b4291fe3987186db408.
* Add test cases
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Add udisks2 dbus support
* assert mountpoints
* Comment
* Add reference links
* docstring
* fix type
* fix type
* add typing extensions as import
* isort
* additional changes
* Simplify classes and conversions, fix bugs
* More simplification
* Fix imports
* fix pip
* Add additional properties and fix requirements
* fix tests maybe
* Handle optionality of certain configuration details
* black
* connect to devices before returning them
* Refactor for latest dbus work
* Not .items
* fix mountpoints logic
* use variants
* Use variants for options too
* isort
* Switch to dbus fast
* Move import to parent
* Add some fixture data
* Add another fixture and reduce the block devices list
* Implement changes discussed with mike
* Add property fixtures
* update object path
* Fix get_block_devices call
* Tests and refactor to minimize dbus reconnects
* Call super init in DBusInterfaceProxy
* Fix permissions on introspection files
---------
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
* Make dbus-fast calls more robust
* Handle all exceptions and add test
* DBus minimal can't return commands list
Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
Sometimes NetworkManager network devices disappear while we are query
them. Presumably those are network interfaces generated by Docker.
Skip them and print a warning.
* Allow to update connection information
* Introduce Signal wrapper class using async context manager
This allows to start monitoring signals and execute code before
processing signals. With that it is possible to check for state changes
in a race free manor.
* Fix unit tests
* Introduce enum for network connectivity
* Differentiate between Bus Name and Interface consts
The Bus names and interfaces look quite similar in D-Bus: Both use dots
to separate words. Usually all interfaces available below a certan Bus
name start with the Bus name. Quite often the Bus name itself is also
available as an interface.
However, those are different things. To avoid confusion, add the type of
const to the const name.
* Remove unused const
* Disconnect D-Bus when not used
Make sure Python disconnects from D-Bus when objects get destroyed. This
avoids exhausting D-Bus connection limit which causes the following
error message:
[system] The maximum number of active connections for UID 0 has been reached (max_connections_per_user=256)
* Filter signals by object as well
Make sure we only listen to signals on that particular object. Also
support filtering messages via message filter callback.
* Explicitly wait until Connection is activated
Wait for activated or raise an error. This avoids too early/errornous
updates when state of the connection changes to "activating" or similar
intermediate signal states.
Fixes: #2639
* Fix VLAN configuration
* Add link to D-Bus object documentation
* Fix network settings update test
* Make MessageBus object optional
* Use the correct interface name to get properties of systemd
It seems that gdbus (or systemd) automatically pick the correct
interface and return the properties. However, dbussy requires the
correct interface name to get all properties.
* Don't expect array from Strength property
The property returns a type "y" which equates to "guchar":
https://developer-old.gnome.org/NetworkManager/stable/gdbus-org.freedesktop.NetworkManager.AccessPoint.html#gdbus-property-org-freedesktop-NetworkManager-AccessPoint.Strength
It seems that the old D-Bus implementation returned an array. With
dbus-next a integer is returned, so no list indexing required.
* Support signals and remove no longer used tests and code
* Pass rauc update file path as string
That is what the interface is expecting, otherwise the new lib chocks on
the Pathlib type.
* Support Network configuration with dbus-next
Assemble Python native objects and pass them to dbus-next. Use dbus-next
specific Variant class where necessary.
* Use org.freedesktop.NetworkManager.Connection.Active.StateChanged
org.freedesktop.NetworkManager.Connection.Active.PropertyChanged is
depricated. Also it seems that StateChanged leads to fewer and more
accurate signals.
* Pass correct data type to RequestScan.
RequestScan expects an option dictionary. Pass an empty option
dictionary to it.
* Update unit tests
Replace gdbus specific fixtures with json files representing the return
values. Those can be easily converted into native Python objects.
* Rename D-Bus utils module gdbus to dbus
