Compare commits

..

26 Commits

Author SHA1 Message Date
ludeeus
e415923553 Trigger backup sync when backup is complete 2024-10-16 04:54:29 +00:00
dependabot[bot]
95c638991d Bump coverage from 7.6.2 to 7.6.3 (#5351) 2024-10-14 08:16:23 +02:00
Stefan Agner
e2ada42001 Fix log follow mode without range header (#5347) 2024-10-11 19:54:53 +02:00
dependabot[bot]
22e50b4ace Bump aiohttp from 3.10.9 to 3.10.10 (#5345)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-11 10:31:09 +02:00
dependabot[bot]
334484de7f Bump debugpy from 1.8.6 to 1.8.7 (#5346)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-11 10:12:26 +02:00
Stefan Agner
180a7c3990 Throttle connectivity check on connectivity issue (#5342)
* Throttle connectivity check on connectivity issue

If Supervisor detects a connectivity issue, currenlty every function
which requires internet get delayed by 10s due to the connectivity
check. This especially slows down initial startup when there are
connectivity issues. It is unlikely to resolve immeaditly, so throttle
the connectivity check to check every 30s.

* Fix pytest

* Reset throttle in test and refactor helper

* CodeRabbit suggestion

---------

Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
2024-10-10 22:57:16 +02:00
dependabot[bot]
d5f33de808 Bump coverage from 7.6.1 to 7.6.2 (#5344)
Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.6.1 to 7.6.2.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](https://github.com/nedbat/coveragepy/compare/7.6.1...7.6.2)

---
updated-dependencies:
- dependency-name: coverage
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-10 09:46:41 +02:00
dependabot[bot]
6539f0df6f Bump actions/upload-artifact from 4.4.2 to 4.4.3 (#5343)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.4.2 to 4.4.3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4.4.2...v4.4.3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-10 09:46:18 +02:00
Stefan Agner
1504278223 Recreate aiohttp session on connectivity check (#5332)
It seems to actually get a proper connectivity check run, we need a new
vanilla ClientSession() object.
2024-10-09 21:41:46 +02:00
Jan Čermák
9f3767b23d Return cursor of the first host logs entry via headers (#5333)
* Return cursor of the first host logs entry via headers

Return first entry's cursor via custom `X-First-Cursor` header that can
be consumed by the client and used for continual requesting of the
historic logs. Once the first fetch returns data, the cursor can be
supplied as the first argument to the Range header in another call,
fetching accurate slice of the journal with the previous log entries
using the `Range: entries=cursor[[:num_skip]:num_entries]` syntax.

Let's say we fetch logs with the Range header `entries=:-19:20` (to
fetch very last 20 lines of the logs, see below why not
`entries:-20:20`) and we get `cursor50` as the reply (the actual value
will be much more complex and with no guaranteed format). To fetch
previous slice of the logs, we use `entries=cursor50:-20:20`, which
would return 20 lines previous to `cursor50` and `cursor30` in the
cursor header. This way we can go all the way back to the history.

One problem with the cursor is that it's not possible to determine when
the negative num_skip points beyond the first log entry. In that case
the client either needs to know what the first entry is (via
`entries=:0:1`) or can iterate naively and stop once two subsequent
requests return the same first cursor.

Another caveat, even though it's unlikely it will be hit in real usage,
is that it's not possible to fetch the last line only - if no cursor is
provided, negative num_skip argument is needed, and in that case we're
pointing one record back from the current cursor, which is the previous
record. The least we can return without knowing any cursor is thus
`entries=👎2` (where the `2` can be omitted, however with
`entries=👎1` we would lose the last line). This also explains why
different `num_skip` and `num_entries` must be used for the first fetch.

* Fix typo (fallback->callback)

* Refactor journal_logs_reader to always return the cursor

* Update tests for new cursor handling
2024-10-09 20:27:29 +02:00
Jan Čermák
e0d7985369 Fix number of lines returned with host logs' query argument (#5334)
If no cursor is specified and negative num_skip is used, we're pointing
one record back from the last one, so host logs always returned 101
lines as the default. This was also the case of the lines query argument
that used the number directly as num_skip. Instead of doing that, point
N-1 records to the back and then get N records. Handle 1 record and
invalid numbers silently to avoid the need for error handling in
unpractical edge cases.
2024-10-09 20:27:18 +02:00
Stefan Agner
2968a5717c Update DNS plug-in on network change (#5331)
* Update DNS plug-in on network change

Restart the DNS plug-in when the primary network changes network
changes. This makes sure any potential host OS DNS configuration
changes get picked up by the DNS plug-in as well.

* Add a test case

---------

Co-authored-by: Mike Degatano <michael.degatano@gmail.com>
2024-10-09 20:16:36 +02:00
Matheson Steplock
e2b25fe7ce Drop support for Debian 11 Bullseye (#5335) 2024-10-09 12:18:45 +02:00
dependabot[bot]
8601f5c49a Bump sentry-sdk from 2.15.0 to 2.16.0 (#5338)
Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 2.15.0 to 2.16.0.
- [Release notes](https://github.com/getsentry/sentry-python/releases)
- [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-python/compare/2.15.0...2.16.0)

---
updated-dependencies:
- dependency-name: sentry-sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-09 11:19:47 +02:00
dependabot[bot]
42279461e0 Bump pre-commit from 4.0.0 to 4.0.1 (#5337)
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v4.0.0...v4.0.1)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-09 11:19:31 +02:00
dependabot[bot]
409447d6ca Bump actions/cache from 4.1.0 to 4.1.1 (#5340)
Bumps [actions/cache](https://github.com/actions/cache) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v4.1.0...v4.1.1)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-09 11:19:02 +02:00
dependabot[bot]
5b313db49d Bump time-machine from 2.15.0 to 2.16.0 (#5336)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-09 09:27:57 +02:00
dependabot[bot]
d64618600d Bump actions/upload-artifact from 4.4.1 to 4.4.2 (#5339)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-09 09:26:23 +02:00
dependabot[bot]
1ee01b1d5e Bump actions/checkout from 4.2.0 to 4.2.1 (#5329)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 09:04:52 +02:00
dependabot[bot]
af590202c3 Bump actions/upload-artifact from 4.4.0 to 4.4.1 (#5330)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 09:04:36 +02:00
dependabot[bot]
12ca2fb624 Bump ruff from 0.6.8 to 0.6.9 (#5326)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 09:22:50 +02:00
dependabot[bot]
ea95f83742 Bump dbus-fast from 2.24.2 to 2.24.3 (#5328)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 09:18:12 +02:00
dependabot[bot]
e4d4da601c Bump aiohttp from 3.10.8 to 3.10.9 (#5327)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 09:01:00 +02:00
dependabot[bot]
0582f6fd39 Bump pre-commit from 3.8.0 to 4.0.0 (#5325)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 08:45:43 +02:00
dependabot[bot]
f254af8326 Bump actions/cache from 4.0.2 to 4.1.0 (#5323)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 08:40:52 +02:00
dependabot[bot]
3333770246 Bump sigstore/cosign-installer from 3.6.0 to 3.7.0 (#5324)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 08:37:08 +02:00
24 changed files with 263 additions and 82 deletions

View File

@@ -53,7 +53,7 @@ jobs:
requirements: ${{ steps.requirements.outputs.changed }}
steps:
- name: Checkout the repository
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
with:
fetch-depth: 0
@@ -92,7 +92,7 @@ jobs:
arch: ${{ fromJson(needs.init.outputs.architectures) }}
steps:
- name: Checkout the repository
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
with:
fetch-depth: 0
@@ -131,7 +131,7 @@ jobs:
- name: Install Cosign
if: needs.init.outputs.publish == 'true'
uses: sigstore/cosign-installer@v3.6.0
uses: sigstore/cosign-installer@v3.7.0
with:
cosign-release: "v2.4.0"
@@ -178,7 +178,7 @@ jobs:
steps:
- name: Checkout the repository
if: needs.init.outputs.publish == 'true'
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Initialize git
if: needs.init.outputs.publish == 'true'
@@ -203,7 +203,7 @@ jobs:
timeout-minutes: 60
steps:
- name: Checkout the repository
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Build the Supervisor
if: needs.init.outputs.publish != 'true'

View File

@@ -25,7 +25,7 @@ jobs:
name: Prepare Python dependencies
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Set up Python
id: python
uses: actions/setup-python@v5.2.0
@@ -33,7 +33,7 @@ jobs:
python-version: ${{ env.DEFAULT_PYTHON }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: venv
key: |
@@ -47,7 +47,7 @@ jobs:
pip install -r requirements.txt -r requirements_tests.txt
- name: Restore pre-commit environment from cache
id: cache-precommit
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: ${{ env.PRE_COMMIT_CACHE }}
lookup-only: true
@@ -67,7 +67,7 @@ jobs:
needs: prepare
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Set up Python ${{ needs.prepare.outputs.python-version }}
uses: actions/setup-python@v5.2.0
id: python
@@ -75,7 +75,7 @@ jobs:
python-version: ${{ needs.prepare.outputs.python-version }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: venv
key: |
@@ -87,7 +87,7 @@ jobs:
exit 1
- name: Restore pre-commit environment from cache
id: cache-precommit
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: ${{ env.PRE_COMMIT_CACHE }}
key: |
@@ -110,7 +110,7 @@ jobs:
needs: prepare
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Set up Python ${{ needs.prepare.outputs.python-version }}
uses: actions/setup-python@v5.2.0
id: python
@@ -118,7 +118,7 @@ jobs:
python-version: ${{ needs.prepare.outputs.python-version }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: venv
key: |
@@ -130,7 +130,7 @@ jobs:
exit 1
- name: Restore pre-commit environment from cache
id: cache-precommit
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: ${{ env.PRE_COMMIT_CACHE }}
key: |
@@ -153,7 +153,7 @@ jobs:
needs: prepare
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Register hadolint problem matcher
run: |
echo "::add-matcher::.github/workflows/matchers/hadolint.json"
@@ -168,7 +168,7 @@ jobs:
needs: prepare
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Set up Python ${{ needs.prepare.outputs.python-version }}
uses: actions/setup-python@v5.2.0
id: python
@@ -176,7 +176,7 @@ jobs:
python-version: ${{ needs.prepare.outputs.python-version }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: venv
key: |
@@ -188,7 +188,7 @@ jobs:
exit 1
- name: Restore pre-commit environment from cache
id: cache-precommit
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: ${{ env.PRE_COMMIT_CACHE }}
key: |
@@ -212,7 +212,7 @@ jobs:
needs: prepare
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Set up Python ${{ needs.prepare.outputs.python-version }}
uses: actions/setup-python@v5.2.0
id: python
@@ -220,7 +220,7 @@ jobs:
python-version: ${{ needs.prepare.outputs.python-version }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: venv
key: |
@@ -232,7 +232,7 @@ jobs:
exit 1
- name: Restore pre-commit environment from cache
id: cache-precommit
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: ${{ env.PRE_COMMIT_CACHE }}
key: |
@@ -256,7 +256,7 @@ jobs:
needs: prepare
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Set up Python ${{ needs.prepare.outputs.python-version }}
uses: actions/setup-python@v5.2.0
id: python
@@ -264,7 +264,7 @@ jobs:
python-version: ${{ needs.prepare.outputs.python-version }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: venv
key: |
@@ -288,19 +288,19 @@ jobs:
name: Run tests Python ${{ needs.prepare.outputs.python-version }}
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Set up Python ${{ needs.prepare.outputs.python-version }}
uses: actions/setup-python@v5.2.0
id: python
with:
python-version: ${{ needs.prepare.outputs.python-version }}
- name: Install Cosign
uses: sigstore/cosign-installer@v3.6.0
uses: sigstore/cosign-installer@v3.7.0
with:
cosign-release: "v2.4.0"
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: venv
key: |
@@ -335,7 +335,7 @@ jobs:
-o console_output_style=count \
tests
- name: Upload coverage artifact
uses: actions/upload-artifact@v4.4.0
uses: actions/upload-artifact@v4.4.3
with:
name: coverage-${{ matrix.python-version }}
path: .coverage
@@ -347,7 +347,7 @@ jobs:
needs: ["pytest", "prepare"]
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Set up Python ${{ needs.prepare.outputs.python-version }}
uses: actions/setup-python@v5.2.0
id: python
@@ -355,7 +355,7 @@ jobs:
python-version: ${{ needs.prepare.outputs.python-version }}
- name: Restore Python virtual environment
id: cache-venv
uses: actions/cache@v4.0.2
uses: actions/cache@v4.1.1
with:
path: venv
key: |

View File

@@ -11,7 +11,7 @@ jobs:
name: Release Drafter
steps:
- name: Checkout the repository
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
with:
fetch-depth: 0

View File

@@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out code from GitHub
uses: actions/checkout@v4.2.0
uses: actions/checkout@v4.2.1
- name: Sentry Release
uses: getsentry/action-release@v1.7.0
env:

View File

@@ -1,5 +1,5 @@
aiodns==3.2.0
aiohttp==3.10.8
aiohttp==3.10.10
atomicwrites-homeassistant==1.4.1
attrs==24.2.0
awesomeversion==24.6.0
@@ -8,7 +8,7 @@ ciso8601==2.3.1
colorlog==6.8.2
cpe==1.3.1
cryptography==43.0.1
debugpy==1.8.6
debugpy==1.8.7
deepmerge==2.0
dirhash==0.5.0
docker==7.1.0
@@ -21,9 +21,9 @@ pyudev==0.24.3
PyYAML==6.0.2
requests==2.32.3
securetar==2024.2.1
sentry-sdk==2.15.0
sentry-sdk==2.16.0
setuptools==75.1.0
voluptuous==0.15.2
dbus-fast==2.24.2
dbus-fast==2.24.3
typing_extensions==4.12.2
zlib-fast==0.2.0

View File

@@ -1,12 +1,12 @@
coverage==7.6.1
pre-commit==3.8.0
coverage==7.6.3
pre-commit==4.0.1
pylint==3.3.1
pytest-aiohttp==1.0.5
pytest-asyncio==0.23.6
pytest-cov==5.0.0
pytest-timeout==2.3.1
pytest==8.3.3
ruff==0.6.8
time-machine==2.15.0
ruff==0.6.9
time-machine==2.16.0
typing_extensions==4.12.2
urllib3==2.2.3

View File

@@ -61,7 +61,7 @@ _LOGGER: logging.Logger = logging.getLogger(__name__)
IDENTIFIER = "identifier"
BOOTID = "bootid"
DEFAULT_RANGE = 100
DEFAULT_LINES = 100
SCHEMA_OPTIONS = vol.Schema({vol.Optional(ATTR_HOSTNAME): str})
@@ -226,13 +226,26 @@ class APIHost(CoreSysAttributes):
log_formatter = LogFormatter.VERBOSE
if "lines" in request.query:
lines = request.query.get("lines", DEFAULT_RANGE)
lines = request.query.get("lines", DEFAULT_LINES)
try:
lines = int(lines)
except ValueError:
# If the user passed a non-integer value, just use the default instead of error.
lines = DEFAULT_LINES
finally:
# We can't use the entries= Range header syntax to refer to the last 1 line,
# and passing 1 to the calculation below would return the 1st line of the logs
# instead. Since this is really an edge case that doesn't matter much, we'll just
# return 2 lines at minimum.
lines = max(2, lines)
# entries=cursor[[:num_skip]:num_entries]
range_header = f"entries=:-{lines}:"
range_header = f"entries=:-{lines-1}:{'' if follow else lines}"
elif RANGE in request.headers:
range_header = request.headers.get(RANGE)
else:
range_header = f"entries=:-{DEFAULT_RANGE}:"
range_header = (
f"entries=:-{DEFAULT_LINES-1}:{'' if follow else DEFAULT_LINES}"
)
async with self.sys_host.logs.journald_logs(
params=params, range_header=range_header, accept=LogFormat.JOURNAL
@@ -240,8 +253,13 @@ class APIHost(CoreSysAttributes):
try:
response = web.StreamResponse()
response.content_type = CONTENT_TYPE_TEXT
await response.prepare(request)
async for line in journal_logs_reader(resp, log_formatter):
headers_returned = False
async for cursor, line in journal_logs_reader(resp, log_formatter):
if not headers_returned:
if cursor:
response.headers["X-First-Cursor"] = cursor
await response.prepare(request)
headers_returned = True
await response.write(line.encode("utf-8") + b"\n")
except ConnectionResetError as ex:
raise APIError(

View File

@@ -10,7 +10,10 @@ from pathlib import Path
from ..addons.addon import Addon
from ..const import (
ATTR_DATA,
ATTR_DAYS_UNTIL_STALE,
ATTR_SLUG,
ATTR_TYPE,
FILE_HASSIO_BACKUPS,
FOLDER_HOMEASSISTANT,
CoreState,
@@ -21,7 +24,9 @@ from ..exceptions import (
BackupInvalidError,
BackupJobError,
BackupMountDownError,
HomeAssistantWSError,
)
from ..homeassistant.const import WSType
from ..jobs.const import JOB_GROUP_BACKUP_MANAGER, JobCondition, JobExecutionLimit
from ..jobs.decorator import Job
from ..jobs.job_group import JobGroup
@@ -299,6 +304,18 @@ class BackupManager(FileConfiguration, JobGroup):
# Ignore exceptions from waiting for addon startup, addon errors handled elsewhere
await asyncio.gather(*addon_start_tasks, return_exceptions=True)
try:
await self.sys_homeassistant.websocket.async_send_command(
{
ATTR_TYPE: WSType.BACKUP_SYNC,
ATTR_DATA: {
ATTR_SLUG: backup.slug,
},
},
)
except HomeAssistantWSError as err:
_LOGGER.error("Can't send backup sync to Home Assistant: %s", err)
return backup
finally:
self.sys_core.state = CoreState.RUNNING

View File

@@ -63,7 +63,7 @@ class CoreSys:
# External objects
self._loop: asyncio.BaseEventLoop = asyncio.get_running_loop()
self._websession: aiohttp.ClientSession = aiohttp.ClientSession()
self._websession = None
# Global objects
self._config: CoreConfig = CoreConfig()
@@ -96,10 +96,8 @@ class CoreSys:
self._bus: Bus | None = None
self._mounts: MountManager | None = None
# Set default header for aiohttp
self._websession._default_headers = MappingProxyType(
{aiohttp.hdrs.USER_AGENT: SERVER_SOFTWARE}
)
# Setup aiohttp session
self.create_websession()
# Task factory attributes
self._set_task_context: list[Callable[[Context], Context]] = []
@@ -548,6 +546,16 @@ class CoreSys:
return self.loop.run_in_executor(None, funct, *args)
def create_websession(self) -> None:
"""Create a new aiohttp session."""
if self._websession:
self.create_task(self._websession.close())
# Create session and set default header for aiohttp
self._websession: aiohttp.ClientSession = aiohttp.ClientSession(
headers=MappingProxyType({aiohttp.hdrs.USER_AGENT: SERVER_SOFTWARE})
)
def _create_context(self) -> Context:
"""Create a new context for a task."""
context = copy_context()

View File

@@ -32,6 +32,7 @@ class WSType(StrEnum):
SUPERVISOR_EVENT = "supervisor/event"
BACKUP_START = "backup/start"
BACKUP_END = "backup/end"
BACKUP_SYNC = "backup/sync"
class WSEvent(StrEnum):

View File

@@ -34,6 +34,7 @@ MIN_VERSION = {
WSType.SUPERVISOR_EVENT: "2021.2.4",
WSType.BACKUP_START: "2022.1.0",
WSType.BACKUP_END: "2022.1.0",
WSType.BACKUP_SYNC: "2024.10.99",
}
_LOGGER: logging.Logger = logging.getLogger(__name__)

View File

@@ -10,7 +10,9 @@ from ..coresys import CoreSys, CoreSysAttributes
from ..dbus.const import (
DBUS_ATTR_CONNECTION_ENABLED,
DBUS_ATTR_CONNECTIVITY,
DBUS_ATTR_PRIMARY_CONNECTION,
DBUS_IFACE_NM,
DBUS_OBJECT_BASE,
DBUS_SIGNAL_NM_CONNECTION_ACTIVE_CHANGED,
ConnectionStateType,
ConnectivityState,
@@ -148,6 +150,15 @@ class NetworkManager(CoreSysAttributes):
connectivity_check: bool | None = changed.get(DBUS_ATTR_CONNECTION_ENABLED)
connectivity: bool | None = changed.get(DBUS_ATTR_CONNECTIVITY)
# This potentially updated the DNS configuration. Make sure the DNS plug-in
# picks up the latest settings.
if (
DBUS_ATTR_PRIMARY_CONNECTION in changed
and changed[DBUS_ATTR_PRIMARY_CONNECTION]
and changed[DBUS_ATTR_PRIMARY_CONNECTION] != DBUS_OBJECT_BASE
):
await self.sys_plugins.dns.restart()
if (
connectivity_check is True
or DBUS_ATTR_CONNECTION_ENABLED in invalidated

View File

@@ -5,7 +5,7 @@ from ...coresys import CoreSys
from ..const import UnsupportedReason
from .base import EvaluateBase
SUPPORTED_OS = ["Debian GNU/Linux 11 (bullseye)", "Debian GNU/Linux 12 (bookworm)"]
SUPPORTED_OS = ["Debian GNU/Linux 12 (bookworm)"]
def setup(coresys: CoreSys) -> EvaluateBase:

View File

@@ -41,7 +41,7 @@ def _check_connectivity_throttle_period(coresys: CoreSys, *_) -> timedelta:
if coresys.supervisor.connectivity:
return timedelta(minutes=10)
return timedelta()
return timedelta(seconds=30)
class Supervisor(CoreSysAttributes):
@@ -274,6 +274,8 @@ class Supervisor(CoreSysAttributes):
"https://checkonline.home-assistant.io/online.txt", timeout=timeout
)
except (ClientError, TimeoutError):
# Need to recreate the websession to avoid stale connection checks
self.coresys.create_websession()
self.connectivity = False
else:
self.connectivity = True

View File

@@ -22,7 +22,7 @@ def formatter(required_fields: list[str]):
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
wrapper.required_fields = required_fields
wrapper.required_fields = ["__CURSOR"] + required_fields
return wrapper
return decorator
@@ -60,10 +60,12 @@ def journal_verbose_formatter(entries: dict[str, str]) -> str:
async def journal_logs_reader(
journal_logs: ClientResponse,
log_formatter: LogFormatter = LogFormatter.PLAIN,
) -> AsyncGenerator[str, None]:
"""Read logs from systemd journal line by line, formatted using the given formatter."""
journal_logs: ClientResponse, log_formatter: LogFormatter = LogFormatter.PLAIN
) -> AsyncGenerator[(str | None, str), None]:
"""Read logs from systemd journal line by line, formatted using the given formatter.
Returns a generator of (cursor, formatted_entry) tuples.
"""
match log_formatter:
case LogFormatter.PLAIN:
formatter_ = journal_plain_formatter
@@ -80,7 +82,7 @@ async def journal_logs_reader(
# at EOF (likely race between at_eof and EOF check in readuntil)
if line == b"\n" or not line:
if entries:
yield formatter_(entries)
yield entries.get("__CURSOR"), formatter_(entries)
entries = {}
continue

View File

@@ -6,7 +6,8 @@ from aiohttp.test_utils import TestClient
from supervisor.host.const import LogFormat
DEFAULT_LOG_RANGE = "entries=:-100:"
DEFAULT_LOG_RANGE = "entries=:-99:100"
DEFAULT_LOG_RANGE_FOLLOW = "entries=:-99:"
async def common_test_api_advanced_logs(
@@ -34,7 +35,7 @@ async def common_test_api_advanced_logs(
journald_logs.assert_called_once_with(
params={"SYSLOG_IDENTIFIER": syslog_identifier, "follow": ""},
range_header=DEFAULT_LOG_RANGE,
range_header=DEFAULT_LOG_RANGE_FOLLOW,
accept=LogFormat.JOURNAL,
)
@@ -62,6 +63,6 @@ async def common_test_api_advanced_logs(
"_BOOT_ID": "ccc",
"follow": "",
},
range_header=DEFAULT_LOG_RANGE,
range_header=DEFAULT_LOG_RANGE_FOLLOW,
accept=LogFormat.JOURNAL,
)

View File

@@ -14,7 +14,8 @@ from supervisor.host.control import SystemControl
from tests.dbus_service_mocks.base import DBusServiceMock
from tests.dbus_service_mocks.systemd import Systemd as SystemdService
DEFAULT_RANGE = "entries=:-100:"
DEFAULT_RANGE = "entries=:-99:100"
DEFAULT_RANGE_FOLLOW = "entries=:-99:"
# pylint: disable=protected-access
@@ -233,7 +234,7 @@ async def test_advanced_logs(
"SYSLOG_IDENTIFIER": coresys.host.logs.default_identifiers,
"follow": "",
},
range_header=DEFAULT_RANGE,
range_header=DEFAULT_RANGE_FOLLOW,
accept=LogFormat.JOURNAL,
)
@@ -249,7 +250,7 @@ async def test_advaced_logs_query_parameters(
await api_client.get("/host/logs?lines=53")
journald_logs.assert_called_once_with(
params={"SYSLOG_IDENTIFIER": coresys.host.logs.default_identifiers},
range_header="entries=:-53:",
range_header="entries=:-52:53",
accept=LogFormat.JOURNAL,
)
@@ -277,7 +278,7 @@ async def test_advaced_logs_query_parameters(
)
journald_logs.assert_called_once_with(
params={"SYSLOG_IDENTIFIER": coresys.host.logs.default_identifiers},
range_header="entries=:-53:",
range_header="entries=:-52:53",
accept=LogFormat.JOURNAL,
)
journal_logs_reader.assert_called_with(ANY, LogFormatter.VERBOSE)

View File

@@ -1169,6 +1169,40 @@ async def test_backup_progress(
]
async def test_backup_sync_notification(
coresys: CoreSys,
install_addon_ssh: Addon,
container: MagicMock,
ha_ws_client: AsyncMock,
tmp_supervisor_data,
path_extern,
):
"""Test backup sync notification."""
container.status = "running"
install_addon_ssh.path_data.mkdir()
coresys.core.state = CoreState.RUNNING
ha_ws_client.ha_version = AwesomeVersion("9999.9.9")
coresys.hardware.disk.get_disk_free_space = lambda x: 5000
ha_ws_client.async_send_command.reset_mock()
partial_backup: Backup = await coresys.backups.do_backup_partial()
await asyncio.sleep(0)
messages = [
call.args[0]
for call in ha_ws_client.async_send_command.call_args_list
if call.args[0]["type"] == WSType.BACKUP_SYNC
]
assert messages == [
{
"data": {
"slug": partial_backup.slug,
},
"type": WSType.BACKUP_SYNC,
},
]
async def test_restore_progress(
request: pytest.FixtureRequest,
coresys: CoreSys,

View File

@@ -1,12 +1,15 @@
"""Common test functions."""
from datetime import datetime
from importlib import import_module
from inspect import getclosurevars
import json
from pathlib import Path
from typing import Any
from dbus_fast.aio.message_bus import MessageBus
from supervisor.jobs.decorator import Job
from supervisor.resolution.validate import get_valid_modules
from supervisor.utils.yaml import read_yaml_file
@@ -82,3 +85,17 @@ async def mock_dbus_services(
services[module] = service_module.setup(to_mock[module]).export(bus)
return services
def get_job_decorator(func) -> Job:
"""Get Job object of decorated function."""
# Access the closure of the wrapper function
job = getclosurevars(func).nonlocals["self"]
if not isinstance(job, Job):
raise TypeError(f"{func.__qualname__} is not a Job")
return job
def reset_last_call(func, group: str | None = None) -> None:
"""Reset last call for a function using the Job decorator."""
get_job_decorator(func).set_last_call(datetime.min, group)

View File

@@ -7,6 +7,7 @@ from unittest.mock import PropertyMock, patch
import pytest
from supervisor.coresys import CoreSys
from supervisor.plugins.dns import PluginDns
from tests.dbus_service_mocks.network_manager import (
NetworkManager as NetworkManagerService,
@@ -84,3 +85,20 @@ async def test_connectivity_events(coresys: CoreSys, force: bool):
},
}
)
async def test_dns_restart_on_connection_change(
coresys: CoreSys, network_manager_service: NetworkManagerService
):
"""Test dns plugin is restarted when primary connection changes."""
await coresys.host.network.load()
with patch.object(PluginDns, "restart") as restart:
network_manager_service.emit_properties_changed({"PrimaryConnection": "/"})
await network_manager_service.ping()
restart.assert_not_called()
network_manager_service.emit_properties_changed(
{"PrimaryConnection": "/org/freedesktop/NetworkManager/ActiveConnection/2"}
)
await network_manager_service.ping()
restart.assert_called_once()

View File

@@ -40,9 +40,13 @@ async def test_logs(coresys: CoreSys, journald_gateway: MagicMock):
journald_gateway.feed_eof()
async with coresys.host.logs.journald_logs() as resp:
line = await anext(
cursor, line = await anext(
journal_logs_reader(resp, log_formatter=LogFormatter.VERBOSE)
)
assert (
cursor
== "s=83fee99ca0c3466db5fc120d52ca7dd8;i=203f2ce;b=f5a5c442fa6548cf97474d2d57c920b3;m=3191a3c620;t=612ccd299e7af;x=8675b540119d10bb"
)
assert (
line
== "2024-03-04 02:52:56.193 homeassistant systemd[1]: Started Hostname Service."
@@ -64,7 +68,11 @@ async def test_logs_coloured(coresys: CoreSys, journald_gateway: MagicMock):
journald_gateway.feed_eof()
async with coresys.host.logs.journald_logs() as resp:
line = await anext(journal_logs_reader(resp))
cursor, line = await anext(journal_logs_reader(resp))
assert (
cursor
== "s=83fee99ca0c3466db5fc120d52ca7dd8;i=2049389;b=f5a5c442fa6548cf97474d2d57c920b3;m=4263828e8c;t=612dda478b01b;x=9ae12394c9326930"
)
assert (
line
== "\x1b[32m24-03-04 23:56:56 INFO (MainThread) [__main__] Closing Supervisor\x1b[0m"

View File

@@ -26,8 +26,11 @@ from supervisor.jobs.job_group import JobGroup
from supervisor.os.manager import OSManager
from supervisor.plugins.audio import PluginAudio
from supervisor.resolution.const import UnhealthyReason
from supervisor.supervisor import Supervisor
from supervisor.utils.dt import utcnow
from tests.common import reset_last_call
async def test_healthy(coresys: CoreSys, caplog: pytest.LogCaptureFixture):
"""Test the healty decorator."""
@@ -73,6 +76,7 @@ async def test_internet(
):
"""Test the internet decorator."""
coresys.core.state = CoreState.RUNNING
reset_last_call(Supervisor.check_connectivity)
class TestClass:
"""Test class."""

View File

@@ -1,6 +1,6 @@
"""Test supervisor object."""
from datetime import datetime
from datetime import datetime, timedelta
import errno
from unittest.mock import AsyncMock, Mock, PropertyMock, patch
@@ -8,6 +8,7 @@ from aiohttp import ClientTimeout
from aiohttp.client_exceptions import ClientError
from awesomeversion import AwesomeVersion
import pytest
from time_machine import travel
from supervisor.const import UpdateChannel
from supervisor.coresys import CoreSys
@@ -22,6 +23,8 @@ from supervisor.resolution.const import ContextType, IssueType
from supervisor.resolution.data import Issue
from supervisor.supervisor import Supervisor
from tests.common import reset_last_call
@pytest.fixture(name="websession", scope="function")
async def fixture_webession(coresys: CoreSys) -> AsyncMock:
@@ -58,21 +61,33 @@ async def test_connectivity_check(
assert supervisor_unthrottled.connectivity is connectivity
@pytest.mark.parametrize("side_effect,call_count", [(ClientError(), 3), (None, 1)])
@pytest.mark.parametrize(
"side_effect,call_interval,throttled",
[
(None, timedelta(minutes=5), True),
(None, timedelta(minutes=15), False),
(ClientError(), timedelta(seconds=20), True),
(ClientError(), timedelta(seconds=40), False),
],
)
async def test_connectivity_check_throttling(
coresys: CoreSys,
websession: AsyncMock,
side_effect: Exception | None,
call_count: int,
call_interval: timedelta,
throttled: bool,
):
"""Test connectivity check throttled when checks succeed."""
coresys.supervisor.connectivity = None
websession.head.side_effect = side_effect
for _ in range(3):
reset_last_call(Supervisor.check_connectivity)
with travel(datetime.now(), tick=False) as traveller:
await coresys.supervisor.check_connectivity()
traveller.shift(call_interval)
await coresys.supervisor.check_connectivity()
assert websession.head.call_count == call_count
assert websession.head.call_count == (1 if throttled else 2)
async def test_update_failed(coresys: CoreSys, capture_exception: Mock):

View File

@@ -1,7 +1,7 @@
"""Test systemd journal utilities."""
import asyncio
from unittest.mock import MagicMock
from unittest.mock import ANY, MagicMock
import pytest
@@ -89,7 +89,7 @@ async def test_parsing_simple():
"""Test plain formatter."""
journal_logs, stream = _journal_logs_mock()
stream.feed_data(b"MESSAGE=Hello, world!\n\n")
line = await anext(journal_logs_reader(journal_logs))
_, line = await anext(journal_logs_reader(journal_logs))
assert line == "Hello, world!"
@@ -103,7 +103,7 @@ async def test_parsing_verbose():
b"_PID=666\n"
b"MESSAGE=Hello, world!\n\n"
)
line = await anext(
_, line = await anext(
journal_logs_reader(journal_logs, log_formatter=LogFormatter.VERBOSE)
)
assert line == "2013-09-17 07:32:51.000 homeassistant python[666]: Hello, world!"
@@ -118,7 +118,7 @@ async def test_parsing_newlines_in_message():
b"AFTER=after\n\n"
)
line = await anext(journal_logs_reader(journal_logs))
_, line = await anext(journal_logs_reader(journal_logs))
assert line == "Hello,\nworld!"
@@ -135,8 +135,8 @@ async def test_parsing_newlines_in_multiple_fields():
b"AFTER=after\n\n"
)
assert await anext(journal_logs_reader(journal_logs)) == "Hello,\nworld!\n"
assert await anext(journal_logs_reader(journal_logs)) == "Hello,\nworld!"
assert await anext(journal_logs_reader(journal_logs)) == (ANY, "Hello,\nworld!\n")
assert await anext(journal_logs_reader(journal_logs)) == (ANY, "Hello,\nworld!")
async def test_parsing_two_messages():
@@ -151,8 +151,31 @@ async def test_parsing_two_messages():
stream.feed_eof()
reader = journal_logs_reader(journal_logs)
assert await anext(reader) == "Hello, world!"
assert await anext(reader) == "Hello again, world!"
assert await anext(reader) == (ANY, "Hello, world!")
assert await anext(reader) == (ANY, "Hello again, world!")
with pytest.raises(StopAsyncIteration):
await anext(reader)
async def test_cursor_parsing():
"""Test cursor is extracted correctly."""
journal_logs, stream = _journal_logs_mock()
stream.feed_data(
b"__CURSOR=cursor1\n"
b"MESSAGE=Hello, world!\n"
b"ID=1\n\n"
b"__CURSOR=cursor2\n"
b"MESSAGE=Hello again, world!\n"
b"ID=2\n\n"
b"MESSAGE=No cursor\n"
b"ID=2\n\n"
)
stream.feed_eof()
reader = journal_logs_reader(journal_logs)
assert await anext(reader) == ("cursor1", "Hello, world!")
assert await anext(reader) == ("cursor2", "Hello again, world!")
assert await anext(reader) == (None, "No cursor")
with pytest.raises(StopAsyncIteration):
await anext(reader)
@@ -174,7 +197,7 @@ async def test_parsing_journal_host_logs():
"""Test parsing of real host logs."""
journal_logs, stream = _journal_logs_mock()
stream.feed_data(load_fixture("logs_export_host.txt").encode("utf-8"))
line = await anext(journal_logs_reader(journal_logs))
_, line = await anext(journal_logs_reader(journal_logs))
assert line == "Started Hostname Service."
@@ -182,7 +205,7 @@ async def test_parsing_colored_supervisor_logs():
"""Test parsing of real logs with ANSI escape sequences."""
journal_logs, stream = _journal_logs_mock()
stream.feed_data(load_fixture("logs_export_supervisor.txt").encode("utf-8"))
line = await anext(journal_logs_reader(journal_logs))
_, line = await anext(journal_logs_reader(journal_logs))
assert (
line
== "\x1b[32m24-03-04 23:56:56 INFO (MainThread) [__main__] Closing Supervisor\x1b[0m"