Compare commits

...

6 Commits

Author SHA1 Message Date
Stefan Agner
20f993e891 Avoid getting changed files for releases (#6381)
The changed files GitHub Action is not available for release events, so
we skip that step and directly set the output to false for releases.
This restores how releases worked before #6374.
2025-12-02 20:23:37 +01:00
Stefan Agner
d220fa801f Await aiodocker import_image coroutine (#6378)
The aiodocker images.import_image() method returns a coroutine that
needs to be awaited, but the code was iterating over it directly,
causing "TypeError: 'coroutine' object is not iterable".

Fixes SUPERVISOR-13D9

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-02 14:11:06 -05:00
Stefan Agner
abeee95eb1 Fix blocking I/O in git repository pull operation (#6380)
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-02 19:03:28 +01:00
Stefan Agner
50d31202ae Use Docker's official registry domain detection logic (#6360)
* Use Docker's official registry domain detection logic

Replace the custom IMAGE_WITH_HOST regex with a proper implementation
based on Docker's reference parser (vendor/github.com/distribution/
reference/normalize.go).

Changes:
- Change DOCKER_HUB from "hub.docker.com" to "docker.io" (official default)
- Add DOCKER_HUB_LEGACY for backward compatibility with "hub.docker.com"
- Add IMAGE_DOMAIN_REGEX and get_domain() function that properly detects:
  - localhost (with optional port)
  - Domains with "." (e.g., ghcr.io, 127.0.0.1)
  - Domains with ":" port (e.g., myregistry:5000)
  - IPv6 addresses (e.g., [::1]:5000)
- Update credential handling to support both docker.io and hub.docker.com
- Add comprehensive tests for domain detection

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Refactor Docker domain detection to utils module

Move get_domain function to supervisor/docker/utils.py and rename it
to get_domain_from_image for consistency with get_registry_for_image.
Use named group in the regex for better readability.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Rename domain to registry for consistency

Use consistent "registry" terminology throughout the codebase:
- Rename get_domain_from_image to get_registry_from_image
- Rename IMAGE_DOMAIN_REGEX to IMAGE_REGISTRY_REGEX
- Update named group from "domain" to "registry"
- Update all related comments and variable names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-02 14:30:03 +01:00
Jan Čermák
bac072a985 Use unpublished local wheels during PR builds (#6374)
* Use unpublished local wheels during PR builds

Refactor wheel building to use the new `local-wheels-repo-path` and move wheels
building into a separate CI job. Wheels are only published on published (i.e.
release or merged dev), for PR builds they are passed as artifacts to the build
job instead.

* Address review comments

* Add trailing slash for wheels folder
* Always run the changed_files check to ensure build_wheels runs on publish
* Use full path for workflow and escape dots in changed files regexp
2025-12-02 14:08:07 +01:00
dependabot[bot]
2fc6a7dcab Bump types-docker from 7.1.0.20251129 to 7.1.0.20251202 (#6376) 2025-12-02 07:36:51 +01:00
15 changed files with 296 additions and 67 deletions

View File

@@ -1,6 +1,7 @@
# General files
.git
.github
.gitkeep
.devcontainer
.vscode

View File

@@ -72,19 +72,89 @@ jobs:
- name: Get changed files
id: changed_files
if: steps.version.outputs.publish == 'false'
if: github.event_name != 'release'
uses: masesgroup/retrieve-changed-files@491e80760c0e28d36ca6240a27b1ccb8e1402c13 # v3.0.0
- name: Check if requirements files changed
id: requirements
run: |
if [[ "${{ steps.changed_files.outputs.all }}" =~ (requirements.txt|build.yaml) ]]; then
# No wheels build necessary for releases
if [[ "${{ github.event_name }}" == "release" ]]; then
echo "changed=false" >> "$GITHUB_OUTPUT"
elif [[ "${{ steps.changed_files.outputs.all }}" =~ (requirements\.txt|build\.yaml|\.github/workflows/builder\.yml) ]]; then
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
build_wheels:
name: Build wheels for ${{ matrix.arch }}
needs: init
if: needs.init.outputs.requirements == 'true'
runs-on: ${{ matrix.runs-on }}
strategy:
matrix:
arch: ${{ fromJson(needs.init.outputs.architectures) }}
include:
- runs-on: ubuntu-24.04
- arch: aarch64
runs-on: ubuntu-24.04-arm
env:
ABI: cp313
TAG: musllinux_1_2
APK_DEPS: "libffi-dev;openssl-dev;yaml-dev"
SKIP_BINARY: aiohttp
steps:
- name: Checkout the repository
uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0
- name: Write env-file
run: |
(
# Fix out of memory issues with rust
echo "CARGO_NET_GIT_FETCH_WITH_CLI=true"
) > .env_file
- name: Build and publish wheels
if: needs.init.outputs.publish == 'true'
uses: home-assistant/wheels@e5742a69d69f0e274e2689c998900c7d19652c21 # 2025.12.0
with:
wheels-key: ${{ secrets.WHEELS_KEY }}
abi: ${{ env.ABI }}
tag: ${{ env.TAG }}
arch: ${{ matrix.arch }}
apk: ${{ env.APK_DEPS }}
skip-binary: ${{ env.SKIP_BINARY }}
env-file: true
requirements: "requirements.txt"
- name: Build local wheels
uses: home-assistant/wheels@e5742a69d69f0e274e2689c998900c7d19652c21 # 2025.12.0
if: needs.init.outputs.publish == 'false'
with:
wheels-host: ""
wheels-user: ""
wheels-key: ""
local-wheels-repo-path: "wheels/"
abi: ${{ env.ABI }}
tag: ${{ env.TAG }}
arch: ${{ matrix.arch }}
apk: ${{ env.APK_DEPS }}
skip-binary: ${{ env.SKIP_BINARY }}
env-file: true
requirements: "requirements.txt"
- name: Upload local wheels artifact
if: needs.init.outputs.publish == 'false'
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: wheels-${{ matrix.arch }}
path: wheels
retention-days: 1
build:
name: Build ${{ matrix.arch }} supervisor
needs: init
needs: [init, build_wheels]
if: ${{ !cancelled() && !failure() }}
runs-on: ubuntu-latest
permissions:
contents: read
@@ -99,27 +169,12 @@ jobs:
with:
fetch-depth: 0
- name: Write env-file
if: needs.init.outputs.requirements == 'true'
run: |
(
# Fix out of memory issues with rust
echo "CARGO_NET_GIT_FETCH_WITH_CLI=true"
) > .env_file
# home-assistant/wheels doesn't support sha pinning
- name: Build wheels
if: needs.init.outputs.requirements == 'true'
uses: home-assistant/wheels@2025.11.0
- name: Download local wheels artifact
if: needs.init.outputs.requirements == 'true' && needs.init.outputs.publish == 'false'
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with:
abi: cp313
tag: musllinux_1_2
arch: ${{ matrix.arch }}
wheels-key: ${{ secrets.WHEELS_KEY }}
apk: "libffi-dev;openssl-dev;yaml-dev"
skip-binary: aiohttp
env-file: true
requirements: "requirements.txt"
name: wheels-${{ matrix.arch }}
path: wheels
- name: Set version
if: needs.init.outputs.publish == 'true'
@@ -208,6 +263,13 @@ jobs:
- name: Checkout the repository
uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0
- name: Download local wheels artifact
if: needs.init.outputs.requirements == 'true' && needs.init.outputs.publish == 'false'
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
with:
name: wheels-amd64
path: wheels
# home-assistant/builder doesn't support sha pinning
- name: Build the Supervisor
if: needs.init.outputs.publish != 'true'

5
.gitignore vendored
View File

@@ -24,6 +24,9 @@ var/
.installed.cfg
*.egg
# Local wheels
wheels/**/*.whl
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
@@ -102,4 +105,4 @@ ENV/
/.dmypy.json
# Mac
.DS_Store
.DS_Store

View File

@@ -32,7 +32,17 @@ RUN \
# Install requirements
RUN \
--mount=type=bind,source=./requirements.txt,target=/usr/src/requirements.txt \
uv pip install --compile-bytecode --no-cache --no-build -r requirements.txt
--mount=type=bind,source=./wheels,target=/usr/src/wheels \
if ls /usr/src/wheels/musllinux/* >/dev/null 2>&1; then \
LOCAL_WHEELS=/usr/src/wheels/musllinux; \
echo "Using local wheels from: $LOCAL_WHEELS"; \
else \
LOCAL_WHEELS=; \
echo "No local wheels found"; \
fi && \
uv pip install --compile-bytecode --no-cache --no-build \
-r requirements.txt \
${LOCAL_WHEELS:+--find-links $LOCAL_WHEELS}
# Install Home Assistant Supervisor
COPY . supervisor

View File

@@ -10,7 +10,7 @@ pytest-timeout==2.4.0
pytest==9.0.1
ruff==0.14.7
time-machine==3.1.0
types-docker==7.1.0.20251129
types-docker==7.1.0.20251202
types-pyyaml==6.0.12.20250915
types-requests==2.32.4.20250913
urllib3==2.5.0

View File

@@ -23,7 +23,7 @@ from ..const import (
CpuArch,
)
from ..coresys import CoreSys, CoreSysAttributes
from ..docker.const import DOCKER_HUB
from ..docker.const import DOCKER_HUB, DOCKER_HUB_LEGACY
from ..docker.interface import MAP_ARCH
from ..exceptions import ConfigurationFileError, HassioArchNotFound
from ..utils.common import FileConfiguration, find_one_filetype
@@ -155,8 +155,11 @@ class AddonBuild(FileConfiguration, CoreSysAttributes):
# Use the actual registry URL for the key
# Docker Hub uses "https://index.docker.io/v1/" as the key
# Support both docker.io (official) and hub.docker.com (legacy)
registry_key = (
"https://index.docker.io/v1/" if registry == DOCKER_HUB else registry
"https://index.docker.io/v1/"
if registry in (DOCKER_HUB, DOCKER_HUB_LEGACY)
else registry
)
config = {"auths": {registry_key: {"auth": auth_string}}}

View File

@@ -15,11 +15,12 @@ from ..const import MACHINE_ID
RE_RETRYING_DOWNLOAD_STATUS = re.compile(r"Retrying in \d+ seconds?")
# Docker Hub registry identifier
DOCKER_HUB = "hub.docker.com"
# Docker Hub registry identifier (official default)
# Docker's default registry is docker.io
DOCKER_HUB = "docker.io"
# Regex to match images with a registry host (e.g., ghcr.io/org/image)
IMAGE_WITH_HOST = re.compile(r"^((?:[a-z0-9]+(?:-[a-z0-9]+)*\.)+[a-z]{2,})\/.+")
# Legacy Docker Hub identifier for backward compatibility
DOCKER_HUB_LEGACY = "hub.docker.com"
class Capabilities(StrEnum):

View File

@@ -45,7 +45,13 @@ from ..jobs.decorator import Job
from ..jobs.job_group import JobGroup
from ..resolution.const import ContextType, IssueType, SuggestionType
from ..utils.sentry import async_capture_exception
from .const import DOCKER_HUB, ContainerState, PullImageLayerStage, RestartPolicy
from .const import (
DOCKER_HUB,
DOCKER_HUB_LEGACY,
ContainerState,
PullImageLayerStage,
RestartPolicy,
)
from .manager import CommandReturn, PullLogEntry
from .monitor import DockerContainerStateEvent
from .stats import DockerStats
@@ -184,7 +190,8 @@ class DockerInterface(JobGroup, ABC):
stored = self.sys_docker.config.registries[registry]
credentials[ATTR_USERNAME] = stored[ATTR_USERNAME]
credentials[ATTR_PASSWORD] = stored[ATTR_PASSWORD]
if registry != DOCKER_HUB:
# Don't include registry for Docker Hub (both official and legacy)
if registry not in (DOCKER_HUB, DOCKER_HUB_LEGACY):
credentials[ATTR_REGISTRY] = registry
_LOGGER.debug(

View File

@@ -49,9 +49,10 @@ from ..exceptions import (
)
from ..utils.common import FileConfiguration
from ..validate import SCHEMA_DOCKER_CONFIG
from .const import DOCKER_HUB, IMAGE_WITH_HOST, LABEL_MANAGED
from .const import DOCKER_HUB, DOCKER_HUB_LEGACY, LABEL_MANAGED
from .monitor import DockerMonitor
from .network import DockerNetwork
from .utils import get_registry_from_image
_LOGGER: logging.Logger = logging.getLogger(__name__)
@@ -212,19 +213,25 @@ class DockerConfig(FileConfiguration):
Matches the image against configured registries and returns the registry
name if found, or None if no matching credentials are configured.
Uses Docker's domain detection logic from:
vendor/github.com/distribution/reference/normalize.go
"""
if not self.registries:
return None
# Check if image uses a custom registry (e.g., ghcr.io/org/image)
matcher = IMAGE_WITH_HOST.match(image)
if matcher:
registry = matcher.group(1)
registry = get_registry_from_image(image)
if registry:
if registry in self.registries:
return registry
# If no registry prefix, check for Docker Hub credentials
elif DOCKER_HUB in self.registries:
return DOCKER_HUB
else:
# No registry prefix means Docker Hub
# Support both docker.io (official) and hub.docker.com (legacy)
if DOCKER_HUB in self.registries:
return DOCKER_HUB
if DOCKER_HUB_LEGACY in self.registries:
return DOCKER_HUB_LEGACY
return None
@@ -761,7 +768,7 @@ class DockerAPI(CoreSysAttributes):
"""Import a tar file as image."""
try:
with tar_file.open("rb") as read_tar:
resp: list[dict[str, Any]] = self.images.import_image(read_tar)
resp: list[dict[str, Any]] = await self.images.import_image(read_tar)
except (aiodocker.DockerError, OSError) as err:
raise DockerError(
f"Can't import image from tar: {err}", _LOGGER.error

View File

@@ -0,0 +1,57 @@
"""Docker utilities."""
from __future__ import annotations
import re
# Docker image reference domain regex
# Based on Docker's reference implementation:
# vendor/github.com/distribution/reference/normalize.go
#
# A domain is detected if the part before the first / contains:
# - "localhost" (with optional port)
# - Contains "." (like registry.example.com or 127.0.0.1)
# - Contains ":" (like myregistry:5000)
# - IPv6 addresses in brackets (like [::1]:5000)
#
# Note: Docker also treats uppercase letters as registry indicators since
# namespaces must be lowercase, but this regex handles lowercase matching
# and the get_registry_from_image() function validates the registry rules.
IMAGE_REGISTRY_REGEX = re.compile(
r"^(?P<registry>"
r"localhost(?::[0-9]+)?|" # localhost with optional port
r"(?:[a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9])" # domain component
r"(?:\.(?:[a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9]))*" # more components
r"(?::[0-9]+)?|" # optional port
r"\[[a-fA-F0-9:]+\](?::[0-9]+)?" # IPv6 with optional port
r")/" # must be followed by /
)
def get_registry_from_image(image_ref: str) -> str | None:
"""Extract registry from Docker image reference.
Returns the registry if the image reference contains one,
or None if the image uses Docker Hub (docker.io).
Based on Docker's reference implementation:
vendor/github.com/distribution/reference/normalize.go
Examples:
get_registry_from_image("nginx") -> None (docker.io)
get_registry_from_image("library/nginx") -> None (docker.io)
get_registry_from_image("myregistry.com/nginx") -> "myregistry.com"
get_registry_from_image("localhost/myimage") -> "localhost"
get_registry_from_image("localhost:5000/myimage") -> "localhost:5000"
get_registry_from_image("registry.io:5000/org/app:v1") -> "registry.io:5000"
get_registry_from_image("[::1]:5000/myimage") -> "[::1]:5000"
"""
match = IMAGE_REGISTRY_REGEX.match(image_ref)
if match:
registry = match.group("registry")
# Must contain '.' or ':' or be 'localhost' to be a real registry
# This prevents treating "myuser/myimage" as having registry "myuser"
if "." in registry or ":" in registry or registry == "localhost":
return registry
return None # No registry = Docker Hub (docker.io)

View File

@@ -183,19 +183,22 @@ class GitRepo(CoreSysAttributes):
raise StoreGitError() from err
try:
branch = self.repo.active_branch.name
repo = self.repo
# Download data
await self.sys_run_in_executor(
ft.partial(
self.repo.remotes.origin.fetch,
**{"update-shallow": True, "depth": 1}, # type: ignore
def _fetch_and_check() -> tuple[str, bool]:
"""Fetch from origin and check if changed."""
# This property access is I/O bound
branch = repo.active_branch.name
repo.remotes.origin.fetch(
**{"update-shallow": True, "depth": 1} # type: ignore[arg-type]
)
)
changed = repo.commit(branch) != repo.commit(f"origin/{branch}")
return branch, changed
if changed := self.repo.commit(branch) != self.repo.commit(
f"origin/{branch}"
):
# Download data and check for changes
branch, changed = await self.sys_run_in_executor(_fetch_and_check)
if changed:
# Jump on top of that
await self.sys_run_in_executor(
ft.partial(self.repo.git.reset, f"origin/{branch}", hard=True)

View File

@@ -144,9 +144,9 @@ async def docker() -> DockerAPI:
docker_images.inspect.return_value = image_inspect
docker_images.list.return_value = [image_inspect]
docker_images.import_image.return_value = [
{"stream": "Loaded image: test:latest\n"}
]
docker_images.import_image = AsyncMock(
return_value=[{"stream": "Loaded image: test:latest\n"}]
)
docker_images.pull.return_value = AsyncIterator([{}])

View File

@@ -1,9 +1,49 @@
"""Test docker login."""
import pytest
# pylint: disable=protected-access
from supervisor.coresys import CoreSys
from supervisor.docker.const import DOCKER_HUB
from supervisor.docker.const import DOCKER_HUB, DOCKER_HUB_LEGACY
from supervisor.docker.interface import DockerInterface
from supervisor.docker.utils import get_registry_from_image
@pytest.mark.parametrize(
("image_ref", "expected_registry"),
[
# No registry - Docker Hub images
("nginx", None),
("nginx:latest", None),
("library/nginx", None),
("library/nginx:latest", None),
("homeassistant/amd64-supervisor", None),
("homeassistant/amd64-supervisor:1.2.3", None),
# Registry with dot
("ghcr.io/homeassistant/amd64-supervisor", "ghcr.io"),
("ghcr.io/homeassistant/amd64-supervisor:latest", "ghcr.io"),
("myregistry.com/nginx", "myregistry.com"),
("registry.example.com/org/image:v1", "registry.example.com"),
("127.0.0.1/myimage", "127.0.0.1"),
# Registry with port
("myregistry:5000/myimage", "myregistry:5000"),
("localhost:5000/myimage", "localhost:5000"),
("registry.io:5000/org/app:v1", "registry.io:5000"),
# localhost special case
("localhost/myimage", "localhost"),
("localhost/myimage:tag", "localhost"),
# IPv6
("[::1]:5000/myimage", "[::1]:5000"),
("[2001:db8::1]:5000/myimage:tag", "[2001:db8::1]:5000"),
],
)
def test_get_registry_from_image(image_ref: str, expected_registry: str | None):
"""Test get_registry_from_image extracts registry from image reference.
Based on Docker's reference implementation:
vendor/github.com/distribution/reference/normalize.go
"""
assert get_registry_from_image(image_ref) == expected_registry
def test_no_credentials(coresys: CoreSys, test_docker_interface: DockerInterface):
@@ -47,3 +87,36 @@ def test_matching_credentials(coresys: CoreSys, test_docker_interface: DockerInt
)
assert credentials["username"] == "Spongebob Squarepants"
assert "registry" not in credentials
def test_legacy_docker_hub_credentials(
coresys: CoreSys, test_docker_interface: DockerInterface
):
"""Test legacy hub.docker.com credentials are used for Docker Hub images."""
coresys.docker.config._data["registries"] = {
DOCKER_HUB_LEGACY: {"username": "LegacyUser", "password": "Password1!"},
}
credentials = test_docker_interface._get_credentials(
"homeassistant/amd64-supervisor"
)
assert credentials["username"] == "LegacyUser"
# No registry should be included for Docker Hub
assert "registry" not in credentials
def test_docker_hub_preferred_over_legacy(
coresys: CoreSys, test_docker_interface: DockerInterface
):
"""Test docker.io is preferred over legacy hub.docker.com when both exist."""
coresys.docker.config._data["registries"] = {
DOCKER_HUB: {"username": "NewUser", "password": "Password1!"},
DOCKER_HUB_LEGACY: {"username": "LegacyUser", "password": "Password2!"},
}
credentials = test_docker_interface._get_credentials(
"homeassistant/amd64-supervisor"
)
# docker.io should be preferred
assert credentials["username"] == "NewUser"
assert "registry" not in credentials

View File

@@ -2,7 +2,7 @@
import asyncio
from pathlib import Path
from unittest.mock import MagicMock, patch
from unittest.mock import AsyncMock, MagicMock, patch
from docker.errors import APIError, DockerException, NotFound
import pytest
@@ -412,9 +412,9 @@ async def test_repair_failures(coresys: CoreSys, caplog: pytest.LogCaptureFixtur
async def test_import_image(coresys: CoreSys, tmp_path: Path, log_starter: str):
"""Test importing an image into docker."""
(test_tar := tmp_path / "test.tar").touch()
coresys.docker.images.import_image.return_value = [
{"stream": f"{log_starter}: imported"}
]
coresys.docker.images.import_image = AsyncMock(
return_value=[{"stream": f"{log_starter}: imported"}]
)
coresys.docker.images.inspect.return_value = {"Id": "imported"}
image = await coresys.docker.import_image(test_tar)
@@ -426,9 +426,9 @@ async def test_import_image(coresys: CoreSys, tmp_path: Path, log_starter: str):
async def test_import_image_error(coresys: CoreSys, tmp_path: Path):
"""Test failure importing an image into docker."""
(test_tar := tmp_path / "test.tar").touch()
coresys.docker.images.import_image.return_value = [
{"errorDetail": {"message": "fail"}}
]
coresys.docker.images.import_image = AsyncMock(
return_value=[{"errorDetail": {"message": "fail"}}]
)
with pytest.raises(DockerError, match="Can't import image from tar: fail"):
await coresys.docker.import_image(test_tar)
@@ -441,10 +441,12 @@ async def test_import_multiple_images_in_tar(
):
"""Test importing an image into docker."""
(test_tar := tmp_path / "test.tar").touch()
coresys.docker.images.import_image.return_value = [
{"stream": "Loaded image: imported-1"},
{"stream": "Loaded image: imported-2"},
]
coresys.docker.images.import_image = AsyncMock(
return_value=[
{"stream": "Loaded image: imported-1"},
{"stream": "Loaded image: imported-2"},
]
)
assert await coresys.docker.import_image(test_tar) is None

0
wheels/.gitkeep Normal file
View File