mirror of
https://github.com/home-assistant/developers.home-assistant.git
synced 2025-07-15 21:36:31 +00:00
Extend OS development docs (#1075)
* Add Operating System developer documentation This moves some of the existing documentation from the operating-system repository, updates it to reflect the current state and extens certain parts. * Reword and add deployment documentation * Add new pages to Sidebar * Reword and extend various aspects * Small fixes here and there * Clarify builds
This commit is contained in:
Stefan Agner
GitHub
parent
be98cd9303
commit
020a41fd18
@ -3,4 +3,23 @@ title: Home Assistant Operating System
|
||||
sidebar_label: Introduction
|
||||
---
|
||||
|
||||
This is the developer documentation for the Home Assistant Operating System.
|
||||
The Home Assistant Operating System is a purpose built operating system specifically designed to run Home Assistant on single board computers and x86-64 systems. It aims to provide a robust and maintenance free operating system to run Home Assistant.
|
||||
|
||||
Home Assistant Operating System (HAOS) is using the [Buildroot](https://buildroot.org/) build system. Buildroot is not a Linux distribution in the classic sense. It provides the infrastructure and build system to build a Linux distribution. Buildroot allows us to cross compile for different architectures which makes it especially useful when compiling for architectures which typically come with fewer resources such as Arm based systems. HAOS consists of a fairly regular stack of Linux and GNU software, using Linux, the GNU C library, systemd init daemon and the Docker container engine required by the Home Assistant Supervisor.
|
||||
|
||||
### Components
|
||||
|
||||
- **Bootloader:**
|
||||
- [Barebox](https://barebox.org/) for devices that support EFI
|
||||
- [U-Boot](https://www.denx.de/wiki/U-Boot) for devices that don't support EFI
|
||||
- **Operating System:**
|
||||
- [Buildroot](https://buildroot.org/) build system to generate Linux distributions
|
||||
- **File Systems:**
|
||||
- [SquashFS](https://www.kernel.org/doc/Documentation/filesystems/squashfs.txt) for read-only file systems (using LZ4 compression)
|
||||
- [ZRAM](https://www.kernel.org/doc/Documentation/blockdev/zram.txt) for `/tmp`, `/var` and swap (using LZ4 compression)
|
||||
- **Container Platform:**
|
||||
- [Docker Engine](https://docs.docker.com/engine/) for running Home Assistant components in containers
|
||||
- **Updates:**
|
||||
- [RAUC](https://rauc.io/) for Over The Air (OTA) and USB updates
|
||||
- **Security:**
|
||||
- [AppArmor](https://apparmor.net/) Linux kernel security module
|
||||
|
||||
85
docs/operating-system/board-metadata.md
Normal file
85
docs/operating-system/board-metadata.md
Normal file
@ -0,0 +1,85 @@
|
||||
---
|
||||
title: "Board Metadata"
|
||||
sidebar_label: Metadata
|
||||
---
|
||||
|
||||
Each supported board has Home Assistant Operating System (HAOS) specific metadata file named `meta`. This documents the variables and the available options.
|
||||
|
||||
## Board specific variables
|
||||
|
||||
`BOARD_ID`:
|
||||
|
||||
Board identifier. Typically all lower case. Required to generate the file name and used as `VARIANT_ID` in the os-release file.
|
||||
|
||||
`BOARD_NAME`:
|
||||
|
||||
User friendly board name. Used in the `VERSION` and `VARIANT` variable in the os-release file.
|
||||
|
||||
## Boot related variables
|
||||
|
||||
`BOOT_ENV_SIZE`:
|
||||
|
||||
Maximum size of the boot loader environment (in hex). Required for rauc.
|
||||
|
||||
`BOOT_SYS`:
|
||||
|
||||
- efi
|
||||
- hybrid
|
||||
- mbr
|
||||
|
||||
HAOS is trying to use GPT whenever possible. To use GPT the second logical block (LBA) needs to be available. On some boards this block is reserved/required by the boot firmware. If that is the case the class MBR approach needs to be used.
|
||||
|
||||
Hybrid uses both partition tables in case GPT can be used, but lower level firmware still requires MBR.
|
||||
|
||||
`BOOT_SPL`:
|
||||
|
||||
- true
|
||||
- false
|
||||
|
||||
Enable SPL (secondary program loader) handling. Some U-Boot targets generate a small loader (SPL) besides the main U-Boot binary.
|
||||
|
||||
`BOOTLOADER`:
|
||||
|
||||
- uboot
|
||||
- barebox
|
||||
|
||||
HAOS uses mainly [U-Boot](https://www.denx.de/wiki/U-Boot). For UEFI systems [barebox](https://barebox.org/) is used.
|
||||
|
||||
`DISK_SIZE`:
|
||||
|
||||
Default 2. Size of the final (uncompressed) image in GB.
|
||||
|
||||
`KERNEL_FILE`:
|
||||
|
||||
File name of the kernel binary. Typically `Image` for aarch64, `zImage` for `armv7` and `bzImage` for `amd64`.
|
||||
|
||||
## Supervisor related variables
|
||||
|
||||
`SUPERVISOR_MACHINE`:
|
||||
|
||||
- generic-x86-64
|
||||
- khadas-vim3
|
||||
- odroid-c2
|
||||
- odroid-c4
|
||||
- odroid-n2
|
||||
- odroid-xu
|
||||
- qemuarm
|
||||
- qemuarm-64
|
||||
- qemux86
|
||||
- qemux86-64
|
||||
- raspberrypi
|
||||
- raspberrypi2
|
||||
- raspberrypi3
|
||||
- raspberrypi4
|
||||
- raspberrypi3-64
|
||||
- raspberrypi4-64
|
||||
- tinker
|
||||
|
||||
`SUPERVISOR_ARCH`:
|
||||
|
||||
- amd64
|
||||
- i386
|
||||
- armhf
|
||||
- armv7
|
||||
- aarch64
|
||||
|
||||
36
docs/operating-system/deployment.md
Normal file
36
docs/operating-system/deployment.md
Normal file
@ -0,0 +1,36 @@
|
||||
---
|
||||
title: "Deployment/Releases"
|
||||
sidebar_label: Deployment
|
||||
---
|
||||
|
||||
Home Assistant Operating System releases are built from the release branch. GitHub Actions are used to build all public releases. There is no fixed schedule, builds are triggered as needed by the HAOS maintainer. Changes need to get applied to the development branch first and labeled with the `rel-x` label. The maintainer will backport those patches onto the release branch before the next release.
|
||||
|
||||
## Branches
|
||||
|
||||
- `dev`: Development branch. Carries the next major version during development. During release candidate phase, release candidates are tagged on this branch.
|
||||
- `rel-X`: Release branch. One per major release. Typically new releases only get built from the last major release version. Each release gets tagged with its version number.
|
||||
|
||||
## Versioning
|
||||
|
||||
The format of version is *MAJOR.BUILD*. Every time a new release is released the BUILD number gets incremented (stored in `buildroot-external/meta`). The MAJOR number is inherited from the development branch, and gets bumped right after the release branch has been created.
|
||||
|
||||
The build system by default automatically adds a *dev{DATE}* suffix to mark development builds.
|
||||
|
||||
Before a new major release, release candidates can be built on the development branch. A release candidate suffix is used to mark them, e.g. *MAJOR.0.rc1*.
|
||||
|
||||
## Deployment types
|
||||
|
||||
HAOS provides 3 different types of deployments. The deployment differ in which public keys they include for over the air updates. The deployment type is shown in the Supervisor web frontend in the System tab on the Host card.
|
||||
|
||||
- development (dev)
|
||||
- staging (beta)
|
||||
- production (stable)
|
||||
|
||||
## Build pipelines
|
||||
|
||||
GitHub Actions are used to build HAOS development and release builds. Two workflow exist:
|
||||
|
||||
- `.github/workflows/dev.yml`: Development builds, triggered manually, images stored on [os-builds.home-assistant.io](https://os-builds.home-assistant.io/).
|
||||
- `.github/workflows/release.yml`: Release (and release candidate) builds, triggered when a GitHub release gets published, images stored as GitHub release asset.
|
||||
|
||||
The development build pipeline can also be triggered from a PR: The appropriate board labels need to be set first, a build for those boards is then triggered when adding the `run-dev-build` label.
|
||||
134
docs/operating-system/getting-started.md
Normal file
134
docs/operating-system/getting-started.md
Normal file
@ -0,0 +1,134 @@
|
||||
---
|
||||
title: "Getting Started with Home Assistant Operating System Development"
|
||||
sidebar_label: Getting Started
|
||||
---
|
||||
|
||||
## Prepare Development Environment
|
||||
|
||||
### Check-out Source Code
|
||||
|
||||
The main repository located at [github.com/home-assistant/operating-system/](https://github.com/home-assistant/operating-system/) contains Buildroot customizations via the [br2-external mechanism](https://buildroot.org/downloads/manual/manual.html#outside-br-custom) as well as helper scripts and GitHub Action CI scripts. The main repository uses the Git Submodule mechanism to point to Buildroot itself. While most customizations can be done by the br2-mechanism, some modifications are made to Buildroot itself. For that reason we also maintain a fork of Buildroot under [github.com/home-assistant/buildroot/](https://github.com/home-assistant/buildroot/). The aim is to keep the amount of patches on-top of upstream Buildroot minimal.
|
||||
|
||||
Make sure you have `git` available and clone the main HAOS repository as follows:
|
||||
|
||||
```shell
|
||||
git clone https://github.com/home-assistant/operating-system/
|
||||
cd operating-system/
|
||||
git submodule update --init
|
||||
```
|
||||
|
||||
When you update your local git repository, make sure to also update the `buildroot` submodule. This makes sure you'll get the matching Buildroot in case it got updated as well.
|
||||
|
||||
```shell
|
||||
git pull
|
||||
git submodule update
|
||||
```
|
||||
|
||||
To get back to a pristine state, use the following two commands (this deletes all local modifications!)
|
||||
|
||||
```shell
|
||||
git reset --hard
|
||||
git submodule update --init --force
|
||||
```
|
||||
|
||||
### Install prerequisites
|
||||
|
||||
HAOS is using build containers to run Buildroot. Install the Docker container engine and make sure you have a working `docker` command which allows to run privileged containers. The build scripts are meant to be run as user, but some commands use privileges, hence a working `sudo` command is required as well.
|
||||
|
||||
While Buildroot can run on most Linux distributions natively, its strongly recommended to use the Debian based build container. This allows for a stable and known build environment with all dependencies pre-installed.
|
||||
|
||||
:::info
|
||||
The build container needs to get started with privileges since at some point during the build process a new loopback device-backed filesystem image will be mounted inside a Docker container. Hence rootless containers won't work to build HAOS.
|
||||
:::
|
||||
|
||||
## Build Images using Build Container
|
||||
|
||||
The script `scripts/enter.sh` builds the build container image and starts a container using that image. Arguments passed to the script get executed inside the container.
|
||||
|
||||
HAOS uses a configuration file for each supported target. To build for a specific target (board), the configuration file needs to be passed to `make`. The configuration files are stored in `buildroot-external/configs/`. Note that the ending `_defconfig` will be appended automatically and *must not* be passed to `make`. E.g. to build the Raspberry Pi 4 64-bit configuration `buildroot-external/configs/rpi4_64_defconfig` use the following command:
|
||||
|
||||
```
|
||||
$ sudo scripts/enter.sh make rpi4_64
|
||||
Sending build context to Docker daemon 159.7kB
|
||||
Step 1/8 : FROM debian:bullseye
|
||||
---> a178460bae57
|
||||
[...]
|
||||
Successfully built 11d679ac51be
|
||||
Successfully tagged hassos:local
|
||||
[...]
|
||||
/usr/bin/make -C /build/buildroot O=/build/output BR2_EXTERNAL=/build/buildroot-external "rpi4_64_defconfig"
|
||||
[...]
|
||||
```
|
||||
|
||||
This invokes make using the `Makefile` in the root of the source repository inside the container. This makefile in turn invokes Buildroot's makefile.
|
||||
|
||||
Depending on the speed of your machine the build process takes 0.5 to 1h. The build files (object files, intermediate binaries etc.) are stored in the folder `output/` (used to be in `buildroot/output/` in rel-6 and older branches). The final image files are stored in the `release/` directory.
|
||||
|
||||
### Rebuild packages
|
||||
|
||||
Buildroot uses packages like a regular distribution. But instead of just downloading a pre-built package, Buildroot packages download the source files and compile the binaries directly. Buildroot remembers which package has been built already. This makes the second build much faster, since only the final image gets regenerated. If you want to force Buildroot to rebuild a particular package, just delete it from the `output/build/` directory:
|
||||
|
||||
```shell
|
||||
rm -rf output/build/linux-custom/
|
||||
```
|
||||
|
||||
:::tip
|
||||
You can check `output/build/packages-file-list.txt` to learn which file in the final image belongs to what package. This makes it easier to find the package you would like to change.
|
||||
:::
|
||||
|
||||
### Build for Multiple Targets
|
||||
|
||||
To build for multiple targets in a single source directory, separate output directories must be used. The output directory can be specified with the `O=` argument. A recommended pattern is to just use an output directory named after the targets configuration file:
|
||||
|
||||
|
||||
```shell
|
||||
sudo scripts/enter.sh make O=output_rpi4_64 rpi4_64
|
||||
```
|
||||
|
||||
### Use the Build Container Interactively
|
||||
|
||||
If no argument to `scripts/enter.sh` is passed, a shell will be presented.
|
||||
|
||||
```bash
|
||||
$ sudo scripts/enter.sh
|
||||
Sending build context to Docker daemon 159.7kB
|
||||
Step 1/8 : FROM debian:bullseye
|
||||
---> a178460bae57
|
||||
[...]
|
||||
builder@c6dfb4cd4036:/build$
|
||||
```
|
||||
|
||||
From this shell, the same build above could be started using `make O=output_rpi4_64 rpi4_64`.
|
||||
|
||||
This allows to invoke other Buildroot targets, e.g. to [graph dependencies between packages](https://buildroot.org/downloads/manual/manual.html#_graphing_the_dependencies_between_packages). To use other Buildroot targets, make sure to change to the `buildroot/` directory and execute commands from there
|
||||
|
||||
```bash
|
||||
builder@c6dfb4cd4036:/build$ cd buildroot/
|
||||
builder@c6dfb4cd4036:/build$ make O=../output_rpi4_64 graph-depends
|
||||
Getting dependency tree...
|
||||
dot -Tpdf \
|
||||
-o /build/output_rpi4/graphs/graph-depends.pdf \
|
||||
/build/output_rpi4/graphs/graph-depends.dot
|
||||
builder@c6dfb4cd4036:/build$
|
||||
```
|
||||
|
||||
## Use Qemu to Test Images
|
||||
|
||||
The target OVA (Open Virtual Appliance) contains images for various virtual machines. One of the image format is QCOW2, the native image format for QEMU. It can be used to test a new HAOS build using QEMU.
|
||||
|
||||
Since HAOS requires UEFI support, this is slightly more tricky than with "classic"(/legacy) MBR-based images. On a *Debian* host install the [ovmf package](https://packages.debian.org/stable/ovmf) which provides the "UEFI firmware for 64-bit x86 virtual machines". That package will install a **TianoCore**-derived QEMU UEFI image at `/usr/share/OVMF/OVMF_CODE.fd`, which can be used with QEMU to boot the generated QCOW2 image.
|
||||
|
||||
```bash
|
||||
$ sudo scripts/enter.sh make O=output_ova ova
|
||||
[...]
|
||||
$ unxz release/haos_ova-7.0.dev20211003.qcow2.xz
|
||||
$ qemu-system-x86_64 -enable-kvm -name haos -smp 2 -m 1G -drive file=release/haos_ova-7.0.dev20211003.qcow2,index=0,media=disk,if=virtio,format=qcow2 -drive file=/usr/share/ovmf/x64/OVMF_CODE.fd,if=pflash,format=raw,readonly=on
|
||||
```
|
||||
|
||||
This will show QEMU's SDL interface and should boot Home Assistant Operating System. Once the boot completes and the Home Assistant CLI prompt `ha > ` is shown, you can use `login` to access the root shell.
|
||||
|
||||
## Create a pull request for review
|
||||
|
||||
Once you are happy with your changes create a separate git branch and commit them. Try to describe *why* you think that change is important and should be applied to HAOS. E.g "update kernel" is also obvious from the changes itself. The maintainer is more interested why you think the kernel should be updated. The *why* can be fairly trivial (update kernel to make sure we keep up with latest changes), or it can have some interesting details (update kernel since this latest version fixes ethernet on board xy).
|
||||
|
||||
Create a fork of the upstream [github.com/home-assistant/operating-system](https://github.com/home-assistant/operating-system) repository (if you haven't already) and push your branch to your forked GitHub repository. Then open a new pull request. All changes should be made against the development branch `dev`. If you like your change in the next stable release, add the `rel-x` label so it is marked for backporting.
|
||||
55
docs/operating-system/partition.md
Normal file
55
docs/operating-system/partition.md
Normal file
@ -0,0 +1,55 @@
|
||||
---
|
||||
title: "Partitioning"
|
||||
sidebar_label: Partitions
|
||||
---
|
||||
|
||||
The Home Assistant Operating System (HAOS) partition layout is a bit different than what is typically used on a Linux system.
|
||||
|
||||
## Partition Table
|
||||
|
||||
HAOS prefers GPT (GUID Partition Table) whenever possible. Boot ROMs of some SoCs don't support GPT, in that case a hybrid GPT/MBR is used if possible and legacy MBR otherwise (see also [Metadata](board-metadata.md) documentation).
|
||||
|
||||
## Partitions
|
||||
|
||||
```text
|
||||
-------------------------
|
||||
| Boot |
|
||||
-------------------------
|
||||
| Kernel A |
|
||||
-------------------------
|
||||
| System A |
|
||||
| |
|
||||
-------------------------
|
||||
| Kernel B |
|
||||
-------------------------
|
||||
| System B |
|
||||
| |
|
||||
-------------------------
|
||||
| Bootstate |
|
||||
-------------------------
|
||||
| Overlay |
|
||||
| |
|
||||
...
|
||||
|
||||
-------------------------
|
||||
| Data |
|
||||
| |
|
||||
| |
|
||||
-------------------------
|
||||
```
|
||||
|
||||
### System partitions
|
||||
|
||||
The boot partition is typically a FAT partition and contains system specific content to enable booting. On x86-64 systems this is the EFI system partition containing Barebox.
|
||||
|
||||
Next two versions of the Linux kernel and the main operating systems are stored (Kernel A/B and System A/B, a total of 4 partitions). This allows the system to fall back to the previous release in case booting on the new release fails (A/B update method). The system partitions are only written to during an update and are read-only under regular operation.
|
||||
|
||||
The overlay partition is used to store certain operating system level settings (e.g. networking settings). The file system label `hassos-overlay` is used to find and mount this partition.
|
||||
|
||||
### Data partition
|
||||
|
||||
The data partition is the main partition containing all containers (Supervisor/Core/Plug-Ins and Add-Ons) as well as user data. It has by far the most I/O operations and hence benefits most if mounted on a fast storage (e.g. via the data disk feature). It is mounted to `/mnt/data`, and some subdirectories are bind mounted to other places (like `/var/lib/docker`). The file system label `hassos-data` is used to find and mount this partition.
|
||||
|
||||
On a fresh installation, the data partition contains the latest version (at build time) of the Supervisor and its Plug-Ins. There is no Home Assistant Core pre-installed, instead a smaller landing page. The Supervisor downloads the latest version of Home Assistant Core on first boot. This makes sure that users start with the latest version of Home Assistant Core after starting the HAOS the first time.
|
||||
|
||||
The data disk feature makes use of the fact that HAOS uses the `hassos-data` label: The feature prepares the disk by partitioning it and creating a file system with the label `hassos-data-external`. On reboot the file system utility `dumpe2fs` is used to move all data from the existing `hassos-data` partition to the new partition. Finally the file system label of the existing data partition is changed to `hassos-data-old` (to avoid getting mounted again) and the new data partition on the data disk to `hassos-data`.
|
||||
@ -82,7 +82,14 @@ module.exports = {
|
||||
"internationalization/core",
|
||||
"internationalization/custom_integration",
|
||||
],
|
||||
OperatingSystem: ["operating-system", "operating-system/debugging"],
|
||||
OperatingSystem: [
|
||||
"operating-system",
|
||||
"operating-system/getting-started",
|
||||
"operating-system/debugging",
|
||||
"operating-system/partition",
|
||||
"operating-system/board-metadata",
|
||||
"operating-system/deployment"
|
||||
],
|
||||
Supervisor: ["supervisor", "supervisor/development", "supervisor/debugging"],
|
||||
// Old structure, still to move/migrate
|
||||
Architecture: {
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user