umbrel/packages/os

umbrelOS

GPU acceleration

The amd64 image keeps GPU userspace in application containers while providing the host kernel interfaces those runtimes require:

Hardware/backend	Host support	Container support
AMD Vulkan	In-tree amdgpu, /dev/dri	Mesa/RADV
AMD ROCm	In-tree amdgpu/KFD, /dev/kfd and /dev/dri	ROCm/HIP
NVIDIA Vulkan	NVIDIA open kernel module, Vulkan ICD, NVIDIA Container Toolkit	Vulkan application/runtime
NVIDIA CUDA	NVIDIA open kernel module, libcuda, NVIDIA Container Toolkit	CUDA runtime
NVIDIA video	NVIDIA open kernel module, NVDEC/NVENC libraries, NVIDIA Container Toolkit	Video application/runtime

NVIDIA's Vulkan, CUDA, and video implementations share the NVIDIA kernel driver and can be used simultaneously. They coexist with AMDGPU and RADV on mixed-GPU systems; the Vulkan loader selects the ICD matching each device. Containers using NVIDIA Vulkan must request the graphics NVIDIA driver capability, expose /dev/dri, and include the Vulkan and GLVND EGL loaders (libvulkan1 and libegl1 on Debian-based images). Containers using NVDEC or NVENC must request the video NVIDIA driver capability and include a compatible video application such as FFmpeg.

An image that only needs Vulkan acceleration on NVIDIA hardware can set the NVIDIA_CUDA_SUPPORT=false build argument to omit the NVIDIA host packages and use the in-tree nouveau driver with containerized Mesa/NVK instead. NVK cannot drive a GPU while that GPU is bound to the NVIDIA kernel driver.

Build Process

The following diagram visualizes the build process and possible build artifacts:

graph TD
    subgraph "Docker"
        root-amd64
        root-arm64
    end
    subgraph "Rugix (Pi)"
        pi --> pi4
        pi --> pi-tryboot
        pi --> pi-mbr
    end
    subgraph "Rugix (AMD64)"
        amd64
        mender-amd64
    end
    root-amd64 --> amd64
    root-amd64 --> mender-amd64
    root-arm64 --> pi
    amd64 --> amd64-img([amd64.img])
    amd64 --> amd64-rugixb([amd64.rugixb])
    mender-amd64 --> mender-amd64-mender([mender-amd64.mender])
    mender-amd64 --> mender-amd64-rugixb([mender-amd64.rugixb])
    pi-mbr --> pi-mbr-mender([pi.mender])
    pi4 --> pi4-img([pi4.img])
    pi-tryboot --> pi-tryboot-img([pi.img])
    pi-tryboot --> pi-tryboot-rugixb([pi.rugixb])

OS Variants

There are three main umbrelOS variants:

• umbrelos-pi: Rugix-native umbrelOS for Raspberry Pi.
• umbrelos-amd64: Rugix-native umbrelOS for AMD64.
• umbrelos-mender-amd64: Rugix-based but Mender-compatible umbrelOS for AMD64.

Important

Legacy devices provisioned with Mender require the umbrelos-mender-amd64 variant. This variant includes a Rugix configuration to enable a safe migration from Mender to Rugix. Rugix will interface with GRUB in the same way Mender does to facilitate A/B switching. In addition, a Rugix hook is installed to migrate state into Rugix's state management mechanism. This enables factory resets and all other Rugix state management features.

Build Artifacts

The build process can produce the following artifacts.

Images

Images for provisioning new umbrelOS devices.

• umbrelos-amd64.img: Image for provisioning AMD64 devices and VMs.
• umbrelos-pi.img: Image for provisioning Raspberry Pi devices (GPT-based).
• umbrelos-pi4.img: Optional image for provisioning Raspberry Pi 4 devices (GPT-based).

The image for Raspberry Pi 4 includes a firmware update to enable the tryboot mechanism used for A/B switching. Otherwise, the image is identical to the image for Raspberry Pi 5.

Mender Update Artifacts

Optional Mender update artifacts for updating existing systems through Mender:

• umbrelos-pi.mender: Mender update artifact for Raspberry Pi.
• umbrelos-mender-amd64.mender: Mender update artifact for AMD64 devices.

The update artifact for Raspberry Pi works for Raspberry Pi 4 and 5.

Rugix Update Bundles

For each of the umbrelOS variants, there is a respective Rugix update bundle:

• umbrelos-pi.rugixb: Rugix update bundle for Raspberry Pi.
• umbrelos-amd64.rugixb: Rugix update bundle for Rugix-native AMD64 devices.
• umbrelos-mender-amd64.rugixb: Rugix update bundle for legacy Mender devices.

Mender to Rugix Migration

Devices can be migrated to Rugix by installing the respective Mender update artifact.

Raspberry Pi

As umbrelOS for Raspberry Pi has always been based on Rugix, no special migration is needed.

AMD64

In case of Mender-based AMD64 devices, the directory structure on the data partition must be migrated such that Rugix's state management can be used. This migration is performed by a boot/post-init Rugix hook: 10-migrate-state.sh. This hook works as follows: Initially, a symlink /data/umbrel-os is placed in the /data directory managed by Rugix linking back to the bare data partition. This way, after booting, all the bind mounts to /data/umbrel-os are set up correctly. The migration hook replaces this symlink atomically with the directory from the data partition, but only after the system has been committed. This is done to leave the state intact, in case there is a rollback. Note that the system needs to be rebooted once after the commit to trigger the migration. To this end, after committing, umbreld may check whether /data/umbrel-os is a symlink and reboot the system if it is.

Factory Resets

Factory resets can be triggered through Rugix Ctrl's state management mechanism as usual. To trigger a factory reset, run rugix-ctrl state reset. This will reboot the system and remove the state on the data partition in the process. In case of a RAID configuration, a state-reset/prepare hook is used to remove the RAID configuration and reset the main data partition. Note that Rugix only supports resetting the state on the actively used data partition (which is the RAID, if a RAID has been configured), hence, we need to use the hook here to make sure that the main data partition is wiped as well.

Removing state can take some time during the boot process. It is therefore recommended to use Rugix's state reset mechanism with the --backup flag. This will simply rename the old state directory according to the following pattern:

/run/rugix/mounts/data/state/default -> /run/rugix/mounts/data/state/default.XXXXXXXXXXXXXX

Here, XXXXXXXXXXXXXX is the date and time of the reset. The residual state can then removed after booting by umbreld.