Proxmox LXC Local AI Serving Stack

Production-ready configs + scripts for serving Qwen3.8-27B (LLM), harrier-oss-v1-0.6b (embeddings), and jina-reranker-v3.5 (reranking) on a Minisforum MS-S1 Max-class box: Strix Halo iGPU (gfx1151) + RX 7900 XTX (gfx1100, 24 GiB VRAM) over OCuLink Gen4 x4, under an unprivileged Proxmox LXC running nested Podman on Ubuntu 24.04.

Everything here was grounded in live verification on 2026-08-24 — engine support matrices, model architecture, device-major behavior. Claims that could not be verified are marked as such inline; nothing is asserted from vibes.

Architecture

Proxmox host — amdgpu+amdkfd stay loaded (LXC shares the host driver)
  GRUB: amd_iommu=on iommu=pt pcie_aspm=off ttm.pages_limit/page_pool_size=<auto>
   └─ LXC <VMID> unprivileged Ubuntu 24.04 — nesting/keyctl/fuse + apparmor unconfined
       └─ podman rootful, Quadlet-managed:
          ai-llm     :8000  Qwen3.8-27B GGUF · ROCmFPX llama-server · dGPU (HIP gfx1100)
                            q8 KV @131k default · optional layer-split for >262k stretch
          ai-embed   :8001  harrier GGUF · llama-server Vulkan RADV · iGPU render node
          ai-rerank  :8002  jina-v3.5 Q8_0 GGUF · llama-server Vulkan RADV · iGPU render node
          ai-gateway :8080  nginx path router (/llm/ /embed/ /rerank/)

Key design decisions and why:

• ROCmFPX (llama.cpp fork) as the LLM engine — the only family that both supports the qwen3_5 hybrid linear-attention arch AND can layer-split across heterogeneous AMD GPUs over a slow link. vLLM cannot do the split (homogeneous-GPU requirement; gfx1151 unsupported upstream); SGLang rejects gfx1100 outright.
• Embed/rerank on the iGPU's GTT pool — tiny models, frees ~3–4 GiB of dVRAM so the LLM owns the full card.
• Hybrid attention = cheap long context: only 16 of 64 layers carry KV (64 KiB/token bf16 → 16 KiB at q4). Full math: docs/memory-budget.md. Native max is 262k; >262k toward 1M is a documented stretch tier via layer-split + q4 KV — arithmetically feasible, hours-scale prefill.
• Health-gated updates with genuine rollback (maintenance/update-stack.sh re-tags the previous image ID on failure — not just a restart).

Deploy order

Phase 1 — Proxmox host

# physical first: host/oculink-checklist.md (DEG1 DIP switches, power-before-POST, BIOS)
cd proxmox-llm && sudo bash host/host-setup.sh     # grub + udev, idempotent, --check available
sudo reboot                                        # only if grub changed
cat /proc/cmdline                                  # verify flags landed
ls -l /dev/kfd /dev/dri/                           # kfd + 2 render nodes must exist

Phase 2 — LXC container

cd ../lxc
sudo ./gpu-passthrough.sh 100 --dry-run            # inspect rendered conf
sudo ./gpu-passthrough.sh 100                      # writes /etc/pve/lxc/100.conf
sudo pct create 100 local-lvm:vztmpl-ubuntu-24.04_...tar.zst --config /etc/pve/lxc/100.conf
sudo pct start 100 && sudo pct enter 100

Phase 3 — inside the container

# copy this repo into the container (e.g. scp/git clone), then:
./guest-bootstrap.sh                # podman, groups, quadlets, model staging, services
podman build -f /opt/ai-stack/stack/Dockerfile.rocmfpx -t localhost/rocmfpx:latest /opt/ai-stack/stack
podman build -f /opt/ai-stack/stack/Containerfile.llamacpp-vulkan -t localhost/llamacpp-vulkan:latest /opt/ai-stack/stack
systemctl restart ai-llm ai-embed ai-rerank ai-gateway

Phase 4 — automation

systemctl enable --now ai-update.timer             # weekly: podman auto-update + gated updates

Verification ladder (execution gates)

Run each gate before trusting the next:

1. GPUs visible in-container: rocminfo | grep -E 'gfx1100|gfx1151', ls -l /dev/kfd /dev/dri/. If ROCm tools aren't installed bare (they shouldn't be), run them inside any rocm container instead.
2. Dual-target image enumerates both GPUs: build log shows gfx1100;gfx1151; llama-server --list-devices inside the container shows two HIP devices. If HIP enumeration fails → rebuild with Vulkan-only targets (guaranteed device visibility, small perf cost).
3. Functional health: /opt/ai-stack/stack/healthcheck.sh all --deep — includes an output-sanity probe on the LLM (layer-split corruption has been observed in the wild on other stacks; never skip deep mode after topology changes).
4. Context-tier ladder: at each ctx (128k → 262k → stretch tiers), load a max-context prompt and record rocm-smi --showmeminfo vram + GTT used (healthcheck.sh all --deep prints occupancy). Promote LLM_CTX in stack/llm.env only on measured headroom.
5. MTP/speculative flags (if you enable them via LLM_EXTRA_FLAGS): validate output sanity vs MTP-off on identical prompts; disable if divergent (PromptForge routes were validated on gfx1151 — dGPU-side paths may differ).

Troubleshooting quick table

Symptom	Check
Engine runs, but CPU-speed inference	GID mismatch: stat -c %g /dev/kfd inside CT vs group membership of the process user. This is the #1 silent failure
GPU vanishes after cold boot	OCuLink power sequencing / DIP switches — host/oculink-checklist.md
GFX ring timeouts under load	Deep-idle wake issue — see checklist §5
OOM at high context	Drop LLM_CTX one rung or switch KV to q4_0; check docs/memory-budget.md
Garbage/nonsense completions	Split corruption probe failed — try single-GPU (LLM_TENSOR_SPLIT= empty), then HIP-only vs Vulkan-only builds
ai-embed unhealthy	harrier GGUF missing — conversion recipe in docs/gguf-notes.md

Kernel-upgrade coupling

The container's ROCm userspace rides on the host kernel's amdgpu/amdkfd. Either hold kernel updates out of unattended-upgrades on the host, or accept a reboot window and re-run host/host-setup.sh --check + healthchecks after kernel bumps. This is the one coupling containers cannot absorb.

Deliberate alternatives (documented, not default)

• vLLM profile (stack/quadlet/ai-vllm.container, off by default): NVFP4 checkpoint fits 24 GiB, but vLLM's NVFP4 kernels are NVIDIA-centric — prove weight-only-NVFP4 loads on RDNA3 before relying on it. Single-GPU only.
• llama-swap gateway (~5.5k★): excellent if you later want hot-swapped, TTL-unloaded models — but it launches upstreams itself (cmd per model) and has no pure-proxy mode, which conflicts with this stack's always-on dedicated Quadlet services. If you adopt it, let it own the llama-server processes directly rather than wrapping these units.
• hal0: Strix-Halo-native platform manager with podman slots and separate llm/embedding/reranking slot types — the heavier-weight alternative to this hand-rolled Quadlet set.

File map

Path	Purpose
host/	GRUB cmdline, udev rules, OCuLink checklist, idempotent host setup
lxc/	Conf template, GID/node-resolving passthrough renderer, guest bootstrap
stack/	Quadlets, env tunables, gateway config, container recipes, healthcheck
maintenance/	Update scripts with rollback, pinned model fetcher, systemd timer
docs/	Memory budget math, GGUF pins/conversions