EngramHalo Automated Builds (AMD ROCm™ 7)

High-performance automated nightly and on-demand builds of EngramHalo.cpp with built-in AMD ROCm™ 7 runtime libraries for AMD Strix Halo (gfx1151).

---

⚡ Project Overview

EngramHalo.cpp (Aristo94/EngramHalo.cpp, strix-halo-qwen4exp branch) is a specialized fork of llama.cpp tuned specifically for Qwen 3.8 Flash-Next on AMD Strix Halo (Ryzen AI MAX+ 395 / Radeon 8060S, gfx1151, 96–128 GB unified memory):

flowchart TD
    subgraph Engine ["EngramHalo Inference Stack"]
        E1["llama.cpp Core + Strix Halo Qwen4Exp Kernel Series"]
        E2["True QSA Sparse Gather (Selected KV row gather instead of dense masking)"]
        E3["Working MTP Draft Head (Speculative decoding: 24.4 → 39.3 tok/s)"]
        E4["SSD-Backed Engram Table (~1 GiB resident RAM footprint at 262K context)"]
        E5["Portable AMD ROCm™ 7 Closure ($ORIGIN RPATH)"]
        E1 --> E2 --> E3 --> E4 --> E5
    end

Key Performance Innovations:

1. True QSA Sparse Gather: Instead of running dense attention with a mask (full KV memory bandwidth), decode gathers only the ~2.3K selected rows (top_k 2048 plus block tail), doubling prefill throughput at depth (91 → 192 t/s at 131K).
2. Working MTP Draft Head: Integrated Multi-Token Prediction with draft weights from the official checkpoint, boosting decode on code from 24.4 → 39.3 tok/s.
3. NVMe / SSD Engram Table Streaming: Keeps the model's 26.8 GiB engram (n-gram) table on SSD (-lm mmap), consuming only ~1 GiB of resident RAM while unlocking full 262K context.
4. Standalone Portable Binaries: Bundles complete ROCm 7 runtime libraries (libamdhip64, librocblas, libhipblas, libhipblaslt, librocsolver, .kpack) with portable $ORIGIN RPATHs. No system ROCm installation required!

---

🎯 Supported AMD GPU Targets

Target Code	GPU Architecture	Target Hardware / Devices
gfx1151	Strix Halo APU (RDNA3.5)	AMD Ryzen AI MAX+ Pro 395, Ryzen AI MAX 390, Radeon 8060S (96GB / 128GB Unified Memory)

---

🍺 Homebrew Installation

Install directly via the Heretek-AI/homebrew-tap:

# 1. Tap the repository
brew tap Heretek-AI/tap

# 2. Install EngramHalo
brew install engramhalo

# 3. Verify installation
engramhalo-cli --help

Running the Server

# Launch server with Qwen 3.8 Flash-Next and MTP sidecar
engramhalo \
  -m /path/to/Qwen3.8-Flash-Next-ROCmFP4.gguf \
  --spec-draft /path/to/mtp-Qwen3.8-Flash-Next-Q8_0.gguf \
  --spec-draft-p-min 0.75 \
  -lm mmap \
  -ctk q8_0 \
  -ctv q8_0 \
  --port 8080 \
  --host 0.0.0.0

# Or manage as a background service via Homebrew
brew services start engramhalo

---

🚀 CI Workflows

• .github/workflows/build-engramhalo.yml: Multi-target build and release pipeline for Aristo94/EngramHalo.cpp.
• .github/workflows/test-engramhalo.yml: Hardware smoke-test workflow for Strix Halo runners.
• .github/actions/test-engramhalo-build: Composite verification action.

---

📄 License & Attribution

• EngramHalo.cpp: Developed by Aristo94 (Aristo94/EngramHalo.cpp) under the MIT License.
• llama.cpp: Developed by Georgi Gerganov and contributors under the MIT License.
• EngramHalo Builder Pipeline: Developed by Heretek AI under the MIT License.