Local Qwen 3.8 27B: scripts, a how-to, and a three-way comparison (#63)

* Local Qwen 3.8 27B: design and plan

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-local.sh: start, stop and status for the local llama-server

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-opencode.sh: OpenCode on the local Qwen server

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-claude.sh: Claude Code on the local Qwen server, in its own config dir

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Local Qwen: setup procedure and verified server flags

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-local.sh: native reasoning flags and MTP on by default

llama-server has --reasoning-effort and --reasoning off, so the plan's
--chat-template-kwargs is replaced (plan corrected in Task 2). The MTP
default, checked against this build's --help, drafts with mtp.gguf on the GPU.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Claude Code on local Qwen: fix the chat template, pass the context size

Claude Code sends hook context as a system message mid-conversation, and
Qwen's template refuses any system message after the first, so every
request failed with HTTP 500. qwen-local.sh template saves a copy that
renders it as a system turn, and start uses it. qwen-claude.sh tells
Claude Code the server's context (CLAUDE_CODE_MAX_CONTEXT_TOKENS).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-opencode.sh: always --standalone

OpenCode 2 runs sessions in a background service that keeps the config
and environment it started with, so a later OPENCODE_CONFIG_CONTENT (and
the safety variables) were ignored. --standalone gives each session a
private server with exactly this config.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Local Qwen: measured speed, context and reasoning effort

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-local.sh: cap the prompt cache in system RAM at 2 GB

llama-server keeps a prompt cache in system RAM, 8 GB by default. In an
agent session it grew past WSL's 12 GB memory cap, and the kernel's
out-of-memory killer stopped the server. With --cache-ram 2048 the
server stays at about 3.4 GB after five long prompts.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-local.sh: keep 4 context checkpoints, not 32

Qwen 3.8 is a hybrid model: each context checkpoint holds its recurrent
state in system RAM. With llama-server's default of 32, a long session
grew the server from 2 GB to 5.7 GB, and with an agent and Go tests
beside it, WSL's 12 GB cap was hit and the server was killed. With 4 it
stays at about 2.3 GB through 50K tokens of conversation.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Local Qwen: three-way comparison and lessons

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-claude.sh: guard the account dirs in both modes; strip cloud switches

The config-dir guard covered only init and missed $HOME, so a run with
QWEN_CLAUDE_DIR=~/.claude would have used the work config, and init with
$HOME would have written ~/.claude.json. Both modes now refuse a dir that
is inside an account dir or holds one. The Bedrock, Vertex and Foundry
switches are removed, since they take Claude Code past the base URL.
Found by the final review.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Local Qwen notes: the remaining config differences and the skills link

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-measure.py and qwen-capture.py: the measurement and capture tools

The speed, long-prompt and memory-growth measurements in the notes, and
the request capture that found the template and OpenCode service bugs,
were throwaway scripts. They are now repo tools with tests, so the
experiment can be repeated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* qwen-compare.sh: run one task three ways and keep the evidence

The three-way comparison (OpenCode on local Qwen, Claude Code on local
Qwen, Claude Code on a hosted model) was a throwaway runner. It is now a
repo script with tests: throwaway worktrees at one commit, the same
prompt, and per run the output, diff, server-log slice and wall time.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* Local Qwen: a reproduce how-to, the comparison's evidence, docs accepted

The how-to holds every step: setup, daily use, the measurements, the
comparison and request capture, with troubleshooting for what broke. The
notes keep the findings. The three B-071 diffs, the prompt and the run
summary are kept, since the worktrees are removed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Bruno Guilhermo de Barros Valério <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
authored and GitHub committed 2026-10-20 02:12:29 +02:00
1 parent b92de01967
commit 2858c115f7
22 files changed
+2389

No files matched your search

@@ -0,0 +1,54 @@
diff --git a/internal/engine/engine.go b/internal/engine/engine.go
index 0e5db0d..6dd06d0 100644
--- a/internal/engine/engine.go
+++ b/internal/engine/engine.go
@@ -103,6 +103,9 @@ func advance(ctx context.Context, r *run.Run) (Outcome, error) {
err = runInit(ctx, r, n)
case schema.KindGrade:
err = runGrade(ctx, r, n)
+ default:
+ // A kind the engine does not know must never pass as if it had run: stop the run on the manifest.
+ err = unknownKind(r, n)
}
var sb *sentBack
var ex *expanded
@@ -150,6 +153,15 @@ func advance(ctx context.Context, r *run.Run) (Outcome, error) {
return Outcome{Status: run.StatusDone}, r.Save()
}
+// unknownKind fails the run for a node whose kind the engine has no way to run: the graph is a bad manifest.
+func unknownKind(r *run.Run, n schema.Node) error {
+ cause := fmt.Errorf("%s: node %s has the kind %q, which this Odin does not know", gungnir.GradeBadManifest, n.ID, n.Kind)
+ if err := r.Log(gungnir.Event{Name: gungnir.EventNodeFailed, Node: n.ID, Result: cause.Error()}); err != nil {
+ return err
+ }
+ return &nodeFailure{cause}
+}
+
// doneResult is how a finished run is described: "done", or the tasks it gave up, listed first (spec §7.3).
func doneResult(s *run.State) string {
var blocked []string
diff --git a/internal/engine/grade_failure_test.go b/internal/engine/grade_failure_test.go
index 24fe315..78fd25e 100644
--- a/internal/engine/grade_failure_test.go
+++ b/internal/engine/grade_failure_test.go
@@ -35,6 +35,19 @@ func TestAResultsFileThatDoesNotMatchTheManifestIsRefusedAsBadManifest(t *testin
}
}
+func TestANodeOfAKindTheEngineDoesNotKnowStopsTheRunAsBadManifest(t *testing.T) {
+ g := startGraded(t, gradedNodes, tasktest.Results(nil), nil)
+ nodes := g.r.Setup.Graph.Nodes
+ nodes[len(nodes)-1].Kind = "teleport" // the graph loader rejects this kind, so the test sets it after the load
+ out := g.advance(t)
+ if out.Status != run.StatusFailed || out.Err == nil || !strings.Contains(out.Err.Error(), "bad_manifest") || !strings.Contains(out.Err.Error(), "teleport") {
+ t.Fatalf("Advance() = %+v, want a failed run whose error says bad_manifest", out)
+ }
+ if len(g.grader.Specs) != 0 {
+ t.Errorf("the grader ran %d time(s) for a node of an unknown kind", len(g.grader.Specs))
+ }
+}
+
func TestAGradingRunThatFailsFailsTheNodeAndAResumeGradesAgain(t *testing.T) {
g := startGraded(t, gradedNodes, tasktest.Results(nil), nil)
g.grader.Err = errors.New("the grader timed out after 5m0s")
@@ -0,0 +1,40 @@
diff --git a/internal/engine/engine.go b/internal/engine/engine.go
index 0e5db0d..41676ac 100644
--- a/internal/engine/engine.go
+++ b/internal/engine/engine.go
@@ -103,6 +103,9 @@ func advance(ctx context.Context, r *run.Run) (Outcome, error) {
err = runInit(ctx, r, n)
case schema.KindGrade:
err = runGrade(ctx, r, n)
+ default:
+ // The manifest names a kind the engine does not run: the run stops failed (backlog B-071).
+ err = &nodeFailure{fmt.Errorf("bad_manifest: the graph names a node kind, %q, that the engine does not run", n.Kind)}
}
var sb *sentBack
var ex *expanded
diff --git a/internal/engine/engine_test.go b/internal/engine/engine_test.go
index 853b58a..1d67276 100644
--- a/internal/engine/engine_test.go
+++ b/internal/engine/engine_test.go
@@ -254,3 +254,21 @@ func TestEveryNodeInALeafEndsWithACheckpoint(t *testing.T) {
t.Error("node a's node_finished line doesn't carry its checkpoint commit")
}
}
+
+func TestAKindTheEngineDoesNotRunStopsTheRunWithBadManifest(t *testing.T) {
+ r := start(t, "key-points", "", " - { id: a, kind: exec, run: [echo, ok], repo: repos/app }\n", "steps: {}\n")
+ // ParseGraph enforces the kind's enum, so the bad kind is put into the parsed graph: a manifest
+ // the engine is given names a kind it does not run (backlog B-071).
+ r.Setup.Graph.Nodes[0].Kind = "frobnicate"
+ out, err := Advance(context.Background(), r)
+ if err != nil || out.Status != run.StatusFailed {
+ t.Fatalf("Advance() = %+v, %v; want the run stopped failed", out, err)
+ }
+ if !strings.Contains(out.Err.Error(), "bad_manifest") {
+ t.Errorf("Advance() = %+v; the failure must name bad_manifest", out)
+ }
+ text := ledgerText(t, r)
+ if !strings.Contains(text, `"event":"run_failed"`) || !strings.Contains(text, "bad_manifest") {
+ t.Errorf("the ledger:\n%s", text)
+ }
+}
@@ -0,0 +1 @@
In this Go repository (odin), fix backlog item B-071: `Advance` in internal/engine/engine.go has no default case in its switch on the node kind, so a graph node of a kind the engine does not know is skipped as if it had passed. Add a default case that stops the run with the `bad_manifest` reason, following how the engine already stops runs for other manifest problems. Add a test: a graph with a made-up node kind stops with bad_manifest. Run `GOWORK=off go test ./internal/engine/` and make sure it passes (the package's tests take about 90 seconds). Do not commit. Do not ask clarifying questions; make reasonable choices and say what you chose.
@@ -0,0 +1,41 @@
diff --git a/internal/engine/engine.go b/internal/engine/engine.go
index 0e5db0d..bc9844e 100644
--- a/internal/engine/engine.go
+++ b/internal/engine/engine.go
@@ -103,6 +103,9 @@ func advance(ctx context.Context, r *run.Run) (Outcome, error) {
err = runInit(ctx, r, n)
case schema.KindGrade:
err = runGrade(ctx, r, n)
+ default:
+ // A kind the engine does not know is a bad manifest, never a pass.
+ err = &nodeFailure{fmt.Errorf("bad_manifest: the graph names the node kind %q, which the engine does not know", n.Kind)}
}
var sb *sentBack
var ex *expanded
diff --git a/internal/engine/engine_test.go b/internal/engine/engine_test.go
index 853b58a..b128838 100644
--- a/internal/engine/engine_test.go
+++ b/internal/engine/engine_test.go
@@ -8,6 +8,7 @@ import (
"testing"
"time"
+ "github.com/ouroboric-dev/gungnir/schema"
"github.com/ouroboric-dev/odin/internal/ledger"
"github.com/ouroboric-dev/odin/internal/run"
"github.com/ouroboric-dev/odin/internal/tree/treetest"
@@ -254,3 +255,14 @@ func TestEveryNodeInALeafEndsWithACheckpoint(t *testing.T) {
t.Error("node a's node_finished line doesn't carry its checkpoint commit")
}
}
+
+func TestAnUnknownNodeKindIsABadManifest(t *testing.T) {
+ r := start(t, "key-points", "", " - { id: a, kind: exec, run: [echo, hi], repo: repos/app }\n", "steps: {}\n")
+ // ParseGraph refuses the kinds its schema doesn't know; the case this covers is a graph whose
+ // schema is newer than the engine's (backlog B-071), so the node is put in after the run started.
+ r.Setup.Graph.Nodes = append([]schema.Node{{ID: "odd", Kind: "quincunx", Repo: "repos/app"}}, r.Setup.Graph.Nodes...)
+ out, err := Advance(context.Background(), r)
+ if err != nil || out.Status != run.StatusFailed || !strings.Contains(out.Err.Error(), "bad_manifest") {
+ t.Fatalf("Advance() = %+v, %v; want failed with bad_manifest", out, err)
+ }
+}
@@ -0,0 +1,4 @@
run exit wall_s
opencode 0 508
qwen-claude 0 551
hosted 0 147
1 run exit wall_s
2 opencode 0 508
3 qwen-claude 0 551
4 hosted 0 147
@@ -0,0 +1,176 @@
---
title: Local Qwen 3.8 27B coding assistant
status: accepted
date: 2026-10-19
style: spec
---
# Local Qwen 3.8 27B coding assistant
## 1. Intent
The founder wants a coding assistant that runs on their own machine: Qwen 3.8 27B on the RTX 4090,
driven by OpenCode, and also by Claude Code for comparison. There are three reasons:
- **Limits and cost.** Claude subscription limits and paid API usage cap how much agentic work
happens in a day. A local model has no rate limit and no per-token bill.
- **Curiosity and learning.** The founder wants to understand local inference: quantization,
context memory, reasoning effort, speculative decoding, and how agent tools behave on a smaller
model.
- **Groundwork.** OpenCode is the harness that Yggdrasil's Gleipnir already drives. Lessons about its
configuration and failure modes carry over.
**Split from Yggdrasil.** Routing Yggdrasil roles to Qwen is a separate track. It uses the OpenRouter
path that already exists (`openrouter/qwen/qwen3.8-27b`) and is out of scope here. Hosted Qwen 3.8 27B
costs about $0.02 per million input tokens and $0.50 per million output tokens on
[OpenRouter](https://openrouter.ai/qwen/qwen3.8-27b), so local inference does not win on cost for
Yggdrasil. It wins on limits, privacy, offline use and learning.
## 2. Goals
1. One command starts the model server, and one command stops it.
2. OpenCode, connected to the local server, completes a real small task in one of the founder's
repos.
3. Claude Code, connected to the same server, completes the same task, so the two harnesses can be
compared on one model.
4. The setup reaches about 100K tokens of context and records the measured decode speed. The target
is 40 tokens per second or more.
5. A short notes page records what the founder learned: the settings that mattered, where each
harness struggled, and how the result compares with Claude.
## 3. Non-goals
- Yggdrasil integration (see §1).
- Parallel sessions or serving other people. One session at a time.
- vLLM, SGLang, or a Windows-native server.
- Community forks of llama.cpp (for example [ninfer-4090](https://github.com/sergiuszm/ninfer-4090)
or BeeLlama). They are candidate follow-ups for curiosity, not part of this work.
- Vision input, fine-tuning, or other models.
## 4. Anti-goals
These make the work a failure even if the model runs:
- **The Claude account split breaks.** The founder runs two Claude Code accounts, split by folder
(work under `~/flink` with `~/.claude`; personal under `~/brunogbv` with `~/.claude-personal`),
with `claude`, `claude-work` and `claude-personal` shell functions in
`~/.config/claude-accounts.zsh` and an `account-guard.sh` hook in each config. The local setup MUST
NOT change either config dir, either login, or the `claude` function's routing. Local Claude Code
MUST use its own config dir and its own command, and MUST NOT log in to an Anthropic account.
- **Requests leave the machine.** The local Claude Code and OpenCode MUST send model requests only to
the local server. Local Claude Code SHOULD turn off non-essential traffic
(`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1`).
- **A pile of one-off commands.** The setup MUST be repeatable from the repo (see §6).
## 5. Constraints
Measured on 2026-10-19:
| Item | Value |
| --- | --- |
| GPU | NVIDIA RTX 4090, 24 GB; about 2.1 GB already used by the Windows desktop |
| Driver | NVIDIA 610.57 (WSL), CUDA 13.3 user-mode driver |
| CPU | AMD Ryzen 7 7800X3D, 8 cores, 16 threads; WSL gets 12 |
| RAM | 32 GB on Windows; WSL is capped at 12 GB by `C:\Users\Bruno\.wslconfig` |
| Disk | about 708 GB free on `/` |
| OS | WSL2, Linux 6.18; shell zsh |
| Already installed | gcc, podman (aliased as `docker`, runtime crun). Not installed: CUDA toolkit (`nvcc`), cmake, llama.cpp, OpenCode. No NVIDIA CDI spec for podman |
The model (from [Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B)):
- 27B dense, Apache 2.0, 262,144-token native context.
- Hybrid layout: of the 64 layers, only 16 use full attention (4 KV heads, head dim 256). The other
48 are Gated DeltaNet with a fixed-size state. The KV cache is therefore about 64 KB per token at
16 bits, or about 32 KB at 8 bits. [ANALYSIS: 16 layers × 2 × 4 × 256 × 2 bytes.]
- Trained with multi-token prediction (MTP), which llama.cpp can use for speculative decoding.
- Thinking is on by default, and `reasoning_effort` defaults to `xhigh`. Levels are `xhigh`,
`medium` and `low`; `enable_thinking` turns thinking off. Qwen notes that low effort can make
multi-turn agentic tasks slower overall, because they fail and retry more.
- Recommended sampling with thinking: temperature 1.0, top_p 0.95, top_k 20, min_p 0.
- Qwen ran its coding benchmarks in the Claude Code harness with a 256K context.
Memory budget [ANALYSIS]: a Q4-class GGUF is about 17–18 GB. About 2 GB is already used and about
1 GB goes to compute buffers, which leaves roughly 3–4 GB for the KV cache. That is about 100–128K
tokens at 8-bit KV. MTP buffers also take memory: one user hit the VRAM limit at 196K context
([unsloth discussion](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/discussions/32)).
Reported speeds on a 4090 ([computingforgeeks](https://computingforgeeks.com/run-qwen3-8-27b-locally/),
[gist](https://gist.github.com/ryan4yin/19db9fa44972c5735c1d181e8888d4fe)): about 47 tokens per
second without MTP and about 60–110 with MTP, depending on context length and prompt. In
[Simon Willison's review](https://simonwillison.net/2026/Aug/16/qwen-38-27b/), tool calls in a coding
agent worked, but the default `xhigh` effort spent about 22K reasoning tokens on one small prompt.
## 6. Decided
These are the founder's decisions, or calls Claude made in the conversation that the founder
accepted:
- **Runtime:** `llama-server` from upstream llama.cpp, built with CUDA, running in WSL with every
layer on the GPU. It serves the OpenAI-compatible API, and the Anthropic Messages API at
`/v1/messages` that Claude Code uses
([Hugging Face blog](https://huggingface.co/blog/ggml-org/anthropic-messages-api-in-llamacpp)).
- **Model file:** a Q4-class GGUF of Qwen 3.8 27B, plus the MTP draft file if llama.cpp needs one.
- **Main harness:** OpenCode, with the local server configured as an OpenAI-compatible provider.
- **Comparison harness:** Claude Code with `ANTHROPIC_BASE_URL` pointing at the local server, under a
separate config dir and a separate command (see §4).
- **Hosted baseline:** the same task also runs once on hosted Claude, through the founder's
personal account, to give a measured baseline: wall time, tokens, retries, and pass or fail. The
task MUST be small, so the baseline uses few subscription tokens.
- **Defaults:** one session at a time; about 100K tokens of context; low reasoning effort by
default, which the founder can raise for each task.
- **Location:** scripts in `ouroboric/scripts`, with tests in `ouroboric/scripts/tests` in the same
style as the existing ones (fake commands, no GPU or network needed). The work goes in through a
pull request. The notes page goes in `ouroboric/docs/` beside this document.
Claude's calls after the builder check (the founder can override them):
- **When the targets collide.** 40 tokens per second is a target that gets recorded, not a pass/fail
gate. Measure speed twice: on a near-empty context, and with about 100K tokens filled. If the
Q4 file, MTP and 100K context don't all fit in 24 GB, keep the Q4 file and MTP, and reduce the
context. The floor is 64K tokens. Record what fits.
- **One-time setup.** Installing the CUDA toolkit, building llama.cpp, installing OpenCode and
downloading the model are a written procedure in the notes page. The founder runs the steps that
need `sudo`. The download and build steps MAY be scripts. They need tests only if they hold logic
that can break. Start, stop and the local Claude Code launcher MUST be scripts with tests.
- **WSL memory cap.** Leave `.wslconfig` as it is, because changing it needs `wsl --shutdown`. If the
build or model load fails at 12 GB, use fewer build jobs, then ask the founder.
- **Network.** Configuration is enough; no firewall or network namespace. "Model requests" means
calls to an LLM only. Turn off OpenCode's auto-update and sharing, and point its small model at
the local server too. Check once, by reading the server log, that every model call reached the
local server.
- **The test task's result.** Each run (OpenCode, local Claude Code, hosted Claude) works in its own
throwaway git worktree, directly on the host, in the harness's default ask-before-acting
permission mode. The founder judges each diff, then the worktree is removed. Nothing is pushed,
and the local agents get no GitHub access.
- **The local Claude Code config dir (founder's decision: match the personal setup, for a fair
comparison).** It gets the same plugins as `~/.claude-personal` (`superpowers`, `gopls-lsp`,
`skill-creator` from `claude-plugins-official`), the same user skills (`~/.claude-personal/skills`),
and the `merge-requires-green.sh` hook. It leaves out three things:
- the `account-guard.sh` hook, because the local config has no account;
- claude.ai connectors (Claude Docs, Artifacts), because they need an account and would send data
off the machine;
- the personal auto-memory.
Install the plugins into the local config dir, or link to them. Do not change the files in
`~/.claude-personal`. Record the remaining differences in the notes page. The task prompt for all
three runs is the same, and it says to skip clarifying questions, so the `superpowers` brainstorming
gate doesn't stall the comparison.
- **Reasoning effort.** Changing the default effort MAY need a server restart with a different
flag. Per-request control inside a harness is a nice-to-have. Record whether each harness
supports it.
## 7. Left to the builder
- How to install the CUDA toolkit and build llama.cpp: build from source in WSL, or use the
`server-cuda` container under podman with an NVIDIA CDI spec. Building from source is preferred if
it is not much harder, because MTP support is recent.
- The exact quant: unsloth Q4_K_M or UD-Q4_K_XL, or another Q4-class file. Pick one, and record why.
- Where model files live (outside every repo) and how the scripts find them.
- The `llama-server` flags: context size, KV cache type, MTP settings, sampling, and how the default
reasoning effort is set.
- Script names and the command names for start, stop, and local Claude Code. Do not shadow `claude`,
`claude-work` or `claude-personal`.
- How OpenCode's config is installed: global config or an env var such as `OPENCODE_CONFIG`.
- The task used for goals 2 and 3. Pick a small, real task from a backlog in one of the founder's
repos, and confirm it with the founder before running it.
- How to measure speed and context: the server's own timings are enough.
@@ -0,0 +1,183 @@
---
title: Run Qwen 3.8 27B locally and repeat the experiment
status: accepted
date: 2026-10-20
style: guide
related: [2026-10-19-local-qwen-design.md, 2026-10-19-local-qwen-notes.md]
---
# Run Qwen 3.8 27B locally and repeat the experiment
This guide sets up Qwen 3.8 27B on one NVIDIA RTX 4090 under WSL2. Then you use it from OpenCode and
from Claude Code, and repeat the measurements and the three-way comparison in
[the notes](2026-10-19-local-qwen-notes.md). Every command runs in zsh, from the root of the
`ouroboric` repo, unless a step says otherwise.
## Before you start
You need:
- An RTX 4090, or another 24 GB NVIDIA card, with a Windows driver that offers CUDA 13.3 or newer in
WSL. Check with `nvidia-smi`.
- WSL2 with Ubuntu 22.04, about 25 GB of free disk, Homebrew, `uv`, and `sudo` for one step.
- Claude Code (`claude`) on `PATH`, and the personal Claude Code config in `~/.claude-personal`.
The model, the llama.cpp build and the run state live in `~/.local/share/qwen-local` (`QWEN_HOME`),
outside every repo.
## Set up the server
1. Install the CUDA toolkit. This is the only step that needs `sudo`. Install only
`cuda-toolkit-13-3`. Never install `cuda`, `cuda-drivers` or `nvidia-driver-*`: in WSL, the Windows
driver is the GPU driver.
```zsh
cd /tmp && wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get install -y cuda-toolkit-13-3
```
`/usr/local/cuda/bin/nvcc --version` now prints `release 13.3`.
1. Install cmake and OpenCode: `brew install cmake opencode`.
1. Build `llama-server` with CUDA. Keep Homebrew off `PATH` for the build: Homebrew's linker misses
the CUDA libraries, and Homebrew's OpenSSL needs a newer glibc than Ubuntu 22.04 has.
```zsh
git clone https://github.com/ggml-org/llama.cpp ~/.local/share/qwen-local/llama.cpp
cd ~/.local/share/qwen-local/llama.cpp
git checkout 4c3c581 # the build the notes measured; leave this out for the latest
export PATH=/usr/local/cuda/bin:/usr/bin:/bin
/home/linuxbrew/.linuxbrew/bin/cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89 \
-DCMAKE_BUILD_TYPE=Release -DLLAMA_OPENSSL=OFF
/home/linuxbrew/.linuxbrew/bin/cmake --build build --config Release -j 6 --target llama-server
```
Open a new terminal afterwards, to get your usual `PATH` back. `-j 6` keeps the build inside a
12 GB WSL memory cap.
1. Download the model (16.5 GB) and its MTP draft file (1.4 GB), and link them to the names the
scripts use:
```zsh
mkdir -p ~/.local/share/qwen-local/models && cd ~/.local/share/qwen-local/models
uvx --from huggingface_hub hf download unsloth/Qwen3.8-27B-GGUF \
Qwen3.8-27B-UD-Q4_K_M.gguf MTP/mtp-Qwen3.8-27B-Q4_0.gguf --local-dir .
ln -sfn Qwen3.8-27B-UD-Q4_K_M.gguf model.gguf
ln -sfn MTP/mtp-Qwen3.8-27B-Q4_0.gguf mtp.gguf
```
1. Start the server: `scripts/qwen-local.sh start`. It is ready in about 25 seconds. To see the
flags, run `scripts/qwen-local.sh cmd`.
1. Save the fixed chat template, then restart, so Claude Code works:
```zsh
scripts/qwen-local.sh template
scripts/qwen-local.sh stop && scripts/qwen-local.sh start
```
1. Set up the local Claude Code config dir, `~/.claude-qwen`: `scripts/qwen-claude.sh init`. It
installs the personal setup's three plugins and links its skills. It never touches `~/.claude` or
`~/.claude-personal`.
The server's settings come from environment variables: `QWEN_CTX` (102400), `QWEN_EFFORT` (`low`,
`medium`, `xhigh` or `off`), `QWEN_MTP_ARGS` (empty turns MTP off), `QWEN_CACHE_RAM` (2048) and
`QWEN_CTX_CHECKPOINTS` (4). The comment block at the top of `scripts/qwen-local.sh` explains each one.
## Use it
- **OpenCode:** `scripts/qwen-opencode.sh` opens the interactive session. For a headless run, use
`scripts/qwen-opencode.sh run --auto "..."`; without `--auto`, a run that calls a tool waits for an
approval that it cannot get.
- **Claude Code:** `scripts/qwen-claude.sh` opens the interactive session, and
`scripts/qwen-claude.sh -p "..."` runs one prompt headless. It asks before it acts. Claude Code's own
effort setting has no effect here; restart the server with another `QWEN_EFFORT` instead.
- **Status and stop:** `scripts/qwen-local.sh status` and `scripts/qwen-local.sh stop`.
Both launchers send model calls only to the local server, and remove `ANTHROPIC_API_KEY`,
`CLAUDE_CODE_OAUTH_TOKEN` and `OPENROUTER_API_KEY` from the session. `qwen-claude.sh` also removes the
Bedrock, Vertex and Foundry switches.
## Repeat the measurements
Start the server, then run these commands. The figures in each answer come from the server itself.
1. Decode speed on a short prompt, three runs:
```zsh
printf 'Write a Go function that parses a duration like "1h30m" or "45s" into seconds, with table-driven tests. Explain your choices briefly.' > /tmp/short.txt
python3 -I scripts/qwen-measure.py speed --runs 3 --max-tokens 512 /tmp/short.txt
```
To measure without MTP, restart with `QWEN_MTP_ARGS="" scripts/qwen-local.sh start` and run it
again.
1. Speed with about 95K tokens of context:
```zsh
python3 -I scripts/qwen-measure.py prompt --tokens 95000 --out /tmp/long.txt \
"$HOME/brunogbv/ouroboric/yggdrasil/gleipnir/**/*.go"
python3 -I scripts/qwen-measure.py speed --runs 1 --max-tokens 256 /tmp/long.txt
```
1. Reasoning effort: restart with `QWEN_EFFORT=low`, `medium` and `xhigh` in turn, and run
`speed --runs 1 --max-tokens 20000 /tmp/short.txt` each time.
1. The server's memory over a growing conversation:
```zsh
python3 -I scripts/qwen-measure.py growth --turns 20 --pid "$(cat ~/.local/share/qwen-local/run/server.pid)" /tmp/long.txt
```
To see the problem the defaults fix, restart with `QWEN_CTX_CHECKPOINTS=32 QWEN_CACHE_RAM=8192`
and run it again.
## Repeat the comparison
The comparison runs one task three times. Each run gets a throwaway worktree of the same commit and
the same prompt. Tools are approved without asking, so use a task you trust the agents with.
```zsh
scripts/qwen-local.sh start
scripts/qwen-compare.sh --repo ~/brunogbv/ouroboric/yggdrasil/odin --commit bc973e3 \
--prompt docs/5-research/2026-10-19-local-qwen-comparison/prompt.txt --out /tmp/qwen-compare-b071
```
- **What it runs:** OpenCode and Claude Code on the local model, then Claude Code on Sonnet through the
personal account. The Sonnet run uses that subscription. Change the model with `--hosted-model`, or
skip runs with `--only opencode,qwen-claude`.
- **What it keeps:** each run's output, diff and slice of the server log, a `summary.tsv` with exit
codes and wall times, and `rss.log`.
- **The task:** `bc973e3` is the `odin` commit the notes used, before B-071 was fixed. The diffs from
the recorded runs are in
[2026-10-19-local-qwen-comparison/](2026-10-19-local-qwen-comparison/).
- **Clean up:** when you have read the diffs, run the cleanup commands that the script prints at the
end.
## Debug what a tool sends
`scripts/qwen-capture.py` stands in for the model server. It saves each request body and answers it
with an error:
```zsh
python3 -I scripts/qwen-capture.py 8099 /tmp/capture &
ANTHROPIC_BASE_URL=http://127.0.0.1:8099 CLAUDE_CONFIG_DIR=~/.claude-qwen ANTHROPIC_AUTH_TOKEN=x command claude -p hi
kill %1
```
The bodies are in `/tmp/capture/00-v1_messages.json` and the files after it. For OpenCode, set the
provider's `baseURL` to `http://127.0.0.1:8099/v1`, and pass `--standalone`.
## Troubleshooting
- **Every Claude Code request fails with HTTP 500, "System message must be at the beginning".** The
fixed chat template is missing. Run the template step in [Set up the server](#set-up-the-server).
- **The server disappears in a long session, and `dmesg` shows `Out of memory: Killed process
(llama-server)`.** WSL ran out of system RAM. Keep `QWEN_CACHE_RAM` and `QWEN_CTX_CHECKPOINTS` low,
or raise `memory=` in `C:\Users\<you>\.wslconfig`. Run `wsl --shutdown` from Windows to apply it; this
closes every WSL session.
- **OpenCode ignores a changed config.** A background `opencode serve --service` from an earlier
session holds the old one. `scripts/qwen-opencode.sh` avoids it with `--standalone`. To stop the
service, run `opencode service stop`.
- **`start` says the port already answers.** Another program uses port 8088. Stop it, or set
`QWEN_PORT`.
@@ -0,0 +1,217 @@
---
title: Local Qwen 3.8 27B coding assistant, notes
status: accepted
date: 2026-10-20
style: research
related: [2026-10-19-local-qwen-design.md, 2026-10-19-local-qwen-plan.md]
---
# Local Qwen 3.8 27B coding assistant, notes
What it took to run Qwen 3.8 27B on one RTX 4090 under WSL2, how fast it is, and how it compares
with hosted Claude. The design is [2026-10-19-local-qwen-design.md](2026-10-19-local-qwen-design.md).
## Setup
Done on 2026-10-19 and 2026-10-20. The steps to repeat it are in
[2026-10-19-local-qwen-howto.md](2026-10-19-local-qwen-howto.md). What happened along the way:
- **Versions.** CUDA toolkit 13.3 (`cuda-toolkit-13-3`, which matches the driver's CUDA 13.3), cmake
4.4.4, OpenCode 2.0.32 (the Homebrew formula is `opencode`, not `sst/tap/opencode`), and llama.cpp at
`4c3c581` (2026-10-19, version 0.7.0-dev, build 50).
- **The build failed twice with Homebrew first on `PATH`.**
- Homebrew's `ld` did not search the CUDA library folder (`undefined reference to
cudaGetLastError@libcudart.so.13`).
- Homebrew's OpenSSL needs glibc 2.38, and Ubuntu 22.04 has 2.35 (`undefined reference to
__isoc23_strtol@GLIBC_2.38`). llama-server uses OpenSSL only to download models, so it is turned
off.
- **The model file.** UD-Q4_K_M (16.46 GB), with the 1.37 GB MTP draft file. It leaves about 1 GB more
for the context than UD-Q4_K_XL (17.56 GB) does. The design targets about 100K tokens of context,
and Windows already holds about 2.1 GB of the card's 24 GB. [ANALYSIS]
- **The flags, checked against this build's `--help`.**
- MTP is `--spec-type draft-mtp -md mtp.gguf -ngld all`, and the draft's KV cache is `q8_0` too.
- Reasoning effort has native flags: `--reasoning-effort low|medium|xhigh`, and `--reasoning off`.
- At 102,400 tokens of context with MTP, the card holds 23.5 of 24.6 GB, Windows included.
- **Claude Code needed two fixes.**
- It sends hook context, such as superpowers' SessionStart text, as a `system` message in the
middle of the conversation. The model's template refuses any system message after the first
(`Jinja Exception: System message must be at the beginning.`), so every request failed with HTTP
500. `qwen-local.sh template` saves a copy that renders such a message as a system turn.
- Claude Code does not know the model, and assumes 200K tokens of context. `qwen-claude.sh` sets
`CLAUDE_CODE_MAX_CONTEXT_TOKENS` to the server's real context.
First run (2026-10-20), with a 30-token coding prompt at low effort:
- Decode speed was 98.5 tokens per second.
- MTP accepted 210 of 309 drafted tokens (68%).
- The reasoning was 887 characters long.
`/v1/messages`, the Anthropic Messages API, answered with a `thinking` block and a `text` block.
Smoke runs (2026-10-20). Each was asked which files in `scripts/` start with `qwen`, and each made a
tool call and answered correctly:
- OpenCode: 8 seconds. After the `--standalone` fix (see [Measurements](#measurements)), 4.5
seconds with `run --auto`.
- Claude Code, after the template fix: 14 seconds.
## Measurements
All measurements were taken on 2026-10-20, with UD-Q4_K_M at a 102,400-token context and a q8_0 KV
cache. The figures are the server's own timings for each request (`timings` in the
`/v1/chat/completions` response, and `print_timing` in the server log). The short prompt asked for a
Go duration parser with table-driven tests (75 tokens, 512 tokens out).
| Run | Decode (tokens/s) | Prompt (tokens/s) | MTP drafts accepted | VRAM in use |
| --- | --- | --- | --- | --- |
| Short prompt, MTP on (3 runs) | 83.9 median (82.3–92.2) | about 180 (cold) | 58–67% | 23.5 GB |
| Short prompt, MTP off (3 runs) | 46.5 median (46.3–46.6) | about 200 (cold) | none | 21.2 GB |
| 94,975-token prompt of Go source, MTP on | 68.6 | 1,987 (48 s to read it) | 82% | 23.6 GB |
- **MTP pays.** It gives about 1.8 times the decode speed, for about 2.3 GB of VRAM. [ANALYSIS]
- **The full context fits.** About 95K tokens of context loaded and answered, with no out-of-memory
error. The design's floor of 64K was not needed, so the default stays at 102,400.
- **Speed at long context.** Decode slowed to about 69 tokens per second at 95K tokens filled, which
is still above the 40 target. The cost of a long context is in reading it: about 2,000 tokens per
second, so a fresh 95K-token prompt takes about 48 seconds. Later turns reuse the prompt cache.
**Reasoning effort.** The server was restarted at each level, then given the short prompt with a
20,000-token cap. Each level was run once, so these are rough figures:
| `QWEN_EFFORT` | Output tokens | Reasoning (characters) | Wall time |
| --- | --- | --- | --- |
| `low` | 2,635 | 4,777 | 27.0 s |
| `medium` | 2,365 | 4,138 | 23.9 s |
| `xhigh` (the model's default) | 6,042 | 19,928 | 67.8 s |
`xhigh` spent about four times the reasoning of `low`, for the same small task. `low` and `medium`
were about the same on this prompt. [ANALYSIS]
**Per-request effort.** The template adds different instructions for each level, so the prompt's
token count shows which level a request got: 75 tokens at `low`, 45 at `medium` and 87 at `xhigh`.
- **The OpenAI API honours it.** `"reasoning_effort": "xhigh"` in a `/v1/chat/completions` request
overrides the server's default.
- **OpenCode can send it.** A capture of its requests showed that a model's
`"options": {"reasoningEffort": "xhigh"}` arrives as `reasoning_effort`. Its title requests ask for
`low`.
- **The Anthropic API ignores it.** `/v1/messages` gives the same prompt whatever
`output_config.effort` or `thinking` says. Claude Code sends `effort: high` and adaptive thinking,
and the server ignores both. For Claude Code, only the server's `QWEN_EFFORT` counts.
**OpenCode's background service.**
- OpenCode 2 starts `opencode serve --service` on first use. Later `opencode run` calls connect to it,
and it keeps the config and environment it started with. A new `OPENCODE_CONFIG_CONTENT` was
silently ignored, so the launcher's provider limit and credential stripping would not have applied.
- `qwen-opencode.sh` now always passes `--standalone`, which gives each session a private server. The
flag must come after `run` (`opencode run --standalone`), or first for the interactive session.
- In standalone mode, a headless `run` that calls a tool waits for approval, with no way to give
it. Pass `--auto` to approve tools in a run you trust.
**Only the local server got model calls.**
- Every model call in the smoke runs appears in the server log as a slot task.
- The launcher tests prove that `ANTHROPIC_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN` and
`OPENROUTER_API_KEY` never reach either tool.
- The two capture runs, pointed at a dummy server, showed that each tool sends its model calls only
to the configured base URL. Claude Code sent one `/v1/messages` request; OpenCode sent 13
`/v1/chat/completions` requests.
Verified on 2026-10-20.
## Comparison
**The task.** The founder chose backlog item B-071 in `odin`: `Advance` had no `default` case, so a
graph node of an unknown kind passed silently. Each run started from `odin` `bc973e3`, in a
throwaway worktree.
**The setup.**
- All three runs got the same prompt: fix it, add a test, run `go test ./internal/engine/` (about 90
seconds), do not commit, and do not ask questions.
- They ran headless, by the founder's choice:
- OpenCode with `run --auto`.
- Claude Code with `-p --permission-mode acceptEdits`, and Bash limited to `go`, `gofmt`,
`git diff` and `git status`.
- The server ran at `QWEN_EFFORT=low`, with MTP on.
| Run | Wall time | Turns or requests | Tokens | Result |
| --- | --- | --- | --- | --- |
| OpenCode + local Qwen | 508 s | 46 tool calls, 42 model requests | 131,591 prompt tokens read and 21,137 generated (server log); largest context 91,707 | `default` case + test; `go test` and `go vet` pass |
| Claude Code + local Qwen | 551 s | 64 turns, 54 model requests | 109,008 prompt tokens read and 25,005 generated (server log); largest context 76,439 | `default` case + test; `go test` and `go vet` pass |
| Claude Code + hosted Sonnet | 146 s | 12 turns | 20 input + 295,376 cache-read + 31,908 cache-write, and 4,403 output; $0.20 of usage | `default` case + helper + test; `go test` and `go vet` pass |
**How the results were checked.**
- `go vet ./...` and `go test ./internal/engine/` ran in each worktree after the runs.
- Each new test was run again with `-count=1`.
- Each new test was then run against the original `engine.go`. All three failed there with the bug's
symptom (`Status:done`, when the test wants a failed run), so each test does catch the bug.
**The three fixes.**
- All three chose the engine's existing failure path: a `nodeFailure` that `fail()` turns into a
failed run, with `bad_manifest` in the error.
- **Hosted Sonnet** added a helper. It uses the `gungnir.GradeBadManifest` constant, logs a
`node_failed` event, and repeats the node ID in the message (which `fail()` already adds).
- **OpenCode** noticed that `fail()` adds the node ID, and left it out of its message on purpose.
- **Claude Code on Qwen** wrote the smallest change. Both Qwen runs wrote `bad_manifest` as a
string, not as the constant.
**Retries.** None of the runs needed a correction. Two earlier attempts of the local runs ended
when the kernel's out-of-memory killer stopped `llama-server` (see [Lessons](#lessons)); those
attempts are not counted above.
The diffs, the prompt and the run summary are kept in
[2026-10-19-local-qwen-comparison/](2026-10-19-local-qwen-comparison/). The founder took no fix from these
runs into `odin`, and B-071 stays open in the backlog. The worktrees were removed after the pull
request.
Claude Code puts an "estimated cost" of $2.37 on the local run. It prices the local model's tokens
as if they were Anthropic's; the real cost was GPU time and electricity.
## Lessons
- **Local Qwen did the real task.** Both local harnesses produced a correct, tested fix to a real
backlog item, with a test that fails without the fix. It took about 3.5 to 4 times the hosted
run's wall time, and no subscription usage. [ANALYSIS]
- **The hosted model was more efficient and slightly more thorough.** It took 12 turns, against 46 to
64 for the local model, and used the package's constant and event log. The local runs read more
code before they acted: OpenCode's context reached 91,707 tokens, close to the 102,400 limit.
[ANALYSIS]
- **On this task, OpenCode and Claude Code were about even on Qwen.** OpenCode was a little faster and
more careful about the message. Claude Code made a smaller diff. [ANALYSIS]
- **Qwen handled superpowers well enough.** The local Claude Code had the founder's superpowers
plugin, whose SessionStart text tells the model to use skills first. The task prompt said not to
ask questions, and the run went straight to the work. [ANALYSIS: one run only.]
- **System RAM, not VRAM, was the binding limit.** The GPU held the model and the context with room
to spare. The kernel's out-of-memory killer stopped `llama-server` three times under WSL's 12 GB
cap, because two llama-server defaults keep state in system RAM:
- the prompt cache, `--cache-ram`, 8 GB by default;
- for this hybrid model, 32 context checkpoints of its recurrent state, which grew the server from
2.0 to 5.7 GB in a 20-turn test.
With `--cache-ram 2048` and `--ctx-checkpoints 4`, the server peaked at 3.0 GB during both local
runs. On the founder's request, `.wslconfig` now gives WSL 20 GB (it was 12 GB). That takes effect
after `wsl --shutdown`.
- **Each harness had a surprise.**
- Claude Code sends hook context as a mid-conversation `system` message, which Qwen's template
refuses. The fix is a saved template.
- Claude Code does not know the model's context size and assumes 200K.
- Claude Code's effort settings are ignored by `/v1/messages`.
- OpenCode 2's background service ignores a new config, so the launcher passes `--standalone`.
- OpenCode's `run` waits for a tool approval that a headless run cannot give, so pass `--auto`.
- **The remaining unfairness.** The local Claude Code shares the personal setup's three plugins,
its skills and its merge hook. It differs from the hosted run in these ways:
- The hosted run had the personal auto-memory and the claude.ai connectors. Neither was relevant
to this task.
- `~/.claude-personal/settings.json` sets `defaultMode: auto`, a permissions allowlist, and
`effortLevel` (`high` for Opus 5.5). The local config uses `defaultMode: default` and has no
allowlist. All three headless runs overrode the mode with `acceptEdits` and the same Bash
allowlist. The local server ignores Claude Code's effort setting.
- Each configuration ran once, so the timing and token figures are one sample each.
- **The skills link is a write path.** `~/.claude-qwen/skills` links to `~/.claude-personal/skills`, so
the local model sees the same skills, as the founder asked. A local session could change a personal
skill through that link, for example with skill-creator, but only through an action that the
founder approves. Copy the skills instead if that matters more than staying in sync.
@@ -0,0 +1,626 @@
---
title: Local Qwen 3.8 27B coding assistant, implementation plan
status: accepted
date: 2026-10-19
style: spec
---
# Local Qwen 3.8 27B Coding Assistant Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Run Qwen 3.8 27B on the local RTX 4090 with `llama-server`, use it from OpenCode and from a
separate Claude Code config, and compare both with hosted Claude on one small real task.
**Architecture:** Upstream llama.cpp, built with CUDA in WSL, serves the model on `127.0.0.1`. One
script starts and stops the server. Two launchers start OpenCode and Claude Code with environment
variables that point them only at that server. A shared library holds the defaults and the running
server's state. A notes page holds the founder-run setup procedure, the measurements and the
comparison.
**Tech Stack:** bash (shellcheck-clean, `set -euo pipefail`), llama.cpp `llama-server`, CUDA toolkit
for WSL-Ubuntu, a Q4-class GGUF from Hugging Face, OpenCode (Homebrew), Claude Code (Homebrew,
already installed at `/home/linuxbrew/.linuxbrew/bin/claude`), python3 only inside tests.
**Spec:** [`2026-10-19-local-qwen-design.md`](2026-10-19-local-qwen-design.md)
## Global Constraints
- Repo: `~/brunogbv/ouroboric/ouroboric`, branch `local-qwen`. Stage explicit paths only; never
`git add -A` (the tree has an untracked `docs/assets/logo/` that is not ours).
- Scripts go in `scripts/`, tests in `scripts/tests/<name>.test.sh`, in the style of
`scripts/tests/ygg-engine.test.sh`: a temp `HOME`, fake commands, no GPU and no network. Scripts are
AGPL-3.0-only through `REUSE.toml`; add no SPDX headers.
- Each script opens with a comment block: what it does, one usage line per mode, the environment it
reads, in the style of `scripts/ygg-engine.sh`.
- The server binds `127.0.0.1` only. Default port `8088`. Model alias `qwen3.8-27b`.
- Default context `102400` tokens; floor `65536`. KV cache `q8_0` for K and V.
- Default reasoning effort `low`. Allowed: `low`, `medium`, `xhigh`, `off`.
- Sampling: temperature 1.0, top_p 0.95, top_k 20, min_p 0.
- One session at a time: `--parallel 1`.
- Model files, the llama.cpp build and run state live in `QWEN_HOME`, default
`$HOME/.local/share/qwen-local`, never in a repo.
- The local Claude Code config dir is `QWEN_CLAUDE_DIR`, default `$HOME/.claude-qwen`. No script may
write to `~/.claude`, `~/.claude.json`, `~/.claude-personal`, or `~/.config/claude-accounts.zsh`, or
shadow the `claude`, `claude-work` or `claude-personal` commands.
- Launchers MUST unset `ANTHROPIC_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN` and `OPENROUTER_API_KEY`.
- `.wslconfig` stays as it is. Steps that need `sudo` are founder-run; the agent prints them and
waits.
- No session links in commits or the PR. Commits end with the `Co-Authored-By` line in use.
- Lint must pass: `pre-commit run --files <changed files>` (shellcheck, markdownlint, reuse,
gitleaks).
## Review Focus
1. **A stale pid file** after a crash or `wsl --shutdown`: `status` says "not running" and `start`
works, without manual cleanup. (Task 2)
2. **A reused pid:** the pid file names a live process that is not `llama-server`. `stop` MUST NOT
kill it. (Task 2)
3. **Another program already on port 8088:** `start` refuses with a clear message instead of
reporting that program as the model. (Task 2)
4. **The model does not fit in VRAM:** `llama-server` exits while loading. `start` reports failure
and shows the end of the log, instead of waiting out the timeout. (Task 2)
5. **Launchers with the server down, or with a cloud credential in the caller's shell:** they refuse,
or they strip the credential, so no session reaches a hosted model by accident. (Tasks 3 and 4)
---
## File Structure
| File | Responsibility |
| --- | --- |
| `scripts/qwen-lib.sh` | sourced only: defaults, the server URL, the health check, the running server's state file |
| `scripts/qwen-local.sh` | `start`, `stop`, `status`, `cmd` for `llama-server` |
| `scripts/qwen-opencode.sh` | runs OpenCode against the local server |
| `scripts/qwen-claude.sh` | `init` the local Claude Code config dir; run Claude Code against the local server |
| `scripts/tests/qwen-local.test.sh` | tests `qwen-lib.sh` and `qwen-local.sh` |
| `scripts/tests/qwen-opencode.test.sh` | tests `qwen-opencode.sh` |
| `scripts/tests/qwen-claude.test.sh` | tests `qwen-claude.sh` |
| `docs/5-research/2026-10-19-local-qwen-notes.md` | setup procedure, measurements, comparison, lessons (style `research`) |
---
### Task 0: Branch
- [ ] **Step 1: Create the branch and commit the design and this plan**
```bash
cd ~/brunogbv/ouroboric/ouroboric
git switch main && git pull --ff-only
git switch -c local-qwen
git add docs/5-research/2026-10-19-local-qwen-design.md docs/5-research/2026-10-19-local-qwen-plan.md
git commit -m "Local Qwen 3.8 27B: design and plan"
```
---
### Task 1: Setup procedure and a first manual run
Founder-run where `sudo` is needed. This task produces the verified facts that Task 2 encodes.
**Files:**
- Create: `docs/5-research/2026-10-19-local-qwen-notes.md`: front matter (`title`, `status: draft`,
`date`, `style: research`) and sections `## Setup`, `## Measurements`, `## Comparison`,
`## Lessons`. This task fills `## Setup`.
- [ ] **Step 1: Print the `sudo` steps for the founder, and wait until they confirm they ran them**
- The CUDA toolkit, from NVIDIA's `wsl-ubuntu` repository for Ubuntu 22.04, at a version no newer
than CUDA 13.3 (the driver's user-mode version). **Install only a `cuda-toolkit-*` package, never
`cuda`, `cuda-drivers` or any `nvidia-driver-*` package: in WSL the Windows driver provides the GPU
driver.**
- `cmake` and `build-essential`, from apt (or `brew install cmake`, which needs no `sudo`).
- [ ] **Step 2: Verify the toolkit**
Run: `/usr/local/cuda/bin/nvcc --version && nvidia-smi --query-gpu=name,memory.total --format=csv`
Expected: a CUDA 13.x release line, and `NVIDIA GeForce RTX 4090, 24564 MiB`.
- [ ] **Step 3: Build llama.cpp in `$QWEN_HOME/llama.cpp`**
Clone `https://github.com/ggml-org/llama.cpp`, record the commit, then
`cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89 && cmake --build build --config Release -j 6 --target llama-server`.
Use `-j 6` because WSL has 12 GB. If the build is killed for lack of memory, retry with `-j 3`, then
ask the founder.
Run: `$QWEN_HOME/llama.cpp/build/bin/llama-server --version`
Expected: a version line that names CUDA.
- [ ] **Step 4: Pick the quant and download it**
Look at `unsloth/Qwen3.8-27B-GGUF` (and the official Qwen GGUF repo, if there is one). Pick a
Q4-class file of 18 GB or less: Q4_K_M, or UD-Q4_K_XL. Download it with
`uvx --from huggingface_hub hf download <repo> <file> --local-dir $QWEN_HOME/models`. Also download
the MTP draft file, if llama.cpp needs one for this model. Link the files:
`$QWEN_HOME/models/model.gguf` (and `$QWEN_HOME/models/mtp.gguf`, if there is a draft file). Record
the repo, the file names, the sizes and the reason for the choice in `## Setup`.
- [ ] **Step 5: Confirm the server flags against the built binary**
Run `llama-server --help` and confirm the spelling of every flag in Task 2's `cmd` list. In
particular, confirm:
- the flash-attention flag (`-fa on`);
- `--chat-template-kwargs`;
- the KV cache type flags;
- `-ngl`;
- the MTP or speculative-decoding flags for this model.
Write the confirmed MTP flags as one string. That string becomes the default of `QWEN_MTP_ARGS` in
Task 2. Willison used `--spec-type draft-mtp`; the upstream spelling may differ. If a flag in Task 2's
list is spelled differently, correct Task 2 in this plan and say so in the commit message.
- [ ] **Step 6: First manual run**
Start the server by hand with the Task 2 flags, at context `102400` and with MTP. Then:
- `curl -s 127.0.0.1:8088/v1/chat/completions` with a short prompt. It must answer, with a reasoning
part that is short at `low` effort.
- `curl -s 127.0.0.1:8088/v1/messages` with the Anthropic body shape (`model`, `max_tokens`,
`messages`). It must answer, which confirms the endpoint Claude Code needs.
- Read `nvidia-smi` and record the VRAM in use.
If loading fails for lack of memory, reduce the context in steps of 8192 tokens, but not below
`65536`. Record the largest value that loads. That value replaces `102400` as the default in Task 2.
- [ ] **Step 7: Install OpenCode, without `sudo`**
Run: `brew install sst/tap/opencode` (or the current Homebrew name), then `opencode --version`.
Record the version.
- [ ] **Step 8: Write `## Setup` as numbered steps that the founder can repeat, then commit**
```bash
git add docs/5-research/2026-10-19-local-qwen-notes.md
git commit -m "Local Qwen: setup procedure and verified server flags"
```
---
### Task 2: `qwen-lib.sh` and `qwen-local.sh`
**Files:**
- Create: `scripts/qwen-lib.sh`, `scripts/qwen-local.sh` (executable)
- Test: `scripts/tests/qwen-local.test.sh` (executable)
**Interfaces:**
- Produces (`qwen-lib.sh`, sourced; it sets these unless the caller already did):
- `QWEN_HOME` (default `$HOME/.local/share/qwen-local`), `QWEN_PORT` (`8088`), `QWEN_CTX`
(`102400`, or the value Task 1 recorded), `QWEN_ALIAS` (`qwen3.8-27b`).
- `QWEN_STATE="$QWEN_HOME/run/server.env"`: written by `start`, removed by `stop`. It holds the
lines `QWEN_PORT=<n>` and `QWEN_CTX=<n>` of the running server.
- `qwen_load_state`: if `$QWEN_STATE` exists, set `QWEN_PORT` and `QWEN_CTX` from it. Read it as
data, with `sed`; do not `source` it.
- `qwen_url`: prints `http://127.0.0.1:$QWEN_PORT`.
- `qwen_healthy`: exit 0 if `curl -fsS -m 2 "$(qwen_url)/health"` succeeds.
- `qwen_require_server`: runs `qwen_load_state`, then fails with exit 2 and
`the local model server is not running on <url>; start it with scripts/qwen-local.sh start`
when it is not healthy.
- Produces (`qwen-local.sh start|stop|status|cmd`). Environment: `QWEN_LLAMA_SERVER` (default
`$QWEN_HOME/llama.cpp/build/bin/llama-server`), `QWEN_MODEL` (default
`$QWEN_HOME/models/model.gguf`), `QWEN_EFFORT` (`low`), `QWEN_MTP_ARGS` (the string from Task 1,
word-split; empty means no MTP), `QWEN_START_TIMEOUT` (seconds, `300`). Files: pid
`$QWEN_HOME/run/server.pid`, log `$QWEN_HOME/logs/server.log`. The previous log is kept as
`server.log.1`.
The `cmd` argument list, in this order:
`-m $QWEN_MODEL --alias $QWEN_ALIAS --host 127.0.0.1 --port $QWEN_PORT -ngl 999 -c $QWEN_CTX -fa on --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1 --jinja --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 <reasoning> $QWEN_MTP_ARGS`.
`<reasoning>` is `--reasoning-effort <effort>`, or `--reasoning off` for `off`. (Corrected in Task 2: the
plan first passed `--chat-template-kwargs`; llama-server has native reasoning flags.) The default
`QWEN_MTP_ARGS`, confirmed in Task 1, is
`--spec-type draft-mtp -md $QWEN_HOME/models/mtp.gguf -ngld all -ctkd q8_0 -ctvd q8_0`.
- [ ] **Step 1: Write the failing tests** in `scripts/tests/qwen-local.test.sh`
Use the `check`, `run` and `has` helpers from `ygg-engine.test.sh`. Put fakes in `$T/bin`, first on
`PATH`:
- a fake `llama-server` that writes its arguments, one per line, to `$T/args`, then sleeps (or exits
with code 1 at once when `FAKE_DIE` is set);
- a fake `curl` that exits 0 when `$T/healthy` exists, and exits 7 otherwise.
`QWEN_LLAMA_SERVER` points to the fake server, `QWEN_MODEL` to an empty file, and
`QWEN_START_TIMEOUT=3`. The cases:
```text
cmd: default line contains "--host 127.0.0.1", "--port 8088", "-c 102400", "--cache-type-k q8_0",
"--parallel 1", and "--reasoning-effort low"; MTP flags on by default
cmd: QWEN_EFFORT=off -> contains "--reasoning off", no "--reasoning-effort"
cmd: QWEN_EFFORT=high -> exit 2, message names QWEN_EFFORT and the four allowed values
cmd: QWEN_MTP_ARGS="--spec-a x" -> last two args are "--spec-a" "x"; QWEN_MTP_ARGS="" -> no "--spec-a"
start: QWEN_MODEL missing -> exit 2, message names the path; no fake server started
start: fake becomes healthy -> exit 0, prints "ready on http://127.0.0.1:8088", pid file names a live
process, $T/args has "-m" "<model path>", server.env has QWEN_PORT=8088 and QWEN_CTX=102400
start: run again while running -> exit 0, "already running", still one fake process
start: FAKE_DIE -> exit 1, prints "exited while starting" and the log tail; no pid file, no server.env
start: never healthy -> exit 1 within ~3 s, "did not become ready", fake process killed, no pid file
start: port already healthy and no pid file -> exit 2, "port 8088 already answers"; fake not started
status: running -> exit 0, "running", the URL; not running -> exit 3, "not running"
status: stale pid file (dead pid) -> exit 3, "not running", pid file and server.env removed
stop: running -> exit 0, process gone within 30 s, pid file and server.env removed
stop: not running -> exit 0, "not running"
stop: pid file names a live `sleep` (not llama-server) -> exit 0, "stale", the sleep still alive,
pid file removed
qwen_load_state: server.env with QWEN_PORT=9000 -> qwen_url prints http://127.0.0.1:9000
```
- [ ] **Step 2: Run the tests, and check that they fail**
Run: `bash scripts/tests/qwen-local.test.sh`
Expected: `FAIL` lines, because `qwen-local.sh` does not exist.
- [ ] **Step 3: Implement `scripts/qwen-lib.sh` and `scripts/qwen-local.sh`**
`qwen-local.sh` sources `qwen-lib.sh` from its own folder.
- Liveness means: the pid is alive, and `/proc/<pid>/cmdline` contains `llama-server`. Anything else
is a stale pid file.
- `start`: launch the server in the background with `nohup`, writing to the log. Poll
`qwen_healthy` once a second, and stop polling early if the process dies.
- `stop`: send TERM, wait up to 30 s, then send KILL.
- [ ] **Step 4: Run the tests and shellcheck, and check that they pass**
Run: `bash scripts/tests/qwen-local.test.sh && shellcheck scripts/qwen-lib.sh scripts/qwen-local.sh scripts/tests/qwen-local.test.sh`
Expected: only `ok` lines, and no shellcheck output.
- [ ] **Step 5: Real smoke**
Run: `scripts/qwen-local.sh start && scripts/qwen-local.sh status && scripts/qwen-local.sh stop`
Expected: `ready on http://127.0.0.1:8088`, then `running`, then a clean stop. `nvidia-smi` shows the
VRAM freed.
- [ ] **Step 6: Commit**
```bash
git add scripts/qwen-lib.sh scripts/qwen-local.sh scripts/tests/qwen-local.test.sh
git commit -m "qwen-local.sh: start, stop and status for the local llama-server"
```
---
### Task 3: `qwen-opencode.sh`
**Files:**
- Create: `scripts/qwen-opencode.sh` (executable)
- Test: `scripts/tests/qwen-opencode.test.sh` (executable)
**Interfaces:**
- Consumes: `qwen_require_server`, `qwen_url`, `QWEN_CTX`, `QWEN_ALIAS` from `qwen-lib.sh`.
- Produces: `qwen-opencode.sh [OPENCODE ARGS...]`. It execs `${QWEN_OPENCODE_BIN:-opencode}` with the
arguments unchanged, and with:
- `OPENCODE_CONFIG_CONTENT`: the JSON below;
- `OPENCODE_DISABLE_CLAUDE_CODE=1` and `OPENCODE_DISABLE_EXTERNAL_SKILLS=1`, so that OpenCode does
not read `~/.claude`, which is the work account;
- `OPENCODE_DISABLE_AUTOUPDATE=true`, `OPENCODE_DISABLE_SHARE=true` and
`OPENCODE_DISABLE_MODELS_FETCH=true`.
The project's own OpenCode config stays enabled.
The config, as built by the script (`<url>`, `<ctx>` and `<alias>` filled in):
```json
{
"$schema": "https://opencode.ai/config.json",
"autoupdate": false,
"share": "disabled",
"enabled_providers": ["qwen-local"],
"provider": {
"qwen-local": {
"npm": "@ai-sdk/openai-compatible",
"name": "Qwen local",
"options": { "baseURL": "<url>/v1" },
"models": { "<alias>": { "name": "Qwen 3.8 27B (local)", "tool_call": true, "reasoning": true,
"limit": { "context": <ctx>, "output": 32768 } } }
}
},
"model": "qwen-local/<alias>",
"small_model": "qwen-local/<alias>"
}
```
Check `enabled_providers` against the installed OpenCode's config schema. If the key has a different
name, use the schema's name, and record the change.
- [ ] **Step 1: Write the failing tests**
Use a fake `opencode` that writes its arguments and its full environment to files, and the fake
`curl` from Task 2. Parse the config with `python3 -c 'import json,os; ...'`. The cases:
```text
server down -> exit 2, message from qwen_require_server; fake opencode not run
server up, server.env says QWEN_PORT=9000 QWEN_CTX=65536 -> baseURL "http://127.0.0.1:9000/v1",
limit.context 65536, model and small_model "qwen-local/qwen3.8-27b", enabled_providers ["qwen-local"]
args pass through: qwen-opencode.sh run "hi" -> fake got "run" "hi"
env: the five OPENCODE_DISABLE_* variables set as above
env: caller exports ANTHROPIC_API_KEY, CLAUDE_CODE_OAUTH_TOKEN, OPENROUTER_API_KEY -> none reach the fake
```
- [ ] **Step 2: Run the tests, and check that they fail**
Run: `bash scripts/tests/qwen-opencode.test.sh`
Expected: `FAIL` lines.
- [ ] **Step 3: Implement `scripts/qwen-opencode.sh`**
Build the JSON with `printf`. Every value is a number or a fixed string, so no escaping is needed.
Use `exec env -u … VAR=… "$bin" "$@"`.
- [ ] **Step 4: Run the tests and shellcheck, and check that they pass**
Run: `bash scripts/tests/qwen-opencode.test.sh && shellcheck scripts/qwen-opencode.sh scripts/tests/qwen-opencode.test.sh`
Expected: only `ok` lines, and no shellcheck output.
- [ ] **Step 5: Real smoke**
With the server running, in `~/brunogbv/ouroboric/ouroboric`, run:
`scripts/qwen-opencode.sh run "List the files in scripts/ that start with qwen, using a tool"`.
Expected: an answer that names the qwen scripts, after at least one tool call. The server log shows
the requests. Record the OpenCode version and any setting that needed a change in `## Setup`.
- [ ] **Step 6: Commit**
```bash
git add scripts/qwen-opencode.sh scripts/tests/qwen-opencode.test.sh docs/5-research/2026-10-19-local-qwen-notes.md
git commit -m "qwen-opencode.sh: OpenCode on the local Qwen server"
```
---
### Task 4: `qwen-claude.sh`
**Files:**
- Create: `scripts/qwen-claude.sh` (executable)
- Test: `scripts/tests/qwen-claude.test.sh` (executable)
**Interfaces:**
- Consumes: `qwen_require_server`, `qwen_url`, `QWEN_ALIAS`.
- Produces: `qwen-claude.sh init` and `qwen-claude.sh [CLAUDE ARGS...]`.
- Environment: `QWEN_CLAUDE_DIR` (default `$HOME/.claude-qwen`), `QWEN_CLAUDE_BIN` (default: the
`claude` binary that `command -v claude` finds in bash, which is not the zsh function),
`QWEN_PERSONAL_DIR` (default `$HOME/.claude-personal`; it is only read).
- `init` refuses with exit 2 when `QWEN_CLAUDE_DIR` resolves to `$HOME/.claude`, to
`$QWEN_PERSONAL_DIR`, or to a path inside either one.
- `init` writes `$QWEN_CLAUDE_DIR/settings.json` (it regenerates the file each time) and
`$QWEN_CLAUDE_DIR/.claude.json` with `{"hasCompletedOnboarding": true}`, but only if that file
does not exist yet.
- `init` links `$QWEN_CLAUDE_DIR/skills` to `$QWEN_PERSONAL_DIR/skills`.
- `init` then runs, with `CLAUDE_CONFIG_DIR=$QWEN_CLAUDE_DIR`:
`claude plugin marketplace add anthropics/claude-plugins-official`, and
`claude plugin install <p>@claude-plugins-official` for each of `superpowers`, `gopls-lsp` and
`skill-creator`. Check these subcommands against `claude --help` on the installed version.
- Run mode requires the server and `settings.json`. Otherwise it fails with exit 2 and
`run scripts/qwen-claude.sh init first`.
- Run mode then execs the binary with the arguments unchanged. It unsets `ANTHROPIC_API_KEY`,
`CLAUDE_CODE_OAUTH_TOKEN` and `OPENROUTER_API_KEY`, and sets:
- `CLAUDE_CONFIG_DIR=$QWEN_CLAUDE_DIR`
- `ANTHROPIC_BASE_URL=$(qwen_url)`
- `ANTHROPIC_AUTH_TOKEN=local-qwen`
- `ANTHROPIC_MODEL`, `ANTHROPIC_DEFAULT_OPUS_MODEL`, `ANTHROPIC_DEFAULT_SONNET_MODEL`,
`ANTHROPIC_DEFAULT_HAIKU_MODEL` and `CLAUDE_CODE_SUBAGENT_MODEL`, all set to `$QWEN_ALIAS`
- `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` and `DISABLE_AUTOUPDATER=1`
`settings.json`:
```json
{
"permissions": { "defaultMode": "default" },
"model": "qwen3.8-27b",
"enabledPlugins": {
"superpowers@claude-plugins-official": true,
"gopls-lsp@claude-plugins-official": true,
"skill-creator@claude-plugins-official": true
},
"extraKnownMarketplaces": {
"claude-plugins-official": { "source": { "source": "github", "repo": "anthropics/claude-plugins-official" } }
},
"hooks": {
"PreToolUse": [ { "matcher": "Bash", "hooks": [
{ "type": "command", "command": "<QWEN_PERSONAL_DIR>/hooks/merge-requires-green.sh" } ] } ]
}
}
```
- [ ] **Step 1: Write the failing tests**
Use a temp `HOME` that holds a `.claude-personal` with `settings.json`, `skills/a/SKILL.md` and
`hooks/merge-requires-green.sh`, plus a `.claude/settings.json`. Use a fake `claude` that appends its
arguments, `CLAUDE_CONFIG_DIR` and its full environment to files, and the fake `curl` from Task 2. The
cases:
```text
init: writes settings.json with the three plugins, defaultMode "default", the hook path under
$HOME/.claude-personal; .claude.json has hasCompletedOnboarding true; skills is a link to
$HOME/.claude-personal/skills; fake claude got the marketplace add and three plugin installs,
each with CLAUDE_CONFIG_DIR=$HOME/.claude-qwen
init: an existing .claude.json is left as it is
init: QWEN_CLAUDE_DIR=$HOME/.claude-personal -> exit 2, nothing written, fake not run
init: QWEN_CLAUDE_DIR=$HOME/.claude/sub -> exit 2
init + run: sha256 of every file under $HOME/.claude-personal and $HOME/.claude unchanged
run: before init -> exit 2 "init first"
run: server down -> exit 2, fake not run
run: args pass through; env has ANTHROPIC_BASE_URL=http://127.0.0.1:8088, CLAUDE_CONFIG_DIR=$HOME/.claude-qwen,
ANTHROPIC_MODEL=qwen3.8-27b, CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
run: caller exports ANTHROPIC_API_KEY, CLAUDE_CODE_OAUTH_TOKEN, OPENROUTER_API_KEY -> none reach the fake
```
- [ ] **Step 2: Run the tests, and check that they fail**
Run: `bash scripts/tests/qwen-claude.test.sh`
Expected: `FAIL` lines.
- [ ] **Step 3: Implement `scripts/qwen-claude.sh`**
Resolve the paths with `realpath -m` before the refusal check. Build `settings.json` with `printf`;
the only variable value is the hook path, and it has no quotes in it.
- [ ] **Step 4: Run the tests and shellcheck, and check that they pass**
Run: `bash scripts/tests/qwen-claude.test.sh && shellcheck scripts/qwen-claude.sh scripts/tests/qwen-claude.test.sh`
Expected: only `ok` lines, and no shellcheck output.
- [ ] **Step 5: Real smoke**
1. Run `scripts/qwen-claude.sh init`.
2. With the server running, in `~/brunogbv/ouroboric/ouroboric`, run
`scripts/qwen-claude.sh -p "Which scripts in scripts/ start with qwen? Use a tool."`.
3. Check the answer. Expected: it names the qwen scripts. No login prompt appears, and
`ls ~/.claude-qwen/plugins` shows the three plugins.
4. Check the server log. Expected: the server log shows `/v1/messages` requests.
5. Check the personal config. Expected: `claude-whoami` in `~/brunogbv` still reports the personal
account.
- [ ] **Step 6: Commit**
```bash
git add scripts/qwen-claude.sh scripts/tests/qwen-claude.test.sh
git commit -m "qwen-claude.sh: Claude Code on the local Qwen server, in its own config dir"
```
---
### Task 5: Measurements
**Files:**
- Modify: `docs/5-research/2026-10-19-local-qwen-notes.md` (`## Measurements`)
- Modify, only if the measurements say so: `scripts/qwen-lib.sh` (the `QWEN_CTX` default) and its
test
- [ ] **Step 1: Decode speed at a near-empty context**
1. With MTP on, send three `/v1/chat/completions` requests. Each has a coding prompt of about 200
tokens and `max_tokens: 512`.
2. Read `timings.predicted_per_second` and `timings.prompt_per_second` from each response.
3. Repeat with `QWEN_MTP_ARGS=""`.
4. Record the median of each run, and the draft acceptance rate from the log.
- [ ] **Step 2: Speed with about 100K tokens filled**
Build a prompt of about 95,000 tokens from real source files (for example, the contents of
`~/brunogbv/ouroboric/yggdrasil/gleipnir`, repeated). Send it with `max_tokens: 256`, with MTP on.
Record the prompt speed, the decode speed and the VRAM in use. If the request fails for lack of
memory, lower `QWEN_CTX` in steps of 8192 tokens, but not below `65536`. Record the result.
- [ ] **Step 3: Reasoning effort**
Send the same prompt at `QWEN_EFFORT=low`, `medium` and `xhigh` (restart the server each time).
Record the reasoning tokens and the wall time.
Then test per-request control:
- OpenCode: does a model `options` entry, or a variant with `reasoning_effort`, reach the server?
Check the server log.
- Claude Code: does a thinking or effort setting change the request that the server gets?
Record yes or no for each.
- [ ] **Step 4: Check that only the local server gets model calls**
During the Task 4 smoke and the Task 6 runs, every model call appears in the server log, and neither
launcher's environment holds a cloud credential (the tests prove this). Record "verified on YYYY-MM-DD".
- [ ] **Step 5: If the measured context differs from the default**
Change the `QWEN_CTX` default in `qwen-lib.sh` and the matching test case. Then run all three test
files.
- [ ] **Step 6: Commit**
```bash
git add docs/5-research/2026-10-19-local-qwen-notes.md # plus qwen-lib.sh and its test, if changed
git commit -m "Local Qwen: measured speed, context and reasoning effort"
```
---
### Task 6: Three-way comparison
**Files:**
- Modify: `docs/5-research/2026-10-19-local-qwen-notes.md` (`## Comparison`, `## Lessons`)
- [ ] **Step 1: Propose the task, and wait for the founder's choice**
Propose two or three small, real tasks from a backlog in the founder's repos, for example
`yggdrasil/docs/2-engineering/backlog.md` or a small script fix in `ouroboric`. A good task touches
one or two files and has a check that can pass or fail. Ask the founder:
- which task to use;
- which hosted model to use for the baseline (their default, or Sonnet, which costs less).
Write down the one prompt that all three runs get. It ends with:
"Do not ask clarifying questions; make reasonable choices and say what you chose."
- [ ] **Step 2: Make three throwaway worktrees from the same commit**
Run in the target repo: `git worktree add ../<repo>-cmp-<tool> -b cmp-<tool>`, for `<tool>` =
`opencode`, `qwen-claude` and `hosted`.
- [ ] **Step 3: Run the task three times, one after the other**
1. Run `scripts/qwen-opencode.sh` in the opencode worktree.
2. Run `scripts/qwen-claude.sh` in the qwen-claude worktree.
3. Run `claude-personal --model <chosen>` in the hosted worktree.
All three runs are interactive, in the default ask-before-acting mode, with the founder approving
each step. For each run, record:
- the wall time;
- the tokens: the server log for the local runs, `/cost` or the session file for hosted;
- the turns and the retries: failed tool calls, or the founder correcting it;
- whether the task's check passed;
- the founder's verdict on the diff.
- [ ] **Step 4: Write `## Comparison` and `## Lessons`**
`## Comparison` holds a table with one row per run. `## Lessons` covers:
- the settings that mattered;
- where each harness struggled, including how Qwen handled the superpowers process;
- the remaining unfairness: the hosted run had the personal memory and the claude.ai connectors.
Mark analysis `[ANALYSIS]`, as the `research` style requires.
- [ ] **Step 5: Remove the worktrees, after the founder has judged the diffs and says so**
For each worktree, run `git worktree remove ../<repo>-cmp-<tool>` and then `git branch -D cmp-<tool>`.
Run one short command per worktree.
- [ ] **Step 6: Commit**
```bash
git add docs/5-research/2026-10-19-local-qwen-notes.md
git commit -m "Local Qwen: three-way comparison and lessons"
```
---
### Task 7: Pull request
- [ ] **Step 1: Run the whole check**
```bash
for t in qwen-local qwen-opencode qwen-claude; do bash scripts/tests/$t.test.sh || echo "FAILED $t"; done
pre-commit run --files scripts/qwen-*.sh scripts/tests/qwen-*.test.sh docs/5-research/2026-10-19-local-qwen-*.md
```
Expected: only `ok` lines, no `FAILED` line, and every hook `Passed`.
- [ ] **Step 2: Set the design and the notes to `status: accepted`, then push and open the PR**
Run `scripts/as-bot.sh ouroboric git push -u origin local-qwen`, then
`scripts/as-bot.sh ouroboric gh pr create --title "Local Qwen 3.8 27B: scripts, setup and comparison" --body-file <file>`.
The body names the design, the plan and the notes. It sums up the measurements and the comparison,
and ends with the attribution line. It holds no session links.
- [ ] **Step 3: After the founder merges, return to main**
Run `git switch main && git pull --ff-only`.
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env python3
# A stand-in model server that records what a coding agent sends, for debugging the local Qwen setup
# (docs/5-research/2026-10-19-local-qwen-notes.md): it found Claude Code's mid-conversation system message
# and OpenCode's ignored config. Run it as `python3 -I`.
# qwen-capture.py PORT OUTDIR
# It listens on 127.0.0.1:PORT and saves each POST body to OUTDIR/<nn>-<path>.json, numbered from 00.
# Each POST gets HTTP 500 with an API error, so the agent stops instead of waiting; a GET (a health check)
# gets 200. Point the agent at it, for example ANTHROPIC_BASE_URL=http://127.0.0.1:PORT, or an OpenCode
# provider baseURL of http://127.0.0.1:PORT/v1. Stop it with Ctrl-C.
import http.server
import itertools
import os
import sys
if len(sys.argv) != 3:
sys.exit("usage: qwen-capture.py PORT OUTDIR")
port, out = int(sys.argv[1]), sys.argv[2]
os.makedirs(out, exist_ok=True)
seq = itertools.count()
class Handler(http.server.BaseHTTPRequestHandler):
def do_POST(self):
body = self.rfile.read(int(self.headers.get("content-length", 0)))
name = self.path.split("?")[0].strip("/").replace("/", "_") or "root"
path = os.path.join(out, f"{next(seq):02d}-{name}.json")
with open(path, "wb") as f:
f.write(body)
print(f"saved {path}", flush=True)
data = b'{"type": "error", "error": {"type": "api_error", "message": "captured by qwen-capture.py"}}'
self.send_response(500)
self.send_header("content-type", "application/json")
self.send_header("content-length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def do_GET(self):
self.send_response(200)
self.send_header("content-length", "2")
self.end_headers()
self.wfile.write(b"{}")
def log_message(self, *args):
pass
http.server.HTTPServer(("127.0.0.1", port), Handler).serve_forever()
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env bash
# Run Claude Code against the local Qwen 3.8 27B server, in a config dir of its own that has no
# account (docs/5-research/2026-10-19-local-qwen-design.md). Start the server first: qwen-local.sh start.
# qwen-claude.sh init set up the config dir: the personal setup's plugins, skills and
# merge hook, without its account, memory or account guard
# qwen-claude.sh [CLAUDE ARGS...] run Claude Code on the local model
# Environment, besides qwen-lib.sh's:
# QWEN_CLAUDE_DIR the local config dir ($HOME/.claude-qwen); never the work or personal one
# QWEN_PERSONAL_DIR the personal config dir, only read ($HOME/.claude-personal)
# QWEN_CLAUDE_BIN the claude binary (the one on PATH; the zsh `claude` function is not used)
# Claude Code does not know this model, so CLAUDE_CODE_MAX_CONTEXT_TOKENS tells it the server's context.
# It never writes to ~/.claude, ~/.claude.json or the personal config dir, and removes the cloud
# credentials of the caller's shell, so a session cannot reach a hosted model by accident.
set -euo pipefail
# shellcheck source-path=SCRIPTDIR source=qwen-lib.sh
. "$(dirname "${BASH_SOURCE[0]}")/qwen-lib.sh"
dir="${QWEN_CLAUDE_DIR:-$HOME/.claude-qwen}"
personal="${QWEN_PERSONAL_DIR:-$HOME/.claude-personal}"
bin="${QWEN_CLAUDE_BIN:-$(command -v claude || true)}"
[ -n "$bin" ] || { echo "claude is not on PATH; set QWEN_CLAUDE_BIN" >&2; exit 2; }
plugins=(superpowers gopls-lsp skill-creator)
market=claude-plugins-official
# inside DIR ROOT: DIR is ROOT or a path under it, after resolving links.
inside() {
local d r
d="$(realpath -m "$1")" r="$(realpath -m "$2")"
[ "$d" = "$r" ] || [[ "$d" == "$r"/* ]]
}
# Both modes refuse a config dir that is, or is inside, a real account's config dir, and one that holds
# them (such as $HOME, where ~/.claude.json lives): Claude Code would write that account's files.
for r in "$HOME/.claude" "$HOME/.claude-personal" "$personal"; do
if inside "$dir" "$r" || inside "$r" "$dir"; then
echo "QWEN_CLAUDE_DIR $dir overlaps the account config dir $r; the local config dir must be separate" >&2
exit 2
fi
done
if [ "${1:-}" = init ]; then
mkdir -p "$dir"
enabled="" sep=""
for p in "${plugins[@]}"; do enabled+="$sep\"$p@$market\":true"; sep=","; done
printf '{"permissions":{"defaultMode":"default"},"model":"%s","enabledPlugins":{%s},"extraKnownMarketplaces":{"%s":{"source":{"source":"github","repo":"anthropics/%s"}}},"hooks":{"PreToolUse":[{"matcher":"Bash","hooks":[{"type":"command","command":"%s/hooks/merge-requires-green.sh"}]}]}}\n' \
"$QWEN_ALIAS" "$enabled" "$market" "$market" "$personal" > "$dir/settings.json"
[ -e "$dir/.claude.json" ] || echo '{"hasCompletedOnboarding":true}' > "$dir/.claude.json"
[ -L "$dir/skills" ] || ln -s "$personal/skills" "$dir/skills"
CLAUDE_CONFIG_DIR="$dir" "$bin" plugin marketplace add "anthropics/$market"
for p in "${plugins[@]}"; do CLAUDE_CONFIG_DIR="$dir" "$bin" plugin install "$p@$market"; done
echo "set up $dir; run scripts/qwen-claude.sh to start a session"
exit 0
fi
[ -f "$dir/settings.json" ] || { echo "$dir is not set up; run scripts/qwen-claude.sh init first" >&2; exit 2; }
qwen_require_server
# The Bedrock, Vertex and Foundry switches would send the session to a hosted model, past the base URL.
exec env -u ANTHROPIC_API_KEY -u CLAUDE_CODE_OAUTH_TOKEN -u OPENROUTER_API_KEY \
-u CLAUDE_CODE_USE_BEDROCK -u CLAUDE_CODE_USE_VERTEX -u CLAUDE_CODE_USE_FOUNDRY \
-u ANTHROPIC_BEDROCK_BASE_URL -u ANTHROPIC_VERTEX_BASE_URL \
CLAUDE_CONFIG_DIR="$dir" ANTHROPIC_BASE_URL="$(qwen_url)" ANTHROPIC_AUTH_TOKEN=local-qwen \
ANTHROPIC_MODEL="$QWEN_ALIAS" ANTHROPIC_DEFAULT_OPUS_MODEL="$QWEN_ALIAS" ANTHROPIC_DEFAULT_SONNET_MODEL="$QWEN_ALIAS" \
ANTHROPIC_DEFAULT_HAIKU_MODEL="$QWEN_ALIAS" CLAUDE_CODE_SUBAGENT_MODEL="$QWEN_ALIAS" \
CLAUDE_CODE_MAX_CONTEXT_TOKENS="$QWEN_CTX" CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 DISABLE_AUTOUPDATER=1 \
"$bin" "$@"
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env bash
# Run one coding task three ways and keep what each did: OpenCode on the local Qwen server, Claude Code on
# the local Qwen server, and Claude Code on a hosted model (docs/5-research/2026-10-19-local-qwen-notes.md,
# "Comparison"). The runs are headless, one after the other, each in a throwaway worktree of REPO at REV.
# qwen-compare.sh --repo REPO --commit REV --prompt FILE --out DIR
# [--only opencode,qwen-claude,hosted] [--hosted-model MODEL] [--allow PATTERN]...
# Tools are approved without asking: OpenCode gets `run --auto`, and Claude Code gets acceptEdits with the
# --allow Bash patterns (by default Go's: "Bash(go *)", "Bash(GOWORK=off go *)", "Bash(gofmt *)", plus
# "Bash(git diff*)" and "Bash(git status*)"). The worktrees are throwaway, but the agents run on this host.
# Start the server first (qwen-local.sh start) for the local runs. The hosted run uses the personal config
# dir (QWEN_HOSTED_CONFIG_DIR, $HOME/.claude-personal) and MODEL (sonnet), so it uses that subscription.
# DIR, which must be new or empty, gets for each run <name>.out and <name>.err (the tool's output),
# <name>.patch (its diff, new files included) and <name>.server.log (the server log lines written during the
# run); summary.tsv (name, exit code, wall seconds); and rss.log (llama-server's memory every 30 seconds).
# The worktrees stay in DIR/wt-<name> for review; the cleanup commands are printed at the end.
# QWEN_COMPARE_TIMEOUT caps each run (1800 seconds). QWEN_COMPARE_OPENCODE, QWEN_COMPARE_QWEN_CLAUDE and
# QWEN_CLAUDE_BIN override the three commands.
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source-path=SCRIPTDIR source=qwen-lib.sh
. "$HERE/qwen-lib.sh"
usage() { echo "usage: qwen-compare.sh --repo REPO --commit REV --prompt FILE --out DIR [--only LIST] [--hosted-model MODEL] [--allow PATTERN]..." >&2; exit 2; }
repo="" rev="" prompt="" out="" only="opencode,qwen-claude,hosted" model="sonnet" allow=()
while [ $# -gt 0 ]; do
case "$1" in
--repo) repo="${2:?}"; shift 2 ;;
--commit) rev="${2:?}"; shift 2 ;;
--prompt) prompt="${2:?}"; shift 2 ;;
--out) out="${2:?}"; shift 2 ;;
--only) only="${2:?}"; shift 2 ;;
--hosted-model) model="${2:?}"; shift 2 ;;
--allow) allow+=("${2:?}"); shift 2 ;;
*) usage ;;
esac
done
[ -n "$repo" ] && [ -n "$rev" ] && [ -n "$prompt" ] && [ -n "$out" ] || usage
[ ${#allow[@]} -gt 0 ] || allow=("Bash(go *)" "Bash(GOWORK=off go *)" "Bash(gofmt *)" "Bash(git diff*)" "Bash(git status*)")
IFS=, read -r -a runs <<< "$only"
for r in "${runs[@]}"; do
case "$r" in opencode | qwen-claude | hosted) ;; *) echo "unknown run '$r'; use opencode, qwen-claude or hosted" >&2; usage ;; esac
done
[ -f "$prompt" ] || { echo "the prompt file $prompt does not exist" >&2; exit 2; }
if [ -d "$out" ] && [ -n "$(ls -A "$out")" ]; then echo "--out $out is not empty; give a new folder" >&2; exit 2; fi
mkdir -p "$out"
out="$(cd "$out" && pwd)"
text="$(cat "$prompt")"
log="$QWEN_HOME/logs/server.log"
stamp="$(date +%Y%m%d%H%M%S)-$$"
claude_flags=(--permission-mode acceptEdits --allowedTools "${allow[@]}" --output-format json)
# Sample llama-server's memory while the runs go, if it is running.
pidf="$QWEN_HOME/run/server.pid"
if [ -f "$pidf" ] && [ -r "/proc/$(cat "$pidf")/status" ]; then
spid="$(cat "$pidf")"
( while [ -r "/proc/$spid/status" ]; do
echo "$(date +%T) $(awk '/VmRSS/ {printf "%.2f GB", $2 / 1048576}' "/proc/$spid/status")" >> "$out/rss.log"
sleep 30
done ) &
sampler=$!
trap 'kill "$sampler" 2>/dev/null || true' EXIT
fi
for r in "${runs[@]}"; do
wt="$out/wt-$r"
git -C "$repo" worktree add -q -b "cmp-$r-$stamp" "$wt" "$rev"
case "$r" in
opencode) cmd=("${QWEN_COMPARE_OPENCODE:-$HERE/qwen-opencode.sh}" run --auto --format json "$text") ;;
qwen-claude) cmd=("${QWEN_COMPARE_QWEN_CLAUDE:-$HERE/qwen-claude.sh}" -p "$text" "${claude_flags[@]}") ;;
hosted) cmd=(env -u ANTHROPIC_BASE_URL -u ANTHROPIC_AUTH_TOKEN -u ANTHROPIC_MODEL -u CLAUDE_CODE_MAX_CONTEXT_TOKENS
CLAUDE_CONFIG_DIR="${QWEN_HOSTED_CONFIG_DIR:-$HOME/.claude-personal}"
"${QWEN_CLAUDE_BIN:-$(command -v claude)}" --model "$model" -p "$text" "${claude_flags[@]}") ;;
esac
before=0
[ ! -f "$log" ] || before="$(wc -l < "$log")"
t0="$(date +%s)"
rc=0
(cd "$wt" && timeout "${QWEN_COMPARE_TIMEOUT:-1800}" "${cmd[@]}") > "$out/$r.out" 2> "$out/$r.err" < /dev/null || rc=$?
printf '%s\t%s\t%s\n' "$r" "$rc" "$(($(date +%s) - t0))" | tee -a "$out/summary.tsv"
[ ! -f "$log" ] || tail -n +"$((before + 1))" "$log" > "$out/$r.server.log"
git -C "$wt" add -A
git -C "$wt" diff --cached > "$out/$r.patch"
done
echo "Results in $out. Review the worktrees, then remove them:"
for r in "${runs[@]}"; do
echo " git -C $repo worktree remove --force $out/wt-$r && git -C $repo branch -D cmp-$r-$stamp"
done
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# Shared defaults and helpers for the qwen-* scripts, which run Qwen 3.8 27B on the local GPU
# (docs/5-research/2026-10-19-local-qwen-design.md). Source this file; do not run it.
# The caller's values win over these defaults:
# QWEN_HOME model files, the llama.cpp build, logs and run state ($HOME/.local/share/qwen-local)
# QWEN_PORT the server's port on 127.0.0.1 (8088)
# QWEN_CTX the context size in tokens (102400)
# QWEN_ALIAS the model name the server answers to (qwen3.8-27b)
# QWEN_STATE, written by `qwen-local.sh start` and removed by `stop`, holds the running server's
# QWEN_PORT and QWEN_CTX, so that the launchers use the values the server really has.
QWEN_HOME="${QWEN_HOME:-$HOME/.local/share/qwen-local}"
QWEN_PORT="${QWEN_PORT:-8088}"
QWEN_CTX="${QWEN_CTX:-102400}"
QWEN_ALIAS="${QWEN_ALIAS:-qwen3.8-27b}"
QWEN_STATE="$QWEN_HOME/run/server.env"
# qwen_load_state: take QWEN_PORT and QWEN_CTX from the running server's state file, if there is one.
# The file is read as data, never sourced.
qwen_load_state() {
local v
[ -f "$QWEN_STATE" ] || return 0
v="$(sed -n 's/^QWEN_PORT=\([0-9][0-9]*\)$/\1/p' "$QWEN_STATE" | head -n 1)"
[ -z "$v" ] || QWEN_PORT="$v"
v="$(sed -n 's/^QWEN_CTX=\([0-9][0-9]*\)$/\1/p' "$QWEN_STATE" | head -n 1)"
[ -z "$v" ] || QWEN_CTX="$v"
}
qwen_url() { echo "http://127.0.0.1:$QWEN_PORT"; }
qwen_healthy() { curl -fsS -m 2 "$(qwen_url)/health" >/dev/null 2>&1; }
qwen_require_server() {
qwen_load_state
if ! qwen_healthy; then
echo "the local model server is not running on $(qwen_url); start it with scripts/qwen-local.sh start" >&2
exit 2
fi
}
+147
View File
@@ -0,0 +1,147 @@
#!/usr/bin/env bash
# Start, stop and check llama-server serving Qwen 3.8 27B on 127.0.0.1, all of it on the GPU
# (docs/5-research/2026-10-19-local-qwen-design.md; setup in docs/5-research/2026-10-19-local-qwen-notes.md).
# qwen-local.sh start start the server in the background; wait until it answers /health
# qwen-local.sh stop stop it (TERM, then KILL after 30 seconds)
# qwen-local.sh status exit 0 and print the URL when it runs, exit 3 when it does not
# qwen-local.sh cmd print the server command line, and run nothing
# qwen-local.sh template save the running model's chat template, changed so that a system message
# after the first one is rendered instead of refused (Claude Code sends one,
# for hook context); start uses the saved file from then on
# Environment, besides qwen-lib.sh's QWEN_HOME, QWEN_PORT, QWEN_CTX and QWEN_ALIAS:
# QWEN_LLAMA_SERVER the binary ($QWEN_HOME/llama.cpp/build/bin/llama-server)
# QWEN_MODEL the GGUF file ($QWEN_HOME/models/model.gguf)
# QWEN_EFFORT reasoning effort: low (default), medium, xhigh, or off (no thinking)
# QWEN_MTP_ARGS the multi-token prediction flags, split on spaces; empty turns MTP off. The default
# drafts with $QWEN_HOME/models/mtp.gguf, all on the GPU
# QWEN_CACHE_RAM the most MiB of system RAM for llama-server's prompt cache (2048). Its own default,
# 8192, grew past WSL's 12 GB memory cap in an agent session, and the kernel's
# out-of-memory killer stopped the server
# QWEN_CTX_CHECKPOINTS context checkpoints kept per slot (4). This hybrid model saves its recurrent
# state in system RAM at each checkpoint; the default, 32, grew the server by about
# 4 GB over a long session. Fewer means more prompt re-reading after a rewind
# QWEN_START_TIMEOUT seconds to wait for /health (300: loading 17 GB takes a while)
# QWEN_CHAT_TEMPLATE the saved chat template ($QWEN_HOME/models/chat-template.jinja), used when it exists
# The log is $QWEN_HOME/logs/server.log; the previous one is kept as server.log.1.
set -euo pipefail
# shellcheck source-path=SCRIPTDIR source=qwen-lib.sh
. "$(dirname "${BASH_SOURCE[0]}")/qwen-lib.sh"
usage() { echo "usage: qwen-local.sh start|stop|status|cmd|template" >&2; exit 2; }
[ $# -eq 1 ] || usage
bin="${QWEN_LLAMA_SERVER:-$QWEN_HOME/llama.cpp/build/bin/llama-server}"
model="${QWEN_MODEL:-$QWEN_HOME/models/model.gguf}"
effort="${QWEN_EFFORT:-low}"
mtp="${QWEN_MTP_ARGS---spec-type draft-mtp -md $QWEN_HOME/models/mtp.gguf -ngld all -ctkd q8_0 -ctvd q8_0}"
timeout="${QWEN_START_TIMEOUT:-300}"
tplf="${QWEN_CHAT_TEMPLATE:-$QWEN_HOME/models/chat-template.jinja}"
pidf="$QWEN_HOME/run/server.pid"
log="$QWEN_HOME/logs/server.log"
case "$effort" in
off) reasoning=(--reasoning off) ;;
low | medium | xhigh) reasoning=(--reasoning-effort "$effort") ;;
*) echo "QWEN_EFFORT is '$effort'; use one of low, medium, xhigh, off" >&2; exit 2 ;;
esac
args=(-m "$model" --alias "$QWEN_ALIAS" --host 127.0.0.1 --port "$QWEN_PORT" -ngl 999 -c "$QWEN_CTX"
-fa on --cache-type-k q8_0 --cache-type-v q8_0 --cache-ram "${QWEN_CACHE_RAM:-2048}"
--ctx-checkpoints "${QWEN_CTX_CHECKPOINTS:-4}" --parallel 1 --jinja
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 "${reasoning[@]}")
[ ! -f "$tplf" ] || args+=(--chat-template-file "$tplf")
# shellcheck disable=SC2206 # QWEN_MTP_ARGS is a list of flags, split on spaces on purpose
[ -z "$mtp" ] || args+=($mtp)
# running: print the pid of our llama-server, if the pid file names one that is alive. A pid file
# whose process is gone, or is some other program (the pid was reused), is stale and is removed.
running() {
local pid
[ -f "$pidf" ] || return 1
pid="$(cat "$pidf")"
if [[ "$pid" =~ ^[0-9]+$ ]] && kill -0 "$pid" 2>/dev/null && tr '\0' ' ' < "/proc/$pid/cmdline" 2>/dev/null | grep -q llama-server; then
echo "$pid"
return 0
fi
echo "removing a stale pid file ($pidf: $pid)" >&2
rm -f "$pidf" "$QWEN_STATE"
return 1
}
quote() {
local a out=()
for a in "$@"; do
if [[ "$a" =~ ^[A-Za-z0-9_./:=+,-]+$ ]]; then out+=("$a"); else out+=("'$a'"); fi
done
echo "${out[*]}"
}
stop_pid() {
local pid="$1" i
kill "$pid" 2>/dev/null || return 0
for ((i = 0; i < 30; i++)); do
kill -0 "$pid" 2>/dev/null || return 0
sleep 1
done
kill -KILL "$pid" 2>/dev/null || true
}
# The model's template refuses a system message after the first. This renders it as a system turn.
fix_template='
import json, sys
t = json.load(sys.stdin)["chat_template"]
old = "{{- raise_exception(\x27System message must be at the beginning.\x27) }}"
if t.count(old) != 1:
sys.exit("did not find the line that refuses later system messages in the chat template; check the template by hand")
open(sys.argv[1], "w").write(t.replace(old, "{{- \x27<|im_start|>system\\n\x27 + content + \x27<|im_end|>\\n\x27 }}"))
'
case "$1" in
cmd) quote "$bin" "${args[@]}" ;;
template)
qwen_require_server
mkdir -p "$(dirname "$tplf")"
if ! curl -fsS -m 10 "$(qwen_url)/props" | python3 -I -c "$fix_template" "$tplf.tmp"; then rm -f "$tplf.tmp"; exit 1; fi
mv -f "$tplf.tmp" "$tplf"
echo "saved $tplf; restart the server to use it: qwen-local.sh stop && qwen-local.sh start"
;;
status)
qwen_load_state
if running >/dev/null; then echo "running on $(qwen_url)"; else echo "not running"; exit 3; fi
;;
stop)
if pid="$(running)"; then
stop_pid "$pid"
rm -f "$pidf" "$QWEN_STATE"
echo "stopped"
else
echo "not running"
fi
;;
start)
if running >/dev/null; then qwen_load_state; echo "already running on $(qwen_url)"; exit 0; fi
[ -f "$model" ] || { echo "the model $model does not exist; see the setup in docs/5-research/2026-10-19-local-qwen-notes.md" >&2; exit 2; }
[ -x "$bin" ] || { echo "llama-server $bin does not exist or is not executable" >&2; exit 2; }
if qwen_healthy; then echo "port $QWEN_PORT already answers on $(qwen_url), and it is not our server; stop that program or set QWEN_PORT" >&2; exit 2; fi
mkdir -p "$QWEN_HOME/run" "$QWEN_HOME/logs"
[ ! -f "$log" ] || mv -f "$log" "$log.1"
nohup "$bin" "${args[@]}" >"$log" 2>&1 &
pid=$!
echo "$pid" > "$pidf"
printf 'QWEN_PORT=%s\nQWEN_CTX=%s\n' "$QWEN_PORT" "$QWEN_CTX" > "$QWEN_STATE"
for ((i = 0; i < timeout; i++)); do
if ! kill -0 "$pid" 2>/dev/null; then
rm -f "$pidf" "$QWEN_STATE"
echo "llama-server exited while starting; the end of $log:" >&2
tail -n 20 "$log" >&2
exit 1
fi
if qwen_healthy; then echo "ready on $(qwen_url) (pid $pid, log $log)"; exit 0; fi
sleep 1
done
stop_pid "$pid"
rm -f "$pidf" "$QWEN_STATE"
echo "llama-server did not become ready within $timeout seconds; stopped it. The end of $log:" >&2
tail -n 20 "$log" >&2
exit 1
;;
*) usage ;;
esac
+124
View File
@@ -0,0 +1,124 @@
#!/usr/bin/env python3
# Measure the local Qwen server (docs/5-research/2026-10-19-local-qwen-notes.md, "Measurements"). Every
# figure is the server's own: the timings in each /v1/chat/completions answer. Run it as `python3 -I`.
# qwen-measure.py speed [--runs N] [--max-tokens M] PROMPT_FILE
# N requests (3) with the prompt; one line per run, then the median decode and prompt speeds
# qwen-measure.py prompt --tokens N [--question Q] --out FILE GLOB...
# write a prompt of about N tokens, counted by the server's /tokenize: the files the globs match
# (** allowed), each headed "// FILE <path>", repeated as needed, then the question
# qwen-measure.py growth [--turns N] [--chunk C] --pid PID SOURCE_FILE
# a conversation that grows by C characters (11000, about 3K tokens) of SOURCE_FILE per turn, for N
# turns (20); after each, the context size and the resident memory of process PID (llama-server)
# QWEN_URL is the server (http://127.0.0.1:8088).
import argparse
import glob
import json
import os
import statistics
import sys
import time
import urllib.error
import urllib.request
URL = os.environ.get("QWEN_URL", "http://127.0.0.1:8088")
def post(path, body, timeout=1800):
req = urllib.request.Request(URL + path, json.dumps(body).encode(), {"content-type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=timeout) as r:
return json.load(r)
except (urllib.error.URLError, ConnectionError) as e:
sys.exit(f"the local model server does not answer on {URL} ({e}); start it with scripts/qwen-local.sh start")
def ntokens(text):
return len(post("/tokenize", {"content": text})["tokens"])
def speed(a):
prompt = open(a.prompt_file).read()
rows = []
for i in range(a.runs):
t0 = time.time()
r = post("/v1/chat/completions", {"model": "q", "max_tokens": a.max_tokens,
"messages": [{"role": "user", "content": prompt}]})
t, m = r["timings"], r["choices"][0]["message"]
rows.append(t)
print(f"run {i + 1}: prompt {t['prompt_n']} tokens at {t['prompt_per_second']:.0f} tokens/s"
f" | decode {t['predicted_per_second']:.1f} tokens/s ({t['predicted_n']} tokens)"
f" | draft {t.get('draft_n_accepted', '-')}/{t.get('draft_n', '-')}"
f" | reasoning {len(m.get('reasoning_content') or '')} chars | wall {time.time() - t0:.1f}s", flush=True)
print(f"median decode {statistics.median(t['predicted_per_second'] for t in rows):.1f} tokens/s,"
f" median prompt {statistics.median(t['prompt_per_second'] for t in rows):.0f} tokens/s")
def prompt(a):
paths = sorted({p for g in a.globs for p in glob.glob(g, recursive=True) if os.path.isfile(p)})
if not paths:
print(f"no files match {' '.join(a.globs)}", file=sys.stderr)
sys.exit(2)
src = "".join(f"// FILE {p}\n" + open(p, errors="replace").read() + "\n" for p in paths)
per = max(1, ntokens(src))
body = src * (a.tokens // per + 1)
body = body[: int(a.tokens * len(src) / per)]
text = body + f"\n\nQuestion: {a.question}"
with open(a.out, "w") as f:
f.write(text)
print(f"wrote {a.out}: {ntokens(text)} tokens, from {len(paths)} files")
def rss_gb(pid):
with open(f"/proc/{pid}/status") as f:
for line in f:
if line.startswith("VmRSS"):
return int(line.split()[1]) / 1048576
return 0.0
def growth(a):
if not os.path.exists(f"/proc/{a.pid}/status"):
print(f"no process {a.pid}; pass the pid of llama-server ($QWEN_HOME/run/server.pid)", file=sys.stderr)
sys.exit(2)
src = open(a.source_file).read()
msgs = []
for i in range(a.turns):
chunk = src[(i * a.chunk) % max(1, len(src)):][: a.chunk]
msgs.append({"role": "user", "content": f"Turn {i}. Here is more code:\n{chunk}\nReply with one word."})
r = post("/v1/chat/completions", {"model": "q", "max_tokens": 30, "messages": msgs}, timeout=600)
msgs.append({"role": "assistant", "content": r["choices"][0]["message"].get("content") or "ok"})
print(f"turn {i:2d} ctx {r['usage']['prompt_tokens']:6d} rss {rss_gb(a.pid):.2f} GB", flush=True)
def main():
p = argparse.ArgumentParser(prog="qwen-measure.py")
sub = p.add_subparsers(dest="cmd", required=True)
s = sub.add_parser("speed")
s.add_argument("--runs", type=int, default=3)
s.add_argument("--max-tokens", type=int, default=512)
s.add_argument("prompt_file")
s.set_defaults(fn=speed)
s = sub.add_parser("prompt")
s.add_argument("--tokens", type=int, required=True)
s.add_argument("--question", default="Which package in the code above does the most work? Answer in two sentences.")
s.add_argument("--out", required=True)
s.add_argument("globs", nargs="+")
s.set_defaults(fn=prompt)
s = sub.add_parser("growth")
s.add_argument("--turns", type=int, default=20)
s.add_argument("--chunk", type=int, default=11000)
s.add_argument("--pid", required=True)
s.add_argument("source_file")
s.set_defaults(fn=growth)
a = p.parse_args()
try:
a.fn(a)
except SystemExit as e:
if isinstance(e.code, str):
print(e.code, file=sys.stderr)
sys.exit(2)
raise
if __name__ == "__main__":
main()
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
# Run OpenCode against the local Qwen 3.8 27B server, and nothing else
# (docs/5-research/2026-10-19-local-qwen-design.md). Start the server first: qwen-local.sh start.
# qwen-opencode.sh [OPENCODE ARGS...] for example: qwen-opencode.sh, or qwen-opencode.sh run "..."
# The configuration goes in through OPENCODE_CONFIG_CONTENT, with the running server's port and
# context. Only the local provider is enabled, and the cloud credentials of the caller's shell are
# removed. OpenCode does not read ~/.claude (the work account's Claude Code config) or outside
# skills, and does not update, share sessions or fetch the models.dev catalogue. The project's own
# OpenCode config still applies. QWEN_OPENCODE_BIN overrides the binary (opencode).
# OpenCode 2 runs sessions in a background service that keeps the config and environment it started
# with, so this script always passes --standalone (after `run`, or first for the interactive session):
# a private server that gets exactly this config.
set -euo pipefail
# shellcheck source-path=SCRIPTDIR source=qwen-lib.sh
. "$(dirname "${BASH_SOURCE[0]}")/qwen-lib.sh"
qwen_require_server
# OpenCode 2 still reads these keys (provider, enabled_providers, small_model, autoupdate), though its
# schema marks them deprecated. Gleipnir's adapter uses the same ones.
# shellcheck disable=SC2016 # "$schema" is a JSON key, not a variable
conf="$(printf '{"$schema":"https://opencode.ai/config.json","autoupdate":false,"share":"disabled","enabled_providers":["qwen-local"],"provider":{"qwen-local":{"npm":"@ai-sdk/openai-compatible","name":"Qwen local","options":{"baseURL":"%s/v1"},"models":{"%s":{"name":"Qwen 3.8 27B (local)","tool_call":true,"reasoning":true,"limit":{"context":%s,"output":32768}}}}},"model":"qwen-local/%s","small_model":"qwen-local/%s"}' \
"$(qwen_url)" "$QWEN_ALIAS" "$QWEN_CTX" "$QWEN_ALIAS" "$QWEN_ALIAS")"
if [ "${1:-}" = run ]; then shift; set -- run --standalone "$@"; else set -- --standalone "$@"; fi
exec env -u ANTHROPIC_API_KEY -u CLAUDE_CODE_OAUTH_TOKEN -u OPENROUTER_API_KEY \
OPENCODE_CONFIG_CONTENT="$conf" \
OPENCODE_DISABLE_CLAUDE_CODE=1 OPENCODE_DISABLE_EXTERNAL_SKILLS=1 \
OPENCODE_DISABLE_AUTOUPDATE=true OPENCODE_DISABLE_SHARE=true OPENCODE_DISABLE_MODELS_FETCH=true \
"${QWEN_OPENCODE_BIN:-opencode}" "$@"
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# Each check is a condition string run by eval: the helpers are read there, so
# the linter cannot see them used.
# shellcheck disable=SC2016,SC2034,SC2329
# Tests for qwen-capture.py. scripts/tests/qwen-capture.test.sh
# Starts the capture server on a free port, sends it requests with curl, and reads what it saved.
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
S="$HERE/.."
T="$(mktemp -d)"
cleanup() { [ -z "${CAP:-}" ] || kill "$CAP" 2>/dev/null; rm -rf -- "$T"; }
trap cleanup EXIT
fail=0
check() {
local d="$1"; shift
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; fi
}
port="$(python3 -I -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1])')"
python3 -I "$S/qwen-capture.py" "$port" "$T/cap" > "$T/log" 2>&1 & CAP=$!
for _ in $(seq 50); do curl -fsS -m 1 "http://127.0.0.1:$port/health" >/dev/null 2>&1 && break; sleep 0.1; done
curl -sS -m 5 -o "$T/r1" -w '%{http_code}' -H 'content-type: application/json' -d '{"model":"m","messages":[{"role":"system","content":"s"}]}' \
"http://127.0.0.1:$port/v1/messages?beta=true" > "$T/code1"
curl -sS -m 5 -o /dev/null -H 'content-type: application/json' -d '{"model":"m","n":2}' "http://127.0.0.1:$port/v1/chat/completions"
check "the first request is saved by number and path, as sent" \
eval '[ -f "$T/cap/00-v1_messages.json" ] && python3 -I -c "import json,sys; assert json.load(open(sys.argv[1]))[\"messages\"][0][\"role\"] == \"system\"" "$T/cap/00-v1_messages.json"'
check "the second request gets the next number" eval '[ -f "$T/cap/01-v1_chat_completions.json" ]'
check "each request is answered with an API error, so the tool stops instead of waiting" \
eval '[ "$(cat "$T/code1")" = 500 ] && grep -q "\"type\": *\"error\"" "$T/r1"'
check "each saved request is announced on stdout" eval 'grep -q "00-v1_messages.json" "$T/log" && grep -q "01-v1_chat_completions.json" "$T/log"'
check "it listens on 127.0.0.1 only" eval '! curl -fsS -m 1 "http://$(hostname -I | cut -d" " -f1):$port/health" >/dev/null 2>&1'
exit $fail
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env bash
# Each check is a condition string run by eval: $rc and the helpers are read there, so
# the linter cannot see them used.
# shellcheck disable=SC2016,SC2034,SC2329
# Tests for qwen-claude.sh. scripts/tests/qwen-claude.test.sh
# A fake `claude` and a fake `curl` stand in for the real tools: no server, no account, no network.
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
S="$HERE/.."
T="$(mktemp -d)"
trap 'rm -rf -- "$T"' EXIT
fail=0
check() {
local d="$1"; shift
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
}
run() { rm -f "$T/calls" "$T/env"; "$@" >"$T/out" 2>&1; rc=$?; }
has() { grep -q -F -- "$1" "$T/out"; }
called() { grep -qx -F -- "$1" "$T/calls"; }
envhas() { grep -qx -F -- "$1" "$T/env"; }
# js FILE EXPR: evaluate a python expression over the JSON file (as c).
js() { python3 -I -c 'import json,sys; c=json.load(open(sys.argv[1])); sys.exit(0 if eval(sys.argv[2]) else 1)' "$1" "$2"; }
sums() { find "$HOME/.claude-personal" "$HOME/.claude" "$HOME/.claude.json" -type f -exec sha256sum {} + | sort; }
export HOME="$T/home"
P="$HOME/.claude-personal" Q="$HOME/.claude-qwen"
mkdir -p "$P/skills/a" "$P/hooks" "$HOME/.claude"
echo '{"model":"opus"}' > "$P/settings.json"
echo 'skill a' > "$P/skills/a/SKILL.md"
printf '#!/bin/sh\nexit 0\n' > "$P/hooks/merge-requires-green.sh"
echo '{"model":"work"}' > "$HOME/.claude/settings.json"
echo '{"work":true}' > "$HOME/.claude.json"
before="$(sums)"
# The fake claude records each call as one line, "<CLAUDE_CONFIG_DIR> <args...>", and its environment.
mkdir -p "$T/bin"
cat > "$T/bin/claude" <<EOF2
#!/usr/bin/env bash
echo "\${CLAUDE_CONFIG_DIR:-none} \$*" >> "$T/calls"
env > "$T/env"
EOF2
cat > "$T/bin/curl" <<EOF2
#!/usr/bin/env bash
[ -e "$T/healthy" ] || exit 7
echo '{"status":"ok"}'
EOF2
chmod +x "$T/bin/claude" "$T/bin/curl"
export PATH="$T/bin:$PATH" QWEN_HOME="$T/qh" QWEN_CLAUDE_BIN="$T/bin/claude"
C="$S/qwen-claude.sh"
run bash "$C" -p hi
check "run before init: refuses and says to init first; claude does not run" \
eval '[ $rc = 2 ] && has "qwen-claude.sh init" && [ ! -e "$T/calls" ]'
QWEN_CLAUDE_DIR="$P" run bash "$C" init
check "init refuses the personal config dir, and writes nothing" \
eval '[ $rc = 2 ] && [ ! -e "$T/calls" ] && [ "$(sums)" = "$before" ]'
QWEN_CLAUDE_DIR="$HOME/.claude/sub" run bash "$C" init
check "init refuses a dir inside the work config dir" eval '[ $rc = 2 ] && [ ! -e "$HOME/.claude/sub" ]'
QWEN_CLAUDE_DIR="$HOME" run bash "$C" init
check "init refuses \$HOME, which holds both config dirs and ~/.claude.json" eval '[ $rc = 2 ] && [ ! -e "$T/calls" ] && [ ! -e "$HOME/settings.json" ]'
QWEN_PERSONAL_DIR="$T/elsewhere" QWEN_CLAUDE_DIR="$P" run bash "$C" init
check "init refuses ~/.claude-personal even when QWEN_PERSONAL_DIR points elsewhere" eval '[ $rc = 2 ] && [ ! -e "$T/calls" ]'
run bash "$C" init
check "init writes settings: the three plugins, ask-before-acting mode, the personal merge hook" \
eval '[ $rc = 0 ] && js "$Q/settings.json" "c[\"permissions\"][\"defaultMode\"] == \"default\" and sorted(c[\"enabledPlugins\"]) == [\"gopls-lsp@claude-plugins-official\", \"skill-creator@claude-plugins-official\", \"superpowers@claude-plugins-official\"] and c[\"hooks\"][\"PreToolUse\"][0][\"hooks\"][0][\"command\"] == \"$P/hooks/merge-requires-green.sh\" and c[\"model\"] == \"qwen3.8-27b\""'
check "init marks onboarding done and links the personal skills" \
eval 'js "$Q/.claude.json" "c[\"hasCompletedOnboarding\"] is True" && [ "$(readlink "$Q/skills")" = "$P/skills" ]'
check "init adds the marketplace and installs the three plugins, all into the local config dir" \
eval 'called "$Q plugin marketplace add anthropics/claude-plugins-official" && called "$Q plugin install superpowers@claude-plugins-official" && called "$Q plugin install gopls-lsp@claude-plugins-official" && called "$Q plugin install skill-creator@claude-plugins-official"'
echo '{"hasCompletedOnboarding":true,"mine":1}' > "$Q/.claude.json"
run bash "$C" init
check "init again keeps an existing .claude.json and the skills link" \
eval '[ $rc = 0 ] && js "$Q/.claude.json" "c.get(\"mine\") == 1" && [ "$(readlink "$Q/skills")" = "$P/skills" ]'
run bash "$C" -p hi
check "run with the server down: refuses with the start hint; claude does not run" \
eval '[ $rc = 2 ] && has "qwen-local.sh start" && [ ! -e "$T/calls" ]'
touch "$T/healthy"
QWEN_CLAUDE_DIR="$HOME/.claude" run bash "$C" -p hi
check "run refuses the work config dir, though it has a settings.json" eval '[ $rc = 2 ] && [ ! -e "$T/calls" ]'
QWEN_CLAUDE_DIR="$P" run bash "$C" -p hi
check "run refuses the personal config dir" eval '[ $rc = 2 ] && [ ! -e "$T/calls" ]'
CLAUDE_CODE_USE_BEDROCK=1 CLAUDE_CODE_USE_VERTEX=1 CLAUDE_CODE_USE_FOUNDRY=1 ANTHROPIC_BEDROCK_BASE_URL=x ANTHROPIC_VERTEX_BASE_URL=y \
ANTHROPIC_API_KEY=a CLAUDE_CODE_OAUTH_TOKEN=b OPENROUTER_API_KEY=c run bash "$C" -p "hi there"
check "run passes the arguments and points Claude Code at the local server and config dir" \
eval '[ $rc = 0 ] && called "$Q -p hi there" && envhas "ANTHROPIC_BASE_URL=http://127.0.0.1:8088" && envhas "CLAUDE_CONFIG_DIR=$Q" && envhas ANTHROPIC_MODEL=qwen3.8-27b && envhas ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen3.8-27b && envhas CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 && envhas DISABLE_AUTOUPDATER=1 && envhas CLAUDE_CODE_MAX_CONTEXT_TOKENS=102400'
check "no cloud credential from the caller's shell reaches claude" \
eval '[ -e "$T/env" ] && ! grep -Eq "^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN|OPENROUTER_API_KEY)=" "$T/env"'
check "no Bedrock, Vertex or Foundry switch reaches claude, so it cannot bypass the local base URL" \
eval '[ -e "$T/env" ] && ! grep -Eq "^(CLAUDE_CODE_USE_BEDROCK|CLAUDE_CODE_USE_VERTEX|CLAUDE_CODE_USE_FOUNDRY|ANTHROPIC_BEDROCK_BASE_URL|ANTHROPIC_VERTEX_BASE_URL)=" "$T/env"'
check "nothing under ~/.claude-personal or ~/.claude changed" eval '[ "$(sums)" = "$before" ]'
exit $fail
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env bash
# Each check is a condition string run by eval: $rc and the helpers are read there, so
# the linter cannot see them used.
# shellcheck disable=SC2016,SC2034,SC2329
# Tests for qwen-compare.sh. scripts/tests/qwen-compare.test.sh
# Fakes stand in for qwen-opencode.sh, qwen-claude.sh and claude: each records its arguments, folder and
# environment, appends a line to a fake server log, and changes a file. A temp git repo is the project.
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
S="$HERE/.."
T="$(mktemp -d)"
trap 'rm -rf -- "$T"' EXIT
fail=0
check() {
local d="$1"; shift
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
}
run() { "$@" >"$T/out" 2>&1; rc=$?; }
has() { grep -q -F -- "$1" "$T/out"; }
export HOME="$T/home"; mkdir -p "$HOME/.claude-personal"
R="$T/repo"
git init -q -b main "$R"
git -C "$R" config user.email [email protected]; git -C "$R" config user.name t
echo base > "$R/a.txt"; git -C "$R" add a.txt; git -C "$R" commit -q -m base
BASE="$(git -C "$R" rev-parse HEAD)"
echo later > "$R/a.txt"; git -C "$R" commit -q -am later
export QWEN_HOME="$T/qh"; mkdir -p "$QWEN_HOME/logs"; echo "old line" > "$QWEN_HOME/logs/server.log"
mkdir -p "$T/bin"
for f in opencode qwen-claude claude; do
cat > "$T/bin/$f" <<EOF2
#!/usr/bin/env bash
printf '%s\n' "\$@" > "$T/$f.args"
pwd > "$T/$f.pwd"
env > "$T/$f.env"
echo "request from $f" >> "$QWEN_HOME/logs/server.log"
echo "$f was here" > "$f.txt"
echo "$f output"
exit "\${FAKE_RC_$(echo "$f" | tr -- '-' '_'):-0}"
EOF2
chmod +x "$T/bin/$f"
done
export QWEN_COMPARE_OPENCODE="$T/bin/opencode" QWEN_COMPARE_QWEN_CLAUDE="$T/bin/qwen-claude" QWEN_CLAUDE_BIN="$T/bin/claude"
echo "Fix the bug. Do not ask questions." > "$T/prompt.txt"
X="$S/qwen-compare.sh"
run bash "$X" --repo "$R" --commit "$BASE"
check "missing --prompt or --out is a usage error" eval '[ $rc = 2 ] && has usage'
mkdir -p "$T/full"; touch "$T/full/x"
run bash "$X" --repo "$R" --commit "$BASE" --prompt "$T/prompt.txt" --out "$T/full"
check "an --out folder that is not empty is refused" eval '[ $rc = 2 ] && has "not empty"'
O="$T/cmp"
ANTHROPIC_BASE_URL=http://127.0.0.1:8088 FAKE_RC_qwen_claude=3 run bash "$X" --repo "$R" --commit "$BASE" --prompt "$T/prompt.txt" --out "$O"
check "all three runs happen, in their own worktrees at the asked commit" \
eval '[ $rc = 0 ] && for t in opencode qwen-claude hosted; do [ "$(git -C "$O/wt-$t" rev-parse HEAD)" = "$BASE" ] || exit 1; done'
check "each tool runs in its own worktree" \
eval '[ "$(cat "$T/opencode.pwd")" = "$O/wt-opencode" ] && [ "$(cat "$T/qwen-claude.pwd")" = "$O/wt-qwen-claude" ] && [ "$(cat "$T/claude.pwd")" = "$O/wt-hosted" ]'
check "OpenCode gets run --auto --format json and the prompt" \
eval '[ "$(head -n 4 "$T/opencode.args")" = "$(printf "run\n--auto\n--format\njson")" ] && grep -qxF "Fix the bug. Do not ask questions." "$T/opencode.args"'
check "local Claude Code gets the prompt, acceptEdits and the default Go allowlist, with JSON output" \
eval 'grep -qx -- -p "$T/qwen-claude.args" && grep -qx acceptEdits "$T/qwen-claude.args" && grep -qxF "Bash(go *)" "$T/qwen-claude.args" && grep -qx json "$T/qwen-claude.args"'
check "hosted Claude Code uses the personal config dir, Sonnet by default, and never the local base URL" \
eval 'grep -qx "CLAUDE_CONFIG_DIR=$HOME/.claude-personal" "$T/claude.env" && grep -qx sonnet "$T/claude.args" && ! grep -q "^ANTHROPIC_BASE_URL=" "$T/claude.env"'
check "the summary has each run's exit code and wall time; a failed run does not stop the others" \
eval 'grep -qP "^opencode\t0\t[0-9]+$" "$O/summary.tsv" && grep -qP "^qwen-claude\t3\t[0-9]+$" "$O/summary.tsv" && grep -qP "^hosted\t0\t[0-9]+$" "$O/summary.tsv"'
check "each run's output and diff (new files included) are saved" \
eval 'grep -q "opencode output" "$O/opencode.out" && grep -q "+opencode was here" "$O/opencode.patch" && grep -q "+claude was here" "$O/hosted.patch"'
check "each run gets only its own slice of the server log" \
eval '[ "$(cat "$O/opencode.server.log")" = "request from opencode" ] && [ "$(cat "$O/qwen-claude.server.log")" = "request from qwen-claude" ]'
check "the cleanup commands are printed, and the worktrees are left for review" \
eval 'has "worktree remove --force $O/wt-opencode" && has "branch -D" && [ -d "$O/wt-hosted" ]'
run bash "$X" --repo "$R" --commit "$BASE" --prompt "$T/prompt.txt" --out "$T/only" --only hosted --hosted-model opus \
--allow "Bash(npm test*)"
check "--only, --hosted-model and --allow choose the runs, the model and the allowlist" \
eval '[ $rc = 0 ] && [ -d "$T/only/wt-hosted" ] && [ ! -e "$T/only/wt-opencode" ] && grep -qx opus "$T/claude.args" && grep -qxF "Bash(npm test*)" "$T/claude.args" && ! grep -qxF "Bash(go *)" "$T/claude.args"'
run bash "$X" --repo "$R" --commit "$BASE" --prompt "$T/prompt.txt" --out "$T/bad" --only gemini
check "an unknown run name is a usage error" eval '[ $rc = 2 ] && has gemini && [ ! -e "$T/bad/wt-gemini" ]'
exit $fail
+153
View File
@@ -0,0 +1,153 @@
#!/usr/bin/env bash
# Each check is a condition string run by eval: $rc, $took and the helpers are read there, so
# the linter cannot see them used.
# shellcheck disable=SC2016,SC2034,SC2329
# Tests for qwen-local.sh and qwen-lib.sh. scripts/tests/qwen-local.test.sh
# A fake `llama-server` and a fake `curl` stand in for the real tools: no GPU, no model, no network.
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
S="$HERE/.."
T="$(mktemp -d)"
cleanup() { pkill -f "$T/bin/llama-server" 2>/dev/null; [ -z "${SLEEPER:-}" ] || kill "$SLEEPER" 2>/dev/null; rm -rf -- "$T"; }
trap cleanup EXIT
fail=0
check() {
local d="$1"; shift
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
}
run() { "$@" >"$T/out" 2>&1; rc=$?; }
has() { grep -q -F -- "$1" "$T/out"; }
export HOME="$T/home"; mkdir -p "$HOME"
# The fake server writes its arguments, one per line, to $T/args, and marks itself healthy (for the fake
# curl) unless FAKE_NEVER_READY is set. FAKE_DIE makes it fail at once, as a model too big for VRAM does.
mkdir -p "$T/bin"
cat > "$T/bin/llama-server" <<EOF2
#!/usr/bin/env bash
printf '%s\n' "\$@" > "$T/args"
echo "fake llama-server loading"
if [ -n "\${FAKE_DIE:-}" ]; then echo "fake: out of memory"; exit 1; fi
trap 'rm -f "$T/healthy"; kill \$! 2>/dev/null; exit 0' TERM
[ -n "\${FAKE_NEVER_READY:-}" ] || touch "$T/healthy"
sleep 300 & wait
EOF2
# The fake curl answers /props from $T/props, and anything else with a health answer.
cat > "$T/bin/curl" <<EOF2
#!/usr/bin/env bash
[ -e "$T/healthy" ] || exit 7
case "\${*: -1}" in */props) cat "$T/props" ;; *) echo '{"status":"ok"}' ;; esac
EOF2
chmod +x "$T/bin/llama-server" "$T/bin/curl"
touch "$T/model.gguf"
export PATH="$T/bin:$PATH" QWEN_HOME="$T/qh" QWEN_LLAMA_SERVER="$T/bin/llama-server" QWEN_MODEL="$T/model.gguf" \
QWEN_START_TIMEOUT=3
L="$S/qwen-local.sh"
pidf="$T/qh/run/server.pid" envf="$T/qh/run/server.env"
nservers() { pgrep -fc "$T/bin/llama-server"; }
# cmd
run bash "$L" cmd
check "cmd: binds 127.0.0.1:8088 with a 102400 context, a q8_0 KV cache, one slot and low effort" \
eval '[ $rc = 0 ] && has "--host 127.0.0.1" && has "--port 8088" && has "-c 102400" && has "--cache-type-k q8_0" && has "--cache-type-v q8_0" && has "--parallel 1" && has "--reasoning-effort low"'
check "cmd: the prompt cache in system RAM is capped at 2048 MiB, so it fits WSL's memory cap" eval 'has "--cache-ram 2048"'
check "cmd: the model's context checkpoints in system RAM are limited to 4" eval 'has "--ctx-checkpoints 4"'
QWEN_CACHE_RAM=0 QWEN_CTX_CHECKPOINTS=8 run bash "$L" cmd
check "cmd: QWEN_CACHE_RAM and QWEN_CTX_CHECKPOINTS set the limits" eval 'has "--cache-ram 0" && has "--ctx-checkpoints 8"'
run bash "$L" cmd
check "cmd: MTP is on by default, with the draft file from QWEN_HOME on the GPU" \
eval 'has "--spec-type draft-mtp -md $T/qh/models/mtp.gguf -ngld all -ctkd q8_0 -ctvd q8_0"'
QWEN_EFFORT=off run bash "$L" cmd
check "cmd: effort off turns thinking off" eval 'has "--reasoning off" && ! has "--reasoning-effort"'
QWEN_EFFORT=high run bash "$L" cmd
check "cmd: an unknown effort is a usage error that names the allowed values" \
eval '[ $rc = 2 ] && has QWEN_EFFORT && has "low, medium, xhigh, off"'
QWEN_MTP_ARGS="--spec-a x" run bash "$L" cmd
check "cmd: QWEN_MTP_ARGS ends the argument list" eval 'grep -Eq -- "--spec-a x ?\$" "$T/out"'
QWEN_MTP_ARGS="" run bash "$L" cmd
check "cmd: an empty QWEN_MTP_ARGS turns MTP off" eval '! has "--spec-a" && ! has "--spec-type"'
tpl="$T/qh/models/chat-template.jinja"
mkdir -p "$T/qh/models"; echo x > "$tpl"
run bash "$L" cmd
check "cmd: a saved chat template is passed to the server" eval 'has "--chat-template-file $tpl"'
rm -f "$tpl"
run bash "$L" cmd
check "cmd: without a saved chat template, the model's own is used" eval '! has "--chat-template-file"'
# template
run bash "$L" template
check "template: needs the server running" eval '[ $rc = 2 ] && has "server is not running" && [ ! -e "$tpl" ]'
touch "$T/healthy"
python3 -I -c 'import json,sys; print(json.dumps({"chat_template": "{%- if a %}\n {{- raise_exception(\x27System message must be at the beginning.\x27) }}\n{%- endif %}"}))' > "$T/props"
run bash "$L" template
check "template: saves the model's template with later system messages rendered as system turns" \
eval '[ $rc = 0 ] && has "$tpl" && ! grep -q raise_exception "$tpl" && grep -qF "<|im_start|>system\\n" "$tpl" && grep -qF " + content + " "$tpl" && grep -qF "{%- if a %}" "$tpl"'
rm -f "$tpl"; echo '{"chat_template": "no such line"}' > "$T/props"
run bash "$L" template
check "template: a template without the expected line is refused, and nothing is saved" \
eval '[ $rc = 1 ] && has "did not find" && [ ! -e "$tpl" ]'
rm -f "$T/healthy" "$T/props"
# start
QWEN_MODEL="$T/missing.gguf" run bash "$L" start
check "start: a missing model fails with its path, and starts nothing" \
eval '[ $rc = 2 ] && has "$T/missing.gguf" && [ ! -e "$T/args" ]'
run bash "$L" start
check "start: waits for health, says where it is ready, and records the pid and the state" \
eval '[ $rc = 0 ] && has "ready on http://127.0.0.1:8088" && kill -0 "$(cat "$pidf")" && grep -qx -- "-m" "$T/args" && grep -qx -- "$T/model.gguf" "$T/args" && grep -qx QWEN_PORT=8088 "$envf" && grep -qx QWEN_CTX=102400 "$envf"'
run bash "$L" start
check "start: a second start while running says so and starts no second server" \
eval '[ $rc = 0 ] && has "already running" && [ "$(nservers)" = 1 ]'
run bash "$L" status
check "status: running exits 0 with the URL" eval '[ $rc = 0 ] && has running && has "http://127.0.0.1:8088"'
run bash "$L" stop
check "stop: the server is gone and the pid and state files are removed" \
eval '[ $rc = 0 ] && [ "$(nservers)" = 0 ] && [ ! -e "$pidf" ] && [ ! -e "$envf" ]'
run bash "$L" stop
check "stop: not running is not an error" eval '[ $rc = 0 ] && has "not running"'
run bash "$L" status
check "status: not running exits 3" eval '[ $rc = 3 ] && has "not running"'
FAKE_DIE=1 run bash "$L" start
check "start: a server that exits while loading fails at once and shows the log tail" \
eval '[ $rc = 1 ] && has "exited while starting" && has "fake: out of memory" && [ ! -e "$pidf" ] && [ ! -e "$envf" ]'
SECONDS=0
FAKE_NEVER_READY=1 run bash "$L" start
took=$SECONDS
check "start: a server that never gets healthy is stopped after the timeout" \
eval '[ $rc = 1 ] && has "did not become ready" && [ "$took" -le 8 ] && [ "$(nservers)" = 0 ] && [ ! -e "$pidf" ]'
rm -f "$T/args"; touch "$T/healthy"
run bash "$L" start
check "start: another program answering on the port is refused, and nothing starts" \
eval '[ $rc = 2 ] && has "port 8088 already answers" && [ ! -e "$T/args" ]'
rm -f "$T/healthy"
# A stale pid file: its process is gone (after a crash or wsl --shutdown).
mkdir -p "$T/qh/run"; sh -c 'exit 0' & dead=$!; wait "$dead"
echo "$dead" > "$pidf"; printf 'QWEN_PORT=8088\nQWEN_CTX=102400\n' > "$envf"
run bash "$L" status
check "status: a stale pid file means not running, and is cleaned up" \
eval '[ $rc = 3 ] && has "not running" && [ ! -e "$pidf" ] && [ ! -e "$envf" ]'
# A reused pid: the pid file names a live process that is not llama-server.
sleep 60 & SLEEPER=$!
echo "$SLEEPER" > "$pidf"
run bash "$L" stop
check "stop: a pid that is not llama-server is never killed; the pid file is removed as stale" \
eval '[ $rc = 0 ] && has stale && kill -0 "$SLEEPER" && [ ! -e "$pidf" ]'
# qwen_load_state
printf 'QWEN_PORT=9000\nQWEN_CTX=65536\n' > "$envf"
run bash -c '. "$1/qwen-lib.sh"; qwen_load_state; qwen_url; echo "ctx=$QWEN_CTX"' _ "$S"
check "qwen_load_state: the running server's port and context win over the defaults" \
eval 'has "http://127.0.0.1:9000" && has "ctx=65536"'
exit $fail
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env bash
# Each check is a condition string run by eval: $rc and the helpers are read there, so
# the linter cannot see them used.
# shellcheck disable=SC2016,SC2034,SC2329
# Tests for qwen-measure.py. scripts/tests/qwen-measure.test.sh
# A fake llama-server (a small Python HTTP server) answers /tokenize (one token per word) and
# /v1/chat/completions (fixed timings, and the prompt's word count as prompt_tokens). No GPU, no model.
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
S="$HERE/.."
T="$(mktemp -d)"
cleanup() { [ -z "${FAKE:-}" ] || kill "$FAKE" 2>/dev/null; rm -rf -- "$T"; }
trap cleanup EXIT
fail=0
check() {
local d="$1"; shift
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
}
run() { "$@" >"$T/out" 2>&1; rc=$?; }
has() { grep -q -F -- "$1" "$T/out"; }
cat > "$T/fake.py" <<'EOF2'
import http.server, json, sys
port_file, log = sys.argv[1], sys.argv[2]
class H(http.server.BaseHTTPRequestHandler):
def do_POST(self):
body = json.loads(self.rfile.read(int(self.headers["content-length"])))
with open(log, "a") as f: f.write(json.dumps({"path": self.path, "body": body}) + "\n")
if self.path == "/tokenize":
out = {"tokens": list(range(len(body["content"].split())))}
else:
words = sum(len(m["content"].split()) for m in body["messages"])
out = {"choices": [{"message": {"content": "ok", "reasoning_content": "think"}}],
"usage": {"prompt_tokens": words, "completion_tokens": 10},
"timings": {"prompt_n": words, "prompt_per_second": 1000.0, "predicted_n": 10,
"predicted_per_second": 50.0 + len(body["messages"]), "draft_n": 8, "draft_n_accepted": 6}}
data = json.dumps(out).encode()
self.send_response(200); self.send_header("content-type", "application/json")
self.send_header("content-length", str(len(data))); self.end_headers(); self.wfile.write(data)
def log_message(self, *a): pass
s = http.server.HTTPServer(("127.0.0.1", 0), H)
open(port_file, "w").write(str(s.server_port))
s.serve_forever()
EOF2
python3 -I "$T/fake.py" "$T/port" "$T/requests.log" & FAKE=$!
for _ in $(seq 50); do [ -s "$T/port" ] && break; sleep 0.1; done
QWEN_URL="http://127.0.0.1:$(cat "$T/port")"; export QWEN_URL
M="$S/qwen-measure.py"
echo "one two three four" > "$T/prompt.txt"
run python3 -I "$M" speed --runs 3 --max-tokens 64 "$T/prompt.txt"
check "speed: one line per run with the server's timings, then the medians" \
eval '[ $rc = 0 ] && [ "$(grep -c "^run " "$T/out")" = 3 ] && has "decode 51.0 tokens/s" && has "draft 6/8" && has "median decode 51.0 tokens/s, median prompt 1000 tokens/s"'
check "speed: sends the prompt file and the token cap" \
eval 'grep -q "\"max_tokens\": 64" "$T/requests.log" && grep -q "one two three four" "$T/requests.log"'
mkdir -p "$T/src"; for i in 1 2 3; do printf 'word%.0s ' $(seq 40) > "$T/src/f$i.go"; done
run python3 -I "$M" prompt --tokens 300 --question "What is it?" --out "$T/long.txt" "$T/src/*.go"
check "prompt: builds a prompt of about the asked size from the files, repeated, and ends with the question" \
eval '[ $rc = 0 ] && has "tokens" && n=$(wc -w < "$T/long.txt") && [ "$n" -ge 280 ] && [ "$n" -le 330 ] && tail -c 20 "$T/long.txt" | grep -q "What is it?"'
run python3 -I "$M" prompt --tokens 300 --out "$T/none.txt" "$T/src/*.nothing"
check "prompt: no matching files is an error" eval '[ $rc = 2 ] && has "no files" && [ ! -e "$T/none.txt" ]'
run python3 -I "$M" growth --turns 3 --pid "$FAKE" --chunk 20 "$T/long.txt"
check "growth: one line per turn with the context, which grows, and the server's resident memory" \
eval '[ $rc = 0 ] && [ "$(grep -c "^turn " "$T/out")" = 3 ] && has "rss" && a=$(sed -n "1s/.*ctx *\([0-9]*\).*/\1/p" "$T/out") && b=$(sed -n "3s/.*ctx *\([0-9]*\).*/\1/p" "$T/out") && [ "$b" -gt "$a" ]'
check "growth: each turn sends the whole conversation so far" \
eval '[ "$(grep -c "/v1/chat/completions" "$T/requests.log")" -ge 3 ] && tail -n 1 "$T/requests.log" | grep -q "Turn 2" && tail -n 1 "$T/requests.log" | grep -q "Turn 0"'
run python3 -I "$M" growth --turns 1 --pid 999999999 "$T/long.txt"
check "growth: a pid that does not exist is an error" eval '[ $rc = 2 ] && has "999999999"'
QWEN_URL="http://127.0.0.1:1" run python3 -I "$M" speed --runs 1 "$T/prompt.txt"
check "speed: a server that does not answer is a clear error, not a traceback" \
eval '[ $rc = 2 ] && has "qwen-local.sh start" && ! has Traceback'
exit $fail
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Each check is a condition string run by eval: $rc and the helpers are read there, so
# the linter cannot see them used.
# shellcheck disable=SC2016,SC2034,SC2329
# Tests for qwen-opencode.sh. scripts/tests/qwen-opencode.test.sh
# A fake `opencode` and a fake `curl` stand in for the real tools: no server, no network.
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
S="$HERE/.."
T="$(mktemp -d)"
trap 'rm -rf -- "$T"' EXIT
fail=0
check() {
local d="$1"; shift
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
}
run() { rm -f "$T/args" "$T/env"; "$@" >"$T/out" 2>&1; rc=$?; }
has() { grep -q -F -- "$1" "$T/out"; }
envhas() { grep -qx -F -- "$1" "$T/env"; }
# conf EXPR: evaluate a python expression over the config the fake opencode was given (as c).
conf() { python3 -I -c 'import json,sys; c=json.loads(sys.argv[1]); sys.exit(0 if eval(sys.argv[2]) else 1)' \
"$(sed -n 's/^OPENCODE_CONFIG_CONTENT=//p' "$T/env")" "$1"; }
export HOME="$T/home"; mkdir -p "$HOME"
mkdir -p "$T/bin"
cat > "$T/bin/opencode" <<EOF2
#!/usr/bin/env bash
printf '%s\n' "\$@" > "$T/args"
env > "$T/env"
EOF2
cat > "$T/bin/curl" <<EOF2
#!/usr/bin/env bash
[ -e "$T/healthy" ] || exit 7
echo '{"status":"ok"}'
EOF2
chmod +x "$T/bin/opencode" "$T/bin/curl"
export PATH="$T/bin:$PATH" QWEN_HOME="$T/qh"
O="$S/qwen-opencode.sh"
run bash "$O" run hi
check "server down: refuses with the start hint, and opencode does not run" \
eval '[ $rc = 2 ] && has "qwen-local.sh start" && [ ! -e "$T/args" ]'
touch "$T/healthy"; mkdir -p "$T/qh/run"; printf 'QWEN_PORT=9000\nQWEN_CTX=65536\n' > "$T/qh/run/server.env"
ANTHROPIC_API_KEY=a CLAUDE_CODE_OAUTH_TOKEN=b OPENROUTER_API_KEY=c run bash "$O" run "hi there"
check "the config points at the running server's port and context, and only at the local provider" \
eval '[ $rc = 0 ] && conf "c[\"provider\"][\"qwen-local\"][\"options\"][\"baseURL\"] == \"http://127.0.0.1:9000/v1\"" && conf "c[\"provider\"][\"qwen-local\"][\"models\"][\"qwen3.8-27b\"][\"limit\"][\"context\"] == 65536" && conf "c[\"enabled_providers\"] == [\"qwen-local\"]"'
check "the model and the small model are both the local one; sharing and updates are off" \
eval 'conf "c[\"model\"] == c[\"small_model\"] == \"qwen-local/qwen3.8-27b\"" && conf "c[\"share\"] == \"disabled\" and c[\"autoupdate\"] is False"'
check "run gets --standalone, so a background service with an older config is never used" \
eval '[ "$(cat "$T/args")" = "$(printf "run\n--standalone\nhi there")" ]'
check "OpenCode does not read ~/.claude (the work account) or outside skills, and does not update, share or fetch models" \
eval 'envhas OPENCODE_DISABLE_CLAUDE_CODE=1 && envhas OPENCODE_DISABLE_EXTERNAL_SKILLS=1 && envhas OPENCODE_DISABLE_AUTOUPDATE=true && envhas OPENCODE_DISABLE_SHARE=true && envhas OPENCODE_DISABLE_MODELS_FETCH=true'
check "no cloud credential from the caller's shell reaches opencode" \
eval '[ -e "$T/env" ] && ! grep -Eq "^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN|OPENROUTER_API_KEY)=" "$T/env"'
run bash "$O"
check "the interactive session gets --standalone too" eval '[ $rc = 0 ] && [ "$(cat "$T/args")" = "--standalone" ]'
run bash "$O" ./some/project
check "a project folder for the interactive session follows --standalone" \
eval '[ "$(cat "$T/args")" = "$(printf -- "--standalone\n./some/project")" ]'
exit $fail