mirror of
https://github.com/piabrota/mu.git
synced 2026-11-07 06:15:59 -10:00
Local Qwen 3.8 27B: scripts, a how-to, and a three-way comparison (#63)
* Local Qwen 3.8 27B: design and plan Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-local.sh: start, stop and status for the local llama-server Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-opencode.sh: OpenCode on the local Qwen server Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-claude.sh: Claude Code on the local Qwen server, in its own config dir Co-Authored-By: Claude Opus 5.5 <[email protected]> * Local Qwen: setup procedure and verified server flags Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-local.sh: native reasoning flags and MTP on by default llama-server has --reasoning-effort and --reasoning off, so the plan's --chat-template-kwargs is replaced (plan corrected in Task 2). The MTP default, checked against this build's --help, drafts with mtp.gguf on the GPU. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Claude Code on local Qwen: fix the chat template, pass the context size Claude Code sends hook context as a system message mid-conversation, and Qwen's template refuses any system message after the first, so every request failed with HTTP 500. qwen-local.sh template saves a copy that renders it as a system turn, and start uses it. qwen-claude.sh tells Claude Code the server's context (CLAUDE_CODE_MAX_CONTEXT_TOKENS). Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-opencode.sh: always --standalone OpenCode 2 runs sessions in a background service that keeps the config and environment it started with, so a later OPENCODE_CONFIG_CONTENT (and the safety variables) were ignored. --standalone gives each session a private server with exactly this config. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Local Qwen: measured speed, context and reasoning effort Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-local.sh: cap the prompt cache in system RAM at 2 GB llama-server keeps a prompt cache in system RAM, 8 GB by default. In an agent session it grew past WSL's 12 GB memory cap, and the kernel's out-of-memory killer stopped the server. With --cache-ram 2048 the server stays at about 3.4 GB after five long prompts. Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-local.sh: keep 4 context checkpoints, not 32 Qwen 3.8 is a hybrid model: each context checkpoint holds its recurrent state in system RAM. With llama-server's default of 32, a long session grew the server from 2 GB to 5.7 GB, and with an agent and Go tests beside it, WSL's 12 GB cap was hit and the server was killed. With 4 it stays at about 2.3 GB through 50K tokens of conversation. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Local Qwen: three-way comparison and lessons Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-claude.sh: guard the account dirs in both modes; strip cloud switches The config-dir guard covered only init and missed $HOME, so a run with QWEN_CLAUDE_DIR=~/.claude would have used the work config, and init with $HOME would have written ~/.claude.json. Both modes now refuse a dir that is inside an account dir or holds one. The Bedrock, Vertex and Foundry switches are removed, since they take Claude Code past the base URL. Found by the final review. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Local Qwen notes: the remaining config differences and the skills link Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-measure.py and qwen-capture.py: the measurement and capture tools The speed, long-prompt and memory-growth measurements in the notes, and the request capture that found the template and OpenCode service bugs, were throwaway scripts. They are now repo tools with tests, so the experiment can be repeated. Co-Authored-By: Claude Opus 5.5 <[email protected]> * qwen-compare.sh: run one task three ways and keep the evidence The three-way comparison (OpenCode on local Qwen, Claude Code on local Qwen, Claude Code on a hosted model) was a throwaway runner. It is now a repo script with tests: throwaway worktrees at one commit, the same prompt, and per run the output, diff, server-log slice and wall time. Co-Authored-By: Claude Opus 5.5 <[email protected]> * Local Qwen: a reproduce how-to, the comparison's evidence, docs accepted The how-to holds every step: setup, daily use, the measurements, the comparison and request capture, with troubleshooting for what broke. The notes keep the findings. The three B-071 diffs, the prompt and the run summary are kept, since the worktrees are removed. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Bruno Guilhermo de Barros Valério <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
22 files changed
+2389
No files matched your search
@@ -0,0 +1,54 @@
|
||||
diff --git a/internal/engine/engine.go b/internal/engine/engine.go
|
||||
index 0e5db0d..6dd06d0 100644
|
||||
--- a/internal/engine/engine.go
|
||||
+++ b/internal/engine/engine.go
|
||||
@@ -103,6 +103,9 @@ func advance(ctx context.Context, r *run.Run) (Outcome, error) {
|
||||
err = runInit(ctx, r, n)
|
||||
case schema.KindGrade:
|
||||
err = runGrade(ctx, r, n)
|
||||
+ default:
|
||||
+ // A kind the engine does not know must never pass as if it had run: stop the run on the manifest.
|
||||
+ err = unknownKind(r, n)
|
||||
}
|
||||
var sb *sentBack
|
||||
var ex *expanded
|
||||
@@ -150,6 +153,15 @@ func advance(ctx context.Context, r *run.Run) (Outcome, error) {
|
||||
return Outcome{Status: run.StatusDone}, r.Save()
|
||||
}
|
||||
|
||||
+// unknownKind fails the run for a node whose kind the engine has no way to run: the graph is a bad manifest.
|
||||
+func unknownKind(r *run.Run, n schema.Node) error {
|
||||
+ cause := fmt.Errorf("%s: node %s has the kind %q, which this Odin does not know", gungnir.GradeBadManifest, n.ID, n.Kind)
|
||||
+ if err := r.Log(gungnir.Event{Name: gungnir.EventNodeFailed, Node: n.ID, Result: cause.Error()}); err != nil {
|
||||
+ return err
|
||||
+ }
|
||||
+ return &nodeFailure{cause}
|
||||
+}
|
||||
+
|
||||
// doneResult is how a finished run is described: "done", or the tasks it gave up, listed first (spec §7.3).
|
||||
func doneResult(s *run.State) string {
|
||||
var blocked []string
|
||||
diff --git a/internal/engine/grade_failure_test.go b/internal/engine/grade_failure_test.go
|
||||
index 24fe315..78fd25e 100644
|
||||
--- a/internal/engine/grade_failure_test.go
|
||||
+++ b/internal/engine/grade_failure_test.go
|
||||
@@ -35,6 +35,19 @@ func TestAResultsFileThatDoesNotMatchTheManifestIsRefusedAsBadManifest(t *testin
|
||||
}
|
||||
}
|
||||
|
||||
+func TestANodeOfAKindTheEngineDoesNotKnowStopsTheRunAsBadManifest(t *testing.T) {
|
||||
+ g := startGraded(t, gradedNodes, tasktest.Results(nil), nil)
|
||||
+ nodes := g.r.Setup.Graph.Nodes
|
||||
+ nodes[len(nodes)-1].Kind = "teleport" // the graph loader rejects this kind, so the test sets it after the load
|
||||
+ out := g.advance(t)
|
||||
+ if out.Status != run.StatusFailed || out.Err == nil || !strings.Contains(out.Err.Error(), "bad_manifest") || !strings.Contains(out.Err.Error(), "teleport") {
|
||||
+ t.Fatalf("Advance() = %+v, want a failed run whose error says bad_manifest", out)
|
||||
+ }
|
||||
+ if len(g.grader.Specs) != 0 {
|
||||
+ t.Errorf("the grader ran %d time(s) for a node of an unknown kind", len(g.grader.Specs))
|
||||
+ }
|
||||
+}
|
||||
+
|
||||
func TestAGradingRunThatFailsFailsTheNodeAndAResumeGradesAgain(t *testing.T) {
|
||||
g := startGraded(t, gradedNodes, tasktest.Results(nil), nil)
|
||||
g.grader.Err = errors.New("the grader timed out after 5m0s")
|
||||
@@ -0,0 +1,40 @@
|
||||
diff --git a/internal/engine/engine.go b/internal/engine/engine.go
|
||||
index 0e5db0d..41676ac 100644
|
||||
--- a/internal/engine/engine.go
|
||||
+++ b/internal/engine/engine.go
|
||||
@@ -103,6 +103,9 @@ func advance(ctx context.Context, r *run.Run) (Outcome, error) {
|
||||
err = runInit(ctx, r, n)
|
||||
case schema.KindGrade:
|
||||
err = runGrade(ctx, r, n)
|
||||
+ default:
|
||||
+ // The manifest names a kind the engine does not run: the run stops failed (backlog B-071).
|
||||
+ err = &nodeFailure{fmt.Errorf("bad_manifest: the graph names a node kind, %q, that the engine does not run", n.Kind)}
|
||||
}
|
||||
var sb *sentBack
|
||||
var ex *expanded
|
||||
diff --git a/internal/engine/engine_test.go b/internal/engine/engine_test.go
|
||||
index 853b58a..1d67276 100644
|
||||
--- a/internal/engine/engine_test.go
|
||||
+++ b/internal/engine/engine_test.go
|
||||
@@ -254,3 +254,21 @@ func TestEveryNodeInALeafEndsWithACheckpoint(t *testing.T) {
|
||||
t.Error("node a's node_finished line doesn't carry its checkpoint commit")
|
||||
}
|
||||
}
|
||||
+
|
||||
+func TestAKindTheEngineDoesNotRunStopsTheRunWithBadManifest(t *testing.T) {
|
||||
+ r := start(t, "key-points", "", " - { id: a, kind: exec, run: [echo, ok], repo: repos/app }\n", "steps: {}\n")
|
||||
+ // ParseGraph enforces the kind's enum, so the bad kind is put into the parsed graph: a manifest
|
||||
+ // the engine is given names a kind it does not run (backlog B-071).
|
||||
+ r.Setup.Graph.Nodes[0].Kind = "frobnicate"
|
||||
+ out, err := Advance(context.Background(), r)
|
||||
+ if err != nil || out.Status != run.StatusFailed {
|
||||
+ t.Fatalf("Advance() = %+v, %v; want the run stopped failed", out, err)
|
||||
+ }
|
||||
+ if !strings.Contains(out.Err.Error(), "bad_manifest") {
|
||||
+ t.Errorf("Advance() = %+v; the failure must name bad_manifest", out)
|
||||
+ }
|
||||
+ text := ledgerText(t, r)
|
||||
+ if !strings.Contains(text, `"event":"run_failed"`) || !strings.Contains(text, "bad_manifest") {
|
||||
+ t.Errorf("the ledger:\n%s", text)
|
||||
+ }
|
||||
+}
|
||||
@@ -0,0 +1 @@
|
||||
In this Go repository (odin), fix backlog item B-071: `Advance` in internal/engine/engine.go has no default case in its switch on the node kind, so a graph node of a kind the engine does not know is skipped as if it had passed. Add a default case that stops the run with the `bad_manifest` reason, following how the engine already stops runs for other manifest problems. Add a test: a graph with a made-up node kind stops with bad_manifest. Run `GOWORK=off go test ./internal/engine/` and make sure it passes (the package's tests take about 90 seconds). Do not commit. Do not ask clarifying questions; make reasonable choices and say what you chose.
|
||||
@@ -0,0 +1,41 @@
|
||||
diff --git a/internal/engine/engine.go b/internal/engine/engine.go
|
||||
index 0e5db0d..bc9844e 100644
|
||||
--- a/internal/engine/engine.go
|
||||
+++ b/internal/engine/engine.go
|
||||
@@ -103,6 +103,9 @@ func advance(ctx context.Context, r *run.Run) (Outcome, error) {
|
||||
err = runInit(ctx, r, n)
|
||||
case schema.KindGrade:
|
||||
err = runGrade(ctx, r, n)
|
||||
+ default:
|
||||
+ // A kind the engine does not know is a bad manifest, never a pass.
|
||||
+ err = &nodeFailure{fmt.Errorf("bad_manifest: the graph names the node kind %q, which the engine does not know", n.Kind)}
|
||||
}
|
||||
var sb *sentBack
|
||||
var ex *expanded
|
||||
diff --git a/internal/engine/engine_test.go b/internal/engine/engine_test.go
|
||||
index 853b58a..b128838 100644
|
||||
--- a/internal/engine/engine_test.go
|
||||
+++ b/internal/engine/engine_test.go
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
+ "github.com/ouroboric-dev/gungnir/schema"
|
||||
"github.com/ouroboric-dev/odin/internal/ledger"
|
||||
"github.com/ouroboric-dev/odin/internal/run"
|
||||
"github.com/ouroboric-dev/odin/internal/tree/treetest"
|
||||
@@ -254,3 +255,14 @@ func TestEveryNodeInALeafEndsWithACheckpoint(t *testing.T) {
|
||||
t.Error("node a's node_finished line doesn't carry its checkpoint commit")
|
||||
}
|
||||
}
|
||||
+
|
||||
+func TestAnUnknownNodeKindIsABadManifest(t *testing.T) {
|
||||
+ r := start(t, "key-points", "", " - { id: a, kind: exec, run: [echo, hi], repo: repos/app }\n", "steps: {}\n")
|
||||
+ // ParseGraph refuses the kinds its schema doesn't know; the case this covers is a graph whose
|
||||
+ // schema is newer than the engine's (backlog B-071), so the node is put in after the run started.
|
||||
+ r.Setup.Graph.Nodes = append([]schema.Node{{ID: "odd", Kind: "quincunx", Repo: "repos/app"}}, r.Setup.Graph.Nodes...)
|
||||
+ out, err := Advance(context.Background(), r)
|
||||
+ if err != nil || out.Status != run.StatusFailed || !strings.Contains(out.Err.Error(), "bad_manifest") {
|
||||
+ t.Fatalf("Advance() = %+v, %v; want failed with bad_manifest", out, err)
|
||||
+ }
|
||||
+}
|
||||
@@ -0,0 +1,4 @@
|
||||
run exit wall_s
|
||||
opencode 0 508
|
||||
qwen-claude 0 551
|
||||
hosted 0 147
|
||||
|
@@ -0,0 +1,176 @@
|
||||
---
|
||||
title: Local Qwen 3.8 27B coding assistant
|
||||
status: accepted
|
||||
date: 2026-10-19
|
||||
style: spec
|
||||
---
|
||||
|
||||
# Local Qwen 3.8 27B coding assistant
|
||||
|
||||
## 1. Intent
|
||||
|
||||
The founder wants a coding assistant that runs on their own machine: Qwen 3.8 27B on the RTX 4090,
|
||||
driven by OpenCode, and also by Claude Code for comparison. There are three reasons:
|
||||
|
||||
- **Limits and cost.** Claude subscription limits and paid API usage cap how much agentic work
|
||||
happens in a day. A local model has no rate limit and no per-token bill.
|
||||
- **Curiosity and learning.** The founder wants to understand local inference: quantization,
|
||||
context memory, reasoning effort, speculative decoding, and how agent tools behave on a smaller
|
||||
model.
|
||||
- **Groundwork.** OpenCode is the harness that Yggdrasil's Gleipnir already drives. Lessons about its
|
||||
configuration and failure modes carry over.
|
||||
|
||||
**Split from Yggdrasil.** Routing Yggdrasil roles to Qwen is a separate track. It uses the OpenRouter
|
||||
path that already exists (`openrouter/qwen/qwen3.8-27b`) and is out of scope here. Hosted Qwen 3.8 27B
|
||||
costs about $0.02 per million input tokens and $0.50 per million output tokens on
|
||||
[OpenRouter](https://openrouter.ai/qwen/qwen3.8-27b), so local inference does not win on cost for
|
||||
Yggdrasil. It wins on limits, privacy, offline use and learning.
|
||||
|
||||
## 2. Goals
|
||||
|
||||
1. One command starts the model server, and one command stops it.
|
||||
2. OpenCode, connected to the local server, completes a real small task in one of the founder's
|
||||
repos.
|
||||
3. Claude Code, connected to the same server, completes the same task, so the two harnesses can be
|
||||
compared on one model.
|
||||
4. The setup reaches about 100K tokens of context and records the measured decode speed. The target
|
||||
is 40 tokens per second or more.
|
||||
5. A short notes page records what the founder learned: the settings that mattered, where each
|
||||
harness struggled, and how the result compares with Claude.
|
||||
|
||||
## 3. Non-goals
|
||||
|
||||
- Yggdrasil integration (see §1).
|
||||
- Parallel sessions or serving other people. One session at a time.
|
||||
- vLLM, SGLang, or a Windows-native server.
|
||||
- Community forks of llama.cpp (for example [ninfer-4090](https://github.com/sergiuszm/ninfer-4090)
|
||||
or BeeLlama). They are candidate follow-ups for curiosity, not part of this work.
|
||||
- Vision input, fine-tuning, or other models.
|
||||
|
||||
## 4. Anti-goals
|
||||
|
||||
These make the work a failure even if the model runs:
|
||||
|
||||
- **The Claude account split breaks.** The founder runs two Claude Code accounts, split by folder
|
||||
(work under `~/flink` with `~/.claude`; personal under `~/brunogbv` with `~/.claude-personal`),
|
||||
with `claude`, `claude-work` and `claude-personal` shell functions in
|
||||
`~/.config/claude-accounts.zsh` and an `account-guard.sh` hook in each config. The local setup MUST
|
||||
NOT change either config dir, either login, or the `claude` function's routing. Local Claude Code
|
||||
MUST use its own config dir and its own command, and MUST NOT log in to an Anthropic account.
|
||||
- **Requests leave the machine.** The local Claude Code and OpenCode MUST send model requests only to
|
||||
the local server. Local Claude Code SHOULD turn off non-essential traffic
|
||||
(`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1`).
|
||||
- **A pile of one-off commands.** The setup MUST be repeatable from the repo (see §6).
|
||||
|
||||
## 5. Constraints
|
||||
|
||||
Measured on 2026-10-19:
|
||||
|
||||
| Item | Value |
|
||||
| --- | --- |
|
||||
| GPU | NVIDIA RTX 4090, 24 GB; about 2.1 GB already used by the Windows desktop |
|
||||
| Driver | NVIDIA 610.57 (WSL), CUDA 13.3 user-mode driver |
|
||||
| CPU | AMD Ryzen 7 7800X3D, 8 cores, 16 threads; WSL gets 12 |
|
||||
| RAM | 32 GB on Windows; WSL is capped at 12 GB by `C:\Users\Bruno\.wslconfig` |
|
||||
| Disk | about 708 GB free on `/` |
|
||||
| OS | WSL2, Linux 6.18; shell zsh |
|
||||
| Already installed | gcc, podman (aliased as `docker`, runtime crun). Not installed: CUDA toolkit (`nvcc`), cmake, llama.cpp, OpenCode. No NVIDIA CDI spec for podman |
|
||||
|
||||
The model (from [Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B)):
|
||||
|
||||
- 27B dense, Apache 2.0, 262,144-token native context.
|
||||
- Hybrid layout: of the 64 layers, only 16 use full attention (4 KV heads, head dim 256). The other
|
||||
48 are Gated DeltaNet with a fixed-size state. The KV cache is therefore about 64 KB per token at
|
||||
16 bits, or about 32 KB at 8 bits. [ANALYSIS: 16 layers × 2 × 4 × 256 × 2 bytes.]
|
||||
- Trained with multi-token prediction (MTP), which llama.cpp can use for speculative decoding.
|
||||
- Thinking is on by default, and `reasoning_effort` defaults to `xhigh`. Levels are `xhigh`,
|
||||
`medium` and `low`; `enable_thinking` turns thinking off. Qwen notes that low effort can make
|
||||
multi-turn agentic tasks slower overall, because they fail and retry more.
|
||||
- Recommended sampling with thinking: temperature 1.0, top_p 0.95, top_k 20, min_p 0.
|
||||
- Qwen ran its coding benchmarks in the Claude Code harness with a 256K context.
|
||||
|
||||
Memory budget [ANALYSIS]: a Q4-class GGUF is about 17–18 GB. About 2 GB is already used and about
|
||||
1 GB goes to compute buffers, which leaves roughly 3–4 GB for the KV cache. That is about 100–128K
|
||||
tokens at 8-bit KV. MTP buffers also take memory: one user hit the VRAM limit at 196K context
|
||||
([unsloth discussion](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/discussions/32)).
|
||||
|
||||
Reported speeds on a 4090 ([computingforgeeks](https://computingforgeeks.com/run-qwen3-8-27b-locally/),
|
||||
[gist](https://gist.github.com/ryan4yin/19db9fa44972c5735c1d181e8888d4fe)): about 47 tokens per
|
||||
second without MTP and about 60–110 with MTP, depending on context length and prompt. In
|
||||
[Simon Willison's review](https://simonwillison.net/2026/Aug/16/qwen-38-27b/), tool calls in a coding
|
||||
agent worked, but the default `xhigh` effort spent about 22K reasoning tokens on one small prompt.
|
||||
|
||||
## 6. Decided
|
||||
|
||||
These are the founder's decisions, or calls Claude made in the conversation that the founder
|
||||
accepted:
|
||||
|
||||
- **Runtime:** `llama-server` from upstream llama.cpp, built with CUDA, running in WSL with every
|
||||
layer on the GPU. It serves the OpenAI-compatible API, and the Anthropic Messages API at
|
||||
`/v1/messages` that Claude Code uses
|
||||
([Hugging Face blog](https://huggingface.co/blog/ggml-org/anthropic-messages-api-in-llamacpp)).
|
||||
- **Model file:** a Q4-class GGUF of Qwen 3.8 27B, plus the MTP draft file if llama.cpp needs one.
|
||||
- **Main harness:** OpenCode, with the local server configured as an OpenAI-compatible provider.
|
||||
- **Comparison harness:** Claude Code with `ANTHROPIC_BASE_URL` pointing at the local server, under a
|
||||
separate config dir and a separate command (see §4).
|
||||
- **Hosted baseline:** the same task also runs once on hosted Claude, through the founder's
|
||||
personal account, to give a measured baseline: wall time, tokens, retries, and pass or fail. The
|
||||
task MUST be small, so the baseline uses few subscription tokens.
|
||||
- **Defaults:** one session at a time; about 100K tokens of context; low reasoning effort by
|
||||
default, which the founder can raise for each task.
|
||||
- **Location:** scripts in `ouroboric/scripts`, with tests in `ouroboric/scripts/tests` in the same
|
||||
style as the existing ones (fake commands, no GPU or network needed). The work goes in through a
|
||||
pull request. The notes page goes in `ouroboric/docs/` beside this document.
|
||||
|
||||
Claude's calls after the builder check (the founder can override them):
|
||||
|
||||
- **When the targets collide.** 40 tokens per second is a target that gets recorded, not a pass/fail
|
||||
gate. Measure speed twice: on a near-empty context, and with about 100K tokens filled. If the
|
||||
Q4 file, MTP and 100K context don't all fit in 24 GB, keep the Q4 file and MTP, and reduce the
|
||||
context. The floor is 64K tokens. Record what fits.
|
||||
- **One-time setup.** Installing the CUDA toolkit, building llama.cpp, installing OpenCode and
|
||||
downloading the model are a written procedure in the notes page. The founder runs the steps that
|
||||
need `sudo`. The download and build steps MAY be scripts. They need tests only if they hold logic
|
||||
that can break. Start, stop and the local Claude Code launcher MUST be scripts with tests.
|
||||
- **WSL memory cap.** Leave `.wslconfig` as it is, because changing it needs `wsl --shutdown`. If the
|
||||
build or model load fails at 12 GB, use fewer build jobs, then ask the founder.
|
||||
- **Network.** Configuration is enough; no firewall or network namespace. "Model requests" means
|
||||
calls to an LLM only. Turn off OpenCode's auto-update and sharing, and point its small model at
|
||||
the local server too. Check once, by reading the server log, that every model call reached the
|
||||
local server.
|
||||
- **The test task's result.** Each run (OpenCode, local Claude Code, hosted Claude) works in its own
|
||||
throwaway git worktree, directly on the host, in the harness's default ask-before-acting
|
||||
permission mode. The founder judges each diff, then the worktree is removed. Nothing is pushed,
|
||||
and the local agents get no GitHub access.
|
||||
- **The local Claude Code config dir (founder's decision: match the personal setup, for a fair
|
||||
comparison).** It gets the same plugins as `~/.claude-personal` (`superpowers`, `gopls-lsp`,
|
||||
`skill-creator` from `claude-plugins-official`), the same user skills (`~/.claude-personal/skills`),
|
||||
and the `merge-requires-green.sh` hook. It leaves out three things:
|
||||
- the `account-guard.sh` hook, because the local config has no account;
|
||||
- claude.ai connectors (Claude Docs, Artifacts), because they need an account and would send data
|
||||
off the machine;
|
||||
- the personal auto-memory.
|
||||
|
||||
Install the plugins into the local config dir, or link to them. Do not change the files in
|
||||
`~/.claude-personal`. Record the remaining differences in the notes page. The task prompt for all
|
||||
three runs is the same, and it says to skip clarifying questions, so the `superpowers` brainstorming
|
||||
gate doesn't stall the comparison.
|
||||
- **Reasoning effort.** Changing the default effort MAY need a server restart with a different
|
||||
flag. Per-request control inside a harness is a nice-to-have. Record whether each harness
|
||||
supports it.
|
||||
|
||||
## 7. Left to the builder
|
||||
|
||||
- How to install the CUDA toolkit and build llama.cpp: build from source in WSL, or use the
|
||||
`server-cuda` container under podman with an NVIDIA CDI spec. Building from source is preferred if
|
||||
it is not much harder, because MTP support is recent.
|
||||
- The exact quant: unsloth Q4_K_M or UD-Q4_K_XL, or another Q4-class file. Pick one, and record why.
|
||||
- Where model files live (outside every repo) and how the scripts find them.
|
||||
- The `llama-server` flags: context size, KV cache type, MTP settings, sampling, and how the default
|
||||
reasoning effort is set.
|
||||
- Script names and the command names for start, stop, and local Claude Code. Do not shadow `claude`,
|
||||
`claude-work` or `claude-personal`.
|
||||
- How OpenCode's config is installed: global config or an env var such as `OPENCODE_CONFIG`.
|
||||
- The task used for goals 2 and 3. Pick a small, real task from a backlog in one of the founder's
|
||||
repos, and confirm it with the founder before running it.
|
||||
- How to measure speed and context: the server's own timings are enough.
|
||||
@@ -0,0 +1,183 @@
|
||||
---
|
||||
title: Run Qwen 3.8 27B locally and repeat the experiment
|
||||
status: accepted
|
||||
date: 2026-10-20
|
||||
style: guide
|
||||
related: [2026-10-19-local-qwen-design.md, 2026-10-19-local-qwen-notes.md]
|
||||
---
|
||||
|
||||
# Run Qwen 3.8 27B locally and repeat the experiment
|
||||
|
||||
This guide sets up Qwen 3.8 27B on one NVIDIA RTX 4090 under WSL2. Then you use it from OpenCode and
|
||||
from Claude Code, and repeat the measurements and the three-way comparison in
|
||||
[the notes](2026-10-19-local-qwen-notes.md). Every command runs in zsh, from the root of the
|
||||
`ouroboric` repo, unless a step says otherwise.
|
||||
|
||||
## Before you start
|
||||
|
||||
You need:
|
||||
|
||||
- An RTX 4090, or another 24 GB NVIDIA card, with a Windows driver that offers CUDA 13.3 or newer in
|
||||
WSL. Check with `nvidia-smi`.
|
||||
- WSL2 with Ubuntu 22.04, about 25 GB of free disk, Homebrew, `uv`, and `sudo` for one step.
|
||||
- Claude Code (`claude`) on `PATH`, and the personal Claude Code config in `~/.claude-personal`.
|
||||
|
||||
The model, the llama.cpp build and the run state live in `~/.local/share/qwen-local` (`QWEN_HOME`),
|
||||
outside every repo.
|
||||
|
||||
## Set up the server
|
||||
|
||||
1. Install the CUDA toolkit. This is the only step that needs `sudo`. Install only
|
||||
`cuda-toolkit-13-3`. Never install `cuda`, `cuda-drivers` or `nvidia-driver-*`: in WSL, the Windows
|
||||
driver is the GPU driver.
|
||||
|
||||
```zsh
|
||||
cd /tmp && wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-keyring_1.1-1_all.deb
|
||||
sudo dpkg -i cuda-keyring_1.1-1_all.deb
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y cuda-toolkit-13-3
|
||||
```
|
||||
|
||||
`/usr/local/cuda/bin/nvcc --version` now prints `release 13.3`.
|
||||
|
||||
1. Install cmake and OpenCode: `brew install cmake opencode`.
|
||||
1. Build `llama-server` with CUDA. Keep Homebrew off `PATH` for the build: Homebrew's linker misses
|
||||
the CUDA libraries, and Homebrew's OpenSSL needs a newer glibc than Ubuntu 22.04 has.
|
||||
|
||||
```zsh
|
||||
git clone https://github.com/ggml-org/llama.cpp ~/.local/share/qwen-local/llama.cpp
|
||||
cd ~/.local/share/qwen-local/llama.cpp
|
||||
git checkout 4c3c581 # the build the notes measured; leave this out for the latest
|
||||
export PATH=/usr/local/cuda/bin:/usr/bin:/bin
|
||||
/home/linuxbrew/.linuxbrew/bin/cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89 \
|
||||
-DCMAKE_BUILD_TYPE=Release -DLLAMA_OPENSSL=OFF
|
||||
/home/linuxbrew/.linuxbrew/bin/cmake --build build --config Release -j 6 --target llama-server
|
||||
```
|
||||
|
||||
Open a new terminal afterwards, to get your usual `PATH` back. `-j 6` keeps the build inside a
|
||||
12 GB WSL memory cap.
|
||||
|
||||
1. Download the model (16.5 GB) and its MTP draft file (1.4 GB), and link them to the names the
|
||||
scripts use:
|
||||
|
||||
```zsh
|
||||
mkdir -p ~/.local/share/qwen-local/models && cd ~/.local/share/qwen-local/models
|
||||
uvx --from huggingface_hub hf download unsloth/Qwen3.8-27B-GGUF \
|
||||
Qwen3.8-27B-UD-Q4_K_M.gguf MTP/mtp-Qwen3.8-27B-Q4_0.gguf --local-dir .
|
||||
ln -sfn Qwen3.8-27B-UD-Q4_K_M.gguf model.gguf
|
||||
ln -sfn MTP/mtp-Qwen3.8-27B-Q4_0.gguf mtp.gguf
|
||||
```
|
||||
|
||||
1. Start the server: `scripts/qwen-local.sh start`. It is ready in about 25 seconds. To see the
|
||||
flags, run `scripts/qwen-local.sh cmd`.
|
||||
1. Save the fixed chat template, then restart, so Claude Code works:
|
||||
|
||||
```zsh
|
||||
scripts/qwen-local.sh template
|
||||
scripts/qwen-local.sh stop && scripts/qwen-local.sh start
|
||||
```
|
||||
|
||||
1. Set up the local Claude Code config dir, `~/.claude-qwen`: `scripts/qwen-claude.sh init`. It
|
||||
installs the personal setup's three plugins and links its skills. It never touches `~/.claude` or
|
||||
`~/.claude-personal`.
|
||||
|
||||
The server's settings come from environment variables: `QWEN_CTX` (102400), `QWEN_EFFORT` (`low`,
|
||||
`medium`, `xhigh` or `off`), `QWEN_MTP_ARGS` (empty turns MTP off), `QWEN_CACHE_RAM` (2048) and
|
||||
`QWEN_CTX_CHECKPOINTS` (4). The comment block at the top of `scripts/qwen-local.sh` explains each one.
|
||||
|
||||
## Use it
|
||||
|
||||
- **OpenCode:** `scripts/qwen-opencode.sh` opens the interactive session. For a headless run, use
|
||||
`scripts/qwen-opencode.sh run --auto "..."`; without `--auto`, a run that calls a tool waits for an
|
||||
approval that it cannot get.
|
||||
- **Claude Code:** `scripts/qwen-claude.sh` opens the interactive session, and
|
||||
`scripts/qwen-claude.sh -p "..."` runs one prompt headless. It asks before it acts. Claude Code's own
|
||||
effort setting has no effect here; restart the server with another `QWEN_EFFORT` instead.
|
||||
- **Status and stop:** `scripts/qwen-local.sh status` and `scripts/qwen-local.sh stop`.
|
||||
|
||||
Both launchers send model calls only to the local server, and remove `ANTHROPIC_API_KEY`,
|
||||
`CLAUDE_CODE_OAUTH_TOKEN` and `OPENROUTER_API_KEY` from the session. `qwen-claude.sh` also removes the
|
||||
Bedrock, Vertex and Foundry switches.
|
||||
|
||||
## Repeat the measurements
|
||||
|
||||
Start the server, then run these commands. The figures in each answer come from the server itself.
|
||||
|
||||
1. Decode speed on a short prompt, three runs:
|
||||
|
||||
```zsh
|
||||
printf 'Write a Go function that parses a duration like "1h30m" or "45s" into seconds, with table-driven tests. Explain your choices briefly.' > /tmp/short.txt
|
||||
python3 -I scripts/qwen-measure.py speed --runs 3 --max-tokens 512 /tmp/short.txt
|
||||
```
|
||||
|
||||
To measure without MTP, restart with `QWEN_MTP_ARGS="" scripts/qwen-local.sh start` and run it
|
||||
again.
|
||||
|
||||
1. Speed with about 95K tokens of context:
|
||||
|
||||
```zsh
|
||||
python3 -I scripts/qwen-measure.py prompt --tokens 95000 --out /tmp/long.txt \
|
||||
"$HOME/brunogbv/ouroboric/yggdrasil/gleipnir/**/*.go"
|
||||
python3 -I scripts/qwen-measure.py speed --runs 1 --max-tokens 256 /tmp/long.txt
|
||||
```
|
||||
|
||||
1. Reasoning effort: restart with `QWEN_EFFORT=low`, `medium` and `xhigh` in turn, and run
|
||||
`speed --runs 1 --max-tokens 20000 /tmp/short.txt` each time.
|
||||
1. The server's memory over a growing conversation:
|
||||
|
||||
```zsh
|
||||
python3 -I scripts/qwen-measure.py growth --turns 20 --pid "$(cat ~/.local/share/qwen-local/run/server.pid)" /tmp/long.txt
|
||||
```
|
||||
|
||||
To see the problem the defaults fix, restart with `QWEN_CTX_CHECKPOINTS=32 QWEN_CACHE_RAM=8192`
|
||||
and run it again.
|
||||
|
||||
## Repeat the comparison
|
||||
|
||||
The comparison runs one task three times. Each run gets a throwaway worktree of the same commit and
|
||||
the same prompt. Tools are approved without asking, so use a task you trust the agents with.
|
||||
|
||||
```zsh
|
||||
scripts/qwen-local.sh start
|
||||
scripts/qwen-compare.sh --repo ~/brunogbv/ouroboric/yggdrasil/odin --commit bc973e3 \
|
||||
--prompt docs/5-research/2026-10-19-local-qwen-comparison/prompt.txt --out /tmp/qwen-compare-b071
|
||||
```
|
||||
|
||||
- **What it runs:** OpenCode and Claude Code on the local model, then Claude Code on Sonnet through the
|
||||
personal account. The Sonnet run uses that subscription. Change the model with `--hosted-model`, or
|
||||
skip runs with `--only opencode,qwen-claude`.
|
||||
- **What it keeps:** each run's output, diff and slice of the server log, a `summary.tsv` with exit
|
||||
codes and wall times, and `rss.log`.
|
||||
- **The task:** `bc973e3` is the `odin` commit the notes used, before B-071 was fixed. The diffs from
|
||||
the recorded runs are in
|
||||
[2026-10-19-local-qwen-comparison/](2026-10-19-local-qwen-comparison/).
|
||||
- **Clean up:** when you have read the diffs, run the cleanup commands that the script prints at the
|
||||
end.
|
||||
|
||||
## Debug what a tool sends
|
||||
|
||||
`scripts/qwen-capture.py` stands in for the model server. It saves each request body and answers it
|
||||
with an error:
|
||||
|
||||
```zsh
|
||||
python3 -I scripts/qwen-capture.py 8099 /tmp/capture &
|
||||
ANTHROPIC_BASE_URL=http://127.0.0.1:8099 CLAUDE_CONFIG_DIR=~/.claude-qwen ANTHROPIC_AUTH_TOKEN=x command claude -p hi
|
||||
kill %1
|
||||
```
|
||||
|
||||
The bodies are in `/tmp/capture/00-v1_messages.json` and the files after it. For OpenCode, set the
|
||||
provider's `baseURL` to `http://127.0.0.1:8099/v1`, and pass `--standalone`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Every Claude Code request fails with HTTP 500, "System message must be at the beginning".** The
|
||||
fixed chat template is missing. Run the template step in [Set up the server](#set-up-the-server).
|
||||
- **The server disappears in a long session, and `dmesg` shows `Out of memory: Killed process
|
||||
(llama-server)`.** WSL ran out of system RAM. Keep `QWEN_CACHE_RAM` and `QWEN_CTX_CHECKPOINTS` low,
|
||||
or raise `memory=` in `C:\Users\<you>\.wslconfig`. Run `wsl --shutdown` from Windows to apply it; this
|
||||
closes every WSL session.
|
||||
- **OpenCode ignores a changed config.** A background `opencode serve --service` from an earlier
|
||||
session holds the old one. `scripts/qwen-opencode.sh` avoids it with `--standalone`. To stop the
|
||||
service, run `opencode service stop`.
|
||||
- **`start` says the port already answers.** Another program uses port 8088. Stop it, or set
|
||||
`QWEN_PORT`.
|
||||
@@ -0,0 +1,217 @@
|
||||
---
|
||||
title: Local Qwen 3.8 27B coding assistant, notes
|
||||
status: accepted
|
||||
date: 2026-10-20
|
||||
style: research
|
||||
related: [2026-10-19-local-qwen-design.md, 2026-10-19-local-qwen-plan.md]
|
||||
---
|
||||
|
||||
# Local Qwen 3.8 27B coding assistant, notes
|
||||
|
||||
What it took to run Qwen 3.8 27B on one RTX 4090 under WSL2, how fast it is, and how it compares
|
||||
with hosted Claude. The design is [2026-10-19-local-qwen-design.md](2026-10-19-local-qwen-design.md).
|
||||
|
||||
## Setup
|
||||
|
||||
Done on 2026-10-19 and 2026-10-20. The steps to repeat it are in
|
||||
[2026-10-19-local-qwen-howto.md](2026-10-19-local-qwen-howto.md). What happened along the way:
|
||||
|
||||
- **Versions.** CUDA toolkit 13.3 (`cuda-toolkit-13-3`, which matches the driver's CUDA 13.3), cmake
|
||||
4.4.4, OpenCode 2.0.32 (the Homebrew formula is `opencode`, not `sst/tap/opencode`), and llama.cpp at
|
||||
`4c3c581` (2026-10-19, version 0.7.0-dev, build 50).
|
||||
- **The build failed twice with Homebrew first on `PATH`.**
|
||||
- Homebrew's `ld` did not search the CUDA library folder (`undefined reference to
|
||||
cudaGetLastError@libcudart.so.13`).
|
||||
- Homebrew's OpenSSL needs glibc 2.38, and Ubuntu 22.04 has 2.35 (`undefined reference to
|
||||
__isoc23_strtol@GLIBC_2.38`). llama-server uses OpenSSL only to download models, so it is turned
|
||||
off.
|
||||
- **The model file.** UD-Q4_K_M (16.46 GB), with the 1.37 GB MTP draft file. It leaves about 1 GB more
|
||||
for the context than UD-Q4_K_XL (17.56 GB) does. The design targets about 100K tokens of context,
|
||||
and Windows already holds about 2.1 GB of the card's 24 GB. [ANALYSIS]
|
||||
- **The flags, checked against this build's `--help`.**
|
||||
- MTP is `--spec-type draft-mtp -md mtp.gguf -ngld all`, and the draft's KV cache is `q8_0` too.
|
||||
- Reasoning effort has native flags: `--reasoning-effort low|medium|xhigh`, and `--reasoning off`.
|
||||
- At 102,400 tokens of context with MTP, the card holds 23.5 of 24.6 GB, Windows included.
|
||||
- **Claude Code needed two fixes.**
|
||||
- It sends hook context, such as superpowers' SessionStart text, as a `system` message in the
|
||||
middle of the conversation. The model's template refuses any system message after the first
|
||||
(`Jinja Exception: System message must be at the beginning.`), so every request failed with HTTP
|
||||
500. `qwen-local.sh template` saves a copy that renders such a message as a system turn.
|
||||
- Claude Code does not know the model, and assumes 200K tokens of context. `qwen-claude.sh` sets
|
||||
`CLAUDE_CODE_MAX_CONTEXT_TOKENS` to the server's real context.
|
||||
|
||||
First run (2026-10-20), with a 30-token coding prompt at low effort:
|
||||
|
||||
- Decode speed was 98.5 tokens per second.
|
||||
- MTP accepted 210 of 309 drafted tokens (68%).
|
||||
- The reasoning was 887 characters long.
|
||||
|
||||
`/v1/messages`, the Anthropic Messages API, answered with a `thinking` block and a `text` block.
|
||||
|
||||
Smoke runs (2026-10-20). Each was asked which files in `scripts/` start with `qwen`, and each made a
|
||||
tool call and answered correctly:
|
||||
|
||||
- OpenCode: 8 seconds. After the `--standalone` fix (see [Measurements](#measurements)), 4.5
|
||||
seconds with `run --auto`.
|
||||
- Claude Code, after the template fix: 14 seconds.
|
||||
|
||||
## Measurements
|
||||
|
||||
All measurements were taken on 2026-10-20, with UD-Q4_K_M at a 102,400-token context and a q8_0 KV
|
||||
cache. The figures are the server's own timings for each request (`timings` in the
|
||||
`/v1/chat/completions` response, and `print_timing` in the server log). The short prompt asked for a
|
||||
Go duration parser with table-driven tests (75 tokens, 512 tokens out).
|
||||
|
||||
| Run | Decode (tokens/s) | Prompt (tokens/s) | MTP drafts accepted | VRAM in use |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Short prompt, MTP on (3 runs) | 83.9 median (82.3–92.2) | about 180 (cold) | 58–67% | 23.5 GB |
|
||||
| Short prompt, MTP off (3 runs) | 46.5 median (46.3–46.6) | about 200 (cold) | none | 21.2 GB |
|
||||
| 94,975-token prompt of Go source, MTP on | 68.6 | 1,987 (48 s to read it) | 82% | 23.6 GB |
|
||||
|
||||
- **MTP pays.** It gives about 1.8 times the decode speed, for about 2.3 GB of VRAM. [ANALYSIS]
|
||||
- **The full context fits.** About 95K tokens of context loaded and answered, with no out-of-memory
|
||||
error. The design's floor of 64K was not needed, so the default stays at 102,400.
|
||||
- **Speed at long context.** Decode slowed to about 69 tokens per second at 95K tokens filled, which
|
||||
is still above the 40 target. The cost of a long context is in reading it: about 2,000 tokens per
|
||||
second, so a fresh 95K-token prompt takes about 48 seconds. Later turns reuse the prompt cache.
|
||||
|
||||
**Reasoning effort.** The server was restarted at each level, then given the short prompt with a
|
||||
20,000-token cap. Each level was run once, so these are rough figures:
|
||||
|
||||
| `QWEN_EFFORT` | Output tokens | Reasoning (characters) | Wall time |
|
||||
| --- | --- | --- | --- |
|
||||
| `low` | 2,635 | 4,777 | 27.0 s |
|
||||
| `medium` | 2,365 | 4,138 | 23.9 s |
|
||||
| `xhigh` (the model's default) | 6,042 | 19,928 | 67.8 s |
|
||||
|
||||
`xhigh` spent about four times the reasoning of `low`, for the same small task. `low` and `medium`
|
||||
were about the same on this prompt. [ANALYSIS]
|
||||
|
||||
**Per-request effort.** The template adds different instructions for each level, so the prompt's
|
||||
token count shows which level a request got: 75 tokens at `low`, 45 at `medium` and 87 at `xhigh`.
|
||||
|
||||
- **The OpenAI API honours it.** `"reasoning_effort": "xhigh"` in a `/v1/chat/completions` request
|
||||
overrides the server's default.
|
||||
- **OpenCode can send it.** A capture of its requests showed that a model's
|
||||
`"options": {"reasoningEffort": "xhigh"}` arrives as `reasoning_effort`. Its title requests ask for
|
||||
`low`.
|
||||
- **The Anthropic API ignores it.** `/v1/messages` gives the same prompt whatever
|
||||
`output_config.effort` or `thinking` says. Claude Code sends `effort: high` and adaptive thinking,
|
||||
and the server ignores both. For Claude Code, only the server's `QWEN_EFFORT` counts.
|
||||
|
||||
**OpenCode's background service.**
|
||||
|
||||
- OpenCode 2 starts `opencode serve --service` on first use. Later `opencode run` calls connect to it,
|
||||
and it keeps the config and environment it started with. A new `OPENCODE_CONFIG_CONTENT` was
|
||||
silently ignored, so the launcher's provider limit and credential stripping would not have applied.
|
||||
- `qwen-opencode.sh` now always passes `--standalone`, which gives each session a private server. The
|
||||
flag must come after `run` (`opencode run --standalone`), or first for the interactive session.
|
||||
- In standalone mode, a headless `run` that calls a tool waits for approval, with no way to give
|
||||
it. Pass `--auto` to approve tools in a run you trust.
|
||||
|
||||
**Only the local server got model calls.**
|
||||
|
||||
- Every model call in the smoke runs appears in the server log as a slot task.
|
||||
- The launcher tests prove that `ANTHROPIC_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN` and
|
||||
`OPENROUTER_API_KEY` never reach either tool.
|
||||
- The two capture runs, pointed at a dummy server, showed that each tool sends its model calls only
|
||||
to the configured base URL. Claude Code sent one `/v1/messages` request; OpenCode sent 13
|
||||
`/v1/chat/completions` requests.
|
||||
|
||||
Verified on 2026-10-20.
|
||||
|
||||
## Comparison
|
||||
|
||||
**The task.** The founder chose backlog item B-071 in `odin`: `Advance` had no `default` case, so a
|
||||
graph node of an unknown kind passed silently. Each run started from `odin` `bc973e3`, in a
|
||||
throwaway worktree.
|
||||
|
||||
**The setup.**
|
||||
|
||||
- All three runs got the same prompt: fix it, add a test, run `go test ./internal/engine/` (about 90
|
||||
seconds), do not commit, and do not ask questions.
|
||||
- They ran headless, by the founder's choice:
|
||||
- OpenCode with `run --auto`.
|
||||
- Claude Code with `-p --permission-mode acceptEdits`, and Bash limited to `go`, `gofmt`,
|
||||
`git diff` and `git status`.
|
||||
- The server ran at `QWEN_EFFORT=low`, with MTP on.
|
||||
|
||||
| Run | Wall time | Turns or requests | Tokens | Result |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| OpenCode + local Qwen | 508 s | 46 tool calls, 42 model requests | 131,591 prompt tokens read and 21,137 generated (server log); largest context 91,707 | `default` case + test; `go test` and `go vet` pass |
|
||||
| Claude Code + local Qwen | 551 s | 64 turns, 54 model requests | 109,008 prompt tokens read and 25,005 generated (server log); largest context 76,439 | `default` case + test; `go test` and `go vet` pass |
|
||||
| Claude Code + hosted Sonnet | 146 s | 12 turns | 20 input + 295,376 cache-read + 31,908 cache-write, and 4,403 output; $0.20 of usage | `default` case + helper + test; `go test` and `go vet` pass |
|
||||
|
||||
**How the results were checked.**
|
||||
|
||||
- `go vet ./...` and `go test ./internal/engine/` ran in each worktree after the runs.
|
||||
- Each new test was run again with `-count=1`.
|
||||
- Each new test was then run against the original `engine.go`. All three failed there with the bug's
|
||||
symptom (`Status:done`, when the test wants a failed run), so each test does catch the bug.
|
||||
|
||||
**The three fixes.**
|
||||
|
||||
- All three chose the engine's existing failure path: a `nodeFailure` that `fail()` turns into a
|
||||
failed run, with `bad_manifest` in the error.
|
||||
- **Hosted Sonnet** added a helper. It uses the `gungnir.GradeBadManifest` constant, logs a
|
||||
`node_failed` event, and repeats the node ID in the message (which `fail()` already adds).
|
||||
- **OpenCode** noticed that `fail()` adds the node ID, and left it out of its message on purpose.
|
||||
- **Claude Code on Qwen** wrote the smallest change. Both Qwen runs wrote `bad_manifest` as a
|
||||
string, not as the constant.
|
||||
|
||||
**Retries.** None of the runs needed a correction. Two earlier attempts of the local runs ended
|
||||
when the kernel's out-of-memory killer stopped `llama-server` (see [Lessons](#lessons)); those
|
||||
attempts are not counted above.
|
||||
|
||||
The diffs, the prompt and the run summary are kept in
|
||||
[2026-10-19-local-qwen-comparison/](2026-10-19-local-qwen-comparison/). The founder took no fix from these
|
||||
runs into `odin`, and B-071 stays open in the backlog. The worktrees were removed after the pull
|
||||
request.
|
||||
|
||||
Claude Code puts an "estimated cost" of $2.37 on the local run. It prices the local model's tokens
|
||||
as if they were Anthropic's; the real cost was GPU time and electricity.
|
||||
|
||||
## Lessons
|
||||
|
||||
- **Local Qwen did the real task.** Both local harnesses produced a correct, tested fix to a real
|
||||
backlog item, with a test that fails without the fix. It took about 3.5 to 4 times the hosted
|
||||
run's wall time, and no subscription usage. [ANALYSIS]
|
||||
- **The hosted model was more efficient and slightly more thorough.** It took 12 turns, against 46 to
|
||||
64 for the local model, and used the package's constant and event log. The local runs read more
|
||||
code before they acted: OpenCode's context reached 91,707 tokens, close to the 102,400 limit.
|
||||
[ANALYSIS]
|
||||
- **On this task, OpenCode and Claude Code were about even on Qwen.** OpenCode was a little faster and
|
||||
more careful about the message. Claude Code made a smaller diff. [ANALYSIS]
|
||||
- **Qwen handled superpowers well enough.** The local Claude Code had the founder's superpowers
|
||||
plugin, whose SessionStart text tells the model to use skills first. The task prompt said not to
|
||||
ask questions, and the run went straight to the work. [ANALYSIS: one run only.]
|
||||
- **System RAM, not VRAM, was the binding limit.** The GPU held the model and the context with room
|
||||
to spare. The kernel's out-of-memory killer stopped `llama-server` three times under WSL's 12 GB
|
||||
cap, because two llama-server defaults keep state in system RAM:
|
||||
- the prompt cache, `--cache-ram`, 8 GB by default;
|
||||
- for this hybrid model, 32 context checkpoints of its recurrent state, which grew the server from
|
||||
2.0 to 5.7 GB in a 20-turn test.
|
||||
|
||||
With `--cache-ram 2048` and `--ctx-checkpoints 4`, the server peaked at 3.0 GB during both local
|
||||
runs. On the founder's request, `.wslconfig` now gives WSL 20 GB (it was 12 GB). That takes effect
|
||||
after `wsl --shutdown`.
|
||||
- **Each harness had a surprise.**
|
||||
- Claude Code sends hook context as a mid-conversation `system` message, which Qwen's template
|
||||
refuses. The fix is a saved template.
|
||||
- Claude Code does not know the model's context size and assumes 200K.
|
||||
- Claude Code's effort settings are ignored by `/v1/messages`.
|
||||
- OpenCode 2's background service ignores a new config, so the launcher passes `--standalone`.
|
||||
- OpenCode's `run` waits for a tool approval that a headless run cannot give, so pass `--auto`.
|
||||
- **The remaining unfairness.** The local Claude Code shares the personal setup's three plugins,
|
||||
its skills and its merge hook. It differs from the hosted run in these ways:
|
||||
- The hosted run had the personal auto-memory and the claude.ai connectors. Neither was relevant
|
||||
to this task.
|
||||
- `~/.claude-personal/settings.json` sets `defaultMode: auto`, a permissions allowlist, and
|
||||
`effortLevel` (`high` for Opus 5.5). The local config uses `defaultMode: default` and has no
|
||||
allowlist. All three headless runs overrode the mode with `acceptEdits` and the same Bash
|
||||
allowlist. The local server ignores Claude Code's effort setting.
|
||||
- Each configuration ran once, so the timing and token figures are one sample each.
|
||||
- **The skills link is a write path.** `~/.claude-qwen/skills` links to `~/.claude-personal/skills`, so
|
||||
the local model sees the same skills, as the founder asked. A local session could change a personal
|
||||
skill through that link, for example with skill-creator, but only through an action that the
|
||||
founder approves. Copy the skills instead if that matters more than staying in sync.
|
||||
@@ -0,0 +1,626 @@
|
||||
---
|
||||
title: Local Qwen 3.8 27B coding assistant, implementation plan
|
||||
status: accepted
|
||||
date: 2026-10-19
|
||||
style: spec
|
||||
---
|
||||
|
||||
# Local Qwen 3.8 27B Coding Assistant Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Run Qwen 3.8 27B on the local RTX 4090 with `llama-server`, use it from OpenCode and from a
|
||||
separate Claude Code config, and compare both with hosted Claude on one small real task.
|
||||
|
||||
**Architecture:** Upstream llama.cpp, built with CUDA in WSL, serves the model on `127.0.0.1`. One
|
||||
script starts and stops the server. Two launchers start OpenCode and Claude Code with environment
|
||||
variables that point them only at that server. A shared library holds the defaults and the running
|
||||
server's state. A notes page holds the founder-run setup procedure, the measurements and the
|
||||
comparison.
|
||||
|
||||
**Tech Stack:** bash (shellcheck-clean, `set -euo pipefail`), llama.cpp `llama-server`, CUDA toolkit
|
||||
for WSL-Ubuntu, a Q4-class GGUF from Hugging Face, OpenCode (Homebrew), Claude Code (Homebrew,
|
||||
already installed at `/home/linuxbrew/.linuxbrew/bin/claude`), python3 only inside tests.
|
||||
|
||||
**Spec:** [`2026-10-19-local-qwen-design.md`](2026-10-19-local-qwen-design.md)
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Repo: `~/brunogbv/ouroboric/ouroboric`, branch `local-qwen`. Stage explicit paths only; never
|
||||
`git add -A` (the tree has an untracked `docs/assets/logo/` that is not ours).
|
||||
- Scripts go in `scripts/`, tests in `scripts/tests/<name>.test.sh`, in the style of
|
||||
`scripts/tests/ygg-engine.test.sh`: a temp `HOME`, fake commands, no GPU and no network. Scripts are
|
||||
AGPL-3.0-only through `REUSE.toml`; add no SPDX headers.
|
||||
- Each script opens with a comment block: what it does, one usage line per mode, the environment it
|
||||
reads, in the style of `scripts/ygg-engine.sh`.
|
||||
- The server binds `127.0.0.1` only. Default port `8088`. Model alias `qwen3.8-27b`.
|
||||
- Default context `102400` tokens; floor `65536`. KV cache `q8_0` for K and V.
|
||||
- Default reasoning effort `low`. Allowed: `low`, `medium`, `xhigh`, `off`.
|
||||
- Sampling: temperature 1.0, top_p 0.95, top_k 20, min_p 0.
|
||||
- One session at a time: `--parallel 1`.
|
||||
- Model files, the llama.cpp build and run state live in `QWEN_HOME`, default
|
||||
`$HOME/.local/share/qwen-local`, never in a repo.
|
||||
- The local Claude Code config dir is `QWEN_CLAUDE_DIR`, default `$HOME/.claude-qwen`. No script may
|
||||
write to `~/.claude`, `~/.claude.json`, `~/.claude-personal`, or `~/.config/claude-accounts.zsh`, or
|
||||
shadow the `claude`, `claude-work` or `claude-personal` commands.
|
||||
- Launchers MUST unset `ANTHROPIC_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN` and `OPENROUTER_API_KEY`.
|
||||
- `.wslconfig` stays as it is. Steps that need `sudo` are founder-run; the agent prints them and
|
||||
waits.
|
||||
- No session links in commits or the PR. Commits end with the `Co-Authored-By` line in use.
|
||||
- Lint must pass: `pre-commit run --files <changed files>` (shellcheck, markdownlint, reuse,
|
||||
gitleaks).
|
||||
|
||||
## Review Focus
|
||||
|
||||
1. **A stale pid file** after a crash or `wsl --shutdown`: `status` says "not running" and `start`
|
||||
works, without manual cleanup. (Task 2)
|
||||
2. **A reused pid:** the pid file names a live process that is not `llama-server`. `stop` MUST NOT
|
||||
kill it. (Task 2)
|
||||
3. **Another program already on port 8088:** `start` refuses with a clear message instead of
|
||||
reporting that program as the model. (Task 2)
|
||||
4. **The model does not fit in VRAM:** `llama-server` exits while loading. `start` reports failure
|
||||
and shows the end of the log, instead of waiting out the timeout. (Task 2)
|
||||
5. **Launchers with the server down, or with a cloud credential in the caller's shell:** they refuse,
|
||||
or they strip the credential, so no session reaches a hosted model by accident. (Tasks 3 and 4)
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
| File | Responsibility |
|
||||
| --- | --- |
|
||||
| `scripts/qwen-lib.sh` | sourced only: defaults, the server URL, the health check, the running server's state file |
|
||||
| `scripts/qwen-local.sh` | `start`, `stop`, `status`, `cmd` for `llama-server` |
|
||||
| `scripts/qwen-opencode.sh` | runs OpenCode against the local server |
|
||||
| `scripts/qwen-claude.sh` | `init` the local Claude Code config dir; run Claude Code against the local server |
|
||||
| `scripts/tests/qwen-local.test.sh` | tests `qwen-lib.sh` and `qwen-local.sh` |
|
||||
| `scripts/tests/qwen-opencode.test.sh` | tests `qwen-opencode.sh` |
|
||||
| `scripts/tests/qwen-claude.test.sh` | tests `qwen-claude.sh` |
|
||||
| `docs/5-research/2026-10-19-local-qwen-notes.md` | setup procedure, measurements, comparison, lessons (style `research`) |
|
||||
|
||||
---
|
||||
|
||||
### Task 0: Branch
|
||||
|
||||
- [ ] **Step 1: Create the branch and commit the design and this plan**
|
||||
|
||||
```bash
|
||||
cd ~/brunogbv/ouroboric/ouroboric
|
||||
git switch main && git pull --ff-only
|
||||
git switch -c local-qwen
|
||||
git add docs/5-research/2026-10-19-local-qwen-design.md docs/5-research/2026-10-19-local-qwen-plan.md
|
||||
git commit -m "Local Qwen 3.8 27B: design and plan"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Setup procedure and a first manual run
|
||||
|
||||
Founder-run where `sudo` is needed. This task produces the verified facts that Task 2 encodes.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `docs/5-research/2026-10-19-local-qwen-notes.md`: front matter (`title`, `status: draft`,
|
||||
`date`, `style: research`) and sections `## Setup`, `## Measurements`, `## Comparison`,
|
||||
`## Lessons`. This task fills `## Setup`.
|
||||
|
||||
- [ ] **Step 1: Print the `sudo` steps for the founder, and wait until they confirm they ran them**
|
||||
- The CUDA toolkit, from NVIDIA's `wsl-ubuntu` repository for Ubuntu 22.04, at a version no newer
|
||||
than CUDA 13.3 (the driver's user-mode version). **Install only a `cuda-toolkit-*` package, never
|
||||
`cuda`, `cuda-drivers` or any `nvidia-driver-*` package: in WSL the Windows driver provides the GPU
|
||||
driver.**
|
||||
- `cmake` and `build-essential`, from apt (or `brew install cmake`, which needs no `sudo`).
|
||||
- [ ] **Step 2: Verify the toolkit**
|
||||
|
||||
Run: `/usr/local/cuda/bin/nvcc --version && nvidia-smi --query-gpu=name,memory.total --format=csv`
|
||||
Expected: a CUDA 13.x release line, and `NVIDIA GeForce RTX 4090, 24564 MiB`.
|
||||
|
||||
- [ ] **Step 3: Build llama.cpp in `$QWEN_HOME/llama.cpp`**
|
||||
|
||||
Clone `https://github.com/ggml-org/llama.cpp`, record the commit, then
|
||||
`cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89 && cmake --build build --config Release -j 6 --target llama-server`.
|
||||
Use `-j 6` because WSL has 12 GB. If the build is killed for lack of memory, retry with `-j 3`, then
|
||||
ask the founder.
|
||||
|
||||
Run: `$QWEN_HOME/llama.cpp/build/bin/llama-server --version`
|
||||
Expected: a version line that names CUDA.
|
||||
|
||||
- [ ] **Step 4: Pick the quant and download it**
|
||||
|
||||
Look at `unsloth/Qwen3.8-27B-GGUF` (and the official Qwen GGUF repo, if there is one). Pick a
|
||||
Q4-class file of 18 GB or less: Q4_K_M, or UD-Q4_K_XL. Download it with
|
||||
`uvx --from huggingface_hub hf download <repo> <file> --local-dir $QWEN_HOME/models`. Also download
|
||||
the MTP draft file, if llama.cpp needs one for this model. Link the files:
|
||||
`$QWEN_HOME/models/model.gguf` (and `$QWEN_HOME/models/mtp.gguf`, if there is a draft file). Record
|
||||
the repo, the file names, the sizes and the reason for the choice in `## Setup`.
|
||||
|
||||
- [ ] **Step 5: Confirm the server flags against the built binary**
|
||||
|
||||
Run `llama-server --help` and confirm the spelling of every flag in Task 2's `cmd` list. In
|
||||
particular, confirm:
|
||||
|
||||
- the flash-attention flag (`-fa on`);
|
||||
- `--chat-template-kwargs`;
|
||||
- the KV cache type flags;
|
||||
- `-ngl`;
|
||||
- the MTP or speculative-decoding flags for this model.
|
||||
|
||||
Write the confirmed MTP flags as one string. That string becomes the default of `QWEN_MTP_ARGS` in
|
||||
Task 2. Willison used `--spec-type draft-mtp`; the upstream spelling may differ. If a flag in Task 2's
|
||||
list is spelled differently, correct Task 2 in this plan and say so in the commit message.
|
||||
|
||||
- [ ] **Step 6: First manual run**
|
||||
|
||||
Start the server by hand with the Task 2 flags, at context `102400` and with MTP. Then:
|
||||
|
||||
- `curl -s 127.0.0.1:8088/v1/chat/completions` with a short prompt. It must answer, with a reasoning
|
||||
part that is short at `low` effort.
|
||||
- `curl -s 127.0.0.1:8088/v1/messages` with the Anthropic body shape (`model`, `max_tokens`,
|
||||
`messages`). It must answer, which confirms the endpoint Claude Code needs.
|
||||
- Read `nvidia-smi` and record the VRAM in use.
|
||||
|
||||
If loading fails for lack of memory, reduce the context in steps of 8192 tokens, but not below
|
||||
`65536`. Record the largest value that loads. That value replaces `102400` as the default in Task 2.
|
||||
|
||||
- [ ] **Step 7: Install OpenCode, without `sudo`**
|
||||
|
||||
Run: `brew install sst/tap/opencode` (or the current Homebrew name), then `opencode --version`.
|
||||
Record the version.
|
||||
|
||||
- [ ] **Step 8: Write `## Setup` as numbered steps that the founder can repeat, then commit**
|
||||
|
||||
```bash
|
||||
git add docs/5-research/2026-10-19-local-qwen-notes.md
|
||||
git commit -m "Local Qwen: setup procedure and verified server flags"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: `qwen-lib.sh` and `qwen-local.sh`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `scripts/qwen-lib.sh`, `scripts/qwen-local.sh` (executable)
|
||||
- Test: `scripts/tests/qwen-local.test.sh` (executable)
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Produces (`qwen-lib.sh`, sourced; it sets these unless the caller already did):
|
||||
- `QWEN_HOME` (default `$HOME/.local/share/qwen-local`), `QWEN_PORT` (`8088`), `QWEN_CTX`
|
||||
(`102400`, or the value Task 1 recorded), `QWEN_ALIAS` (`qwen3.8-27b`).
|
||||
- `QWEN_STATE="$QWEN_HOME/run/server.env"`: written by `start`, removed by `stop`. It holds the
|
||||
lines `QWEN_PORT=<n>` and `QWEN_CTX=<n>` of the running server.
|
||||
- `qwen_load_state`: if `$QWEN_STATE` exists, set `QWEN_PORT` and `QWEN_CTX` from it. Read it as
|
||||
data, with `sed`; do not `source` it.
|
||||
- `qwen_url`: prints `http://127.0.0.1:$QWEN_PORT`.
|
||||
- `qwen_healthy`: exit 0 if `curl -fsS -m 2 "$(qwen_url)/health"` succeeds.
|
||||
- `qwen_require_server`: runs `qwen_load_state`, then fails with exit 2 and
|
||||
`the local model server is not running on <url>; start it with scripts/qwen-local.sh start`
|
||||
when it is not healthy.
|
||||
- Produces (`qwen-local.sh start|stop|status|cmd`). Environment: `QWEN_LLAMA_SERVER` (default
|
||||
`$QWEN_HOME/llama.cpp/build/bin/llama-server`), `QWEN_MODEL` (default
|
||||
`$QWEN_HOME/models/model.gguf`), `QWEN_EFFORT` (`low`), `QWEN_MTP_ARGS` (the string from Task 1,
|
||||
word-split; empty means no MTP), `QWEN_START_TIMEOUT` (seconds, `300`). Files: pid
|
||||
`$QWEN_HOME/run/server.pid`, log `$QWEN_HOME/logs/server.log`. The previous log is kept as
|
||||
`server.log.1`.
|
||||
|
||||
The `cmd` argument list, in this order:
|
||||
`-m $QWEN_MODEL --alias $QWEN_ALIAS --host 127.0.0.1 --port $QWEN_PORT -ngl 999 -c $QWEN_CTX -fa on --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1 --jinja --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 <reasoning> $QWEN_MTP_ARGS`.
|
||||
`<reasoning>` is `--reasoning-effort <effort>`, or `--reasoning off` for `off`. (Corrected in Task 2: the
|
||||
plan first passed `--chat-template-kwargs`; llama-server has native reasoning flags.) The default
|
||||
`QWEN_MTP_ARGS`, confirmed in Task 1, is
|
||||
`--spec-type draft-mtp -md $QWEN_HOME/models/mtp.gguf -ngld all -ctkd q8_0 -ctvd q8_0`.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests** in `scripts/tests/qwen-local.test.sh`
|
||||
|
||||
Use the `check`, `run` and `has` helpers from `ygg-engine.test.sh`. Put fakes in `$T/bin`, first on
|
||||
`PATH`:
|
||||
|
||||
- a fake `llama-server` that writes its arguments, one per line, to `$T/args`, then sleeps (or exits
|
||||
with code 1 at once when `FAKE_DIE` is set);
|
||||
- a fake `curl` that exits 0 when `$T/healthy` exists, and exits 7 otherwise.
|
||||
|
||||
`QWEN_LLAMA_SERVER` points to the fake server, `QWEN_MODEL` to an empty file, and
|
||||
`QWEN_START_TIMEOUT=3`. The cases:
|
||||
|
||||
```text
|
||||
cmd: default line contains "--host 127.0.0.1", "--port 8088", "-c 102400", "--cache-type-k q8_0",
|
||||
"--parallel 1", and "--reasoning-effort low"; MTP flags on by default
|
||||
cmd: QWEN_EFFORT=off -> contains "--reasoning off", no "--reasoning-effort"
|
||||
cmd: QWEN_EFFORT=high -> exit 2, message names QWEN_EFFORT and the four allowed values
|
||||
cmd: QWEN_MTP_ARGS="--spec-a x" -> last two args are "--spec-a" "x"; QWEN_MTP_ARGS="" -> no "--spec-a"
|
||||
start: QWEN_MODEL missing -> exit 2, message names the path; no fake server started
|
||||
start: fake becomes healthy -> exit 0, prints "ready on http://127.0.0.1:8088", pid file names a live
|
||||
process, $T/args has "-m" "<model path>", server.env has QWEN_PORT=8088 and QWEN_CTX=102400
|
||||
start: run again while running -> exit 0, "already running", still one fake process
|
||||
start: FAKE_DIE -> exit 1, prints "exited while starting" and the log tail; no pid file, no server.env
|
||||
start: never healthy -> exit 1 within ~3 s, "did not become ready", fake process killed, no pid file
|
||||
start: port already healthy and no pid file -> exit 2, "port 8088 already answers"; fake not started
|
||||
status: running -> exit 0, "running", the URL; not running -> exit 3, "not running"
|
||||
status: stale pid file (dead pid) -> exit 3, "not running", pid file and server.env removed
|
||||
stop: running -> exit 0, process gone within 30 s, pid file and server.env removed
|
||||
stop: not running -> exit 0, "not running"
|
||||
stop: pid file names a live `sleep` (not llama-server) -> exit 0, "stale", the sleep still alive,
|
||||
pid file removed
|
||||
qwen_load_state: server.env with QWEN_PORT=9000 -> qwen_url prints http://127.0.0.1:9000
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests, and check that they fail**
|
||||
|
||||
Run: `bash scripts/tests/qwen-local.test.sh`
|
||||
Expected: `FAIL` lines, because `qwen-local.sh` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement `scripts/qwen-lib.sh` and `scripts/qwen-local.sh`**
|
||||
|
||||
`qwen-local.sh` sources `qwen-lib.sh` from its own folder.
|
||||
|
||||
- Liveness means: the pid is alive, and `/proc/<pid>/cmdline` contains `llama-server`. Anything else
|
||||
is a stale pid file.
|
||||
- `start`: launch the server in the background with `nohup`, writing to the log. Poll
|
||||
`qwen_healthy` once a second, and stop polling early if the process dies.
|
||||
- `stop`: send TERM, wait up to 30 s, then send KILL.
|
||||
|
||||
- [ ] **Step 4: Run the tests and shellcheck, and check that they pass**
|
||||
|
||||
Run: `bash scripts/tests/qwen-local.test.sh && shellcheck scripts/qwen-lib.sh scripts/qwen-local.sh scripts/tests/qwen-local.test.sh`
|
||||
Expected: only `ok` lines, and no shellcheck output.
|
||||
|
||||
- [ ] **Step 5: Real smoke**
|
||||
|
||||
Run: `scripts/qwen-local.sh start && scripts/qwen-local.sh status && scripts/qwen-local.sh stop`
|
||||
Expected: `ready on http://127.0.0.1:8088`, then `running`, then a clean stop. `nvidia-smi` shows the
|
||||
VRAM freed.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add scripts/qwen-lib.sh scripts/qwen-local.sh scripts/tests/qwen-local.test.sh
|
||||
git commit -m "qwen-local.sh: start, stop and status for the local llama-server"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: `qwen-opencode.sh`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `scripts/qwen-opencode.sh` (executable)
|
||||
- Test: `scripts/tests/qwen-opencode.test.sh` (executable)
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: `qwen_require_server`, `qwen_url`, `QWEN_CTX`, `QWEN_ALIAS` from `qwen-lib.sh`.
|
||||
- Produces: `qwen-opencode.sh [OPENCODE ARGS...]`. It execs `${QWEN_OPENCODE_BIN:-opencode}` with the
|
||||
arguments unchanged, and with:
|
||||
- `OPENCODE_CONFIG_CONTENT`: the JSON below;
|
||||
- `OPENCODE_DISABLE_CLAUDE_CODE=1` and `OPENCODE_DISABLE_EXTERNAL_SKILLS=1`, so that OpenCode does
|
||||
not read `~/.claude`, which is the work account;
|
||||
- `OPENCODE_DISABLE_AUTOUPDATE=true`, `OPENCODE_DISABLE_SHARE=true` and
|
||||
`OPENCODE_DISABLE_MODELS_FETCH=true`.
|
||||
|
||||
The project's own OpenCode config stays enabled.
|
||||
|
||||
The config, as built by the script (`<url>`, `<ctx>` and `<alias>` filled in):
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"autoupdate": false,
|
||||
"share": "disabled",
|
||||
"enabled_providers": ["qwen-local"],
|
||||
"provider": {
|
||||
"qwen-local": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "Qwen local",
|
||||
"options": { "baseURL": "<url>/v1" },
|
||||
"models": { "<alias>": { "name": "Qwen 3.8 27B (local)", "tool_call": true, "reasoning": true,
|
||||
"limit": { "context": <ctx>, "output": 32768 } } }
|
||||
}
|
||||
},
|
||||
"model": "qwen-local/<alias>",
|
||||
"small_model": "qwen-local/<alias>"
|
||||
}
|
||||
```
|
||||
|
||||
Check `enabled_providers` against the installed OpenCode's config schema. If the key has a different
|
||||
name, use the schema's name, and record the change.
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
Use a fake `opencode` that writes its arguments and its full environment to files, and the fake
|
||||
`curl` from Task 2. Parse the config with `python3 -c 'import json,os; ...'`. The cases:
|
||||
|
||||
```text
|
||||
server down -> exit 2, message from qwen_require_server; fake opencode not run
|
||||
server up, server.env says QWEN_PORT=9000 QWEN_CTX=65536 -> baseURL "http://127.0.0.1:9000/v1",
|
||||
limit.context 65536, model and small_model "qwen-local/qwen3.8-27b", enabled_providers ["qwen-local"]
|
||||
args pass through: qwen-opencode.sh run "hi" -> fake got "run" "hi"
|
||||
env: the five OPENCODE_DISABLE_* variables set as above
|
||||
env: caller exports ANTHROPIC_API_KEY, CLAUDE_CODE_OAUTH_TOKEN, OPENROUTER_API_KEY -> none reach the fake
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests, and check that they fail**
|
||||
|
||||
Run: `bash scripts/tests/qwen-opencode.test.sh`
|
||||
Expected: `FAIL` lines.
|
||||
|
||||
- [ ] **Step 3: Implement `scripts/qwen-opencode.sh`**
|
||||
|
||||
Build the JSON with `printf`. Every value is a number or a fixed string, so no escaping is needed.
|
||||
Use `exec env -u … VAR=… "$bin" "$@"`.
|
||||
|
||||
- [ ] **Step 4: Run the tests and shellcheck, and check that they pass**
|
||||
|
||||
Run: `bash scripts/tests/qwen-opencode.test.sh && shellcheck scripts/qwen-opencode.sh scripts/tests/qwen-opencode.test.sh`
|
||||
Expected: only `ok` lines, and no shellcheck output.
|
||||
|
||||
- [ ] **Step 5: Real smoke**
|
||||
|
||||
With the server running, in `~/brunogbv/ouroboric/ouroboric`, run:
|
||||
`scripts/qwen-opencode.sh run "List the files in scripts/ that start with qwen, using a tool"`.
|
||||
|
||||
Expected: an answer that names the qwen scripts, after at least one tool call. The server log shows
|
||||
the requests. Record the OpenCode version and any setting that needed a change in `## Setup`.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add scripts/qwen-opencode.sh scripts/tests/qwen-opencode.test.sh docs/5-research/2026-10-19-local-qwen-notes.md
|
||||
git commit -m "qwen-opencode.sh: OpenCode on the local Qwen server"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: `qwen-claude.sh`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `scripts/qwen-claude.sh` (executable)
|
||||
- Test: `scripts/tests/qwen-claude.test.sh` (executable)
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: `qwen_require_server`, `qwen_url`, `QWEN_ALIAS`.
|
||||
- Produces: `qwen-claude.sh init` and `qwen-claude.sh [CLAUDE ARGS...]`.
|
||||
- Environment: `QWEN_CLAUDE_DIR` (default `$HOME/.claude-qwen`), `QWEN_CLAUDE_BIN` (default: the
|
||||
`claude` binary that `command -v claude` finds in bash, which is not the zsh function),
|
||||
`QWEN_PERSONAL_DIR` (default `$HOME/.claude-personal`; it is only read).
|
||||
- `init` refuses with exit 2 when `QWEN_CLAUDE_DIR` resolves to `$HOME/.claude`, to
|
||||
`$QWEN_PERSONAL_DIR`, or to a path inside either one.
|
||||
- `init` writes `$QWEN_CLAUDE_DIR/settings.json` (it regenerates the file each time) and
|
||||
`$QWEN_CLAUDE_DIR/.claude.json` with `{"hasCompletedOnboarding": true}`, but only if that file
|
||||
does not exist yet.
|
||||
- `init` links `$QWEN_CLAUDE_DIR/skills` to `$QWEN_PERSONAL_DIR/skills`.
|
||||
- `init` then runs, with `CLAUDE_CONFIG_DIR=$QWEN_CLAUDE_DIR`:
|
||||
`claude plugin marketplace add anthropics/claude-plugins-official`, and
|
||||
`claude plugin install <p>@claude-plugins-official` for each of `superpowers`, `gopls-lsp` and
|
||||
`skill-creator`. Check these subcommands against `claude --help` on the installed version.
|
||||
- Run mode requires the server and `settings.json`. Otherwise it fails with exit 2 and
|
||||
`run scripts/qwen-claude.sh init first`.
|
||||
- Run mode then execs the binary with the arguments unchanged. It unsets `ANTHROPIC_API_KEY`,
|
||||
`CLAUDE_CODE_OAUTH_TOKEN` and `OPENROUTER_API_KEY`, and sets:
|
||||
- `CLAUDE_CONFIG_DIR=$QWEN_CLAUDE_DIR`
|
||||
- `ANTHROPIC_BASE_URL=$(qwen_url)`
|
||||
- `ANTHROPIC_AUTH_TOKEN=local-qwen`
|
||||
- `ANTHROPIC_MODEL`, `ANTHROPIC_DEFAULT_OPUS_MODEL`, `ANTHROPIC_DEFAULT_SONNET_MODEL`,
|
||||
`ANTHROPIC_DEFAULT_HAIKU_MODEL` and `CLAUDE_CODE_SUBAGENT_MODEL`, all set to `$QWEN_ALIAS`
|
||||
- `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` and `DISABLE_AUTOUPDATER=1`
|
||||
|
||||
`settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"permissions": { "defaultMode": "default" },
|
||||
"model": "qwen3.8-27b",
|
||||
"enabledPlugins": {
|
||||
"superpowers@claude-plugins-official": true,
|
||||
"gopls-lsp@claude-plugins-official": true,
|
||||
"skill-creator@claude-plugins-official": true
|
||||
},
|
||||
"extraKnownMarketplaces": {
|
||||
"claude-plugins-official": { "source": { "source": "github", "repo": "anthropics/claude-plugins-official" } }
|
||||
},
|
||||
"hooks": {
|
||||
"PreToolUse": [ { "matcher": "Bash", "hooks": [
|
||||
{ "type": "command", "command": "<QWEN_PERSONAL_DIR>/hooks/merge-requires-green.sh" } ] } ]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
Use a temp `HOME` that holds a `.claude-personal` with `settings.json`, `skills/a/SKILL.md` and
|
||||
`hooks/merge-requires-green.sh`, plus a `.claude/settings.json`. Use a fake `claude` that appends its
|
||||
arguments, `CLAUDE_CONFIG_DIR` and its full environment to files, and the fake `curl` from Task 2. The
|
||||
cases:
|
||||
|
||||
```text
|
||||
init: writes settings.json with the three plugins, defaultMode "default", the hook path under
|
||||
$HOME/.claude-personal; .claude.json has hasCompletedOnboarding true; skills is a link to
|
||||
$HOME/.claude-personal/skills; fake claude got the marketplace add and three plugin installs,
|
||||
each with CLAUDE_CONFIG_DIR=$HOME/.claude-qwen
|
||||
init: an existing .claude.json is left as it is
|
||||
init: QWEN_CLAUDE_DIR=$HOME/.claude-personal -> exit 2, nothing written, fake not run
|
||||
init: QWEN_CLAUDE_DIR=$HOME/.claude/sub -> exit 2
|
||||
init + run: sha256 of every file under $HOME/.claude-personal and $HOME/.claude unchanged
|
||||
run: before init -> exit 2 "init first"
|
||||
run: server down -> exit 2, fake not run
|
||||
run: args pass through; env has ANTHROPIC_BASE_URL=http://127.0.0.1:8088, CLAUDE_CONFIG_DIR=$HOME/.claude-qwen,
|
||||
ANTHROPIC_MODEL=qwen3.8-27b, CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
|
||||
run: caller exports ANTHROPIC_API_KEY, CLAUDE_CODE_OAUTH_TOKEN, OPENROUTER_API_KEY -> none reach the fake
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests, and check that they fail**
|
||||
|
||||
Run: `bash scripts/tests/qwen-claude.test.sh`
|
||||
Expected: `FAIL` lines.
|
||||
|
||||
- [ ] **Step 3: Implement `scripts/qwen-claude.sh`**
|
||||
|
||||
Resolve the paths with `realpath -m` before the refusal check. Build `settings.json` with `printf`;
|
||||
the only variable value is the hook path, and it has no quotes in it.
|
||||
|
||||
- [ ] **Step 4: Run the tests and shellcheck, and check that they pass**
|
||||
|
||||
Run: `bash scripts/tests/qwen-claude.test.sh && shellcheck scripts/qwen-claude.sh scripts/tests/qwen-claude.test.sh`
|
||||
Expected: only `ok` lines, and no shellcheck output.
|
||||
|
||||
- [ ] **Step 5: Real smoke**
|
||||
|
||||
1. Run `scripts/qwen-claude.sh init`.
|
||||
2. With the server running, in `~/brunogbv/ouroboric/ouroboric`, run
|
||||
`scripts/qwen-claude.sh -p "Which scripts in scripts/ start with qwen? Use a tool."`.
|
||||
3. Check the answer. Expected: it names the qwen scripts. No login prompt appears, and
|
||||
`ls ~/.claude-qwen/plugins` shows the three plugins.
|
||||
4. Check the server log. Expected: the server log shows `/v1/messages` requests.
|
||||
5. Check the personal config. Expected: `claude-whoami` in `~/brunogbv` still reports the personal
|
||||
account.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add scripts/qwen-claude.sh scripts/tests/qwen-claude.test.sh
|
||||
git commit -m "qwen-claude.sh: Claude Code on the local Qwen server, in its own config dir"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Measurements
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/5-research/2026-10-19-local-qwen-notes.md` (`## Measurements`)
|
||||
- Modify, only if the measurements say so: `scripts/qwen-lib.sh` (the `QWEN_CTX` default) and its
|
||||
test
|
||||
|
||||
- [ ] **Step 1: Decode speed at a near-empty context**
|
||||
|
||||
1. With MTP on, send three `/v1/chat/completions` requests. Each has a coding prompt of about 200
|
||||
tokens and `max_tokens: 512`.
|
||||
2. Read `timings.predicted_per_second` and `timings.prompt_per_second` from each response.
|
||||
3. Repeat with `QWEN_MTP_ARGS=""`.
|
||||
4. Record the median of each run, and the draft acceptance rate from the log.
|
||||
|
||||
- [ ] **Step 2: Speed with about 100K tokens filled**
|
||||
|
||||
Build a prompt of about 95,000 tokens from real source files (for example, the contents of
|
||||
`~/brunogbv/ouroboric/yggdrasil/gleipnir`, repeated). Send it with `max_tokens: 256`, with MTP on.
|
||||
Record the prompt speed, the decode speed and the VRAM in use. If the request fails for lack of
|
||||
memory, lower `QWEN_CTX` in steps of 8192 tokens, but not below `65536`. Record the result.
|
||||
|
||||
- [ ] **Step 3: Reasoning effort**
|
||||
|
||||
Send the same prompt at `QWEN_EFFORT=low`, `medium` and `xhigh` (restart the server each time).
|
||||
Record the reasoning tokens and the wall time.
|
||||
|
||||
Then test per-request control:
|
||||
|
||||
- OpenCode: does a model `options` entry, or a variant with `reasoning_effort`, reach the server?
|
||||
Check the server log.
|
||||
- Claude Code: does a thinking or effort setting change the request that the server gets?
|
||||
|
||||
Record yes or no for each.
|
||||
|
||||
- [ ] **Step 4: Check that only the local server gets model calls**
|
||||
|
||||
During the Task 4 smoke and the Task 6 runs, every model call appears in the server log, and neither
|
||||
launcher's environment holds a cloud credential (the tests prove this). Record "verified on YYYY-MM-DD".
|
||||
|
||||
- [ ] **Step 5: If the measured context differs from the default**
|
||||
|
||||
Change the `QWEN_CTX` default in `qwen-lib.sh` and the matching test case. Then run all three test
|
||||
files.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/5-research/2026-10-19-local-qwen-notes.md # plus qwen-lib.sh and its test, if changed
|
||||
git commit -m "Local Qwen: measured speed, context and reasoning effort"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Three-way comparison
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/5-research/2026-10-19-local-qwen-notes.md` (`## Comparison`, `## Lessons`)
|
||||
|
||||
- [ ] **Step 1: Propose the task, and wait for the founder's choice**
|
||||
|
||||
Propose two or three small, real tasks from a backlog in the founder's repos, for example
|
||||
`yggdrasil/docs/2-engineering/backlog.md` or a small script fix in `ouroboric`. A good task touches
|
||||
one or two files and has a check that can pass or fail. Ask the founder:
|
||||
|
||||
- which task to use;
|
||||
- which hosted model to use for the baseline (their default, or Sonnet, which costs less).
|
||||
|
||||
Write down the one prompt that all three runs get. It ends with:
|
||||
"Do not ask clarifying questions; make reasonable choices and say what you chose."
|
||||
|
||||
- [ ] **Step 2: Make three throwaway worktrees from the same commit**
|
||||
|
||||
Run in the target repo: `git worktree add ../<repo>-cmp-<tool> -b cmp-<tool>`, for `<tool>` =
|
||||
`opencode`, `qwen-claude` and `hosted`.
|
||||
|
||||
- [ ] **Step 3: Run the task three times, one after the other**
|
||||
|
||||
1. Run `scripts/qwen-opencode.sh` in the opencode worktree.
|
||||
2. Run `scripts/qwen-claude.sh` in the qwen-claude worktree.
|
||||
3. Run `claude-personal --model <chosen>` in the hosted worktree.
|
||||
|
||||
All three runs are interactive, in the default ask-before-acting mode, with the founder approving
|
||||
each step. For each run, record:
|
||||
|
||||
- the wall time;
|
||||
- the tokens: the server log for the local runs, `/cost` or the session file for hosted;
|
||||
- the turns and the retries: failed tool calls, or the founder correcting it;
|
||||
- whether the task's check passed;
|
||||
- the founder's verdict on the diff.
|
||||
|
||||
- [ ] **Step 4: Write `## Comparison` and `## Lessons`**
|
||||
|
||||
`## Comparison` holds a table with one row per run. `## Lessons` covers:
|
||||
|
||||
- the settings that mattered;
|
||||
- where each harness struggled, including how Qwen handled the superpowers process;
|
||||
- the remaining unfairness: the hosted run had the personal memory and the claude.ai connectors.
|
||||
|
||||
Mark analysis `[ANALYSIS]`, as the `research` style requires.
|
||||
|
||||
- [ ] **Step 5: Remove the worktrees, after the founder has judged the diffs and says so**
|
||||
|
||||
For each worktree, run `git worktree remove ../<repo>-cmp-<tool>` and then `git branch -D cmp-<tool>`.
|
||||
Run one short command per worktree.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/5-research/2026-10-19-local-qwen-notes.md
|
||||
git commit -m "Local Qwen: three-way comparison and lessons"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Pull request
|
||||
|
||||
- [ ] **Step 1: Run the whole check**
|
||||
|
||||
```bash
|
||||
for t in qwen-local qwen-opencode qwen-claude; do bash scripts/tests/$t.test.sh || echo "FAILED $t"; done
|
||||
pre-commit run --files scripts/qwen-*.sh scripts/tests/qwen-*.test.sh docs/5-research/2026-10-19-local-qwen-*.md
|
||||
```
|
||||
|
||||
Expected: only `ok` lines, no `FAILED` line, and every hook `Passed`.
|
||||
|
||||
- [ ] **Step 2: Set the design and the notes to `status: accepted`, then push and open the PR**
|
||||
|
||||
Run `scripts/as-bot.sh ouroboric git push -u origin local-qwen`, then
|
||||
`scripts/as-bot.sh ouroboric gh pr create --title "Local Qwen 3.8 27B: scripts, setup and comparison" --body-file <file>`.
|
||||
|
||||
The body names the design, the plan and the notes. It sums up the measurements and the comparison,
|
||||
and ends with the attribution line. It holds no session links.
|
||||
|
||||
- [ ] **Step 3: After the founder merges, return to main**
|
||||
|
||||
Run `git switch main && git pull --ff-only`.
|
||||
Executable
+47
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env python3
|
||||
# A stand-in model server that records what a coding agent sends, for debugging the local Qwen setup
|
||||
# (docs/5-research/2026-10-19-local-qwen-notes.md): it found Claude Code's mid-conversation system message
|
||||
# and OpenCode's ignored config. Run it as `python3 -I`.
|
||||
# qwen-capture.py PORT OUTDIR
|
||||
# It listens on 127.0.0.1:PORT and saves each POST body to OUTDIR/<nn>-<path>.json, numbered from 00.
|
||||
# Each POST gets HTTP 500 with an API error, so the agent stops instead of waiting; a GET (a health check)
|
||||
# gets 200. Point the agent at it, for example ANTHROPIC_BASE_URL=http://127.0.0.1:PORT, or an OpenCode
|
||||
# provider baseURL of http://127.0.0.1:PORT/v1. Stop it with Ctrl-C.
|
||||
import http.server
|
||||
import itertools
|
||||
import os
|
||||
import sys
|
||||
|
||||
if len(sys.argv) != 3:
|
||||
sys.exit("usage: qwen-capture.py PORT OUTDIR")
|
||||
port, out = int(sys.argv[1]), sys.argv[2]
|
||||
os.makedirs(out, exist_ok=True)
|
||||
seq = itertools.count()
|
||||
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def do_POST(self):
|
||||
body = self.rfile.read(int(self.headers.get("content-length", 0)))
|
||||
name = self.path.split("?")[0].strip("/").replace("/", "_") or "root"
|
||||
path = os.path.join(out, f"{next(seq):02d}-{name}.json")
|
||||
with open(path, "wb") as f:
|
||||
f.write(body)
|
||||
print(f"saved {path}", flush=True)
|
||||
data = b'{"type": "error", "error": {"type": "api_error", "message": "captured by qwen-capture.py"}}'
|
||||
self.send_response(500)
|
||||
self.send_header("content-type", "application/json")
|
||||
self.send_header("content-length", str(len(data)))
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
|
||||
def do_GET(self):
|
||||
self.send_response(200)
|
||||
self.send_header("content-length", "2")
|
||||
self.end_headers()
|
||||
self.wfile.write(b"{}")
|
||||
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
|
||||
http.server.HTTPServer(("127.0.0.1", port), Handler).serve_forever()
|
||||
Executable
+65
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run Claude Code against the local Qwen 3.8 27B server, in a config dir of its own that has no
|
||||
# account (docs/5-research/2026-10-19-local-qwen-design.md). Start the server first: qwen-local.sh start.
|
||||
# qwen-claude.sh init set up the config dir: the personal setup's plugins, skills and
|
||||
# merge hook, without its account, memory or account guard
|
||||
# qwen-claude.sh [CLAUDE ARGS...] run Claude Code on the local model
|
||||
# Environment, besides qwen-lib.sh's:
|
||||
# QWEN_CLAUDE_DIR the local config dir ($HOME/.claude-qwen); never the work or personal one
|
||||
# QWEN_PERSONAL_DIR the personal config dir, only read ($HOME/.claude-personal)
|
||||
# QWEN_CLAUDE_BIN the claude binary (the one on PATH; the zsh `claude` function is not used)
|
||||
# Claude Code does not know this model, so CLAUDE_CODE_MAX_CONTEXT_TOKENS tells it the server's context.
|
||||
# It never writes to ~/.claude, ~/.claude.json or the personal config dir, and removes the cloud
|
||||
# credentials of the caller's shell, so a session cannot reach a hosted model by accident.
|
||||
set -euo pipefail
|
||||
# shellcheck source-path=SCRIPTDIR source=qwen-lib.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/qwen-lib.sh"
|
||||
|
||||
dir="${QWEN_CLAUDE_DIR:-$HOME/.claude-qwen}"
|
||||
personal="${QWEN_PERSONAL_DIR:-$HOME/.claude-personal}"
|
||||
bin="${QWEN_CLAUDE_BIN:-$(command -v claude || true)}"
|
||||
[ -n "$bin" ] || { echo "claude is not on PATH; set QWEN_CLAUDE_BIN" >&2; exit 2; }
|
||||
plugins=(superpowers gopls-lsp skill-creator)
|
||||
market=claude-plugins-official
|
||||
|
||||
# inside DIR ROOT: DIR is ROOT or a path under it, after resolving links.
|
||||
inside() {
|
||||
local d r
|
||||
d="$(realpath -m "$1")" r="$(realpath -m "$2")"
|
||||
[ "$d" = "$r" ] || [[ "$d" == "$r"/* ]]
|
||||
}
|
||||
|
||||
# Both modes refuse a config dir that is, or is inside, a real account's config dir, and one that holds
|
||||
# them (such as $HOME, where ~/.claude.json lives): Claude Code would write that account's files.
|
||||
for r in "$HOME/.claude" "$HOME/.claude-personal" "$personal"; do
|
||||
if inside "$dir" "$r" || inside "$r" "$dir"; then
|
||||
echo "QWEN_CLAUDE_DIR $dir overlaps the account config dir $r; the local config dir must be separate" >&2
|
||||
exit 2
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "${1:-}" = init ]; then
|
||||
mkdir -p "$dir"
|
||||
enabled="" sep=""
|
||||
for p in "${plugins[@]}"; do enabled+="$sep\"$p@$market\":true"; sep=","; done
|
||||
printf '{"permissions":{"defaultMode":"default"},"model":"%s","enabledPlugins":{%s},"extraKnownMarketplaces":{"%s":{"source":{"source":"github","repo":"anthropics/%s"}}},"hooks":{"PreToolUse":[{"matcher":"Bash","hooks":[{"type":"command","command":"%s/hooks/merge-requires-green.sh"}]}]}}\n' \
|
||||
"$QWEN_ALIAS" "$enabled" "$market" "$market" "$personal" > "$dir/settings.json"
|
||||
[ -e "$dir/.claude.json" ] || echo '{"hasCompletedOnboarding":true}' > "$dir/.claude.json"
|
||||
[ -L "$dir/skills" ] || ln -s "$personal/skills" "$dir/skills"
|
||||
CLAUDE_CONFIG_DIR="$dir" "$bin" plugin marketplace add "anthropics/$market"
|
||||
for p in "${plugins[@]}"; do CLAUDE_CONFIG_DIR="$dir" "$bin" plugin install "$p@$market"; done
|
||||
echo "set up $dir; run scripts/qwen-claude.sh to start a session"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
[ -f "$dir/settings.json" ] || { echo "$dir is not set up; run scripts/qwen-claude.sh init first" >&2; exit 2; }
|
||||
qwen_require_server
|
||||
# The Bedrock, Vertex and Foundry switches would send the session to a hosted model, past the base URL.
|
||||
exec env -u ANTHROPIC_API_KEY -u CLAUDE_CODE_OAUTH_TOKEN -u OPENROUTER_API_KEY \
|
||||
-u CLAUDE_CODE_USE_BEDROCK -u CLAUDE_CODE_USE_VERTEX -u CLAUDE_CODE_USE_FOUNDRY \
|
||||
-u ANTHROPIC_BEDROCK_BASE_URL -u ANTHROPIC_VERTEX_BASE_URL \
|
||||
CLAUDE_CONFIG_DIR="$dir" ANTHROPIC_BASE_URL="$(qwen_url)" ANTHROPIC_AUTH_TOKEN=local-qwen \
|
||||
ANTHROPIC_MODEL="$QWEN_ALIAS" ANTHROPIC_DEFAULT_OPUS_MODEL="$QWEN_ALIAS" ANTHROPIC_DEFAULT_SONNET_MODEL="$QWEN_ALIAS" \
|
||||
ANTHROPIC_DEFAULT_HAIKU_MODEL="$QWEN_ALIAS" CLAUDE_CODE_SUBAGENT_MODEL="$QWEN_ALIAS" \
|
||||
CLAUDE_CODE_MAX_CONTEXT_TOKENS="$QWEN_CTX" CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 DISABLE_AUTOUPDATER=1 \
|
||||
"$bin" "$@"
|
||||
Executable
+88
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run one coding task three ways and keep what each did: OpenCode on the local Qwen server, Claude Code on
|
||||
# the local Qwen server, and Claude Code on a hosted model (docs/5-research/2026-10-19-local-qwen-notes.md,
|
||||
# "Comparison"). The runs are headless, one after the other, each in a throwaway worktree of REPO at REV.
|
||||
# qwen-compare.sh --repo REPO --commit REV --prompt FILE --out DIR
|
||||
# [--only opencode,qwen-claude,hosted] [--hosted-model MODEL] [--allow PATTERN]...
|
||||
# Tools are approved without asking: OpenCode gets `run --auto`, and Claude Code gets acceptEdits with the
|
||||
# --allow Bash patterns (by default Go's: "Bash(go *)", "Bash(GOWORK=off go *)", "Bash(gofmt *)", plus
|
||||
# "Bash(git diff*)" and "Bash(git status*)"). The worktrees are throwaway, but the agents run on this host.
|
||||
# Start the server first (qwen-local.sh start) for the local runs. The hosted run uses the personal config
|
||||
# dir (QWEN_HOSTED_CONFIG_DIR, $HOME/.claude-personal) and MODEL (sonnet), so it uses that subscription.
|
||||
# DIR, which must be new or empty, gets for each run <name>.out and <name>.err (the tool's output),
|
||||
# <name>.patch (its diff, new files included) and <name>.server.log (the server log lines written during the
|
||||
# run); summary.tsv (name, exit code, wall seconds); and rss.log (llama-server's memory every 30 seconds).
|
||||
# The worktrees stay in DIR/wt-<name> for review; the cleanup commands are printed at the end.
|
||||
# QWEN_COMPARE_TIMEOUT caps each run (1800 seconds). QWEN_COMPARE_OPENCODE, QWEN_COMPARE_QWEN_CLAUDE and
|
||||
# QWEN_CLAUDE_BIN override the three commands.
|
||||
set -euo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# shellcheck source-path=SCRIPTDIR source=qwen-lib.sh
|
||||
. "$HERE/qwen-lib.sh"
|
||||
usage() { echo "usage: qwen-compare.sh --repo REPO --commit REV --prompt FILE --out DIR [--only LIST] [--hosted-model MODEL] [--allow PATTERN]..." >&2; exit 2; }
|
||||
|
||||
repo="" rev="" prompt="" out="" only="opencode,qwen-claude,hosted" model="sonnet" allow=()
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--repo) repo="${2:?}"; shift 2 ;;
|
||||
--commit) rev="${2:?}"; shift 2 ;;
|
||||
--prompt) prompt="${2:?}"; shift 2 ;;
|
||||
--out) out="${2:?}"; shift 2 ;;
|
||||
--only) only="${2:?}"; shift 2 ;;
|
||||
--hosted-model) model="${2:?}"; shift 2 ;;
|
||||
--allow) allow+=("${2:?}"); shift 2 ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
done
|
||||
[ -n "$repo" ] && [ -n "$rev" ] && [ -n "$prompt" ] && [ -n "$out" ] || usage
|
||||
[ ${#allow[@]} -gt 0 ] || allow=("Bash(go *)" "Bash(GOWORK=off go *)" "Bash(gofmt *)" "Bash(git diff*)" "Bash(git status*)")
|
||||
IFS=, read -r -a runs <<< "$only"
|
||||
for r in "${runs[@]}"; do
|
||||
case "$r" in opencode | qwen-claude | hosted) ;; *) echo "unknown run '$r'; use opencode, qwen-claude or hosted" >&2; usage ;; esac
|
||||
done
|
||||
[ -f "$prompt" ] || { echo "the prompt file $prompt does not exist" >&2; exit 2; }
|
||||
if [ -d "$out" ] && [ -n "$(ls -A "$out")" ]; then echo "--out $out is not empty; give a new folder" >&2; exit 2; fi
|
||||
mkdir -p "$out"
|
||||
out="$(cd "$out" && pwd)"
|
||||
text="$(cat "$prompt")"
|
||||
log="$QWEN_HOME/logs/server.log"
|
||||
stamp="$(date +%Y%m%d%H%M%S)-$$"
|
||||
claude_flags=(--permission-mode acceptEdits --allowedTools "${allow[@]}" --output-format json)
|
||||
|
||||
# Sample llama-server's memory while the runs go, if it is running.
|
||||
pidf="$QWEN_HOME/run/server.pid"
|
||||
if [ -f "$pidf" ] && [ -r "/proc/$(cat "$pidf")/status" ]; then
|
||||
spid="$(cat "$pidf")"
|
||||
( while [ -r "/proc/$spid/status" ]; do
|
||||
echo "$(date +%T) $(awk '/VmRSS/ {printf "%.2f GB", $2 / 1048576}' "/proc/$spid/status")" >> "$out/rss.log"
|
||||
sleep 30
|
||||
done ) &
|
||||
sampler=$!
|
||||
trap 'kill "$sampler" 2>/dev/null || true' EXIT
|
||||
fi
|
||||
|
||||
for r in "${runs[@]}"; do
|
||||
wt="$out/wt-$r"
|
||||
git -C "$repo" worktree add -q -b "cmp-$r-$stamp" "$wt" "$rev"
|
||||
case "$r" in
|
||||
opencode) cmd=("${QWEN_COMPARE_OPENCODE:-$HERE/qwen-opencode.sh}" run --auto --format json "$text") ;;
|
||||
qwen-claude) cmd=("${QWEN_COMPARE_QWEN_CLAUDE:-$HERE/qwen-claude.sh}" -p "$text" "${claude_flags[@]}") ;;
|
||||
hosted) cmd=(env -u ANTHROPIC_BASE_URL -u ANTHROPIC_AUTH_TOKEN -u ANTHROPIC_MODEL -u CLAUDE_CODE_MAX_CONTEXT_TOKENS
|
||||
CLAUDE_CONFIG_DIR="${QWEN_HOSTED_CONFIG_DIR:-$HOME/.claude-personal}"
|
||||
"${QWEN_CLAUDE_BIN:-$(command -v claude)}" --model "$model" -p "$text" "${claude_flags[@]}") ;;
|
||||
esac
|
||||
before=0
|
||||
[ ! -f "$log" ] || before="$(wc -l < "$log")"
|
||||
t0="$(date +%s)"
|
||||
rc=0
|
||||
(cd "$wt" && timeout "${QWEN_COMPARE_TIMEOUT:-1800}" "${cmd[@]}") > "$out/$r.out" 2> "$out/$r.err" < /dev/null || rc=$?
|
||||
printf '%s\t%s\t%s\n' "$r" "$rc" "$(($(date +%s) - t0))" | tee -a "$out/summary.tsv"
|
||||
[ ! -f "$log" ] || tail -n +"$((before + 1))" "$log" > "$out/$r.server.log"
|
||||
git -C "$wt" add -A
|
||||
git -C "$wt" diff --cached > "$out/$r.patch"
|
||||
done
|
||||
|
||||
echo "Results in $out. Review the worktrees, then remove them:"
|
||||
for r in "${runs[@]}"; do
|
||||
echo " git -C $repo worktree remove --force $out/wt-$r && git -C $repo branch -D cmp-$r-$stamp"
|
||||
done
|
||||
@@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env bash
|
||||
# Shared defaults and helpers for the qwen-* scripts, which run Qwen 3.8 27B on the local GPU
|
||||
# (docs/5-research/2026-10-19-local-qwen-design.md). Source this file; do not run it.
|
||||
# The caller's values win over these defaults:
|
||||
# QWEN_HOME model files, the llama.cpp build, logs and run state ($HOME/.local/share/qwen-local)
|
||||
# QWEN_PORT the server's port on 127.0.0.1 (8088)
|
||||
# QWEN_CTX the context size in tokens (102400)
|
||||
# QWEN_ALIAS the model name the server answers to (qwen3.8-27b)
|
||||
# QWEN_STATE, written by `qwen-local.sh start` and removed by `stop`, holds the running server's
|
||||
# QWEN_PORT and QWEN_CTX, so that the launchers use the values the server really has.
|
||||
|
||||
QWEN_HOME="${QWEN_HOME:-$HOME/.local/share/qwen-local}"
|
||||
QWEN_PORT="${QWEN_PORT:-8088}"
|
||||
QWEN_CTX="${QWEN_CTX:-102400}"
|
||||
QWEN_ALIAS="${QWEN_ALIAS:-qwen3.8-27b}"
|
||||
QWEN_STATE="$QWEN_HOME/run/server.env"
|
||||
|
||||
# qwen_load_state: take QWEN_PORT and QWEN_CTX from the running server's state file, if there is one.
|
||||
# The file is read as data, never sourced.
|
||||
qwen_load_state() {
|
||||
local v
|
||||
[ -f "$QWEN_STATE" ] || return 0
|
||||
v="$(sed -n 's/^QWEN_PORT=\([0-9][0-9]*\)$/\1/p' "$QWEN_STATE" | head -n 1)"
|
||||
[ -z "$v" ] || QWEN_PORT="$v"
|
||||
v="$(sed -n 's/^QWEN_CTX=\([0-9][0-9]*\)$/\1/p' "$QWEN_STATE" | head -n 1)"
|
||||
[ -z "$v" ] || QWEN_CTX="$v"
|
||||
}
|
||||
|
||||
qwen_url() { echo "http://127.0.0.1:$QWEN_PORT"; }
|
||||
|
||||
qwen_healthy() { curl -fsS -m 2 "$(qwen_url)/health" >/dev/null 2>&1; }
|
||||
|
||||
qwen_require_server() {
|
||||
qwen_load_state
|
||||
if ! qwen_healthy; then
|
||||
echo "the local model server is not running on $(qwen_url); start it with scripts/qwen-local.sh start" >&2
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
Executable
+147
@@ -0,0 +1,147 @@
|
||||
#!/usr/bin/env bash
|
||||
# Start, stop and check llama-server serving Qwen 3.8 27B on 127.0.0.1, all of it on the GPU
|
||||
# (docs/5-research/2026-10-19-local-qwen-design.md; setup in docs/5-research/2026-10-19-local-qwen-notes.md).
|
||||
# qwen-local.sh start start the server in the background; wait until it answers /health
|
||||
# qwen-local.sh stop stop it (TERM, then KILL after 30 seconds)
|
||||
# qwen-local.sh status exit 0 and print the URL when it runs, exit 3 when it does not
|
||||
# qwen-local.sh cmd print the server command line, and run nothing
|
||||
# qwen-local.sh template save the running model's chat template, changed so that a system message
|
||||
# after the first one is rendered instead of refused (Claude Code sends one,
|
||||
# for hook context); start uses the saved file from then on
|
||||
# Environment, besides qwen-lib.sh's QWEN_HOME, QWEN_PORT, QWEN_CTX and QWEN_ALIAS:
|
||||
# QWEN_LLAMA_SERVER the binary ($QWEN_HOME/llama.cpp/build/bin/llama-server)
|
||||
# QWEN_MODEL the GGUF file ($QWEN_HOME/models/model.gguf)
|
||||
# QWEN_EFFORT reasoning effort: low (default), medium, xhigh, or off (no thinking)
|
||||
# QWEN_MTP_ARGS the multi-token prediction flags, split on spaces; empty turns MTP off. The default
|
||||
# drafts with $QWEN_HOME/models/mtp.gguf, all on the GPU
|
||||
# QWEN_CACHE_RAM the most MiB of system RAM for llama-server's prompt cache (2048). Its own default,
|
||||
# 8192, grew past WSL's 12 GB memory cap in an agent session, and the kernel's
|
||||
# out-of-memory killer stopped the server
|
||||
# QWEN_CTX_CHECKPOINTS context checkpoints kept per slot (4). This hybrid model saves its recurrent
|
||||
# state in system RAM at each checkpoint; the default, 32, grew the server by about
|
||||
# 4 GB over a long session. Fewer means more prompt re-reading after a rewind
|
||||
# QWEN_START_TIMEOUT seconds to wait for /health (300: loading 17 GB takes a while)
|
||||
# QWEN_CHAT_TEMPLATE the saved chat template ($QWEN_HOME/models/chat-template.jinja), used when it exists
|
||||
# The log is $QWEN_HOME/logs/server.log; the previous one is kept as server.log.1.
|
||||
set -euo pipefail
|
||||
# shellcheck source-path=SCRIPTDIR source=qwen-lib.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/qwen-lib.sh"
|
||||
usage() { echo "usage: qwen-local.sh start|stop|status|cmd|template" >&2; exit 2; }
|
||||
[ $# -eq 1 ] || usage
|
||||
|
||||
bin="${QWEN_LLAMA_SERVER:-$QWEN_HOME/llama.cpp/build/bin/llama-server}"
|
||||
model="${QWEN_MODEL:-$QWEN_HOME/models/model.gguf}"
|
||||
effort="${QWEN_EFFORT:-low}"
|
||||
mtp="${QWEN_MTP_ARGS---spec-type draft-mtp -md $QWEN_HOME/models/mtp.gguf -ngld all -ctkd q8_0 -ctvd q8_0}"
|
||||
timeout="${QWEN_START_TIMEOUT:-300}"
|
||||
tplf="${QWEN_CHAT_TEMPLATE:-$QWEN_HOME/models/chat-template.jinja}"
|
||||
pidf="$QWEN_HOME/run/server.pid"
|
||||
log="$QWEN_HOME/logs/server.log"
|
||||
|
||||
case "$effort" in
|
||||
off) reasoning=(--reasoning off) ;;
|
||||
low | medium | xhigh) reasoning=(--reasoning-effort "$effort") ;;
|
||||
*) echo "QWEN_EFFORT is '$effort'; use one of low, medium, xhigh, off" >&2; exit 2 ;;
|
||||
esac
|
||||
args=(-m "$model" --alias "$QWEN_ALIAS" --host 127.0.0.1 --port "$QWEN_PORT" -ngl 999 -c "$QWEN_CTX"
|
||||
-fa on --cache-type-k q8_0 --cache-type-v q8_0 --cache-ram "${QWEN_CACHE_RAM:-2048}"
|
||||
--ctx-checkpoints "${QWEN_CTX_CHECKPOINTS:-4}" --parallel 1 --jinja
|
||||
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 "${reasoning[@]}")
|
||||
[ ! -f "$tplf" ] || args+=(--chat-template-file "$tplf")
|
||||
# shellcheck disable=SC2206 # QWEN_MTP_ARGS is a list of flags, split on spaces on purpose
|
||||
[ -z "$mtp" ] || args+=($mtp)
|
||||
|
||||
# running: print the pid of our llama-server, if the pid file names one that is alive. A pid file
|
||||
# whose process is gone, or is some other program (the pid was reused), is stale and is removed.
|
||||
running() {
|
||||
local pid
|
||||
[ -f "$pidf" ] || return 1
|
||||
pid="$(cat "$pidf")"
|
||||
if [[ "$pid" =~ ^[0-9]+$ ]] && kill -0 "$pid" 2>/dev/null && tr '\0' ' ' < "/proc/$pid/cmdline" 2>/dev/null | grep -q llama-server; then
|
||||
echo "$pid"
|
||||
return 0
|
||||
fi
|
||||
echo "removing a stale pid file ($pidf: $pid)" >&2
|
||||
rm -f "$pidf" "$QWEN_STATE"
|
||||
return 1
|
||||
}
|
||||
|
||||
quote() {
|
||||
local a out=()
|
||||
for a in "$@"; do
|
||||
if [[ "$a" =~ ^[A-Za-z0-9_./:=+,-]+$ ]]; then out+=("$a"); else out+=("'$a'"); fi
|
||||
done
|
||||
echo "${out[*]}"
|
||||
}
|
||||
|
||||
stop_pid() {
|
||||
local pid="$1" i
|
||||
kill "$pid" 2>/dev/null || return 0
|
||||
for ((i = 0; i < 30; i++)); do
|
||||
kill -0 "$pid" 2>/dev/null || return 0
|
||||
sleep 1
|
||||
done
|
||||
kill -KILL "$pid" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# The model's template refuses a system message after the first. This renders it as a system turn.
|
||||
fix_template='
|
||||
import json, sys
|
||||
t = json.load(sys.stdin)["chat_template"]
|
||||
old = "{{- raise_exception(\x27System message must be at the beginning.\x27) }}"
|
||||
if t.count(old) != 1:
|
||||
sys.exit("did not find the line that refuses later system messages in the chat template; check the template by hand")
|
||||
open(sys.argv[1], "w").write(t.replace(old, "{{- \x27<|im_start|>system\\n\x27 + content + \x27<|im_end|>\\n\x27 }}"))
|
||||
'
|
||||
|
||||
case "$1" in
|
||||
cmd) quote "$bin" "${args[@]}" ;;
|
||||
template)
|
||||
qwen_require_server
|
||||
mkdir -p "$(dirname "$tplf")"
|
||||
if ! curl -fsS -m 10 "$(qwen_url)/props" | python3 -I -c "$fix_template" "$tplf.tmp"; then rm -f "$tplf.tmp"; exit 1; fi
|
||||
mv -f "$tplf.tmp" "$tplf"
|
||||
echo "saved $tplf; restart the server to use it: qwen-local.sh stop && qwen-local.sh start"
|
||||
;;
|
||||
status)
|
||||
qwen_load_state
|
||||
if running >/dev/null; then echo "running on $(qwen_url)"; else echo "not running"; exit 3; fi
|
||||
;;
|
||||
stop)
|
||||
if pid="$(running)"; then
|
||||
stop_pid "$pid"
|
||||
rm -f "$pidf" "$QWEN_STATE"
|
||||
echo "stopped"
|
||||
else
|
||||
echo "not running"
|
||||
fi
|
||||
;;
|
||||
start)
|
||||
if running >/dev/null; then qwen_load_state; echo "already running on $(qwen_url)"; exit 0; fi
|
||||
[ -f "$model" ] || { echo "the model $model does not exist; see the setup in docs/5-research/2026-10-19-local-qwen-notes.md" >&2; exit 2; }
|
||||
[ -x "$bin" ] || { echo "llama-server $bin does not exist or is not executable" >&2; exit 2; }
|
||||
if qwen_healthy; then echo "port $QWEN_PORT already answers on $(qwen_url), and it is not our server; stop that program or set QWEN_PORT" >&2; exit 2; fi
|
||||
mkdir -p "$QWEN_HOME/run" "$QWEN_HOME/logs"
|
||||
[ ! -f "$log" ] || mv -f "$log" "$log.1"
|
||||
nohup "$bin" "${args[@]}" >"$log" 2>&1 &
|
||||
pid=$!
|
||||
echo "$pid" > "$pidf"
|
||||
printf 'QWEN_PORT=%s\nQWEN_CTX=%s\n' "$QWEN_PORT" "$QWEN_CTX" > "$QWEN_STATE"
|
||||
for ((i = 0; i < timeout; i++)); do
|
||||
if ! kill -0 "$pid" 2>/dev/null; then
|
||||
rm -f "$pidf" "$QWEN_STATE"
|
||||
echo "llama-server exited while starting; the end of $log:" >&2
|
||||
tail -n 20 "$log" >&2
|
||||
exit 1
|
||||
fi
|
||||
if qwen_healthy; then echo "ready on $(qwen_url) (pid $pid, log $log)"; exit 0; fi
|
||||
sleep 1
|
||||
done
|
||||
stop_pid "$pid"
|
||||
rm -f "$pidf" "$QWEN_STATE"
|
||||
echo "llama-server did not become ready within $timeout seconds; stopped it. The end of $log:" >&2
|
||||
tail -n 20 "$log" >&2
|
||||
exit 1
|
||||
;;
|
||||
*) usage ;;
|
||||
esac
|
||||
Executable
+124
@@ -0,0 +1,124 @@
|
||||
#!/usr/bin/env python3
|
||||
# Measure the local Qwen server (docs/5-research/2026-10-19-local-qwen-notes.md, "Measurements"). Every
|
||||
# figure is the server's own: the timings in each /v1/chat/completions answer. Run it as `python3 -I`.
|
||||
# qwen-measure.py speed [--runs N] [--max-tokens M] PROMPT_FILE
|
||||
# N requests (3) with the prompt; one line per run, then the median decode and prompt speeds
|
||||
# qwen-measure.py prompt --tokens N [--question Q] --out FILE GLOB...
|
||||
# write a prompt of about N tokens, counted by the server's /tokenize: the files the globs match
|
||||
# (** allowed), each headed "// FILE <path>", repeated as needed, then the question
|
||||
# qwen-measure.py growth [--turns N] [--chunk C] --pid PID SOURCE_FILE
|
||||
# a conversation that grows by C characters (11000, about 3K tokens) of SOURCE_FILE per turn, for N
|
||||
# turns (20); after each, the context size and the resident memory of process PID (llama-server)
|
||||
# QWEN_URL is the server (http://127.0.0.1:8088).
|
||||
import argparse
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import statistics
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
URL = os.environ.get("QWEN_URL", "http://127.0.0.1:8088")
|
||||
|
||||
|
||||
def post(path, body, timeout=1800):
|
||||
req = urllib.request.Request(URL + path, json.dumps(body).encode(), {"content-type": "application/json"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as r:
|
||||
return json.load(r)
|
||||
except (urllib.error.URLError, ConnectionError) as e:
|
||||
sys.exit(f"the local model server does not answer on {URL} ({e}); start it with scripts/qwen-local.sh start")
|
||||
|
||||
|
||||
def ntokens(text):
|
||||
return len(post("/tokenize", {"content": text})["tokens"])
|
||||
|
||||
|
||||
def speed(a):
|
||||
prompt = open(a.prompt_file).read()
|
||||
rows = []
|
||||
for i in range(a.runs):
|
||||
t0 = time.time()
|
||||
r = post("/v1/chat/completions", {"model": "q", "max_tokens": a.max_tokens,
|
||||
"messages": [{"role": "user", "content": prompt}]})
|
||||
t, m = r["timings"], r["choices"][0]["message"]
|
||||
rows.append(t)
|
||||
print(f"run {i + 1}: prompt {t['prompt_n']} tokens at {t['prompt_per_second']:.0f} tokens/s"
|
||||
f" | decode {t['predicted_per_second']:.1f} tokens/s ({t['predicted_n']} tokens)"
|
||||
f" | draft {t.get('draft_n_accepted', '-')}/{t.get('draft_n', '-')}"
|
||||
f" | reasoning {len(m.get('reasoning_content') or '')} chars | wall {time.time() - t0:.1f}s", flush=True)
|
||||
print(f"median decode {statistics.median(t['predicted_per_second'] for t in rows):.1f} tokens/s,"
|
||||
f" median prompt {statistics.median(t['prompt_per_second'] for t in rows):.0f} tokens/s")
|
||||
|
||||
|
||||
def prompt(a):
|
||||
paths = sorted({p for g in a.globs for p in glob.glob(g, recursive=True) if os.path.isfile(p)})
|
||||
if not paths:
|
||||
print(f"no files match {' '.join(a.globs)}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
src = "".join(f"// FILE {p}\n" + open(p, errors="replace").read() + "\n" for p in paths)
|
||||
per = max(1, ntokens(src))
|
||||
body = src * (a.tokens // per + 1)
|
||||
body = body[: int(a.tokens * len(src) / per)]
|
||||
text = body + f"\n\nQuestion: {a.question}"
|
||||
with open(a.out, "w") as f:
|
||||
f.write(text)
|
||||
print(f"wrote {a.out}: {ntokens(text)} tokens, from {len(paths)} files")
|
||||
|
||||
|
||||
def rss_gb(pid):
|
||||
with open(f"/proc/{pid}/status") as f:
|
||||
for line in f:
|
||||
if line.startswith("VmRSS"):
|
||||
return int(line.split()[1]) / 1048576
|
||||
return 0.0
|
||||
|
||||
|
||||
def growth(a):
|
||||
if not os.path.exists(f"/proc/{a.pid}/status"):
|
||||
print(f"no process {a.pid}; pass the pid of llama-server ($QWEN_HOME/run/server.pid)", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
src = open(a.source_file).read()
|
||||
msgs = []
|
||||
for i in range(a.turns):
|
||||
chunk = src[(i * a.chunk) % max(1, len(src)):][: a.chunk]
|
||||
msgs.append({"role": "user", "content": f"Turn {i}. Here is more code:\n{chunk}\nReply with one word."})
|
||||
r = post("/v1/chat/completions", {"model": "q", "max_tokens": 30, "messages": msgs}, timeout=600)
|
||||
msgs.append({"role": "assistant", "content": r["choices"][0]["message"].get("content") or "ok"})
|
||||
print(f"turn {i:2d} ctx {r['usage']['prompt_tokens']:6d} rss {rss_gb(a.pid):.2f} GB", flush=True)
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser(prog="qwen-measure.py")
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
s = sub.add_parser("speed")
|
||||
s.add_argument("--runs", type=int, default=3)
|
||||
s.add_argument("--max-tokens", type=int, default=512)
|
||||
s.add_argument("prompt_file")
|
||||
s.set_defaults(fn=speed)
|
||||
s = sub.add_parser("prompt")
|
||||
s.add_argument("--tokens", type=int, required=True)
|
||||
s.add_argument("--question", default="Which package in the code above does the most work? Answer in two sentences.")
|
||||
s.add_argument("--out", required=True)
|
||||
s.add_argument("globs", nargs="+")
|
||||
s.set_defaults(fn=prompt)
|
||||
s = sub.add_parser("growth")
|
||||
s.add_argument("--turns", type=int, default=20)
|
||||
s.add_argument("--chunk", type=int, default=11000)
|
||||
s.add_argument("--pid", required=True)
|
||||
s.add_argument("source_file")
|
||||
s.set_defaults(fn=growth)
|
||||
a = p.parse_args()
|
||||
try:
|
||||
a.fn(a)
|
||||
except SystemExit as e:
|
||||
if isinstance(e.code, str):
|
||||
print(e.code, file=sys.stderr)
|
||||
sys.exit(2)
|
||||
raise
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+29
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run OpenCode against the local Qwen 3.8 27B server, and nothing else
|
||||
# (docs/5-research/2026-10-19-local-qwen-design.md). Start the server first: qwen-local.sh start.
|
||||
# qwen-opencode.sh [OPENCODE ARGS...] for example: qwen-opencode.sh, or qwen-opencode.sh run "..."
|
||||
# The configuration goes in through OPENCODE_CONFIG_CONTENT, with the running server's port and
|
||||
# context. Only the local provider is enabled, and the cloud credentials of the caller's shell are
|
||||
# removed. OpenCode does not read ~/.claude (the work account's Claude Code config) or outside
|
||||
# skills, and does not update, share sessions or fetch the models.dev catalogue. The project's own
|
||||
# OpenCode config still applies. QWEN_OPENCODE_BIN overrides the binary (opencode).
|
||||
# OpenCode 2 runs sessions in a background service that keeps the config and environment it started
|
||||
# with, so this script always passes --standalone (after `run`, or first for the interactive session):
|
||||
# a private server that gets exactly this config.
|
||||
set -euo pipefail
|
||||
# shellcheck source-path=SCRIPTDIR source=qwen-lib.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/qwen-lib.sh"
|
||||
qwen_require_server
|
||||
|
||||
# OpenCode 2 still reads these keys (provider, enabled_providers, small_model, autoupdate), though its
|
||||
# schema marks them deprecated. Gleipnir's adapter uses the same ones.
|
||||
# shellcheck disable=SC2016 # "$schema" is a JSON key, not a variable
|
||||
conf="$(printf '{"$schema":"https://opencode.ai/config.json","autoupdate":false,"share":"disabled","enabled_providers":["qwen-local"],"provider":{"qwen-local":{"npm":"@ai-sdk/openai-compatible","name":"Qwen local","options":{"baseURL":"%s/v1"},"models":{"%s":{"name":"Qwen 3.8 27B (local)","tool_call":true,"reasoning":true,"limit":{"context":%s,"output":32768}}}}},"model":"qwen-local/%s","small_model":"qwen-local/%s"}' \
|
||||
"$(qwen_url)" "$QWEN_ALIAS" "$QWEN_CTX" "$QWEN_ALIAS" "$QWEN_ALIAS")"
|
||||
|
||||
if [ "${1:-}" = run ]; then shift; set -- run --standalone "$@"; else set -- --standalone "$@"; fi
|
||||
exec env -u ANTHROPIC_API_KEY -u CLAUDE_CODE_OAUTH_TOKEN -u OPENROUTER_API_KEY \
|
||||
OPENCODE_CONFIG_CONTENT="$conf" \
|
||||
OPENCODE_DISABLE_CLAUDE_CODE=1 OPENCODE_DISABLE_EXTERNAL_SKILLS=1 \
|
||||
OPENCODE_DISABLE_AUTOUPDATE=true OPENCODE_DISABLE_SHARE=true OPENCODE_DISABLE_MODELS_FETCH=true \
|
||||
"${QWEN_OPENCODE_BIN:-opencode}" "$@"
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
# Each check is a condition string run by eval: the helpers are read there, so
|
||||
# the linter cannot see them used.
|
||||
# shellcheck disable=SC2016,SC2034,SC2329
|
||||
# Tests for qwen-capture.py. scripts/tests/qwen-capture.test.sh
|
||||
# Starts the capture server on a free port, sends it requests with curl, and reads what it saved.
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
S="$HERE/.."
|
||||
T="$(mktemp -d)"
|
||||
cleanup() { [ -z "${CAP:-}" ] || kill "$CAP" 2>/dev/null; rm -rf -- "$T"; }
|
||||
trap cleanup EXIT
|
||||
fail=0
|
||||
check() {
|
||||
local d="$1"; shift
|
||||
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; fi
|
||||
}
|
||||
|
||||
port="$(python3 -I -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1])')"
|
||||
python3 -I "$S/qwen-capture.py" "$port" "$T/cap" > "$T/log" 2>&1 & CAP=$!
|
||||
for _ in $(seq 50); do curl -fsS -m 1 "http://127.0.0.1:$port/health" >/dev/null 2>&1 && break; sleep 0.1; done
|
||||
|
||||
curl -sS -m 5 -o "$T/r1" -w '%{http_code}' -H 'content-type: application/json' -d '{"model":"m","messages":[{"role":"system","content":"s"}]}' \
|
||||
"http://127.0.0.1:$port/v1/messages?beta=true" > "$T/code1"
|
||||
curl -sS -m 5 -o /dev/null -H 'content-type: application/json' -d '{"model":"m","n":2}' "http://127.0.0.1:$port/v1/chat/completions"
|
||||
|
||||
check "the first request is saved by number and path, as sent" \
|
||||
eval '[ -f "$T/cap/00-v1_messages.json" ] && python3 -I -c "import json,sys; assert json.load(open(sys.argv[1]))[\"messages\"][0][\"role\"] == \"system\"" "$T/cap/00-v1_messages.json"'
|
||||
check "the second request gets the next number" eval '[ -f "$T/cap/01-v1_chat_completions.json" ]'
|
||||
check "each request is answered with an API error, so the tool stops instead of waiting" \
|
||||
eval '[ "$(cat "$T/code1")" = 500 ] && grep -q "\"type\": *\"error\"" "$T/r1"'
|
||||
check "each saved request is announced on stdout" eval 'grep -q "00-v1_messages.json" "$T/log" && grep -q "01-v1_chat_completions.json" "$T/log"'
|
||||
check "it listens on 127.0.0.1 only" eval '! curl -fsS -m 1 "http://$(hostname -I | cut -d" " -f1):$port/health" >/dev/null 2>&1'
|
||||
|
||||
exit $fail
|
||||
Executable
+97
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env bash
|
||||
# Each check is a condition string run by eval: $rc and the helpers are read there, so
|
||||
# the linter cannot see them used.
|
||||
# shellcheck disable=SC2016,SC2034,SC2329
|
||||
# Tests for qwen-claude.sh. scripts/tests/qwen-claude.test.sh
|
||||
# A fake `claude` and a fake `curl` stand in for the real tools: no server, no account, no network.
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
S="$HERE/.."
|
||||
T="$(mktemp -d)"
|
||||
trap 'rm -rf -- "$T"' EXIT
|
||||
fail=0
|
||||
check() {
|
||||
local d="$1"; shift
|
||||
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
|
||||
}
|
||||
run() { rm -f "$T/calls" "$T/env"; "$@" >"$T/out" 2>&1; rc=$?; }
|
||||
has() { grep -q -F -- "$1" "$T/out"; }
|
||||
called() { grep -qx -F -- "$1" "$T/calls"; }
|
||||
envhas() { grep -qx -F -- "$1" "$T/env"; }
|
||||
# js FILE EXPR: evaluate a python expression over the JSON file (as c).
|
||||
js() { python3 -I -c 'import json,sys; c=json.load(open(sys.argv[1])); sys.exit(0 if eval(sys.argv[2]) else 1)' "$1" "$2"; }
|
||||
sums() { find "$HOME/.claude-personal" "$HOME/.claude" "$HOME/.claude.json" -type f -exec sha256sum {} + | sort; }
|
||||
export HOME="$T/home"
|
||||
P="$HOME/.claude-personal" Q="$HOME/.claude-qwen"
|
||||
mkdir -p "$P/skills/a" "$P/hooks" "$HOME/.claude"
|
||||
echo '{"model":"opus"}' > "$P/settings.json"
|
||||
echo 'skill a' > "$P/skills/a/SKILL.md"
|
||||
printf '#!/bin/sh\nexit 0\n' > "$P/hooks/merge-requires-green.sh"
|
||||
echo '{"model":"work"}' > "$HOME/.claude/settings.json"
|
||||
echo '{"work":true}' > "$HOME/.claude.json"
|
||||
before="$(sums)"
|
||||
|
||||
# The fake claude records each call as one line, "<CLAUDE_CONFIG_DIR> <args...>", and its environment.
|
||||
mkdir -p "$T/bin"
|
||||
cat > "$T/bin/claude" <<EOF2
|
||||
#!/usr/bin/env bash
|
||||
echo "\${CLAUDE_CONFIG_DIR:-none} \$*" >> "$T/calls"
|
||||
env > "$T/env"
|
||||
EOF2
|
||||
cat > "$T/bin/curl" <<EOF2
|
||||
#!/usr/bin/env bash
|
||||
[ -e "$T/healthy" ] || exit 7
|
||||
echo '{"status":"ok"}'
|
||||
EOF2
|
||||
chmod +x "$T/bin/claude" "$T/bin/curl"
|
||||
export PATH="$T/bin:$PATH" QWEN_HOME="$T/qh" QWEN_CLAUDE_BIN="$T/bin/claude"
|
||||
C="$S/qwen-claude.sh"
|
||||
|
||||
run bash "$C" -p hi
|
||||
check "run before init: refuses and says to init first; claude does not run" \
|
||||
eval '[ $rc = 2 ] && has "qwen-claude.sh init" && [ ! -e "$T/calls" ]'
|
||||
|
||||
QWEN_CLAUDE_DIR="$P" run bash "$C" init
|
||||
check "init refuses the personal config dir, and writes nothing" \
|
||||
eval '[ $rc = 2 ] && [ ! -e "$T/calls" ] && [ "$(sums)" = "$before" ]'
|
||||
QWEN_CLAUDE_DIR="$HOME/.claude/sub" run bash "$C" init
|
||||
check "init refuses a dir inside the work config dir" eval '[ $rc = 2 ] && [ ! -e "$HOME/.claude/sub" ]'
|
||||
QWEN_CLAUDE_DIR="$HOME" run bash "$C" init
|
||||
check "init refuses \$HOME, which holds both config dirs and ~/.claude.json" eval '[ $rc = 2 ] && [ ! -e "$T/calls" ] && [ ! -e "$HOME/settings.json" ]'
|
||||
QWEN_PERSONAL_DIR="$T/elsewhere" QWEN_CLAUDE_DIR="$P" run bash "$C" init
|
||||
check "init refuses ~/.claude-personal even when QWEN_PERSONAL_DIR points elsewhere" eval '[ $rc = 2 ] && [ ! -e "$T/calls" ]'
|
||||
|
||||
run bash "$C" init
|
||||
check "init writes settings: the three plugins, ask-before-acting mode, the personal merge hook" \
|
||||
eval '[ $rc = 0 ] && js "$Q/settings.json" "c[\"permissions\"][\"defaultMode\"] == \"default\" and sorted(c[\"enabledPlugins\"]) == [\"gopls-lsp@claude-plugins-official\", \"skill-creator@claude-plugins-official\", \"superpowers@claude-plugins-official\"] and c[\"hooks\"][\"PreToolUse\"][0][\"hooks\"][0][\"command\"] == \"$P/hooks/merge-requires-green.sh\" and c[\"model\"] == \"qwen3.8-27b\""'
|
||||
check "init marks onboarding done and links the personal skills" \
|
||||
eval 'js "$Q/.claude.json" "c[\"hasCompletedOnboarding\"] is True" && [ "$(readlink "$Q/skills")" = "$P/skills" ]'
|
||||
check "init adds the marketplace and installs the three plugins, all into the local config dir" \
|
||||
eval 'called "$Q plugin marketplace add anthropics/claude-plugins-official" && called "$Q plugin install superpowers@claude-plugins-official" && called "$Q plugin install gopls-lsp@claude-plugins-official" && called "$Q plugin install skill-creator@claude-plugins-official"'
|
||||
|
||||
echo '{"hasCompletedOnboarding":true,"mine":1}' > "$Q/.claude.json"
|
||||
run bash "$C" init
|
||||
check "init again keeps an existing .claude.json and the skills link" \
|
||||
eval '[ $rc = 0 ] && js "$Q/.claude.json" "c.get(\"mine\") == 1" && [ "$(readlink "$Q/skills")" = "$P/skills" ]'
|
||||
|
||||
run bash "$C" -p hi
|
||||
check "run with the server down: refuses with the start hint; claude does not run" \
|
||||
eval '[ $rc = 2 ] && has "qwen-local.sh start" && [ ! -e "$T/calls" ]'
|
||||
|
||||
touch "$T/healthy"
|
||||
QWEN_CLAUDE_DIR="$HOME/.claude" run bash "$C" -p hi
|
||||
check "run refuses the work config dir, though it has a settings.json" eval '[ $rc = 2 ] && [ ! -e "$T/calls" ]'
|
||||
QWEN_CLAUDE_DIR="$P" run bash "$C" -p hi
|
||||
check "run refuses the personal config dir" eval '[ $rc = 2 ] && [ ! -e "$T/calls" ]'
|
||||
|
||||
CLAUDE_CODE_USE_BEDROCK=1 CLAUDE_CODE_USE_VERTEX=1 CLAUDE_CODE_USE_FOUNDRY=1 ANTHROPIC_BEDROCK_BASE_URL=x ANTHROPIC_VERTEX_BASE_URL=y \
|
||||
ANTHROPIC_API_KEY=a CLAUDE_CODE_OAUTH_TOKEN=b OPENROUTER_API_KEY=c run bash "$C" -p "hi there"
|
||||
check "run passes the arguments and points Claude Code at the local server and config dir" \
|
||||
eval '[ $rc = 0 ] && called "$Q -p hi there" && envhas "ANTHROPIC_BASE_URL=http://127.0.0.1:8088" && envhas "CLAUDE_CONFIG_DIR=$Q" && envhas ANTHROPIC_MODEL=qwen3.8-27b && envhas ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen3.8-27b && envhas CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 && envhas DISABLE_AUTOUPDATER=1 && envhas CLAUDE_CODE_MAX_CONTEXT_TOKENS=102400'
|
||||
check "no cloud credential from the caller's shell reaches claude" \
|
||||
eval '[ -e "$T/env" ] && ! grep -Eq "^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN|OPENROUTER_API_KEY)=" "$T/env"'
|
||||
check "no Bedrock, Vertex or Foundry switch reaches claude, so it cannot bypass the local base URL" \
|
||||
eval '[ -e "$T/env" ] && ! grep -Eq "^(CLAUDE_CODE_USE_BEDROCK|CLAUDE_CODE_USE_VERTEX|CLAUDE_CODE_USE_FOUNDRY|ANTHROPIC_BEDROCK_BASE_URL|ANTHROPIC_VERTEX_BASE_URL)=" "$T/env"'
|
||||
check "nothing under ~/.claude-personal or ~/.claude changed" eval '[ "$(sums)" = "$before" ]'
|
||||
|
||||
exit $fail
|
||||
Executable
+84
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env bash
|
||||
# Each check is a condition string run by eval: $rc and the helpers are read there, so
|
||||
# the linter cannot see them used.
|
||||
# shellcheck disable=SC2016,SC2034,SC2329
|
||||
# Tests for qwen-compare.sh. scripts/tests/qwen-compare.test.sh
|
||||
# Fakes stand in for qwen-opencode.sh, qwen-claude.sh and claude: each records its arguments, folder and
|
||||
# environment, appends a line to a fake server log, and changes a file. A temp git repo is the project.
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
S="$HERE/.."
|
||||
T="$(mktemp -d)"
|
||||
trap 'rm -rf -- "$T"' EXIT
|
||||
fail=0
|
||||
check() {
|
||||
local d="$1"; shift
|
||||
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
|
||||
}
|
||||
run() { "$@" >"$T/out" 2>&1; rc=$?; }
|
||||
has() { grep -q -F -- "$1" "$T/out"; }
|
||||
export HOME="$T/home"; mkdir -p "$HOME/.claude-personal"
|
||||
|
||||
R="$T/repo"
|
||||
git init -q -b main "$R"
|
||||
git -C "$R" config user.email [email protected]; git -C "$R" config user.name t
|
||||
echo base > "$R/a.txt"; git -C "$R" add a.txt; git -C "$R" commit -q -m base
|
||||
BASE="$(git -C "$R" rev-parse HEAD)"
|
||||
echo later > "$R/a.txt"; git -C "$R" commit -q -am later
|
||||
|
||||
export QWEN_HOME="$T/qh"; mkdir -p "$QWEN_HOME/logs"; echo "old line" > "$QWEN_HOME/logs/server.log"
|
||||
mkdir -p "$T/bin"
|
||||
for f in opencode qwen-claude claude; do
|
||||
cat > "$T/bin/$f" <<EOF2
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' "\$@" > "$T/$f.args"
|
||||
pwd > "$T/$f.pwd"
|
||||
env > "$T/$f.env"
|
||||
echo "request from $f" >> "$QWEN_HOME/logs/server.log"
|
||||
echo "$f was here" > "$f.txt"
|
||||
echo "$f output"
|
||||
exit "\${FAKE_RC_$(echo "$f" | tr -- '-' '_'):-0}"
|
||||
EOF2
|
||||
chmod +x "$T/bin/$f"
|
||||
done
|
||||
export QWEN_COMPARE_OPENCODE="$T/bin/opencode" QWEN_COMPARE_QWEN_CLAUDE="$T/bin/qwen-claude" QWEN_CLAUDE_BIN="$T/bin/claude"
|
||||
echo "Fix the bug. Do not ask questions." > "$T/prompt.txt"
|
||||
X="$S/qwen-compare.sh"
|
||||
|
||||
run bash "$X" --repo "$R" --commit "$BASE"
|
||||
check "missing --prompt or --out is a usage error" eval '[ $rc = 2 ] && has usage'
|
||||
|
||||
mkdir -p "$T/full"; touch "$T/full/x"
|
||||
run bash "$X" --repo "$R" --commit "$BASE" --prompt "$T/prompt.txt" --out "$T/full"
|
||||
check "an --out folder that is not empty is refused" eval '[ $rc = 2 ] && has "not empty"'
|
||||
|
||||
O="$T/cmp"
|
||||
ANTHROPIC_BASE_URL=http://127.0.0.1:8088 FAKE_RC_qwen_claude=3 run bash "$X" --repo "$R" --commit "$BASE" --prompt "$T/prompt.txt" --out "$O"
|
||||
check "all three runs happen, in their own worktrees at the asked commit" \
|
||||
eval '[ $rc = 0 ] && for t in opencode qwen-claude hosted; do [ "$(git -C "$O/wt-$t" rev-parse HEAD)" = "$BASE" ] || exit 1; done'
|
||||
check "each tool runs in its own worktree" \
|
||||
eval '[ "$(cat "$T/opencode.pwd")" = "$O/wt-opencode" ] && [ "$(cat "$T/qwen-claude.pwd")" = "$O/wt-qwen-claude" ] && [ "$(cat "$T/claude.pwd")" = "$O/wt-hosted" ]'
|
||||
check "OpenCode gets run --auto --format json and the prompt" \
|
||||
eval '[ "$(head -n 4 "$T/opencode.args")" = "$(printf "run\n--auto\n--format\njson")" ] && grep -qxF "Fix the bug. Do not ask questions." "$T/opencode.args"'
|
||||
check "local Claude Code gets the prompt, acceptEdits and the default Go allowlist, with JSON output" \
|
||||
eval 'grep -qx -- -p "$T/qwen-claude.args" && grep -qx acceptEdits "$T/qwen-claude.args" && grep -qxF "Bash(go *)" "$T/qwen-claude.args" && grep -qx json "$T/qwen-claude.args"'
|
||||
check "hosted Claude Code uses the personal config dir, Sonnet by default, and never the local base URL" \
|
||||
eval 'grep -qx "CLAUDE_CONFIG_DIR=$HOME/.claude-personal" "$T/claude.env" && grep -qx sonnet "$T/claude.args" && ! grep -q "^ANTHROPIC_BASE_URL=" "$T/claude.env"'
|
||||
check "the summary has each run's exit code and wall time; a failed run does not stop the others" \
|
||||
eval 'grep -qP "^opencode\t0\t[0-9]+$" "$O/summary.tsv" && grep -qP "^qwen-claude\t3\t[0-9]+$" "$O/summary.tsv" && grep -qP "^hosted\t0\t[0-9]+$" "$O/summary.tsv"'
|
||||
check "each run's output and diff (new files included) are saved" \
|
||||
eval 'grep -q "opencode output" "$O/opencode.out" && grep -q "+opencode was here" "$O/opencode.patch" && grep -q "+claude was here" "$O/hosted.patch"'
|
||||
check "each run gets only its own slice of the server log" \
|
||||
eval '[ "$(cat "$O/opencode.server.log")" = "request from opencode" ] && [ "$(cat "$O/qwen-claude.server.log")" = "request from qwen-claude" ]'
|
||||
check "the cleanup commands are printed, and the worktrees are left for review" \
|
||||
eval 'has "worktree remove --force $O/wt-opencode" && has "branch -D" && [ -d "$O/wt-hosted" ]'
|
||||
|
||||
run bash "$X" --repo "$R" --commit "$BASE" --prompt "$T/prompt.txt" --out "$T/only" --only hosted --hosted-model opus \
|
||||
--allow "Bash(npm test*)"
|
||||
check "--only, --hosted-model and --allow choose the runs, the model and the allowlist" \
|
||||
eval '[ $rc = 0 ] && [ -d "$T/only/wt-hosted" ] && [ ! -e "$T/only/wt-opencode" ] && grep -qx opus "$T/claude.args" && grep -qxF "Bash(npm test*)" "$T/claude.args" && ! grep -qxF "Bash(go *)" "$T/claude.args"'
|
||||
|
||||
run bash "$X" --repo "$R" --commit "$BASE" --prompt "$T/prompt.txt" --out "$T/bad" --only gemini
|
||||
check "an unknown run name is a usage error" eval '[ $rc = 2 ] && has gemini && [ ! -e "$T/bad/wt-gemini" ]'
|
||||
|
||||
exit $fail
|
||||
Executable
+153
@@ -0,0 +1,153 @@
|
||||
#!/usr/bin/env bash
|
||||
# Each check is a condition string run by eval: $rc, $took and the helpers are read there, so
|
||||
# the linter cannot see them used.
|
||||
# shellcheck disable=SC2016,SC2034,SC2329
|
||||
# Tests for qwen-local.sh and qwen-lib.sh. scripts/tests/qwen-local.test.sh
|
||||
# A fake `llama-server` and a fake `curl` stand in for the real tools: no GPU, no model, no network.
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
S="$HERE/.."
|
||||
T="$(mktemp -d)"
|
||||
cleanup() { pkill -f "$T/bin/llama-server" 2>/dev/null; [ -z "${SLEEPER:-}" ] || kill "$SLEEPER" 2>/dev/null; rm -rf -- "$T"; }
|
||||
trap cleanup EXIT
|
||||
fail=0
|
||||
check() {
|
||||
local d="$1"; shift
|
||||
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
|
||||
}
|
||||
run() { "$@" >"$T/out" 2>&1; rc=$?; }
|
||||
has() { grep -q -F -- "$1" "$T/out"; }
|
||||
export HOME="$T/home"; mkdir -p "$HOME"
|
||||
|
||||
# The fake server writes its arguments, one per line, to $T/args, and marks itself healthy (for the fake
|
||||
# curl) unless FAKE_NEVER_READY is set. FAKE_DIE makes it fail at once, as a model too big for VRAM does.
|
||||
mkdir -p "$T/bin"
|
||||
cat > "$T/bin/llama-server" <<EOF2
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' "\$@" > "$T/args"
|
||||
echo "fake llama-server loading"
|
||||
if [ -n "\${FAKE_DIE:-}" ]; then echo "fake: out of memory"; exit 1; fi
|
||||
trap 'rm -f "$T/healthy"; kill \$! 2>/dev/null; exit 0' TERM
|
||||
[ -n "\${FAKE_NEVER_READY:-}" ] || touch "$T/healthy"
|
||||
sleep 300 & wait
|
||||
EOF2
|
||||
# The fake curl answers /props from $T/props, and anything else with a health answer.
|
||||
cat > "$T/bin/curl" <<EOF2
|
||||
#!/usr/bin/env bash
|
||||
[ -e "$T/healthy" ] || exit 7
|
||||
case "\${*: -1}" in */props) cat "$T/props" ;; *) echo '{"status":"ok"}' ;; esac
|
||||
EOF2
|
||||
chmod +x "$T/bin/llama-server" "$T/bin/curl"
|
||||
touch "$T/model.gguf"
|
||||
export PATH="$T/bin:$PATH" QWEN_HOME="$T/qh" QWEN_LLAMA_SERVER="$T/bin/llama-server" QWEN_MODEL="$T/model.gguf" \
|
||||
QWEN_START_TIMEOUT=3
|
||||
L="$S/qwen-local.sh"
|
||||
pidf="$T/qh/run/server.pid" envf="$T/qh/run/server.env"
|
||||
nservers() { pgrep -fc "$T/bin/llama-server"; }
|
||||
|
||||
# cmd
|
||||
run bash "$L" cmd
|
||||
check "cmd: binds 127.0.0.1:8088 with a 102400 context, a q8_0 KV cache, one slot and low effort" \
|
||||
eval '[ $rc = 0 ] && has "--host 127.0.0.1" && has "--port 8088" && has "-c 102400" && has "--cache-type-k q8_0" && has "--cache-type-v q8_0" && has "--parallel 1" && has "--reasoning-effort low"'
|
||||
check "cmd: the prompt cache in system RAM is capped at 2048 MiB, so it fits WSL's memory cap" eval 'has "--cache-ram 2048"'
|
||||
check "cmd: the model's context checkpoints in system RAM are limited to 4" eval 'has "--ctx-checkpoints 4"'
|
||||
QWEN_CACHE_RAM=0 QWEN_CTX_CHECKPOINTS=8 run bash "$L" cmd
|
||||
check "cmd: QWEN_CACHE_RAM and QWEN_CTX_CHECKPOINTS set the limits" eval 'has "--cache-ram 0" && has "--ctx-checkpoints 8"'
|
||||
run bash "$L" cmd
|
||||
check "cmd: MTP is on by default, with the draft file from QWEN_HOME on the GPU" \
|
||||
eval 'has "--spec-type draft-mtp -md $T/qh/models/mtp.gguf -ngld all -ctkd q8_0 -ctvd q8_0"'
|
||||
QWEN_EFFORT=off run bash "$L" cmd
|
||||
check "cmd: effort off turns thinking off" eval 'has "--reasoning off" && ! has "--reasoning-effort"'
|
||||
QWEN_EFFORT=high run bash "$L" cmd
|
||||
check "cmd: an unknown effort is a usage error that names the allowed values" \
|
||||
eval '[ $rc = 2 ] && has QWEN_EFFORT && has "low, medium, xhigh, off"'
|
||||
QWEN_MTP_ARGS="--spec-a x" run bash "$L" cmd
|
||||
check "cmd: QWEN_MTP_ARGS ends the argument list" eval 'grep -Eq -- "--spec-a x ?\$" "$T/out"'
|
||||
QWEN_MTP_ARGS="" run bash "$L" cmd
|
||||
check "cmd: an empty QWEN_MTP_ARGS turns MTP off" eval '! has "--spec-a" && ! has "--spec-type"'
|
||||
|
||||
tpl="$T/qh/models/chat-template.jinja"
|
||||
mkdir -p "$T/qh/models"; echo x > "$tpl"
|
||||
run bash "$L" cmd
|
||||
check "cmd: a saved chat template is passed to the server" eval 'has "--chat-template-file $tpl"'
|
||||
rm -f "$tpl"
|
||||
run bash "$L" cmd
|
||||
check "cmd: without a saved chat template, the model's own is used" eval '! has "--chat-template-file"'
|
||||
|
||||
# template
|
||||
run bash "$L" template
|
||||
check "template: needs the server running" eval '[ $rc = 2 ] && has "server is not running" && [ ! -e "$tpl" ]'
|
||||
touch "$T/healthy"
|
||||
python3 -I -c 'import json,sys; print(json.dumps({"chat_template": "{%- if a %}\n {{- raise_exception(\x27System message must be at the beginning.\x27) }}\n{%- endif %}"}))' > "$T/props"
|
||||
run bash "$L" template
|
||||
check "template: saves the model's template with later system messages rendered as system turns" \
|
||||
eval '[ $rc = 0 ] && has "$tpl" && ! grep -q raise_exception "$tpl" && grep -qF "<|im_start|>system\\n" "$tpl" && grep -qF " + content + " "$tpl" && grep -qF "{%- if a %}" "$tpl"'
|
||||
rm -f "$tpl"; echo '{"chat_template": "no such line"}' > "$T/props"
|
||||
run bash "$L" template
|
||||
check "template: a template without the expected line is refused, and nothing is saved" \
|
||||
eval '[ $rc = 1 ] && has "did not find" && [ ! -e "$tpl" ]'
|
||||
rm -f "$T/healthy" "$T/props"
|
||||
|
||||
# start
|
||||
QWEN_MODEL="$T/missing.gguf" run bash "$L" start
|
||||
check "start: a missing model fails with its path, and starts nothing" \
|
||||
eval '[ $rc = 2 ] && has "$T/missing.gguf" && [ ! -e "$T/args" ]'
|
||||
|
||||
run bash "$L" start
|
||||
check "start: waits for health, says where it is ready, and records the pid and the state" \
|
||||
eval '[ $rc = 0 ] && has "ready on http://127.0.0.1:8088" && kill -0 "$(cat "$pidf")" && grep -qx -- "-m" "$T/args" && grep -qx -- "$T/model.gguf" "$T/args" && grep -qx QWEN_PORT=8088 "$envf" && grep -qx QWEN_CTX=102400 "$envf"'
|
||||
|
||||
run bash "$L" start
|
||||
check "start: a second start while running says so and starts no second server" \
|
||||
eval '[ $rc = 0 ] && has "already running" && [ "$(nservers)" = 1 ]'
|
||||
|
||||
run bash "$L" status
|
||||
check "status: running exits 0 with the URL" eval '[ $rc = 0 ] && has running && has "http://127.0.0.1:8088"'
|
||||
|
||||
run bash "$L" stop
|
||||
check "stop: the server is gone and the pid and state files are removed" \
|
||||
eval '[ $rc = 0 ] && [ "$(nservers)" = 0 ] && [ ! -e "$pidf" ] && [ ! -e "$envf" ]'
|
||||
|
||||
run bash "$L" stop
|
||||
check "stop: not running is not an error" eval '[ $rc = 0 ] && has "not running"'
|
||||
|
||||
run bash "$L" status
|
||||
check "status: not running exits 3" eval '[ $rc = 3 ] && has "not running"'
|
||||
|
||||
FAKE_DIE=1 run bash "$L" start
|
||||
check "start: a server that exits while loading fails at once and shows the log tail" \
|
||||
eval '[ $rc = 1 ] && has "exited while starting" && has "fake: out of memory" && [ ! -e "$pidf" ] && [ ! -e "$envf" ]'
|
||||
|
||||
SECONDS=0
|
||||
FAKE_NEVER_READY=1 run bash "$L" start
|
||||
took=$SECONDS
|
||||
check "start: a server that never gets healthy is stopped after the timeout" \
|
||||
eval '[ $rc = 1 ] && has "did not become ready" && [ "$took" -le 8 ] && [ "$(nservers)" = 0 ] && [ ! -e "$pidf" ]'
|
||||
|
||||
rm -f "$T/args"; touch "$T/healthy"
|
||||
run bash "$L" start
|
||||
check "start: another program answering on the port is refused, and nothing starts" \
|
||||
eval '[ $rc = 2 ] && has "port 8088 already answers" && [ ! -e "$T/args" ]'
|
||||
rm -f "$T/healthy"
|
||||
|
||||
# A stale pid file: its process is gone (after a crash or wsl --shutdown).
|
||||
mkdir -p "$T/qh/run"; sh -c 'exit 0' & dead=$!; wait "$dead"
|
||||
echo "$dead" > "$pidf"; printf 'QWEN_PORT=8088\nQWEN_CTX=102400\n' > "$envf"
|
||||
run bash "$L" status
|
||||
check "status: a stale pid file means not running, and is cleaned up" \
|
||||
eval '[ $rc = 3 ] && has "not running" && [ ! -e "$pidf" ] && [ ! -e "$envf" ]'
|
||||
|
||||
# A reused pid: the pid file names a live process that is not llama-server.
|
||||
sleep 60 & SLEEPER=$!
|
||||
echo "$SLEEPER" > "$pidf"
|
||||
run bash "$L" stop
|
||||
check "stop: a pid that is not llama-server is never killed; the pid file is removed as stale" \
|
||||
eval '[ $rc = 0 ] && has stale && kill -0 "$SLEEPER" && [ ! -e "$pidf" ]'
|
||||
|
||||
# qwen_load_state
|
||||
printf 'QWEN_PORT=9000\nQWEN_CTX=65536\n' > "$envf"
|
||||
run bash -c '. "$1/qwen-lib.sh"; qwen_load_state; qwen_url; echo "ctx=$QWEN_CTX"' _ "$S"
|
||||
check "qwen_load_state: the running server's port and context win over the defaults" \
|
||||
eval 'has "http://127.0.0.1:9000" && has "ctx=65536"'
|
||||
|
||||
exit $fail
|
||||
Executable
+77
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env bash
|
||||
# Each check is a condition string run by eval: $rc and the helpers are read there, so
|
||||
# the linter cannot see them used.
|
||||
# shellcheck disable=SC2016,SC2034,SC2329
|
||||
# Tests for qwen-measure.py. scripts/tests/qwen-measure.test.sh
|
||||
# A fake llama-server (a small Python HTTP server) answers /tokenize (one token per word) and
|
||||
# /v1/chat/completions (fixed timings, and the prompt's word count as prompt_tokens). No GPU, no model.
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
S="$HERE/.."
|
||||
T="$(mktemp -d)"
|
||||
cleanup() { [ -z "${FAKE:-}" ] || kill "$FAKE" 2>/dev/null; rm -rf -- "$T"; }
|
||||
trap cleanup EXIT
|
||||
fail=0
|
||||
check() {
|
||||
local d="$1"; shift
|
||||
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
|
||||
}
|
||||
run() { "$@" >"$T/out" 2>&1; rc=$?; }
|
||||
has() { grep -q -F -- "$1" "$T/out"; }
|
||||
|
||||
cat > "$T/fake.py" <<'EOF2'
|
||||
import http.server, json, sys
|
||||
port_file, log = sys.argv[1], sys.argv[2]
|
||||
class H(http.server.BaseHTTPRequestHandler):
|
||||
def do_POST(self):
|
||||
body = json.loads(self.rfile.read(int(self.headers["content-length"])))
|
||||
with open(log, "a") as f: f.write(json.dumps({"path": self.path, "body": body}) + "\n")
|
||||
if self.path == "/tokenize":
|
||||
out = {"tokens": list(range(len(body["content"].split())))}
|
||||
else:
|
||||
words = sum(len(m["content"].split()) for m in body["messages"])
|
||||
out = {"choices": [{"message": {"content": "ok", "reasoning_content": "think"}}],
|
||||
"usage": {"prompt_tokens": words, "completion_tokens": 10},
|
||||
"timings": {"prompt_n": words, "prompt_per_second": 1000.0, "predicted_n": 10,
|
||||
"predicted_per_second": 50.0 + len(body["messages"]), "draft_n": 8, "draft_n_accepted": 6}}
|
||||
data = json.dumps(out).encode()
|
||||
self.send_response(200); self.send_header("content-type", "application/json")
|
||||
self.send_header("content-length", str(len(data))); self.end_headers(); self.wfile.write(data)
|
||||
def log_message(self, *a): pass
|
||||
s = http.server.HTTPServer(("127.0.0.1", 0), H)
|
||||
open(port_file, "w").write(str(s.server_port))
|
||||
s.serve_forever()
|
||||
EOF2
|
||||
python3 -I "$T/fake.py" "$T/port" "$T/requests.log" & FAKE=$!
|
||||
for _ in $(seq 50); do [ -s "$T/port" ] && break; sleep 0.1; done
|
||||
QWEN_URL="http://127.0.0.1:$(cat "$T/port")"; export QWEN_URL
|
||||
M="$S/qwen-measure.py"
|
||||
echo "one two three four" > "$T/prompt.txt"
|
||||
|
||||
run python3 -I "$M" speed --runs 3 --max-tokens 64 "$T/prompt.txt"
|
||||
check "speed: one line per run with the server's timings, then the medians" \
|
||||
eval '[ $rc = 0 ] && [ "$(grep -c "^run " "$T/out")" = 3 ] && has "decode 51.0 tokens/s" && has "draft 6/8" && has "median decode 51.0 tokens/s, median prompt 1000 tokens/s"'
|
||||
check "speed: sends the prompt file and the token cap" \
|
||||
eval 'grep -q "\"max_tokens\": 64" "$T/requests.log" && grep -q "one two three four" "$T/requests.log"'
|
||||
|
||||
mkdir -p "$T/src"; for i in 1 2 3; do printf 'word%.0s ' $(seq 40) > "$T/src/f$i.go"; done
|
||||
run python3 -I "$M" prompt --tokens 300 --question "What is it?" --out "$T/long.txt" "$T/src/*.go"
|
||||
check "prompt: builds a prompt of about the asked size from the files, repeated, and ends with the question" \
|
||||
eval '[ $rc = 0 ] && has "tokens" && n=$(wc -w < "$T/long.txt") && [ "$n" -ge 280 ] && [ "$n" -le 330 ] && tail -c 20 "$T/long.txt" | grep -q "What is it?"'
|
||||
run python3 -I "$M" prompt --tokens 300 --out "$T/none.txt" "$T/src/*.nothing"
|
||||
check "prompt: no matching files is an error" eval '[ $rc = 2 ] && has "no files" && [ ! -e "$T/none.txt" ]'
|
||||
|
||||
run python3 -I "$M" growth --turns 3 --pid "$FAKE" --chunk 20 "$T/long.txt"
|
||||
check "growth: one line per turn with the context, which grows, and the server's resident memory" \
|
||||
eval '[ $rc = 0 ] && [ "$(grep -c "^turn " "$T/out")" = 3 ] && has "rss" && a=$(sed -n "1s/.*ctx *\([0-9]*\).*/\1/p" "$T/out") && b=$(sed -n "3s/.*ctx *\([0-9]*\).*/\1/p" "$T/out") && [ "$b" -gt "$a" ]'
|
||||
check "growth: each turn sends the whole conversation so far" \
|
||||
eval '[ "$(grep -c "/v1/chat/completions" "$T/requests.log")" -ge 3 ] && tail -n 1 "$T/requests.log" | grep -q "Turn 2" && tail -n 1 "$T/requests.log" | grep -q "Turn 0"'
|
||||
|
||||
run python3 -I "$M" growth --turns 1 --pid 999999999 "$T/long.txt"
|
||||
check "growth: a pid that does not exist is an error" eval '[ $rc = 2 ] && has "999999999"'
|
||||
|
||||
QWEN_URL="http://127.0.0.1:1" run python3 -I "$M" speed --runs 1 "$T/prompt.txt"
|
||||
check "speed: a server that does not answer is a clear error, not a traceback" \
|
||||
eval '[ $rc = 2 ] && has "qwen-local.sh start" && ! has Traceback'
|
||||
|
||||
exit $fail
|
||||
Executable
+62
@@ -0,0 +1,62 @@
|
||||
#!/usr/bin/env bash
|
||||
# Each check is a condition string run by eval: $rc and the helpers are read there, so
|
||||
# the linter cannot see them used.
|
||||
# shellcheck disable=SC2016,SC2034,SC2329
|
||||
# Tests for qwen-opencode.sh. scripts/tests/qwen-opencode.test.sh
|
||||
# A fake `opencode` and a fake `curl` stand in for the real tools: no server, no network.
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
S="$HERE/.."
|
||||
T="$(mktemp -d)"
|
||||
trap 'rm -rf -- "$T"' EXIT
|
||||
fail=0
|
||||
check() {
|
||||
local d="$1"; shift
|
||||
if "$@"; then echo "ok $d"; else echo "FAIL $d"; fail=1; [ -z "${VERBOSE:-}" ] || sed 's/^/ | /' "$T/out"; fi
|
||||
}
|
||||
run() { rm -f "$T/args" "$T/env"; "$@" >"$T/out" 2>&1; rc=$?; }
|
||||
has() { grep -q -F -- "$1" "$T/out"; }
|
||||
envhas() { grep -qx -F -- "$1" "$T/env"; }
|
||||
# conf EXPR: evaluate a python expression over the config the fake opencode was given (as c).
|
||||
conf() { python3 -I -c 'import json,sys; c=json.loads(sys.argv[1]); sys.exit(0 if eval(sys.argv[2]) else 1)' \
|
||||
"$(sed -n 's/^OPENCODE_CONFIG_CONTENT=//p' "$T/env")" "$1"; }
|
||||
export HOME="$T/home"; mkdir -p "$HOME"
|
||||
|
||||
mkdir -p "$T/bin"
|
||||
cat > "$T/bin/opencode" <<EOF2
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' "\$@" > "$T/args"
|
||||
env > "$T/env"
|
||||
EOF2
|
||||
cat > "$T/bin/curl" <<EOF2
|
||||
#!/usr/bin/env bash
|
||||
[ -e "$T/healthy" ] || exit 7
|
||||
echo '{"status":"ok"}'
|
||||
EOF2
|
||||
chmod +x "$T/bin/opencode" "$T/bin/curl"
|
||||
export PATH="$T/bin:$PATH" QWEN_HOME="$T/qh"
|
||||
O="$S/qwen-opencode.sh"
|
||||
|
||||
run bash "$O" run hi
|
||||
check "server down: refuses with the start hint, and opencode does not run" \
|
||||
eval '[ $rc = 2 ] && has "qwen-local.sh start" && [ ! -e "$T/args" ]'
|
||||
|
||||
touch "$T/healthy"; mkdir -p "$T/qh/run"; printf 'QWEN_PORT=9000\nQWEN_CTX=65536\n' > "$T/qh/run/server.env"
|
||||
ANTHROPIC_API_KEY=a CLAUDE_CODE_OAUTH_TOKEN=b OPENROUTER_API_KEY=c run bash "$O" run "hi there"
|
||||
check "the config points at the running server's port and context, and only at the local provider" \
|
||||
eval '[ $rc = 0 ] && conf "c[\"provider\"][\"qwen-local\"][\"options\"][\"baseURL\"] == \"http://127.0.0.1:9000/v1\"" && conf "c[\"provider\"][\"qwen-local\"][\"models\"][\"qwen3.8-27b\"][\"limit\"][\"context\"] == 65536" && conf "c[\"enabled_providers\"] == [\"qwen-local\"]"'
|
||||
check "the model and the small model are both the local one; sharing and updates are off" \
|
||||
eval 'conf "c[\"model\"] == c[\"small_model\"] == \"qwen-local/qwen3.8-27b\"" && conf "c[\"share\"] == \"disabled\" and c[\"autoupdate\"] is False"'
|
||||
check "run gets --standalone, so a background service with an older config is never used" \
|
||||
eval '[ "$(cat "$T/args")" = "$(printf "run\n--standalone\nhi there")" ]'
|
||||
check "OpenCode does not read ~/.claude (the work account) or outside skills, and does not update, share or fetch models" \
|
||||
eval 'envhas OPENCODE_DISABLE_CLAUDE_CODE=1 && envhas OPENCODE_DISABLE_EXTERNAL_SKILLS=1 && envhas OPENCODE_DISABLE_AUTOUPDATE=true && envhas OPENCODE_DISABLE_SHARE=true && envhas OPENCODE_DISABLE_MODELS_FETCH=true'
|
||||
check "no cloud credential from the caller's shell reaches opencode" \
|
||||
eval '[ -e "$T/env" ] && ! grep -Eq "^(ANTHROPIC_API_KEY|CLAUDE_CODE_OAUTH_TOKEN|OPENROUTER_API_KEY)=" "$T/env"'
|
||||
|
||||
run bash "$O"
|
||||
check "the interactive session gets --standalone too" eval '[ $rc = 0 ] && [ "$(cat "$T/args")" = "--standalone" ]'
|
||||
run bash "$O" ./some/project
|
||||
check "a project folder for the interactive session follows --standalone" \
|
||||
eval '[ "$(cat "$T/args")" = "$(printf -- "--standalone\n./some/project")" ]'
|
||||
exit $fail
|
||||
Reference in new issue
Block a user