anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 18:25:32 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

OpenCode sends tools (and Jinja tool template) to llama.cpp results in 500 error unless --jinja; with --jinja, template crashes (reject filter) #1280

New Issue
Open
opened 2026-02-16 17:30:16 -05:00 by yindo · 8 comments
yindo commented 2026-02-16 17:30:16 -05:00
Owner
Copy Link
Copy Source

Originally created by @brianluby on GitHub (Aug 13, 2025).

Originally assigned to: @thdxr on GitHub.

Environment

  • Client: OpenCode via Homebrew on macOS — v0.4.41 (also seen with 0.4.40)
  • Server: llama.cpp OpenAI‑compatible server (Windows)
  • Model: Qwen3‑30B A3B Coder (GGUF), served via llama.cpp; alias qwen3-30b
  • Endpoint: http://{server-ip}:8080/v1
  • Auth header: Authorization: Bearer test-api-key

Summary
OpenCode includes a tools parameter and injects a Jinja <tools> prompt block even in “plain chat”.

  • If llama.cpp runs without --jinja, it rejects the request with: tools param requires --jinja flag.
  • If llama.cpp runs with --jinja, it attempts to render OpenCode’s Jinja tools block and fails with a 500 due to missing filters (e.g., reject), producing a long “Value is not callable” stack trace.

Reproduction (no --jinja)

  1. Start llama.cpp:

    llama-server.exe -m C:\models\Qwen3-30B-A3B-Coder-480B-Distill-v2-Q8_0.gguf --alias qwen3-30b --host 0.0.0.0 --port 8080 --api-key test-api-key

  2. OpenCode provider points to http://{server-ip}:8080/v1, agent set to “plain chat” (no tools in config).

  3. Send “hi” in OpenCode.

Actual: Server 500 + log: tools param requires --jinja flag.
Expected: If no tools are configured, OpenCode should omit the tools key entirely.

Reproduction (with --jinja)

  1. Start llama.cpp:

    llama-server.exe -m C:\models\Qwen3-30B-A3B-Coder-480B-Distill-v2-Q8_0.gguf --alias qwen3-30b --host 0.0.0.0 --port 8080 --api-key test-api-key --jinja

  2. Same OpenCode config; send “hi”.

Actual: llama.cpp 500 with Jinja crash while rendering tool template. Excerpt:

Value is not callable: null at row 58, column 110:
{%- for json_key in param_fields.keys() | reject("in", handled_keys) %}
...
{{- "<tools>" }} {%- for tool in tools %} ...

Expected: Either don’t send Jinja in system prompt, or send OpenAI‑native tool schema without relying on server‑side Jinja filters.

Sanity checks (curl)

  • With --jinja, server accepts tools: []:

    curl http://{server-ip}:8080/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer test-api-key" \
  -d '{"model":"qwen3-30b","messages":[{"role":"user","content":"Say hi"}],"tools":[]}'

Returns completion.

  • Without --jinja, server accepts when no tools key is present:

    curl http://{server-ip}:8080/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer test-api-key" \
  -d '{"model":"qwen3-30b","messages":[{"role":"user","content":"Say hi"}]}'

Returns completion.

Why this seems client‑side
Even with a “plain chat” agent and no tools in my opencode.json, OpenCode still sends either:

  • a tools field in the JSON payload, or
  • a Jinja <tools> block in the system prompt.

This causes failures on llama.cpp depending on --jinja. Changelog 0.4.40 mentioned “disable todo tools for qwen models”, but the failures persist (they’re triggered by the prompt template/tool scaffolding, not the model).

Requested improvements

  1. Conditional tools emission: If an agent has no tools configured or the provider is generic OpenAI‑compatible (e.g., llama.cpp), omit tools and tool_choice entirely. Consider auto‑retry without tools on 4xx/5xx indicating unsupported tools.
  2. Config switch: A per‑agent flag like "sendTools": false that guarantees no tools are sent.
  3. Jinja‑free prompting mode: Provide a “raw OpenAI” prompt path (no {% ... %} in system prompt).
  4. Qwen defaults: Extend 0.4.40’s Qwen compatibility so all tool scaffolding is disabled by default for Qwen on OpenAI‑compatible providers, unless explicitly enabled.

Workarounds that fixed it for me

  • Start llama.cpp with --jinja and supply a minimal chat template that ignores tools, or
  • Run OpenCode through a tiny local proxy that strips tools/tool_choice from requests (works flawlessly with llama.cpp without --jinja).

Artifacts (can provide on request)

  • Server logs with 500 traces (redacted)
  • opencode.json (redacted)
  • Minimal proxy script that strips tools and demonstrates successful operation

Thanks! Happy to test a dev build or provide more logs.

Originally created by @brianluby on GitHub (Aug 13, 2025). Originally assigned to: @thdxr on GitHub. **Environment** * **Client:** OpenCode via Homebrew on macOS — **v0.4.41** (also seen with 0.4.40) * **Server:** llama.cpp OpenAI‑compatible server (Windows) * **Model:** Qwen3‑30B A3B Coder (GGUF), served via llama.cpp; alias `qwen3-30b` * **Endpoint:** `http://{server-ip}:8080/v1` * **Auth header:** `Authorization: Bearer test-api-key` **Summary** OpenCode includes a `tools` parameter and injects a Jinja `<tools>` prompt block even in “plain chat”. * If llama.cpp runs **without** `--jinja`, it rejects the request with: **`tools param requires --jinja flag`**. * If llama.cpp runs **with** `--jinja`, it attempts to render OpenCode’s Jinja tools block and fails with a 500 due to missing filters (e.g., `reject`), producing a long “Value is not callable” stack trace. **Reproduction (no `--jinja`)** 1. Start llama.cpp: ```powershell llama-server.exe -m C:\models\Qwen3-30B-A3B-Coder-480B-Distill-v2-Q8_0.gguf --alias qwen3-30b --host 0.0.0.0 --port 8080 --api-key test-api-key ``` 2. OpenCode provider points to `http://{server-ip}:8080/v1`, agent set to “plain chat” (no tools in config). 3. Send “hi” in OpenCode. **Actual:** Server 500 + log: `tools param requires --jinja flag`. **Expected:** If no tools are configured, OpenCode should **omit** the `tools` key entirely. **Reproduction (with `--jinja`)** 1. Start llama.cpp: ```powershell llama-server.exe -m C:\models\Qwen3-30B-A3B-Coder-480B-Distill-v2-Q8_0.gguf --alias qwen3-30b --host 0.0.0.0 --port 8080 --api-key test-api-key --jinja ``` 2. Same OpenCode config; send “hi”. **Actual:** llama.cpp 500 with Jinja crash while rendering tool template. Excerpt: ``` Value is not callable: null at row 58, column 110: {%- for json_key in param_fields.keys() | reject("in", handled_keys) %} ... {{- "<tools>" }} {%- for tool in tools %} ... ``` **Expected:** Either don’t send Jinja in system prompt, or send OpenAI‑native tool schema without relying on server‑side Jinja filters. **Sanity checks (curl)** * With `--jinja`, server accepts `tools: []`: ```bash curl http://{server-ip}:8080/v1/chat/completions \ -H "Content-Type: application/json" -H "Authorization: Bearer test-api-key" \ -d '{"model":"qwen3-30b","messages":[{"role":"user","content":"Say hi"}],"tools":[]}' ``` **Returns completion.** * Without `--jinja`, server accepts when **no** `tools` key is present: ```bash curl http://{server-ip}:8080/v1/chat/completions \ -H "Content-Type: application/json" -H "Authorization: Bearer test-api-key" \ -d '{"model":"qwen3-30b","messages":[{"role":"user","content":"Say hi"}]}' ``` **Returns completion.** **Why this seems client‑side** Even with a “plain chat” agent and no `tools` in my `opencode.json`, OpenCode still sends either: * a `tools` field in the JSON payload, **or** * a Jinja `<tools>` block in the system prompt. This causes failures on llama.cpp depending on `--jinja`. Changelog 0.4.40 mentioned “disable todo tools for qwen models”, but the failures persist (they’re triggered by the **prompt template/tool scaffolding**, not the model). **Requested improvements** 1. **Conditional tools emission:** If an agent has no tools configured or the provider is generic OpenAI‑compatible (e.g., llama.cpp), omit `tools` and `tool_choice` entirely. Consider auto‑retry without tools on 4xx/5xx indicating unsupported tools. 2. **Config switch:** A per‑agent flag like `"sendTools": false` that guarantees no tools are sent. 3. **Jinja‑free prompting mode:** Provide a “raw OpenAI” prompt path (no `{% ... %}` in system prompt). 4. **Qwen defaults:** Extend 0.4.40’s Qwen compatibility so **all** tool scaffolding is disabled by default for Qwen on OpenAI‑compatible providers, unless explicitly enabled. **Workarounds that fixed it for me** * Start llama.cpp with `--jinja` **and** supply a minimal chat template that ignores tools, **or** * Run OpenCode through a tiny local proxy that strips `tools`/`tool_choice` from requests (works flawlessly with llama.cpp **without** `--jinja`). **Artifacts (can provide on request)** * Server logs with 500 traces (redacted) * `opencode.json` (redacted) * Minimal proxy script that strips `tools` and demonstrates successful operation **Thanks!** Happy to test a dev build or provide more logs. ---
yindo commented 2026-02-16 17:30:17 -05:00
Author
Owner
Copy Link
Copy Source

@shantur commented on GitHub (Sep 29, 2025):

@thdxr - Any workaround or plan on this?

@shantur commented on GitHub (Sep 29, 2025): @thdxr - Any workaround or plan on this?
yindo commented 2026-02-16 17:30:17 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Sep 29, 2025):

@shantur so to confirm you want to be able to run opencode without any tools enabled at all? Or is it that opencode needs to send in different format for llama.cpp? Just trying to make sure i am following here because I haven't used llama.cpp yet

@rekram1-node commented on GitHub (Sep 29, 2025): @shantur so to confirm you want to be able to run opencode without any tools enabled at all? Or is it that opencode needs to send in different format for llama.cpp? Just trying to make sure i am following here because I haven't used llama.cpp yet
yindo commented 2026-02-16 17:30:17 -05:00
Author
Owner
Copy Link
Copy Source

@hollandThomas commented on GitHub (Oct 30, 2025):

Not entirely sure if related but I also can't seem to get OpenCode functioning properly with llama.cpp

OpenCode 0.15.29

# Start llama.cpp
~/llama.cpp/build/bin/llama-server -m ~/models/Q5_K_M/GLM-4.5-Air-Q5_K_M-merged.gguf --alias "GLM-4.5-Air-Q5_K_M" --host 0.0.0.0 --port 8080 --ctx-size 131072 --jinja

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "@ai-sdk/openai-compatible": {
      "name": "llama.cpp",
      "options": {
        "baseURL": "http://0.0.0.0:8080/v1"
      },
      "models": {
        "GLM-4.5": {
          "name": "GLM-4.5"
        }
      }
    }
  }
}

# Start an OpenCode session, Plan agent (this works). E.g.
> Please create a plan for a TypeScript project that is a Snake game that runs in the terminal. Use Bun as a runtime.

Here's a comprehensive plan for a TypeScript terminal Snake game using Bun:

## Project Structure

snake-game/
├── src/
│   ├── game/
│   │   ├── Game.ts         # Main game class
│   │   ├── Snake.ts        # Snake entity
│   │   ├── Food.ts         # Food entity
│   │   └── GameBoard.ts    # Game board and rendering
[…]

# Change to Build agent
> Please proceed with implementation

I'll implement the terminal Snake game in TypeScript using Bun. Let me start by creating the project structure and core
files. <tool_call>list <arg_key>path</arg_key> <arg_value>/home/tho/Projects/dummy</arg_value> </tool_call>

And there it just stops. I might be configuring it wrong tho, not sure 🤷‍♂️

Edit:
I hit it with "proceed" a couple of times and now it does something. It's all along these lines though. It managed to create a package.json but then failed to move on. Seems like I should figure out how to configure this better.

Terminal-Snake

The arguments provided to the tool are invalid: Model tried to call unavailable tool 'terminal-snake'. Available tools:
invalid, bash, edit, webfetch, glob, grep, list, read, write, todowrite, todoread, task.

<tool_call>write <arg_key>filePath</arg_key> <arg_value>/home/tho/Projects/dummy/tsconfig.json</arg_value>
<arg_key>content</arg_key> <arg_value>{ "compilerOptions": { "target": "ES2022", "module": "ESNext", "moduleResolution":
"bundler", "lib": ["ES2022"], "outDir": "./dist", "rootDir": "./src", "strict": true, "esModuleInterop": true,
"skipLibCheck": true, "forceConsistentCasingInFileNames": true, "declaration": true, "declarationMap": true, "sourceMap":
true, "removeComments": true, "noImplicitAny": true, "strictNullChecks": true, "strictFunctionTypes": true,
"noImplicitThis": true, "noImplicitReturns": true, "noFallthroughCasesInSwitch": true, "noUncheckedIndexedAccess": true,
"noImplicitOverride": true, "allowUnusedLabels": false, "allowUnreachableCode": false, "exactOptionalPropertyTypes": true,
"baseUrl": ".", "paths": { "@/": ["./src/"] } }, "include": ["src/**/*"], "exclude": ["node_modules", "dist"] }</arg_value>
</tool_call>
@hollandThomas commented on GitHub (Oct 30, 2025): Not entirely sure if related but I also can't seem to get OpenCode functioning properly with llama.cpp OpenCode 0.15.29 ```bash # Start llama.cpp ~/llama.cpp/build/bin/llama-server -m ~/models/Q5_K_M/GLM-4.5-Air-Q5_K_M-merged.gguf --alias "GLM-4.5-Air-Q5_K_M" --host 0.0.0.0 --port 8080 --ctx-size 131072 --jinja ``` ```json { "$schema": "https://opencode.ai/config.json", "provider": { "@ai-sdk/openai-compatible": { "name": "llama.cpp", "options": { "baseURL": "http://0.0.0.0:8080/v1" }, "models": { "GLM-4.5": { "name": "GLM-4.5" } } } } } ``` ```markdown # Start an OpenCode session, Plan agent (this works). E.g. > Please create a plan for a TypeScript project that is a Snake game that runs in the terminal. Use Bun as a runtime. Here's a comprehensive plan for a TypeScript terminal Snake game using Bun: ## Project Structure snake-game/ ├── src/ │ ├── game/ │ │ ├── Game.ts # Main game class │ │ ├── Snake.ts # Snake entity │ │ ├── Food.ts # Food entity │ │ └── GameBoard.ts # Game board and rendering […] ``` ```markdown # Change to Build agent > Please proceed with implementation I'll implement the terminal Snake game in TypeScript using Bun. Let me start by creating the project structure and core files. <tool_call>list <arg_key>path</arg_key> <arg_value>/home/tho/Projects/dummy</arg_value> </tool_call> ``` And there it just stops. I might be configuring it wrong tho, not sure 🤷‍♂️ Edit: I hit it with "proceed" a couple of times and now it does something. It's all along these lines though. It managed to create a package.json but then failed to move on. Seems like I should figure out how to configure this better. ``` Terminal-Snake The arguments provided to the tool are invalid: Model tried to call unavailable tool 'terminal-snake'. Available tools: invalid, bash, edit, webfetch, glob, grep, list, read, write, todowrite, todoread, task. <tool_call>write <arg_key>filePath</arg_key> <arg_value>/home/tho/Projects/dummy/tsconfig.json</arg_value> <arg_key>content</arg_key> <arg_value>{ "compilerOptions": { "target": "ES2022", "module": "ESNext", "moduleResolution": "bundler", "lib": ["ES2022"], "outDir": "./dist", "rootDir": "./src", "strict": true, "esModuleInterop": true, "skipLibCheck": true, "forceConsistentCasingInFileNames": true, "declaration": true, "declarationMap": true, "sourceMap": true, "removeComments": true, "noImplicitAny": true, "strictNullChecks": true, "strictFunctionTypes": true, "noImplicitThis": true, "noImplicitReturns": true, "noFallthroughCasesInSwitch": true, "noUncheckedIndexedAccess": true, "noImplicitOverride": true, "allowUnusedLabels": false, "allowUnreachableCode": false, "exactOptionalPropertyTypes": true, "baseUrl": ".", "paths": { "@/": ["./src/"] } }, "include": ["src/**/*"], "exclude": ["node_modules", "dist"] }</arg_value> </tool_call> ```
yindo commented 2026-02-16 17:30:17 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Oct 30, 2025):

@hollandThomas what is the context size you have llama cpp? This usually happens with local models when the llm doesnt get enough context, or the model may just suck at tool calling but it can happen in either case

@rekram1-node commented on GitHub (Oct 30, 2025): @hollandThomas what is the context size you have llama cpp? This usually happens with local models when the llm doesnt get enough context, or the model may just suck at tool calling but it can happen in either case
yindo commented 2026-02-16 17:30:17 -05:00
Author
Owner
Copy Link
Copy Source

@grigio commented on GitHub (Nov 2, 2025):

@rekram1-node I've 32684

      /app/llama-server
      --model /models/Qwen3-Coder-30B-A3B-Instruct-IQ4_NL.gguf
      --rope-scaling yarn
      --rope-scale 3.2
      --ctx-size 32684
      --seed 3407
      --jinja
      --temp 0.7
      --top-p 0.9
      --top-k 20
      --min-p 0.0
      --repeat-penalty 1.05
      -ngl 99
      --port 9999
@grigio commented on GitHub (Nov 2, 2025): @rekram1-node I've 32684 ``` /app/llama-server --model /models/Qwen3-Coder-30B-A3B-Instruct-IQ4_NL.gguf --rope-scaling yarn --rope-scale 3.2 --ctx-size 32684 --seed 3407 --jinja --temp 0.7 --top-p 0.9 --top-k 20 --min-p 0.0 --repeat-penalty 1.05 -ngl 99 --port 9999 ```
yindo commented 2026-02-16 17:30:17 -05:00
Author
Owner
Copy Link
Copy Source

@grigio commented on GitHub (Nov 2, 2025):

OK, this smaller model seems to work

  "Qwen3-0.6B-Q4_K_M":
    proxy: "http://127.0.0.1:9999"
    cmd: >
      /app/llama-server
      --model /models/Qwen3-0.6B-Q4_K_M.gguf
      --jinja
      --ctx-size 16384
      --seed 3407
      --prio 3
      --temp 0.6
      --min-p 0.0
      --top-p 0.95
      --top-k 20
      --port 9999
@grigio commented on GitHub (Nov 2, 2025): OK, this smaller model seems to work ``` "Qwen3-0.6B-Q4_K_M": proxy: "http://127.0.0.1:9999" cmd: > /app/llama-server --model /models/Qwen3-0.6B-Q4_K_M.gguf --jinja --ctx-size 16384 --seed 3407 --prio 3 --temp 0.6 --min-p 0.0 --top-p 0.95 --top-k 20 --port 9999 ```
yindo commented 2026-02-16 17:30:17 -05:00
Author
Owner
Copy Link
Copy Source

@marceldev89 commented on GitHub (Nov 5, 2025):

llama.cpp + Qwen3-Coder-* requires https://github.com/ggml-org/llama.cpp/pull/16755 to work properly. Been using this since august-ish and it's working fine besides some minor model specific issues.

@marceldev89 commented on GitHub (Nov 5, 2025): llama.cpp + Qwen3-Coder-* requires https://github.com/ggml-org/llama.cpp/pull/16755 to work properly. Been using this since august-ish and it's working fine besides some minor model specific issues.
yindo commented 2026-02-16 17:30:17 -05:00
Author
Owner
Copy Link
Copy Source

@grigio commented on GitHub (Nov 7, 2025):

@marceldev89 weird I found that the current llama.cpp + unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-GGUF works without changes, so probably is that some Qwen3-Coder-30B-A3B-Instruct models on HF have a broken chat template attached

@grigio commented on GitHub (Nov 7, 2025): @marceldev89 weird I found that the current llama.cpp + unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-GGUF works without changes, so probably is that some Qwen3-Coder-30B-A3B-Instruct models on HF have a broken chat template attached
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#1280
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?