anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-21 17:56:33 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

Why is opencode not working with local llms via Ollama? #2926

New Issue
Open
opened 2026-02-16 17:37:50 -05:00 by yindo · 36 comments
yindo commented 2026-02-16 17:37:50 -05:00
Owner
Copy Link
Copy Source

Originally created by @TheLevic on GitHub (Nov 17, 2025).

Originally assigned to: @rekram1-node on GitHub.

Question

Why is opencode not working with local llms via Ollama?
Hello. I have tried numerous local llms with opencode and I can not seem to get any to work. I have a decent PC that can run up to a 30b model smoothly. I have tried them. I can not get anything to work. Below is an example of what keeps happening. This is with llama3.2:3b.

Image

Any help is appreciated.

Image

EDIT: Added my config.

Originally created by @TheLevic on GitHub (Nov 17, 2025). Originally assigned to: @rekram1-node on GitHub. ### Question Why is opencode not working with local llms via Ollama? Hello. I have tried numerous local llms with opencode and I can not seem to get any to work. I have a decent PC that can run up to a 30b model smoothly. I have tried them. I can not get anything to work. Below is an example of what keeps happening. This is with llama3.2:3b. <img width="1871" height="842" alt="Image" src="https://github.com/user-attachments/assets/6d37997f-e75b-422b-8c1a-42ce1ff2bf65" /> Any help is appreciated. <img width="619" height="436" alt="Image" src="https://github.com/user-attachments/assets/82f85b0e-2d3c-4e3c-9217-61331776703b" /> EDIT: Added my config.
yindo added the model-problem label 2026-02-16 17:37:50 -05:00
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Nov 17, 2025):

Hm I don't think the llama models will treat you very well, also what is your context size for ollama

@rekram1-node commented on GitHub (Nov 17, 2025): Hm I don't think the llama models will treat you very well, also what is your context size for ollama
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@TheLevic commented on GitHub (Nov 17, 2025):

Hm I don't think the llama models will treat you very well

  • What would you recommmend?

also what is your context size for ollama

  • Right now it is set at 32k. Is this too small?
@TheLevic commented on GitHub (Nov 17, 2025): > Hm I don't think the llama models will treat you very well - What would you recommmend? > also what is your context size for ollama - Right now it is set at 32k. Is this too small?
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@Datal0re commented on GitHub (Nov 17, 2025):

Image

I think that it's viewing OpenCode as the context instead of the actual project directory.

@Datal0re commented on GitHub (Nov 17, 2025): <img width="1084" height="862" alt="Image" src="https://github.com/user-attachments/assets/9490de33-2010-4ad6-a89a-1252e0c8b05f" /> I think that it's viewing OpenCode as the context instead of the actual project directory.
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@TheLevic commented on GitHub (Nov 17, 2025):

I think that it's viewing OpenCode as the context instead of the actual project directory.

Exactly what I was thinking as well. Seems weird.

@TheLevic commented on GitHub (Nov 17, 2025): > I think that it's viewing OpenCode as the context instead of the actual project directory. Exactly what I was thinking as well. Seems weird.
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@1337hero commented on GitHub (Nov 17, 2025):

What you are experiencing is actually common with local models. This is classic tool calling capability gap. I use Anthropic most of the time, which has native, robust tool calling support.

But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem.

Some of the guys at unsloth have been working on this issue: https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/discussions/10

To provide you guys with context here:

Models must be specifically fine-tuned to understand structured tool calling

  • They gotta learn to output valid JSON in a specific format when tools are available
  • The HuggingFace discussion on Qwen3-Coder states it struggles because:
    • It outputs XML-style tags (<tool_call>, <function=...>) instead of JSON
    • It hallucinates tool names (like todonext) that don't exist
    • It doesn't consistently follow the expected format
@1337hero commented on GitHub (Nov 17, 2025): What you are experiencing is actually common with local models. This is classic tool calling capability gap. I use Anthropic most of the time, which has native, robust tool calling support. But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem. Some of the guys at unsloth have been working on this issue: https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/discussions/10 To provide you guys with context here: Models must be specifically fine-tuned to understand structured tool calling - They gotta learn to output valid JSON in a specific format when tools are available - The HuggingFace discussion on Qwen3-Coder states it struggles because: - It outputs XML-style tags (<tool_call>, <function=...>) instead of JSON - It hallucinates tool names (like todonext) that don't exist - It doesn't consistently follow the expected format
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@1337hero commented on GitHub (Nov 17, 2025):

I've been hinking about this - and there might be a PR here in theory. I believe LM Studio has a transformation layer that intercepts the models raw output, spots xml tool calls and changes them to json format.

hmm....

Looking at /packages/opencode/src/provider/transform.ts, in theory one could add a new transformation for Ollama/local models and Parse XML-style outputs and convert to JSON then hook into the wrapLanguageModel from AI SDK

maybe... I pulled this out my rear

@1337hero commented on GitHub (Nov 17, 2025): I've been hinking about this - and there might be a PR here in theory. I believe LM Studio has a transformation layer that intercepts the models raw output, spots xml tool calls and changes them to json format. hmm.... Looking at `/packages/opencode/src/provider/transform.ts,` in theory one could add a new transformation for Ollama/local models and Parse XML-style outputs and convert to JSON then hook into the `wrapLanguageModel` from AI SDK maybe... I pulled this out my rear
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Nov 17, 2025):

Yeah I think its possible technically

@rekram1-node commented on GitHub (Nov 17, 2025): Yeah I think its possible technically
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@1337hero commented on GitHub (Nov 17, 2025):

I started fiddling with this because I have ADHD and am now hyper fixated. This is def a challenge. I think however the model might still be the challenge. TBD

Image
@1337hero commented on GitHub (Nov 17, 2025): I started fiddling with this because I have ADHD and am now hyper fixated. This is def a challenge. I think however the model might still be the challenge. TBD <img width="2141" height="665" alt="Image" src="https://github.com/user-attachments/assets/ed2e4f48-d803-4e4b-a4ac-8e45b53a708c" />
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@Datal0re commented on GitHub (Nov 17, 2025):

But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem.

The issue isn't solely due to tiny models like llama3.2
This is an example with gpt-oss:20b

Image
@Datal0re commented on GitHub (Nov 17, 2025): > But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem. The issue isn't solely due to tiny models like llama3.2 This is an example with gpt-oss:20b <img width="714" height="346" alt="Image" src="https://github.com/user-attachments/assets/1818fead-2237-4e68-8922-04a9e0b7f20f" />
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@1337hero commented on GitHub (Nov 17, 2025):

But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem.

The issue isn't solely due to tiny models like llama3.2 This is an example with gpt-oss:20b

Image

Yeah it's the models ability to call the correct tools and use them. I've been testing with Qwen3-coder:30B for an hour now running with an idea. It hallucinates tool names nonstop. So an even smaller model will do even worse.

I built an adapter that catches when Qwen+Ollama mess up tool names, uses fuzzy matching to figure out what they actually meant (like mapping exec to bash), and fixes the call before it fails.

That was the easy part .... but it's still 💩

The problem is I get inconsistent behavior out the wazoo.

  • Sometimes outputs XML as plain text: <function=files></function></tool_call>
  • Sometimes makes actual tool calls (but with wrong arguments)
  • Sometimes just describes what it would do instead of doing it :|

TLDR:
✅ Fuzzy matching works
✅ Alias mapping works
❌ Model still can't edit files
❌ Model still can't run commands
❌ User experience is broken

I gotta think on this one. Maybe test more models. It might just be that the model itself is the limitation. Claude works great though. I love this TUI. I just discovered this whole project randomly earlier today when browsing Reddit about Ollama.

@1337hero commented on GitHub (Nov 17, 2025): > > But I also do a lot of local model stuff. Models like qwen3-coder:30b don't have it. I can see you're using Llama3.2:3b which is an incredibly small model which is likely also the problem. > > The issue isn't solely due to tiny models like llama3.2 This is an example with gpt-oss:20b > > <img alt="Image" width="714" height="346" src="https://private-user-images.githubusercontent.com/42654566/515520177-1818fead-2237-4e68-8922-04a9e0b7f20f.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjM0NDEzOTIsIm5iZiI6MTc2MzQ0MTA5MiwicGF0aCI6Ii80MjY1NDU2Ni81MTU1MjAxNzctMTgxOGZlYWQtMjIzNy00ZTY4LTg5MjItMDRhOWUwYjdmMjBmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTExMTglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUxMTE4VDA0NDQ1MlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTAyOWZkYmE4Nzk4OWUwNDI1MjVjNmY0MmRlMmJkMmIzMjBmYmRhNWRiYWY2M2E3OTEzYjMwZDAzM2M5ZjY3NjEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.ZxqQcBfZ2fqNoKC-E2fxOaO8pJwLnywyApmonBXF3mU"> Yeah it's the models ability to call the correct tools and use them. I've been testing with Qwen3-coder:30B for an hour now running with an idea. It hallucinates tool names nonstop. So an even smaller model will do even worse. I built an adapter that catches when Qwen+Ollama mess up tool names, uses fuzzy matching to figure out what they actually meant (like mapping exec to bash), and fixes the call before it fails. That was the easy part .... but it's still 💩 The problem is I get inconsistent behavior out the wazoo. - Sometimes outputs XML as plain text: `<function=files></function></tool_call>` - Sometimes makes actual tool calls (but with wrong arguments) - Sometimes just describes what it would do instead of doing it :| TLDR: ✅ Fuzzy matching works ✅ Alias mapping works ❌ Model still can't edit files ❌ Model still can't run commands ❌ User experience is broken I gotta think on this one. Maybe test more models. It might just be that the model itself is the limitation. Claude works great though. I love this TUI. I just discovered this whole project randomly earlier today when browsing Reddit about Ollama.
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Nov 18, 2025):

I have heard most people have better experiences with lmstudio over ollama

There isn't a ton we can do on our end besides maybe extra prompting for local models?

@rekram1-node commented on GitHub (Nov 18, 2025): I have heard most people have better experiences with lmstudio over ollama There isn't a ton we can do on our end besides maybe extra prompting for local models?
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@p0ns commented on GitHub (Nov 18, 2025):

Just tested both Ollama and LM Studio on Opencode with multiple models the past couple days.

After many issues with Ollama (mostly that all models default to a very small context window and you have to modify them or find versions with bigger context window settings, and tool call formatting issues), after installing LM Studio I was able to consistently use qwen/qwen3-30b-a3b-2507 with tools, and had varied success with openai/gpt-oss-20b, but it was still usable for shorter tasks.

TL;DR: ditch Ollama, try LM Studio

@p0ns commented on GitHub (Nov 18, 2025): Just tested both Ollama and LM Studio on Opencode with multiple models the past couple days. After many issues with Ollama (mostly that all models default to a very small context window and you have to modify them or find versions with bigger context window settings, and tool call formatting issues), after installing LM Studio I was able to consistently use `qwen/qwen3-30b-a3b-2507` with tools, and had varied success with `openai/gpt-oss-20b`, but it was still usable for shorter tasks. TL;DR: ditch Ollama, try LM Studio
yindo commented 2026-02-16 17:37:50 -05:00
Author
Owner
Copy Link
Copy Source

@1337hero commented on GitHub (Nov 18, 2025):

I was able to verify today that the MODEL (qwen3-coder:30b-a3b-q4_K_M) actually is making tool calls. The disconnect was happening between Ollama and the time it hit the AI SDK. Pretty sure Ollama's parser layer is the problem.

By the time it got to the SDK it was plain text.

// What Ollama returns:
  {
    "content": "<function=read>\n<parameter=file_path>/etc/hostname</parameter>"
  }

So need to intercept that at the middleware layer BEFORE it hits the AI SDK. But there might be another issue... TBD

I'll come back to this if I figure something out

@1337hero commented on GitHub (Nov 18, 2025): I was able to verify today that the MODEL (qwen3-coder:30b-a3b-q4_K_M) actually is making tool calls. The disconnect was happening between Ollama and the time it hit the AI SDK. Pretty sure Ollama's parser layer is the problem. By the time it got to the SDK it was plain text. ``` // What Ollama returns: { "content": "<function=read>\n<parameter=file_path>/etc/hostname</parameter>" } ``` So need to intercept that at the middleware layer BEFORE it hits the AI SDK. But there might be another issue... TBD I'll come back to this if I figure something out
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@jonahbenton commented on GitHub (Nov 18, 2025):

Ollama has 2 API surfaces- a "native" one that most ollama clients seem to use, and a claimed openai compatible API surface (the /v1 prefixed endpoints). From dev work I have done against them, the openai endpoints are not entirely openai compatible, some subtle issues both on normal streaming calls and on tool use. I see that opencode suggests treating ollama as openai-compatible- well, and their doc references llama 2, which is ancient- not sure what to say about that. I would think acting as an ollama client via openai endpoints isn't really going to work.

Ollama requires extra configuration to be able to use larger than 8k context sizes. Either the model has to know itself, or the server has to be told at startup via environment variables or whatever mechanism.

llama3.2:3b is really not very capable, I would not expect it to actually be able to do tool calls. I would only use for very simple small classification tasks. qwen3-coder with full context should be fine for tool and code analysis and small generation tasks.

I had been using cline against ollama using the qwen3-coder models (30b) very successfully, but for reasons switched to llama.cpp. cline seems to have context management issues when not talking to ollama or to a foundation model, so I was looking for a replacement. in limited testing so far opencode is working well using qwen3-coder 30b against llama.cpp, able to do all the range of tool use tasks I would expect.

@jonahbenton commented on GitHub (Nov 18, 2025): Ollama has 2 API surfaces- a "native" one that most ollama clients seem to use, and a claimed openai compatible API surface (the /v1 prefixed endpoints). From dev work I have done against them, the openai endpoints are not entirely openai compatible, some subtle issues both on normal streaming calls and on tool use. I see that opencode suggests treating ollama as openai-compatible- well, and their doc references llama 2, which is ancient- not sure what to say about that. I would think acting as an ollama client via openai endpoints isn't really going to work. Ollama requires extra configuration to be able to use larger than 8k context sizes. Either the model has to know itself, or the server has to be told at startup via environment variables or whatever mechanism. llama3.2:3b is really not very capable, I would not expect it to actually be able to do tool calls. I would only use for very simple small classification tasks. qwen3-coder with full context should be fine for tool and code analysis and small generation tasks. I had been using cline against ollama using the qwen3-coder models (30b) very successfully, but for reasons switched to llama.cpp. cline seems to have context management issues when not talking to ollama or to a foundation model, so I was looking for a replacement. in limited testing so far opencode is working well using qwen3-coder 30b against llama.cpp, able to do all the range of tool use tasks I would expect.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@1337hero commented on GitHub (Nov 18, 2025):

Yeah - I was able to get further down the rabbit hole if I turned off streaming.... this is just.... not the experience I imagine the dev team wants in the app. There is probably a solution here somewhere... but it's def one that is going to take a touch longer than vibe coding a solution in an hour. That ain't happening.

Still props to the dev team. I used this a bunch today with Claude and it might be my new favorite interface. The keybindings are great, config is easy to setup - you can tell it was crafted with love by NVIM users.

@1337hero commented on GitHub (Nov 18, 2025): Yeah - I was able to get further down the rabbit hole if I turned off streaming.... this is just.... not the experience I imagine the dev team wants in the app. There is probably a solution here somewhere... but it's def one that is going to take a touch longer than vibe coding a solution in an hour. That ain't happening. Still props to the dev team. I used this a bunch today with Claude and it might be my new favorite interface. The keybindings are great, config is easy to setup - you can tell it was crafted with love by NVIM users.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@TheLevic commented on GitHub (Nov 18, 2025):

Thanks everyone for the help and for looking into it. I will definitely switch models.

I completely agree. This tool is awesome. I will be using it for a while I suspect.

@TheLevic commented on GitHub (Nov 18, 2025): Thanks everyone for the help and for looking into it. I will definitely switch models. I completely agree. This tool is awesome. I will be using it for a while I suspect.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Nov 19, 2025):

We can prolly ask the ollama team what they'd recommend

@rekram1-node commented on GitHub (Nov 19, 2025): We can prolly ask the ollama team what they'd recommend
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@Mr-Philipp commented on GitHub (Dec 4, 2025):

i pretty much had the exact same experience not only with opencode but also with void. Using ollama with litellm in between.

Next i am going to test vllm because it seems to be way more efficient as well. Does anyone already has experience with it?

@Mr-Philipp commented on GitHub (Dec 4, 2025): i pretty much had the exact same experience not only with opencode but also with void. Using ollama with litellm in between. Next i am going to test vllm because it seems to be way more efficient as well. Does anyone already has experience with it?
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@1337hero commented on GitHub (Dec 6, 2025):

I saw this being asked a few times in the LocalLLM on Reddit... realizing many are still trying. I'll have to share what my attempt was and maybe some of us can work up a solution to put into a PR.

@1337hero commented on GitHub (Dec 6, 2025): I saw this being asked a few times in the LocalLLM on Reddit... realizing many are still trying. I'll have to share what my attempt was and maybe some of us can work up a solution to put into a PR.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@DragonDev1906 commented on GitHub (Dec 8, 2025):

I wish the opencode docs would contain not only recommended cloud models but also a list of open source models that work well (or at least are somewhat functional, if there are any). Together with required config changes or workarounds like increasing the context window size. Preferably not just for those with multiple 5090 but also for those limited to different sizes: 3b, 8b or 30b.

So far I've only managed to get a single 30b local model to emit valid tooling commands (after lots of searching, trying and getting annoyed at the current situation+docs).

@DragonDev1906 commented on GitHub (Dec 8, 2025): I wish the opencode docs would contain not only recommended cloud models but also a list of open source models that work well (or at least are somewhat functional, if there are any). Together with required config changes or workarounds like increasing the context window size. Preferably not just for those with multiple 5090 but also for those limited to different sizes: 3b, 8b or 30b. So far I've only managed to get a single 30b local model to emit valid tooling commands (after lots of searching, trying and getting annoyed at the current situation+docs).
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@yigitcanozdemir commented on GitHub (Dec 9, 2025):

I was able to verify today that the MODEL (qwen3-coder:30b-a3b-q4_K_M) actually is making tool calls. The disconnect was happening between Ollama and the time it hit the AI SDK. Pretty sure Ollama's parser layer is the problem.

By the time it got to the SDK it was plain text.

// What Ollama returns:
  {
    "content": "<function=read>\n<parameter=file_path>/etc/hostname</parameter>"
  }

So need to intercept that at the middleware layer BEFORE it hits the AI SDK. But there might be another issue... TBD

I'll come back to this if I figure something out

Hey yeah I saw qwen3-coder:30b making call tho. I tried gpt-oss:latest which is 20b model, it fails sometimes when making tool calls, its interesting. I have another problem maybe you have encountered. My model calls grep commands directly at home folder. Why this happens any idea?
(I run the opencode in repo folder and my repo folders in external drive as you can see right bottom side in the image)
NOTE: It works fine with cloud model providers (claude, zen, openai etc.), they use grep or other commands in repo folder.
Image

@yigitcanozdemir commented on GitHub (Dec 9, 2025): > I was able to verify today that the MODEL (qwen3-coder:30b-a3b-q4_K_M) actually is making tool calls. The disconnect was happening between Ollama and the time it hit the AI SDK. Pretty sure Ollama's parser layer is the problem. > > By the time it got to the SDK it was plain text. > > ``` > // What Ollama returns: > { > "content": "<function=read>\n<parameter=file_path>/etc/hostname</parameter>" > } > ``` > > So need to intercept that at the middleware layer BEFORE it hits the AI SDK. But there might be another issue... TBD > > I'll come back to this if I figure something out Hey yeah I saw qwen3-coder:30b making call tho. I tried gpt-oss:latest which is 20b model, it fails sometimes when making tool calls, its interesting. I have another problem maybe you have encountered. My model calls grep commands directly at home folder. Why this happens any idea? (I run the opencode in repo folder and my repo folders in external drive as you can see right bottom side in the image) NOTE: It works fine with cloud model providers (claude, zen, openai etc.), they use grep or other commands in repo folder. <img width="2119" height="478" alt="Image" src="https://github.com/user-attachments/assets/2a27b37d-3223-4d23-a296-4653df9945bc" />
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@1337hero commented on GitHub (Dec 24, 2025):

I am working on a training data set to fine tune a model for specifically React, Tanstack stuff, a bunch of other things I use - and part of that is going to include a training set to tune the model to properly output the needed formats for Open Code.

It will be a 30B model.

Aspect (opencode-toolcalls.jsonl)
System prompt Full tool definitions with JSON schema
Format Pure tool_calls JSON
Tool names Exact: read, write, edit, glob, grep, bash
Arguments Always valid JSON strings
Output null content + tool_calls array

Example format the model will learn:

  {
    "role": "assistant",
    "content": null,
    "tool_calls": [{
      "id": "call_001",
      "type": "function",
      "function": {
        "name": "read",
        "arguments": "{\"filePath\": \"/project/src/App.tsx\"}"
      }
    }]
  }

https://gist.github.com/1337hero/3fc50ca6e2080d246140afcc8f483997

Just pulled out of the tool schemas ex: opencode/packages/opencode/src/tool/glob.ts

@1337hero commented on GitHub (Dec 24, 2025): I am working on a training data set to fine tune a model for specifically React, Tanstack stuff, a bunch of other things I use - and part of that is going to include a training set to tune the model to properly output the needed formats for Open Code. It will be a 30B model. | Aspect | (opencode-toolcalls.jsonl) | |---------------|--------------------------------------------| | System prompt | Full tool definitions with JSON schema | | Format | Pure tool_calls JSON | | Tool names | Exact: read, write, edit, glob, grep, bash | | Arguments | Always valid JSON strings | | Output | null content + tool_calls array | Example format the model will learn: ``` { "role": "assistant", "content": null, "tool_calls": [{ "id": "call_001", "type": "function", "function": { "name": "read", "arguments": "{\"filePath\": \"/project/src/App.tsx\"}" } }] } ``` https://gist.github.com/1337hero/3fc50ca6e2080d246140afcc8f483997 Just pulled out of the tool schemas ex: `opencode/packages/opencode/src/tool/glob.ts`
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@davidbernat commented on GitHub (Jan 2, 2026):

@1337hero @rekram1-node
For people who find their way to this ticket via search engine as I did:
Is there a dedicated OpenCode working group to repairing this OpenCode + Local LLM use case?

I am hugely impressed with OpenCode, and mildly shocked by this gap. The industry relies on on-premise and open source methods in a huge number of use cases, far beyond privacy concerns, and and most engineers should empowered to presume tools exist to mirror (in parity) their commercial workflows, especially going into 2026. It is actually quite disconcerting, industry-wide, which is why I am here writing. Without discussing technical hurdles on this issue here, my experience with this issue is similar to yours: online searches for information mostly find only Reddit (owned by the same cohort of YC as OpenAI), and repeatedly the bots (and occasional human) of Reddit suggest LM Studio; however LM Studio is private business, and operates in-cloud with an SDK. It is like trying to drive your dune buggy around Baja California and each local shop tells you to buy a Tesla after showing your insurance card to Pennsylvania, or buggy without a throttle and steering system.

OpenCode is an excellent system. If you have a team that is dedicated to this issue I would appreciate being subscribed.

@davidbernat commented on GitHub (Jan 2, 2026): @1337hero @rekram1-node For people who find their way to this ticket via search engine as I did: Is there a dedicated OpenCode working group to repairing this OpenCode + Local LLM use case? I am hugely impressed with OpenCode, and mildly shocked by this gap. The industry relies on on-premise and open source methods in a huge number of use cases, far beyond privacy concerns, and and most engineers should empowered to presume tools exist to mirror (in parity) their commercial workflows, especially going into 2026. It is actually quite disconcerting, industry-wide, which is why I am here writing. Without discussing technical hurdles on this issue here, my experience with this issue is similar to yours: online searches for information mostly find only Reddit (owned by the same cohort of YC as OpenAI), and repeatedly the bots (and occasional human) of Reddit suggest LM Studio; however LM Studio is private business, and operates in-cloud with an SDK. It is like trying to drive your dune buggy around Baja California and each local shop tells you to buy a Tesla after showing your insurance card to Pennsylvania, or buggy without a throttle and steering system. OpenCode is an excellent system. If you have a team that is dedicated to this issue I would appreciate being subscribed.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@ateixeiramunoz commented on GitHub (Jan 9, 2026):

Hello people:

1.- I was banned from anthropic 4 days ago..... XD (fun, just 1 week of vibe coding got me banned) cause I created a "clever" super wrapper for claude code (That's what I understood).
2.- I was working at the same time on a very very detailed architecture for other really big project that, when considered by AI models, allows them to ez build a lot of stuff just cause the architecture is pre-defining "nearly everything".
3.- After being banned from anthropic, I started creating my own "model/agent/claude-code-like". It's been like 2 days of coding and it's kinda "intelligent". it can do commits and basic stuff, but still hallucinating.
4.- How? I created a bunch of deteministic rules for behaviour, for hallucination, differente agentic personalities, different roles, orchestrators, etc...
5.- For those wondering: yes, the basic Idea is to input always rules to the agents or skills (whatever you want to do, does not matter) in order to format with Json. This way you can define like an own "MCP" or bunch of skills and funcionts that can talk between them
4.- When you have all of this, then you need to create a binary that controls and starts the model with ur preferences, etc. and routes commands and responses to / from the console-

it's hard work, not ez task, but not impossible. Maybe I'll update if I continue pushing for it.

hope it lhelps

@ateixeiramunoz commented on GitHub (Jan 9, 2026): Hello people: 1.- I was banned from anthropic 4 days ago..... XD (fun, just 1 week of vibe coding got me banned) cause I created a "clever" super wrapper for claude code (That's what I understood). 2.- I was working at the same time on a very very detailed architecture for other really big project that, when considered by AI models, allows them to ez build a lot of stuff just cause the architecture is pre-defining "nearly everything". 3.- After being banned from anthropic, I started creating my own "model/agent/claude-code-like". It's been like 2 days of coding and it's kinda "intelligent". it can do commits and basic stuff, but still hallucinating. 4.- How? I created a bunch of deteministic rules for behaviour, for hallucination, differente agentic personalities, different roles, orchestrators, etc... 5.- For those wondering: yes, the basic Idea is to input always rules to the agents or skills (whatever you want to do, does not matter) in order to format with Json. This way you can define like an own "MCP" or bunch of skills and funcionts that can talk between them 4.- When you have all of this, then you need to create a binary that controls and starts the model with ur preferences, etc. and routes commands and responses to / from the console- it's hard work, not ez task, but not impossible. Maybe I'll update if I continue pushing for it. hope it lhelps
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@pawanjenu commented on GitHub (Jan 17, 2026):

facing the same issue with:
GLM4.7/UD-IQ2_M

below is the ollama model file used:

FROM /custom_models/GLM4.7/UD-IQ2_M/GLM-4.7-MERGED.gguf

PARAMETER num_ctx 65536
# Use a low temperature for better JSON/Tool accuracy
PARAMETER temperature 0.1
PARAMETER num_gpu 999
PARAMETER num_predict 4096

# --- Tool-Enabled Agentic Template ---
TEMPLATE """[gMASK]<sop>{{ if .System }}<|system|>
{{ .System }}{{ end }}{{ if .Tools }}<|system|>
In addition to plain text responses, you can choose to call one or more of the provided functions.
Use the following rule to decide when to call a function:
* if you need external information or need to edit files, generate a function call.
If you decide to call functions:
* all function calls should be generated in a single JSON list formatted as functools[{"name": [function name], "arguments": [function arguments as JSON]}, ...]
Available functions as JSON spec:
{{ .Tools }}<|eot_id|>{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}{{ end }}<|assistant|>
{{- if .Response }}
{{ .Response }}
{{- end }}
{{- if .ToolCalls }}
<|thought|>
I will call the following tools to fulfill the requirement:
functools[
{{- range .ToolCalls }}
{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}
]
{{- end }}"""

# --- Senior Engineer System Prompt ---
SYSTEM """You are a Senior Software Engineer with deep access to the system.
You MUST use the provided file-editing tools (edit, write, read) for all building tasks.
Always provide a brief reasoning block (<|thought|>) before emitting a tool call.
Output production-grade, secure code following Clean Code principles."""
@pawanjenu commented on GitHub (Jan 17, 2026): facing the same issue with: `GLM4.7/UD-IQ2_M` below is the ollama model file used: ``` FROM /custom_models/GLM4.7/UD-IQ2_M/GLM-4.7-MERGED.gguf PARAMETER num_ctx 65536 # Use a low temperature for better JSON/Tool accuracy PARAMETER temperature 0.1 PARAMETER num_gpu 999 PARAMETER num_predict 4096 # --- Tool-Enabled Agentic Template --- TEMPLATE """[gMASK]<sop>{{ if .System }}<|system|> {{ .System }}{{ end }}{{ if .Tools }}<|system|> In addition to plain text responses, you can choose to call one or more of the provided functions. Use the following rule to decide when to call a function: * if you need external information or need to edit files, generate a function call. If you decide to call functions: * all function calls should be generated in a single JSON list formatted as functools[{"name": [function name], "arguments": [function arguments as JSON]}, ...] Available functions as JSON spec: {{ .Tools }}<|eot_id|>{{ end }}{{ if .Prompt }}<|user|> {{ .Prompt }}{{ end }}<|assistant|> {{- if .Response }} {{ .Response }} {{- end }} {{- if .ToolCalls }} <|thought|> I will call the following tools to fulfill the requirement: functools[ {{- range .ToolCalls }} {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{- end }} ] {{- end }}""" # --- Senior Engineer System Prompt --- SYSTEM """You are a Senior Software Engineer with deep access to the system. You MUST use the provided file-editing tools (edit, write, read) for all building tasks. Always provide a brief reasoning block (<|thought|>) before emitting a tool call. Output production-grade, secure code following Clean Code principles.""" ```
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@Mr-Philipp commented on GitHub (Jan 17, 2026):

Update: I played a few days with opencode, ollama, litellm and other LLM engines and what i found out is: just don't use ollama. It's simple to deploy but more for testing purposes. It does not do proper tool calling and in general lacks a lot of features.

@Mr-Philipp commented on GitHub (Jan 17, 2026): Update: I played a few days with opencode, ollama, litellm and other LLM engines and what i found out is: just don't use ollama. It's simple to deploy but more for testing purposes. It does not do proper tool calling and in general lacks a lot of features.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@pawanjenu commented on GitHub (Jan 17, 2026):

@Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models.
I have around 250-300VRAM for it.

@pawanjenu commented on GitHub (Jan 17, 2026): @Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@Mr-Philipp commented on GitHub (Jan 17, 2026):

@Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it.

i moved to vllm and all my problems basically fixed themselves.

nice setup! can you elaborate what GPU's you use and how they are connected?

@Mr-Philipp commented on GitHub (Jan 17, 2026): > [@Mr-Philipp](https://github.com/Mr-Philipp) , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it. i moved to vllm and all my problems basically fixed themselves. nice setup! can you elaborate what GPU's you use and how they are connected?
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@pawanjenu commented on GitHub (Jan 17, 2026):

@Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it.

i moved to vllm and all my problems basically fixed themselves.

nice setup! can you elaborate what GPU's you use and how they are connected?

if possible can you guide me with the setup you did?

  1. which model you used
  2. parameter you used etc
  3. how did you connect opencode and vllm ?

Also my setup:
I have 4 NVIDIA H200, its setup in a store room with AC's

@pawanjenu commented on GitHub (Jan 17, 2026): > > [@Mr-Philipp](https://github.com/Mr-Philipp) , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it. > > i moved to vllm and all my problems basically fixed themselves. > > nice setup! can you elaborate what GPU's you use and how they are connected? if possible can you guide me with the setup you did? 1. which model you used 2. parameter you used etc 3. how did you connect opencode and vllm ? Also my setup: I have 4 NVIDIA H200, its setup in a store room with AC's
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@Mr-Philipp commented on GitHub (Jan 19, 2026):

@Mr-Philipp , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it.

i moved to vllm and all my problems basically fixed themselves.
nice setup! can you elaborate what GPU's you use and how they are connected?

if possible can you guide me with the setup you did?

  1. which model you used
  2. parameter you used etc
  3. how did you connect opencode and vllm ?

Also my setup: I have 4 NVIDIA H200, its setup in a store room with AC's

have a look at https://github.com/BoltzmannEntropy/vLLM-5090 to get some inspiration.

@Mr-Philipp commented on GitHub (Jan 19, 2026): > > > [@Mr-Philipp](https://github.com/Mr-Philipp) , what other alternate do you suggest if not ollama, my main goal is to make it work with opencode with local models. I have around 250-300VRAM for it. > > > > > > i moved to vllm and all my problems basically fixed themselves. > > nice setup! can you elaborate what GPU's you use and how they are connected? > > if possible can you guide me with the setup you did? > > 1. which model you used > 2. parameter you used etc > 3. how did you connect opencode and vllm ? > > Also my setup: I have 4 NVIDIA H200, its setup in a store room with AC's have a look at https://github.com/BoltzmannEntropy/vLLM-5090 to get some inspiration.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@nesterow commented on GitHub (Jan 25, 2026):

I made Qwen 30B work by using appropriate context length: https://docs.ollama.com/context-length

@nesterow commented on GitHub (Jan 25, 2026): I made Qwen 30B work by using appropriate context length: https://docs.ollama.com/context-length
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@rasoliveira commented on GitHub (Feb 5, 2026):

Setting the context length as mentioned by @nesterow worked for me!

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

Unbelievable difference. Worked for me with with gpt-oss:20b, qwen3-coder:30b.

Bumped OLLAMA_CONTEXT_LENGTH=128000 for nemotron-3-nano:30b. It thinks too much!

@rasoliveira commented on GitHub (Feb 5, 2026): Setting the context length as mentioned by @nesterow worked for me! ``` OLLAMA_CONTEXT_LENGTH=64000 ollama serve ``` Unbelievable difference. Worked for me with with `gpt-oss:20b`, `qwen3-coder:30b`. Bumped `OLLAMA_CONTEXT_LENGTH=128000` for `nemotron-3-nano:30b`. It thinks too much!
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@shochdoerfer commented on GitHub (Feb 5, 2026):

Ollama was working fine for me with the 1.1.20ish versions of Opencode. After upgrading to 1.1.30 and 1.1.40 I am not getting any feedback in OpenCode. The Ollama setup works fine, e.g. in our OpenWebUI setup. So it seems to be an Opencode issue.

@shochdoerfer commented on GitHub (Feb 5, 2026): Ollama was working fine for me with the 1.1.20ish versions of Opencode. After upgrading to 1.1.30 and 1.1.40 I am not getting any feedback in OpenCode. The Ollama setup works fine, e.g. in our OpenWebUI setup. So it seems to be an Opencode issue.
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@davidbernat commented on GitHub (Feb 12, 2026):

To summarize this ticket since I last posted:

@rasoliveira suggests that OLLAMA_CONTEXT_LENGTH really does solve this issue, although keep in mind that that sets the context length for every API request, not per-request, which would be ideal.

@pawanjenu Could you kindly describe where you built your "# --- Tool-Enabled Agentic Template ---" from? That looks very technical for an LLM to process, and I am curious about the reasoning behind that?

For me, I leveraged the 32GB RAM to load 20GB models onto an 8GB GPU and got typical robust tool usages with Ollama. I have not had a chance to use the new 32GB GPU with 96GB RAM to load large-LLMs which may be sufficient to do the multi-sequence OpenCode style calls, switching off between MCPs, tool uses, file reads, and generations with large contexts. I will return when I get a chance to do that.

@shochdoerfer what did you experience before & after the update?

@Mr-Philipp is vLLM (from Berkeley) a model service leveraging most HuggingFace models? In that regard, you are suggesting that is a one-to-one replacement for Ollama? (Except perhaps the CLI interface of Ollama?) Is it validated, tested?

@davidbernat commented on GitHub (Feb 12, 2026): To summarize this ticket since I last posted: @rasoliveira suggests that `OLLAMA_CONTEXT_LENGTH` really does solve this issue, although keep in mind that that sets the context length for every API request, not per-request, which would be ideal. @pawanjenu Could you kindly describe where you built your "# --- Tool-Enabled Agentic Template ---" from? That looks very technical for an LLM to process, and I am curious about the reasoning behind that? For me, I leveraged the 32GB RAM to load 20GB models onto an 8GB GPU and got typical robust tool usages with Ollama. I have not had a chance to use the new 32GB GPU with 96GB RAM to load large-LLMs which may be sufficient to do the multi-sequence OpenCode style calls, switching off between MCPs, tool uses, file reads, and generations with large contexts. I will return when I get a chance to do that. @shochdoerfer what did you experience before & after the update? @Mr-Philipp is vLLM (from Berkeley) a model service leveraging most HuggingFace models? In that regard, you are suggesting that is a one-to-one replacement for Ollama? (Except perhaps the CLI interface of Ollama?) Is it validated, tested?
yindo commented 2026-02-16 17:37:51 -05:00
Author
Owner
Copy Link
Copy Source

@shochdoerfer commented on GitHub (Feb 13, 2026):

@davidbernat after upgrading to Ollama 0.15, I got opencode working again.

Before that the opencode UI would just do nothing. It did communicate with Ollama - as in the LLM instance got properly started - but even waiting for 30min showed no reponse in the opencode UI. Also no errors in the logs which made it really hard to debug.

@shochdoerfer commented on GitHub (Feb 13, 2026): @davidbernat after upgrading to Ollama 0.15, I got opencode working again. Before that the opencode UI would just do nothing. It did communicate with Ollama - as in the LLM instance got properly started - but even waiting for 30min showed no reponse in the opencode UI. Also no errors in the logs which made it really hard to debug.
yindo commented 2026-02-16 17:37:52 -05:00
Author
Owner
Copy Link
Copy Source

@Mr-Philipp commented on GitHub (Feb 13, 2026):

@Mr-Philipp is vLLM (from Berkeley) a model service leveraging most HuggingFace models? In that regard, you are suggesting that is a one-to-one replacement for Ollama? (Except perhaps the CLI interface of Ollama?) Is it validated, tested?

Yes, vllm can load HuggingFace models natively but you need to select exactly what model type you want to load. vllm is openai-API compatible but in my case i am using litellm in front of it. It is not a one-to-one replacement because of how it works and how it needs to be configured is different and more complex.

Some differences:

  • vllm does not support switching models on demand. you start it with one model and that's it
  • vllm is able to handle multiple requests at a time

It's like MacOS vs Arch Linux. There is no answer which one is better - it depends on what you want it to do.

@Mr-Philipp commented on GitHub (Feb 13, 2026): > [@Mr-Philipp](https://github.com/Mr-Philipp) is vLLM (from Berkeley) a model service leveraging most HuggingFace models? In that regard, you are suggesting that is a one-to-one replacement for Ollama? (Except perhaps the CLI interface of Ollama?) Is it validated, tested? Yes, vllm can load HuggingFace models natively but you need to select exactly what model type you want to load. vllm is openai-API compatible but in my case i am using litellm in front of it. It is not a one-to-one replacement because of how it works and how it needs to be configured is different and more complex. Some differences: * vllm does not support switching models on demand. you start it with one model and that's it * vllm is able to handle multiple requests at a time It's like MacOS vs Arch Linux. There is no answer which one is better - it depends on what you want it to do.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label model-problem
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#2926
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?