anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 18:25:32 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

Using local Ollama models doesnt return any results. #4304

New Issue
Open
opened 2026-02-16 17:43:21 -05:00 by yindo · 20 comments
yindo commented 2026-02-16 17:43:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @padsbanger on GitHub (Jan 6, 2026).

Originally assigned to: @rekram1-node on GitHub.

Question

Hello, I am running Opencode using local llms from ollama:

  • qwen2.5-coder:7b
  • qwen2.5-coder:32b
  • qwen2.5-coder:32b
  • codellama:34b

os: Omarchy
gpu: rtx 4070ti

When I run them on my projects (generic React crud apps) using /init command I get following responses:

{"name": "todoread", "arguments": {}}

or

{"name": "read", "arguments": {"path": "/home/Projects/react-app/AGENTS.md"}}

Here is my opencode.json file:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (Local)",
      "options": {
        "baseURL": "http://127.0.0.1:11434/v1",
        "apiKey": "ollama"  // Optional dummy key; some setups need it
      },
      "models": {
        "qwen2.5-coder:7b": {
          "name": "Qwen2.5-Coder 7B"
        },
        "qwen2.5-coder:32b": {
          "name": "Qwen2.5-Coder 32B"
        },
        "deepseek-coder-v2": {
          "name": "DeepSeek-Coder-V2 16B"
        },
        "qwen3-coder": {
          "name": "Qwen3-Coder 30B"
        },
        "glm4": {
          "name": "GLM-4"
        },
        "codellama:34b": {
          "name": "CodeLlama 34B"
        },
        "codestral": {
          "name": "Codestral"
        },
        "gpt-oss": {
          "name": "GPT-OSS"
        }
      }
    }
  },
  "model": "ollama/qwen2.5-coder:32b" // Default: provider_id/model_id format
}

When I switch to cloud providers such as Copilot of Grok Code Fast, I get desired results (agents.md files). Any suggetions what I am doing wrong ?

Originally created by @padsbanger on GitHub (Jan 6, 2026). Originally assigned to: @rekram1-node on GitHub. ### Question Hello, I am running Opencode using local llms from ollama: - qwen2.5-coder:7b - qwen2.5-coder:32b - qwen2.5-coder:32b - codellama:34b os: Omarchy gpu: rtx 4070ti When I run them on my projects (generic React crud apps) using /init command I get following responses: `{"name": "todoread", "arguments": {}}` or `{"name": "read", "arguments": {"path": "/home/Projects/react-app/AGENTS.md"}}` Here is my opencode.json file: ``` { "$schema": "https://opencode.ai/config.json", "provider": { "ollama": { "npm": "@ai-sdk/openai-compatible", "name": "Ollama (Local)", "options": { "baseURL": "http://127.0.0.1:11434/v1", "apiKey": "ollama" // Optional dummy key; some setups need it }, "models": { "qwen2.5-coder:7b": { "name": "Qwen2.5-Coder 7B" }, "qwen2.5-coder:32b": { "name": "Qwen2.5-Coder 32B" }, "deepseek-coder-v2": { "name": "DeepSeek-Coder-V2 16B" }, "qwen3-coder": { "name": "Qwen3-Coder 30B" }, "glm4": { "name": "GLM-4" }, "codellama:34b": { "name": "CodeLlama 34B" }, "codestral": { "name": "Codestral" }, "gpt-oss": { "name": "GPT-OSS" } } } }, "model": "ollama/qwen2.5-coder:32b" // Default: provider_id/model_id format } ``` When I switch to cloud providers such as Copilot of Grok Code Fast, I get desired results (agents.md files). Any suggetions what I am doing wrong ?
yindo commented 2026-02-16 17:43:21 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Jan 6, 2026):

This issue might be a duplicate of existing issues. Please check:

  • #5694: Local Ollama models are not agentic
  • #4488: Ollama integration: API returns HTTP 200 but responses not displayed
  • #5187: Ollama: User message content arrives as empty array - model cannot see user input
  • #3265: ollama integration no results returned
  • #5210: Custom OpenAI-compatible provider returns no text content
  • #7030: Ollama (qwen2.5-coder): tool calls (edit/write) show as executed but no files are created/modified

Feel free to ignore if none of these address your specific case.

@github-actions[bot] commented on GitHub (Jan 6, 2026): This issue might be a duplicate of existing issues. Please check: - #5694: Local Ollama models are not agentic - #4488: Ollama integration: API returns HTTP 200 but responses not displayed - #5187: Ollama: User message content arrives as empty array - model cannot see user input - #3265: ollama integration no results returned - #5210: Custom OpenAI-compatible provider returns no text content - #7030: Ollama (qwen2.5-coder): tool calls (edit/write) show as executed but no files are created/modified Feel free to ignore if none of these address your specific case.
yindo commented 2026-02-16 17:43:21 -05:00
Author
Owner
Copy Link
Copy Source

@aaronnewsome commented on GitHub (Jan 6, 2026):

if you're able to use those models, you are probably able to use qwen3-coder:30b-a3b, which is a much better model for doing tool calls. i've just tried qwen3-coder:30b-a3b on my setup and the tool calls do work, at least for the quick test i did. the biggest issue i've seen with ollama models is they mostly are horrible at tool calling, nothing specific to ollama though, they are horrible at tool calling no matter what they're running on. you might also look into mistral models or most models much newer than qwen2.5, which are specifically engineered for more reliable tool calling.

@aaronnewsome commented on GitHub (Jan 6, 2026): if you're able to use those models, you are probably able to use qwen3-coder:30b-a3b, which is a much better model for doing tool calls. i've just tried qwen3-coder:30b-a3b on my setup and the tool calls do work, at least for the quick test i did. the biggest issue i've seen with ollama models is they mostly are horrible at tool calling, nothing specific to ollama though, they are horrible at tool calling no matter what they're running on. you might also look into mistral models or most models much newer than qwen2.5, which are specifically engineered for more reliable tool calling.
yindo commented 2026-02-16 17:43:21 -05:00
Author
Owner
Copy Link
Copy Source

@kempy007 commented on GitHub (Jan 6, 2026):

Plus one!!!!

I also have a similar issue trying nemotron-3-nano:30b from ollama.
If I start opencode leave it in build mode on the splash prompt and ask a question, it works along with subsequent prompts, tool use fails trying to get it to ping 8.8.8.8 three times, it used the wrong tools and lacked definition to use bash when explicitly asked to.
if the mode is switched to plan or even back to build, it just replies to the system prompt.

Ok seems this is all related to the context window, see https://opencode.ai/docs/providers/#ollama and https://blog.driftingruby.com/ollama-context-window/
A simple greeting uses 10k context and the ollama default is 8k. @padsbanger try this ctx window increase.

@kempy007 commented on GitHub (Jan 6, 2026): Plus one!!!! I also have a similar issue trying nemotron-3-nano:30b from ollama. If I start opencode leave it in build mode on the splash prompt and ask a question, it works along with subsequent prompts, tool use fails trying to get it to ping 8.8.8.8 three times, it used the wrong tools and lacked definition to use bash when explicitly asked to. if the mode is switched to plan or even back to build, it just replies to the system prompt. Ok seems this is all related to the context window, see https://opencode.ai/docs/providers/#ollama and https://blog.driftingruby.com/ollama-context-window/ A simple greeting uses 10k context and the ollama default is 8k. @padsbanger try this ctx window increase.
yindo commented 2026-02-16 17:43:21 -05:00
Author
Owner
Copy Link
Copy Source

@padsbanger commented on GitHub (Jan 6, 2026):

Plus one!!!!

I also have a similar issue trying nemotron-3-nano:30b from ollama. If I start opencode leave it in build mode on the splash prompt and ask a question, it works along with subsequent prompts, tool use fails trying to get it to ping 8.8.8.8 three times, it used the wrong tools and lacked definition to use bash when explicitly asked to. if the mode is switched to plan or even back to build, it just replies to the system prompt.

Ok seems this is all related to the context window, see https://opencode.ai/docs/providers/#ollama and https://blog.driftingruby.com/ollama-context-window/ A simple greeting uses 10k context and the ollama default is 8k. @padsbanger try this ctx window increase.

Thank you, increasing context windows did something (I increased to 32k). However I have run into another issue. When I run /init on repo in Build mode, it outputs makrdown file, however it is unable to create this file.

I get this response from devstral:

I'm sorry for any confusion, but I don't have the capability to directly create files or use tools in the way you're asking. However, I can certainly help guide you on how to create a agents.md file yourself! If you provide me with the content you'd like to include in the file, I can help you format it properly for Markdown.
Here's an example of what the file might look like if you want to document different types of agents:

It is weird, because devstral supports tools and file access.

@padsbanger commented on GitHub (Jan 6, 2026): > Plus one!!!! > > I also have a similar issue trying nemotron-3-nano:30b from ollama. If I start opencode leave it in build mode on the splash prompt and ask a question, it works along with subsequent prompts, tool use fails trying to get it to ping 8.8.8.8 three times, it used the wrong tools and lacked definition to use bash when explicitly asked to. if the mode is switched to plan or even back to build, it just replies to the system prompt. > > Ok seems this is all related to the context window, see https://opencode.ai/docs/providers/#ollama and https://blog.driftingruby.com/ollama-context-window/ A simple greeting uses 10k context and the ollama default is 8k. [@padsbanger](https://github.com/padsbanger) try this ctx window increase. Thank you, increasing context windows did something (I increased to 32k). However I have run into another issue. When I run /init on repo in Build mode, it outputs makrdown file, however it is unable to create this file. I get this response from devstral: ``` I'm sorry for any confusion, but I don't have the capability to directly create files or use tools in the way you're asking. However, I can certainly help guide you on how to create a agents.md file yourself! If you provide me with the content you'd like to include in the file, I can help you format it properly for Markdown. Here's an example of what the file might look like if you want to document different types of agents: ``` It is weird, because devstral supports tools and file access.
yindo commented 2026-02-16 17:43:21 -05:00
Author
Owner
Copy Link
Copy Source

@aaronnewsome commented on GitHub (Jan 6, 2026):

i notice you don't have the tool use param in your opencode config. you should probably put that in. here's what I have for my ollama server

"ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://hawk.swift.local:11434/v1",
        "includeUsage": true,
        "timeout": 1000000
      },
      "models": {
        "qwen3-coder:30b": {
          "name": "qwen3-coder:30b",
          "tool_call": true,
          "reasoning": true,
          "cost": {
            "input": 0.10,
            "output": 1.20
          }
        },
	"gpt-oss:120b": {
          "name": "gpt-oss:120b",
          "tool_call": true,
          "reasoning": true,
          "cost": {
            "input": 0.10,
            "output": 1.20
          }
        },
        "ministral-3:14b": {
          "name": "ministral-3:14b",
          "tool_call": true,
          "reasoning": true,
          "cost": {
            "input": 0.10,
            "output": 1.20
          }
        },
	"deepseek-r1:32b": {
          "name": "deepseek-r1:32b",
          "tool_call": true,
          "reasoning": true,
          "cost": {
            "input": 0.10,
            "output": 1.20
          }
        }
@aaronnewsome commented on GitHub (Jan 6, 2026): i notice you don't have the tool use param in your opencode config. you should probably put that in. here's what I have for my ollama server ``` "ollama": { "npm": "@ai-sdk/openai-compatible", "options": { "baseURL": "http://hawk.swift.local:11434/v1", "includeUsage": true, "timeout": 1000000 }, "models": { "qwen3-coder:30b": { "name": "qwen3-coder:30b", "tool_call": true, "reasoning": true, "cost": { "input": 0.10, "output": 1.20 } }, "gpt-oss:120b": { "name": "gpt-oss:120b", "tool_call": true, "reasoning": true, "cost": { "input": 0.10, "output": 1.20 } }, "ministral-3:14b": { "name": "ministral-3:14b", "tool_call": true, "reasoning": true, "cost": { "input": 0.10, "output": 1.20 } }, "deepseek-r1:32b": { "name": "deepseek-r1:32b", "tool_call": true, "reasoning": true, "cost": { "input": 0.10, "output": 1.20 } } ```
yindo commented 2026-02-16 17:43:21 -05:00
Author
Owner
Copy Link
Copy Source

@GlenM42 commented on GitHub (Jan 6, 2026):

@aaronnewsome, adding tool_call still does not help

@GlenM42 commented on GitHub (Jan 6, 2026): @aaronnewsome, adding `tool_call` still does not help
yindo commented 2026-02-16 17:43:21 -05:00
Author
Owner
Copy Link
Copy Source

@padsbanger commented on GitHub (Jan 7, 2026):

I think I have tested most popular models, that can run on my graphics card (rtx 4070ti, 12gb vram). Tried different context windows, different flags, like "tool_call", "reasoning" etc. All of the time, LLMs just keep hallucinating, not generating any results (I have tried /init as a benchmark). Are there any ollama models that can actually run locally and still be useful in Opencode with programming tasks ? When I run these models in ollama cli they work fine.

@padsbanger commented on GitHub (Jan 7, 2026): I think I have tested most popular models, that can run on my graphics card (rtx 4070ti, 12gb vram). Tried different context windows, different flags, like "tool_call", "reasoning" etc. All of the time, LLMs just keep hallucinating, not generating any results (I have tried /init as a benchmark). Are there any ollama models that can actually run locally and still be useful in Opencode with programming tasks ? When I run these models in ollama cli they work fine.
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@aaronnewsome commented on GitHub (Jan 7, 2026):

i too have had REALLY bad luck with small models and opencode. I've found qwen3-coder:30b to be one of the better ones, gpt-oss:120b is supported well in Ollama and runs decently, but really any model I would run on Ollama has a ton of failed tool calls over time, which really slows down your workflow. My best, most stable local models for coding specifically have been, glm-4.5, glm-4.7, minimax m2 and minimax m2.1. i've also had pretty good luck with qwen3-coder:480b. if you must use ollama rather than llama.cpp, i'd suggest at least using the Q8 quant of your preferred model, those seem a bit smarter than the Q4 ollama quants.

@aaronnewsome commented on GitHub (Jan 7, 2026): i too have had REALLY bad luck with small models and opencode. I've found qwen3-coder:30b to be one of the better ones, gpt-oss:120b is supported well in Ollama and runs decently, but really any model I would run on Ollama has a ton of failed tool calls over time, which really slows down your workflow. My best, most stable local models for coding specifically have been, glm-4.5, glm-4.7, minimax m2 and minimax m2.1. i've also had pretty good luck with qwen3-coder:480b. if you must use ollama rather than llama.cpp, i'd suggest at least using the Q8 quant of your preferred model, those seem a bit smarter than the Q4 ollama quants.
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@padsbanger commented on GitHub (Jan 7, 2026):

qwen3-coder:480b fails to run my on PC, i have only 32gb ram :/

@padsbanger commented on GitHub (Jan 7, 2026): qwen3-coder:480b fails to run my on PC, i have only 32gb ram :/
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@davidbernat commented on GitHub (Jan 7, 2026):

For those who are arriving here from search engines, this appears to be a major ongoing difficulty with OpenCode for on premise usage. I did a substantial review across the web of this issue several days ago, and found very many different branches of user problems, though all reduce to the notion that OpenCode does not have a local open source LLM solution. As summarized above, most users discuss the Ollama context length limitations (which has its own foibles, and does not solve all problems) and others discuss the use of the Qwen30b model as the only one which works. Generally, the discussion tends to circle around a drain right at the point someone says "the way Ollama uses tools is [negative word]". I cannot speak to knowing how the cool calling mechanisms are implemented (other than enforcing JSON responses, and some light parsing of non-JSON string responses to coerce them into JSON), and I have also seen problems with PydanticAI being too opaque to see in what ways they are editing their prompts and response closed-loops for a basis of comparison (structured data, instead of tool calls, specifically). This all puzzles me, as tool calling is more or less across the board the same action requirements and these packages are open source. It seems unclear to me why Ollama would implement its own tool coercion logic, for instance, or why six months later the tools would still not work as well. This is an issue different than the broad LLMs capacity to return correct inputs, but these models, especially up near 32GB ought to be way, way more than sufficient, especially if the specific tool requirements available are part of the input context. What all this information in this paragraph is trying to communicate is a single-source of this industry-wide open source problem, and one that is major in its implications. And to follow-on the conversation to help as I may be able to. For all intents and purposes I have a similar setup as the ticket described above and the same problem to solve.

Thanks.

@davidbernat commented on GitHub (Jan 7, 2026): For those who are arriving here from search engines, this appears to be a major ongoing difficulty with OpenCode for on premise usage. I did a substantial review across the web of this issue several days ago, and found very many different branches of user problems, though all reduce to the notion that OpenCode does not have a local open source LLM solution. As summarized above, most users discuss the Ollama context length limitations (which has its own foibles, and does not solve all problems) and others discuss the use of the Qwen30b model as the only one which works. Generally, the discussion tends to circle around a drain right at the point someone says "the way Ollama uses tools is [negative word]". I cannot speak to knowing how the cool calling mechanisms are implemented (other than enforcing JSON responses, and some light parsing of non-JSON string responses to coerce them into JSON), and I have also seen problems with PydanticAI being too opaque to see in what ways they are editing their prompts and response closed-loops for a basis of comparison (structured data, instead of tool calls, specifically). This all puzzles me, as tool calling is more or less across the board the same action requirements and these packages are open source. It seems unclear to me why Ollama would implement its own tool coercion logic, for instance, or why six months later the tools would still not work as well. This is an issue different than the broad LLMs capacity to return correct inputs, but these models, especially up near 32GB ought to be way, way more than sufficient, especially if the specific tool requirements available are part of the input context. What all this information in this paragraph is trying to communicate is a single-source of this industry-wide open source problem, and one that is major in its implications. And to follow-on the conversation to help as I may be able to. For all intents and purposes I have a similar setup as the ticket described above and the same problem to solve. Thanks.
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@aaronnewsome commented on GitHub (Jan 7, 2026):

Give the qwen3-coder:30b a try, at least see if the Q8_K_XL quant is more reliable at tool calling compared to what's in the ollama library. try this one:

https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF

on the page, just click Use This Model button, then click Ollama which will give you a command line for ollama run or pull.

plus, Unsloth always does a great job of doing chat template fixes to make tool calls more reliable, a MUST with opencode.

@aaronnewsome commented on GitHub (Jan 7, 2026): Give the qwen3-coder:30b a try, at least see if the Q8_K_XL quant is more reliable at tool calling compared to what's in the ollama library. try this one: [https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF](https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF) on the page, just click `Use This Model` button, then click Ollama which will give you a command line for ollama run or pull. plus, Unsloth always does a great job of doing chat template fixes to make tool calls more reliable, a MUST with opencode.
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@davidbernat commented on GitHub (Jan 7, 2026):

I think a more thorough pulling back the veils on the various options and problems, to reach congruity, is necessary.

After all, once a dozen different tribes of people face a problem as large as "open source LLMs with open source OpenCode" marked as unusable, we have an industry mistake larger than one GitHub ticket and one recommendation of a model.

And I am not an expert in recent model performance problems with tools, but I am nearly absolutely certain this is not a model size problem. Providing function schema in context and coercion of outputs (similar to PydanticAI), with a few closed-loop retries, ought to be far more than enough with edge-device sized models. Here is a quick example with a relatively small model.

This demonstration shows to me that small models are more than sufficient for low quality effective tool use.
What we seem to have is a configuration and obfuscation problem. This is the reason for my posting to the ticket above.

Image
@davidbernat commented on GitHub (Jan 7, 2026): I think a more thorough pulling back the veils on the various options and problems, to reach congruity, is necessary. After all, once a dozen different tribes of people face a problem as large as "open source LLMs with open source OpenCode" marked as unusable, we have an industry mistake larger than one GitHub ticket and one recommendation of a model. And I am not an expert in recent model performance problems with tools, but I am nearly absolutely certain this is not a model size problem. Providing function schema in context and coercion of outputs (similar to PydanticAI), with a few closed-loop retries, ought to be far more than enough with edge-device sized models. Here is a quick example with a relatively small model. This demonstration shows to me that small models are more than sufficient for low quality effective tool use. What we seem to have is a configuration and obfuscation problem. This is the reason for my posting to the ticket above. <img width="860" height="813" alt="Image" src="https://github.com/user-attachments/assets/60d20561-16f7-4742-a8e9-80231373f9a1" />
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@aaronnewsome commented on GitHub (Jan 7, 2026):

I'm not sure I agree with any of this. Ollama GGUF models use the built-in chat template. Unless you create the GGUF yourself, it's using the chat template inside the GGUF. As I've learned, chat templates really matter when it comes to tool calling. Especially with the wildly different tool calling implementations across the models. Some use xml, some use json, some do weird things with how they do thinking between tool calls (no additional USER input), and newer models are doing even more weird things with interleaved thinking and probably a bunch of stuff I've never even heard of. Tool calling is messy and super-inconsistent. From my personal experience, older, smaller "tool calling capable" models are just not as reliable as newer, larger models - which have been better for me with consistent tool calling.

i do agree with one point though, tool calling should be more consistent and reliable for ANY tool calling capable model. unfortunately, I don't think that's a problem that can be solved by opencode OR ollama.

every model maker makes a decision to follow an established "standard" for tool calling, chat templates, etc or just invent new ones.

@aaronnewsome commented on GitHub (Jan 7, 2026): I'm not sure I agree with any of this. Ollama GGUF models use the built-in chat template. Unless you create the GGUF yourself, it's using the chat template inside the GGUF. As I've learned, chat templates really matter when it comes to tool calling. Especially with the wildly different tool calling implementations across the models. Some use xml, some use json, some do weird things with how they do thinking between tool calls (no additional USER input), and newer models are doing even more weird things with interleaved thinking and probably a bunch of stuff I've never even heard of. Tool calling is messy and super-inconsistent. From my personal experience, older, smaller "tool calling capable" models are just not as reliable as newer, larger models - which have been better for me with consistent tool calling. i do agree with one point though, tool calling should be more consistent and reliable for ANY tool calling capable model. unfortunately, I don't think that's a problem that can be solved by opencode OR ollama. every model maker makes a decision to follow an established "standard" for tool calling, chat templates, etc or just invent new ones.
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@Yupu-Wang commented on GitHub (Jan 8, 2026):

@davidbernat
Hi, I encountered the same issue when deploying Qwen2.5-Coder with Ollama.
The model can run normally and generate outputs, but actual file operations never happen.
I’ve already adjusted the context length and enabled tool calling.

What I’m wondering is: Is this problem caused by OpenCode, by Ollama, or by the combination of OpenCode + Ollama?

And if I switch to OpenHands + Ollama, will I run into the same issue?

@Yupu-Wang commented on GitHub (Jan 8, 2026): @davidbernat Hi, I encountered the same issue when deploying Qwen2.5-Coder with Ollama. The model can run normally and generate outputs, but actual file operations never happen. I’ve already adjusted the context length and enabled tool calling. What I’m wondering is: Is this problem caused by OpenCode, by Ollama, or by the combination of OpenCode + Ollama? And if I switch to OpenHands + Ollama, will I run into the same issue?
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@padsbanger commented on GitHub (Jan 8, 2026):

It Ollama consistence, based on what was posted by @davidbernat (If I understand correctly).

@padsbanger commented on GitHub (Jan 8, 2026): It Ollama consistence, based on what was posted by @davidbernat (If I understand correctly).
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@davidbernat commented on GitHub (Jan 8, 2026):

I agree with your statements @aaronnewsome and to some extend the trouble is that different networks expect different input and output string formats (XML or JSON, for instance). But this is not a problem to Ollama, tools with sufficient logic under the hood like Pydantic and PydanticAI exist, there is probably no larger use case in AI today (huge industry motivation), and the various requirements of each network should be well-documented (since they were data munged at training procedure). For the second part of this statement, absolutely no alternative to Ollama seems to exist in the open source community, which is verging on unwise if these problems with Ollama have persisted for more than a year into the emergence of open coding and tools. (The few non-Ollama solutions I see provided as an alternative all suspiciously accumulate around the same for-profit company, which makes me not trust this situational blocker as genuine even more.) I do not know enough about the internal workings of this tool problem (at the cross section of OpenCode, PydanticAI, and Ollama) right now to comment with authority any further, hence this call to action as a central meeting place for discussion on the OpenCode board (who should handle this, and its education, as the utility is so core to their value proposition). I am very excited about what their awesome team can do!

There are three clear components to the problem, and three clear sources of first-pass learning:

  • PydanticAI is a structured output munger which modifies prompts, intermediate chat cycles, and outputs to get a clean JSON
  • Ollama is a system-wide interface between networks' inputs and outputs, which should be enabled to make zero modification
  • OpenCode does additional transformations of prompt engineering and caching local context for its repeating thinking pattern

Concrete information is always welcomed. (cc: @YupuWang2001 @padsbanger)

@davidbernat commented on GitHub (Jan 8, 2026): I agree with your statements @aaronnewsome and to some extend the trouble is that different networks expect different input and output string formats (XML or JSON, for instance). But this is not a problem to Ollama, tools with sufficient logic under the hood like Pydantic and PydanticAI exist, there is probably no larger use case in AI today (huge industry motivation), and the various requirements of each network should be well-documented (since they were data munged at training procedure). For the second part of this statement, absolutely no alternative to Ollama seems to exist in the open source community, which is verging on unwise if these problems with Ollama have persisted for more than a year into the emergence of open coding and tools. (The few non-Ollama solutions I see provided as an alternative all suspiciously accumulate around the same for-profit company, which makes me not trust this situational blocker as genuine even more.) I do not know enough about the internal workings of this tool problem (at the cross section of OpenCode, PydanticAI, and Ollama) right now to comment with authority any further, hence this call to action as a central meeting place for discussion on the OpenCode board (who should handle this, and its education, as the utility is so core to their value proposition). I am very excited about what their awesome team can do! There are three clear components to the problem, and three clear sources of first-pass learning: - PydanticAI is a structured output munger which modifies prompts, intermediate chat cycles, and outputs to get a clean JSON - Ollama is a system-wide interface between networks' inputs and outputs, which should be enabled to make zero modification - OpenCode does additional transformations of prompt engineering and caching local context for its repeating thinking pattern Concrete information is always welcomed. (cc: @YupuWang2001 @padsbanger)
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@minger0 commented on GitHub (Jan 9, 2026):

See https://github.com/anomalyco/opencode/issues/5694#issuecomment-3667094022

@minger0 commented on GitHub (Jan 9, 2026): See https://github.com/anomalyco/opencode/issues/5694#issuecomment-3667094022
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@davidbernat commented on GitHub (Jan 9, 2026):

@minger0 Great. That comment is insightful; and people of this ticket should know about that ongoing discussion.
In short, as it suggests: I will check out ollama v2, and see whether this resolves the tool calling problems of ollama v1.

still, however: this all is far too opaque for users. There is no reason for tool calling success to happen layers deep.
We should know what changes can be made in their update, why those were made, and how those change procedure.
It is not clear to me that, if tool calling is this big a problem, we should all understand how to tinker in the guts of those three packages described above. For instance, why does PydanticAI not simply tell users (or me, a scientist), or log to console when requested, exactly how PydanticAI munges prompts and exactly what raw results the LLM returns? Even the requirement that PydanticAI users use cloud-based LogFire is highly suspect in conjunction with its other problems.

@davidbernat commented on GitHub (Jan 9, 2026): @minger0 Great. That comment is insightful; and people of this ticket should know about that ongoing discussion. In short, as it suggests: I will check out ollama v2, and see whether this resolves the tool calling problems of ollama v1. still, however: this all is far too opaque for users. There is no reason for tool calling success to happen layers deep. We should know what changes can be made in their update, why those were made, and how those change procedure. It is not clear to me that, if tool calling is this big a problem, we should all understand how to tinker in the guts of those three packages described above. For instance, why does PydanticAI not simply tell users (or me, a scientist), or log to console when requested, exactly how PydanticAI munges prompts and exactly what raw results the LLM returns? Even the requirement that PydanticAI users use cloud-based LogFire is highly suspect in conjunction with its other problems.
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@ElArkk commented on GitHub (Jan 9, 2026):

I was struggling with my own Ollama deployment on Modal for a while (turns out my SSE proxy was only emitting \n instead of \n\n between events, breaking the parser).

Now both the ollama Qwen3 30B and the Huggingface A3B version are working well. Llama 3.2 3B does not work well as it struggles with tool calls.

Here's my working OpenCode config:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "modal-ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Modal Ollama",
      "options": {
        "baseURL": "https://<my_remote_ollama_endpoint>/v1",
        "num_ctx": "65536"
      },
      "models": {
        "llama3.2:3b": {
          "name": "Llama 3.2 3B",
          "limit": {
            "context": 128000,
            "output": 64000
          }
        },
        "qwen3-coder:30b": {
          "name": "Qwen 3 Coder 30B",
          "tool_call": true,
          "reasoning": true,
          "limit": {
            "context": 256000,
            "output": 64000
            }
          },
        "hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q8_K_XL": {
          "name": "Qwen 3 Coder 30B A3B Q8 HF",
          "tool_call": true,
          "reasoning": true,
          "limit": {
            "context": 256000,
            "output": 64000
          },
        }
      }
    }
@ElArkk commented on GitHub (Jan 9, 2026): I was struggling with my own Ollama deployment on Modal for a while (turns out my SSE proxy was only emitting `\n` instead of `\n\n` between events, breaking the parser). Now both the ollama Qwen3 30B and the Huggingface A3B version are working well. Llama 3.2 3B does not work well as it struggles with tool calls. Here's my working OpenCode config: ``` { "$schema": "https://opencode.ai/config.json", "provider": { "modal-ollama": { "npm": "@ai-sdk/openai-compatible", "name": "Modal Ollama", "options": { "baseURL": "https://<my_remote_ollama_endpoint>/v1", "num_ctx": "65536" }, "models": { "llama3.2:3b": { "name": "Llama 3.2 3B", "limit": { "context": 128000, "output": 64000 } }, "qwen3-coder:30b": { "name": "Qwen 3 Coder 30B", "tool_call": true, "reasoning": true, "limit": { "context": 256000, "output": 64000 } }, "hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q8_K_XL": { "name": "Qwen 3 Coder 30B A3B Q8 HF", "tool_call": true, "reasoning": true, "limit": { "context": 256000, "output": 64000 }, } } } ```
yindo commented 2026-02-16 17:43:22 -05:00
Author
Owner
Copy Link
Copy Source

@davidbernat commented on GitHub (Feb 12, 2026):

@ElArkk And you are absolutely positive your Ollama is configured properly to accept these per-call context configurations?

Would you mind sharing your Ollama ollama serve bootup configuration?

          "tool_call": true,
          "reasoning": true,
          "limit": {
            "context": 256000,
            "output": 64000
          },
@davidbernat commented on GitHub (Feb 12, 2026): @ElArkk And you are absolutely positive your Ollama is configured properly to accept these per-call context configurations? Would you mind sharing your Ollama ollama serve bootup configuration? ``` "tool_call": true, "reasoning": true, "limit": { "context": 256000, "output": 64000 }, ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#4304
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?