anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

When use gpt-oss-120B by vLLM locally, opencode doesn't call the tools #4349

New Issue
Open
opened 2026-02-16 17:43:31 -05:00 by yindo · 18 comments
yindo commented 2026-02-16 17:43:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @inv1s10n on GitHub (Jan 7, 2026).

Originally assigned to: @rekram1-node on GitHub.

Description

When I try use local model (such as gpt-oss-120b and qwen3-32b), there is only content of thinking in response, no tools calling(even if model thinks it should call tools from thinking content) and no other response. But sometimes it be normally(very rare). This is very strange.

Image Image

Plugins

No response

OpenCode version

1.1.4

Steps to reproduce

opencode.json:

Image

vllm serve:

Image

Screenshot and/or share link

No response

Operating System

No response

Terminal

No response

Originally created by @inv1s10n on GitHub (Jan 7, 2026). Originally assigned to: @rekram1-node on GitHub. ### Description When I try use local model (such as gpt-oss-120b and qwen3-32b), there is only content of thinking in response, no tools calling(even if model thinks it should call tools from thinking content) and no other response. But sometimes it be normally(very rare). This is very strange. <img width="880" height="160" alt="Image" src="https://github.com/user-attachments/assets/81f99273-98af-4c9d-8d30-f59ab843edac" /> <img width="897" height="359" alt="Image" src="https://github.com/user-attachments/assets/d71cda97-8a63-48fc-b581-ad8b366a1b36" /> ### Plugins _No response_ ### OpenCode version 1.1.4 ### Steps to reproduce opencode.json: <img width="416" height="527" alt="Image" src="https://github.com/user-attachments/assets/bcbc131a-b588-4c43-b20e-b7dffad06378" /> vllm serve: <img width="1520" height="37" alt="Image" src="https://github.com/user-attachments/assets/941a3604-f7bd-4478-9e78-5f5075b3f0ec" /> ### Screenshot and/or share link _No response_ ### Operating System _No response_ ### Terminal _No response_
yindo added the bug label 2026-02-16 17:43:31 -05:00
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Jan 7, 2026):

This issue might be a duplicate of existing issues. Please check:

  • #7083: Using local Ollama models doesnt return any results - same symptoms with local models returning only JSON tool calls instead of actual responses
  • #6649: Weird error with toolcalling on local model - similar tool calling issues with local models via Llama.cpp
  • #5694: Local Ollama models are not agentic - local models not able to make tool calls properly
  • #6223: Model not returning the answers - Qwen2.5-coder with Ollama not responding, only showing JSON tool calls
  • #4428: Why is opencode not working with local llms via Ollama - widespread issue with local LLMs and tool calling
  • #4255: OpenCode v1.0.25 Hangs Indefinitely with LM Studio + Qwen Models - detailed analysis of empty tool_calls array handling issue with local models
  • #234: Tool Calling Issues with Open Source Models - discusses case sensitivity and tool calling inconsistencies with open source models

All of these report similar symptoms: local models (gpt-oss, qwen variants) via vLLM/Ollama either failing to call tools, only returning thinking content, or hanging indefinitely.

Feel free to ignore if none of these address your specific case.

@github-actions[bot] commented on GitHub (Jan 7, 2026): This issue might be a duplicate of existing issues. Please check: - #7083: Using local Ollama models doesnt return any results - same symptoms with local models returning only JSON tool calls instead of actual responses - #6649: Weird error with toolcalling on local model - similar tool calling issues with local models via Llama.cpp - #5694: Local Ollama models are not agentic - local models not able to make tool calls properly - #6223: Model not returning the answers - Qwen2.5-coder with Ollama not responding, only showing JSON tool calls - #4428: Why is opencode not working with local llms via Ollama - widespread issue with local LLMs and tool calling - #4255: OpenCode v1.0.25 Hangs Indefinitely with LM Studio + Qwen Models - detailed analysis of empty tool_calls array handling issue with local models - #234: Tool Calling Issues with Open Source Models - discusses case sensitivity and tool calling inconsistencies with open source models All of these report similar symptoms: local models (gpt-oss, qwen variants) via vLLM/Ollama either failing to call tools, only returning thinking content, or hanging indefinitely. Feel free to ignore if none of these address your specific case.
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@inv1s10n commented on GitHub (Jan 7, 2026):

I found there is no introduction about vLLM in official docs. Whether vLLM is not supported well ?

@inv1s10n commented on GitHub (Jan 7, 2026): I found there is no introduction about vLLM in official docs. Whether vLLM is not supported well ?
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@padsbanger commented on GitHub (Jan 7, 2026):

I have running similar issue, as described here: https://github.com/anomalyco/opencode/issues/7083 and it seems that locall llms from ollama are not capable of code assistant tasks in Opencode.

What hardware are you using, btw ?

@padsbanger commented on GitHub (Jan 7, 2026): I have running similar issue, as described here: https://github.com/anomalyco/opencode/issues/7083 and it seems that locall llms from ollama are not capable of code assistant tasks in Opencode. What hardware are you using, btw ?
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@inv1s10n commented on GitHub (Jan 7, 2026):

a A100, linux.
I'm not sure whether this is a problem with the model or the startup parameters.
Or it need other plugins ?

@inv1s10n commented on GitHub (Jan 7, 2026): a A100, linux. I'm not sure whether this is a problem with the model or the startup parameters. Or it need other plugins ?
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@nilomind-code commented on GitHub (Jan 8, 2026):

I am using 'GPT-OSS' on Ollama on local machine. I have ollama setup on Ubuntu 22 LTS with a Nvidia 3090.
I have it already setup with 'OpenWebUI'. I started using 'opencode' yesterday. And 'opencode' is realy a big step forward in AI in my perspective. I was hooked with the agentic work that Network Chuck showed on his channel: https://www.youtube.com/watch?v=MsQACpcuTkU
It is preetty fantastic if this could work. I was realy fascinated, when i have seen on the above video, how better an experience can be achieved. Especialy if i can put agentic work to work on my '.md' files that i have saved on 'NextCloud'.

And i would realy like if this would also work with local ollama models like gpt-oss. ollama shows me that tools are also supported with 'Mistral'. Which i also could not get to work.

Are there any more detailed tutorials how to get this to work?
Is this even an 'opencode' issue or is this a model issue?
I would be very happy if someone can point me in a direction how i can get this to work. A lot of hat-downs to the creators of 'opencode'. this is realy fantastic.

@nilomind-code commented on GitHub (Jan 8, 2026): I am using 'GPT-OSS' on Ollama on local machine. I have ollama setup on Ubuntu 22 LTS with a Nvidia 3090. I have it already setup with 'OpenWebUI'. I started using 'opencode' yesterday. And 'opencode' is realy a big step forward in AI in my perspective. I was hooked with the agentic work that Network Chuck showed on his channel: https://www.youtube.com/watch?v=MsQACpcuTkU It is preetty fantastic if this could work. I was realy fascinated, when i have seen on the above video, how better an experience can be achieved. Especialy if i can put agentic work to work on my '.md' files that i have saved on 'NextCloud'. And i would realy like if this would also work with local ollama models like gpt-oss. ollama shows me that tools are also supported with 'Mistral'. Which i also could not get to work. Are there any more detailed tutorials how to get this to work? Is this even an 'opencode' issue or is this a model issue? I would be very happy if someone can point me in a direction how i can get this to work. A lot of hat-downs to the creators of 'opencode'. this is realy fantastic.
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Jan 8, 2026):

if u are using vllm i thought there was a long open issue about tool calling not working through vllm for like months w/ gpt oss

@rekram1-node commented on GitHub (Jan 8, 2026): if u are using vllm i thought there was a long open issue about tool calling not working through vllm for like months w/ gpt oss
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@inv1s10n commented on GitHub (Jan 8, 2026):

This is a bad news. Is there a link to this issue? And is there a plan to address this issue?

@inv1s10n commented on GitHub (Jan 8, 2026): This is a bad news. Is there a link to this issue? And is there a plan to address this issue?
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@famitzsy8 commented on GitHub (Jan 8, 2026):

Had the same issue. Was a problem of the context window. Ollama uses a default context window of 2048, which isn't enough to put all tool descriptions and examples schemas in. Create a Modelfile for gpt-oss:120b like this:

FROM gpt-oss:120b

PARAMETER num_ctx 16384
PARAMETER temperature 1

then run:

ollama create gpt-oss:120b-16384 -f Modelfile

This should ensure that the model has sufficient context to consider all tool definitions. If this doesn't work, increase to 32k

@famitzsy8 commented on GitHub (Jan 8, 2026): Had the same issue. Was a problem of the context window. Ollama uses a default context window of 2048, which isn't enough to put all tool descriptions and examples schemas in. Create a `Modelfile` for gpt-oss:120b like this: ```txt FROM gpt-oss:120b PARAMETER num_ctx 16384 PARAMETER temperature 1 ``` then run: ```bash ollama create gpt-oss:120b-16384 -f Modelfile ``` This should ensure that the model has sufficient context to consider all tool definitions. If this doesn't work, increase to 32k
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@inv1s10n commented on GitHub (Jan 8, 2026):

Thank you, but I am not using ollama, and when I started the model with vLLM, I set the max-model-len to 40960

@inv1s10n commented on GitHub (Jan 8, 2026): Thank you, but I am not using ollama, and when I started the model with vLLM, I set the max-model-len to 40960
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@Charmnut commented on GitHub (Jan 8, 2026):

Hi, @rekram1-node. Can using sglang solve this problem？

@Charmnut commented on GitHub (Jan 8, 2026): Hi, @rekram1-node. Can using sglang solve this problem？
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@TrashcanOfLove commented on GitHub (Jan 11, 2026):

I am also having this issue but have 128k context on a DGX spark,

my vLLM

services:
  vllm:
    image: nvcr.io/nvidia/vllm:25.11-py3
    restart: always
    container_name: vllm-gpt-oss-120b
    ports:
      - "8000:8000"
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
    environment:
      VLLM_USE_NATIVE_FP4: "1"
      VLLM_FP4_IMPL: "nvfp4"
    ipc: host
    ulimits:
      memlock: -1
      stack: 67108864
    volumes:
      - ./model-cache:/root/.cache/huggingface
      - ./vllm-cache:/root/.cache/vllm
    command: >
      vllm serve openai/gpt-oss-120b
        --host 0.0.0.0
        --port 8000
        --quantization mxfp4
        --trust-remote-code
        --gpu-memory-utilization 0.90
        --max-model-len 131072
        --enable-prefix-caching
        --enable-auto-tool-choice
        --tool-call-parser openai
@TrashcanOfLove commented on GitHub (Jan 11, 2026): I am also having this issue but have 128k context on a DGX spark, my vLLM ``` services: vllm: image: nvcr.io/nvidia/vllm:25.11-py3 restart: always container_name: vllm-gpt-oss-120b ports: - "8000:8000" deploy: resources: reservations: devices: - capabilities: [gpu] environment: VLLM_USE_NATIVE_FP4: "1" VLLM_FP4_IMPL: "nvfp4" ipc: host ulimits: memlock: -1 stack: 67108864 volumes: - ./model-cache:/root/.cache/huggingface - ./vllm-cache:/root/.cache/vllm command: > vllm serve openai/gpt-oss-120b --host 0.0.0.0 --port 8000 --quantization mxfp4 --trust-remote-code --gpu-memory-utilization 0.90 --max-model-len 131072 --enable-prefix-caching --enable-auto-tool-choice --tool-call-parser openai ```
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@inv1s10n commented on GitHub (Jan 12, 2026):

I tried the following methods, and they seem to be effective.

Image
@inv1s10n commented on GitHub (Jan 12, 2026): I tried the following methods, and they seem to be effective. <img width="405" height="524" alt="Image" src="https://github.com/user-attachments/assets/ae2e41ca-2e0d-4a57-a7a7-f2325c2db690" />
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@logan-jun commented on GitHub (Jan 12, 2026):

@inv1s10n is it working well? What vllm version are u using??

@logan-jun commented on GitHub (Jan 12, 2026): @inv1s10n is it working well? What vllm version are u using??
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@inv1s10n commented on GitHub (Jan 13, 2026):

I recently have no issues with its use. My vllm version is 0.14.0rc1.

@inv1s10n commented on GitHub (Jan 13, 2026): I recently have no issues with its use. My vllm version is 0.14.0rc1.
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@marc2332 commented on GitHub (Jan 15, 2026):

Had the same issue. Was a problem of the context window. Ollama uses a default context window of 2048, which isn't enough to put all tool descriptions and examples schemas in. Create a Modelfile for gpt-oss:120b like this:

FROM gpt-oss:120b

PARAMETER num_ctx 16384
PARAMETER temperature 1

then run:

ollama create gpt-oss:120b-16384 -f Modelfile

This should ensure that the model has sufficient context to consider all tool definitions. If this doesn't work, increase to 32k

Thank you!!!!!!! This worked for me (I am using ollama)

@marc2332 commented on GitHub (Jan 15, 2026): > Had the same issue. Was a problem of the context window. Ollama uses a default context window of 2048, which isn't enough to put all tool descriptions and examples schemas in. Create a `Modelfile` for gpt-oss:120b like this: > > FROM gpt-oss:120b > > PARAMETER num_ctx 16384 > PARAMETER temperature 1 > > then run: > > ollama create gpt-oss:120b-16384 -f Modelfile > > This should ensure that the model has sufficient context to consider all tool definitions. If this doesn't work, increase to 32k Thank you!!!!!!! This worked for me (I am using ollama)
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Jan 15, 2026):

I think Ideal solution here is doing a pass on ollama so that we automatically send the num_ctx param so u dont ahve to, I need to look into it, but last time you werent able to do that via openai compat endpoints... If this has changed it should be an easy fix

@rekram1-node commented on GitHub (Jan 15, 2026): I think Ideal solution here is doing a pass on ollama so that we automatically send the num_ctx param so u dont ahve to, I need to look into it, but last time you werent able to do that via openai compat endpoints... If this has changed it should be an easy fix
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@FreshLucas-git commented on GitHub (Jan 18, 2026):

@inv1s10n

I recently have no issues with its use. My vllm version is 0.14.0rc1.

I am using vLLM 0.14.0rc1, and it works as expected. Thank you.

@FreshLucas-git commented on GitHub (Jan 18, 2026): @inv1s10n > I recently have no issues with its use. My vllm version is 0.14.0rc1. I am using vLLM 0.14.0rc1, and it works as expected. Thank you.
yindo commented 2026-02-16 17:43:32 -05:00
Author
Owner
Copy Link
Copy Source

@lee-b commented on GitHub (Jan 26, 2026):

Had the same issue. Was a problem of the context window. Ollama uses a default context window of 2048, which isn't enough to put all tool descriptions and examples schemas in. Create a Modelfile for gpt-oss:120b like this:
FROM gpt-oss:120b
PARAMETER num_ctx 16384
PARAMETER temperature 1
then run:
ollama create gpt-oss:120b-16384 -f Modelfile
This should ensure that the model has sufficient context to consider all tool definitions. If this doesn't work, increase to 32k

Thank you!!!!!!! This worked for me (I am using ollama)

Yep, I thought I had the same "issue" (but with llama.cpp). It turns out that I'd left out a zero in my compaction limit, so the "responses" were actually constant compaction summaries.

@lee-b commented on GitHub (Jan 26, 2026): > > Had the same issue. Was a problem of the context window. Ollama uses a default context window of 2048, which isn't enough to put all tool descriptions and examples schemas in. Create a `Modelfile` for gpt-oss:120b like this: > > FROM gpt-oss:120b > > PARAMETER num_ctx 16384 > > PARAMETER temperature 1 > > then run: > > ollama create gpt-oss:120b-16384 -f Modelfile > > This should ensure that the model has sufficient context to consider all tool definitions. If this doesn't work, increase to 32k > > Thank you!!!!!!! This worked for me (I am using ollama) Yep, I *thought* I had the same "issue" (but with llama.cpp). It turns out that I'd left out a zero in my compaction limit, so the "responses" were actually constant compaction summaries.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#4349
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?