weird behavior on devstral-small #279

Closed
opened 2026-02-16 17:26:02 -05:00 by yindo · 4 comments
Owner

Originally created by @bernardolsp on GitHub (Jun 25, 2025).

Originally assigned to: @thdxr on GitHub.

hey! opencode is awesome and works very much well with sonnet.

I have been trying devstral on roocode (via openrouter) and that seems to behave pretty well - i.e, performing agentic (and function calling) very, very well:

Image

With the same question on Roo Code and in OpenCode, I get very different results:

Image (notice it keeps going and vibe coding its way to the task) Image

On the other hand, the same prompt to OpenCode generated this https://opencode.ai/s/J1nnnktj - pretty much not calling the functions.

I know most of these tools are more optimized for Claude - though, if this makes sense, I would think that opencode is specially interesting for allowing the usage of other models here to perform in similar scenarios.
am I just bad at prompting, or would this technically require more prompt finagling?

For what is worth, this is my global config:

➜  test-roocode-devstral cat ~/.config/opencode/config.json                                                                               (⎈|omni-Production:default)
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "openrouter": {
      "npm": "@openrouter/ai-sdk-provider",
      "name": "OpenRouter",
      "options": {},
      "models": {
        "mistralai/devstral-small": {
          "name": "Devstral Small",
	  "limit": { "context": 131072, "output": 1000000 }
        },
	"deepseek/deepseek-chat-v3-0324": {
	  "name": "DeepSeek V3"
	},
	"openai/o3-mini-high": {}
      }
    }
  }
}

I have tried messing with the context/output limits, but seemingly that went nowhere.

thank you!

Originally created by @bernardolsp on GitHub (Jun 25, 2025). Originally assigned to: @thdxr on GitHub. hey! opencode is awesome and works very much well with sonnet. I have been trying devstral on roocode (via openrouter) and that seems to behave pretty well - i.e, performing agentic (and function calling) very, very well: <img width="388" alt="Image" src="https://github.com/user-attachments/assets/eccfbcc5-4b5b-48df-a3e1-0096526fa0a1" /> With the same question on Roo Code and in OpenCode, I get very different results: <img width="804" alt="Image" src="https://github.com/user-attachments/assets/2428e2fb-b93f-44f7-b5a1-920cd7cfefa5" /> (notice it keeps going and vibe coding its way to the task) <img width="804" alt="Image" src="https://github.com/user-attachments/assets/dad1e6e6-57bf-401f-906e-4e26bf4d1bd9" /> On the other hand, the same prompt to OpenCode generated this https://opencode.ai/s/J1nnnktj - pretty much not calling the functions. I know most of these tools are more optimized for Claude - though, if this makes sense, I would think that opencode is specially interesting for allowing the usage of other models here to perform in similar scenarios. am I just bad at prompting, or would this technically require more prompt finagling? For what is worth, this is my global config: ``` ➜ test-roocode-devstral cat ~/.config/opencode/config.json (⎈|omni-Production:default) { "$schema": "https://opencode.ai/config.json", "provider": { "openrouter": { "npm": "@openrouter/ai-sdk-provider", "name": "OpenRouter", "options": {}, "models": { "mistralai/devstral-small": { "name": "Devstral Small", "limit": { "context": 131072, "output": 1000000 } }, "deepseek/deepseek-chat-v3-0324": { "name": "DeepSeek V3" }, "openai/o3-mini-high": {} } } } } ``` I have tried messing with the context/output limits, but seemingly that went nowhere. thank you!
yindo closed this issue 2026-02-16 17:26:02 -05:00
Author
Owner

@bernardolsp commented on GitHub (Jun 25, 2025):

I have removed the limits (in the json config) to avoid any issue - this is what I got https://opencode.ai/s/hBysNy7f

this is mac os, iterm2, latest version of opencode (that was released yesterday)

(fwiw, I understand that roocode and opencode do several different things - effectively grepping vs using LSPs, etc. - I am not comparing the final outcome or characteristics of the two different tools - rather trying to understand why it completely failed to actually call functions on opencode)

@bernardolsp commented on GitHub (Jun 25, 2025): I have removed the limits (in the json config) to avoid any issue - this is what I got https://opencode.ai/s/hBysNy7f this is mac os, iterm2, latest version of opencode (that was released yesterday) (fwiw, I understand that roocode and opencode do several different things - effectively grepping vs using LSPs, etc. - I am not comparing the final outcome or characteristics of the two different tools - rather trying to understand why it completely failed to actually call functions on opencode)
Author
Owner

@thdxr commented on GitHub (Jun 25, 2025):

can you ask it "what tools do you have?"

@thdxr commented on GitHub (Jun 25, 2025): can you ask it "what tools do you have?"
Author
Owner

@bernardolsp commented on GitHub (Jun 25, 2025):

hey, there you go:

Image
@bernardolsp commented on GitHub (Jun 25, 2025): hey, there you go: <img width="822" alt="Image" src="https://github.com/user-attachments/assets/64aa8f30-d2c6-44ea-af02-58cfca90a69a" />
Author
Owner

@bernardolsp commented on GitHub (Jun 25, 2025):

this might be relevant:

https://opencode.ai/s/ykg1N5tS

which triggered an error (btw, copying stuff in the TUI would be awesome):

Image
@bernardolsp commented on GitHub (Jun 25, 2025): this might be relevant: https://opencode.ai/s/ykg1N5tS which triggered an error (btw, copying stuff in the TUI would be awesome): <img width="822" alt="Image" src="https://github.com/user-attachments/assets/02fa21f8-a2d9-4de3-ae04-67bbb423ffde" />
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#279