anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

BUG: TemplateError #1507

New Issue
Closed
opened 2026-02-16 17:31:18 -05:00 by yindo · 18 comments
yindo commented 2026-02-16 17:31:18 -05:00
Owner
Copy Link
Copy Source

Originally created by @oatmealm on GitHub (Aug 27, 2025).

Originally assigned to: @thdxr on GitHub.

I keep running into these type of errors with most models I've tried (served by mlx-omni-server):

TemplateError: Conversation roles must alternate user/assistant/user/assistant/…
Thrown in transformers/utils/chat_template_utils.py:446.

mlx-omni-server` version: current HEAD, but this happening only in OpenCode not with other clients.

According to Kimi2, the issue is when a client sends two consecutive messages with the same role (e.g. two system messages), the server never reaches model memory allocation; it aborts during the Jinja2 chat-template pre-processing step with
RuntimeError: Stream generation failed: Conversation roles must alternate user/assistant/user/assistant/…

Partial log attached.

extract.log

Originally created by @oatmealm on GitHub (Aug 27, 2025). Originally assigned to: @thdxr on GitHub. I keep running into these type of errors with most models I've tried (served by mlx-omni-server): ``` TemplateError: Conversation roles must alternate user/assistant/user/assistant/… Thrown in transformers/utils/chat_template_utils.py:446. ``` mlx-omni-server` version: current HEAD, but this happening only in OpenCode not with other clients. According to Kimi2, the issue is when a client sends two consecutive messages with the same role (e.g. two system messages), the server never reaches model memory allocation; it aborts during the Jinja2 chat-template pre-processing step with RuntimeError: Stream generation failed: Conversation roles must alternate user/assistant/user/assistant/… Partial log attached. [extract.log](https://github.com/user-attachments/files/22002553/extract.log)
yindo closed this issue 2026-02-16 17:31:18 -05:00
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Aug 27, 2025):

This issue might be a duplicate of existing issues. Please check:

  • #1301: Similar issue with duplicate system messages being sent, which can cause conversation role alternation errors with certain providers

Feel free to ignore if none of these address your specific case.

@github-actions[bot] commented on GitHub (Aug 27, 2025): This issue might be a duplicate of existing issues. Please check: - #1301: Similar issue with duplicate system messages being sent, which can cause conversation role alternation errors with certain providers Feel free to ignore if none of these address your specific case.
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@oatmealm commented on GitHub (Aug 27, 2025):

Ooops. Seems like it's the same issue. Though if there was a fix issued (in July?) it's still happening.

@oatmealm commented on GitHub (Aug 27, 2025): Ooops. Seems like it's the same issue. Though if there was a fix issued (in July?) it's still happening.
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@OpeOginni commented on GitHub (Aug 27, 2025):

Hello @oatmealm , could can you show how you configured this local model to run on opencode?

@OpeOginni commented on GitHub (Aug 27, 2025): Hello @oatmealm , could can you show how you configured this local model to run on opencode?
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@oatmealm commented on GitHub (Aug 27, 2025):

Hi there, right now I have this in opendoce.json (assuming that's what you need to see?):

opencode.json

@oatmealm commented on GitHub (Aug 27, 2025): Hi there, right now I have this in opendoce.json (assuming that's what you need to see?): [opencode.json](https://github.com/user-attachments/files/22003794/opencode.json)
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@OpeOginni commented on GitHub (Aug 27, 2025):

Thanks, from what I can see your models or way you run them don't support multiple leading system messages, and I think the guys at Opencode would rather send 2 system messages as default to enable caching on systems like Anthropic. Still the AI SDK apparently has a way to check if a model allows this and if not it merges multiple system messages into 1 to prevent this error, but since you are running a local with a custom provider, it might think your models accept multiple system prompts when it doesn't.

This is what I can see for now, I am still doing some looking, best case right now would be to find the proper provider for this your way of running the models. Not sure why the @ai-sdk/openai-compatible one isn't merging the system prompts.

@OpeOginni commented on GitHub (Aug 27, 2025): Thanks, from what I can see your models or way you run them don't support multiple leading system messages, and I think the guys at Opencode would rather send 2 system messages as default to enable caching on systems like Anthropic. Still the AI SDK apparently has a way to check if a model allows this and if not it merges multiple system messages into 1 to prevent this error, but since you are running a local with a custom provider, it might think your models accept multiple system prompts when it doesn't. This is what I can see for now, I am still doing some looking, best case right now would be to find the proper provider for this your way of running the models. Not sure why the `@ai-sdk/openai-compatible` one isn't merging the system prompts.
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@OpeOginni commented on GitHub (Aug 27, 2025):

A long shot, but could you try this in your opencode config and let me know how it goes?

    "mlx-server": {
      "npm": "@ai-sdk/anthropic",
      "name": "MLX Omni Server (LAN)",
      "options": {
        "baseURL": "http://mbp.lan:10240/anthropic",
        "headers": {
          "Content-Type": "application/json"
        }
      },
@OpeOginni commented on GitHub (Aug 27, 2025): A long shot, but could you try this in your opencode config and let me know how it goes? ```json "mlx-server": { "npm": "@ai-sdk/anthropic", "name": "MLX Omni Server (LAN)", "options": { "baseURL": "http://mbp.lan:10240/anthropic", "headers": { "Content-Type": "application/json" } }, ```
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@oatmealm commented on GitHub (Aug 27, 2025):

Thanks. I was going to test that too :) Seems to work, though lots of other problems, like performance and irrelevant replys

@oatmealm commented on GitHub (Aug 27, 2025): Thanks. I was going to test that too :) Seems to work, though lots of other problems, like performance and irrelevant replys
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Aug 27, 2025):

did @OpeOginni help you get this solved or do you need further assistance?

@rekram1-node commented on GitHub (Aug 27, 2025): did @OpeOginni help you get this solved or do you need further assistance?
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@OpeOginni commented on GitHub (Aug 27, 2025):

@oatmealm what I think is the best solution would be an AI SDK custom provider specifically for this way of running models. Maybe you can push the community to make one. But idk if it's the only solution anyone else with better clue can chip in.

@OpeOginni commented on GitHub (Aug 27, 2025): @oatmealm what I think is the best solution would be an AI SDK custom provider specifically for this way of running models. Maybe you can push the community to make one. But idk if it's the only solution anyone else with better clue can chip in.
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Aug 27, 2025):

Ultimately this seems like a provider / configuration issue rather than opencode, I can use kimi k2 through a variety of providers just fine

@rekram1-node commented on GitHub (Aug 27, 2025): Ultimately this seems like a provider / configuration issue rather than opencode, I can use kimi k2 through a variety of providers just fine
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@oatmealm commented on GitHub (Aug 27, 2025):

I will continue investigating with more models. I prefer local, though I'm wondering how realistic it is given context size requirements alone...?

@oatmealm commented on GitHub (Aug 27, 2025): I will continue investigating with more models. I prefer local, though I'm wondering how realistic it is given context size requirements alone...?
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Aug 27, 2025):

if you want to use local models I recommend LLM Studio they seem to do best job in that department I always have people filing issues about ollama and others

@rekram1-node commented on GitHub (Aug 27, 2025): if you want to use local models I recommend LLM Studio they seem to do best job in that department I always have people filing issues about ollama and others
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@oatmealm commented on GitHub (Aug 27, 2025):

Maybe I should. They do have full support for mlx now afaik but I thought the overhead of the gui or the single model standalone server (afaik it can't switch and load models after it's started) was too restrictive (i.e. I would need something else serving embeddings at least on the same machine.)

@oatmealm commented on GitHub (Aug 27, 2025): Maybe I should. They do have full support for mlx now afaik but I thought the overhead of the gui or the single model standalone server (afaik it can't switch and load models after it's started) was too restrictive (i.e. I would need something else serving embeddings at least on the same machine.)
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@thdxr commented on GitHub (Aug 27, 2025):

when you say not other clients what clients is it working in?

@thdxr commented on GitHub (Aug 27, 2025): when you say not other clients what clients is it working in?
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@oatmealm commented on GitHub (Aug 29, 2025):

I was using emacs tools such as efrit (anthropic api as well) and gptel. But testing more trivial workflows with openwebui, I'm having too many problems with both mlx-omni-server and mlx-gui, i.e. embedding sometimes fails, models return 500 and even reboots (possibly realted to resource utiliaztion).

So I'm back on my original ollama setup, but of course the most immediate issue to tool calling support. I have some agents defined that don't need tools at all. I tried this:

        "deepseek-r1:8b": {
          "name": "DeepSeek R1 8B",
          "tools": false,
          "reasoning": true,
          "temperature": true,
          "options": {
            "temperature": 0.3
          }
        },

for this particular model, which afaik does not support tools at all, but opencode still complains:

AI_APICallError: registry.ollama.ai/library/deepseek-
r1:8b does not support tools

Same with

AI_APICallError: registry.ollama.ai/library/phi4-mini-
reasoning:latest does not support tools
@oatmealm commented on GitHub (Aug 29, 2025): I was using emacs tools such as efrit (anthropic api as well) and gptel. But testing more trivial workflows with openwebui, I'm having too many problems with both mlx-omni-server and mlx-gui, i.e. embedding sometimes fails, models return 500 and even reboots (possibly realted to resource utiliaztion). So I'm back on my original ollama setup, but of course the most immediate issue to tool calling support. I have some agents defined that don't need tools at all. I tried this: ``` "deepseek-r1:8b": { "name": "DeepSeek R1 8B", "tools": false, "reasoning": true, "temperature": true, "options": { "temperature": 0.3 } }, ``` for this particular model, which afaik does not support tools at all, but opencode still complains: ``` AI_APICallError: registry.ollama.ai/library/deepseek- r1:8b does not support tools ``` Same with ``` AI_APICallError: registry.ollama.ai/library/phi4-mini- reasoning:latest does not support tools ```
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@oatmealm commented on GitHub (Aug 29, 2025):

Sorry, I see it's "tool_call": false, which works fine.
Question is if there's a way to use option to enable tools on Ollama models which support tool call by might need additional or specific options passed in?

@oatmealm commented on GitHub (Aug 29, 2025): Sorry, I see it's ```"tool_call": false```, which works fine. Question is if there's a way to use option to enable tools on Ollama models which support tool call by might need additional or specific options passed in?
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Aug 29, 2025):

Can you elaborate? I don't think I understand what you are asking there

@rekram1-node commented on GitHub (Aug 29, 2025): Can you elaborate? I don't think I understand what you are asking there
yindo commented 2026-02-16 17:31:19 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Dec 27, 2025):

[automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!

@rekram1-node commented on GitHub (Dec 27, 2025): [automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#1507
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?