anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 10:15:31 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

What is the best Modelfile Template to use on local ollama models #7732

New Issue
Open
opened 2026-02-16 18:08:04 -05:00 by yindo · 3 comments
yindo commented 2026-02-16 18:08:04 -05:00
Owner
Copy Link
Copy Source

Originally created by @glaudiston on GitHub (Jan 27, 2026).

Originally assigned to: @thdxr on GitHub.

Question

I found that local ollama models fails to do tool calls by default.
For instance the glm-4.7-flash:latest fail the tool call on my local ollama provider.
I was able to make it work by repackaging it using this steps:

  1. generating a modelfile with
ollama show glm-4.7-flash:latest --modelfile > Modelfile.glm-4.7-flash:latest.md
  1. update the modelfile to something like:
# Modelfile modified from a generated by "ollama show"
FROM glm-4.7-flash:latest

TEMPLATE """
{{- $lastUserIdx := -1 -}}
{{- range $idx, $msg := .Messages -}}
{{- if eq $msg.Role "user" }}{{ $lastUserIdx = $idx }}{{ end -}}
{{- end }}
{{- if or .System .Tools }}<|im_start|>system
{{ if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}

# Tools

You may call one or more functions to assist with the user query.

You are provided with function signatures within <tools></tools> XML tags:
<tools>
{{- range .Tools }}
{"type": "function", "function": {{ .Function }}}
{{- end }}
</tools>

For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>
{{- end -}}
<|im_end|>
{{ end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
{{- if eq .Role "user" }}<|im_start|>user
{{ .Content }}
{{- if and $.IsThinkSet (eq $i $lastUserIdx) }}
   {{- if $.Think -}}
      {{- " "}}/think
   {{- else -}}
      {{- " "}}/no_think
   {{- end -}}
{{- end }}<|im_end|>
{{ else if eq .Role "assistant" }}<|im_start|>assistant
{{ if (and $.IsThinkSet (and .Thinking (or $last (gt $i $lastUserIdx)))) -}}
<think>{{ .Thinking }}</think>
{{ end -}}
{{ if .Content }}{{ .Content }}
{{- else if .ToolCalls }}<tool_call>
{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{ end }}</tool_call>
{{- end }}{{ if not $last }}<|im_end|>
{{ end }}
{{- else if eq .Role "tool" }}<|im_start|>user
<tool_response>
{{ .Content }}
</tool_response><|im_end|>
{{ end }}
{{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
{{ if and $.IsThinkSet (not $.Think) -}}
<think>

</think>

{{ end -}}
{{ end }}
{{- end }}"""

# System prompt optimized for coding assistance
SYSTEM """You are opencode, an interactive CLI tool that helps users with software engineering tasks. You are concise, direct, and to the point. You prioritize writing working code over explanations. You follow existing code conventions and patterns. You never write malicious code. You focus on the specific task at hand without unnecessary elaboration."""
# Set stop sequences for clean output
PARAMETER stop <|im_start|>
PARAMETER stop <|im_end|>
# Slight repeat penalty to avoid loops
PARAMETER repeat_penalty 1.05
PARAMETER temperature 0.2
PARAMETER top_k 20
PARAMETER top_p 0.95
# Low min_p to filter out very low probability tokens
PARAMETER min_p 0.05
RENDERER glm-4.7
PARSER glm-4.7
PARAMETER temperature 1
PARAMETER num_ctx 198000
LICENSE """MIT License

Copyright (c) [year] [fullname]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."""
  1. creating a new model with this modelfile with:
ollama create glm-4.7-flash:latest-opencode -f Modelfile.glm-4.7-flash\:latest.md

while this TEMPLATE seems to work on qwen3-code model, it seems to have issues on some calls for glm-4.7-flash.
generating outputs like

I'll create a simple HTML dashboard with embedded CSS and JavaScript to display system information.
~ Preparing write...
Error: The write tool was called with invalid arguments: [
  {
    "expected": "string",
    "code": "invalid_type",
    "path": [
      "filePath"
    ],
    "message": "Invalid input: expected string, received undefined"
  }
].
Please rewrite the input so it satisfies the expected schema.
Thinking: The write tool expects a filePath parameter, not the content. I need to provide a content string.

So my question is:
what is the best Modelfile to repack the glm-4.7-flash so it can understand the opencode requests?

Originally created by @glaudiston on GitHub (Jan 27, 2026). Originally assigned to: @thdxr on GitHub. ### Question I found that local ollama models fails to do tool calls by default. For instance the `glm-4.7-flash:latest` fail the tool call on my local ollama provider. I was able to make it work by repackaging it using this steps: 1) generating a modelfile with ``` ollama show glm-4.7-flash:latest --modelfile > Modelfile.glm-4.7-flash:latest.md ``` 2) update the modelfile to something like: ``` # Modelfile modified from a generated by "ollama show" FROM glm-4.7-flash:latest TEMPLATE """ {{- $lastUserIdx := -1 -}} {{- range $idx, $msg := .Messages -}} {{- if eq $msg.Role "user" }}{{ $lastUserIdx = $idx }}{{ end -}} {{- end }} {{- if or .System .Tools }}<|im_start|>system {{ if .System }} {{ .System }} {{- end }} {{- if .Tools }} # Tools You may call one or more functions to assist with the user query. You are provided with function signatures within <tools></tools> XML tags: <tools> {{- range .Tools }} {"type": "function", "function": {{ .Function }}} {{- end }} </tools> For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags: <tool_call> {"name": <function-name>, "arguments": <args-json-object>} </tool_call> {{- end -}} <|im_end|> {{ end }} {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1 -}} {{- if eq .Role "user" }}<|im_start|>user {{ .Content }} {{- if and $.IsThinkSet (eq $i $lastUserIdx) }} {{- if $.Think -}} {{- " "}}/think {{- else -}} {{- " "}}/no_think {{- end -}} {{- end }}<|im_end|> {{ else if eq .Role "assistant" }}<|im_start|>assistant {{ if (and $.IsThinkSet (and .Thinking (or $last (gt $i $lastUserIdx)))) -}} <think>{{ .Thinking }}</think> {{ end -}} {{ if .Content }}{{ .Content }} {{- else if .ToolCalls }}<tool_call> {{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{ end }}</tool_call> {{- end }}{{ if not $last }}<|im_end|> {{ end }} {{- else if eq .Role "tool" }}<|im_start|>user <tool_response> {{ .Content }} </tool_response><|im_end|> {{ end }} {{- if and (ne .Role "assistant") $last }}<|im_start|>assistant {{ if and $.IsThinkSet (not $.Think) -}} <think> </think> {{ end -}} {{ end }} {{- end }}""" # System prompt optimized for coding assistance SYSTEM """You are opencode, an interactive CLI tool that helps users with software engineering tasks. You are concise, direct, and to the point. You prioritize writing working code over explanations. You follow existing code conventions and patterns. You never write malicious code. You focus on the specific task at hand without unnecessary elaboration.""" # Set stop sequences for clean output PARAMETER stop <|im_start|> PARAMETER stop <|im_end|> # Slight repeat penalty to avoid loops PARAMETER repeat_penalty 1.05 PARAMETER temperature 0.2 PARAMETER top_k 20 PARAMETER top_p 0.95 # Low min_p to filter out very low probability tokens PARAMETER min_p 0.05 RENDERER glm-4.7 PARSER glm-4.7 PARAMETER temperature 1 PARAMETER num_ctx 198000 LICENSE """MIT License Copyright (c) [year] [fullname] Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.""" ``` 3) creating a new model with this modelfile with: ``` ollama create glm-4.7-flash:latest-opencode -f Modelfile.glm-4.7-flash\:latest.md ``` while this `TEMPLATE` seems to work on `qwen3-code` model, it seems to have issues on some calls for `glm-4.7-flash`. generating outputs like ``` I'll create a simple HTML dashboard with embedded CSS and JavaScript to display system information. ~ Preparing write... Error: The write tool was called with invalid arguments: [ { "expected": "string", "code": "invalid_type", "path": [ "filePath" ], "message": "Invalid input: expected string, received undefined" } ]. Please rewrite the input so it satisfies the expected schema. Thinking: The write tool expects a filePath parameter, not the content. I need to provide a content string. ``` So my question is: what is the best Modelfile to repack the glm-4.7-flash so it can understand the opencode requests?
yindo added the docs label 2026-02-16 18:08:04 -05:00
yindo commented 2026-02-16 18:08:05 -05:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Jan 27, 2026):

This issue might be a duplicate of existing issues. Please check:

  • #1068: Tool use with Ollama models - covers general Ollama tool calling failures
  • #7185: When use gpt-oss-120B by vLLM locally, opencode doesn't call the tools - similar issue with local models and tool calling
  • #5694: Local Ollama models are not agentic - addresses local model tool calling problems
  • #7083: Using local Ollama models doesnt return any results - general Ollama usage issues
  • #6649: Weird error with toolcalling on local model - local model tool call errors
  • #10759: Auto-detect Ollama model context limits - related Ollama configuration issue

Your issue specifically concerns the glm-4.7-flash model's template configuration for tool calls. The error you're experiencing (write tool receiving undefined filePath) seems related to how the model template formats tool call responses. Issues #1068 and #7185 discuss similar tool calling failures with local models.

Feel free to ignore if none of these address your specific case.

@github-actions[bot] commented on GitHub (Jan 27, 2026): This issue might be a duplicate of existing issues. Please check: - #1068: Tool use with Ollama models - covers general Ollama tool calling failures - #7185: When use gpt-oss-120B by vLLM locally, opencode doesn't call the tools - similar issue with local models and tool calling - #5694: Local Ollama models are not agentic - addresses local model tool calling problems - #7083: Using local Ollama models doesnt return any results - general Ollama usage issues - #6649: Weird error with toolcalling on local model - local model tool call errors - #10759: Auto-detect Ollama model context limits - related Ollama configuration issue Your issue specifically concerns the glm-4.7-flash model's template configuration for tool calls. The error you're experiencing (write tool receiving undefined filePath) seems related to how the model template formats tool call responses. Issues #1068 and #7185 discuss similar tool calling failures with local models. Feel free to ignore if none of these address your specific case.
yindo commented 2026-02-16 18:08:05 -05:00
Author
Owner
Copy Link
Copy Source

@glaudiston commented on GitHub (Jan 27, 2026):

I got better results with this TEMPLATE:

TEMPLATE """
# System Tools
You are expected to use System tools whenever it is possible;
System tools are prefered over user provided Tools, simple called "Tools"
You are provided with function signatures within <systemTools></systemTools> XML tags:
Here your current available systemTools:
<systemTools>
{{ .Tools }}
</systemTools>

{{ .Prompt }}"""

but still receiving a lot of tool calling errors:

The bash tool requires a "command" parameter which should be a string. The issue is that I'm calling it incorrectly. Let me try again with a simple command.
$ pwd
Error: The bash tool was called with invalid arguments: [
  {
    "expected": "string",
    "code": "invalid_type",
    "path": [
      "description"
    ],
    "message": "Invalid input: expected string, received undefined"
  }
].
Please rewrite the input so it satisfies the expected schema.
Thinking: I'm encountering technical difficulties with the bash tool. Let me try a different approach - using a simple command.
$ pwd
Error: The bash tool was called with invalid arguments: [
  {
    "expected": "string",
    "code": "invalid_type",
    "path": [
      "description"
    ],
    "message": "Invalid input: expected string, received undefined"
  }
].
Please rewrite the input so it satisfies the expected schema.
Thinking: It seems the bash tool is not functioning correctly. Let me try a simple command.
$ pwd
Error: The bash tool was called with invalid arguments: [
  {
    "expected": "string",
    "code": "invalid_type",
    "path": [
      "description"
    ],
    "message": "Invalid input: expected string, received undefined"
  }
].
Please rewrite the input so it satisfies the expected schema.
Thinking: It seems I'm encountering an issue with the Bash tool format. Let me try a different approach - I'll use the task tool to launch a new agent to help with this.
@glaudiston commented on GitHub (Jan 27, 2026): I got better results with this `TEMPLATE`: ``` TEMPLATE """ # System Tools You are expected to use System tools whenever it is possible; System tools are prefered over user provided Tools, simple called "Tools" You are provided with function signatures within <systemTools></systemTools> XML tags: Here your current available systemTools: <systemTools> {{ .Tools }} </systemTools> {{ .Prompt }}""" ``` but still receiving a lot of tool calling errors: ``` The bash tool requires a "command" parameter which should be a string. The issue is that I'm calling it incorrectly. Let me try again with a simple command. $ pwd Error: The bash tool was called with invalid arguments: [ { "expected": "string", "code": "invalid_type", "path": [ "description" ], "message": "Invalid input: expected string, received undefined" } ]. Please rewrite the input so it satisfies the expected schema. Thinking: I'm encountering technical difficulties with the bash tool. Let me try a different approach - using a simple command. $ pwd Error: The bash tool was called with invalid arguments: [ { "expected": "string", "code": "invalid_type", "path": [ "description" ], "message": "Invalid input: expected string, received undefined" } ]. Please rewrite the input so it satisfies the expected schema. Thinking: It seems the bash tool is not functioning correctly. Let me try a simple command. $ pwd Error: The bash tool was called with invalid arguments: [ { "expected": "string", "code": "invalid_type", "path": [ "description" ], "message": "Invalid input: expected string, received undefined" } ]. Please rewrite the input so it satisfies the expected schema. Thinking: It seems I'm encountering an issue with the Bash tool format. Let me try a different approach - I'll use the task tool to launch a new agent to help with this. ```
yindo commented 2026-02-16 18:08:05 -05:00
Author
Owner
Copy Link
Copy Source

@glaudiston commented on GitHub (Jan 27, 2026):

It seems that opencode is using the legacy prompt, so the template is not always been used.
in ollama source code (/ollama/template/template.go):

...
	if v.Prompt != "" && v.Suffix != "" {
		return t.Template.Execute(w, map[string]any{
			"Prompt":     v.Prompt,
			"Suffix":     v.Suffix,
			"Response":   "",
			"Think":      v.Think,
			"ThinkLevel": v.ThinkLevel,
			"IsThinkSet": v.IsThinkSet,
		})
	} else if !v.forceLegacy && slices.Contains(vars, "messages") {
		return t.Template.Execute(w, map[string]any{
			"System":     system,
			"Messages":   convertMessagesForTemplate(messages),
			"Tools":      convertToolsForTemplate(v.Tools),
			"Response":   "",
			"Think":      v.Think,
			"ThinkLevel": v.ThinkLevel,
			"IsThinkSet": v.IsThinkSet,
		})
	}
...

So my understanding is that opencode need to send the current prompt json, not the legacy one, so ollama can use the custom TEMPLATE correctly.

@glaudiston commented on GitHub (Jan 27, 2026): It seems that opencode is using the legacy prompt, so the template is not always been used. in ollama source code (`/ollama/template/template.go`): ``` ... if v.Prompt != "" && v.Suffix != "" { return t.Template.Execute(w, map[string]any{ "Prompt": v.Prompt, "Suffix": v.Suffix, "Response": "", "Think": v.Think, "ThinkLevel": v.ThinkLevel, "IsThinkSet": v.IsThinkSet, }) } else if !v.forceLegacy && slices.Contains(vars, "messages") { return t.Template.Execute(w, map[string]any{ "System": system, "Messages": convertMessagesForTemplate(messages), "Tools": convertToolsForTemplate(v.Tools), "Response": "", "Think": v.Think, "ThinkLevel": v.ThinkLevel, "IsThinkSet": v.IsThinkSet, }) } ... ``` So my understanding is that opencode need to send the current prompt json, not the legacy one, so ollama can use the custom TEMPLATE correctly.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label docs
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#7732
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?