Client.create inserts escape quotes \" for the last PARAMETER stop. #75

New Issue

2026-02-15T16:27:52-05:00

yindo commented

2026-02-15 16:27:52 -05:00

Originally created by @chigkim on GitHub (Apr 27, 2024).

Hi,

If you run the following code to copy llama3 to test, it creates a modelfile with escape qotes for the last PARAMETER stop.
If I use ollama create test -f test.modelfile, it works fine.

from ollama import Client
import re

client = Client(host='http://localhost:11434')
modelfile = client.show('llama3')['modelfile']
print('Original modelfile:\n', modelfile)
from_str = re.search('# (FROM.*?\n)', modelfile)[1]
modelfile = re.sub('FROM /Users.*?\n', from_str, modelfile)
print('Modified modelfile:\n', modelfile)
client.create('test', modelfile=modelfile, stream=False)
response = client.generate(model='test', prompt='hello!')
print('Response:\n', response['response'])
modelfile = client.show('test')['modelfile']
print('new modelfile:\n', modelfile)
client.delete('test')

Here's the output.

Original modelfile:
 # Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM llama3:latest

FROM /Users/cgk/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""
PARAMETER num_keep 24
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
Modified modelfile:
 # Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM llama3:latest

FROM llama3:latest
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""
PARAMETER num_keep 24
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
Response:
 Hello there! It's nice to meet you. Is there something I can help you with, or would you like to chat?<|eot_id|>
new modelfile:
 # Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM test:latest

FROM llama3:latest
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""
PARAMETER num_keep 24
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "\"<|eot_id|>\""

If you look at the last line, there's the escape quotes ". Not sure why only the last line has that.
Also if you look at the response string, it ends with <|eot_id|> because the last PARAMETER stop string in the new modelfile doesn't work.
Could you look into this?
Thanks so much!

Originally created by @chigkim on GitHub (Apr 27, 2024). Hi, If you run the following code to copy llama3 to test, it creates a modelfile with escape qotes for the last PARAMETER stop. If I use `ollama create test -f test.modelfile`, it works fine. ```python from ollama import Client import re client = Client(host='http://localhost:11434') modelfile = client.show('llama3')['modelfile'] print('Original modelfile:\n', modelfile) from_str = re.search('# (FROM.*?\n)', modelfile)[1] modelfile = re.sub('FROM /Users.*?\n', from_str, modelfile) print('Modified modelfile:\n', modelfile) client.create('test', modelfile=modelfile, stream=False) response = client.generate(model='test', prompt='hello!') print('Response:\n', response['response']) modelfile = client.show('test')['modelfile'] print('new modelfile:\n', modelfile) client.delete('test') ``` Here's the output. ```bash Original modelfile: # Modelfile generated by "ollama show" # To build a new Modelfile based on this one, replace the FROM line with: # FROM llama3:latest FROM /Users/cgk/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>""" PARAMETER num_keep 24 PARAMETER stop "<|start_header_id|>" PARAMETER stop "<|end_header_id|>" PARAMETER stop "<|eot_id|>" Modified modelfile: # Modelfile generated by "ollama show" # To build a new Modelfile based on this one, replace the FROM line with: # FROM llama3:latest FROM llama3:latest TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>""" PARAMETER num_keep 24 PARAMETER stop "<|start_header_id|>" PARAMETER stop "<|end_header_id|>" PARAMETER stop "<|eot_id|>" Response: Hello there! It's nice to meet you. Is there something I can help you with, or would you like to chat?<|eot_id|> new modelfile: # Modelfile generated by "ollama show" # To build a new Modelfile based on this one, replace the FROM line with: # FROM test:latest FROM llama3:latest TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>""" PARAMETER num_keep 24 PARAMETER stop "<|start_header_id|>" PARAMETER stop "<|end_header_id|>" PARAMETER stop "\"<|eot_id|>\"" ``` If you look at the last line, there's the escape quotes \". Not sure why only the last line has that. Also if you look at the response string, it ends with <|eot_id|> because the last PARAMETER stop string in the new modelfile doesn't work. Could you look into this? Thanks so much!

yindo closed this issue

2026-02-15 16:27:52 -05:00

yindo commented

2026-02-15 16:27:52 -05:00

@mxyng commented on GitHub (May 2, 2024):

thanks for the issue. since this is a ollama serve bug, I'm going to close it in favor of https://github.com/ollama/ollama/issues/3977

@mxyng commented on GitHub (May 2, 2024): thanks for the issue. since this is a ollama serve bug, I'm going to close it in favor of https://github.com/ollama/ollama/issues/3977

yindo referenced this issue

2026-02-15 16:30:05 -05:00

[PR #75] [CLOSED] Stability testing for client #353

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#75