ollama/ollama-python
1
0
Fork 0
mirror of https://github.com/ollama/ollama-python.git synced 2026-07-21 09:05:23 -04:00
Code Issues 101 Packages Projects Releases 10 Wiki Activity

Question: the response I got by using terminal is way better than using ollama.generate #65

New Issue
Closed
opened 2026-02-15 16:27:49 -05:00 by yindo · 5 comments
yindo commented 2026-02-15 16:27:49 -05:00
Owner
Copy Link
Copy Source

Originally created by @wangyeye66 on GitHub (Apr 12, 2024).

I use llama2 7b to for text generation. The prompt I attampted:
"""Task: Turn the input into (subject, predicate, object).
Input: Sam Johnson is eating breakfast.
Output: (Dolores Murphy, eat, breakfast)
Input: Joon Park is brewing coffee.
Output: (Joon Park, brew, coffee)
Input: Jane Cook is sleeping.
Output: (Jane Cook, is, sleep)
Input: Michael Bernstein is writing email on a computer.
Output: (Michael Bernstein, write, email)
Input: Percy Liang is teaching students in a classroom.
Output: (Percy Liang, teach, students)
Input: Merrie Morris is running on a treadmill.
Output: (Merrie Morris, run, treadmill)
Input: John Doe is drinking coffee.
Output: (John Doe,"""

using ollama.generate will generate a chat rather than keep generating the text.
In terminal, it seems understand what I would like to do. Did I call wrong function in python? How can I let the model know I don't need a chat-like response?

Originally created by @wangyeye66 on GitHub (Apr 12, 2024). I use llama2 7b to for text generation. The prompt I attampted: """Task: Turn the input into (subject, predicate, object). Input: Sam Johnson is eating breakfast. Output: (Dolores Murphy, eat, breakfast) Input: Joon Park is brewing coffee. Output: (Joon Park, brew, coffee) Input: Jane Cook is sleeping. Output: (Jane Cook, is, sleep) Input: Michael Bernstein is writing email on a computer. Output: (Michael Bernstein, write, email) Input: Percy Liang is teaching students in a classroom. Output: (Percy Liang, teach, students) Input: Merrie Morris is running on a treadmill. Output: (Merrie Morris, run, treadmill) Input: John Doe is drinking coffee. Output: (John Doe,""" using ollama.generate will generate a chat rather than keep generating the text. In terminal, it seems understand what I would like to do. Did I call wrong function in python? How can I let the model know I don't need a chat-like response?
yindo closed this issue 2026-02-15 16:27:49 -05:00
yindo commented 2026-02-15 16:27:50 -05:00
Author
Owner
Copy Link
Copy Source

@93andresen commented on GitHub (Apr 13, 2024):

I've never tried this libary, but maybe "ollama.chat" works like the terminal amd "ollama.generate" is like autocomplete?

@93andresen commented on GitHub (Apr 13, 2024): I've never tried this libary, but maybe "ollama.chat" works like the terminal amd "ollama.generate" is like autocomplete?
yindo commented 2026-02-15 16:27:50 -05:00
Author
Owner
Copy Link
Copy Source

@ioo0s commented on GitHub (Jun 3, 2024):

I have the same problem, and the results I get from running it through the ollama run xxmodel terminal are much better than the results I get from python sdk client.chat. Why?

@ioo0s commented on GitHub (Jun 3, 2024): I have the same problem, and the results I get from running it through the `ollama run xxmodel` terminal are much better than the results I get from python sdk `client.chat`. Why?
yindo commented 2026-02-15 16:27:50 -05:00
Author
Owner
Copy Link
Copy Source

@BowenKwan commented on GitHub (Jun 13, 2024):

Same problem here. Using ollama run custom_model in the terminal gives a much better result than ollama.chat(model='custom_model.

It seems to me that all the few shot example provided in the modelfile used to train the custom_model is not provided to the custom model when using ollama.chat. The result seems to be just like using the base model that the custom model is trained on.

@BowenKwan commented on GitHub (Jun 13, 2024): Same problem here. Using `ollama run custom_model` in the terminal gives a much better result than `ollama.chat(model='custom_model`. It seems to me that all the few shot example provided in the modelfile used to train the custom_model is not provided to the custom model when using `ollama.chat`. The result seems to be just like using the base model that the custom model is trained on.
yindo commented 2026-02-15 16:27:50 -05:00
Author
Owner
Copy Link
Copy Source

@mxyng commented on GitHub (Jun 13, 2024):

@wangyeye66 can you paste the output you get from the cli and the output from the ollama.chat?

from what I can tell, this behavior is expected. llama2:7b implements a chat template which uses these messages to simulate a user/assistant exchange. this is regardless of what method is used to interact with the llm, cli, ollama.generate, or ollama.chat. here's (roughly) what your prompt will produce as an input to the llm:

[INST] <<SYS>><</SYS>> Task: Turn the input into (subject, predicate, object).
Input: Sam Johnson is eating breakfast.
Output: (Dolores Murphy, eat, breakfast)
Input: Joon Park is brewing coffee.
Output: (Joon Park, brew, coffee)
Input: Jane Cook is sleeping.
Output: (Jane Cook, is, sleep)
Input: Michael Bernstein is writing email on a computer.
Output: (Michael Bernstein, write, email)
Input: Percy Liang is teaching students in a classroom.
Output: (Percy Liang, teach, students)
Input: Merrie Morris is running on a treadmill.
Output: (Merrie Morris, run, treadmill)
Input: John Doe is drinking coffee.
Output: (John Doe, [/INST]

based on your prompt, you're probably more interested in the text completion model, llama2:7b-text, which does not template the input

@mxyng commented on GitHub (Jun 13, 2024): @wangyeye66 can you paste the output you get from the cli and the output from the `ollama.chat`? from what I can tell, this behavior is expected. [llama2:7b](https://ollama.com/library/llama2:7b) implements a chat template which uses these messages to simulate a user/assistant exchange. this is regardless of what method is used to interact with the llm, `cli`, `ollama.generate`, or `ollama.chat`. here's (roughly) what your prompt will produce as an input to the llm: ``` [INST] <<SYS>><</SYS>> Task: Turn the input into (subject, predicate, object). Input: Sam Johnson is eating breakfast. Output: (Dolores Murphy, eat, breakfast) Input: Joon Park is brewing coffee. Output: (Joon Park, brew, coffee) Input: Jane Cook is sleeping. Output: (Jane Cook, is, sleep) Input: Michael Bernstein is writing email on a computer. Output: (Michael Bernstein, write, email) Input: Percy Liang is teaching students in a classroom. Output: (Percy Liang, teach, students) Input: Merrie Morris is running on a treadmill. Output: (Merrie Morris, run, treadmill) Input: John Doe is drinking coffee. Output: (John Doe, [/INST] ``` based on your prompt, you're probably more interested in the text completion model, [llama2:7b-text](https://ollama.com/library/llama2:7b-text), which does not template the input
yindo commented 2026-02-15 16:27:50 -05:00
Author
Owner
Copy Link
Copy Source

@mxyng commented on GitHub (Jun 13, 2024):

@BowenKwan your issue appears different so I'll respond in #188

@mxyng commented on GitHub (Jun 13, 2024): @BowenKwan your issue appears different so I'll respond in #188
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
pull-request
Mirrored from GitHub Pull Request
 thinking
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ollama/ollama-python#65
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?