Persistent chat memory #121

New Issue

2026-02-15T16:28:08-05:00

yindo commented

2026-02-15 16:28:08 -05:00

Originally created by @kikoferrer on GitHub (Jul 31, 2024).

I have ollama-python running with a custom ollama model. It works very well except that it does not remember the conversation at all. Every chat is like a new conversation.

I checked issues and I cant find the same problem I am having.(Or I did not look enough. Forgive me.)

I made this basic prototype with chat stream to test.

import ollama


model = 'Llama3'

def chat(message):
    messages = [{
    'role': 'user',
    'content': message,
    }]
    response = ollama.chat(model=model, messages=messages, stream=True)
    for line in response:
        print(line['message']['content'], end='', flush=True)

while True:
    print('\nQ to quit')
    prompt = input('Enter your message: ')
    if prompt.lower() == 'q':
        break
    else:
        chat(prompt)

Is there a doc somewhere I can use as guide for more use cases like persistent memory/context using the model's context size? Similar to how Ollama run works where the chat is continuous. Thanks.

Originally created by @kikoferrer on GitHub (Jul 31, 2024). I have ollama-python running with a custom ollama model. It works very well except that it does not remember the conversation at all. Every chat is like a new conversation. I checked issues and I cant find the same problem I am having.(Or I did not look enough. Forgive me.) I made this basic prototype with chat stream to test. ``` import ollama model = 'Llama3' def chat(message): messages = [{ 'role': 'user', 'content': message, }] response = ollama.chat(model=model, messages=messages, stream=True) for line in response: print(line['message']['content'], end='', flush=True) while True: print('\nQ to quit') prompt = input('Enter your message: ') if prompt.lower() == 'q': break else: chat(prompt) ``` Is there a doc somewhere I can use as guide for more use cases like persistent memory/context using the model's context size? Similar to how Ollama run works where the chat is continuous. Thanks.

yindo closed this issue

2026-02-15 16:28:08 -05:00

yindo commented

2026-02-15 16:28:09 -05:00

@ArnoldIOI commented on GitHub (Aug 1, 2024):

You need to keep the conversation in your messages here response = ollama.chat(model=model, messages=messages, stream=True)
Ollama's official doc provides an example on this.

@ArnoldIOI commented on GitHub (Aug 1, 2024): You need to keep the conversation in your `messages` here `response = ollama.chat(model=model, messages=messages, stream=True)` Ollama's official doc provides an [example](https://github.com/ollama/ollama/blob/main/docs/api.md#chat-request-with-history) on this.

yindo commented

2026-02-15 16:28:09 -05:00

@kikoferrer commented on GitHub (Aug 9, 2024):

Does this mean declaring an empty list [] as messages variable and inserting each conversation turn to creat a conversation history, then using the while chat history as prompt for the chat?

@kikoferrer commented on GitHub (Aug 9, 2024): Does this mean declaring an empty list `[]` as `messages` variable and inserting each conversation turn to creat a conversation history, then using the while chat history as prompt for the chat?

yindo commented

2026-02-15 16:28:09 -05:00

@kikoferrer commented on GitHub (Aug 9, 2024):

Figured it out. For anyone who might encounter the same question, I will leave the bare bone code here. It can certainly remember previous conversation.

import ollama


model = 'Test'
messages = []

def chat(message):
    user_message = [{
    'role': 'user',
    'content': message,
    }]
    messages.append(user_message[0])
    response = ollama.chat(model=model, messages=messages)
    answer = response['message']['content']
    messages.append(response['message'])
    return answer


while True:
    print('\nQ to quit')
    prompt = input('Enter your message: ')
    if prompt.lower() == 'q':
        break
    else:
        reply = chat(prompt)
        print(reply)

Though I am unable to recreate this with streaming=True. Anyone who can figure out to do this in streaming mode please let me know. Thanks

@kikoferrer commented on GitHub (Aug 9, 2024): Figured it out. For anyone who might encounter the same question, I will leave the bare bone code here. It can certainly remember previous conversation. ``` import ollama model = 'Test' messages = [] def chat(message): user_message = [{ 'role': 'user', 'content': message, }] messages.append(user_message[0]) response = ollama.chat(model=model, messages=messages) answer = response['message']['content'] messages.append(response['message']) return answer while True: print('\nQ to quit') prompt = input('Enter your message: ') if prompt.lower() == 'q': break else: reply = chat(prompt) print(reply) ``` Though I am unable to recreate this with streaming=True. Anyone who can figure out to do this in streaming mode please let me know. Thanks

yindo commented

2026-02-15 16:28:09 -05:00

@ArnoldIOI commented on GitHub (Aug 10, 2024):

I wrote this based on the examples you provided, which worked for me.
I think the problem you met (to maintain the history with steaming=True) is to get the complete response properly.

import ollama


model = 'Gemma2:2b'
messages = []
# Roles
USER = 'user'
ASSISTANT = 'assistant'

def add_history(content, role):
    messages.append({'role': role, 'content': content})

def chat(message):
    add_history(message, USER)
    response = ollama.chat(model=model, messages=messages, stream=True)
    complete_message = ''
    for line in response:
        complete_message += line['message']['content']
        print(line['message']['content'], end='', flush=True)
    add_history(complete_message, ASSISTANT)

while True:
    print('\nQ to quit')
    prompt = input('Enter your message: ')
    if prompt.lower() == 'q':
        break
    else:
        chat(prompt)

@ArnoldIOI commented on GitHub (Aug 10, 2024): I wrote this based on the examples you provided, which worked for me. I think the problem you met (to maintain the history with steaming=True) is to get the complete response properly. ``` import ollama model = 'Gemma2:2b' messages = [] # Roles USER = 'user' ASSISTANT = 'assistant' def add_history(content, role): messages.append({'role': role, 'content': content}) def chat(message): add_history(message, USER) response = ollama.chat(model=model, messages=messages, stream=True) complete_message = '' for line in response: complete_message += line['message']['content'] print(line['message']['content'], end='', flush=True) add_history(complete_message, ASSISTANT) while True: print('\nQ to quit') prompt = input('Enter your message: ') if prompt.lower() == 'q': break else: chat(prompt) ```

yindo commented

2026-02-15 16:28:09 -05:00

@dyexlzc commented on GitHub (Aug 15, 2024):

hi,i hava same question too.
i found there has a simple parameter in api /generate:
https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion

Generate a completion
POST /api/generate

request:

Advanced parameters (optional):

context: the context parameter returned from a previous request to /generate, this can be used to keep a short conversational memory

response:

context: an encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory

i wonder if i use /api/chat and want to keep memory, means only can store message at client(such as python script) and send all history message like {"role":"user",“content":"xxxx"} in the POST request body?
is there have a same things for /api/chat like parameter context in /api/generate? thanks for all!

@dyexlzc commented on GitHub (Aug 15, 2024): hi,i hava same question too. i found there has a simple parameter in api /generate: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion > Generate a completion > POST /api/generate request: > Advanced parameters (optional): > > context: the context parameter returned from a previous request to /generate, this can be used to keep a short conversational memory response: > context: an encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory i wonder if i use /api/chat and want to keep memory, means only can store message at client(such as python script) and send all history message like {"role":"user",“content":"xxxx"} in the POST request body? is there have a same things for /api/chat like parameter `context` in /api/generate? thanks for all!

yindo commented

2026-02-15 16:28:09 -05:00

@dyexlzc commented on GitHub (Aug 15, 2024):

hi,i hava same question too. i found there has a simple parameter in api /generate: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion

Generate a completion
POST /api/generate

request:

Advanced parameters (optional):
context: the context parameter returned from a previous request to /generate, this can be used to keep a short conversational memory

response:

context: an encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory

i wonder if i use /api/chat and want to keep memory, means only can store message at client(such as python script) and send all history message like {"role":"user",“content":"xxxx"} in the POST request body? is there have a same things for /api/chat like parameter in /api/generate? thanks for all!context

or is there are any different between use /api/generate with context and use /api/chat with all history message

@dyexlzc commented on GitHub (Aug 15, 2024): > hi,i hava same question too. i found there has a simple parameter in api /generate: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion > > > Generate a completion > > POST /api/generate > > request: > > > Advanced parameters (optional): > > context: the context parameter returned from a previous request to /generate, this can be used to keep a short conversational memory > > response: > > > context: an encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory > > i wonder if i use /api/chat and want to keep memory, means only can store message at client(such as python script) and send all history message like {"role":"user",“content":"xxxx"} in the POST request body? is there have a same things for /api/chat like parameter in /api/generate? thanks for all!`context` or is there are any different between use /api/generate with `context` and use /api/chat with all history message

yindo commented

2026-02-15 16:28:09 -05:00

@jwaitzel commented on GitHub (Sep 7, 2024):

Hello, I'm having the same issue, I'm learning to use llms so probably I'm missing something, but with Strem=False, I'm sending all the message history, but the model reply to all my messages together. I attach an example.

New msg: 10 + 10
20.
New msg: + 40
To evaluate the expression, I'll follow the order of operations:

1. First, add 10 and 10:
10 + 10 = 20
2. Now, add 20 (the result from step 1) to 40:
20 + 40 = 60

Therefore, the final answer is: **60**

@jwaitzel commented on GitHub (Sep 7, 2024): Hello, I'm having the same issue, I'm learning to use llms so probably I'm missing something, but with Strem=False, I'm sending all the message history, but the model reply to all my messages together. I attach an example. ``` New msg: 10 + 10 20. New msg: + 40 To evaluate the expression, I'll follow the order of operations: 1. First, add 10 and 10: 10 + 10 = 20 2. Now, add 20 (the result from step 1) to 40: 20 + 40 = 60 Therefore, the final answer is: **60** ```

yindo commented

2026-02-15 16:28:09 -05:00

@ParthSareen commented on GitHub (Nov 26, 2024):

Closing this out - we have an example under the examples directory :D

@ParthSareen commented on GitHub (Nov 26, 2024): Closing this out - we have an example under the examples directory :D

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#121