[PR #158] [MERGED] Avoid side effects if chat message contains images #388

New Issue

2026-02-15T16:30:12-05:00

yindo commented

2026-02-15 16:30:12 -05:00

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama-python/pull/158
Author: @tillfalko
Created: 5/18/2024
Status: ✅ Merged
Merged: 6/5/2024
Merged by: @mxyng

Base: main ← Head: main

📝 Commits (1)

0f8c20a Avoid side effects if chat message contains images

📊 Changes

1 file changed (+5 additions, -0 deletions)

View changed files

📝 ollama/_client.py (+5 -0)

📄 Description

The Issue

Generating chat responses with images has a rather nasty side effect in that, if a message has images, they will be overwritten by their Base64 encoding.

import ollama

messages = [{
    'role': 'user',
    'content': 'What is this?',
    'images' : ['ollama.png']
  },]

print(messages)

response = ollama.chat(model='llava-llama3', messages=messages)

print(messages)

[{'role': 'user', 'content': 'What is this?', 'images': ['ollama.png']}]
[{'role': 'user', 'content': 'What is this?', 'images': ['iVBORw0KGgoAAAANSUhEUgAAALUAAAEACAYAAAD1IzfbAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAABzUSURBVHgB7Z0JtGRVdYZ/bTUMDiiKMnYLhKFVJCBOKDxdIM4oEiUaYiNxJRpFkhVwjLTLKVFwiONSGWIaUIFAUNAElUYDR (it goes on for way longer)

If a user passed an image path, it is undesirable to have this path overwritten by extremely verbose Base64-text.

The Cause

Most python users will pass messages as sequences of dictionaries. Dictionaries are inherently mutable object that are passed by reference, which is why the line
https://github.com/ollama/ollama-python/blob/cb81f522b0f0035acbfeeed87b7902856bda501e/ollama/_client.py#L174-L175

also affects the user's list/tuple of messages.

The Solution

Obviously this can be avoided on the user's end by simply making a deep copy themselves before passing messages. But because this is a rather obscure issue that is never mentioned in the documentation, I recommend simply removing the possiblility of this side-effect.

There are multiple possible solutions to this issue, but because the performance cost of making a deep copy is insignificant in comparison to the performance cost of LLM inference, I went with the easiest fix: simply making a deep copy of messages every time ollama.chat is called, but I am looking forward to hearing the opinions of the Ollama maintainers.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama-python/pull/158 **Author:** [@tillfalko](https://github.com/tillfalko) **Created:** 5/18/2024 **Status:** ✅ Merged **Merged:** 6/5/2024 **Merged by:** [@mxyng](https://github.com/mxyng) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (1) - [`0f8c20a`](https://github.com/ollama/ollama-python/commit/0f8c20a596c32832a7f6e293523b7c420bcc4172) Avoid side effects if chat message contains images ### 📊 Changes **1 file changed** (+5 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `ollama/_client.py` (+5 -0) </details> ### 📄 Description ### The Issue Generating chat responses with images has a rather nasty side effect in that, if a message has images, they will be overwritten by their Base64 encoding. ```python import ollama messages = [{ 'role': 'user', 'content': 'What is this?', 'images' : ['ollama.png'] },] print(messages) response = ollama.chat(model='llava-llama3', messages=messages) print(messages) ``` ``` [{'role': 'user', 'content': 'What is this?', 'images': ['ollama.png']}] [{'role': 'user', 'content': 'What is this?', 'images': ['iVBORw0KGgoAAAANSUhEUgAAALUAAAEACAYAAAD1IzfbAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAABzUSURBVHgB7Z0JtGRVdYZ/bTUMDiiKMnYLhKFVJCBOKDxdIM4oEiUaYiNxJRpFkhVwjLTLKVFwiONSGWIaUIFAUNAElUYDR (it goes on for way longer) ``` If a user passed an image path, it is undesirable to have this path overwritten by extremely verbose Base64-text. ### The Cause Most python users will pass messages as sequences of dictionaries. Dictionaries are inherently mutable object that are passed by reference, which is why the line https://github.com/ollama/ollama-python/blob/cb81f522b0f0035acbfeeed87b7902856bda501e/ollama/_client.py#L174-L175 also affects the user's list/tuple of messages. ### The Solution Obviously this can be avoided on the user's end by simply making a deep copy themselves before passing `messages`. But because this is a rather obscure issue that is never mentioned in the documentation, I recommend simply removing the possiblility of this side-effect. There are multiple possible solutions to this issue, but because the performance cost of making a deep copy is insignificant in comparison to the performance cost of LLM inference, I went with the easiest fix: simply making a deep copy of `messages` every time `ollama.chat` is called, but I am looking forward to hearing the opinions of the Ollama maintainers. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

yindo added the pull-request label 2026-02-15 16:30:12 -05:00

yindo closed this issue

2026-02-15 16:30:12 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#388