Support Metrics #70

New Issue

2026-02-15T16:27:51-05:00

yindo commented

2026-02-15 16:27:51 -05:00

Originally created by @slyt on GitHub (Apr 19, 2024).

The Ollama API has a Metrics object that appears to be returned with both the ChatResponse and GenerateResponse.

I can't find a way to get these metrics using a client with ollama-python. Is it supported?

I would expect something like this to work:

from ollama import Client
from ollama import Message

client = Client(host='http://localhost:11434')
client.pull("llama3")

system_message = Message(role="system", content="You are a helpful AI assistant")
user_message = Message(role="user", content=f"Hello there!")
response= client .chat(model="llama3", messages=[system_message, user_message]) 
metrics= response.get("metrics", {}) # This doesn't work
print(f"metrics: {metrics}")

currently, the above code outputs:
metrics: {}

Originally created by @slyt on GitHub (Apr 19, 2024). The Ollama API has a [Metrics](https://github.com/ollama/ollama/blob/8d1995c625e7f2ed2ff98eb099e1bd8d7e6e133e/api/types.go#L105-L112) object that appears to be returned with both the [ChatResponse](https://github.com/ollama/ollama/blob/8d1995c625e7f2ed2ff98eb099e1bd8d7e6e133e/api/types.go#L102) and [GenerateResponse](https://github.com/ollama/ollama/blob/8d1995c625e7f2ed2ff98eb099e1bd8d7e6e133e/api/types.go#L270). I can't find a way to get these metrics using a client with ollama-python. Is it supported? I would expect something like this to work: ```python from ollama import Client from ollama import Message client = Client(host='http://localhost:11434') client.pull("llama3") system_message = Message(role="system", content="You are a helpful AI assistant") user_message = Message(role="user", content=f"Hello there!") response= client .chat(model="llama3", messages=[system_message, user_message]) metrics= response.get("metrics", {}) # This doesn't work print(f"metrics: {metrics}") ``` currently, the above code outputs: `metrics: {}`

yindo closed this issue

2026-02-15 16:27:51 -05:00

yindo commented

2026-02-15 16:27:52 -05:00

@slyt commented on GitHub (Apr 19, 2024):

Here's an example of parsing the metrics from the API

import requests
import json

data = {"model": "llama3", "prompt": "Why is the sky blue?", "stream":True}
response = requests.post("http://localhost:11434/api/generate", json=data)
response.raise_for_status()

metrics = ["total_duration", "load_duration", "prompt_eval_duration", "eval_count", "eval_duration"]

# convert response.content from bytes to json dict
content = response.content.decode("utf-8")
for line in content.split("\n"):
    if not line.strip(): # ignore empty lines
        continue
    content_json = json.loads(line)
    done = content_json.get("done", None)
    print(line)
    if done:
        metrics_dict = {metric: content_json.get(metric, None) for metric in metrics}
        print(metrics_dict)

This works for streaming and non-streaming responses.

Output is:
{'total_duration': 8037858768, 'load_duration': 207136, 'prompt_eval_duration': 235696000, 'eval_count': 464, 'eval_duration': 7801437000}

@slyt commented on GitHub (Apr 19, 2024): Here's an example of parsing the metrics from the API ```python import requests import json data = {"model": "llama3", "prompt": "Why is the sky blue?", "stream":True} response = requests.post("http://localhost:11434/api/generate", json=data) response.raise_for_status() metrics = ["total_duration", "load_duration", "prompt_eval_duration", "eval_count", "eval_duration"] # convert response.content from bytes to json dict content = response.content.decode("utf-8") for line in content.split("\n"): if not line.strip(): # ignore empty lines continue content_json = json.loads(line) done = content_json.get("done", None) print(line) if done: metrics_dict = {metric: content_json.get(metric, None) for metric in metrics} print(metrics_dict) ``` This works for streaming and non-streaming responses. Output is: `{'total_duration': 8037858768, 'load_duration': 207136, 'prompt_eval_duration': 235696000, 'eval_count': 464, 'eval_duration': 7801437000}`

yindo commented

2026-02-15 16:27:52 -05:00

@mxyng commented on GitHub (Apr 24, 2024):

metrics are returned in the top level JSON object. here's the equivalent using the client

from ollama import chat

metrics = ['total_duration', 'load_duration', 'prompt_eval_duration', 'eval_count', 'eval_duration']
r = chat('llama3', [{'role': 'user', 'content': 'Hello, world!'}])
print({metric: r[metric] for metric in metrics})
# Prints:
# {'total_duration': 1325679458, 'load_duration': 538143625, 'prompt_eval_duration': 117545000, 'eval_count': 41, 'eval_duration': 667624000}

@mxyng commented on GitHub (Apr 24, 2024): metrics are returned in the top level JSON object. here's the equivalent using the client ```python from ollama import chat metrics = ['total_duration', 'load_duration', 'prompt_eval_duration', 'eval_count', 'eval_duration'] r = chat('llama3', [{'role': 'user', 'content': 'Hello, world!'}]) print({metric: r[metric] for metric in metrics}) # Prints: # {'total_duration': 1325679458, 'load_duration': 538143625, 'prompt_eval_duration': 117545000, 'eval_count': 41, 'eval_duration': 667624000} ```

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#70