Tools not working when providing a schema via format=... #282

New Issue

2026-02-15T16:29:33-05:00

yindo commented

2026-02-15 16:29:33 -05:00

Originally created by @AlexysGromard on GitHub (Jul 31, 2025).

Description

When using a function tool with a response model and providing its schema via the format argument, the tool is never invoked. The same code works when format is omitted, but breaks when a schema is passed using model_json_schema() from a Pydantic model.

Code to Reproduce

from ollama import ChatResponse, chat
from pydantic import BaseModel, Field
import json

class AddTwoNumbersOutput(BaseModel):
  """
  Output schema for the add_two_numbers function
  """
  a: int = Field(..., description="The first number",)
  b: int = Field(..., description="The second number",)
  result: int = Field(..., description="The result of the addition",)

def add_two_numbers(a: int, b: int) -> int:
  """
  Add two numbers

  Args:
    a (int): The first number
    b (int): The second number

  Returns:
    int: The sum of the two numbers
  """

  # The cast is necessary as returned tool call arguments don't always conform exactly to schema
  # E.g. this would prevent "what is 30 + 12" to produce '3012' instead of 42
  return int(a) + int(b)


messages = [{'role': 'user', 'content': 'What is three plus one?'}]
print('Prompt:', messages[0]['content'])

available_functions = {
  'add_two_numbers': add_two_numbers,
}

response: ChatResponse = chat(
  'llama3.1',
  messages=messages,
  tools=[add_two_numbers],
  format=AddTwoNumbersOutput.model_json_schema(),
)

if response.message.tool_calls:
  tool = response.message.tool_calls[0]
  # Ensure the function is available, and then call it
  if function_to_call := available_functions.get(tool.function.name):
    print('Calling function:', tool.function.name)
    print('Arguments:', tool.function.arguments)
    output = function_to_call(**tool.function.arguments)
    print('Function output:', output)
  else:
    print('Function', tool.function.name, 'not found')

# Only needed to chat with the model using the tool call results
if response.message.tool_calls:
  # Add the function response to messages for the model to use
  messages.append(response.message)
  messages.append({'role': 'tool', 'content': str(output), 'tool_name': tool.function.name})

  # Get final response from model with function outputs
  final_response = chat('llama3.1', messages=messages)
  print('Final response:', json.dumps(final_response.message.content, indent=2, ensure_ascii=False))

else:
  print('No tool calls returned from model')

Observed Behavior

With format=...: No tool call is made, response.message.tool_calls is empty.
Without format=...: The function add_two_numbers is correctly triggered and returns the result.

Expected Behavior

Providing a schema via format=AddTwoNumbersOutput.model_json_schema() should not prevent tool calls from being triggered.

My Environment

OS: macOS 15.6 (M1)
Python: 3.13.5
ollama-python: 0.5.1
Ollama 0.10.1
Model: llama3.1

Originally created by @AlexysGromard on GitHub (Jul 31, 2025). ## Description When using a function tool with a response model and providing its schema via the `format` argument, the tool is never invoked. The same code works when `format` is omitted, but breaks when a schema is passed using `model_json_schema()` from a Pydantic model. ## Code to Reproduce ```python from ollama import ChatResponse, chat from pydantic import BaseModel, Field import json class AddTwoNumbersOutput(BaseModel): """ Output schema for the add_two_numbers function """ a: int = Field(..., description="The first number",) b: int = Field(..., description="The second number",) result: int = Field(..., description="The result of the addition",) def add_two_numbers(a: int, b: int) -> int: """ Add two numbers Args: a (int): The first number b (int): The second number Returns: int: The sum of the two numbers """ # The cast is necessary as returned tool call arguments don't always conform exactly to schema # E.g. this would prevent "what is 30 + 12" to produce '3012' instead of 42 return int(a) + int(b) messages = [{'role': 'user', 'content': 'What is three plus one?'}] print('Prompt:', messages[0]['content']) available_functions = { 'add_two_numbers': add_two_numbers, } response: ChatResponse = chat( 'llama3.1', messages=messages, tools=[add_two_numbers], format=AddTwoNumbersOutput.model_json_schema(), ) if response.message.tool_calls: tool = response.message.tool_calls[0] # Ensure the function is available, and then call it if function_to_call := available_functions.get(tool.function.name): print('Calling function:', tool.function.name) print('Arguments:', tool.function.arguments) output = function_to_call(**tool.function.arguments) print('Function output:', output) else: print('Function', tool.function.name, 'not found') # Only needed to chat with the model using the tool call results if response.message.tool_calls: # Add the function response to messages for the model to use messages.append(response.message) messages.append({'role': 'tool', 'content': str(output), 'tool_name': tool.function.name}) # Get final response from model with function outputs final_response = chat('llama3.1', messages=messages) print('Final response:', json.dumps(final_response.message.content, indent=2, ensure_ascii=False)) else: print('No tool calls returned from model') ``` ## Observed Behavior * With `format=...`: No tool call is made, `response.message.tool_calls` is empty. * Without `format=...`: The function `add_two_numbers` is correctly triggered and returns the result. ## Expected Behavior Providing a schema via `format=AddTwoNumbersOutput.model_json_schema()` should not prevent tool calls from being triggered. ## My Environment * OS: macOS 15.6 (M1) * Python: 3.13.5 * ollama-python: 0.5.1 * Ollama 0.10.1 * Model: `llama3.1`

yindo closed this issue

2026-02-15 16:29:33 -05:00

yindo commented

2026-02-15 16:29:34 -05:00

@lennartpollvogt commented on GitHub (Aug 2, 2025):

I would like to keep it that way. I only use the model schema in format when I want to force the LLM to adhere to it, and it shouldn't have the possibility to call a tool. Otherwise the reliability would decrease.

But maybe you can explain what why you would expect the behavior you've described? Which use cases do you have where this would be necessary?

@lennartpollvogt commented on GitHub (Aug 2, 2025): I would like to keep it that way. I only use the model schema in format when I want to force the LLM to adhere to it, and it shouldn't have the possibility to call a tool. Otherwise the reliability would decrease. But maybe you can explain what why you would expect the behavior you've described? Which use cases do you have where this would be necessary?

yindo commented

2026-02-15 16:29:34 -05:00

@zeerd commented on GitHub (Sep 3, 2025):

The example given is not very good.
In fact, in the vast majority of cases, we do not know when LLM will complete all tool call processes and begin outputting the final results.

The real progress would be like below:

res = chat(tools=[...])
loop tool_calls in res
    function-call
    res = chat(tools=[...]) # key point
end
res = chat(formats=[...])

The last chat(tools=[...]) that give res without tool_calls in the loop will give the final answer without format.
User need to chat again with formats=[...] for the formatted answer.

This is a waste of tokens.

@zeerd commented on GitHub (Sep 3, 2025): The example given is not very good. In fact, in the vast majority of cases, we do not know when LLM will complete all tool call processes and begin outputting the final results. The real progress would be like below: ``` res = chat(tools=[...]) loop tool_calls in res function-call res = chat(tools=[...]) # key point end res = chat(formats=[...]) ``` The last `chat(tools=[...])` that give `res` without `tool_calls` in the `loop` will give the final answer without format. User need to `chat` again with `formats=[...]` for the formatted answer. This is a waste of tokens.

yindo commented

2026-02-15 16:29:35 -05:00

@ParthSareen commented on GitHub (Sep 18, 2025):

We're probably not going to support structured generation around tool calling for now. It's best to use a model better at tool calling. I will be working on erroring better so that the error can be used to inform the model about the incorrect call.

@ParthSareen commented on GitHub (Sep 18, 2025): We're probably not going to support structured generation around tool calling for now. It's best to use a model better at tool calling. I will be working on erroring better so that the error can be used to inform the model about the incorrect call.

yindo referenced this issue

2026-02-15 16:30:27 -05:00

[PR #282] [CLOSED] add OpenAI example in README.md. #450

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#282

Tools not working when providing a schema via format=... #282

Description

Code to Reproduce

Observed Behavior

Expected Behavior

My Environment

Tools not working when providing a schema via `format=...` #282