Model doesn't seem to include tool output in its response #178

New Issue

2026-02-15T16:28:31-05:00

yindo commented

2026-02-15 16:28:31 -05:00

Originally created by @mjvmiller on GitHub (Dec 2, 2024).

Originally assigned to: @ParthSareen on GitHub.

This issue is built off of the code I detailed in #345. After running effectively the same code I included there (with the one alteration being requesting the model to perform "alfalfa" on 2 and 3, rather than 2 and 4), I follow python example on generating a response based on the outcome of the tool call as follows:

if response.message.tool_calls:
    for tool in response.message.tool_calls:
        if function_to_call := toolfuncs[tool.function.name]:
            print('Calling function:', tool.function.name)
            print('Arguments:', tool.function.arguments)
            output = function_to_call(**tool.function.arguments)
            print('Function output:', output)
            messages.append(response.message)
            messages.append({'role': 'tool', 'content': str(output), 'name': tool.function.name})
final_response = oclient.chat(model="llama3.2:3b-instruct-q8_0", messages=messages)

This prints the following:

Calling function: alfalfa
Arguments: {'a': '2', 'b': '3'}
Function output: 72340

This indicates that the tool was chosen and called successfully. For multiple attempts at making the same request this remained consistently true. However, when I then print final_response.message.content I find that the model either doesn't understand what "alfalfa" means (despite understanding it well enough to call the tool), or simply makes up its own meaning for it.

Originally created by @mjvmiller on GitHub (Dec 2, 2024). Originally assigned to: @ParthSareen on GitHub. This issue is built off of the code I detailed in #345. After running effectively the same code I included there (with the one alteration being requesting the model to perform "alfalfa" on 2 and 3, rather than 2 and 4), I follow python example on generating a response based on the outcome of the tool call as follows: ``` if response.message.tool_calls: for tool in response.message.tool_calls: if function_to_call := toolfuncs[tool.function.name]: print('Calling function:', tool.function.name) print('Arguments:', tool.function.arguments) output = function_to_call(**tool.function.arguments) print('Function output:', output) messages.append(response.message) messages.append({'role': 'tool', 'content': str(output), 'name': tool.function.name}) final_response = oclient.chat(model="llama3.2:3b-instruct-q8_0", messages=messages) ``` This prints the following: ``` Calling function: alfalfa Arguments: {'a': '2', 'b': '3'} Function output: 72340 ``` This indicates that the tool was chosen and called successfully. For multiple attempts at making the same request this remained consistently true. However, when I then print `final_response.message.content` I find that the model either doesn't understand what "alfalfa" means (despite understanding it well enough to call the tool), or simply makes up its own meaning for it.

yindo closed this issue

2026-02-15 16:28:31 -05:00

yindo commented

2026-02-15 16:28:31 -05:00

@ParthSareen commented on GitHub (Dec 3, 2024):

So it truly depends on what your initial prompt is for the model, what your prompt is to get the final response, and the size of the model you're using.

Initial prompt - to get the best performance you should probably tell the model what all tools it's has access to even in the system prompt (although it does get passed in later as well)
Would recommend the final prompt to be something along the lines of "the tool output is attached and the function run was {function_to_call}, tell the user the final output of the function"
Larger models will inherently perform better - would recommend at least an 8b model for this

@ParthSareen commented on GitHub (Dec 3, 2024): So it truly depends on what your initial prompt is for the model, what your prompt is to get the final response, and the size of the model you're using. 1. Initial prompt - to get the best performance you should probably tell the model what all tools it's has access to even in the system prompt (although it does get passed in later as well) 2. Would recommend the final prompt to be something along the lines of "the tool output is attached and the function run was {function_to_call}, tell the user the final output of the function" 3. Larger models will inherently perform better - would recommend at least an 8b model for this

yindo referenced this issue

2026-02-15 16:30:15 -05:00

[PR #178] [MERGED] Bump pytest from 8.2.0 to 8.2.2 #396

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#178