function calling #15

New Issue

Closed

opened 2026-02-15 16:27:27 -05:00 by yindo · 13 comments

yindo commented 2026-02-15 16:27:27 -05:00

Owner

Copy Link

Copy Source

Originally created by @iplayfast on GitHub (Jan 29, 2024).

I keep seeing that openai has function calling [https://platform.openai.com/docs/api-reference/chat/create] (now called tooltypes) and some open source llm also support function calling.

This is done by having the models fine tuned to understand when they need to call a function. As we don't have that ability (as far as I know) maybe we could emulate it by adding a layer between ollama and the api, so the api can be added to.

So calling
ollama.tools() will return what tools are available.
ollama.addtool(name="string",prompt,function)

addtool would add to a list of tools the name and the prompt to recognize and the response it should have..
For instance to do an internet search
ollama.addtool("BrowseWeb","If the answer needs to access html pages return 'BrowseWeb:url'",BrowseWeb)
def BrowseWeb(url):
return httpx(url)

This is probably not the best solution, but I thought I would make it an issue to see if any discussion will come from it.

Originally created by @iplayfast on GitHub (Jan 29, 2024). I keep seeing that openai has function calling [https://platform.openai.com/docs/api-reference/chat/create] (now called tooltypes) and some open source llm also support function calling. This is done by having the models fine tuned to understand when they need to call a function. As we don't have that ability (as far as I know) maybe we could emulate it by adding a layer between ollama and the api, so the api can be added to. So calling ollama.tools() will return what tools are available. ollama.addtool(name="string",prompt,function) addtool would add to a list of tools the name and the prompt to recognize and the response it should have.. For instance to do an internet search ollama.addtool("BrowseWeb","If the answer needs to access html pages return 'BrowseWeb:url'",BrowseWeb) def BrowseWeb(url): return httpx(url) This is probably not the best solution, but I thought I would make it an issue to see if any discussion will come from it.

yindo closed this issue 2026-02-15 16:27:27 -05:00

yindo commented 2026-02-15 16:27:28 -05:00

Author

Owner

Copy Link

Copy Source

@jeremytregunna commented on GitHub (Feb 13, 2024):

One possible solution for you, that works well for me... is define a schema to represent your function's arguments, and pass that into the system message, get json formatted response using json mode in the API. Then treat the response you get (don't recommend streaming the json) as the input data you can then use to match to the function you want to invoke given the data.

It's not the same approach that OpenAI uses, but it's similar and does work.

@jeremytregunna commented on GitHub (Feb 13, 2024): One possible solution for you, that works well for me... is define a schema to represent your function's arguments, and pass that into the system message, get json formatted response using json mode in the API. Then treat the response you get (don't recommend streaming the json) as the input data you can then use to match to the function you want to invoke given the data. It's not the same approach that OpenAI uses, but it's similar and does work.

yindo commented 2026-02-15 16:27:28 -05:00

Author

Owner

Copy Link

Copy Source

@eliranwong commented on GitHub (Feb 23, 2024):

Function calling makes a big difference.

It would be perfect if ollama could support function calling

@eliranwong commented on GitHub (Feb 23, 2024): Function calling makes a big difference. It would be perfect if ollama could support function calling

yindo commented 2026-02-15 16:27:28 -05:00

Author

Owner

Copy Link

Copy Source

@eliranwong commented on GitHub (Mar 1, 2024):

An approach to run multiple functions with Ollama: https://github.com/eliranwong/freegenius#approach-to-run-function-calling-equivalent-features-offline-with-common-hardwares I like Ollama and will use ollama as the default engine for the project.

@eliranwong commented on GitHub (Mar 1, 2024): An approach to run multiple functions with Ollama: https://github.com/eliranwong/freegenius#approach-to-run-function-calling-equivalent-features-offline-with-common-hardwares I like Ollama and will use ollama as the default engine for the project.

yindo commented 2026-02-15 16:27:28 -05:00

Author

Owner

Copy Link

Copy Source

@jeremytregunna commented on GitHub (Mar 1, 2024):

I have been able to get function calling (openai style) to work with dolphin-mistral:8x7b-v2.7-q8_0, probably works with others too but probably not them all. Here's the prompt I send:

         To respond to the users message, you have access to the following tools:             
         {                                                                                    
           "name": "duckduckgo_search",                                                       
           "description": "Use this function to search DuckDuckGo for a query.\n\nArgs:\n     
         query(str): The query to search for.\n    max_results (optional, default=5): The     
         maximum number of results to return.\n\nReturns:\n    The result from DuckDuckGo.",  
           "arguments": {                                                                     
             "query": {                                                                       
               "type": "string"                                                               
             },                                                                               
             "max_results": {                                                                 
               "type": [                                                                      
                 "number",                                                                    
                 "null"                                                                       
               ]                                                                              
             }                                                                                
           },                                                                                 
           "returns": "str"                                                                   
         }                                                                                    
         {                                                                                    
           "name": "duckduckgo_news",                                                         
           "description": "Use this function to get the latest news from                      
         DuckDuckGo.\n\nArgs:\n    query(str): The query to search for.\n    max_results      
         (optional, default=5): The maximum number of results to return.\n\nReturns:\n    The 
         latest news from DuckDuckGo.",                                                       
           "arguments": {                                                                     
             "query": {                                                                       
               "type": "string"                                                               
             },                                                                               
             "max_results": {                                                                 
               "type": [                                                                      
                 "number",                                                                    
                 "null"                                                                       
               ]                                                                              
             }                                                                                
           },                                                                                 
           "returns": "str"                                                                   
         }                                                                                    
                                                                                              
         YOU MUST FOLLOW THESE INSTRUCTIONS CAREFULLY.                                        
         <instructions>                                                                       
         1. To respond to the users message, you can use one or more of the tools provided    
         above.                                                                               
         2. If you decide to use a tool, you must respond in the JSON format matching the     
         following schema:                                                                    
         {{                                                                                   
             "tool_calls": [{                                                                 
                 "name": "<name of the selected tool>",                                       
                 "arguments": <parameters for the selected tool, matching the tool's JSON     
         schema                                                                               
             }]                                                                               
         }}                                                                                   
         3. To use a tool, just respond with the JSON matching the schema. Nothing else. Do   
         not add any additional notes or explanations                                         
         4. After you use a tool, the next message you get will contain the result of the tool
         call.                                                                                
         5. REMEMBER: To use a tool, you must respond only in JSON format.                    
         6. After you use a tool and receive the result back, respond regularly to answer the 
         users question.                                                                      
         7. Only use the tools you are provided.                                              
         8. Use markdown to format your answers.                                              
         </instructions>                                                                      
         ============== user ==============                                                   
         What's the weather like in Toronto?

which causes output from the model like this (for illustrative purposes):

         Building tool calls from [{'name': 'duckduckgo_search', 'arguments': {'query':       
         'weather in Toronto'}}]                                                              
         ============== assistant ==============                                              
         {"tool_calls": [                                                                     
             {                                                                                
                 "name": "duckduckgo_search",                                                 
                 "arguments": {                                                               
                     "query": "weather in Toronto"                                            
                 }                                                                            
             }                                                                                
         ]}

Hope that helps illustrate that at least with some models, you can do it directly with prompting and json mode.

@jeremytregunna commented on GitHub (Mar 1, 2024): I have been able to get function calling (openai style) to work with dolphin-mistral:8x7b-v2.7-q8_0, probably works with others too but probably not them all. Here's the prompt I send: ``` To respond to the users message, you have access to the following tools: { "name": "duckduckgo_search", "description": "Use this function to search DuckDuckGo for a query.\n\nArgs:\n query(str): The query to search for.\n max_results (optional, default=5): The maximum number of results to return.\n\nReturns:\n The result from DuckDuckGo.", "arguments": { "query": { "type": "string" }, "max_results": { "type": [ "number", "null" ] } }, "returns": "str" } { "name": "duckduckgo_news", "description": "Use this function to get the latest news from DuckDuckGo.\n\nArgs:\n query(str): The query to search for.\n max_results (optional, default=5): The maximum number of results to return.\n\nReturns:\n The latest news from DuckDuckGo.", "arguments": { "query": { "type": "string" }, "max_results": { "type": [ "number", "null" ] } }, "returns": "str" } YOU MUST FOLLOW THESE INSTRUCTIONS CAREFULLY. <instructions> 1. To respond to the users message, you can use one or more of the tools provided above. 2. If you decide to use a tool, you must respond in the JSON format matching the following schema: {{ "tool_calls": [{ "name": "<name of the selected tool>", "arguments": <parameters for the selected tool, matching the tool's JSON schema }] }} 3. To use a tool, just respond with the JSON matching the schema. Nothing else. Do not add any additional notes or explanations 4. After you use a tool, the next message you get will contain the result of the tool call. 5. REMEMBER: To use a tool, you must respond only in JSON format. 6. After you use a tool and receive the result back, respond regularly to answer the users question. 7. Only use the tools you are provided. 8. Use markdown to format your answers. </instructions> ============== user ============== What's the weather like in Toronto? ``` which causes output from the model like this (for illustrative purposes): ``` Building tool calls from [{'name': 'duckduckgo_search', 'arguments': {'query': 'weather in Toronto'}}] ============== assistant ============== {"tool_calls": [ { "name": "duckduckgo_search", "arguments": { "query": "weather in Toronto" } } ]} ``` Hope that helps illustrate that at least with some models, you can do it directly with prompting and json mode.

yindo commented 2026-02-15 16:27:28 -05:00

Author

Owner

Copy Link

Copy Source

@eliranwong commented on GitHub (Mar 1, 2024):

It is helpful, much appreciated

@eliranwong commented on GitHub (Mar 1, 2024): It is helpful, much appreciated

yindo commented 2026-02-15 16:27:29 -05:00

Author

Owner

Copy Link

Copy Source

@eliranwong commented on GitHub (Mar 2, 2024):

I have been able to get function calling (openai style) to work with dolphin-mistral:8x7b-v2.7-q8_0, probably works with others too but probably not them all. Here's the prompt I send:

         To respond to the users message, you have access to the following tools:             
         {                                                                                    
           "name": "duckduckgo_search",                                                       

Hope that helps illustrate that at least with some models, you can do it directly with prompting and json mode.

This example works in some models. However, when I try to add more than 5 actions or above, the waiting time is getting terrible.

I have a project https://github.com/eliranwong/letmedoit uses ChatGPT by default for function calling among around 40 functions in one go, without an issue. I am working on how to use Ollama instead. So far, waiting time for a single prompt of 40 functions is not acceptable at all. I find a little hope to split the process in steps for generations.

@eliranwong commented on GitHub (Mar 2, 2024): > I have been able to get function calling (openai style) to work with dolphin-mistral:8x7b-v2.7-q8_0, probably works with others too but probably not them all. Here's the prompt I send: > > ``` > To respond to the users message, you have access to the following tools: > { > "name": "duckduckgo_search", > ``` > > Hope that helps illustrate that at least with some models, you can do it directly with prompting and json mode. This example works in some models. However, when I try to add more than 5 actions or above, the waiting time is getting terrible. I have a project https://github.com/eliranwong/letmedoit uses ChatGPT by default for function calling among around 40 functions in one go, without an issue. I am working on how to use Ollama instead. So far, waiting time for a single prompt of 40 functions is not acceptable at all. I find a little hope to split the process in steps for generations.

yindo commented 2026-02-15 16:27:29 -05:00

Author

Owner

Copy Link

Copy Source

@eliranwong commented on GitHub (Mar 12, 2024):

It seems this workaround works with better speed: https://github.com/eliranwong/freegenius#approach-to-run-function-calling-equivalent-features-offline-with-common-hardwares

I am testing with "phi", "mistral" and "llama2" via Ollama

... still in testing, though ...

@eliranwong commented on GitHub (Mar 12, 2024): It seems this workaround works with better speed: https://github.com/eliranwong/freegenius#approach-to-run-function-calling-equivalent-features-offline-with-common-hardwares I am testing with "phi", "mistral" and "llama2" via Ollama  ... still in testing, though ...

yindo commented 2026-02-15 16:27:29 -05:00

Author

Owner

Copy Link

Copy Source

@jquintanilla4 commented on GitHub (Apr 10, 2024):

i just used instructor. It's a bit of a pain to setup, but once you do. It works without issues everytime.
https://jxnl.github.io/instructor/

@jquintanilla4 commented on GitHub (Apr 10, 2024): i just used instructor. It's a bit of a pain to setup, but once you do. It works without issues everytime. https://jxnl.github.io/instructor/

yindo commented 2026-02-15 16:27:29 -05:00

Author

Owner

Copy Link

Copy Source

@lennartpollvogt commented on GitHub (Jun 2, 2024):

@jquintanilla4 I was inspired by instructor and tried to create a similar library just for Ollama. It is still missing some features like supporting async and I haven't tried if it works with images, but I am working on that and the documentation. Here is the link to the repo: ollama-instructor
Would be cool to have some feedback, if it would work for you projects as well

@lennartpollvogt commented on GitHub (Jun 2, 2024): @jquintanilla4 I was inspired by instructor and tried to create a similar library just for Ollama. It is still missing some features like supporting async and I haven't tried if it works with images, but I am working on that and the documentation. Here is the link to the repo: [ollama-instructor](https://github.com/lennartpollvogt/ollama-instructor) Would be cool to have some feedback, if it would work for you projects as well

yindo commented 2026-02-15 16:27:29 -05:00

Author

Owner

Copy Link

Copy Source

@elhumbertoz commented on GitHub (Jul 19, 2024):

I have been able to get function calling (openai style) to work with dolphin-mistral:8x7b-v2.7-q8_0, probably works with others too but probably not them all. Here's the prompt I send:

         To respond to the users message, you have access to the following tools:             
         {                                                                                    
           "name": "duckduckgo_search",                                                       
           "description": "Use this function to search DuckDuckGo for a query.\n\nArgs:\n     
         query(str): The query to search for.\n    max_results (optional, default=5): The     
         maximum number of results to return.\n\nReturns:\n    The result from DuckDuckGo.",  
           "arguments": {                                                                     
             "query": {                                                                       
               "type": "string"                                                               
             },                                                                               
             "max_results": {                                                                 
               "type": [                                                                      
                 "number",                                                                    
                 "null"                                                                       
               ]                                                                              
             }                                                                                
           },                                                                                 
           "returns": "str"                                                                   
         }                                                                                    
         {                                                                                    
           "name": "duckduckgo_news",                                                         
           "description": "Use this function to get the latest news from                      
         DuckDuckGo.\n\nArgs:\n    query(str): The query to search for.\n    max_results      
         (optional, default=5): The maximum number of results to return.\n\nReturns:\n    The 
         latest news from DuckDuckGo.",                                                       
           "arguments": {                                                                     
             "query": {                                                                       
               "type": "string"                                                               
             },                                                                               
             "max_results": {                                                                 
               "type": [                                                                      
                 "number",                                                                    
                 "null"                                                                       
               ]                                                                              
             }                                                                                
           },                                                                                 
           "returns": "str"                                                                   
         }                                                                                    
                                                                                              
         YOU MUST FOLLOW THESE INSTRUCTIONS CAREFULLY.                                        
         <instructions>                                                                       
         1. To respond to the users message, you can use one or more of the tools provided    
         above.                                                                               
         2. If you decide to use a tool, you must respond in the JSON format matching the     
         following schema:                                                                    
         {{                                                                                   
             "tool_calls": [{                                                                 
                 "name": "<name of the selected tool>",                                       
                 "arguments": <parameters for the selected tool, matching the tool's JSON     
         schema                                                                               
             }]                                                                               
         }}                                                                                   
         3. To use a tool, just respond with the JSON matching the schema. Nothing else. Do   
         not add any additional notes or explanations                                         
         4. After you use a tool, the next message you get will contain the result of the tool
         call.                                                                                
         5. REMEMBER: To use a tool, you must respond only in JSON format.                    
         6. After you use a tool and receive the result back, respond regularly to answer the 
         users question.                                                                      
         7. Only use the tools you are provided.                                              
         8. Use markdown to format your answers.                                              
         </instructions>                                                                      
         ============== user ==============                                                   
         What's the weather like in Toronto?

which causes output from the model like this (for illustrative purposes):

         Building tool calls from [{'name': 'duckduckgo_search', 'arguments': {'query':       
         'weather in Toronto'}}]                                                              
         ============== assistant ==============                                              
         {"tool_calls": [                                                                     
             {                                                                                
                 "name": "duckduckgo_search",                                                 
                 "arguments": {                                                               
                     "query": "weather in Toronto"                                            
                 }                                                                            
             }                                                                                
         ]}

Hope that helps illustrate that at least with some models, you can do it directly with prompting and json mode.

Thank you,

I modify a litte:

context += `To respond to the users message, you have access to the following tools:

${JSON.stringify(functions)}

YOU MUST FOLLOW THESE INSTRUCTIONS CAREFULLY.

1. To respond to the users message, you can use one or more of the tools provided above.
2. If you decide to use a tool, you must respond in the JSON format matching the following schema:
   {
   "tool_calls": [{
   "name": "",
   "arguments": <parameters for the selected tool, matching the tool's JSON schema
   }]
   }
   3. Get the arguments for calling functions only from the context of this conversation. Don't concoct them; you will be able to ask the user for any missing information.
3. To use a tool, just respond with the JSON matching the schema. Nothing else. Do not add any additional notes or explanations
4. After you use a tool, the next message you get will contain the result of the tool call.
5. REMEMBER: To use a tool, you must respond only in JSON format.
6. After you use a tool and receive the result back, respond regularly to answer the users question.
7. Only use the tools you are provided.
8. Use markdown to format your answers.
   `
   }

@elhumbertoz commented on GitHub (Jul 19, 2024): > I have been able to get function calling (openai style) to work with dolphin-mistral:8x7b-v2.7-q8_0, probably works with others too but probably not them all. Here's the prompt I send: > > ``` > To respond to the users message, you have access to the following tools: > { > "name": "duckduckgo_search", > "description": "Use this function to search DuckDuckGo for a query.\n\nArgs:\n > query(str): The query to search for.\n max_results (optional, default=5): The > maximum number of results to return.\n\nReturns:\n The result from DuckDuckGo.", > "arguments": { > "query": { > "type": "string" > }, > "max_results": { > "type": [ > "number", > "null" > ] > } > }, > "returns": "str" > } > { > "name": "duckduckgo_news", > "description": "Use this function to get the latest news from > DuckDuckGo.\n\nArgs:\n query(str): The query to search for.\n max_results > (optional, default=5): The maximum number of results to return.\n\nReturns:\n The > latest news from DuckDuckGo.", > "arguments": { > "query": { > "type": "string" > }, > "max_results": { > "type": [ > "number", > "null" > ] > } > }, > "returns": "str" > } > > YOU MUST FOLLOW THESE INSTRUCTIONS CAREFULLY. > <instructions> > 1. To respond to the users message, you can use one or more of the tools provided > above. > 2. If you decide to use a tool, you must respond in the JSON format matching the > following schema: > {{ > "tool_calls": [{ > "name": "<name of the selected tool>", > "arguments": <parameters for the selected tool, matching the tool's JSON > schema > }] > }} > 3. To use a tool, just respond with the JSON matching the schema. Nothing else. Do > not add any additional notes or explanations > 4. After you use a tool, the next message you get will contain the result of the tool > call. > 5. REMEMBER: To use a tool, you must respond only in JSON format. > 6. After you use a tool and receive the result back, respond regularly to answer the > users question. > 7. Only use the tools you are provided. > 8. Use markdown to format your answers. > </instructions> > ============== user ============== > What's the weather like in Toronto? > ``` > > which causes output from the model like this (for illustrative purposes): > > ``` > Building tool calls from [{'name': 'duckduckgo_search', 'arguments': {'query': > 'weather in Toronto'}}] > ============== assistant ============== > {"tool_calls": [ > { > "name": "duckduckgo_search", > "arguments": { > "query": "weather in Toronto" > } > } > ]} > ``` > > Hope that helps illustrate that at least with some models, you can do it directly with prompting and json mode. Thank you, I modify a litte: context += `To respond to the users message, you have access to the following tools: ${JSON.stringify(functions)} YOU MUST FOLLOW THESE INSTRUCTIONS CAREFULLY. <instructions> 1. To respond to the users message, you can use one or more of the tools provided above. 2. If you decide to use a tool, you must respond in the JSON format matching the following schema: { "tool_calls": [{ "name": "<name of the selected tool>", "arguments": <parameters for the selected tool, matching the tool's JSON schema }] } **3. Get the arguments for calling functions only from the context of this conversation. Don't concoct them; you will be able to ask the user for any missing information.** 4. To use a tool, just respond with the JSON matching the schema. Nothing else. Do not add any additional notes or explanations 5. After you use a tool, the next message you get will contain the result of the tool call. 6. REMEMBER: To use a tool, you must respond only in JSON format. 7. After you use a tool and receive the result back, respond regularly to answer the users question. 8. Only use the tools you are provided. 9. Use markdown to format your answers. </instructions>` }

yindo commented 2026-02-15 16:27:30 -05:00

Author

Owner

Copy Link

Copy Source

@SpudGunMan commented on GitHub (Sep 3, 2024):

https://medium.com/@meirgotroot/ollama-support-for-tool-calling-186bcfeb892f

https://ollama.com/blog/tool-support

@SpudGunMan commented on GitHub (Sep 3, 2024): https://medium.com/@meirgotroot/ollama-support-for-tool-calling-186bcfeb892f https://ollama.com/blog/tool-support

yindo commented 2026-02-15 16:27:30 -05:00

Author

Owner

Copy Link

Copy Source

@eliranwong commented on GitHub (Sep 5, 2024):

My previous struggles with ollama function calling was that it was not easy for ollama to handle 20+ tools in one go. However, I finally found a way to run multiple tools with ollama too. In case anyone interested https://github.com/eliranwong/freegenius/wiki/Multiple-Tools-in-One-Go

@eliranwong commented on GitHub (Sep 5, 2024): My previous struggles with ollama function calling was that it was not easy for ollama to handle 20+ tools in one go. However, I finally found a way to run multiple tools with ollama too. In case anyone interested https://github.com/eliranwong/freegenius/wiki/Multiple-Tools-in-One-Go

yindo commented 2026-02-15 16:27:30 -05:00

Author

Owner

Copy Link

Copy Source

@ParthSareen commented on GitHub (Feb 13, 2025):

Closing this out: https://ollama.com/blog/functions-as-tools

@ParthSareen commented on GitHub (Feb 13, 2025): Closing this out: https://ollama.com/blog/functions-as-tools

yindo referenced this issue 2026-02-15 16:29:59 -05:00

[PR #15] [MERGED] examples #326

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dependabot/github_actions/actions/setup-python-7

dependabot/github_actions/actions/checkout-7

dependabot/pip/certifi-2026.4.22

dependabot/pip/pydantic-core-2.46.3

dependabot/pip/httpcore-1.0.9

dependabot/pip/anyio-4.12.1

dependabot/pip/typing-extensions-4.15.0

add-hatch-test-matrix

dependabot/github_actions/astral-sh/setup-uv-7

parth/point-to-local-websearch

nicole/example-refactor

nicole/browser-tool

parth/turbo-auth

parth/proxy-auth

parth/test

parth/tool-prototype

brucemacd/list-capabilities

tool-parsing-json-package

parth/revert-462

parth/tokenize-detokenize

revert-330-parth/qol-disable-tests-for-readmes-and-examples

parth/examples-pydantic-tools-and-more

jyan/tools

mxyng/layers-from-files

v0.6.2

v0.6.1

v0.6.0

v0.5.4

v0.5.3

v0.5.2

v0.5.1

v0.5.0

v0.4.9

v0.4.8

v0.4.7

v0.4.6

v0.4.5

v0.4.4

v0.4.3

v0.4.2

v0.4.1

v0.4.0

v0.3.3

v0.3.2

v0.3.1

v0.3.0

v0.2.1

v0.2.0

v0.1.9

v0.1.8

v0.1.7

v0.1.6

v0.1.5

v0.1.4

v0.1.3

v0.1.2

v0.1.0

v0.0.1

Labels

Clear labels

bug

enhancement

pull-request

Mirrored from GitHub Pull Request

thinking

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

yindo

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#15