langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

Agent Tools: ValueError: <function_name> is not strict. Only strict function tools can be auto-parsed #305

New Issue
Closed
opened 2026-02-20 17:36:00 -05:00 by yindo · 1 comment
yindo commented 2026-02-20 17:36:00 -05:00
Owner
Copy Link
Copy Source

Originally created by @jaimeescano on GitHub (Nov 11, 2024).

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangGraph/LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangGraph/LangChain rather than my code.
  • I am sure this is better as an issue rather than a GitHub discussion, since this is a LangGraph bug and not a design question.

Example Code

############################ Graph flow/src/graph.py
from langgraph.prebuilt import ToolNode, tools_condition
from langgraph.store.memory import InMemoryStore
from langgraph.graph.message import add_messages
from langgraph.graph import StateGraph

from langchain_core.messages import SystemMessage, HumanMessage, ToolMessage, AIMessage
from langchain_openai import AzureChatOpenAI
from langchain_core.tools import tool

from typing_extensions import Annotated, TypedDict
import random
import os

############# Graph #############
class ConfigSchema(TypedDict):
    """Define the schema for the config object."""
    thread_id: str

class StateSchema(TypedDict):
    """Define the schema for the state object."""
    prompt: str
    messages: Annotated[list, add_messages] = []


############# Tools #############
@tool
def get_city_population(city:str) -> int:
    """Get the population of a city"""
    return random.randint(1000, 1000000)

tools = [get_city_population]
############# Nodes #############
class Agent():
    name: str = "Agent"
    agent: AzureChatOpenAI = None
    tools: list = []

    def __init__(self, tools: list = [], name: str = "Agent"):
        self.name = name

        self.agent = AzureChatOpenAI(
            azure_deployment=os.getenv('MODEL', 'gpt-4o-mini'),
            azure_endpoint=os.getenv('AZURE_OPENAI_ENDPOINT'),
            openai_api_key=os.getenv('AZURE_OPENAI_API_KEY'),
            openai_api_version = os.getenv("OPENAI_API_VERSION"),
            model_kwargs = {
                "response_format": {"type": "json_object"}
            }
        )

        # Bind tools to the agent if they are provided
        if len(tools) > 0:
            self.tools = tools

        # Bind tools to the agent if they are already set
        if len(self.tools) > 0:
            self.agent = self.agent.bind_tools(tools)
            # self.agent = self.agent.bind_tools(tools, strict = True)

        # Disabling this line as it seems not having any effect        
        # self.agent.bind(response_format={"type": "json_object"})

    def run(self, state: dict, config: dict) -> dict:
        existing_messages = state.get('messages', [])

        new_messages = []

        if not existing_messages:
            new_messages = [
                SystemMessage("Your task is to find the population of a city requested by the user. Your response should be a JSON object with the city name and the population. {'<city>':'<population>'}"),
                HumanMessage(state.get('prompt')),
            ]

        response = self.agent.invoke(existing_messages + new_messages)

        new_messages.append(response)

        return {
            "messages": new_messages
        }


graph_builder = StateGraph(StateSchema, ConfigSchema)

agent_node = Agent(tools=tools)
tools_node = ToolNode(tools=tools)

# Add the nodes to the graph

graph_builder.add_node(agent_node.name, agent_node.run)
graph_builder.add_node(tools_node.name, tools_node)

graph_builder.add_conditional_edges(
    agent_node.name,
    tools_condition
)
graph_builder.add_edge(tools_node.name, agent_node.name)

# Configure the graph
graph_builder.set_entry_point(agent_node.name)
graph_builder.set_finish_point(agent_node.name)

in_memory_store = InMemoryStore()
graph = graph_builder.compile(store = in_memory_store)




############################ FastAPI api.py
from fastapi import FastAPI, Body
from pydantic import BaseModel
from flow.src.graph import graph as graph_module

app = FastAPI()

class GraphInput(BaseModel):
    prompt: str

@app.post("/graph")
async def create_graph(input: GraphInput):
    return graph_module.invoke({"prompt": input.prompt})

Error Message and Stack Trace (if applicable)

ERROR:    Exception in ASGI application
 Traceback (most recent call last):
   File "/usr/local/lib/python3.11/site-packages/uvicorn-0.32.0-py3.11.egg/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
     result = await app(  # type: ignore[func-returns-value]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/uvicorn-0.32.0-py3.11.egg/uvicorn/middleware/proxy_headers.py", line 60, in __call__
     return await self.app(scope, receive, send)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/fastapi-0.115.4-py3.11.egg/fastapi/applications.py", line 1054, in __call__
     await super().__call__(scope, receive, send)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/applications.py", line 113, in __call__
     await self.middleware_stack(scope, receive, send)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/middleware/errors.py", line 187, in __call__
     raise exc
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/middleware/errors.py", line 165, in __call__
     await self.app(scope, receive, _send)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/middleware/exceptions.py", line 62, in __call__
     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/_exception_handler.py", line 53, in wrapped_app
     raise exc
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/_exception_handler.py", line 42, in wrapped_app
     await app(scope, receive, sender)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 715, in __call__
     await self.middleware_stack(scope, receive, send)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 735, in app
     await route.handle(scope, receive, send)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 288, in handle
     await self.app(scope, receive, send)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 76, in app
     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/_exception_handler.py", line 53, in wrapped_app
     raise exc
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/_exception_handler.py", line 42, in wrapped_app
     await app(scope, receive, sender)
   File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 73, in app
     response = await f(request)
                ^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/fastapi-0.115.4-py3.11.egg/fastapi/routing.py", line 301, in app
     raw_response = await run_endpoint_function(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/fastapi-0.115.4-py3.11.egg/fastapi/routing.py", line 212, in run_endpoint_function
     return await dependant.call(**values)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/app/api.py", line 14, in create_graph
     return graph_module.invoke({"prompt": input.prompt})
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/pregel/__init__.py", line 1608, in invoke
     for chunk in self.stream(
   File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/pregel/__init__.py", line 1336, in stream
     for _ in runner.tick(
   File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/pregel/runner.py", line 58, in tick
     run_with_retry(t, retry_policy)
   File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/pregel/retry.py", line 29, in run_with_retry
     task.proc.invoke(task.input, config)
   File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/utils/runnable.py", line 410, in invoke
     input = context.run(step.invoke, input, config, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/utils/runnable.py", line 184, in invoke
     ret = context.run(self.func, input, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/app/flow/src/graph.py", line 71, in run
     response = self.agent.invoke(messages)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/runnables/base.py", line 5354, in invoke
     return self.bound.invoke(
            ^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 286, in invoke
     self.generate_prompt(
   File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 786, in generate_prompt
     return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 643, in generate
     raise e
   File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 633, in generate
     self._generate_with_cache(
   File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 851, in _generate_with_cache
     result = self._generate(
              ^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/langchain_openai-0.2.6-py3.11.egg/langchain_openai/chat_models/base.py", line 701, in _generate
     response = self.root_client.beta.chat.completions.parse(**payload)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/openai-1.54.3-py3.11.egg/openai/resources/beta/chat/completions.py", line 142, in parse
     _validate_input_tools(tools)
   File "/usr/local/lib/python3.11/site-packages/openai-1.54.3-py3.11.egg/openai/lib/_parsing/_completions.py", line 53, in validate_input_tools
     raise ValueError(
 ValueError: `get_city_population` is not strict. Only `strict` function tools can be auto-parsed

Description

We are experiencing an issue with the AzureChatOpenAI binding tools into it. The odd behaviour is that the graph works in LangGraph Studio .. but onces it's triggered via FastAPI .. an error is thrown indicating "Only strict function tools can be auto-parsed".

The issue seems to be related to the response_format. When declaring the LLM using AzureChatOpenAI, we are passing "response_format": {"type": "json_object"}. We have tried using the option agent.bind(response_format={"type": "json_object"}) but it doesn't seems to have any effect, if we remove "JSON" word from the system prompt ... it won't give any error ... whereas it should respond "'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'.". So utilizing the "model_kwargs" within AzureChatOpenAI did activate the "response_format".

Troubleshooting 1:

  1. Set model_kwargs = {"response_format": {"type": "json_object"}}
  2. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"}
  3. Get ERROR, Only strict function tools can be auto-parsed

Troubleshooting 2:

  1. Set model_kwargs = {"response_format": {"type": "json_object"}}
  2. Set agent.bind_tools(tools, strict = True)
  3. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"}
  4. Get ERROR, langgraph.errors.GraphRecursionError: Recursion limit of 25 reached without hitting a stop condition

Troubleshooting 3:

  1. Setting agent.bind(response_format={"type": "json_object"})
  2. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"}
  3. WORKED .. but looks like response_format is not activated.
  4. Removed "JSON" keyword from the system prompt
  5. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"}
  6. WORKED, didn't trigger any error .. so json_object feature not "used". Which is not a valid option.

Troubleshooting 4:

  1. Set model_kwargs = {"response_format": {"type": "json_object"}}
  2. Removed "JSON" keyword from the system prompt
  3. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"}
  4. Get ERROR (which is expected): "'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'.". Which means ... the "feature" about reponse_format is activated.

And finally, running the initial version using model_kwargs = {"response_format": {"type": "json_object"}} and just using
self.agent = agent.bind_tools(tools) WORKS perfect if the graph if triggered using LangGraph Studio.

Within the stack error, it shows 'openai/resources/beta/chat/completions.py' ... so not sure if this could be related?

Thanks for your help and support ;-)

System Info

python -m langchain_core.sys_info

System Information

OS: Linux
OS Version: #1 SMP Mon Aug 12 08:47:01 UTC 2024
Python Version: 3.11.10 (main, Oct 19 2024, 18:56:55) [GCC 12.2.0]

Package Information

langchain_core: 0.3.15
langchain: 0.3.7
langchain_community: 0.3.5
langsmith: 0.1.142
langchain_openai: 0.2.6
langchain_postgres: 0.0.12
langchain_text_splitters: 0.3.2
langgraph: 0.2.45

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.11.0rc1
async-timeout: Installed. No version info available.
dataclasses-json: 0.6.7
httpx: 0.27.2
httpx-sse: 0.4.0
jsonpatch: 1.33
langgraph-checkpoint: 2.0.2
langgraph-sdk: 0.1.35
numpy: 1.26.4
openai: 1.54.3
orjson: 3.10.11
packaging: 24.1
pgvector: 0.2.5
psycopg: 3.2.3
psycopg-pool: 3.2.3
pydantic: 2.10.0b1
pydantic-settings: 2.6.1
PyYAML: 6.0.2
requests: 2.32.3
requests-toolbelt: 1.0.0
sqlalchemy: 2.0.35
SQLAlchemy: 2.0.35
tenacity: 9.0.0
tiktoken: 0.8.0
typing-extensions: 4.12.2

Originally created by @jaimeescano on GitHub (Nov 11, 2024). ### Checked other resources - [X] I added a very descriptive title to this issue. - [X] I searched the [LangGraph](https://langchain-ai.github.io/langgraph/)/LangChain documentation with the integrated search. - [X] I used the GitHub search to find a similar question and didn't find it. - [X] I am sure that this is a bug in LangGraph/LangChain rather than my code. - [X] I am sure this is better as an issue [rather than a GitHub discussion](https://github.com/langchain-ai/langgraph/discussions/new/choose), since this is a LangGraph bug and not a design question. ### Example Code ```python ############################ Graph flow/src/graph.py from langgraph.prebuilt import ToolNode, tools_condition from langgraph.store.memory import InMemoryStore from langgraph.graph.message import add_messages from langgraph.graph import StateGraph from langchain_core.messages import SystemMessage, HumanMessage, ToolMessage, AIMessage from langchain_openai import AzureChatOpenAI from langchain_core.tools import tool from typing_extensions import Annotated, TypedDict import random import os ############# Graph ############# class ConfigSchema(TypedDict): """Define the schema for the config object.""" thread_id: str class StateSchema(TypedDict): """Define the schema for the state object.""" prompt: str messages: Annotated[list, add_messages] = [] ############# Tools ############# @tool def get_city_population(city:str) -> int: """Get the population of a city""" return random.randint(1000, 1000000) tools = [get_city_population] ############# Nodes ############# class Agent(): name: str = "Agent" agent: AzureChatOpenAI = None tools: list = [] def __init__(self, tools: list = [], name: str = "Agent"): self.name = name self.agent = AzureChatOpenAI( azure_deployment=os.getenv('MODEL', 'gpt-4o-mini'), azure_endpoint=os.getenv('AZURE_OPENAI_ENDPOINT'), openai_api_key=os.getenv('AZURE_OPENAI_API_KEY'), openai_api_version = os.getenv("OPENAI_API_VERSION"), model_kwargs = { "response_format": {"type": "json_object"} } ) # Bind tools to the agent if they are provided if len(tools) > 0: self.tools = tools # Bind tools to the agent if they are already set if len(self.tools) > 0: self.agent = self.agent.bind_tools(tools) # self.agent = self.agent.bind_tools(tools, strict = True) # Disabling this line as it seems not having any effect # self.agent.bind(response_format={"type": "json_object"}) def run(self, state: dict, config: dict) -> dict: existing_messages = state.get('messages', []) new_messages = [] if not existing_messages: new_messages = [ SystemMessage("Your task is to find the population of a city requested by the user. Your response should be a JSON object with the city name and the population. {'<city>':'<population>'}"), HumanMessage(state.get('prompt')), ] response = self.agent.invoke(existing_messages + new_messages) new_messages.append(response) return { "messages": new_messages } graph_builder = StateGraph(StateSchema, ConfigSchema) agent_node = Agent(tools=tools) tools_node = ToolNode(tools=tools) # Add the nodes to the graph graph_builder.add_node(agent_node.name, agent_node.run) graph_builder.add_node(tools_node.name, tools_node) graph_builder.add_conditional_edges( agent_node.name, tools_condition ) graph_builder.add_edge(tools_node.name, agent_node.name) # Configure the graph graph_builder.set_entry_point(agent_node.name) graph_builder.set_finish_point(agent_node.name) in_memory_store = InMemoryStore() graph = graph_builder.compile(store = in_memory_store) ############################ FastAPI api.py from fastapi import FastAPI, Body from pydantic import BaseModel from flow.src.graph import graph as graph_module app = FastAPI() class GraphInput(BaseModel): prompt: str @app.post("/graph") async def create_graph(input: GraphInput): return graph_module.invoke({"prompt": input.prompt}) ``` ### Error Message and Stack Trace (if applicable) ```shell ERROR: Exception in ASGI application Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/uvicorn-0.32.0-py3.11.egg/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn-0.32.0-py3.11.egg/uvicorn/middleware/proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi-0.115.4-py3.11.egg/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/applications.py", line 113, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/middleware/errors.py", line 187, in __call__ raise exc File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/middleware/errors.py", line 165, in __call__ await self.app(scope, receive, _send) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 715, in __call__ await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 735, in app await route.handle(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/usr/local/lib/python3.11/site-packages/starlette-0.41.2-py3.11.egg/starlette/routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi-0.115.4-py3.11.egg/fastapi/routing.py", line 301, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/fastapi-0.115.4-py3.11.egg/fastapi/routing.py", line 212, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api.py", line 14, in create_graph return graph_module.invoke({"prompt": input.prompt}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/pregel/__init__.py", line 1608, in invoke for chunk in self.stream( File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/pregel/__init__.py", line 1336, in stream for _ in runner.tick( File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/pregel/runner.py", line 58, in tick run_with_retry(t, retry_policy) File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/pregel/retry.py", line 29, in run_with_retry task.proc.invoke(task.input, config) File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/utils/runnable.py", line 410, in invoke input = context.run(step.invoke, input, config, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langgraph-0.2.45-py3.11.egg/langgraph/utils/runnable.py", line 184, in invoke ret = context.run(self.func, input, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/flow/src/graph.py", line 71, in run response = self.agent.invoke(messages) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/runnables/base.py", line 5354, in invoke return self.bound.invoke( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 286, in invoke self.generate_prompt( File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 786, in generate_prompt return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 643, in generate raise e File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 633, in generate self._generate_with_cache( File "/usr/local/lib/python3.11/site-packages/langchain_core-0.3.15-py3.11.egg/langchain_core/language_models/chat_models.py", line 851, in _generate_with_cache result = self._generate( ^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_openai-0.2.6-py3.11.egg/langchain_openai/chat_models/base.py", line 701, in _generate response = self.root_client.beta.chat.completions.parse(**payload) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai-1.54.3-py3.11.egg/openai/resources/beta/chat/completions.py", line 142, in parse _validate_input_tools(tools) File "/usr/local/lib/python3.11/site-packages/openai-1.54.3-py3.11.egg/openai/lib/_parsing/_completions.py", line 53, in validate_input_tools raise ValueError( ValueError: `get_city_population` is not strict. Only `strict` function tools can be auto-parsed ``` ### Description We are experiencing an issue with the AzureChatOpenAI binding tools into it. The odd behaviour is that the graph works in LangGraph Studio .. but onces it's triggered via FastAPI .. an error is thrown indicating "Only `strict` function tools can be auto-parsed". The issue seems to be related to the response_format. When declaring the LLM using AzureChatOpenAI, we are passing `"response_format": {"type": "json_object"}`. We have tried using the option `agent.bind(response_format={"type": "json_object"})` but it doesn't seems to have any effect, if we remove "JSON" word from the system prompt ... it won't give any error ... whereas it should respond `"'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'."`. So utilizing the "model_kwargs" within AzureChatOpenAI did activate the "response_format". **Troubleshooting 1**: 1. Set model_kwargs = {"response_format": {"type": "json_object"}} 2. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"} 3. Get ERROR, Only `strict` function tools can be auto-parsed **Troubleshooting 2**: 1. Set model_kwargs = {"response_format": {"type": "json_object"}} 2. Set agent.bind_tools(tools, strict = True) 4. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"} 5. Get ERROR, langgraph.errors.GraphRecursionError: Recursion limit of 25 reached without hitting a stop condition **Troubleshooting 3**: 1. Setting agent.bind(response_format={"type": "json_object"}) 2. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"} 3. WORKED .. but looks like response_format is not activated. 4. Removed "JSON" keyword from the system prompt 5. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"} 6. WORKED, didn't trigger any error .. so json_object feature not "used". Which is not a valid option. **Troubleshooting 4**: 1. Set model_kwargs = {"response_format": {"type": "json_object"}} 2. Removed "JSON" keyword from the system prompt 3. API call to the fastapi endpoint: POST /graph {"prompt":"Population of London?"} 4. Get ERROR (which is expected): `"'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'."`. Which means ... the "feature" about reponse_format is activated. And finally, running the initial version using `model_kwargs = {"response_format": {"type": "json_object"}}` and just using `self.agent = agent.bind_tools(tools)` WORKS perfect if the graph if triggered using LangGraph Studio. Within the stack error, it shows 'openai/resources/beta/chat/completions.py' ... so not sure if this could be related? Thanks for your help and support ;-) ### System Info python -m langchain_core.sys_info System Information ------------------ > OS: Linux > OS Version: #1 SMP Mon Aug 12 08:47:01 UTC 2024 > Python Version: 3.11.10 (main, Oct 19 2024, 18:56:55) [GCC 12.2.0] Package Information ------------------- > langchain_core: 0.3.15 > langchain: 0.3.7 > langchain_community: 0.3.5 > langsmith: 0.1.142 > langchain_openai: 0.2.6 > langchain_postgres: 0.0.12 > langchain_text_splitters: 0.3.2 > langgraph: 0.2.45 Optional packages not installed ------------------------------- > langserve Other Dependencies ------------------ > aiohttp: 3.11.0rc1 > async-timeout: Installed. No version info available. > dataclasses-json: 0.6.7 > httpx: 0.27.2 > httpx-sse: 0.4.0 > jsonpatch: 1.33 > langgraph-checkpoint: 2.0.2 > langgraph-sdk: 0.1.35 > numpy: 1.26.4 > openai: 1.54.3 > orjson: 3.10.11 > packaging: 24.1 > pgvector: 0.2.5 > psycopg: 3.2.3 > psycopg-pool: 3.2.3 > pydantic: 2.10.0b1 > pydantic-settings: 2.6.1 > PyYAML: 6.0.2 > requests: 2.32.3 > requests-toolbelt: 1.0.0 > sqlalchemy: 2.0.35 > SQLAlchemy: 2.0.35 > tenacity: 9.0.0 > tiktoken: 0.8.0 > typing-extensions: 4.12.2
yindo closed this issue 2026-02-20 17:36:00 -05:00
yindo commented 2026-02-20 17:36:00 -05:00
Author
Owner
Copy Link
Copy Source

@jaimeescano commented on GitHub (Jan 14, 2025):

For anyone having this issue. Turned out the root case (in our case) was the mismatch between the "tool" decorator from langchain which have the "input" schema in Pydantic model and the actual function definition with "default" paramterer settings.

Here sample:

class SearchInput(BaseModel):
    search_query: str = Field(description="should be a search query")
    top_k: int = Field(description="should be a number")
    language: str = Field(description="the product documentation language", default="all")

@tool("documentation", args_schema=SearchInput)
def documentation(search_query: str, top_k: int, language: str = "all") -> MutableSequence[Document]:
   .....

When the Model JSON response is enabled, then request can't have "default" values ... so removing from the function definition = "all", then it will complain. Changing to this .. make it throw:

class SearchInput(BaseModel):
    search_query: str = Field(description="should be a search query")
    top_k: int = Field(description="should be a number")
    language: str = Field(description="the product documentation language")

@tool("documentation", args_schema=SearchInput)
def documentation(search_query: str, top_k: int, language: str) -> MutableSequence[Document]:
@jaimeescano commented on GitHub (Jan 14, 2025): For anyone having this issue. Turned out the root case (in our case) was the mismatch between the "tool" decorator from langchain which have the "input" schema in Pydantic model and the actual function definition with "default" paramterer settings. Here sample: ```python class SearchInput(BaseModel): search_query: str = Field(description="should be a search query") top_k: int = Field(description="should be a number") language: str = Field(description="the product documentation language", default="all") @tool("documentation", args_schema=SearchInput) def documentation(search_query: str, top_k: int, language: str = "all") -> MutableSequence[Document]: ..... ``` When the Model JSON response is enabled, then request can't have "default" values ... so removing from the function definition `= "all"`, then it will complain. Changing to this .. make it throw: ```python class SearchInput(BaseModel): search_query: str = Field(description="should be a search query") top_k: int = Field(description="should be a number") language: str = Field(description="the product documentation language") @tool("documentation", args_schema=SearchInput) def documentation(search_query: str, top_k: int, language: str) -> MutableSequence[Document]: ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#305
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?