langchain-ai/langserve
1
0
Fork 0
mirror of https://github.com/langchain-ai/langserve.git synced 2026-07-21 00:55:22 -04:00
Code Issues 136 Packages Projects Releases 10 Wiki Activity

batch endpoint triggers PydanticUserError due to List[config] in dynamic model creation #248

New Issue
Open
opened 2026-02-16 00:19:47 -05:00 by yindo · 3 comments
yindo commented 2026-02-16 00:19:47 -05:00
Owner
Copy Link
Copy Source

Originally created by @k1dav on GitHub (May 19, 2025).

I'm encountering an error when enabling the batch endpoint using the add_routes function with a dynamically created batch request model.

Here is a minimal reproducible example:

from fastapi import APIRouter
from langserve import add_routes
from langchain_core.runnables import Runnable
from langchain_core.prompts import ChatPromptTemplate
from langchain_anthropic import ChatAnthropic

router = APIRouter()

model = ChatAnthropic(model="claude-3-haiku-20240307")
prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")

add_routes(
    router,
    prompt | model,
    path="/joke",
    enabled_endpoints=[
        "invoke",
        "stream",
        "stream_log",
        "stream_events",
        "playground",
        "feedback",
        "public_trace_link",
        "input_schema",
        "config_schema",
        "output_schema",
        "config_hashes",
        "batch"  # causes error
    ]
)

When the batch endpoint is included, I get the following error:

pydantic.errors.PydanticUserError: `TypeAdapter[typing.Annotated[langserve.validation.jokeBatchRequest, <class 'langserve.validation.jokeBatchRequest'>, Body(PydanticUndefined)]]` is not fully defined; you should define `typing.Annotated[...]` and all referenced types, then call `.rebuild()` on the instance.

I traced it to the following code in langserve/validation.py:

batch_request_type = create_model(
    f"{namespace}BatchRequest",
    inputs=(List[input_type], ...),
    config=(
        Union[config, List[config]],
        Field(
            default_factory=dict,
            description=(
                "Subset of RunnableConfig object in LangChain. Either specify one "
                "config for all inputs or a list of configs with one per input."
            ),
        ),
    ),
    kwargs=(
        dict,
        Field(
            default_factory=dict,
            description="Keyword arguments to the runnable. Currently ignored.",
        ),
    ),
)
batch_request_type.model_rebuild()

Temporary workaround:
If I remove the List[config] from the Union in the config field and just use config, everything works fine. So it seems the dynamic create_model doesn’t correctly handle List[config] in this context.

Thanks in advance!

Requirements

langserve==0.3.1
pydantic==2.11.4
pydantic-settings==2.9.1
pydantic_core==2.33.2
fastapi==0.115.11
Originally created by @k1dav on GitHub (May 19, 2025). I'm encountering an error when enabling the batch endpoint using the add_routes function with a dynamically created batch request model. Here is a minimal reproducible example: ```python from fastapi import APIRouter from langserve import add_routes from langchain_core.runnables import Runnable from langchain_core.prompts import ChatPromptTemplate from langchain_anthropic import ChatAnthropic router = APIRouter() model = ChatAnthropic(model="claude-3-haiku-20240307") prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}") add_routes( router, prompt | model, path="/joke", enabled_endpoints=[ "invoke", "stream", "stream_log", "stream_events", "playground", "feedback", "public_trace_link", "input_schema", "config_schema", "output_schema", "config_hashes", "batch" # causes error ] ) ``` When the batch endpoint is included, I get the following error: ```sh pydantic.errors.PydanticUserError: `TypeAdapter[typing.Annotated[langserve.validation.jokeBatchRequest, <class 'langserve.validation.jokeBatchRequest'>, Body(PydanticUndefined)]]` is not fully defined; you should define `typing.Annotated[...]` and all referenced types, then call `.rebuild()` on the instance. ``` I traced it to the following code in langserve/validation.py: ```python batch_request_type = create_model( f"{namespace}BatchRequest", inputs=(List[input_type], ...), config=( Union[config, List[config]], Field( default_factory=dict, description=( "Subset of RunnableConfig object in LangChain. Either specify one " "config for all inputs or a list of configs with one per input." ), ), ), kwargs=( dict, Field( default_factory=dict, description="Keyword arguments to the runnable. Currently ignored.", ), ), ) batch_request_type.model_rebuild() ``` Temporary workaround: If I remove the List[config] from the Union in the config field and just use config, everything works fine. So it seems the dynamic create_model doesn’t correctly handle List[config] in this context. Thanks in advance! --- Requirements ``` langserve==0.3.1 pydantic==2.11.4 pydantic-settings==2.9.1 pydantic_core==2.33.2 fastapi==0.115.11 ```
yindo commented 2026-02-16 00:19:48 -05:00
Author
Owner
Copy Link
Copy Source

@DhavalThkkar commented on GitHub (May 21, 2025):

All of a sudden this popped up

@DhavalThkkar commented on GitHub (May 21, 2025): All of a sudden this popped up
yindo commented 2026-02-16 00:19:48 -05:00
Author
Owner
Copy Link
Copy Source

@rsradulescu commented on GitHub (May 21, 2025):

We found similar error trying to show openapi docs and was fixed when we downgrade to pydantic==2.10.6

@rsradulescu commented on GitHub (May 21, 2025): We found similar error trying to show openapi docs and was fixed when we downgrade to pydantic==2.10.6
yindo commented 2026-02-16 00:19:48 -05:00
Author
Owner
Copy Link
Copy Source

@officialgoat commented on GitHub (Jul 7, 2025):

"LangServe>=0.3 fully supports Pydantic 2."

  • Well, no it doesnt.

This issue is open for a while now, and no reaction.

@officialgoat commented on GitHub (Jul 7, 2025): "[LangServe>=0.3 fully supports Pydantic 2.](https://python.langchain.com/v0.2/docs/langserve/#pydantic)" - Well, no it doesnt. This issue is open for a while now, and no reaction.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
duplicate
enhancement
help wanted
in-progress
investigate
not langserve
playground
pull-request
Mirrored from GitHub Pull Request
 question
stale
streaming
unable-to-reproduce
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langserve#248
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?