langchain-ai/langserve
1
0
Fork 0
mirror of https://github.com/langchain-ai/langserve.git synced 2026-07-16 09:34:30 -04:00
Code Issues 136 Packages Projects Releases 10 Wiki Activity

trying to set up a LangServe application with RunnableWithMessageHistory and a chat playground #223

New Issue
Closed
opened 2026-02-16 00:19:37 -05:00 by yindo · 4 comments
yindo commented 2026-02-16 00:19:37 -05:00
Owner
Copy Link
Copy Source

Originally created by @DeeraDigiSpoc on GitHub (Aug 23, 2024).

I'm trying to integrate both RunnableWithMessageHistory as chat with persistence playground_type='chat'

app = FastAPI()

llm_model_path = "model/llama-3-8b.gguf"

llm = ChatLlamaCpp(model_path=llm_model_path,
                   n_gpu_layers=-1,
                   n_ctx=512,
                   chat_format='chatml')

# Declare a chain
# prompt = ChatPromptTemplate.from_messages(
#     [
#         ("system", "You are a helpful AI assistant. Please provide a concise list of course with university that suits users question."),
#         MessagesPlaceholder(variable_name="history"),
#         ("human", "{human_input}"),
#     ]
# )

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful AI assistant that recommends courses along with universities to the students. Please provide a concise list of course with university that suits users question."),
        MessagesPlaceholder(variable_name="messages"),
    ]
)

chain = prompt | llm | StrOutputParser()

def get_session_history():
    return InMemoryChatMessageHistory()

class InputChat(BaseModel):
    """Input for the chat endpoint."""

    messages: List[Union[HumanMessage, AIMessage, SystemMessage]] = Field(
        ...,
        description="The chat messages representing the current conversation.",
    )

chain_with_history = RunnableWithMessageHistory(
    chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history"
).with_types(input_type=InputChat)

add_routes(
    app,
    chain_with_history,
    path="/chat",
    playground_type="chat"
)

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

TypeError: issubclass() arg 1 must be a class

But RunnableWithMessageHistory is a class right?

Is there a option to do this?

Originally created by @DeeraDigiSpoc on GitHub (Aug 23, 2024). I'm trying to integrate both `RunnableWithMessageHistory` as chat with persistence `playground_type='chat'` ``` app = FastAPI() llm_model_path = "model/llama-3-8b.gguf" llm = ChatLlamaCpp(model_path=llm_model_path, n_gpu_layers=-1, n_ctx=512, chat_format='chatml') # Declare a chain # prompt = ChatPromptTemplate.from_messages( # [ # ("system", "You are a helpful AI assistant. Please provide a concise list of course with university that suits users question."), # MessagesPlaceholder(variable_name="history"), # ("human", "{human_input}"), # ] # ) prompt = ChatPromptTemplate.from_messages( [ ("system", "You are a helpful AI assistant that recommends courses along with universities to the students. Please provide a concise list of course with university that suits users question."), MessagesPlaceholder(variable_name="messages"), ] ) chain = prompt | llm | StrOutputParser() def get_session_history(): return InMemoryChatMessageHistory() class InputChat(BaseModel): """Input for the chat endpoint.""" messages: List[Union[HumanMessage, AIMessage, SystemMessage]] = Field( ..., description="The chat messages representing the current conversation.", ) chain_with_history = RunnableWithMessageHistory( chain, get_session_history, input_messages_key="input", history_messages_key="chat_history" ).with_types(input_type=InputChat) add_routes( app, chain_with_history, path="/chat", playground_type="chat" ) if __name__ == "__main__": import uvicorn uvicorn.run(app, host="0.0.0.0", port=8000) ``` `TypeError: issubclass() arg 1 must be a class` But `RunnableWithMessageHistory` is a class right? Is there a option to do this?
yindo closed this issue 2026-02-16 00:19:38 -05:00
yindo commented 2026-02-16 00:19:38 -05:00
Author
Owner
Copy Link
Copy Source

@tylerwray commented on GitHub (Aug 24, 2024):

I'm having this same error. Code is very similar —

agent_executor = create_agent_executor(database=DATABASE)

def get_message_history(jwt):
    return RedisChatMessageHistory(url=REDIS_URL, ttl=600, session_id=jwt)


with_message_history = RunnableWithMessageHistory(
    agent_executor,
    get_message_history,
    input_messages_key="input",
    history_messages_key="history",
    history_factory_config=[
        ConfigurableFieldSpec(
            id="jwt",
            annotation=str,
            name="User JWT",
            description="Unique session identifier for the user.",
            default="",
            is_shared=True,
        ),
    ],
)

add_routes(
    app,
    with_message_history,
    per_req_config_modifier=_per_req_config_modifier,
    path="/agent",
)

I suspect something with pydantic versions. I'm using pydantic = "^2.8.2" in my pyproject.toml.

I'm using an app generated with the langchain-cli last week.

@tylerwray commented on GitHub (Aug 24, 2024): I'm having this same error. Code is very similar — ```python agent_executor = create_agent_executor(database=DATABASE) def get_message_history(jwt): return RedisChatMessageHistory(url=REDIS_URL, ttl=600, session_id=jwt) with_message_history = RunnableWithMessageHistory( agent_executor, get_message_history, input_messages_key="input", history_messages_key="history", history_factory_config=[ ConfigurableFieldSpec( id="jwt", annotation=str, name="User JWT", description="Unique session identifier for the user.", default="", is_shared=True, ), ], ) add_routes( app, with_message_history, per_req_config_modifier=_per_req_config_modifier, path="/agent", ) ``` I suspect something with pydantic versions. I'm using `pydantic = "^2.8.2"` in my `pyproject.toml`. I'm using an app generated with the langchain-cli last week.
yindo commented 2026-02-16 00:19:38 -05:00
Author
Owner
Copy Link
Copy Source

@DeeraDigiSpoc commented on GitHub (Aug 26, 2024):

I'm trying to integrate both RunnableWithMessageHistory as chat with persistence playground_type='chat'

app = FastAPI()

llm_model_path = "model/llama-3-8b.gguf"

llm = ChatLlamaCpp(model_path=llm_model_path,
                   n_gpu_layers=-1,
                   n_ctx=512,
                   chat_format='chatml')

# Declare a chain
# prompt = ChatPromptTemplate.from_messages(
#     [
#         ("system", "You are a helpful AI assistant. Please provide a concise list of course with university that suits users question."),
#         MessagesPlaceholder(variable_name="history"),
#         ("human", "{human_input}"),
#     ]
# )

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful AI assistant that recommends courses along with universities to the students. Please provide a concise list of course with university that suits users question."),
        MessagesPlaceholder(variable_name="messages"),
    ]
)

chain = prompt | llm | StrOutputParser()

def get_session_history():
    return InMemoryChatMessageHistory()

class InputChat(BaseModel):
    """Input for the chat endpoint."""

    messages: List[Union[HumanMessage, AIMessage, SystemMessage]] = Field(
        ...,
        description="The chat messages representing the current conversation.",
    )

chain_with_history = RunnableWithMessageHistory(
    chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history"
).with_types(input_type=InputChat)

add_routes(
    app,
    chain_with_history,
    path="/chat",
    playground_type="chat"
)

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

TypeError: issubclass() arg 1 must be a class

But RunnableWithMessageHistory is a class right?

Is there a option to do this?

My Configs are:
pydantic-2.8.2
pydantic-core- 2.20.1

@DeeraDigiSpoc commented on GitHub (Aug 26, 2024): > I'm trying to integrate both `RunnableWithMessageHistory` as chat with persistence `playground_type='chat'` > > ``` > app = FastAPI() > > llm_model_path = "model/llama-3-8b.gguf" > > llm = ChatLlamaCpp(model_path=llm_model_path, > n_gpu_layers=-1, > n_ctx=512, > chat_format='chatml') > > # Declare a chain > # prompt = ChatPromptTemplate.from_messages( > # [ > # ("system", "You are a helpful AI assistant. Please provide a concise list of course with university that suits users question."), > # MessagesPlaceholder(variable_name="history"), > # ("human", "{human_input}"), > # ] > # ) > > prompt = ChatPromptTemplate.from_messages( > [ > ("system", "You are a helpful AI assistant that recommends courses along with universities to the students. Please provide a concise list of course with university that suits users question."), > MessagesPlaceholder(variable_name="messages"), > ] > ) > > chain = prompt | llm | StrOutputParser() > > def get_session_history(): > return InMemoryChatMessageHistory() > > class InputChat(BaseModel): > """Input for the chat endpoint.""" > > messages: List[Union[HumanMessage, AIMessage, SystemMessage]] = Field( > ..., > description="The chat messages representing the current conversation.", > ) > > chain_with_history = RunnableWithMessageHistory( > chain, > get_session_history, > input_messages_key="input", > history_messages_key="chat_history" > ).with_types(input_type=InputChat) > > add_routes( > app, > chain_with_history, > path="/chat", > playground_type="chat" > ) > > if __name__ == "__main__": > import uvicorn > uvicorn.run(app, host="0.0.0.0", port=8000) > ``` > > `TypeError: issubclass() arg 1 must be a class` > > But `RunnableWithMessageHistory` is a class right? > > Is there a option to do this? My Configs are: pydantic-2.8.2 pydantic-core- 2.20.1
yindo commented 2026-02-16 00:19:39 -05:00
Author
Owner
Copy Link
Copy Source

@Abd-elr4hman commented on GitHub (Aug 26, 2024):

This issue is in RunnableWithMessageHistory get_output_schema, I was able to work around it by defining my chain as follow:

from langchain_core.output_parsers.string import StrOutputParser

json_parser = StrOutputParser()

chain_with_parser = (
    chain_with_history |
    json_parser
)

then passing chain_with_parser to add_routes:

# Edit this to add the chain you want to add
add_routes(app, chain_with_parser)

not sure exactly how to solve the issue in RunnableWithMessageHistory tho.

@Abd-elr4hman commented on GitHub (Aug 26, 2024): This issue is in RunnableWithMessageHistory get_output_schema, I was able to work around it by defining my chain as follow: ``` from langchain_core.output_parsers.string import StrOutputParser json_parser = StrOutputParser() chain_with_parser = ( chain_with_history | json_parser ) ``` then passing chain_with_parser to add_routes: ``` # Edit this to add the chain you want to add add_routes(app, chain_with_parser) ``` not sure exactly how to solve the issue in RunnableWithMessageHistory tho.
yindo commented 2026-02-16 00:19:39 -05:00
Author
Owner
Copy Link
Copy Source

@DeeraDigiSpoc commented on GitHub (Aug 28, 2024):

This issue is in RunnableWithMessageHistory get_output_schema, I was able to work around it by defining my chain as follow:

from langchain_core.output_parsers.string import StrOutputParser

json_parser = StrOutputParser()

chain_with_parser = (
    chain_with_history |
    json_parser
)

then passing chain_with_parser to add_routes:

# Edit this to add the chain you want to add
add_routes(app, chain_with_parser)

not sure exactly how to solve the issue in RunnableWithMessageHistory tho.

Yes, It's solved!!

Still, The chat playground is only supported for chains that take one of the following as input:

  • a dict with a single key containing a list of messages
  • a dict with two keys: one a string input, one an list of messages

and which return either an AIMessage or a string.

You can test this chain in the default LangServe playground instead.

To use the default playground, set playground_type="default" when adding the route in your backend.

@DeeraDigiSpoc commented on GitHub (Aug 28, 2024): > This issue is in RunnableWithMessageHistory get_output_schema, I was able to work around it by defining my chain as follow: > > ``` > from langchain_core.output_parsers.string import StrOutputParser > > json_parser = StrOutputParser() > > chain_with_parser = ( > chain_with_history | > json_parser > ) > ``` > > then passing chain_with_parser to add_routes: > > ``` > # Edit this to add the chain you want to add > add_routes(app, chain_with_parser) > ``` > > not sure exactly how to solve the issue in RunnableWithMessageHistory tho. Yes, It's solved!! Still, The chat playground is only supported for chains that take one of the following as input: - a dict with a single key containing a list of messages - a dict with two keys: one a string input, one an list of messages and which return either an AIMessage or a string. You can test this chain in the default LangServe playground instead. To use the default playground, set playground_type="default" when adding the route in your backend.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
duplicate
enhancement
help wanted
in-progress
investigate
not langserve
playground
pull-request
Mirrored from GitHub Pull Request
 question
stale
streaming
unable-to-reproduce
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langserve#223
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?