langchain-ai/langserve
1
0
Fork 0
mirror of https://github.com/langchain-ai/langserve.git synced 2026-07-18 10:54:29 -04:00
Code Issues 136 Packages Projects Releases 10 Wiki Activity

Invoke through forwarding message #121

New Issue
Closed
opened 2026-02-16 00:18:53 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 00:18:53 -05:00
Owner
Copy Link
Copy Source

Originally created by @ezelanza on GitHub (Feb 13, 2024).

Hi

My scenario is FRONT_END(:5005/openaiapi)--> BACK_END(:8000/api_openai )-INVOKE -->OPENAI (API)

In my front end, I'm using Fastapi listening to /api_openai, to forward the request to openai wrapper

FRONT_END

from pydantic import BaseModel

class Data(BaseModel):
    question: str

openai_llm = RemoteRunnable("http://localhost:8000/openai_api")
@app.post("/openaiapi")
async def process_text_data(question: Data):
    try:
        print("Entering try block...")
        user_question= str(question.question)
        
        prompt = ChatPromptTemplate.from_messages(
            [
                (
                    "system",
                    "You are a highly educated person who loves to use big words. "
                    + "You are also concise. Never answer in more than three sentences.",
                ),
                ("human", user_question),
            ]       
        ).format_messages()     
        
        # Pass the extracted question
        result=openai_llm.invoke(prompt)

        print(result.content)
        
        return result.content

BACKEND*

model = ChatOpenAI(openai_api_key="sk-xx")
#prompt = ChatPromptTemplate.from_template("Answer user question please :{question}")

add_routes(
    app,
    model,
    path="/openai_api",
)

The funny thing is that the request works when I run on my Jupyter notebook

from langserve import RemoteRunnable
from langchain.prompts.chat import ChatPromptTemplate

openai_llm = RemoteRunnable("http://localhost:8000/openai_api")
question = ("Tell me about k8s")
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a highly educated person who loves to use big words. "
            + "You are also concise. Never answer in more than three sentences.",
        ),
        ("human", question),
    ]
).format_messages()
save=openai_llm.invoke(prompt)
print(save.content)

K8s, short for Kubernetes, is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It provides a highly scalable and flexible infrastructure for running and managing applications in a distributed environment.

BUT It doesn't work when I curl the front end

curl -X POST -H "Content-Type: application/json" -d '{"question": "Tell me about kubernetes"}' http://localhost:5005/api_openai
{"detail":"Error: [Errno 99] Cannot assign requested address"}%

Doing some troubleshooting I found that the problem is in this part of the front_end code. The front end is a fast api implementation intended to fw requests to .invoke. Which works when is directly accessed (Jupyter) but it doesn't work when is curl by the fe

openai_llm.invoke(prompt)

I'm using the same envs and nothing is different. Is there anything I should be aware when forwarding requests? Does .invoke need any additional detail on the request?

Originally created by @ezelanza on GitHub (Feb 13, 2024). Hi My scenario is FRONT_END(:5005/openaiapi)--> BACK_END(:8000/api_openai )-INVOKE -->OPENAI (API) In my front end, I'm using Fastapi listening to /api_openai, to forward the request to openai wrapper *********FRONT_END********* ``` from pydantic import BaseModel class Data(BaseModel): question: str openai_llm = RemoteRunnable("http://localhost:8000/openai_api") @app.post("/openaiapi") async def process_text_data(question: Data): try: print("Entering try block...") user_question= str(question.question) prompt = ChatPromptTemplate.from_messages( [ ( "system", "You are a highly educated person who loves to use big words. " + "You are also concise. Never answer in more than three sentences.", ), ("human", user_question), ] ).format_messages() # Pass the extracted question result=openai_llm.invoke(prompt) print(result.content) return result.content ``` *********BACKEND********** ``` model = ChatOpenAI(openai_api_key="sk-xx") #prompt = ChatPromptTemplate.from_template("Answer user question please :{question}") add_routes( app, model, path="/openai_api", ) ``` The funny thing is that the request works when I run on my Jupyter notebook ``` from langserve import RemoteRunnable from langchain.prompts.chat import ChatPromptTemplate openai_llm = RemoteRunnable("http://localhost:8000/openai_api") question = ("Tell me about k8s") prompt = ChatPromptTemplate.from_messages( [ ( "system", "You are a highly educated person who loves to use big words. " + "You are also concise. Never answer in more than three sentences.", ), ("human", question), ] ).format_messages() save=openai_llm.invoke(prompt) print(save.content) ``` ```K8s, short for Kubernetes, is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It provides a highly scalable and flexible infrastructure for running and managing applications in a distributed environment.``` BUT It doesn't work when I curl the front end ``` curl -X POST -H "Content-Type: application/json" -d '{"question": "Tell me about kubernetes"}' http://localhost:5005/api_openai {"detail":"Error: [Errno 99] Cannot assign requested address"}% ``` Doing some troubleshooting I found that the problem is in this part of the front_end code. The front end is a fast api implementation intended to fw requests to .invoke. Which works when is directly accessed (Jupyter) but it doesn't work when is curl by the fe openai_llm.invoke(prompt) I'm using the same envs and nothing is different. Is there anything I should be aware when forwarding requests? Does .invoke need any additional detail on the request?
yindo closed this issue 2026-02-16 00:18:53 -05:00
yindo commented 2026-02-16 00:18:53 -05:00
Author
Owner
Copy Link
Copy Source

@eyurtsev commented on GitHub (Feb 14, 2024):

curl -X POST -H "Content-Type: application/json" -d '{"question": "Tell me about kubernetes"}' http://localhost:5005/api_openai
{"detail":"Error: [Errno 99] Cannot assign requested address"}%

Try looking for the error message online -- I don't think this is LangServe related

e.g.,

https://support.prodi.gy/t/errno-99-error-while-attempting-to-bind-on-address-1-8080-0-0-cannot-assign-requested-address/3110

@eyurtsev commented on GitHub (Feb 14, 2024): ``` curl -X POST -H "Content-Type: application/json" -d '{"question": "Tell me about kubernetes"}' http://localhost:5005/api_openai {"detail":"Error: [Errno 99] Cannot assign requested address"}% ``` Try looking for the error message online -- I don't think this is LangServe related e.g., https://support.prodi.gy/t/errno-99-error-while-attempting-to-bind-on-address-1-8080-0-0-cannot-assign-requested-address/3110
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
duplicate
enhancement
help wanted
in-progress
investigate
not langserve
playground
pull-request
Mirrored from GitHub Pull Request
 question
stale
streaming
unable-to-reproduce
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langserve#121
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?