Release 0.0.45 (#498 )

Improvements for playground
Adds playground chat widget for message list inputs, additional global callback for chunks (#489 )
2026-07-19 13:54:13 -04:00 · 2024-02-28 14:03:18 -05:00 · 2024-02-28 13:39:54 -05:00 · 2024-02-28 12:19:17 -05:00 · 2024-02-28 12:12:24 -05:00 · 2024-02-28 12:11:57 -05:00
39 changed files with 5551 additions and 437 deletions
@@ -1,4 +1,4 @@
 {
-  "contributors": ["eyurtsev", "hwchase17", "nfcampos", "efriis", "jacoblee93", "dqbd", "kreneskyp", "adarsh-jha-dev", "harris", "baskaryan", "hinthornw", "bracesproul", "jakerachleff", "craigsdennis", "anhi", "169", "LarchLiu", "PaulLockett", "RCMatthias", "jwynia", "majiayu000", "mpskex", "shivachittamuru", "sinashaloudegi", "sowsan", "akira", "lucianotonet", "JGalego"],
+  "contributors": ["eyurtsev", "hwchase17", "nfcampos", "efriis", "jacoblee93", "dqbd", "kreneskyp", "adarsh-jha-dev", "harris", "baskaryan", "hinthornw", "bracesproul", "jakerachleff", "craigsdennis", "anhi", "169", "LarchLiu", "PaulLockett", "RCMatthias", "jwynia", "majiayu000", "mpskex", "shivachittamuru", "sinashaloudegi", "sowsan", "akira", "lucianotonet", "JGalego", "nat-n", "dirien"],
  "message": "Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the username {{usersWithoutCLA}} on file. In order for us to review and merge your code, please complete the Individual Contributor License Agreement here https://forms.gle/AQFbtkWRoHXUgipM6 .\n\nThis process is done manually on our side, so after signing the form one of the maintainers will add you to the contributors list.\n\nFor more details about why we have a CLA and other contribution guidelines please see: https://github.com/langchain-ai/langserve/blob/main/CONTRIBUTING.md."
 }
@@ -6,13 +6,12 @@
 [![](https://dcbadge.vercel.app/api/server/6adMQxSpJS?compact=true&style=flat)](https://discord.com/channels/1038097195422978059/1170024642245832774)

 🚩 We will be releasing a hosted version of LangServe for one-click deployments of
-LangChain
-applications. [Sign up here](https://airtable.com/app0hN6sd93QcKubv/shrAjst60xXa6quV2)
+LangChain applications. [Sign up here](https://airtable.com/app0hN6sd93QcKubv/shrAjst60xXa6quV2)
 to get on the waitlist.

 ## Overview

-`LangServe` helps developers
+[LangServe](https://github.com/langchain-ai/langserve) helps developers
 deploy `LangChain` [runnables and chains](https://python.langchain.com/docs/expression_language/)
 as a REST API.

@@ -21,8 +20,8 @@ uses [pydantic](https://docs.pydantic.dev/latest/) for data validation.

 In addition, it provides a client that can be used to call into runnables deployed on a
 server.
-A javascript client is available
-in [LangChainJS](https://js.langchain.com/docs/api/runnables_remote/classes/RemoteRunnable).
+A JavaScript client is available
+in [LangChain.js](https://js.langchain.com/docs/ecosystem/langserve).

 ## Features

@@ -33,6 +32,7 @@ in [LangChainJS](https://js.langchain.com/docs/api/runnables_remote/classes/Remo
  concurrent requests on a single server
 - `/stream_log/` endpoint for streaming all (or some) intermediate steps from your
  chain/agent
+- **new** as of 0.0.40, supports `astream_events` to make it easier to stream without needing to parse the output of `stream_log`.
 - Playground page at `/playground/` with streaming output and intermediate steps
 - Built-in (optional) tracing to [LangSmith](https://www.langchain.com/langsmith), just
  add your API key (see [Instructions](https://docs.smith.langchain.com/))
@@ -53,7 +53,7 @@ in [LangChainJS](https://js.langchain.com/docs/api/runnables_remote/classes/Remo

 We will be releasing a hosted version of LangServe for one-click deployments of
 LangChain
-applications. [Sign up here](https://airtable.com/app0hN6sd93QcKubv/shrAjst60xXa6quV2)
+applications. [Sign up here](https://airtable.com/apppQ9p5XuujRl3wJ/shrABpHWdxry8Bacm)
 to get on the waitlist.

 ## Security
@@ -99,18 +99,19 @@ directory.
 | **LLMs** Minimal example that reserves OpenAI and Anthropic chat models. Uses async, supports batching and streaming.                                                                                                                                              | [server](https://github.com/langchain-ai/langserve/tree/main/examples/llm/server.py), [client](https://github.com/langchain-ai/langserve/blob/main/examples/llm/client.ipynb)                                                       | 
 | **Retriever** Simple server that exposes a retriever as a runnable.                                                                                                                                                                                                | [server](https://github.com/langchain-ai/langserve/tree/main/examples/retrieval/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/retrieval/client.ipynb)                                           | 
 | **Conversational Retriever** A [Conversational Retriever](https://python.langchain.com/docs/expression_language/cookbook/retrieval#conversational-retrieval-chain) exposed via LangServe                                                                           | [server](https://github.com/langchain-ai/langserve/tree/main/examples/conversational_retrieval_chain/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/conversational_retrieval_chain/client.ipynb) | 
-| **Agent** Implementation of an [Agent based on OpenAI tools](https://python.langchain.com/docs/modules/agents/agent_types/openai_functions_agent)                                                                                                                  | [server](https://github.com/langchain-ai/langserve/tree/main/examples/agent/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/agent/client.ipynb)                                                   | 
+| **Agent** without **conversation history** based on [OpenAI tools](https://python.langchain.com/docs/modules/agents/agent_types/openai_functions_agent)                                                                                             | [server](https://github.com/langchain-ai/langserve/tree/main/examples/agent/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/agent/client.ipynb)                                                   | 
+| **Agent** with **conversation history** based on [OpenAI tools](https://python.langchain.com/docs/modules/agents/agent_types/openai_functions_agent)                                                                                                               | [server](https://github.com/langchain-ai/langserve/blob/main/examples/agent_with_history/server.py), [client](https://github.com/langchain-ai/langserve/blob/main/examples/agent_with_history/client.ipynb)                         | 
 | [RunnableWithMessageHistory](https://python.langchain.com/docs/expression_language/how_to/message_history) to implement chat persisted on backend, keyed off a `session_id` supplied by client.                                                                    | [server](https://github.com/langchain-ai/langserve/tree/main/examples/chat_with_persistence/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/chat_with_persistence/client.ipynb)                   |
 | [RunnableWithMessageHistory](https://python.langchain.com/docs/expression_language/how_to/message_history) to implement chat persisted on backend, keyed off a `conversation_id` supplied by client, and `user_id` (see Auth for implementing `user_id` properly). | [server](https://github.com/langchain-ai/langserve/tree/main/examples/chat_with_persistence_and_user/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/chat_with_persistence_and_user/client.ipynb) | 
 | [Configurable Runnable](https://python.langchain.com/docs/expression_language/how_to/configure) to create a retriever that supports run time configuration of the index name.                                                                                      | [server](https://github.com/langchain-ai/langserve/tree/main/examples/configurable_retrieval/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/configurable_retrieval/client.ipynb)                 | 
 | [Configurable Runnable](https://python.langchain.com/docs/expression_language/how_to/configure) that shows configurable fields and configurable alternatives.                                                                                                      | [server](https://github.com/langchain-ai/langserve/tree/main/examples/configurable_chain/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/configurable_chain/client.ipynb)                         | 
-| **APIHandler** Shows how to use `APIHandler` instead of `add_routes`. This provides more flexibility for developers to define endpoints. Works well with all FastAPI patterns, but takes a bit more effort.                                                        | [server](https://github.com/langchain-ai/langserve/tree/main/examples/api_handler_examples/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/api_handler_examples/client.ipynb)                     |
+| **APIHandler** Shows how to use `APIHandler` instead of `add_routes`. This provides more flexibility for developers to define endpoints. Works well with all FastAPI patterns, but takes a bit more effort.                                                        | [server](https://github.com/langchain-ai/langserve/tree/main/examples/api_handler_examples/server.py)                                                                                                                               |
 | **LCEL Example** Example that uses LCEL to manipulate a dictionary input.                                                                                                                                                                                          | [server](https://github.com/langchain-ai/langserve/tree/main/examples/passthrough_dict/client.ipynb), [client](https://github.com/langchain-ai/langserve/tree/main/examples/passthrough_dict/client.ipynb)                          | 
 | **Auth** with `add_routes`: Simple authentication that can be applied across all endpoints associated with app. (Not useful on its own for implementing per user logic.)                                                                                           | [server](https://github.com/langchain-ai/langserve/tree/main/examples/auth/global_deps/server.py)                                                                                                                                   | 
 | **Auth** with `add_routes`: Simple authentication mechanism based on path dependencies. (No useful on its own for implementing per user logic.)                                                                                                                    | [server](https://github.com/langchain-ai/langserve/tree/main/examples/auth/path_dependencies/server.py)                                                                                                                             | 
 | **Auth** with `add_routes`: Implement per user logic and auth for endpoints that use per request config modifier. (**Note**: At the moment, does not integrate with OpenAPI docs.)                                                                                 | [server](https://github.com/langchain-ai/langserve/tree/main/examples/auth/per_req_config_modifier/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/auth/per_req_config_modifier/client.ipynb)     | 
 | **Auth** with `APIHandler`: Implement per user logic and auth that shows how to search only within user owned documents.                                                                                                                                           | [server](https://github.com/langchain-ai/langserve/tree/main/examples/auth/api_handler/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/auth/api_handler/client.ipynb)                             | 
-| **Widgets** Different widgets that can be used with playground (file upload and chat)                                                                                                                                                                              | [server](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/client.ipynb)                                               | 
+| **Widgets** Different widgets that can be used with playground (file upload and chat)                                                                                                                                                                              | [server](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/chat/tuples/server.py)                                                                                                                                            | 
 | **Widgets** File upload widget used for LangServe playground.                                                                                                                                                                                                      | [server](https://github.com/langchain-ai/langserve/tree/main/examples/file_processing/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/file_processing/client.ipynb)                               | 

 ## Sample Application
@@ -160,6 +161,23 @@ if __name__ == "__main__":
    uvicorn.run(app, host="localhost", port=8000)
 ```

+If you intend to call your endpoint from the browser, you will also need to set CORS headers.
+You can use FastAPI's built-in middleware for that:
+
+```python
+from fastapi.middleware.cors import CORSMiddleware
+
+# Set all CORS enabled origins
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+    expose_headers=["*"],
+)
+```
+
 ### Docs

 If you've deployed the server above, you can view the generated OpenAPI docs using:
@@ -222,7 +240,7 @@ chain.batch([{"topic": "parrots"}, {"topic": "cats"}])
 In TypeScript (requires LangChain.js version 0.0.166 or later):

 ```typescript
-import {RemoteRunnable} from "langchain/runnables/remote";
+import { RemoteRunnable } from "@langchain/core/runnables/remote";

 const chain = new RemoteRunnable({
    url: `http://localhost:8000/joke/`,
@@ -276,6 +294,8 @@ adds of these endpoints to the server:
 - `POST /my_runnable/stream` - invoke on a single input and stream the output
 - `POST /my_runnable/stream_log` - invoke on a single input and stream the output,
  including output of intermediate steps as it's generated
+- `POST /my_runnable/astream_events` - invoke on a single input and stream events as they are generated,
+ including from intermediate steps.
 - `GET /my_runnable/input_schema` - json schema for input to the runnable
 - `GET /my_runnable/output_schema` - json schema for output of the runnable
 - `GET /my_runnable/config_schema` - json schema for config of the runnable
@@ -352,6 +372,27 @@ You can deploy to GCP Cloud Run using the following command:
 gcloud run deploy [your-service-name] --source . --port 8001 --allow-unauthenticated --region us-central1 --set-env-vars=OPENAI_API_KEY=your_key
 ```

+### Deploy using Infrastructure as Code
+
+#### Pulumi
+
+You can deploy your LangServe server with [Pulumi](https://www.pulumi.com/) using your preferred general purpose language. Below are some quickstart 
+examples for deploying LangServe to different cloud providers.
+
+These examples are a good starting point for your own infrastructure as code (IaC) projects. You can easily modify them to suit your needs.
+
+
+| Cloud | Language   | Repository                                          | Quickstart                                                                                                                                  |
+|-------|------------|-----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
+| AWS   | dotnet     | https://github.com/pulumi/examples/aws-cs-langserve | [![Deploy](https://get.pulumi.com/new/button.svg)](https://app.pulumi.com/new?template=https://github.com/pulumi/examples/aws-cs-langserve) |
+| AWS   | golang     | https://github.com/pulumi/examples/aws-go-langserve | [![Deploy](https://get.pulumi.com/new/button.svg)](https://app.pulumi.com/new?template=https://github.com/pulumi/examples/aws-go-langserve) |
+| AWS   | python     | https://github.com/pulumi/examples/aws-py-langserve | [![Deploy](https://get.pulumi.com/new/button.svg)](https://app.pulumi.com/new?template=https://github.com/pulumi/examples/aws-py-langserve) |
+| AWS   | typescript | https://github.com/pulumi/examples/aws-ts-langserve | [![Deploy](https://get.pulumi.com/new/button.svg)](https://app.pulumi.com/new?template=https://github.com/pulumi/examples/aws-ts-langserve) |
+| AWS   | javascript | https://github.com/pulumi/examples/aws-js-langserve | [![Deploy](https://get.pulumi.com/new/button.svg)](https://app.pulumi.com/new?template=https://github.com/pulumi/examples/aws-js-langserve) |
+
+
+
+
 ### Community Contributed

 #### Deploy to Railway
@@ -476,7 +517,7 @@ def func(x: Any) -> int:


 runnable = RunnableLambda(func).with_types(
-    input_schema=int,
+    input_type=int,
 )

 add_routes(app, runnable)
@@ -515,8 +556,8 @@ def func(foo: Foo) -> int:
 # Note that the input and output type are automatically inferred!
 # You do not need to specify them.
 # runnable = RunnableLambda(func).with_types( # <-- Not needed in this case
-#     input_schema=Foo,
-#     output_schema=int,
+#     input_type=Foo,
+#     output_type=int,
 #
 add_routes(app, RunnableLambda(func), path="/foo")
 ```
@@ -529,7 +570,7 @@ Here are a few examples:

 | Description                                                                                                                                                                                                                                                        | Links                                                                                                                                                                                                                                |
 |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **Widgets** Different widgets that can be used with playground (file upload and chat)                                                                                                                                                                              | [server](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/client.ipynb)                                               | 
+| **Widgets** Different widgets that can be used with playground (file upload and chat)                                                                                                                                                                              | [server](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/chat/tuples/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/client.ipynb)                                               | 
 | **Widgets** File upload widget used for LangServe playground.                                                                                                                                                                                                      | [server](https://github.com/langchain-ai/langserve/tree/main/examples/file_processing/server.py), [client](https://github.com/langchain-ai/langserve/tree/main/examples/file_processing/client.ipynb)                               | 

 #### Schema
@@ -601,7 +642,7 @@ Example widget:
 ### Chat Widget

 Look
-at [widget example](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/server.py).
+at the [widget example](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/chat/tuples/server.py).

 To define a chat widget, make sure that you pass "type": "chat".

@@ -613,7 +654,6 @@ To define a chat widget, make sure that you pass "type": "chat".
 Here's a snippet:

 ```python
-
 class ChatHistory(CustomUserType):
    chat_history: List[Tuple[str, str]] = Field(
        ...,
@@ -652,21 +692,53 @@ Example widget:
 <img src="https://github.com/langchain-ai/langserve/assets/3205522/a71ff37b-a6a9-4857-a376-cf27c41d3ca4" width="50%"/>
 </p>

+You can also specify a list of messages as your a parameter directly, as shown in this snippet:
+
+```python
+prompt = ChatPromptTemplate.from_messages(
+    [
+        ("system", "You are a helpful assisstant named Cob."),
+        MessagesPlaceholder(variable_name="messages"),
+    ]
+)
+
+chain = prompt | ChatAnthropic(model="claude-2")
+
+
+class MessageListInput(BaseModel):
+    """Input for the chat endpoint."""
+    messages: List[Union[HumanMessage, AIMessage]] = Field(
+        ...,
+        description="The chat messages representing the current conversation.",
+        extra={"widget": {"type": "chat", "input": "messages"}},
+    )
+
+
+add_routes(
+    app,
+    chain.with_types(input_type=MessageListInput),
+    path="/chat",
+)
+```
+
+See [this sample file](https://github.com/langchain-ai/langserve/tree/main/examples/widgets/chat/message_list/server.py) for an example.
+
 ### Enabling / Disabling Endpoints (LangServe >=0.0.33)

-You can enable / disable which endpoints are exposed. Use `enabled_endpoints` if you
-want to make sure to never get a new endpoint when upgrading langserve to a newer
+You can enable / disable which endpoints are exposed when adding routes for a given chain.
+
+Use `enabled_endpoints` if you want to make sure to never get a new endpoint when upgrading langserve to a newer
 verison.

 Enable: The code below will only enable `invoke`, `batch` and the
 corresponding `config_hash` endpoint variants.

 ```python
-add_routes(app, chain, enabled_endpoints=["invoke", "batch", "config_hashes"])
+add_routes(app, chain, enabled_endpoints=["invoke", "batch", "config_hashes"], path="/mychain")
 ```

 Disable: The code below will disable the playground for the chain

 ```python
-add_routes(app, chain, disabled_endpoints=["playground"])
+add_routes(app, chain, disabled_endpoints=["playground"], path="/mychain")
 ```
@@ -27,7 +27,8 @@
     "data": {
      "text/plain": [
       "{'output': {'output': 'Eugene thinks that cats like fish.'},\n",
-       " 'callback_events': []}"
+       " 'callback_events': [],\n",
+       " 'metadata': {'run_id': 'f16d95e5-dd8f-48d1-8668-4b33a54023fb'}}"
      ]
     },
     "execution_count": 1,
@@ -95,7 +96,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 4,
   "metadata": {
    "tags": []
   },
@@ -106,7 +107,7 @@
       "{'output': 'Eugene thinks that cats like fish.'}"
      ]
     },
-     "execution_count": 6,
+     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
@@ -114,6 +115,816 @@
   "source": [
    "remote_runnable.invoke({\"input\": \"what does eugene think of cats?\"})"
   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Stream\n",
+    "\n",
+    "Please note that streaming alternates between actions and observations. It does not stream individual tokens! If you need to stream individual tokens you will need to use astream_log!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "--\n",
+      "{'actions': [AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])], 'messages': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]}\n",
+      "--\n",
+      "{'steps': [{'action': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]), 'observation': [Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]}], 'messages': [FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')]}\n",
+      "--\n",
+      "{'output': \"Eugene thinks that cats like fish. Now let me tell you a story about that thought.\\n\\nOnce upon a time, in a small village, there lived a mischievous cat named Whiskers. Whiskers was known for his love for fish. Every day, he would venture out to the nearby river in search of his favorite food.\\n\\nOne sunny day, Whiskers set out on his usual fish-hunting expedition. As he approached the riverbank, he noticed a group of fishermen casting their nets into the water. Whiskers couldn't resist the temptation and decided to join in on the action.\\n\\nWith his agile paws, Whiskers skillfully maneuvered through the fishermen, strategically positioning himself to snatch the fish caught in their nets. The fishermen were amazed by Whiskers' quick reflexes and couldn't help but laugh at the sight of a cat fishing alongside them.\\n\\nWhiskers continued his fish-stealing escapades for several days, becoming somewhat of a local legend. People from neighboring villages would come to witness the incredible sight of a cat outsmarting seasoned fishermen.\\n\\nOne day, as Whiskers was enjoying his stolen fish by the river, he noticed a stray dog named Rover approaching him. Rover had heard about Whiskers' fishing talents and was intrigued by the cat's abilities.\\n\\nCurious, Whiskers decided to share his secret with Rover. He taught him the art of stealth and how to snatch fish from the nets without being noticed. Rover, being a quick learner, soon became Whiskers' partner in crime.\\n\\nTogether, Whiskers and Rover formed an unbeatable duo, leaving the fishermen puzzled and amazed at the disappearing fish. The villagers, entertained by their antics, started leaving fish out for Whiskers and Rover as a token of appreciation.\\n\\nAnd so, Whiskers and Rover continued their fish-stealing adventures, bringing joy and laughter to the village. Eugene's thought about cats and their love for fish certainly came to life in the mischievous and clever Whiskers, who proved that cats truly have a special affinity for fish.\", 'messages': [AIMessage(content=\"Eugene thinks that cats like fish. Now let me tell you a story about that thought.\\n\\nOnce upon a time, in a small village, there lived a mischievous cat named Whiskers. Whiskers was known for his love for fish. Every day, he would venture out to the nearby river in search of his favorite food.\\n\\nOne sunny day, Whiskers set out on his usual fish-hunting expedition. As he approached the riverbank, he noticed a group of fishermen casting their nets into the water. Whiskers couldn't resist the temptation and decided to join in on the action.\\n\\nWith his agile paws, Whiskers skillfully maneuvered through the fishermen, strategically positioning himself to snatch the fish caught in their nets. The fishermen were amazed by Whiskers' quick reflexes and couldn't help but laugh at the sight of a cat fishing alongside them.\\n\\nWhiskers continued his fish-stealing escapades for several days, becoming somewhat of a local legend. People from neighboring villages would come to witness the incredible sight of a cat outsmarting seasoned fishermen.\\n\\nOne day, as Whiskers was enjoying his stolen fish by the river, he noticed a stray dog named Rover approaching him. Rover had heard about Whiskers' fishing talents and was intrigued by the cat's abilities.\\n\\nCurious, Whiskers decided to share his secret with Rover. He taught him the art of stealth and how to snatch fish from the nets without being noticed. Rover, being a quick learner, soon became Whiskers' partner in crime.\\n\\nTogether, Whiskers and Rover formed an unbeatable duo, leaving the fishermen puzzled and amazed at the disappearing fish. The villagers, entertained by their antics, started leaving fish out for Whiskers and Rover as a token of appreciation.\\n\\nAnd so, Whiskers and Rover continued their fish-stealing adventures, bringing joy and laughter to the village. Eugene's thought about cats and their love for fish certainly came to life in the mischievous and clever Whiskers, who proved that cats truly have a special affinity for fish.\")]}\n"
+     ]
+    }
+   ],
+   "source": [
+    "async for chunk in remote_runnable.astream({\"input\": \"what does eugene think of cats? Then tell me a story about that thought.\"}):\n",
+    "    print('--')\n",
+    "    print(chunk)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Stream Events\n",
+    "\n",
+    "The client is looking for a runnable name called `agent` for the chain events. This name was defined on the server side using `runnable.with_config({\"run_name\": \"agent\"}`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Starting agent: agent with input: {'input': 'what does eugene think of cats? Then tell me a story about that thought.'}\n",
+      "--\n",
+      "Starting tool: get_eugene_thoughts with inputs: {'query': 'cats'}\n",
+      "Done tool: get_eugene_thoughts\n",
+      "Tool output was: [Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\n",
+      "--\n",
+      "E|ug|ene| thinks| that| cats| like| fish|.| Now| let| me| tell| you| a| story| about| that| thought|:\n",
+      "\n",
+      "|Once| upon| a| time|,| in| a| small| village|,| there| lived| a| curious| cat| named| Wh|isk|ers|.| Wh|isk|ers| was| known| for| his| love| of| fish|.| Every| day|,| he| would| venture| to| the| nearby| river| in| search| of| his| favorite| meal|.\n",
+      "\n",
+      "|One| sunny| morning|,| Wh|isk|ers| set| out| on| his| daily| fish|-h|unting| expedition|.| As| he| approached| the| river|,| he| could| smell| the| fresh| scent| of| the| water| and| feel| the| excitement| building| up| inside| him|.| Wh|isk|ers| knew| that| today| might| be| his| lucky| day|.\n",
+      "\n",
+      "|He| carefully| ti|pto|ed| along| the| river|bank|,| his| eyes| fixed| on| the| water|'s| surface|.| Suddenly|,| he| spotted| a| shimmer|ing| fish| swimming| gracefully| through| the| clear| blue| water|.| Wh|isk|ers| c|rou|ched| low|,| ready| to| p|ounce|.\n",
+      "\n",
+      "|With| lightning| speed|,| he| le|aped| into| the| air|,| his| p|aws| out|st|retched| towards| the| fish|.| Splash|!| Wh|isk|ers| landed| right| in| the| middle| of| the| river|,| causing| r|ipples| to| spread| in| all| directions|.| But| he| didn|'t| care|.| All| he| wanted| was| that| delicious| fish|.\n",
+      "\n",
+      "|Wh|isk|ers| chased| the| fish| with| all| his| might|,| dart|ing| through| the| water| with| elegance| and| precision|.| The| fish| sw|am| gracefully|,| trying| to| escape| Wh|isk|ers|'| determined| pursuit|.| But| the| cat| was| relentless|.\n",
+      "\n",
+      "|After| a| few| moments| of| intense| chase|,| Wh|isk|ers| finally| managed| to| catch| the| fish| in| his| p|aws|.| He| triumph|antly| carried| it| to| the| river|bank|,| where| he| enjoyed| his| well|-des|erved| feast|.| The| taste| of| the| fresh| fish| was| heavenly|,| satisfying| his| hunger| and| bringing| a| content|ed| smile| to| his| face|.\n",
+      "\n",
+      "|From| that| day| on|,| Wh|isk|ers| became| known| as| the| legendary| fish|-catching| cat| in| the| village|.| People| would| often| gather| by| the| river| to| watch| him| in| action|,| amazed| by| his| agility| and| determination|.| Wh|isk|ers| taught| everyone| the| importance| of| perseverance| and| following| one|'s| passion|,| just| like| he| pursued| his| love| for| fish|.\n",
+      "\n",
+      "|And| so|,| Wh|isk|ers| continued| his| fish|-h|unting| adventures|,| spreading| joy| and| inspiration| to| everyone| he| encountered|.| He| proved| that| when| you| have| a| passion| for| something|,| nothing| can| stop| you| from| achieving| it| –| just| like| cats| and| their| love| for| fish|.\n",
+      "\n",
+      "|The| end|.|\n",
+      "--\n",
+      "Done agent: agent with output: Eugene thinks that cats like fish. Now let me tell you a story about that thought:\n",
+      "\n",
+      "Once upon a time, in a small village, there lived a curious cat named Whiskers. Whiskers was known for his love of fish. Every day, he would venture to the nearby river in search of his favorite meal.\n",
+      "\n",
+      "One sunny morning, Whiskers set out on his daily fish-hunting expedition. As he approached the river, he could smell the fresh scent of the water and feel the excitement building up inside him. Whiskers knew that today might be his lucky day.\n",
+      "\n",
+      "He carefully tiptoed along the riverbank, his eyes fixed on the water's surface. Suddenly, he spotted a shimmering fish swimming gracefully through the clear blue water. Whiskers crouched low, ready to pounce.\n",
+      "\n",
+      "With lightning speed, he leaped into the air, his paws outstretched towards the fish. Splash! Whiskers landed right in the middle of the river, causing ripples to spread in all directions. But he didn't care. All he wanted was that delicious fish.\n",
+      "\n",
+      "Whiskers chased the fish with all his might, darting through the water with elegance and precision. The fish swam gracefully, trying to escape Whiskers' determined pursuit. But the cat was relentless.\n",
+      "\n",
+      "After a few moments of intense chase, Whiskers finally managed to catch the fish in his paws. He triumphantly carried it to the riverbank, where he enjoyed his well-deserved feast. The taste of the fresh fish was heavenly, satisfying his hunger and bringing a contented smile to his face.\n",
+      "\n",
+      "From that day on, Whiskers became known as the legendary fish-catching cat in the village. People would often gather by the river to watch him in action, amazed by his agility and determination. Whiskers taught everyone the importance of perseverance and following one's passion, just like he pursued his love for fish.\n",
+      "\n",
+      "And so, Whiskers continued his fish-hunting adventures, spreading joy and inspiration to everyone he encountered. He proved that when you have a passion for something, nothing can stop you from achieving it – just like cats and their love for fish.\n",
+      "\n",
+      "The end.\n"
+     ]
+    }
+   ],
+   "source": [
+    "async for event in remote_runnable.astream_events(\n",
+    "    {\"input\": \"what does eugene think of cats? Then tell me a story about that thought.\"},\n",
+    "    version=\"v1\",\n",
+    "):\n",
+    "    kind = event[\"event\"]\n",
+    "    if kind == \"on_chain_start\":\n",
+    "        if (\n",
+    "            event[\"name\"] == \"agent\"\n",
+    "        ):  # Was assigned when creating the agent with `.with_config({\"run_name\": \"Agent\"})`\n",
+    "            print(\n",
+    "                f\"Starting agent: {event['name']} with input: {event['data'].get('input')}\"\n",
+    "            )\n",
+    "    elif kind == \"on_chain_end\":\n",
+    "        if (\n",
+    "            event[\"name\"] == \"agent\"\n",
+    "        ):  # Was assigned when creating the agent with `.with_config({\"run_name\": \"Agent\"})`\n",
+    "            print()\n",
+    "            print(\"--\")\n",
+    "            print(\n",
+    "                f\"Done agent: {event['name']} with output: {event['data'].get('output')['output']}\"\n",
+    "            )\n",
+    "    if kind == \"on_chat_model_stream\":\n",
+    "        content = event[\"data\"][\"chunk\"].content\n",
+    "        if content:\n",
+    "            # Empty content in the context of OpenAI means\n",
+    "            # that the model is asking for a tool to be invoked.\n",
+    "            # So we only print non-empty content\n",
+    "            print(content, end=\"|\")\n",
+    "    elif kind == \"on_tool_start\":\n",
+    "        print(\"--\")\n",
+    "        print(\n",
+    "            f\"Starting tool: {event['name']} with inputs: {event['data'].get('input')}\"\n",
+    "        )\n",
+    "    elif kind == \"on_tool_end\":\n",
+    "        print(f\"Done tool: {event['name']}\")\n",
+    "        print(f\"Tool output was: {event['data'].get('output')}\")\n",
+    "        print(\"--\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Stream log\n",
+    "\n",
+    "If you need acccess the individual llm tokens from an agent use `astream_log`. Please make sure that you set **streaming=True** on your LLM (see server code). For this to work, the LLM must also support streaming!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "--\n",
+      "RunLogPatch({'op': 'replace',\n",
+      "  'path': '',\n",
+      "  'value': {'final_output': None,\n",
+      "            'id': '1213bf75-c5ff-401a-b3c5-b4663ec7dfca',\n",
+      "            'logs': {},\n",
+      "            'streamed_output': []}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '7629fc73-a2fd-48fa-b9a2-c336e270180d',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'RunnableSequence',\n",
+      "            'start_time': '2024-01-05T22:24:50.737+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': [],\n",
+      "            'type': 'chain'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '72d2e143-d977-4661-b901-430cae1ee739',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'RunnableParallel<input,agent_scratchpad>',\n",
+      "            'start_time': '2024-01-05T22:24:50.738+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:1'],\n",
+      "            'type': 'chain'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '7ec78a86-596f-4904-9888-45ef67302cb8',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': '<lambda>',\n",
+      "            'start_time': '2024-01-05T22:24:50.739+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['map:key:input'],\n",
+      "            'type': 'chain'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '281ca194-3e0c-490c-b637-759f5d971769',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': '<lambda>',\n",
+      "            'start_time': '2024-01-05T22:24:50.739+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['map:key:agent_scratchpad'],\n",
+      "            'type': 'chain'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>/final_output',\n",
+      "  'value': {'output': 'what does eugene think of cats?'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/<lambda>/end_time',\n",
+      "  'value': '2024-01-05T22:24:50.740+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:2/final_output',\n",
+      "  'value': {'output': []}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/<lambda>:2/end_time',\n",
+      "  'value': '2024-01-05T22:24:50.740+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>/final_output',\n",
+      "  'value': {'agent_scratchpad': [],\n",
+      "            'input': 'what does eugene think of cats?'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>/end_time',\n",
+      "  'value': '2024-01-05T22:24:50.740+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '68c6ad9d-b9f0-47e0-9ea7-d0f170a22732',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'ChatPromptTemplate',\n",
+      "            'start_time': '2024-01-05T22:24:50.741+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:2'],\n",
+      "            'type': 'prompt'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate/final_output',\n",
+      "  'value': {'messages': [SystemMessage(content='You are a helpful assistant.'),\n",
+      "                         HumanMessage(content='what does eugene think of cats?')]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate/end_time',\n",
+      "  'value': '2024-01-05T22:24:50.741+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '883e05a9-6b29-4494-92b3-51c72973cb54',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'ChatOpenAI',\n",
+      "            'start_time': '2024-01-05T22:24:50.743+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:3'],\n",
+      "            'type': 'llm'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': ''}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '{\\n'}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' '}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' \"'}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': 'query'}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '\":'}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' \"'}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': 'cats'}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '\"\\n'}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '}'}})})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/final_output',\n",
+      "  'value': LLMResult(generations=[[ChatGeneration(generation_info={'finish_reason': 'function_call'}, message=AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}))]], llm_output=None, run=None)},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI/end_time',\n",
+      "  'value': '2024-01-05T22:24:51.497+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'f8052926-8eb1-4df1-a701-309ee48fd63c',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'OpenAIFunctionsAgentOutputParser',\n",
+      "            'start_time': '2024-01-05T22:24:51.498+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:4'],\n",
+      "            'type': 'parser'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser/final_output',\n",
+      "  'value': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser/end_time',\n",
+      "  'value': '2024-01-05T22:24:51.499+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence/final_output',\n",
+      "  'value': {'output': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence/end_time',\n",
+      "  'value': '2024-01-05T22:24:51.500+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'actions': [AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])],\n",
+      "            'messages': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]}},\n",
+      " {'op': 'replace',\n",
+      "  'path': '/final_output',\n",
+      "  'value': {'actions': [AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])],\n",
+      "            'messages': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/get_eugene_thoughts',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'e745d7ec-5992-4ab8-a3ef-191342c6470c',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'get_eugene_thoughts',\n",
+      "            'start_time': '2024-01-05T22:24:51.501+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': [],\n",
+      "            'type': 'tool'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/get_eugene_thoughts/final_output',\n",
+      "  'value': {'output': \"[Document(page_content='cats like fish'), \"\n",
+      "                      \"Document(page_content='dogs like sticks')]\"}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/get_eugene_thoughts/end_time',\n",
+      "  'value': '2024-01-05T22:24:51.737+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'messages': [FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')],\n",
+      "            'steps': [{'action': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]),\n",
+      "                       'observation': [Document(page_content='cats like fish'),\n",
+      "                                       Document(page_content='dogs like sticks')]}]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/steps',\n",
+      "  'value': [{'action': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]),\n",
+      "             'observation': [Document(page_content='cats like fish'),\n",
+      "                             Document(page_content='dogs like sticks')]}]},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/messages/1',\n",
+      "  'value': FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '15d4ae13-554b-4144-9aa8-05527a15c042',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'RunnableSequence',\n",
+      "            'start_time': '2024-01-05T22:24:51.739+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': [],\n",
+      "            'type': 'chain'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '207dab98-d50c-4f2c-934d-291ba3ffd953',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'RunnableParallel<input,agent_scratchpad>',\n",
+      "            'start_time': '2024-01-05T22:24:51.740+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:1'],\n",
+      "            'type': 'chain'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:3',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '8795a642-3f8e-4e5c-aec6-825b7aac2148',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': '<lambda>',\n",
+      "            'start_time': '2024-01-05T22:24:51.741+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['map:key:input'],\n",
+      "            'type': 'chain'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:4',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '60be3aac-3718-4386-a941-1dd371b9a3ea',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': '<lambda>',\n",
+      "            'start_time': '2024-01-05T22:24:51.741+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['map:key:agent_scratchpad'],\n",
+      "            'type': 'chain'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:3/final_output',\n",
+      "  'value': {'output': 'what does eugene think of cats?'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/<lambda>:3/end_time',\n",
+      "  'value': '2024-01-05T22:24:51.742+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:4/final_output',\n",
+      "  'value': {'output': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}),\n",
+      "                       FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/<lambda>:4/end_time',\n",
+      "  'value': '2024-01-05T22:24:51.742+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>:2/final_output',\n",
+      "  'value': {'agent_scratchpad': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}),\n",
+      "                                 FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')],\n",
+      "            'input': 'what does eugene think of cats?'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>:2/end_time',\n",
+      "  'value': '2024-01-05T22:24:51.742+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'f1ff89b0-ae9c-49d1-8a2e-73c7f001f1e7',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'ChatPromptTemplate',\n",
+      "            'start_time': '2024-01-05T22:24:51.743+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:2'],\n",
+      "            'type': 'prompt'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate:2/final_output',\n",
+      "  'value': {'messages': [SystemMessage(content='You are a helpful assistant.'),\n",
+      "                         HumanMessage(content='what does eugene think of cats?'),\n",
+      "                         AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}),\n",
+      "                         FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate:2/end_time',\n",
+      "  'value': '2024-01-05T22:24:51.743+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'c4aeb3d0-33c3-4f46-80e9-d1ffb0f983f2',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'ChatOpenAI',\n",
+      "            'start_time': '2024-01-05T22:24:51.744+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:3'],\n",
+      "            'type': 'llm'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI:2/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': 'E'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='E')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': 'ug'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='ug')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': 'ene'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='ene')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' has'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' has')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' two'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' two')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' thoughts'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' thoughts')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' on'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' on')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' cats'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' cats')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': '.'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='.')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' One'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' One')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' thought'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' thought')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' is'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' is')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' \"'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' \"')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': 'cats'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='cats')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' like'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' like')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' fish'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' fish')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': '\"'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='\"')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' and'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' and')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' the'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' the')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' other'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' other')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' thought'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' thought')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' is'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' is')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' \"'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' \"')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': 'dogs'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='dogs')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' like'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' like')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': ' sticks'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' sticks')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output_str/-',\n",
+      "  'value': '\".'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='\".')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/ChatOpenAI:2/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/final_output',\n",
+      "  'value': LLMResult(generations=[[ChatGeneration(text='Eugene has two thoughts on cats. One thought is \"cats like fish\" and the other thought is \"dogs like sticks\".', generation_info={'finish_reason': 'stop'}, message=AIMessage(content='Eugene has two thoughts on cats. One thought is \"cats like fish\" and the other thought is \"dogs like sticks\".'))]], llm_output=None, run=None)},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatOpenAI:2/end_time',\n",
+      "  'value': '2024-01-05T22:24:53.082+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '7074d8c2-df0b-4d18-9890-dd050c2f0ca9',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'OpenAIFunctionsAgentOutputParser',\n",
+      "            'start_time': '2024-01-05T22:24:53.083+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:4'],\n",
+      "            'type': 'parser'}})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser:2/final_output',\n",
+      "  'value': AgentFinish(return_values={'output': 'Eugene has two thoughts on cats. One thought is \"cats like fish\" and the other thought is \"dogs like sticks\".'}, log='Eugene has two thoughts on cats. One thought is \"cats like fish\" and the other thought is \"dogs like sticks\".')},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser:2/end_time',\n",
+      "  'value': '2024-01-05T22:24:53.084+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence:2/final_output',\n",
+      "  'value': {'output': AgentFinish(return_values={'output': 'Eugene has two thoughts on cats. One thought is \"cats like fish\" and the other thought is \"dogs like sticks\".'}, log='Eugene has two thoughts on cats. One thought is \"cats like fish\" and the other thought is \"dogs like sticks\".')}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence:2/end_time',\n",
+      "  'value': '2024-01-05T22:24:53.084+00:00'})\n",
+      "--\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'messages': [AIMessage(content='Eugene has two thoughts on cats. One thought is \"cats like fish\" and the other thought is \"dogs like sticks\".')],\n",
+      "            'output': 'Eugene has two thoughts on cats. One thought is \"cats '\n",
+      "                      'like fish\" and the other thought is \"dogs like '\n",
+      "                      'sticks\".'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/output',\n",
+      "  'value': 'Eugene has two thoughts on cats. One thought is \"cats like fish\" '\n",
+      "           'and the other thought is \"dogs like sticks\".'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/messages/2',\n",
+      "  'value': AIMessage(content='Eugene has two thoughts on cats. One thought is \"cats like fish\" and the other thought is \"dogs like sticks\".')})\n"
+     ]
+    }
+   ],
+   "source": [
+    "async for chunk in remote_runnable.astream_log({\"input\": \"what does eugene think of cats?\"}):\n",
+    "    print('--')\n",
+    "    print(chunk)"
+   ]
  }
 ],
 "metadata": {
@@ -132,7 +943,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.9.6"
+   "version": "3.10.8"
  }
 },
 "nbformat": 4,
@@ -1,5 +1,24 @@
 #!/usr/bin/env python
-"""Example LangChain server exposes a conversational retrieval chain."""
+"""Example LangChain server exposes a conversational retrieval agent.
+
+Relevant LangChain documentation:
+
+* Creating a custom agent: https://python.langchain.com/docs/modules/agents/how_to/custom_agent
+* Streaming with agents: https://python.langchain.com/docs/modules/agents/how_to/streaming#custom-streaming-with-events
+* General streaming documentation: https://python.langchain.com/docs/expression_language/streaming
+
+**ATTENTION**
+1. To support streaming individual tokens you will need to use the astream events
+   endpoint rather than the streaming endpoint.
+2. This example does not truncate message history, so it will crash if you
+   send too many messages (exceed token length).
+3. The playground at the moment does not render agent output well! If you want to
+   use the playground you need to customize it's output server side using astream
+   events by wrapping it within another runnable.
+4. See the client notebook it has an example of how to use stream_events client side!
+"""
+from typing import Any
+
 from fastapi import FastAPI
 from langchain.agents import AgentExecutor, tool
 from langchain.agents.format_scratchpad import format_to_openai_functions
@@ -30,12 +49,22 @@ tools = [get_eugene_thoughts]
 prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful assistant."),
+        # Please note that the ordering of the user input vs.
+        # the agent_scratchpad is important.
+        # The agent_scratchpad is a working space for the agent to think,
+        # invoke tools, see tools outputs in order to respond to the given
+        # user input. It has to come AFTER the user input.
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
 )

-llm = ChatOpenAI()
+# We need to set streaming=True on the LLM to support streaming individual tokens.
+# Tokens will be available when using the stream_log / stream events endpoints,
+# but not when using the stream endpoint since the stream implementation for agent
+# streams action observation pairs not individual tokens.
+# See the client notebook that shows how to use the stream events endpoint.
+llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0, streaming=True)

 llm_with_tools = llm.bind(functions=[format_tool_to_openai_function(t) for t in tools])

@@ -56,7 +85,7 @@ agent_executor = AgentExecutor(agent=agent, tools=tools)
 app = FastAPI(
    title="LangChain Server",
    version="1.0",
-    description="Spin up a simple api server using Langchain's Runnable interfaces",
+    description="Spin up a simple api server using LangChain's Runnable interfaces",
 )


@@ -67,14 +96,20 @@ class Input(BaseModel):


 class Output(BaseModel):
-    output: str
+    output: Any


 # Adds routes to the app for using the chain under:
 # /invoke
 # /batch
 # /stream
-add_routes(app, agent_executor.with_types(input_type=Input, output_type=Output))
+# /stream_events
+add_routes(
+    app,
+    agent_executor.with_types(input_type=Input, output_type=Output).with_config(
+        {"run_name": "agent"}
+    ),
+)

 if __name__ == "__main__":
    import uvicorn
@@ -0,0 +1,232 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Client\n",
+    "\n",
+    "Client code interacting with a server that implements: \n",
+    "\n",
+    "* custom streaming for an agent\n",
+    "* agent with user selected tools\n",
+    "\n",
+    "This agent does not have memory! See other examples in LangServe to see how to add memory.\n",
+    "\n",
+    "**ATTENTION** We made the agent stream strings as an output. This is almost certainly not what you would want for your application. Feel free to adapt to return more structured output; however, keep in mind that likely the client can just use `astream_events`!\n",
+    "\n",
+    "See relevant documentation about agents:\n",
+    "\n",
+    "* Creating a custom agent: https://python.langchain.com/docs/modules/agents/how_to/custom_agent\n",
+    "* Streaming with agents: https://python.langchain.com/docs/modules/agents/how_to/streaming#custom-streaming-with-events\n",
+    "* General streaming documentation: https://python.langchain.com/docs/expression_language/streaming"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can interact with this via API directly"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "16"
+      ]
+     },
+     "execution_count": 1,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "word = \"audioeeeeeeeeeee\"\n",
+    "len(word)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Starting agent: agent with input: {'input': 'what is the length of the word audioeeeeeeeeeee?'}\n",
+      "The length of the word \"audioeeeeeeeeeee\" is 15 characters.\n",
+      "Done agent: agent with output: The length of the word \"audioeeeeeeeeeee\" is 15 characters.\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "import requests\n",
+    "\n",
+    "inputs = {\"input\": {\"input\": f\"what is the length of the word {word}?\", \"chat_history\": [], \"tools\": []}}\n",
+    "response = requests.post(\"http://localhost:8000/invoke\", json=inputs)\n",
+    "\n",
+    "print(response.json()['output'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's provide it with a tool to test that tool selection works"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Starting agent: agent with input: {'input': 'what is the length of the word audioeeeeeeeeeee?'}\n",
+      "\n",
+      "Starting tool: word_length with inputs: {'word': 'audioeeeeeeeeeee'}\n",
+      "\n",
+      "Done tool: word_length with output: 16\n",
+      "The length of the word \"audioeeeeeeeeeee\" is 16.\n",
+      "Done agent: agent with output: The length of the word \"audioeeeeeeeeeee\" is 16.\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "import requests\n",
+    "\n",
+    "inputs = {\"input\": {\"input\": f\"what is the length of the word {word}?\", \"chat_history\": [], \"tools\": [\"word_length\", \"favorite_animal\"]}}\n",
+    "response = requests.post(\"http://localhost:8000/invoke\", json=inputs)\n",
+    "\n",
+    "print(response.json()['output'])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can also interact with this via the RemoteRunnable interface (to use in other chains)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langserve import RemoteRunnable\n",
+    "\n",
+    "remote_runnable = RemoteRunnable(\"http://localhost:8000/\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Remote runnable has the same interface as local runnables"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Stream\n",
+    "\n",
+    "Streaming output from a **CUSTOM STREAMING** implementation that streams string representations of intermediate steps. Please see server side implementation for details."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "|Starting agent: agent with input: {'input': 'What is eugenes favorite animal?'}|\n",
+      "|I|'m| sorry|,| but| I| don|'t| have| access| to| personal| information| about| individuals| unless| it| has| been| shared| with| me| in| the| course| of| our| conversation|.|\n",
+      "|Done agent: agent with output: I'm sorry, but I don't have access to personal information about individuals unless it has been shared with me in the course of our conversation.|\n",
+      "|"
+     ]
+    }
+   ],
+   "source": [
+    "async for chunk in remote_runnable.astream({\"input\": \"What is eugenes favorite animal?\", \"tools\": [\"word_length\"]}):\n",
+    "    print(chunk, end='|', flush=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "|Starting agent: agent with input: {'input': 'What is eugenes favorite animal?'}|\n",
+      "|\n",
+      "|Starting tool: favorite_animal with inputs: {'name': 'Eugene'}|\n",
+      "|\n",
+      "|Done tool: favorite_animal with output: cat|\n",
+      "|E|ug|ene|'s| favorite| animal| is| a| cat|.|\n",
+      "|Done agent: agent with output: Eugene's favorite animal is a cat.|\n",
+      "|"
+     ]
+    }
+   ],
+   "source": [
+    "async for chunk in remote_runnable.astream({\"input\": \"What is eugenes favorite animal?\", \"tools\": [\"word_length\", \"favorite_animal\"]}):\n",
+    "    print(chunk, end='|', flush=True)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
@@ -0,0 +1,248 @@
+#!/usr/bin/env python
+"""Example LangChain server that shows how to customize streaming for an agent.
+
+Example uses a RunnableLambda that:
+
+1) Uses the agent's astream events method to create a custom streaming API endpoint.
+2) Instantiates an agent with custom tools (based on the user request).
+
+In this example, we kept things simple and are outputting strings to the client
+with all the intermediate steps of the agent. This is just for demonstration
+purposes, and usually you would want to return more structured output in the form
+of a dictionary.
+
+To add history to the agent you can use RunnableWithHistory. Please see the
+other examples in LangServe for how to use RunnableWithHistory to store history
+on the server side.
+
+Alternatively, you can keep track of history on the client side and send it to the
+server with each request. For that to work, you will definitely want to modify the
+streaming output to yield dictionaries with structured output, so it's easy
+to determine what the final agent output was on the client side.
+
+Customize the streaming output to your use case!
+
+Note that we configure the agent using the `tools` field in the input rather
+than using configurable fields. Using custom runnables and configurable fields
+is another option to customize the agent. 
+
+Please see configurable_agent_executor: https://github.com/langchain-ai/langserve/blob/main/examples/configurable_agent_executor/server.py
+for an example that uses a custom runnable with configurable fields.
+
+Relevant LangChain documentation:
+
+* Creating a custom agent: https://python.langchain.com/docs/modules/agents/how_to/custom_agent
+* Streaming with agents: https://python.langchain.com/docs/modules/agents/how_to/streaming#custom-streaming-with-events
+* General streaming documentation: https://python.langchain.com/docs/expression_language/streaming
+* Message History: https://python.langchain.com/docs/expression_language/how_to/message_history
+
+**ATTENTION**
+1. This example does not truncate message history, so it will crash if you
+   send too many messages (exceed token length).
+2. The playground at the moment does not render agent output well! If you want to
+   use the playground you need to customize it's output server side using astream
+   events by wrapping it within another runnable.
+3. See the client notebook to see how .stream() behaves!
+"""  # noqa: E501
+from typing import Any, AsyncIterator, List, Literal
+
+from fastapi import FastAPI
+from langchain.agents import AgentExecutor, tool
+from langchain.agents.format_scratchpad.openai_tools import (
+    format_to_openai_tool_messages,
+)
+from langchain.agents.output_parsers.openai_tools import OpenAIToolsAgentOutputParser
+from langchain.prompts import MessagesPlaceholder
+from langchain_community.tools.convert_to_openai import format_tool_to_openai_tool
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.runnables import RunnableLambda
+from langchain_openai import ChatOpenAI
+
+from langserve import add_routes
+from langserve.pydantic_v1 import BaseModel
+
+prompt = ChatPromptTemplate.from_messages(
+    [
+        (
+            "system",
+            "You are very powerful assistant, but bad at calculating lengths of words. "
+            "Talk with the user as normal. "
+            "If they ask you to calculate the length of a word, use a tool",
+        ),
+        # Please note the ordering of the fields in the prompt!
+        # The correct ordering is:
+        # 1. user - the user's current input
+        # 2. agent_scratchpad - the agent's working space for thinking and
+        #    invoking tools to respond to the user's input.
+        # If you change the ordering, the agent will not work correctly since
+        # the messages will be shown to the underlying LLM in the wrong order.
+        ("user", "{input}"),
+        MessagesPlaceholder(variable_name="agent_scratchpad"),
+    ]
+)
+
+
+@tool
+def word_length(word: str) -> int:
+    """Returns a counter word"""
+    return len(word)
+
+
+@tool
+def favorite_animal(name: str) -> str:
+    """Get the favorite animal of the person with the given name"""
+    if name.lower().strip() == "eugene":
+        return "cat"
+    return "dog"
+
+
+# We need to set streaming=True on the LLM to support streaming individual tokens.
+# Tokens will be available when using the stream_log / stream events endpoints,
+# but not when using the stream endpoint since the stream implementation for agent
+# streams action observation pairs not individual tokens.
+# See the client notebook that shows how to use the stream events endpoint.
+llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0, streaming=True)
+
+TOOL_MAPPING = {
+    "word_length": word_length,
+    "favorite_animal": favorite_animal,
+}
+KnownTool = Literal["word_length", "favorite_animal"]
+
+
+def _create_agent_with_tools(requested_tools: List[KnownTool]) -> AgentExecutor:
+    """Create an agent with custom tools."""
+    tools = []
+
+    for requested_tool in requested_tools:
+        if requested_tool not in TOOL_MAPPING:
+            raise ValueError(f"Unknown tool: {requested_tool}")
+        tools.append(TOOL_MAPPING[requested_tool])
+
+    if tools:
+        llm_with_tools = llm.bind(
+            tools=[format_tool_to_openai_tool(tool) for tool in tools]
+        )
+    else:
+        llm_with_tools = llm
+
+    agent = (
+        {
+            "input": lambda x: x["input"],
+            "agent_scratchpad": lambda x: format_to_openai_tool_messages(
+                x["intermediate_steps"]
+            ),
+        }
+        | prompt
+        | llm_with_tools
+        | OpenAIToolsAgentOutputParser()
+    )
+    agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True).with_config(
+        {"run_name": "agent"}
+    )
+    return agent_executor
+
+
+app = FastAPI(
+    title="LangChain Server",
+    version="1.0",
+    description="Spin up a simple api server using LangChain's Runnable interfaces",
+)
+
+
+# We need to add these input/output schemas because the current AgentExecutor
+# is lacking in schemas.
+class Input(BaseModel):
+    input: str
+    tools: List[KnownTool]
+
+
+async def custom_stream(input: Input) -> AsyncIterator[str]:
+    """A custom runnable that can stream content.
+
+    Args:
+        input: The input to the agent. See the Input model for more details.
+
+    Yields:
+        strings that are streamed to the client.
+
+
+    Strings were chosen for simplicity, feel free to adapt to your use case.
+
+    You will almost certainly want to return more structured output in the form
+    of a dictionary, so it's easy to determine what the agent is doing without
+    parsing the output.
+
+    Before creating a custom streaming API, you should consider if you can use
+    the existing astream events API and customize the output on the client side
+    (potentially less overall work both server and client side).
+    """
+    agent_executor = _create_agent_with_tools(input["tools"])
+    async for event in agent_executor.astream_events(
+        {
+            "input": input["input"],
+        },
+        version="v1",
+    ):
+        kind = event["event"]
+        if kind == "on_chain_start":
+            if (
+                event["name"] == "agent"
+            ):  # matches `.with_config({"run_name": "Agent"})` in agent_executor
+                yield "\n"
+                yield (
+                    f"Starting agent: {event['name']} "
+                    f"with input: {event['data'].get('input')}"
+                )
+                yield "\n"
+        elif kind == "on_chain_end":
+            if (
+                event["name"] == "agent"
+            ):  # matches `.with_config({"run_name": "Agent"})` in agent_executor
+                yield "\n"
+                yield (
+                    f"Done agent: {event['name']} "
+                    f"with output: {event['data'].get('output')['output']}"
+                )
+                yield "\n"
+        if kind == "on_chat_model_stream":
+            content = event["data"]["chunk"].content
+            if content:
+                # Empty content in the context of OpenAI means
+                # that the model is asking for a tool to be invoked.
+                # So we only print non-empty content
+                yield content
+        elif kind == "on_tool_start":
+            yield "\n"
+            yield (
+                f"Starting tool: {event['name']} "
+                f"with inputs: {event['data'].get('input')}"
+            )
+            yield "\n"
+        elif kind == "on_tool_end":
+            yield "\n"
+            yield (
+                f"Done tool: {event['name']} "
+                f"with output: {event['data'].get('output')}"
+            )
+            yield "\n"
+
+
+class Output(BaseModel):
+    output: Any
+
+
+# Adds routes to the app for using the chain under:
+# /invoke
+# /batch
+# /stream
+# /stream_events
+add_routes(
+    app,
+    RunnableLambda(custom_stream),
+)
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="localhost", port=8000)
@@ -0,0 +1,554 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Client\n",
+    "\n",
+    "Demo of a client interacting with a remote agent that can use history.\n",
+    "\n",
+    "See relevant documentation about agents:\n",
+    "\n",
+    "* Creating a custom agent: https://python.langchain.com/docs/modules/agents/how_to/custom_agent\n",
+    "* Streaming with agents: https://python.langchain.com/docs/modules/agents/how_to/streaming#custom-streaming-with-events\n",
+    "* General streaming documentation: https://python.langchain.com/docs/expression_language/streaming"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can interact with this via API directly"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'output': {'output': 'The length of the word \"audioee\" is 7.'},\n",
+       " 'callback_events': [],\n",
+       " 'metadata': {'run_id': '1847be77-f53c-40ba-b88d-06af3a598b6e'}}"
+      ]
+     },
+     "execution_count": 1,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import requests\n",
+    "\n",
+    "inputs = {\"input\": {\"input\": \"what is the length of the word audioee?\", \"chat_history\": []}}\n",
+    "response = requests.post(\"http://localhost:8000/invoke\", json=inputs)\n",
+    "\n",
+    "response.json()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can also interact with this via the RemoteRunnable interface (to use in other chains)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langserve import RemoteRunnable\n",
+    "\n",
+    "remote_runnable = RemoteRunnable(\"http://localhost:8000/\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Remote runnable has the same interface as local runnables"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain_core.messages import HumanMessage, AIMessage"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  hello\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: Hello! How can I assist you today?\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  my name is eugene\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: Nice to meet you, Eugene! How can I help you today?\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  what is my name\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: Your name is Eugene.\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  what is the length of my name\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: The length of your name, Eugene, is 6 characters.\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  q\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: Bye bye human\n"
+     ]
+    }
+   ],
+   "source": [
+    "chat_history = []\n",
+    "\n",
+    "while True:\n",
+    "    human = input(\"Human (Q/q to quit): \")\n",
+    "    if human in {\"q\", \"Q\"}:\n",
+    "        print('AI: Bye bye human')\n",
+    "        break\n",
+    "    ai = await remote_runnable.ainvoke({\"input\": human, \"chat_history\": chat_history})\n",
+    "    print(f\"AI: {ai['output']}\")\n",
+    "    chat_history.extend([HumanMessage(content=human), AIMessage(content=ai['output'])])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Stream\n",
+    "\n",
+    "Please note that streaming alternates between actions and observations. It does not stream individual tokens!\n",
+    "\n",
+    "To stream individual tokens, we need to use the astream events endpoint (see below)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  hello\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: \n",
+      "Hello! How can I assist you today?\n",
+      "------\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  my name is eugene\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: \n",
+      "Nice to meet you, Eugene! How can I help you today?\n",
+      "------\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  what is my name\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: \n",
+      "Your name is Eugene.\n",
+      "------\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  what's the length of my name?\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: \n",
+      "Calling Tool ```word_length``` with input ```{'word': 'Eugene'}```\n",
+      "------\n",
+      "Got result: ```6```\n",
+      "------\n",
+      "The length of your name, Eugene, is 6 characters.\n",
+      "------\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  q\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: Bye bye human\n"
+     ]
+    }
+   ],
+   "source": [
+    "chat_history = []\n",
+    "\n",
+    "while True:\n",
+    "    human = input(\"Human (Q/q to quit): \")\n",
+    "    if human in {\"q\", \"Q\"}:\n",
+    "        print('AI: Bye bye human')\n",
+    "        break\n",
+    "\n",
+    "    ai = None\n",
+    "    print(\"AI: \")\n",
+    "    async for chunk in remote_runnable.astream({\"input\": human, \"chat_history\": chat_history}):\n",
+    "        # Agent Action\n",
+    "        if \"actions\" in chunk:\n",
+    "            for action in chunk[\"actions\"]:\n",
+    "                print(\n",
+    "                    f\"Calling Tool ```{action['tool']}``` with input ```{action['tool_input']}```\"\n",
+    "                )\n",
+    "        # Observation\n",
+    "        elif \"steps\" in chunk:\n",
+    "            for step in chunk[\"steps\"]:\n",
+    "                print(f\"Got result: ```{step['observation']}```\")\n",
+    "        # Final result\n",
+    "        elif \"output\" in chunk:\n",
+    "            print(chunk['output'])\n",
+    "            ai = AIMessage(content=chunk['output'])\n",
+    "        else:\n",
+    "            raise ValueError\n",
+    "        print(\"------\")        \n",
+    "    chat_history.extend([HumanMessage(content=human), ai])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Stream Events"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  hello! my name is eugene\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: \n",
+      "Starting agent: agent with input: {'input': 'hello! my name is eugene', 'chat_history': []}\n",
+      "Hello| Eugene|!| How| can| I| assist| you| today|?|\n",
+      "--\n",
+      "Done agent: agent with output: Hello Eugene! How can I assist you today?\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  what's the length of my name?\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: \n",
+      "Starting agent: agent with input: {'input': \"what's the length of my name?\", 'chat_history': []}\n",
+      "--\n",
+      "Starting tool: word_length with inputs: {'word': 'my name'}\n",
+      "Done tool: word_length\n",
+      "Tool output was: 7\n",
+      "--\n",
+      "The| length| of| your| name|,| \"|my| name|\",| is| |7| characters|.|\n",
+      "--\n",
+      "Done agent: agent with output: The length of your name, \"my name\", is 7 characters.\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  could you tell me a long story about the length of my name?\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: \n",
+      "Starting agent: agent with input: {'input': 'could you tell me a long story about the length of my name?', 'chat_history': []}\n",
+      "Once| upon| a| time|,| there| was| a| person| named| [|Your| Name|].| Now|,| [|Your| Name|]| had| a| very| unique| and| special| name|.| It| was| a| name| that| carried| a| lot| of| meaning| and| significance|.| People| often| wondered| about| the| length| of| [|Your| Name|]'|s| name| and| how| it| compared| to| others|.\n",
+      "\n",
+      "|One| day|,| [|Your| Name|]| decided| to| embark| on| a| journey| to| discover| the| true| length| of| their| name|.| They| traveled| far| and| wide|,| seeking| the| wisdom| of| s|ages| and| scholars| who| were| known| for| their| knowledge| of| names| and| their| lengths|.\n",
+      "\n",
+      "|Along| the| way|,| [|Your| Name|]| encountered| many| interesting| characters| who| had| their| own| stories| to| tell| about| the| lengths| of| their| names|.| Some| had| short| names| that| were| easy| to| remember|,| while| others| had| long| names| that| seemed| to| go| on| forever|.\n",
+      "\n",
+      "|As| [|Your| Name|]| continued| their| quest|,| they| came| across| a| wise| old| wizard| who| claimed| to| have| a| magical| tool| that| could| calculate| the| exact| length| of| any| name|.| Intr|ig|ued|,| [|Your| Name|]| approached| the| wizard| and| asked| for| their| assistance|.\n",
+      "\n",
+      "|The| wizard| took| out| a| mystical| device| and| asked| [|Your| Name|]| to| spell| out| their| name|.| [|Your| Name|]| eagerly| complied|,| carefully| en|unci|ating| each| letter|.| The| wizard| then| waved| the| device| over| the| name| and| muttered| an| inc|ant|ation|.\n",
+      "\n",
+      "|In| an| instant|,| the| device| displayed| the| length| of| [|Your| Name|]'|s| name|.| It| was| a| number| that| represented| the| total| count| of| characters| in| their| name|,| including| spaces| and| punctuation| marks|.| [|Your| Name|]| was| amazed| to| see| the| result| and| thanked| the| wizard| for| their| help|.\n",
+      "\n",
+      "|Ar|med| with| this| newfound| knowledge|,| [|Your| Name|]| returned| home| and| shared| their| story| with| friends| and| family|.| They| realized| that| the| length| of| their| name| was| not| just| a| random| number|,| but| a| reflection| of| their| identity| and| the| unique| journey| they| had| taken| to| discover| it|.\n",
+      "\n",
+      "|And| so|,| [|Your| Name|]| lived| happily| ever| after|,| cher|ishing| their| name| and| the| story| behind| its| length|.| They| understood| that| the| length| of| a| name| is| not| just| a| matter| of| counting| letters|,| but| a| testament| to| the| individual|ity| and| significance| of| each| person|'s| identity|.|\n",
+      "--\n",
+      "Done agent: agent with output: Once upon a time, there was a person named [Your Name]. Now, [Your Name] had a very unique and special name. It was a name that carried a lot of meaning and significance. People often wondered about the length of [Your Name]'s name and how it compared to others.\n",
+      "\n",
+      "One day, [Your Name] decided to embark on a journey to discover the true length of their name. They traveled far and wide, seeking the wisdom of sages and scholars who were known for their knowledge of names and their lengths.\n",
+      "\n",
+      "Along the way, [Your Name] encountered many interesting characters who had their own stories to tell about the lengths of their names. Some had short names that were easy to remember, while others had long names that seemed to go on forever.\n",
+      "\n",
+      "As [Your Name] continued their quest, they came across a wise old wizard who claimed to have a magical tool that could calculate the exact length of any name. Intrigued, [Your Name] approached the wizard and asked for their assistance.\n",
+      "\n",
+      "The wizard took out a mystical device and asked [Your Name] to spell out their name. [Your Name] eagerly complied, carefully enunciating each letter. The wizard then waved the device over the name and muttered an incantation.\n",
+      "\n",
+      "In an instant, the device displayed the length of [Your Name]'s name. It was a number that represented the total count of characters in their name, including spaces and punctuation marks. [Your Name] was amazed to see the result and thanked the wizard for their help.\n",
+      "\n",
+      "Armed with this newfound knowledge, [Your Name] returned home and shared their story with friends and family. They realized that the length of their name was not just a random number, but a reflection of their identity and the unique journey they had taken to discover it.\n",
+      "\n",
+      "And so, [Your Name] lived happily ever after, cherishing their name and the story behind its length. They understood that the length of a name is not just a matter of counting letters, but a testament to the individuality and significance of each person's identity.\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  that's not what I wanted. My name is eugene. calculate the length of my name and tell me a story about the result.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: \n",
+      "Starting agent: agent with input: {'input': \"that's not what I wanted. My name is eugene. calculate the length of my name and tell me a story about the result.\", 'chat_history': []}\n",
+      "--\n",
+      "Starting tool: word_length with inputs: {'word': 'eugene'}\n",
+      "Done tool: word_length\n",
+      "Tool output was: 6\n",
+      "--\n",
+      "The| length| of| your| name|,| Eugene|,| is| |6| characters|.| Now|,| let| me| tell| you| a| story| about| the| result|.\n",
+      "\n",
+      "|Once| upon| a| time|,| in| a| land| far| away|,| there| was| a| young| boy| named| Eugene|.| He| had| a| special| power| -| the| power| to| bring| joy| and| laughter| to| everyone| he| met|.| Eugene|'s| name|,| with| its| |6| letters|,| perfectly| reflected| his| vibrant| and| energetic| personality|.\n",
+      "\n",
+      "|One| day|,| Eugene| decided| to| embark| on| a| grand| adventure|.| He| set| off| on| a| journey| to| spread| happiness| and| positivity| throughout| the| kingdom|.| Along| the| way|,| he| encountered| people| from| all| walks| of| life| -| from| humble| farmers| to| noble| knights|.\n",
+      "\n",
+      "|With| his| infectious| smile| and| kind| heart|,| Eugene| touched| the| lives| of| everyone| he| met|.| His| name|,| with| its| |6| letters|,| became| synonymous| with| love|,| compassion|,| and| joy|.| People| would| often| say|,| \"|If| you| want| to| experience| true| happiness|,| just| spend| a| moment| with| Eugene|.\"\n",
+      "\n",
+      "|As| Eugene| continued| his| journey|,| word| of| his| incredible| ability| to| bring| happiness| spread| far| and| wide|.| People| from| distant| lands| would| travel| for| miles| just| to| catch| a| glimpse| of| him|.| His| name|,| with| its| |6| letters|,| became| a| symbol| of| hope| and| inspiration|.\n",
+      "\n",
+      "|E|ug|ene|'s| story| serves| as| a| reminder| that| sometimes|,| the| simplest| things| can| have| the| greatest| impact|.| His| name|,| with| its| |6| letters|,| became| a| beacon| of| light| in| a| world| that| often| seemed| dark| and| glo|omy|.\n",
+      "\n",
+      "|And| so|,| Eugene|'s| adventure| continues|,| as| he| spreads| joy| and| happiness| wherever| he| goes|.| His| name|,| with| its| |6| letters|,| will| forever| be| et|ched| in| the| hearts| of| those| whose| lives| he| has| touched|.\n",
+      "\n",
+      "|The| end|.|\n",
+      "--\n",
+      "Done agent: agent with output: The length of your name, Eugene, is 6 characters. Now, let me tell you a story about the result.\n",
+      "\n",
+      "Once upon a time, in a land far away, there was a young boy named Eugene. He had a special power - the power to bring joy and laughter to everyone he met. Eugene's name, with its 6 letters, perfectly reflected his vibrant and energetic personality.\n",
+      "\n",
+      "One day, Eugene decided to embark on a grand adventure. He set off on a journey to spread happiness and positivity throughout the kingdom. Along the way, he encountered people from all walks of life - from humble farmers to noble knights.\n",
+      "\n",
+      "With his infectious smile and kind heart, Eugene touched the lives of everyone he met. His name, with its 6 letters, became synonymous with love, compassion, and joy. People would often say, \"If you want to experience true happiness, just spend a moment with Eugene.\"\n",
+      "\n",
+      "As Eugene continued his journey, word of his incredible ability to bring happiness spread far and wide. People from distant lands would travel for miles just to catch a glimpse of him. His name, with its 6 letters, became a symbol of hope and inspiration.\n",
+      "\n",
+      "Eugene's story serves as a reminder that sometimes, the simplest things can have the greatest impact. His name, with its 6 letters, became a beacon of light in a world that often seemed dark and gloomy.\n",
+      "\n",
+      "And so, Eugene's adventure continues, as he spreads joy and happiness wherever he goes. His name, with its 6 letters, will forever be etched in the hearts of those whose lives he has touched.\n",
+      "\n",
+      "The end.\n"
+     ]
+    },
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "Human (Q/q to quit):  q\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "AI: Bye bye human\n"
+     ]
+    }
+   ],
+   "source": [
+    "chat_history = []\n",
+    "\n",
+    "while True:\n",
+    "    human = input(\"Human (Q/q to quit): \")\n",
+    "    if human in {\"q\", \"Q\"}:\n",
+    "        print('AI: Bye bye human')\n",
+    "        break\n",
+    "    ai = None\n",
+    "    print(\"AI: \")\n",
+    "    async for event in remote_runnable.astream_events(\n",
+    "        {\"input\": human, \"chat_history\": chat_history},\n",
+    "        version=\"v1\",\n",
+    "    ):\n",
+    "        kind = event[\"event\"]\n",
+    "        if kind == \"on_chain_start\":\n",
+    "            if (\n",
+    "                event[\"name\"] == \"agent\"\n",
+    "            ):  # Was assigned when creating the agent with `.with_config({\"run_name\": \"Agent\"})`\n",
+    "                print(\n",
+    "                    f\"Starting agent: {event['name']} with input: {event['data'].get('input')}\"\n",
+    "                )\n",
+    "        elif kind == \"on_chain_end\":\n",
+    "            if (\n",
+    "                event[\"name\"] == \"agent\"\n",
+    "            ):  # Was assigned when creating the agent with `.with_config({\"run_name\": \"Agent\"})`\n",
+    "                print()\n",
+    "                print(\"--\")\n",
+    "                print(\n",
+    "                    f\"Done agent: {event['name']} with output: {event['data'].get('output')['output']}\"\n",
+    "                )\n",
+    "        if kind == \"on_chat_model_stream\":\n",
+    "            content = event[\"data\"][\"chunk\"].content\n",
+    "            if content:\n",
+    "                # Empty content in the context of OpenAI means\n",
+    "                # that the model is asking for a tool to be invoked.\n",
+    "                # So we only print non-empty content\n",
+    "                print(content, end=\"|\")\n",
+    "        elif kind == \"on_tool_start\":\n",
+    "            print(\"--\")\n",
+    "            print(\n",
+    "                f\"Starting tool: {event['name']} with inputs: {event['data'].get('input')}\"\n",
+    "            )\n",
+    "        elif kind == \"on_tool_end\":\n",
+    "            print(f\"Done tool: {event['name']}\")\n",
+    "            print(f\"Tool output was: {event['data'].get('output')}\")\n",
+    "            print(\"--\")    \n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
@@ -0,0 +1,157 @@
+#!/usr/bin/env python
+"""Example LangChain server exposes and agent that has conversation history.
+
+In this example, the history is stored entirely on the client's side.
+
+Please see other examples in LangServe on how to use RunnableWithHistory to
+store history on the server side.
+
+Relevant LangChain documentation:
+
+* Creating a custom agent: https://python.langchain.com/docs/modules/agents/how_to/custom_agent
+* Streaming with agents: https://python.langchain.com/docs/modules/agents/how_to/streaming#custom-streaming-with-events
+* General streaming documentation: https://python.langchain.com/docs/expression_language/streaming
+* Message History: https://python.langchain.com/docs/expression_language/how_to/message_history
+
+**ATTENTION**
+1. To support streaming individual tokens you will need to use the astream events
+   endpoint rather than the streaming endpoint.
+2. This example does not truncate message history, so it will crash if you
+   send too many messages (exceed token length).
+3. The playground at the moment does not render agent output well! If you want to
+   use the playground you need to customize it's output server side using astream
+   events by wrapping it within another runnable.
+4. See the client notebook it has an example of how to use stream_events client side!
+"""  # noqa: E501
+from typing import Any, List, Union
+
+from fastapi import FastAPI
+from langchain.agents import AgentExecutor, tool
+from langchain.agents.format_scratchpad.openai_tools import (
+    format_to_openai_tool_messages,
+)
+from langchain.agents.output_parsers.openai_tools import OpenAIToolsAgentOutputParser
+from langchain.prompts import MessagesPlaceholder
+from langchain_community.tools.convert_to_openai import format_tool_to_openai_tool
+from langchain_core.messages import AIMessage, FunctionMessage, HumanMessage
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_openai import ChatOpenAI
+
+from langserve import add_routes
+from langserve.pydantic_v1 import BaseModel, Field
+
+prompt = ChatPromptTemplate.from_messages(
+    [
+        (
+            "system",
+            "You are very powerful assistant, but bad at calculating lengths of words. "
+            "Talk with the user as normal. "
+            "If they ask you to calculate the length of a word, use a tool",
+        ),
+        # Please note the ordering of the fields in the prompt!
+        # The correct ordering is:
+        # 1. history - the past messages between the user and the agent
+        # 2. user - the user's current input
+        # 3. agent_scratchpad - the agent's working space for thinking and
+        #    invoking tools to respond to the user's input.
+        # If you change the ordering, the agent will not work correctly since
+        # the messages will be shown to the underlying LLM in the wrong order.
+        MessagesPlaceholder(variable_name="chat_history"),
+        ("user", "{input}"),
+        MessagesPlaceholder(variable_name="agent_scratchpad"),
+    ]
+)
+
+
+@tool
+def word_length(word: str) -> int:
+    """Returns a counter word"""
+    return len(word)
+
+
+# We need to set streaming=True on the LLM to support streaming individual tokens.
+# Tokens will be available when using the stream_log / stream events endpoints,
+# but not when using the stream endpoint since the stream implementation for agent
+# streams action observation pairs not individual tokens.
+# See the client notebook that shows how to use the stream events endpoint.
+llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0, streaming=True)
+
+tools = [word_length]
+
+
+llm_with_tools = llm.bind(tools=[format_tool_to_openai_tool(tool) for tool in tools])
+
+# ATTENTION: For production use case, it's a good idea to trim the prompt to avoid
+#            exceeding the context window length used by the model.
+#
+# To fix that simply adjust the chain to trim the prompt in whatever way
+# is appropriate for your use case.
+# For example, you may want to keep the system message and the last 10 messages.
+# Or you may want to trim based on the number of tokens.
+# Or you may want to also summarize the messages to keep information about things
+# that were learned about the user.
+#
+# def prompt_trimmer(messages: List[Union[HumanMessage, AIMessage, FunctionMessage]]):
+#     '''Trims the prompt to a reasonable length.'''
+#     # Keep in mind that when trimming you may want to keep the system message!
+#     return messages[-10:] # Keep last 10 messages.
+
+agent = (
+    {
+        "input": lambda x: x["input"],
+        "agent_scratchpad": lambda x: format_to_openai_tool_messages(
+            x["intermediate_steps"]
+        ),
+        "chat_history": lambda x: x["chat_history"],
+    }
+    | prompt
+    # | prompt_trimmer # See comment above.
+    | llm_with_tools
+    | OpenAIToolsAgentOutputParser()
+)
+agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
+
+
+app = FastAPI(
+    title="LangChain Server",
+    version="1.0",
+    description="Spin up a simple api server using LangChain's Runnable interfaces",
+)
+
+
+# We need to add these input/output schemas because the current AgentExecutor
+# is lacking in schemas.
+class Input(BaseModel):
+    input: str
+    # The field extra defines a chat widget.
+    # Please see documentation about widgets in the main README.
+    # The widget is used in the playground.
+    # Keep in mind that playground support for agents is not great at the moment.
+    # To get a better experience, you'll need to customize the streaming output
+    # for now.
+    chat_history: List[Union[HumanMessage, AIMessage, FunctionMessage]] = Field(
+        ...,
+        extra={"widget": {"type": "chat", "input": "input", "output": "output"}},
+    )
+
+
+class Output(BaseModel):
+    output: Any
+
+
+# Adds routes to the app for using the chain under:
+# /invoke
+# /batch
+# /stream
+# /stream_events
+add_routes(
+    app,
+    agent_executor.with_types(input_type=Input, output_type=Output).with_config(
+        {"run_name": "agent"}
+    ),
+)
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="localhost", port=8000)
@@ -18,9 +18,9 @@ from langchain.memory import FileChatMessageHistory
 from langchain_core.chat_history import BaseChatMessageHistory
 from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
 from langchain_core.runnables.history import RunnableWithMessageHistory
-from typing_extensions import TypedDict

 from langserve import add_routes
+from langserve.pydantic_v1 import BaseModel, Field


 def _is_valid_identifier(value: str) -> bool:
@@ -79,11 +79,19 @@ prompt = ChatPromptTemplate.from_messages(
 chain = prompt | ChatAnthropic(model="claude-2")


-class InputChat(TypedDict):
+class InputChat(BaseModel):
    """Input for the chat endpoint."""

-    human_input: str
-    """Human input"""
+    # The field extra defines a chat widget.
+    # As of 2024-02-05, this chat widget is not fully supported.
+    # It's included in documentation to show how it should be specified, but
+    # will not work until the widget is fully supported for history persistence
+    # on the backend.
+    human_input: str = Field(
+        ...,
+        description="The human input to the chat system.",
+        extra={"widget": {"type": "chat", "input": "human_input"}},
+    )


 chain_with_history = RunnableWithMessageHistory(
@@ -0,0 +1,770 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Client\n",
+    "\n",
+    "Demo of a client interacting with a custom runnable executor that supports configuration.\n",
+    "\n",
+    "This server does not support invoke or batch! only stream and astream log! (see backend code.)\n",
+    "\n",
+    "The underlying backend code is just a demo in this case -- it's working around an existing bug, but uses \n",
+    "the opportunity to show how to create custom runnables."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can interact with this via API directly"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "event: metadata\n",
+      "data: {\"run_id\": \"5e6ce60a-95c4-4fe5-8c4a-ec1d347afd83\"}\n",
+      "\n",
+      "event: data\n",
+      "data: {\"actions\":[{\"tool\":\"get_eugene_thoughts\",\"tool_input\":{\"query\":\"cats\"},\"log\":\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\",\"type\":\"AgentActionMessageLog\",\"message_log\":[{\"content\":\"\",\"additional_kwargs\":{\"function_call\":{\"name\":\"get_eugene_thoughts\",\"arguments\":\"{\\n  \\\"query\\\": \\\"cats\\\"\\n}\"}},\"type\":\"ai\",\"example\":false}]}],\"messages\":[{\"content\":\"\",\"additional_kwargs\":{\"function_call\":{\"name\":\"get_eugene_thoughts\",\"arguments\":\"{\\n  \\\"query\\\": \\\"cats\\\"\\n}\"}},\"type\":\"ai\",\"example\":false}]}\n",
+      "\n",
+      "event: data\n",
+      "data: {\"steps\":[{\"action\":{\"tool\":\"get_eugene_thoughts\",\"tool_input\":{\"query\":\"cats\"},\"log\":\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\",\"type\":\"AgentActionMessageLog\",\"message_log\":[{\"content\":\"\",\"additional_kwargs\":{\"function_call\":{\"name\":\"get_eugene_thoughts\",\"arguments\":\"{\\n  \\\"query\\\": \\\"cats\\\"\\n}\"}},\"type\":\"ai\",\"example\":false}]},\"observation\":[{\"page_content\":\"cats like fish\",\"metadata\":{},\"type\":\"Document\"},{\"page_content\":\"dogs like sticks\",\"metadata\":{},\"type\":\"Document\"}]}],\"messages\":[{\"content\":\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\",\"additional_kwargs\":{},\"type\":\"function\",\"name\":\"get_eugene_thoughts\"}]}\n",
+      "\n",
+      "event: data\n",
+      "data: {\"output\":\"Eugene thinks that cats like fish.\",\"messages\":[{\"content\":\"Eugene thinks that cats like fish.\",\"additional_kwargs\":{},\"type\":\"ai\",\"example\":false}]}\n",
+      "\n",
+      "event: end\n",
+      "\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "import requests\n",
+    "\n",
+    "inputs = {\"input\": {\"input\": \"what does eugene think of cats?\"}}\n",
+    "response = requests.post(\"http://localhost:8000/stream\", json=inputs)\n",
+    "\n",
+    "print(response.text)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can also interact with this via the RemoteRunnable interface (to use in other chains)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langserve import RemoteRunnable\n",
+    "\n",
+    "remote_runnable = RemoteRunnable(\"http://localhost:8000/\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Remote runnable has the same interface as local runnables"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{'output': 'Hello! How can I assist you today?', 'messages': [AIMessage(content='Hello! How can I assist you today?')]}\n"
+     ]
+    }
+   ],
+   "source": [
+    "async for chunk in remote_runnable.astream({\"input\": \"hi!\"}):\n",
+    "    print(chunk)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "RunLogPatch({'op': 'replace',\n",
+      "  'path': '',\n",
+      "  'value': {'final_output': None,\n",
+      "            'id': '9f415b49-ba69-4fdf-9b9c-5ccc1805487f',\n",
+      "            'logs': {},\n",
+      "            'streamed_output': []}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '95a0ba72-9511-4fff-8b8c-410e7108ee60',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'RunnableSequence',\n",
+      "            'start_time': '2024-01-06T03:12:42.213+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': [],\n",
+      "            'type': 'chain'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'a0afa5a6-a408-4a2d-83f8-8e4c8193f963',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'RunnableParallel<input,agent_scratchpad>',\n",
+      "            'start_time': '2024-01-06T03:12:42.214+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:1'],\n",
+      "            'type': 'chain'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '2aed7d13-2d0b-4c37-a4b3-d3f266bbf7e6',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': '<lambda>',\n",
+      "            'start_time': '2024-01-06T03:12:42.215+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['map:key:input'],\n",
+      "            'type': 'chain'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '9328b5cf-fd7b-48fe-affb-2973643189e9',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': '<lambda>',\n",
+      "            'start_time': '2024-01-06T03:12:42.215+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['map:key:agent_scratchpad'],\n",
+      "            'type': 'chain'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>/final_output',\n",
+      "  'value': {'output': 'what does eugene think about cats?'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/<lambda>/end_time',\n",
+      "  'value': '2024-01-06T03:12:42.217+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:2/final_output',\n",
+      "  'value': {'output': []}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/<lambda>:2/end_time',\n",
+      "  'value': '2024-01-06T03:12:42.217+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>/final_output',\n",
+      "  'value': {'agent_scratchpad': [],\n",
+      "            'input': 'what does eugene think about cats?'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>/end_time',\n",
+      "  'value': '2024-01-06T03:12:42.218+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'a08bec71-bf7f-46c5-b082-14f7bff421cd',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'ChatPromptTemplate',\n",
+      "            'start_time': '2024-01-06T03:12:42.219+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:2'],\n",
+      "            'type': 'prompt'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate/final_output',\n",
+      "  'value': {'messages': [SystemMessage(content='You are a helpful assistant.'),\n",
+      "                         HumanMessage(content='what does eugene think about cats?')]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate/end_time',\n",
+      "  'value': '2024-01-06T03:12:42.219+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/LLM',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'bf859261-467b-44b3-84fb-80d8c9e9dff2',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'LLM',\n",
+      "            'start_time': '2024-01-06T03:12:42.221+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:3'],\n",
+      "            'type': 'llm'}})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': ''}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '{\\n'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' '}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' \"'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': 'query'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '\":'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' \"'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': 'cats'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '\"\\n'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '}'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/LLM/final_output',\n",
+      "  'value': LLMResult(generations=[[ChatGeneration(generation_info={'finish_reason': 'function_call'}, message=AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}))]], llm_output=None, run=None)},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/end_time',\n",
+      "  'value': '2024-01-06T03:12:42.806+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'd635ccbb-1db3-4515-b1ca-0c14752d361a',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'OpenAIFunctionsAgentOutputParser',\n",
+      "            'start_time': '2024-01-06T03:12:42.807+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:4'],\n",
+      "            'type': 'parser'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser/final_output',\n",
+      "  'value': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser/end_time',\n",
+      "  'value': '2024-01-06T03:12:42.809+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence/final_output',\n",
+      "  'value': {'output': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence/end_time',\n",
+      "  'value': '2024-01-06T03:12:42.809+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'actions': [AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])],\n",
+      "            'messages': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]}},\n",
+      " {'op': 'replace',\n",
+      "  'path': '/final_output',\n",
+      "  'value': {'actions': [AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])],\n",
+      "            'messages': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/get_eugene_thoughts',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '5eecdbe8-3374-4f0e-8ae8-d8a7e2394ae1',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'get_eugene_thoughts',\n",
+      "            'start_time': '2024-01-06T03:12:42.810+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': [],\n",
+      "            'type': 'tool'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/get_eugene_thoughts/final_output',\n",
+      "  'value': {'output': \"[Document(page_content='cats like fish'), \"\n",
+      "                      \"Document(page_content='dogs like sticks')]\"}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/get_eugene_thoughts/end_time',\n",
+      "  'value': '2024-01-06T03:12:43.095+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'messages': [FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')],\n",
+      "            'steps': [{'action': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]),\n",
+      "                       'observation': [Document(page_content='cats like fish'),\n",
+      "                                       Document(page_content='dogs like sticks')]}]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/steps',\n",
+      "  'value': [{'action': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]),\n",
+      "             'observation': [Document(page_content='cats like fish'),\n",
+      "                             Document(page_content='dogs like sticks')]}]},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/messages/1',\n",
+      "  'value': FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '89bf8195-df48-4fde-81b0-9e1982051caa',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'RunnableSequence',\n",
+      "            'start_time': '2024-01-06T03:12:43.097+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': [],\n",
+      "            'type': 'chain'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '567c178a-4897-4865-af27-7122432df7bd',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'RunnableParallel<input,agent_scratchpad>',\n",
+      "            'start_time': '2024-01-06T03:12:43.098+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:1'],\n",
+      "            'type': 'chain'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:3',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '03530072-e5b2-4d7e-b875-f77e3d47481b',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': '<lambda>',\n",
+      "            'start_time': '2024-01-06T03:12:43.099+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['map:key:input'],\n",
+      "            'type': 'chain'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:4',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': 'c554e615-1b70-4dc1-a4aa-aa09ccdacb09',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': '<lambda>',\n",
+      "            'start_time': '2024-01-06T03:12:43.099+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['map:key:agent_scratchpad'],\n",
+      "            'type': 'chain'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:3/final_output',\n",
+      "  'value': {'output': 'what does eugene think about cats?'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/<lambda>:3/end_time',\n",
+      "  'value': '2024-01-06T03:12:43.100+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/<lambda>:4/final_output',\n",
+      "  'value': {'output': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}),\n",
+      "                       FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/<lambda>:4/end_time',\n",
+      "  'value': '2024-01-06T03:12:43.100+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>:2/final_output',\n",
+      "  'value': {'agent_scratchpad': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}),\n",
+      "                                 FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')],\n",
+      "            'input': 'what does eugene think about cats?'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/RunnableParallel<input,agent_scratchpad>:2/end_time',\n",
+      "  'value': '2024-01-06T03:12:43.101+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '724c5bf8-d475-45ec-a69a-a4f188dc405f',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'ChatPromptTemplate',\n",
+      "            'start_time': '2024-01-06T03:12:43.101+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:2'],\n",
+      "            'type': 'prompt'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate:2/final_output',\n",
+      "  'value': {'messages': [SystemMessage(content='You are a helpful assistant.'),\n",
+      "                         HumanMessage(content='what does eugene think about cats?'),\n",
+      "                         AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}),\n",
+      "                         FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/ChatPromptTemplate:2/end_time',\n",
+      "  'value': '2024-01-06T03:12:43.102+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/LLM:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '4542ae99-f854-41be-9669-41d8cd7dfb24',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'LLM',\n",
+      "            'start_time': '2024-01-06T03:12:43.103+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:3'],\n",
+      "            'type': 'llm'}})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': 'E'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='E')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': 'ug'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='ug')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': 'ene'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='ene')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' thinks'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' thinks')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' that'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' that')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' cats'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' cats')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' like'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' like')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' fish'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' fish')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': '.'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='.')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/LLM:2/final_output',\n",
+      "  'value': LLMResult(generations=[[ChatGeneration(text='Eugene thinks that cats like fish.', generation_info={'finish_reason': 'stop'}, message=AIMessage(content='Eugene thinks that cats like fish.'))]], llm_output=None, run=None)},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/end_time',\n",
+      "  'value': '2024-01-06T03:12:43.776+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '281c8ae8-dbc6-433f-86c8-20b86b3c8eee',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'OpenAIFunctionsAgentOutputParser',\n",
+      "            'start_time': '2024-01-06T03:12:43.776+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:4'],\n",
+      "            'type': 'parser'}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser:2/final_output',\n",
+      "  'value': AgentFinish(return_values={'output': 'Eugene thinks that cats like fish.'}, log='Eugene thinks that cats like fish.')},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/OpenAIFunctionsAgentOutputParser:2/end_time',\n",
+      "  'value': '2024-01-06T03:12:43.777+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence:2/final_output',\n",
+      "  'value': {'output': AgentFinish(return_values={'output': 'Eugene thinks that cats like fish.'}, log='Eugene thinks that cats like fish.')}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/RunnableSequence:2/end_time',\n",
+      "  'value': '2024-01-06T03:12:43.778+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'messages': [AIMessage(content='Eugene thinks that cats like fish.')],\n",
+      "            'output': 'Eugene thinks that cats like fish.'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/output',\n",
+      "  'value': 'Eugene thinks that cats like fish.'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/messages/2',\n",
+      "  'value': AIMessage(content='Eugene thinks that cats like fish.')})\n"
+     ]
+    }
+   ],
+   "source": [
+    "async for chunk in remote_runnable.astream_log({\"input\": \"what does eugene think about cats?\"}):\n",
+    "    print(chunk)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "RunLogPatch({'op': 'replace',\n",
+      "  'path': '',\n",
+      "  'value': {'final_output': None,\n",
+      "            'id': '51c65021-1b40-4a45-81b3-95fc5c5b545a',\n",
+      "            'logs': {},\n",
+      "            'streamed_output': []}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/LLM',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '7a34b2a3-d6c0-4a0b-9939-2bde70a98958',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'LLM',\n",
+      "            'start_time': '2024-01-06T03:12:43.812+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:3'],\n",
+      "            'type': 'llm'}})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': ''}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '{\\n'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' '}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' \"'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': 'query'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '\":'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': ' \"'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': 'cats'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '\"\\n'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='', additional_kwargs={'function_call': {'arguments': '}'}})})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/LLM/final_output',\n",
+      "  'value': LLMResult(generations=[[ChatGeneration(generation_info={'finish_reason': 'function_call'}, message=AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}}))]], llm_output=None, run=None)},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM/end_time',\n",
+      "  'value': '2024-01-06T03:12:44.588+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'actions': [AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])],\n",
+      "            'messages': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]}},\n",
+      " {'op': 'replace',\n",
+      "  'path': '/final_output',\n",
+      "  'value': {'actions': [AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})])],\n",
+      "            'messages': [AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]}})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'messages': [FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')],\n",
+      "            'steps': [{'action': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]),\n",
+      "                       'observation': [Document(page_content='cats like fish'),\n",
+      "                                       Document(page_content='dogs like sticks')]}]}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/steps',\n",
+      "  'value': [{'action': AgentActionMessageLog(tool='get_eugene_thoughts', tool_input={'query': 'cats'}, log=\"\\nInvoking: `get_eugene_thoughts` with `{'query': 'cats'}`\\n\\n\\n\", message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_eugene_thoughts', 'arguments': '{\\n  \"query\": \"cats\"\\n}'}})]),\n",
+      "             'observation': [Document(page_content='cats like fish'),\n",
+      "                             Document(page_content='dogs like sticks')]}]},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/messages/1',\n",
+      "  'value': FunctionMessage(content=\"[Document(page_content='cats like fish'), Document(page_content='dogs like sticks')]\", name='get_eugene_thoughts')})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/LLM:2',\n",
+      "  'value': {'end_time': None,\n",
+      "            'final_output': None,\n",
+      "            'id': '8ff3b248-fb7e-4305-bd57-54e21a85bacb',\n",
+      "            'metadata': {'__langserve_endpoint': 'stream_log',\n",
+      "                         '__langserve_version': '0.0.37',\n",
+      "                         '__useragent': 'python-httpx/0.25.2'},\n",
+      "            'name': 'LLM',\n",
+      "            'start_time': '2024-01-06T03:12:44.756+00:00',\n",
+      "            'streamed_output': [],\n",
+      "            'streamed_output_str': [],\n",
+      "            'tags': ['seq:step:3'],\n",
+      "            'type': 'llm'}})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': 'E'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='E')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': 'ug'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='ug')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': 'ene'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='ene')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' thinks'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' thinks')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' that'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' that')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' cats'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' cats')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' like'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' like')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ' fish'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content=' fish')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': '.'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='.')})\n",
+      "RunLogPatch({'op': 'add', 'path': '/logs/LLM:2/streamed_output_str/-', 'value': ''},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/streamed_output/-',\n",
+      "  'value': AIMessageChunk(content='')})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/logs/LLM:2/final_output',\n",
+      "  'value': LLMResult(generations=[[ChatGeneration(text='Eugene thinks that cats like fish.', generation_info={'finish_reason': 'stop'}, message=AIMessage(content='Eugene thinks that cats like fish.'))]], llm_output=None, run=None)},\n",
+      " {'op': 'add',\n",
+      "  'path': '/logs/LLM:2/end_time',\n",
+      "  'value': '2024-01-06T03:12:45.171+00:00'})\n",
+      "RunLogPatch({'op': 'add',\n",
+      "  'path': '/streamed_output/-',\n",
+      "  'value': {'messages': [AIMessage(content='Eugene thinks that cats like fish.')],\n",
+      "            'output': 'Eugene thinks that cats like fish.'}},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/output',\n",
+      "  'value': 'Eugene thinks that cats like fish.'},\n",
+      " {'op': 'add',\n",
+      "  'path': '/final_output/messages/2',\n",
+      "  'value': AIMessage(content='Eugene thinks that cats like fish.')})\n"
+     ]
+    }
+   ],
+   "source": [
+    "async for chunk in remote_runnable.astream_log({\"input\": \"what does eugene think about cats?\"}, include_names=[\"LLM\"]):\n",
+    "    print(chunk)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
@@ -0,0 +1,155 @@
+#!/usr/bin/env python
+"""An example that shows how to create a custom agent executor like Runnable.
+
+At the time of writing, there is a bug in the current AgentExecutor that
+prevents it from correctly propagating configuration of the underlying
+runnable. While that bug should be fixed, this is an example shows
+how to create a more complex custom runnable.
+
+Please see documentation for custom agent streaming here:
+
+https://python.langchain.com/docs/modules/agents/how_to/streaming#stream-tokens
+
+**ATTENTION**
+To support streaming individual tokens you will need to manually set the streaming=True
+on the LLM and use the stream_log endpoint rather than stream endpoint.
+"""
+from typing import Any, AsyncIterator, Dict, List, Optional, cast
+
+from fastapi import FastAPI
+from langchain.agents import AgentExecutor, tool
+from langchain.agents.format_scratchpad import format_to_openai_functions
+from langchain.agents.output_parsers import OpenAIFunctionsAgentOutputParser
+from langchain.chat_models import ChatOpenAI
+from langchain.embeddings import OpenAIEmbeddings
+from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
+from langchain.pydantic_v1 import BaseModel
+from langchain.tools.render import format_tool_to_openai_function
+from langchain.vectorstores import FAISS
+from langchain_core.runnables import (
+    ConfigurableField,
+    ConfigurableFieldSpec,
+    Runnable,
+    RunnableConfig,
+)
+from langchain_core.runnables.utils import Input, Output
+
+from langserve import add_routes
+
+vectorstore = FAISS.from_texts(
+    ["cats like fish", "dogs like sticks"], embedding=OpenAIEmbeddings()
+)
+retriever = vectorstore.as_retriever()
+
+
+@tool
+def get_eugene_thoughts(query: str) -> list:
+    """Returns Eugene's thoughts on a topic."""
+    return retriever.get_relevant_documents(query)
+
+
+tools = [get_eugene_thoughts]
+
+prompt = ChatPromptTemplate.from_messages(
+    [
+        ("system", "You are a helpful assistant."),
+        ("user", "{input}"),
+        MessagesPlaceholder(variable_name="agent_scratchpad"),
+    ]
+)
+
+# We need to set streaming=True on the LLM to support streaming individual tokens.
+# when using the stream_log endpoint.
+# .stream for agents streams action observation pairs not individual tokens.
+llm = ChatOpenAI(temperature=0, streaming=True).configurable_fields(
+    temperature=ConfigurableField(
+        id="llm_temperature",
+        name="LLM Temperature",
+        description="The temperature of the LLM",
+    )
+)
+
+llm_with_tools = llm.bind(
+    functions=[format_tool_to_openai_function(t) for t in tools]
+).with_config({"run_name": "LLM"})
+
+
+class CustomAgentExecutor(Runnable):
+    """A custom runnable that will be used by the agent executor."""
+
+    def __init__(self, **kwargs):
+        """Initialize the runnable."""
+        super().__init__(**kwargs)
+        self.agent = (
+            {
+                "input": lambda x: x["input"],
+                "agent_scratchpad": lambda x: format_to_openai_functions(
+                    x["intermediate_steps"]
+                ),
+            }
+            | prompt
+            | llm_with_tools
+            | OpenAIFunctionsAgentOutputParser()
+        )
+
+    def invoke(self, input: Input, config: Optional[RunnableConfig] = None) -> Output:
+        """Will not be used."""
+        raise NotImplementedError()
+
+    @property
+    def config_specs(self) -> List[ConfigurableFieldSpec]:
+        return self.agent.config_specs
+
+    async def astream(
+        self,
+        input: Input,
+        config: Optional[RunnableConfig] = None,
+        **kwargs: Optional[Any],
+    ) -> AsyncIterator[Output]:
+        """Stream the agent's output."""
+        configurable = cast(Dict[str, Any], config.pop("configurable", {}))
+
+        if configurable:
+            configured_agent = self.agent.with_config(
+                {
+                    "configurable": configurable,
+                }
+            )
+        else:
+            configured_agent = self.agent
+
+        executor = AgentExecutor(
+            agent=configured_agent,
+            tools=tools,
+        ).with_config({"run_name": "executor"})
+
+        async for output in executor.astream(input, config=config, **kwargs):
+            yield output
+
+
+app = FastAPI()
+
+
+# We need to add these input/output schemas because the current AgentExecutor
+# is lacking in schemas.
+class Input(BaseModel):
+    input: str
+
+
+class Output(BaseModel):
+    output: Any
+
+
+runnable = CustomAgentExecutor()
+
+# Add routes to the app for using the custom agent executor.
+add_routes(
+    app,
+    runnable.with_types(input_type=Input, output_type=Output),
+    disabled_endpoints=["invoke", "batch"],  # not implemented
+)
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="localhost", port=8000)
@@ -0,0 +1,301 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Local LLM\n",
+    "\n",
+    "Here, we'll use a server that's serving a local LLM.\n",
+    "\n",
+    "**Attention** This is OK for prototyping / dev usage, but should not be used for production cases when there might be concurrent requests from different users. As of the time of writing, Ollama is designed for single user and cannot handle concurrent requests see this issue: https://github.com/ollama/ollama/issues/358"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langchain.prompts.chat import ChatPromptTemplate"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from langserve import RemoteRunnable\n",
+    "\n",
+    "model = RemoteRunnable(\"http://localhost:8000/ollama/\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's test out the standard interface of a chat model."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "prompt = \"Tell me a 3 sentence story about a cat.\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content=\"\\nSure! Here is a three sentence story about a cat:\\n\\nMittens the cat purred contentedly on the windowsill, basking in the warm sunlight. Suddenly, a bird perched nearby and Mittens' ears perked up, ready to pounce. With lightning quick reflexes, Mittens leapt into the air, but the bird had flown away, leaving Mittens to settle for just lounging in the sun once again.\")"
+      ]
+     },
+     "execution_count": 17,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "model.invoke(prompt)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AIMessage(content=\"\\nSure! Here is a three sentence story about a cat:\\n\\nMittens the cat purred contentedly on the windowsill, basking in the warm sunlight. Suddenly, a bird flew by, catching Mittens' attention and causing her to leap into action. With lightning quick reflexes, Mittens pounced on the bird, but it flew away just in time, leaving Mittens frustrated but still purring happily.\")"
+      ]
+     },
+     "execution_count": 18,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "await model.ainvoke(prompt)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Batched API works, but b/c ollama does not support parallelism, it's no faster than using .invoke twice."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "CPU times: user 7.65 ms, sys: 6.57 ms, total: 14.2 ms\n",
+      "Wall time: 5.51 s\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[AIMessage(content='\\nSure! Here is a three sentence story about a cat:\\n\\nMr. Whiskers was a sleek black cat with bright green eyes. He spent his days lounging in the sunbeams that streamed through the living room window, chasing the occasional fly, and purring contentedly. Despite his lazy demeanor, Mr. Whiskers was always on the lookout for a warm lap to curl up in.'),\n",
+       " AIMessage(content='\\nSure! Here is a three sentence story about a cat:\\n\\nMittens the cat purred contentedly on the windowsill, basking in the warm sunlight that streamed through the glass. Suddenly, a tiny bird perched on the ledge outside, tweeting nervously as it eyed the cat with suspicion. Without hesitation, Mittens pounced, snatching the bird in mid-air and devouring it in one quick motion.')]"
+      ]
+     },
+     "execution_count": 21,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%%time\n",
+    "model.batch([prompt, prompt])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "CPU times: user 9.72 ms, sys: 7.59 ms, total: 17.3 ms\n",
+      "Wall time: 5.56 s\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%time\n",
+    "for _ in range(2):\n",
+    "    model.invoke(prompt)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[AIMessage(content=\"\\nSure, here's a three sentence story about a cat:\\n\\nMittens the cat purred contentedly on the windowsill, basking in the warm sunlight that streamed through the glass. Her bright green eyes sparkled as she watched a bird flit and flutter outside, wishing she could join it in its flight. Just then, her owner entered the room with a bowl of creamy milk, causing Mittens to jump down from the windowsill and rub against their legs in excitement.\"),\n",
+       " AIMessage(content='\\nSure! Here is a three sentence story about a cat:\\n\\nMittens the cat purred contentedly on my lap, her soft fur a soothing balm for my frazzled nerves. As I stroked her back, she gazed up at me with big, round eyes, purring even louder. It was hard to resist the charm of this little ball of fluff, and I found myself smiling and scratching behind her ears.')]"
+      ]
+     },
+     "execution_count": 23,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "await model.abatch([prompt, prompt])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Streaming is available by default"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 24,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "|S|ure|,| here| is| a| |3| sentence| story| about| a| cat|:|\n",
+      "|\n",
+      "|M|itt|ens| the| cat| pur|red| content|edly| on| the| windows|ill|,| her| tail| tw|itch|ing| as| she| watched| the| birds| outside|.| Sud|den|ly|,| a| squ|ir|rel| sc|am|per|ed| by| and| Mitt|ens| was| on| high| alert|,| her| ears| per|ked| up| and| ready| to| p|ounce|.| With| light|ning| quick| ref|lex|es|,| Mitt|ens| jump|ed| from| the| windows|ill| and| ch|ased| after| the| squ|ir|rel|,| her| tail| streaming| behind| her|.||"
+     ]
+    }
+   ],
+   "source": [
+    "for chunk in model.stream(prompt):\n",
+    "    print(chunk.content, end=\"|\", flush=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "|The| cat| pur|red| content|edly| on| my| lap|,| its| soft| fur| a| so|othing| bal|m| for| my| fra|zz|led| n|erves|.| As| I| st|rok|ed| its| back|,| it| gaz|ed| up| at| me| with| soul|ful| eyes|,| p|urr|ing| loud|ly| in| appro|val|.| In| that| moment|,| all| was| right| with| the| world|.||"
+     ]
+    }
+   ],
+   "source": [
+    "async for chunk in model.astream(prompt):\n",
+    "    print(chunk.content, end=\"|\", flush=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "And so is the event stream API"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{'event': 'on_chat_model_start', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'name': '/ollama', 'tags': [], 'metadata': {}, 'data': {'input': 'Tell me a 3 sentence story about a cat.'}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content='\\n')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content='S')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content='ure')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content=',')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content=' here')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content=' is')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content=' a')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content=' ')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content='3')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content=' sentence')}}\n",
+      "{'event': 'on_chat_model_stream', 'run_id': '6c2bdfc1-d482-4861-886c-c737a50771c3', 'tags': [], 'metadata': {}, 'name': '/ollama', 'data': {'chunk': AIMessageChunk(content=' story')}}\n",
+      "...\n"
+     ]
+    }
+   ],
+   "source": [
+    "i = 0\n",
+    "async for event in model.astream_events(prompt, version='v1'):\n",
+    "    print(event)\n",
+    "    if i > 10:\n",
+    "        print('...')\n",
+    "        break\n",
+    "    i += 1"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
@@ -0,0 +1,38 @@
+#!/usr/bin/env python
+"""Example LangChain Server that runs a local llm.
+
+**Attention** This is OK for prototyping / dev usage, but should not be used
+for production cases when there might be concurrent requests from different
+users. As of the time of writing, Ollama is designed for single user and cannot
+handle concurrent requests see this issue: 
+https://github.com/ollama/ollama/issues/358
+
+When deploying local models, make sure you understand whether the model is able
+to handle concurrent requests or not. If concurrent requests are not handled
+properly, the server will either crash or will just not be able to handle more
+than one user at a time.
+"""
+from fastapi import FastAPI
+from langchain_community.chat_models import ChatOllama
+
+from langserve import add_routes
+
+app = FastAPI(
+    title="LangChain Server",
+    version="1.0",
+    description="Spin up a simple api server using Langchain's Runnable interfaces",
+)
+
+
+llm = ChatOllama(model="llama2")
+
+add_routes(
+    app,
+    llm,
+    path="/ollama",
+)
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="localhost", port=8000)
@@ -0,0 +1,35 @@
+"""Client server that interacts with the main server via a remote runnable.
+
+This server sets up a simple proxy to the main server. It uses the RemoteRunnable
+to interact with the main server. The main server is expected to be running at
+http://localhost:8123.
+
+A client server will likely end up doing something more clever rather than
+just being a proxy.
+"""
+from fastapi import FastAPI
+
+from langserve import RemoteRunnable, add_routes
+
+app = FastAPI()
+
+MAIN_SERVER_URL = (
+    "http://localhost:8123/chat_model/"  # <-- URL of the RUNNABLE on the main server
+)
+# Type inference is not automatic for remote runnables at the moment,
+# so you must specify which types are used for the playground to work.
+remote_runnable = RemoteRunnable(MAIN_SERVER_URL).with_types(input_type=str)
+
+
+# Let's add an example chain
+add_routes(
+    app,
+    remote_runnable,
+    path="/proxied",
+)
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app)
@@ -0,0 +1,20 @@
+"""Main server that exposes one or more chains as HTTP endpoints."""
+from fastapi import FastAPI
+from langchain_openai import ChatOpenAI
+
+from langserve import add_routes
+
+app = FastAPI()
+
+# Let's add an example chain
+add_routes(
+    app,
+    ChatOpenAI(),
+    path="/chat_model",
+)
+
+if __name__ == "__main__":
+    import uvicorn
+
+    # Running on port 8123
+    uvicorn.run(app, port=8123)
@@ -0,0 +1,38 @@
+#!/usr/bin/env python
+"""Example LangChain Server that uses a Fast API Router.
+
+When applications grow, it becomes useful to use FastAPI's Router to organize
+the routes.
+
+See more documentation at:
+
+https://fastapi.tiangolo.com/tutorial/bigger-applications/
+"""
+from fastapi import APIRouter, FastAPI
+from langchain.chat_models import ChatAnthropic, ChatOpenAI
+
+from langserve import add_routes
+
+app = FastAPI()
+
+router = APIRouter(prefix="/models")
+
+# Invocations to this router will appear in trace logs as /models/openai
+add_routes(
+    router,
+    ChatOpenAI(),
+    path="/openai",
+)
+# Invocations to this router will appear in trace logs as /models/anthropic
+add_routes(
+    router,
+    ChatAnthropic(),
+    path="/anthropic",
+)
+
+app.include_router(router)
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="localhost", port=8000)
@@ -0,0 +1,64 @@
+#!/usr/bin/env python
+"""Example of a simple chatbot that just passes current conversation
+state back and forth between server and client.
+"""
+from typing import List, Union
+
+from fastapi import FastAPI
+from fastapi.middleware.cors import CORSMiddleware
+from langchain.chat_models import ChatAnthropic
+from langchain_core.messages import AIMessage, HumanMessage, SystemMessage
+from langchain_core.output_parsers import StrOutputParser
+from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
+
+from langserve import add_routes
+from langserve.pydantic_v1 import BaseModel, Field
+
+app = FastAPI(
+    title="LangChain Server",
+    version="1.0",
+    description="Spin up a simple api server using Langchain's Runnable interfaces",
+)
+
+
+# Set all CORS enabled origins
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+    expose_headers=["*"],
+)
+
+
+# Declare a chain
+prompt = ChatPromptTemplate.from_messages(
+    [
+        ("system", "You are a helpful assisstant named Cob."),
+        MessagesPlaceholder(variable_name="messages"),
+    ]
+)
+
+chain = prompt | ChatAnthropic(model="claude-2") | StrOutputParser()
+
+
+class InputChat(BaseModel):
+    """Input for the chat endpoint."""
+
+    messages: List[Union[HumanMessage, AIMessage, SystemMessage]] = Field(
+        ...,
+        description="The chat messages representing the current conversation.",
+        extra={"widget": {"type": "chat", "input": "messages"}},
+    )
+
+
+add_routes(
+    app,
+    chain.with_types(input_type=InputChat),
+)
+
+if __name__ == "__main__":
+    import uvicorn
+
+    uvicorn.run(app, host="localhost", port=8000)
@@ -51,11 +51,13 @@ from langserve.validation import (
    BatchRequestShallowValidator,
    InvokeBaseResponse,
    InvokeRequestShallowValidator,
+    StreamEventsParameters,
    StreamLogParameters,
    create_batch_request_model,
    create_batch_response_model,
    create_invoke_request_model,
    create_invoke_response_model,
+    create_stream_events_request_model,
    create_stream_log_request_model,
    create_stream_request_model,
 )
@@ -94,6 +96,14 @@ PerRequestConfigModifier = Union[
 ]


+def _strip_internal_keys(metadata: Dict[str, Any]) -> Dict[str, Any]:
+    """Strip out internal metadata keys that should not be sent to the client.
+
+    These keys are defined to be any key that starts with "__".
+    """
+    return {k: v for k, v in metadata.items() if not k.startswith("__")}
+
+
 async def _unpack_request_config(
    *client_sent_configs: Union[BaseModel, Mapping, str],
    config_keys: Sequence[str],
@@ -161,7 +171,7 @@ async def _unpack_request_config(


 def _update_config_with_defaults(
-    path: str,
+    run_name: str,
    incoming_config: RunnableConfig,
    request: Request,
    *,
@@ -199,7 +209,7 @@ def _update_config_with_defaults(
        metadata.update(hosted_metadata)

    non_overridable_default_config = RunnableConfig(
-        run_name=path,
+        run_name=run_name,
        metadata=metadata,
    )

@@ -498,6 +508,11 @@ class APIHandler:
                credentials to a runnable. The RunnableConfig is presented in its
                dictionary form. Note that only keys in `config_keys` will be
                modifiable by this function.
+            stream_log_name_allow_list: optional list of names of logs that can be
+                streamed by the stream_log endpoint.
+                If not provided, then all logs will be allowed to be streamed.
+                Use to also limit the events that can be streamed by the stream_events.
+                TODO: Introduce deprecation for this parameter to rename it
        """
        if importlib.util.find_spec("sse_starlette") is None:
            raise ImportError(
@@ -519,13 +534,21 @@ class APIHandler:

        self._config_keys = config_keys

-        self._path = path
-        self._base_url = prefix + path
+        self._path = path.rstrip("/")
+        self._base_url = prefix + self._path
+        # Setting the run name explicitly
+        # the run name is set to the base url, which takes into account
+        # the prefix (e.g., if there's an APIRouter used) and the path relative
+        # to the router.
+        # If the base path is /foo/bar, the run name will be /foo/bar
+        # and when tracing information is logged, we'll be able to see
+        # traces for the path /foo/bar.
+        self._run_name = self._base_url
        self._include_callback_events = include_callback_events
        self._per_req_config_modifier = per_req_config_modifier
        self._serializer = WellKnownLCSerializer()
        self._enable_feedback_endpoint = enable_feedback_endpoint
-        self._stream_log_name_allow_list = stream_log_name_allow_list
+        self._names_in_stream_allow_list = stream_log_name_allow_list

        # Client is patched using mock.patch, if changing the names
        # remember to make relevant updates in the unit tests.
@@ -576,6 +599,10 @@ class APIHandler:
        self._StreamLogRequest = create_stream_log_request_model(
            model_namespace, input_type_, self._ConfigPayload
        )
+
+        self._StreamEventsRequest = create_stream_events_request_model(
+            model_namespace, input_type_, self._ConfigPayload
+        )
        # Generate the response models
        self._InvokeResponse = create_invoke_response_model(
            model_namespace, output_type_
@@ -602,6 +629,11 @@ class APIHandler:
        """Return the stream log request model."""
        return self._StreamLogRequest

+    @property
+    def StreamEventsRequest(self) -> Type[BaseModel]:
+        """Return the stream events request model."""
+        return self._StreamEventsRequest
+
    @property
    def InvokeResponse(self) -> Type[BaseModel]:
        """Return the invoke response model."""
@@ -639,7 +671,7 @@ class APIHandler:
                server_config=server_config,
            )
            config = _update_config_with_defaults(
-                self._path,
+                self._run_name,
                user_provided_config,
                request,
                endpoint=endpoint,
@@ -790,7 +822,7 @@ class APIHandler:
            _add_callbacks(config_, [aggregator])
            final_configs.append(
                _update_config_with_defaults(
-                    self._path, config_, request, endpoint="batch"
+                    self._run_name, config_, request, endpoint="batch"
                )
            )

@@ -1045,9 +1077,9 @@ class APIHandler:
                            f"Expected a RunLog instance got {type(chunk)}"
                        )
                    if (
-                        self._stream_log_name_allow_list is None
+                        self._names_in_stream_allow_list is None
                        or self._runnable.config.get("run_name")
-                        in self._stream_log_name_allow_list
+                        in self._names_in_stream_allow_list
                    ):
                        data = {
                            "ops": chunk.ops,
@@ -1076,6 +1108,119 @@ class APIHandler:

        return EventSourceResponse(_stream_log())

+    async def astream_events(
+        self,
+        request: Request,
+        *,
+        config_hash: str = "",
+        server_config: Optional[RunnableConfig] = None,
+    ) -> EventSourceResponse:
+        """Stream events from the runnable."""
+        err_event = {}
+        validation_exception: Optional[BaseException] = None
+        try:
+            config, input_ = await self._get_config_and_input(
+                request,
+                config_hash,
+                endpoint="stream_events",
+                server_config=server_config,
+            )
+        except BaseException as e:
+            validation_exception = e
+            if isinstance(e, RequestValidationError):
+                err_event = {
+                    "event": "error",
+                    "data": json.dumps(
+                        {"status_code": 422, "message": repr(e.errors())}
+                    ),
+                }
+            else:
+                err_event = {
+                    "event": "error",
+                    # Do not expose the error message to the client since
+                    # the message may contain sensitive information.
+                    "data": json.dumps(
+                        {"status_code": 500, "message": "Internal Server Error"}
+                    ),
+                }
+
+        try:
+            body = await request.json()
+            with _with_validation_error_translation():
+                stream_events_request = StreamEventsParameters(**body)
+        except json.JSONDecodeError:
+            # Body as text
+            validation_exception = RequestValidationError(errors=["Invalid JSON body"])
+            err_event = {
+                "event": "error",
+                "data": json.dumps(
+                    {"status_code": 422, "message": "Invalid JSON body"}
+                ),
+            }
+        except RequestValidationError as e:
+            validation_exception = e
+            err_event = {
+                "event": "error",
+                "data": json.dumps({"status_code": 422, "message": repr(e.errors())}),
+            }
+
+        async def _stream_events() -> AsyncIterator[dict]:
+            """Stream the output of the runnable."""
+            if not hasattr(self._runnable, "astream_events"):
+                raise NotImplementedError(
+                    "Please upgrade langchain-core>=0.1.14 to use astream_events"
+                )
+
+            if validation_exception:
+                yield err_event
+                if isinstance(validation_exception, RequestValidationError):
+                    return
+                else:
+                    raise AssertionError(
+                        "Internal server error"
+                    ) from validation_exception
+
+            try:
+                async for event in self._runnable.astream_events(
+                    input_,
+                    config=config,
+                    include_names=stream_events_request.include_names,
+                    include_types=stream_events_request.include_types,
+                    include_tags=stream_events_request.include_tags,
+                    exclude_names=stream_events_request.exclude_names,
+                    exclude_types=stream_events_request.exclude_types,
+                    exclude_tags=stream_events_request.exclude_tags,
+                    version="v1",
+                ):
+                    if (
+                        self._names_in_stream_allow_list is None
+                        or self._runnable.config.get("run_name")
+                        in self._names_in_stream_allow_list
+                    ):
+                        # Strip internal metadata from the event
+                        event["metadata"] = _strip_internal_keys(event["metadata"])
+                        yield {
+                            # EventSourceResponse expects a string for data
+                            # so after serializing into bytes, we decode into utf-8
+                            # to get a string.
+                            "data": self._serializer.dumps(event).decode("utf-8"),
+                            "event": "data",
+                        }
+                yield {"event": "end"}
+            except BaseException:
+                yield {
+                    "event": "error",
+                    # Do not expose the error message to the client since
+                    # the message may contain sensitive information.
+                    # We'll add client side errors for validation as well.
+                    "data": json.dumps(
+                        {"status_code": 500, "message": "Internal Server Error"}
+                    ),
+                }
+                raise
+
+        return EventSourceResponse(_stream_events())
+
    async def input_schema(
        self,
        request: Request,
@@ -1099,7 +1244,7 @@ class APIHandler:
                server_config=server_config,
            )
            config = _update_config_with_defaults(
-                self._path, user_provided_config, request
+                self._run_name, user_provided_config, request
            )

        return self._runnable.get_input_schema(config).schema()
@@ -1127,7 +1272,7 @@ class APIHandler:
                server_config=server_config,
            )
            config = _update_config_with_defaults(
-                self._path, user_provided_config, request
+                self._run_name, user_provided_config, request
            )
        return self._runnable.get_output_schema(config).schema()

@@ -1154,7 +1299,7 @@ class APIHandler:
                server_config=server_config,
            )
            config = _update_config_with_defaults(
-                self._path, user_provided_config, request
+                self._run_name, user_provided_config, request
            )
        return (
            self._runnable.with_config(config)
@@ -1187,16 +1332,16 @@ class APIHandler:
            )

            config = _update_config_with_defaults(
-                self._path, user_provided_config, request
+                self._run_name, user_provided_config, request
            )

+        playground_url = (
+            request.scope.get("root_path", "").rstrip("/")
+            + self._base_url
+            + "/playground"
+        )
        feedback_enabled = tracing_is_enabled() and self._enable_feedback_endpoint

-        if self._base_url.endswith("/"):
-            playground_url = self._base_url + "playground"
-        else:
-            playground_url = self._base_url + "/playground"
-
        return await serve_playground(
            self._runnable.with_config(config),
            self._runnable.with_config(config).input_schema,
@@ -36,6 +36,8 @@ from langchain.schema.runnable.config import (
    get_callback_manager_for_config,
 )
 from langchain.schema.runnable.utils import AddableDict, Input, Output
+from langchain_core.runnables.schema import StreamEvent
+from typing_extensions import Literal

 from langserve.callbacks import CallbackEventDict, ahandle_callbacks, handle_callbacks
 from langserve.serialization import (
@@ -47,10 +49,68 @@ from langserve.serialization import (
 logger = logging.getLogger(__name__)


-def _without_callbacks(config: Optional[RunnableConfig]) -> RunnableConfig:
-    """Evict callbacks from the config since those are definitely not supported."""
+def _is_json_serializable(obj: Any) -> bool:
+    """Return True if the object is json serializable."""
+    if isinstance(obj, (tuple, list, dict, str, int, float, bool, type(None))):
+        return True
+    else:
+        return False
+
+
+def _keep_json_serializable(obj: Any) -> Any:
+    """Traverse the object recursively and removes non-json serializable objects."""
+    if isinstance(obj, dict):
+        return {
+            k: _keep_json_serializable(v)
+            for k, v in obj.items()
+            if isinstance(k, str) and _is_json_serializable(v)
+        }
+    elif isinstance(obj, (list, tuple)):
+        return [_keep_json_serializable(v) for v in obj if _is_json_serializable(v)]
+    elif _is_json_serializable(obj):
+        return obj
+    else:
+        raise AssertionError("This code should not be reachable. If it's reached")
+
+
+def _prepare_config_for_server(
+    config: Optional[RunnableConfig], *, ignore_unserializable: bool = True
+) -> RunnableConfig:
+    """Evict information from the config that should not be sent to the server.
+
+    This includes:
+    - callbacks: Callbacks are handled separately
+    - non-json serializable objects: We cannot serialize then the correct behavior
+        these appear frequently in the config of the runnable but are only needed
+        in the local scope of the config (they do not need to be sent to the server).
+        An example are the write / read channel objects populated by langgraph,
+        or the 'messages' field in configurable populated by RunnableWithMessageHistory.
+
+    Args:
+        config: The config to clean up
+        ignore_unserializable: If True, will ignore non-json serializable objects
+            found in the 'configurable' field of the config.
+            This is expected to be the safe default to use since the server
+            should not be specifying configurable objects that are not json
+            serializable. This logic is expected mostly to with non serializable
+            content that was created for local use by the runnable, and
+            is not needed by the server.
+            If False, will raise an error if a non-json serializable object is found.
+
+    Returns:
+        A cleaned up version of the config that can be sent to the server.
+    """
    _config = config or {}
-    return {k: v for k, v in _config.items() if k != "callbacks"}
+    without_callbacks = {k: v for k, v in _config.items() if k != "callbacks"}
+    if "configurable" in without_callbacks:
+        # Get a version of
+
+        if ignore_unserializable:
+            without_callbacks["configurable"] = _keep_json_serializable(
+                without_callbacks["configurable"]
+            )
+
+    return without_callbacks


@lru_cache(maxsize=1_000)  # Will accommodate up to 1_000 different error messages
@@ -275,7 +335,7 @@ class RemoteRunnable(Runnable[Input, Output]):
            "/invoke",
            json={
                "input": self._lc_serializer.dumpd(input),
-                "config": _without_callbacks(config),
+                "config": _prepare_config_for_server(config),
                "kwargs": kwargs,
            },
        )
@@ -306,7 +366,7 @@ class RemoteRunnable(Runnable[Input, Output]):
            "/invoke",
            json={
                "input": self._lc_serializer.dumpd(input),
-                "config": _without_callbacks(config),
+                "config": _prepare_config_for_server(config),
                "kwargs": kwargs,
            },
        )
@@ -341,9 +401,9 @@ class RemoteRunnable(Runnable[Input, Output]):
            )

        if isinstance(config, list):
-            _config = [_without_callbacks(c) for c in config]
+            _config = [_prepare_config_for_server(c) for c in config]
        else:
-            _config = _without_callbacks(config)
+            _config = _prepare_config_for_server(config)

        response = self.sync_client.post(
            "/batch",
@@ -394,9 +454,9 @@ class RemoteRunnable(Runnable[Input, Output]):
            )

        if isinstance(config, list):
-            _config = [_without_callbacks(c) for c in config]
+            _config = [_prepare_config_for_server(c) for c in config]
        else:
-            _config = _without_callbacks(config)
+            _config = _prepare_config_for_server(config)

        response = await self.async_client.post(
            "/batch",
@@ -458,7 +518,7 @@ class RemoteRunnable(Runnable[Input, Output]):
        )
        data = {
            "input": self._lc_serializer.dumpd(input),
-            "config": _without_callbacks(config),
+            "config": _prepare_config_for_server(config),
            "kwargs": kwargs,
        }
        endpoint = urljoin(self.url, "stream")
@@ -544,7 +604,7 @@ class RemoteRunnable(Runnable[Input, Output]):
        )
        data = {
            "input": self._lc_serializer.dumpd(input),
-            "config": _without_callbacks(config),
+            "config": _prepare_config_for_server(config),
            "kwargs": kwargs,
        }
        endpoint = urljoin(self.url, "stream")
@@ -646,7 +706,7 @@ class RemoteRunnable(Runnable[Input, Output]):
        )
        data = {
            "input": self._lc_serializer.dumpd(input),
-            "config": _without_callbacks(config),
+            "config": _prepare_config_for_server(config),
            "kwargs": kwargs,
            "diff": True,
            "include_names": include_names,
@@ -699,3 +759,102 @@ class RemoteRunnable(Runnable[Input, Output]):
            raise
        else:
            await run_manager.on_chain_end(final_output)
+
+    async def astream_events(
+        self,
+        input: Any,
+        config: Optional[RunnableConfig] = None,
+        *,
+        version: Literal["v1"],
+        include_names: Optional[Sequence[str]] = None,
+        include_types: Optional[Sequence[str]] = None,
+        include_tags: Optional[Sequence[str]] = None,
+        exclude_names: Optional[Sequence[str]] = None,
+        exclude_types: Optional[Sequence[str]] = None,
+        exclude_tags: Optional[Sequence[str]] = None,
+        **kwargs: Any,
+    ) -> AsyncIterator[StreamEvent]:
+        """Stream events from the server runnable.
+
+        **Attention**: This method is using a beta API and may change slightly.
+
+        This method can stream events from any step used in the runnable exposed
+        on the server. This includes all inner runs of LLMs, Retrievers, Tools, etc.
+
+        **Recommended**: Only ask for the data you need. This can significantly
+        reduce the amount of data sent over the wire.
+
+        Args:
+            input: The input to the runnable
+            config: The config to use for the runnable
+            version: The version of the astream_events to use.
+                     Currently only "v1" is supported.
+            include_names: The names of the events to include
+            include_types: The types of the events to include
+            include_tags: The tags of the events to include
+            exclude_names: The names of the events to exclude
+            exclude_types: The types of the events to exclude
+            exclude_tags: The tags of the events to exclude
+        """
+        if version != "v1":
+            raise ValueError(f"Unsupported version: {version}. Use 'v1'")
+
+        # Create a stream handler that will emit Log objects
+        config = ensure_config(config)
+        callback_manager = get_async_callback_manager_for_config(config)
+
+        events = []
+
+        run_manager = await callback_manager.on_chain_start(
+            dumpd(self),
+            self._lc_serializer.dumpd(input),
+            name=config.get("run_name"),
+        )
+        data = {
+            "input": self._lc_serializer.dumpd(input),
+            "config": _prepare_config_for_server(config),
+            "kwargs": kwargs,
+            "include_names": include_names,
+            "include_types": include_types,
+            "include_tags": include_tags,
+            "exclude_names": exclude_names,
+            "exclude_types": exclude_types,
+            "exclude_tags": exclude_tags,
+        }
+        endpoint = urljoin(self.url, "stream_events")
+
+        try:
+            from httpx_sse import aconnect_sse
+        except ImportError:
+            raise ImportError("You must install `httpx_sse` to use the stream method.")
+
+        try:
+            async with aconnect_sse(
+                self.async_client, "POST", endpoint, json=data
+            ) as event_source:
+                async for sse in event_source.aiter_sse():
+                    if sse.event == "data":
+                        event = self._lc_serializer.loads(sse.data)
+                        # Create a copy of the data to yield since underlying
+                        # code is using jsonpatch which does some stuff in-place
+                        # that can cause unexpected consequences.
+                        yield event
+                        events.append(event)
+                    elif sse.event == "error":
+                        # This can only be a server side error
+                        _raise_exception_from_data(
+                            sse.data, httpx.Request(method="POST", url=endpoint)
+                        )
+                    elif sse.event == "end":
+                        break
+                    else:
+                        _log_error_message_once(
+                            f"Encountered an unsupported event type: `{sse.event}`. "
+                            f"Try upgrading the remote client to the latest version."
+                            f"Ignoring events of type `{sse.event}`."
+                        )
+        except BaseException as e:
+            await run_manager.on_chain_error(e)
+            raise
+        else:
+            await run_manager.on_chain_end(events)
@@ -5,7 +5,7 @@
    <link rel="icon" href="/____LANGSERVE_BASE_URL/favicon.ico" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Playground</title>
-    <script type="module" crossorigin src="/____LANGSERVE_BASE_URL/assets/index-6a0f524c.js"></script>
+    <script type="module" crossorigin src="/____LANGSERVE_BASE_URL/assets/index-a7c0cdaa.js"></script>
    <link rel="stylesheet" href="/____LANGSERVE_BASE_URL/assets/index-52e8ab2f.css">
  </head>
  <body>
@@ -3,6 +3,7 @@
  "private": true,
  "version": "0.0.0",
  "type": "module",
+  "packageManager": "yarn@1.22.19",
  "scripts": {
    "dev": "vite",
    "build": "tsc && vite build",
@@ -0,0 +1,189 @@
+import * as ToggleGroup from "@radix-ui/react-toggle-group";
+import { AutosizeTextarea } from "./AutosizeTextarea";
+import TrashIcon from "../assets/TrashIcon.svg?react";
+import CodeIcon from "../assets/CodeIcon.svg?react";
+import ChatIcon from "../assets/ChatIcon.svg?react";
+
+export interface MessageFields {
+  content: string;
+  additional_kwargs?: { [key: string]: unknown };
+  name?: string;
+  type?: string;
+  role?: string;
+}
+
+export interface ChatMessageInputArgs {
+  handleChange: (field: string, value: any) => void;
+  handleRemoval: () => void;
+  path: string;
+  message: MessageFields;
+};
+
+function isOpenAiFunctionCall(
+  x: unknown
+): x is { name: string; arguments: string } {
+  if (typeof x !== "object" || x == null) return false;
+  if (!("name" in x) || typeof x.name !== "string") return false;
+  if (!("arguments" in x) || typeof x.arguments !== "string") return false;
+  return true;
+}
+
+export const ChatMessageInput = (props: ChatMessageInputArgs) => {
+  const { message } = props;
+  const isAiFunctionCall = isOpenAiFunctionCall(
+    message.additional_kwargs?.function_call
+  );
+  const type = message.type ?? "chat"
+  return (
+    <div className="control group">
+      <div className="flex items-start justify-between gap-2">
+        <select
+          className="-ml-1 min-w-[100px]"
+          value={type}
+          onChange={(e) => {
+            props.handleChange(
+              "type",
+              e.target.value
+            );
+          }}
+        >
+          <option value="human">Human</option>
+          <option value="ai">AI</option>
+          <option value="system">System</option>
+          <option value="function">Function</option>
+
+          <option value="chat">Chat</option>
+        </select>
+        <div className="flex items-center gap-2">
+          {message.type === "ai" && (
+            <ToggleGroup.Root
+              type="single"
+              aria-label="Message Type"
+              className="opacity-0 transition-opacity group-focus-within:opacity-100 group-hover:opacity-100"
+              value={isAiFunctionCall ? "function" : "text"}
+              onValueChange={(value) => {
+                switch (value) {
+                  case "function": {
+                    props.handleChange(
+                      "additional_kwargs",
+                      {
+                        function_call: {
+                          name: "",
+                          arguments: "{}",
+                        },
+                      }
+                    );
+
+                    break;
+                  }
+                  case "text": {
+                    props.handleChange(
+                      "additional_kwargs",
+                      {}
+                    );
+
+                    break;
+                  }
+                }
+              }}
+            >
+              <ToggleGroup.Item
+                className="rounded-s border border-divider-700 px-2.5 py-1 data-[state=on]:bg-divider-500/50"
+                value="text"
+                aria-label="Text message"
+              >
+                <ChatIcon className="w-4 h-4" />
+              </ToggleGroup.Item>
+              <ToggleGroup.Item
+                className="rounded-e border border-l-0 border-divider-700 px-2.5 py-1 data-[state=on]:bg-divider-500/50"
+                value="function"
+                aria-label="Function call"
+              >
+                <CodeIcon className="w-4 h-4" />
+              </ToggleGroup.Item>
+            </ToggleGroup.Root>
+          )}
+
+          <button
+            className="p-1 border rounded opacity-0 transition-opacity border-divider-700 group-focus-within:opacity-100 group-hover:opacity-100"
+            onClick={props.handleRemoval}
+          >
+            <TrashIcon className="w-4 h-4" />
+          </button>
+        </div>
+      </div>
+
+      {type === "chat" && (
+        <input
+          className="mb-1"
+          placeholder="Role"
+          value={message.role ?? ""}
+          onChange={(e) => {
+            props.handleChange(
+              "role",
+              e.target.value
+            );
+          }}
+        />
+      )}
+
+      {type === "function" && (
+        <input
+          className="mb-1"
+          placeholder="Function Name"
+          value={message.name ?? ""}
+          onChange={(e) => {
+            props.handleChange(
+              "name",
+              e.target.value
+            );
+          }}
+        />
+      )}
+
+      {type === "ai" &&
+      isOpenAiFunctionCall(
+        message.additional_kwargs?.function_call
+      ) ? (
+        <>
+          <input
+            className="mb-1"
+            placeholder="Function Name"
+            value={
+              message.additional_kwargs?.function_call.name ?? ""
+            }
+            onChange={(e) => {
+              props.handleChange(
+                "additional_kwargs.function_call.name",
+                e.target.value
+              );
+            }}
+          />
+
+          <AutosizeTextarea
+            value={
+              message.additional_kwargs?.function_call?.arguments ??
+              ""
+            }
+            onChange={(content) => {
+              props.handleChange(
+                "additional_kwargs.function_call.arguments",
+                content
+              );
+            }}
+          />
+        </>
+      ) : (
+        <AutosizeTextarea
+          value={message.content}
+          onChange={(content) => {
+            props.handleChange(
+              "content",
+              content
+            );
+          }}
+        />
+      )}
+    </div>
+  );
+}
@@ -1,8 +1,5 @@
 import { withJsonFormsControlProps } from "@jsonforms/react";
 import PlusIcon from "../assets/PlusIcon.svg?react";
-import TrashIcon from "../assets/TrashIcon.svg?react";
-import CodeIcon from "../assets/CodeIcon.svg?react";
-import ChatIcon from "../assets/ChatIcon.svg?react";
 import {
  rankWith,
  and,
@@ -10,11 +7,10 @@ import {
  Paths,
  isControl,
 } from "@jsonforms/core";
-import { AutosizeTextarea } from "./AutosizeTextarea";
 import { useStreamCallback } from "../useStreamCallback";
-import { getNormalizedJsonPath, traverseNaiveJsonPath } from "../utils/path";
 import { isJsonSchemaExtra } from "../utils/schema";
-import * as ToggleGroup from "@radix-ui/react-toggle-group";
+import { MessageFields, ChatMessageInput } from "./ChatMessageInput";
+import { useEffect } from "react";

 export const chatMessagesTester = rankWith(
  12,
@@ -61,98 +57,48 @@ export const chatMessagesTester = rankWith(
  )
 );

-interface MessageFields {
-  content: string;
-  additional_kwargs?: { [key: string]: unknown };
-  name?: string;
-  type?: string;
-
-  role?: string;
-}
-
-function isMessageFields(x: unknown): x is MessageFields {
-  if (typeof x !== "object" || x == null) return false;
-  if (!("content" in x) || typeof x.content !== "string") return false;
-  if (
-    "additional_kwargs" in x &&
-    typeof x.additional_kwargs !== "object" &&
-    x.additional_kwargs != null
-  )
-    return false;
-  if ("name" in x && typeof x.name !== "string" && x.name != null) return false;
-  if ("type" in x && typeof x.type !== "string" && x.type != null) return false;
-  if ("role" in x && typeof x.role !== "string" && x.role != null) return false;
-  return true;
-}
-
-function constructMessage(
-  x: unknown,
-  assumedRole: string
-): Array<MessageFields> | null {
-  if (typeof x === "string") {
-    return [{ content: x, type: assumedRole }];
-  }
-
-  if (isMessageFields(x)) {
-    return [x];
-  }
-
-  if (Array.isArray(x) && x.every(isMessageFields)) {
-    return x;
-  }
-
-  return null;
-}
-
-function isOpenAiFunctionCall(
-  x: unknown
-): x is { name: string; arguments: string } {
-  if (typeof x !== "object" || x == null) return false;
-  if (!("name" in x) || typeof x.name !== "string") return false;
-  if (!("arguments" in x) || typeof x.arguments !== "string") return false;
-  return true;
-}
-
 export const ChatMessagesControlRenderer = withJsonFormsControlProps(
  (props) => {
    const data: Array<MessageFields> = props.data ?? [];

-    useStreamCallback("onSuccess", (ctx) => {
+    useEffect(() => {
      if (!isJsonSchemaExtra(props.schema)) return;
-      const widget = props.schema.extra.widget;
-      if (!("input" in widget) && !("output" in widget)) return;
+      if (props.schema.extra.widget.type !== "chat") return;
+      setTimeout(() => props.handleChange(props.path, [
+        ...data,
+        { content: "", type: "human" },
+      ]), 10);
+    }, []);

-      const inputPath = getNormalizedJsonPath(widget.input ?? "");
-      const outputPath = getNormalizedJsonPath(widget.output ?? "");
+    useStreamCallback("onStart", () => {
+      if (!isJsonSchemaExtra(props.schema)) return;
+      if (props.schema.extra.widget.type !== "chat") return;
+      props.handleChange(props.path, [...data, { content: "", type: "ai" }]);
+    });

-      const human = traverseNaiveJsonPath(ctx.input, inputPath);
-      let ai = traverseNaiveJsonPath(ctx.output, outputPath);
-
-      const isSingleOutputKey =
-        ctx.output != null &&
-        Object.keys(ctx.output).length === 1 &&
-        Object.keys(ctx.output)[0] === "output";
-
-      if (isSingleOutputKey) {
-        ai = traverseNaiveJsonPath(ai, ["output", ...outputPath]) ?? ai;
+    useStreamCallback("onChunk", (_chunk, aggregatedState) => {
+      if (!isJsonSchemaExtra(props.schema)) return;
+      if (props.schema.extra.widget.type !== "chat") return;
+      if (aggregatedState?.final_output !== undefined) {
+        const msgPath = Paths.compose(props.path, `${data.length - 1}`);
+        if ((aggregatedState.final_output as MessageFields)?.type === "AIMessageChunk") {
+          props.handleChange(
+            Paths.compose(msgPath, "content"),
+            (aggregatedState.final_output as MessageFields)?.content
+          );
+        } else if (typeof aggregatedState.final_output === "string") {
+          props.handleChange(
+            Paths.compose(msgPath, "content"),
+            aggregatedState.final_output
+          );
+        }
      }
+    });

-      const humanMsg = constructMessage(human, "human");
-      const aiMsg = constructMessage(ai, "ai");
-
-      let newMessages = undefined;
-      if (humanMsg != null) {
-        newMessages ??= [...data];
-        newMessages.push(...humanMsg);
-      }
-      if (aiMsg != null) {
-        newMessages ??= [...data];
-        newMessages.push(...aiMsg);
-      }
-
-      if (newMessages != null) {
-        props.handleChange(props.path, newMessages);
-      }
+    useStreamCallback("onSuccess", () => {
+      if (!isJsonSchemaExtra(props.schema)) return;
+      if (props.schema.extra.widget.type !== "chat") return;
+      props.handleChange(props.path, [...data, { content: "", type: "human" }]);
    });

    return (
@@ -178,174 +124,25 @@ export const ChatMessagesControlRenderer = withJsonFormsControlProps(
        <div className="flex flex-col gap-3 mt-1 empty:hidden">
          {data.map((message, index) => {
            const msgPath = Paths.compose(props.path, `${index}`);
-            const type = message.type ?? "chat";

-            const isAiFunctionCall = isOpenAiFunctionCall(
-              message.additional_kwargs?.function_call
-            );
+            const handleChatMessageChange = (field: string, value: any) => {
+              props.handleChange(Paths.compose(msgPath, field), value);
+            };

+            const handleChatMessageRemoval = () => {
+              props.handleChange(
+                props.path,
+                data.filter((_, i) => i !== index)
+              );
+            }
            return (
-              <div className="control group" key={index}>
-                <div className="flex items-start justify-between gap-2">
-                  <select
-                    className="-ml-1 min-w-[100px]"
-                    value={type}
-                    onChange={(e) => {
-                      props.handleChange(
-                        Paths.compose(msgPath, "type"),
-                        e.target.value
-                      );
-                    }}
-                  >
-                    <option value="human">Human</option>
-                    <option value="ai">AI</option>
-                    <option value="system">System</option>
-                    <option value="function">Function</option>
-
-                    <option value="chat">Chat</option>
-                  </select>
-                  <div className="flex items-center gap-2">
-                    {message.type === "ai" && (
-                      <ToggleGroup.Root
-                        type="single"
-                        aria-label="Message Type"
-                        className="opacity-0 transition-opacity group-focus-within:opacity-100 group-hover:opacity-100"
-                        value={isAiFunctionCall ? "function" : "text"}
-                        onValueChange={(value) => {
-                          switch (value) {
-                            case "function": {
-                              props.handleChange(
-                                Paths.compose(msgPath, "additional_kwargs"),
-                                {
-                                  function_call: {
-                                    name: "",
-                                    arguments: "{}",
-                                  },
-                                }
-                              );
-
-                              break;
-                            }
-                            case "text": {
-                              props.handleChange(
-                                Paths.compose(msgPath, "additional_kwargs"),
-                                {}
-                              );
-
-                              break;
-                            }
-                          }
-                        }}
-                      >
-                        <ToggleGroup.Item
-                          className="rounded-s border border-divider-700 px-2.5 py-1 data-[state=on]:bg-divider-500/50"
-                          value="text"
-                          aria-label="Text message"
-                        >
-                          <ChatIcon className="w-4 h-4" />
-                        </ToggleGroup.Item>
-                        <ToggleGroup.Item
-                          className="rounded-e border border-l-0 border-divider-700 px-2.5 py-1 data-[state=on]:bg-divider-500/50"
-                          value="function"
-                          aria-label="Function call"
-                        >
-                          <CodeIcon className="w-4 h-4" />
-                        </ToggleGroup.Item>
-                      </ToggleGroup.Root>
-                    )}
-
-                    <button
-                      className="p-1 border rounded opacity-0 transition-opacity border-divider-700 group-focus-within:opacity-100 group-hover:opacity-100"
-                      onClick={() => {
-                        props.handleChange(
-                          props.path,
-                          data.filter((_, i) => i !== index)
-                        );
-                      }}
-                    >
-                      <TrashIcon className="w-4 h-4" />
-                    </button>
-                  </div>
-                </div>
-
-                {type === "chat" && (
-                  <input
-                    className="mb-1"
-                    placeholder="Role"
-                    value={message.role ?? ""}
-                    onChange={(e) => {
-                      props.handleChange(
-                        Paths.compose(msgPath, "role"),
-                        e.target.value
-                      );
-                    }}
-                  />
-                )}
-
-                {type === "function" && (
-                  <input
-                    className="mb-1"
-                    placeholder="Function Name"
-                    value={message.name ?? ""}
-                    onChange={(e) => {
-                      props.handleChange(
-                        Paths.compose(msgPath, "name"),
-                        e.target.value
-                      );
-                    }}
-                  />
-                )}
-
-                {type === "ai" &&
-                isOpenAiFunctionCall(
-                  message.additional_kwargs?.function_call
-                ) ? (
-                  <>
-                    <input
-                      className="mb-1"
-                      placeholder="Function Name"
-                      value={
-                        message.additional_kwargs?.function_call.name ?? ""
-                      }
-                      onChange={(e) => {
-                        props.handleChange(
-                          Paths.compose(
-                            msgPath,
-                            "additional_kwargs.function_call.name"
-                          ),
-                          e.target.value
-                        );
-                      }}
-                    />
-
-                    <AutosizeTextarea
-                      value={
-                        message.additional_kwargs?.function_call?.arguments ??
-                        ""
-                      }
-                      onChange={(content) => {
-                        props.handleChange(
-                          Paths.compose(
-                            msgPath,
-                            "additional_kwargs.function_call.arguments"
-                          ),
-                          content
-                        );
-                      }}
-                    />
-                  </>
-                ) : (
-                  <AutosizeTextarea
-                    value={message.content}
-                    onChange={(content) => {
-                      props.handleChange(
-                        Paths.compose(msgPath, "content"),
-                        content
-                      );
-                    }}
-                  />
-                )}
-              </div>
+              <ChatMessageInput
+                message={message}
+                handleChange={handleChatMessageChange}
+                handleRemoval={handleChatMessageRemoval}
+                path={props.path}
+                key={index}
+                ></ChatMessageInput>
            );
          })}
        </div>
@@ -69,16 +69,16 @@ export function CorrectnessFeedback(props: { runId: string }) {
        type="button"
        className={cn(
          "border focus-within:border-ls-blue focus-within:outline-none bg-background rounded p-1 border-divider-700 hover:bg-divider-500/50 active:bg-divider-500",
-          score.mutation.data?.score === -1 && "text-red-500"
+          score.mutation.data?.score === 0 && "text-red-500"
        )}
        disabled={score.mutation.isMutating}
        onClick={() => {
-          if (score.mutation.data?.score !== -1) {
-            score.mutation.trigger({ key: "correctness", score: -1 });
+          if (score.mutation.data?.score !== 0) {
+            score.mutation.trigger({ key: "correctness", score: 0 });
          }
        }}
      >
-        {score.lastArg?.score === -1 ? (
+        {score.lastArg?.score === 0 ? (
          <CircleSpinIcon className="animate-spin w-4 h-4 text-white/50 fill-white" />
        ) : (
          <ThumbsDownIcon className="w-4 h-4" />
@@ -1,5 +1,9 @@
+import type { Operation } from "fast-json-patch";
+import type { RunState } from "./useStreamLog";
+
 export interface StreamCallback {
  onSuccess?: (ctx: { input: unknown; output: unknown }) => void;
+  onChunk?: (chunk: { ops?: Operation[] }, aggregatedState: RunState | null) => void;
  onError?: () => void;
  onStart?: (ctx: { input: unknown }) => void;
 }
@@ -10,6 +10,7 @@ import { StreamCallback } from "./types";

 export const AppCallbackContext = createContext<MutableRefObject<{
  onStart: Exclude<StreamCallback["onStart"], undefined>[];
+  onChunk: Exclude<StreamCallback["onChunk"], undefined>[];
  onSuccess: Exclude<StreamCallback["onSuccess"], undefined>[];
  onError: Exclude<StreamCallback["onError"], undefined>[];
 }> | null>(null);
@@ -18,9 +19,10 @@ export function useAppStreamCallbacks() {
  // callbacks handling
  const context = useRef<{
    onStart: Exclude<StreamCallback["onStart"], undefined>[];
+    onChunk: Exclude<StreamCallback["onChunk"], undefined>[];
    onSuccess: Exclude<StreamCallback["onSuccess"], undefined>[];
    onError: Exclude<StreamCallback["onError"], undefined>[];
-  }>({ onStart: [], onSuccess: [], onError: [] });
+  }>({ onStart: [], onChunk: [], onSuccess: [], onError: [] });

  const callbacks: StreamCallback = {
    onStart(...args) {
@@ -28,6 +30,11 @@ export function useAppStreamCallbacks() {
        callback(...args);
      }
    },
+    onChunk(...args) {
+      for (const callback of context.current.onChunk) {
+        callback(...args);
+      }
+    },
    onSuccess(...args) {
      for (const callback of context.current.onSuccess) {
        callback(...args);
@@ -44,7 +51,7 @@ export function useAppStreamCallbacks() {
 }

 export function useStreamCallback<
-  Type extends "onStart" | "onSuccess" | "onError"
+  Type extends "onStart" | "onChunk" | "onSuccess" | "onError"
 >(type: Type, callback: Exclude<StreamCallback[Type], undefined>) {
  type CallbackType = Exclude<StreamCallback[Type], undefined>;

@@ -52,6 +52,9 @@ export function useStreamLog(callbacks: StreamCallback = {}) {
  const startRef = useRef(callbacks.onStart);
  startRef.current = callbacks.onStart;

+  const chunkRef = useRef(callbacks.onChunk);
+  chunkRef.current = callbacks.onChunk;
+
  const successRef = useRef(callbacks.onSuccess);
  successRef.current = callbacks.onSuccess;

@@ -74,6 +77,7 @@ export function useStreamLog(callbacks: StreamCallback = {}) {
        if (msg.event === "data") {
          innerLatest = reducer(innerLatest, JSON.parse(msg.data)?.ops);
          setLatest(innerLatest);
+          chunkRef.current?.(JSON.parse(msg.data), innerLatest);
        }
      },
      openWhenHidden: true,
@@ -104,11 +104,13 @@ def _register_path_for_app(app: Union[FastAPI, APIRouter], path: str) -> None:
        _APP_TO_PATHS[app] = {path}


+# This is the type annotation
 EndpointName = Literal[
    "invoke",
    "batch",
    "stream",
    "stream_log",
+    "stream_events",
    "playground",
    "feedback",
    "input_schema",
@@ -117,11 +119,14 @@ EndpointName = Literal[
    "config_hashes",
 ]

+# Contains same information as EndpointName but as a set.
+# Used for run-time validation and displaying configuration errors.
 KNOWN_ENDPOINTS = {
    "invoke",
    "batch",
    "stream",
    "stream_log",
+    "stream_events",
    "playground",
    "feedback",
    "input_schema",
@@ -167,6 +172,7 @@ class _EndpointConfiguration:
                is_batch_enabled = True
                is_stream_enabled = True
                is_stream_log_enabled = True
+                is_stream_events_enabled = True
                is_playground_enabled = True
                is_input_schema_enabled = True
                is_output_schema_enabled = True
@@ -183,6 +189,7 @@ class _EndpointConfiguration:
                is_batch_enabled = "batch" not in disabled_endpoints_
                is_stream_enabled = "stream" not in disabled_endpoints_
                is_stream_log_enabled = "stream_log" not in disabled_endpoints_
+                is_stream_events_enabled = "stream_events" not in disabled_endpoints_
                is_playground_enabled = "playground" not in disabled_endpoints_
                is_input_schema_enabled = "input_schema" not in disabled_endpoints_
                is_output_schema_enabled = "output_schema" not in disabled_endpoints_
@@ -198,6 +205,7 @@ class _EndpointConfiguration:
            is_batch_enabled = "batch" in enabled_endpoints_
            is_stream_enabled = "stream" in enabled_endpoints_
            is_stream_log_enabled = "stream_log" in enabled_endpoints_
+            is_stream_events_enabled = "stream_events" in enabled_endpoints_
            is_playground_enabled = "playground" in enabled_endpoints_
            is_input_schema_enabled = "input_schema" in enabled_endpoints_
            is_output_schema_enabled = "output_schema" in enabled_endpoints_
@@ -208,6 +216,7 @@ class _EndpointConfiguration:
        self.is_batch_enabled = is_batch_enabled
        self.is_stream_enabled = is_stream_enabled
        self.is_stream_log_enabled = is_stream_log_enabled
+        self.is_stream_events_enabled = is_stream_events_enabled
        self.is_playground_enabled = is_playground_enabled
        self.is_input_schema_enabled = is_input_schema_enabled
        self.is_output_schema_enabled = is_output_schema_enabled
@@ -359,6 +368,15 @@ def add_routes(
    # Determine the base URL for the playground endpoint
    prefix = app.prefix if isinstance(app, APIRouter) else ""  # type: ignore

+    # Let's check if the runnable has a astream events property
+    # It's a new method on runnables that allows them to stream events.
+    # We'll only add this if folks are on recent versions of langchain-core.
+    # This is done so that folks can upgrade langserve without having to
+    # upgrade langchain-core if they need other fixes.
+    # We can likely remove in a few months and bump minimal version of langchain
+    # required by langserve.
+    has_astream_events = hasattr(runnable, "astream_events")
+
    api_handler = APIHandler(
        runnable,
        path=path,
@@ -540,6 +558,36 @@ def add_routes(
                # that are used by the runnnable (e.g., input, config fields)
                return await api_handler.stream_log(request, config_hash=config_hash)

+    if has_astream_events and endpoint_configuration.is_stream_events_enabled:
+
+        @app.post(
+            f"{namespace}/stream_events",
+            include_in_schema=False,
+            dependencies=dependencies,
+        )
+        async def stream_events(request: Request) -> EventSourceResponse:
+            """Handle a request."""
+            # The API Handler validates the parts of the request
+            # that are used by the runnnable (e.g., input, config fields)
+            return await api_handler.astream_events(request)
+
+        if endpoint_configuration.is_config_hash_enabled:
+
+            @app.post(
+                namespace + "/c/{config_hash}/stream_events",
+                include_in_schema=False,
+                dependencies=dependencies,
+            )
+            async def stream_events_with_config(
+                request: Request, config_hash: str = ""
+            ) -> EventSourceResponse:
+                """Handle a request."""
+                # The API Handler validates the parts of the request
+                # that are used by the runnnable (e.g., input, config fields)
+                return await api_handler.astream_events(
+                    request, config_hash=config_hash
+                )
+
    if endpoint_configuration.is_input_schema_enabled:

        @app.get(
@@ -620,13 +668,16 @@ def add_routes(

    if endpoint_configuration.is_playground_enabled:
        playground = app.get(
-            namespace + "/playground/{file_path:path}", dependencies=dependencies
+            namespace + "/playground/{file_path:path}",
+            dependencies=dependencies,
+            include_in_schema=False,
        )(api_handler.playground)

        if endpoint_configuration.is_config_hash_enabled:
            app.get(
                namespace + "/c/{config_hash}/playground/{file_path:path}",
                dependencies=dependencies,
+                include_in_schema=False,
            )(playground)

    if enable_feedback_endpoint:
@@ -651,6 +702,7 @@ def add_routes(
        BatchResponse = api_handler.BatchResponse
        StreamRequest = api_handler.StreamRequest
        StreamLogRequest = api_handler.StreamLogRequest
+        StreamEventsRequest = api_handler.StreamEventsRequest

        if endpoint_configuration.is_invoke_enabled:

@@ -777,6 +829,7 @@ def add_routes(
                include_in_schema=True,
                tags=route_tags,
                name=_route_name("stream"),
+                dependencies=dependencies,
                description=(
                    "This endpoint allows to stream the output of the runnable. "
                    "The endpoint uses a server sent event stream to stream the "
@@ -874,3 +927,97 @@ def add_routes(
                    ),
                    dependencies=dependencies,
                )(_stream_log_docs)
+
+        if has_astream_events and endpoint_configuration.is_stream_events_enabled:
+
+            async def _stream_events_docs(
+                stream_events_request: Annotated[
+                    StreamEventsRequest, StreamEventsRequest
+                ],
+                config_hash: str = "",
+            ) -> EventSourceResponse:
+                """Stream events from the given runnable.
+
+                This endpoint allows to stream events from the runnable, including
+                events from all intermediate steps.
+
+                **Attention**
+
+                    This is a new endpoint that only works for langchain-core >= 0.1.14.
+
+                    It belongs to a Beta API that may change in the future.
+
+                **Important**
+                    Specify filters to the events you want to receive by setting
+                    the appropriate filters in the request body.
+
+                    This will help avoid sending too much data over the network.
+
+                    It will also prevent serialization issues with
+                    any unsupported types since it won't need to serialize events
+                    that aren't transmitted.
+
+                The endpoint uses a server sent event stream to stream the output.
+
+                https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
+
+                The encoding of events follows the following format:
+
+                * data - for streaming the output of the runnable
+
+                    {
+                        "event": "data",
+                        "data": {
+                        ...
+                        }
+                    }
+
+                * error - for signaling an error in the stream, also ends the stream.
+
+                {
+                    "event": "error",
+                    "data": {
+                        "status_code": 500,
+                        "message": "Internal Server Error"
+                    }
+                }
+
+                * end - for signaling the end of the stream.
+
+                    This helps the client to know when to stop listening for events and
+                    know that the streaming has ended successfully.
+
+                    {
+                        "event": "end",
+                    }
+
+                `data` for the `data` event is a JSON object that corresponds
+                to a serialized representation of a StreamEvent.
+
+                See LangChain documentation for more information about astream_events.
+                """
+                raise AssertionError("This endpoint should not be reachable.")
+
+            app.post(
+                f"{namespace}/stream_events",
+                include_in_schema=True,
+                tags=route_tags,
+                name=_route_name("stream_events"),
+                dependencies=dependencies,
+            )(_stream_events_docs)
+
+            if endpoint_configuration.is_config_hash_enabled:
+                app.post(
+                    namespace + "/c/{config_hash}/stream_events",
+                    include_in_schema=True,
+                    tags=route_tags_with_config,
+                    name=_route_name_with_config("stream_events"),
+                    description=(
+                        "This endpoint is to be used with share links generated by the "
+                        "LangServe playground. "
+                        "The hash is an LZString compressed JSON string. "
+                        "For regular use cases, use the /stream_events endpoint "
+                        "without the `c/{config_hash}` path parameter."
+                    ),
+                    dependencies=dependencies,
+                )(_stream_events_docs)
@@ -195,6 +195,64 @@ def create_stream_log_request_model(
    return stream_log_request


+def create_stream_events_request_model(
+    namespace: str,
+    input_type: Validator,
+    config: Type[BaseModel],
+) -> Type[BaseModel]:
+    """Create a pydantic model for the stream events request."""
+    stream_events_request = create_model(
+        f"{namespace}StreamEventsRequest",
+        input=(input_type, ...),
+        config=(config, Field(default_factory=dict)),
+        include_names=(
+            Optional[Sequence[str]],
+            Field(
+                None,
+                description="If specified, filter to runnables with matching names",
+            ),
+        ),
+        include_types=(
+            Optional[Sequence[str]],
+            Field(
+                None,
+                description="If specified, filter to runnables with matching types",
+            ),
+        ),
+        include_tags=(
+            Optional[Sequence[str]],
+            Field(
+                None,
+                description="If specified, filter to runnables with matching tags",
+            ),
+        ),
+        exclude_names=(
+            Optional[Sequence[str]],
+            Field(
+                None,
+                description="If specified, exclude runnables with matching names",
+            ),
+        ),
+        exclude_types=(
+            Optional[Sequence[str]],
+            Field(
+                None,
+                description="If specified, exclude runnables with matching types",
+            ),
+        ),
+        exclude_tags=(
+            Optional[Sequence[str]],
+            Field(
+                None,
+                description="If specified, exclude runnables with matching tags",
+            ),
+        ),
+        kwargs=(dict, Field(default_factory=dict)),
+    )
+    stream_events_request.update_forward_refs()
+    return stream_events_request
+
+
 class InvokeBaseResponse(BaseModel):
    """Base class for invoke request."""

@@ -308,6 +366,17 @@ class StreamLogParameters(BaseModel):
    exclude_tags: Optional[Sequence[str]] = None


+class StreamEventsParameters(BaseModel):
+    """Shallow validator for Stream Events Request."""
+
+    include_names: Optional[Sequence[str]] = None
+    include_types: Optional[Sequence[str]] = None
+    include_tags: Optional[Sequence[str]] = None
+    exclude_names: Optional[Sequence[str]] = None
+    exclude_types: Optional[Sequence[str]] = None
+    exclude_tags: Optional[Sequence[str]] = None
+
+
 # Pydantic validators for callback events
 # These objects may have a slightly different shape than the callback events
 # used internally in langchain because they represent a serialized version
@@ -1674,19 +1674,19 @@ extended-testing = ["aiosqlite (>=0.19.0,<0.20.0)", "aleph-alpha-client (>=2.15.

 [[package]]
 name = "langchain-core"
-version = "0.1.6"
+version = "0.1.16"
 description = "Building applications with LLMs through composability"
 optional = false
 python-versions = ">=3.8.1,<4.0"
 files = [
-    {file = "langchain_core-0.1.6-py3-none-any.whl", hash = "sha256:1c3e6ba5f6cc70c0934fea99a80a8312e73612ccb727ff8950d8013ea7cd07f6"},
-    {file = "langchain_core-0.1.6.tar.gz", hash = "sha256:cacb5972a05632f6dbea0785eb11323121295bcbf682b53a53c9afcb902f3954"},
+    {file = "langchain_core-0.1.16-py3-none-any.whl", hash = "sha256:c1b2e7363771d64a72cb45032ed5a46facf67de005017fb5e74595cbf433f834"},
+    {file = "langchain_core-0.1.16.tar.gz", hash = "sha256:8cb546eed318009ee1a8a381d108074eddf0395ae61eb243db00d76e1e265e89"},
 ]

 [package.dependencies]
 anyio = ">=3,<5"
 jsonpatch = ">=1.33,<2.0"
-langsmith = ">=0.0.63,<0.1.0"
+langsmith = ">=0.0.83,<0.1"
 packaging = ">=23.2,<24.0"
 pydantic = ">=1,<3"
 PyYAML = ">=5.3"
@@ -1698,13 +1698,13 @@ extended-testing = ["jinja2 (>=3,<4)"]

 [[package]]
 name = "langsmith"
-version = "0.0.77"
+version = "0.0.83"
 description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform."
 optional = false
 python-versions = ">=3.8.1,<4.0"
 files = [
-    {file = "langsmith-0.0.77-py3-none-any.whl", hash = "sha256:750c0aa9177240c64e131d831e009ed08dd59038f7cabbd0bbcf62ccb7c8dcac"},
-    {file = "langsmith-0.0.77.tar.gz", hash = "sha256:c4c8d3a96ad8671a41064f3ccc673e2e22a4153e823b19f915c9c9b8a4f33a2c"},
+    {file = "langsmith-0.0.83-py3-none-any.whl", hash = "sha256:a5bb7ac58c19a415a9d5f51db56dd32ee2cd7343a00825bbc2018312eb3d122a"},
+    {file = "langsmith-0.0.83.tar.gz", hash = "sha256:94427846b334ad9bdbec3266fee12903fe9f5448f628667689d0412012aaf392"},
 ]

 [package.dependencies]
@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "langserve"
-version = "0.0.38"
+version = "0.0.45"
 description = ""
 readme = "README.md"
 authors = ["LangChain"]
@@ -4,7 +4,9 @@ import datetime
 import json
 from asyncio import AbstractEventLoop
 from contextlib import asynccontextmanager, contextmanager
+from dataclasses import dataclass
 from enum import Enum
+from itertools import cycle
 from typing import (
    Any,
    Dict,
@@ -31,6 +33,13 @@ from langchain.schema.messages import HumanMessage, SystemMessage
 from langchain.schema.runnable import Runnable, RunnableConfig, RunnablePassthrough
 from langchain.schema.runnable.base import RunnableLambda
 from langchain.schema.runnable.utils import ConfigurableField, Input, Output
+from langchain_core.chat_history import BaseChatMessageHistory
+from langchain_core.documents import Document
+from langchain_core.messages import AIMessage, AIMessageChunk, BaseMessage
+from langchain_core.output_parsers import StrOutputParser
+from langchain_core.outputs import ChatGenerationChunk, LLMResult
+from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
+from langchain_core.runnables.history import RunnableWithMessageHistory
 from langsmith import schemas as ls_schemas
 from pytest import MonkeyPatch
 from pytest_mock import MockerFixture
@@ -51,7 +60,8 @@ try:
 except ImportError:
    from pydantic import BaseModel, Field
 from langserve.server import add_routes
-from tests.unit_tests.utils import FakeListLLM, FakeTracer
+from tests.unit_tests.utils.llms import FakeListLLM, GenericFakeChatModel
+from tests.unit_tests.utils.tracer import FakeTracer


 def _decode_eventstream(text: str) -> List[Dict[str, Any]]:
@@ -297,6 +307,42 @@ async def test_serve_playground_with_api_router() -> None:
    assert response.status_code == 200


+async def test_root_path_on_playground(event_loop: AbstractEventLoop) -> None:
+    """Test that the playground respects the root_path for requesting assets"""
+
+    for root_path in ("/home/root", "/home/root/"):
+        app = FastAPI(root_path=root_path)
+        add_routes(
+            app,
+            RunnableLambda(lambda foo: "hello"),
+            path="/chat",
+        )
+
+        router = APIRouter(prefix="/router")
+        add_routes(
+            router,
+            RunnableLambda(lambda foo: "hello"),
+            path="/chat",
+        )
+        app.include_router(router)
+
+        async_client = AsyncClient(app=app, base_url="http://localhost:9999")
+
+        response = await async_client.get("/chat/playground/index.html")
+        assert response.status_code == 200
+        assert (
+            f'src="{root_path.rstrip("/")}/chat/playground/assets/'
+            in response.content.decode()
+        ), "html should contain reference to playground assets with root_path prefix"
+
+        response = await async_client.get("/router/chat/playground/index.html")
+        assert response.status_code == 200
+        assert (
+            f'src="{root_path.rstrip("/")}/router/chat/playground/assets/'
+            in response.content.decode()
+        ), "html should contain reference to playground assets with root_path prefix"
+
+
 async def test_server_async(app: FastAPI) -> None:
    """Test the server directly via HTTP requests."""
    async with get_async_test_client(app, raise_app_exceptions=True) as async_client:
@@ -406,6 +452,69 @@ async def test_server_async(app: FastAPI) -> None:
        assert stream_events[0]["data"]["status_code"] == 422


+async def test_server_astream_events(app: FastAPI) -> None:
+    """Test the server directly via HTTP requests.
+
+    Here we test just astream_events server side without a Remote Client.
+    """
+    async with get_async_test_client(app, raise_app_exceptions=True) as async_client:
+        # Test invoke
+        # Test stream
+        response = await async_client.post("/stream_events", json={"input": 1})
+        # Decode the event stream using plain json de-serialization
+        events = _decode_eventstream(response.text)
+
+        for event in events:
+            if "data" in event:
+                assert "run_id" in event["data"]
+                del event["data"]["run_id"]
+                assert "metadata" in event["data"]
+                del event["data"]["metadata"]
+
+        assert events == [
+            {
+                "data": {
+                    "data": {"input": 1},
+                    "event": "on_chain_start",
+                    "name": "add_one_or_passthrough",
+                    "tags": [],
+                },
+                "type": "data",
+            },
+            {
+                "data": {
+                    "data": {"chunk": 2},
+                    "event": "on_chain_stream",
+                    "name": "add_one_or_passthrough",
+                    "tags": [],
+                },
+                "type": "data",
+            },
+            {
+                "data": {
+                    "data": {"output": 2},
+                    "event": "on_chain_end",
+                    "name": "add_one_or_passthrough",
+                    "tags": [],
+                },
+                "type": "data",
+            },
+            {"type": "end"},
+        ]
+
+    # test stream_events with bad requests
+    async with get_async_test_client(app, raise_app_exceptions=True) as async_client:
+        response = await async_client.post("/stream_events", data="bad json []")
+        stream_events = _decode_eventstream(response.text)
+        assert stream_events[0]["type"] == "error"
+        assert stream_events[0]["data"]["status_code"] == 422
+
+        response = await async_client.post("/stream_events", json={})
+        stream_events = _decode_eventstream(response.text)
+        assert stream_events[0]["type"] == "error"
+        assert stream_events[0]["data"]["status_code"] == 422
+
+
 async def test_server_bound_async(app_for_config: FastAPI) -> None:
    """Test the server directly via HTTP requests."""
    async_client = AsyncClient(app=app_for_config, base_url="http://localhost:9999")
@@ -661,6 +770,8 @@ async def test_astream_log_diff_no_effect(
                    "id": uuid,
                    "logs": {},
                    "streamed_output": [],
+                    "type": "chain",
+                    "name": "add_one_or_passthrough",
                },
            }
        ],
@@ -674,6 +785,8 @@ async def test_astream_log_diff_no_effect(
        "id": uuid,
        "logs": {},
        "streamed_output": [2],
+        "type": "chain",
+        "name": "add_one_or_passthrough",
    }


@@ -729,6 +842,8 @@ async def test_astream_log(async_remote_runnable: RemoteRunnable) -> None:
                        "id": uuid,
                        "logs": {},
                        "streamed_output": [],
+                        "type": "chain",
+                        "name": "add_one",
                    },
                }
            ],
@@ -743,10 +858,11 @@ async def test_astream_log(async_remote_runnable: RemoteRunnable) -> None:
            "id": uuid,
            "logs": {},
            "streamed_output": [2],
+            "type": "chain",
+            "name": "add_one",
        }


-@pytest.mark.asyncio
 async def test_astream_log_allowlist(event_loop: AbstractEventLoop) -> None:
    """Test async stream with an allowlist."""

@@ -922,6 +1038,8 @@ async def test_invoke_as_part_of_sequence_async(
            "id": first_op["value"]["id"],
            "logs": {},
            "streamed_output": [],
+            "type": "chain",
+            "name": "RunnableSequence",
        },
    }

@@ -1987,6 +2105,7 @@ async def test_endpoint_configurations() -> None:
        ("POST", "/batch", {"inputs": [1, 2]}),
        ("POST", "/stream", {"input": 1}),
        ("POST", "/stream_log", {"input": 1}),
+        ("POST", "/stream_events", {"input": 1}),
        ("GET", "/input_schema", {}),
        ("GET", "/output_schema", {}),
        ("GET", "/config_schema", {}),
@@ -1998,6 +2117,7 @@ async def test_endpoint_configurations() -> None:
        ("POST", "/c/1234/batch", {"inputs": [1, 2]}),
        ("POST", "/c/1234/stream", {"input": 1}),
        ("POST", "/c/1234/stream_log", {"input": 1}),
+        ("POST", "/c/1234/stream_events", {"input": 1}),
        ("GET", "/c/1234/input_schema", {}),
        ("GET", "/c/1234/output_schema", {}),
        ("GET", "/c/1234/config_schema", {}),
@@ -2084,6 +2204,496 @@ async def test_endpoint_configurations() -> None:
        )


+async def test_astream_events_simple(async_remote_runnable: RemoteRunnable) -> None:
+    """Test astream events using a simple chain.
+
+    This test should not involve any complex serialization logic.
+    """
+
+    app = FastAPI()
+
+    def add_one(x: int) -> int:
+        """Add one to simulate a valid function"""
+        return x + 1
+
+    def mul_two(y: int) -> int:
+        """Add one to simulate a valid function"""
+        return y * 2
+
+    runnable = RunnableLambda(add_one) | RunnableLambda(mul_two)
+    add_routes(app, runnable)
+
+    # Invoke request
+    async with get_async_remote_runnable(app, raise_app_exceptions=False) as runnable:
+        # Test bad requests
+        # test client side error
+        with pytest.raises(httpx.HTTPStatusError) as cb:
+            # Invalid input type (expected string but got int)
+            async for _ in runnable.astream_events("foo", version="v1"):
+                pass
+
+        # Verify that this is a 422 error
+        assert cb.value.response.status_code == 422
+
+        with pytest.raises(httpx.HTTPStatusError) as cb:
+            # Invalid input type (expected string but got int)
+            # include names should not be a list of lists
+            async for _ in runnable.astream_events(1, include_names=[[]], version="v1"):
+                pass
+
+        # Verify that this is a 422 error
+        assert cb.value.response.status_code == 422
+
+        # Test good requests
+        events = []
+
+        async for event in runnable.astream_events(1, version="v1"):
+            events.append(event)
+
+        # validate events
+        for event in events:
+            assert "run_id" in event
+            del event["run_id"]
+            # Assert that we don't include any "internal" metadata
+            # in the events
+            for k, v in event["metadata"].items():
+                assert not k.startswith("__")
+            assert "metadata" in event
+            del event["metadata"]
+
+        assert events == [
+            {
+                "data": {"input": 1},
+                "event": "on_chain_start",
+                "name": "RunnableSequence",
+                "tags": [],
+            },
+            {
+                "data": {},
+                "event": "on_chain_start",
+                "name": "add_one",
+                "tags": ["seq:step:1"],
+            },
+            {
+                "data": {"chunk": 2},
+                "event": "on_chain_stream",
+                "name": "add_one",
+                "tags": ["seq:step:1"],
+            },
+            {
+                "data": {},
+                "event": "on_chain_start",
+                "name": "mul_two",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {"input": 1, "output": 2},
+                "event": "on_chain_end",
+                "name": "add_one",
+                "tags": ["seq:step:1"],
+            },
+            {
+                "data": {"chunk": 4},
+                "event": "on_chain_stream",
+                "name": "mul_two",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {"chunk": 4},
+                "event": "on_chain_stream",
+                "name": "RunnableSequence",
+                "tags": [],
+            },
+            {
+                "data": {"input": 2, "output": 4},
+                "event": "on_chain_end",
+                "name": "mul_two",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {"output": 4},
+                "event": "on_chain_end",
+                "name": "RunnableSequence",
+                "tags": [],
+            },
+        ]
+
+
+def _clean_up_events(events: List[Dict[str, Any]]) -> None:
+    """Clean up events to make it easy to compare them."""
+    for event in events:
+        assert "run_id" in event
+        del event["run_id"]
+        # Assert that we don't include any "internal" metadata
+        # in the events
+        for k, v in event["metadata"].items():
+            assert not k.startswith("__")
+        assert "metadata" in event
+        del event["metadata"]
+
+
+async def test_astream_events_with_serialization(
+    async_remote_runnable: RemoteRunnable,
+) -> None:
+    """Test serialization logic in astream events.
+
+    Intermediate steps in the chain may involve arbitrary types.
+
+    Let's check that we can serialize some of the well known types.
+    """
+
+    app = FastAPI()
+
+    def to_document(query: str) -> List[Document]:
+        """Convert a query to a document"""
+        return [
+            Document(page_content=query, metadata={"a": "b"}),
+            Document(page_content=query[::-1]),
+        ]
+
+    def from_document(documents: List[Document]) -> str:
+        """Convert a document to a string"""
+        return documents[0].page_content
+
+    # This should work since we have built in serializers for Document
+    chain = RunnableLambda(to_document) | RunnableLambda(from_document)
+    add_routes(app, chain, path="/doc_types")
+
+    # Add a test case for serialization of a dataclass
+    # This will be serialized using FastAPI's built in serializer for dataclasses
+    # It will not however be decoded properly into a dataclass on the client side
+    # since the client side does not have enough information to do so.
+    @dataclass
+    class Pet:
+        name: str
+        age: int
+
+    def get_pets(query: str) -> List[Pet]:
+        """Get pets"""
+        return [
+            Pet(name="foo", age=1),
+            Pet(name="bar", age=2),
+        ]
+
+    # Works because of built-in serializer for dataclass from fast api
+    # But it will not deserialize correctly into a dataclass (this is OK)
+    add_routes(app, RunnableLambda(get_pets), path="/get_pets")
+
+    class NotSerializable:
+        def __init__(self, foo: int) -> None:
+            """Create a non-serializable class"""
+            self.foo = foo
+
+    def into_non_serializable(query: str) -> List[NotSerializable]:
+        """Return non serializable data"""
+        return [NotSerializable(foo=1)]
+
+    def back_to_serializable(inputs) -> str:
+        """Return non serializable data"""
+        return "hello"
+
+    # Works because of built-in serializer for dataclass from fast api
+    # But it will not deserialize correctly into a dataclass (this is OK)
+    chain = RunnableLambda(into_non_serializable) | RunnableLambda(back_to_serializable)
+    add_routes(app, chain, path="/break")
+
+    # Invoke request
+    async with get_async_remote_runnable(
+        app, raise_app_exceptions=False, path="/doc_types"
+    ) as runnable:
+        # Test good requests
+        events = [event async for event in runnable.astream_events("foo", version="v1")]
+        _clean_up_events(events)
+
+        assert events == [
+            {
+                "data": {"input": "foo"},
+                "event": "on_chain_start",
+                "name": "/doc_types",
+                "tags": [],
+            },
+            {
+                "data": {},
+                "event": "on_chain_start",
+                "name": "to_document",
+                "tags": ["seq:step:1"],
+            },
+            {
+                "data": {
+                    "chunk": [
+                        Document(page_content="foo", metadata={"a": "b"}),
+                        Document(page_content="oof"),
+                    ]
+                },
+                "event": "on_chain_stream",
+                "name": "to_document",
+                "tags": ["seq:step:1"],
+            },
+            {
+                "data": {},
+                "event": "on_chain_start",
+                "name": "from_document",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {
+                    "input": "foo",
+                    "output": [
+                        Document(page_content="foo", metadata={"a": "b"}),
+                        Document(page_content="oof"),
+                    ],
+                },
+                "event": "on_chain_end",
+                "name": "to_document",
+                "tags": ["seq:step:1"],
+            },
+            {
+                "data": {"chunk": "foo"},
+                "event": "on_chain_stream",
+                "name": "from_document",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {"chunk": "foo"},
+                "event": "on_chain_stream",
+                "name": "/doc_types",
+                "tags": [],
+            },
+            {
+                "data": {
+                    "input": [
+                        Document(page_content="foo", metadata={"a": "b"}),
+                        Document(page_content="oof"),
+                    ],
+                    "output": "foo",
+                },
+                "event": "on_chain_end",
+                "name": "from_document",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {"output": "foo"},
+                "event": "on_chain_end",
+                "name": "/doc_types",
+                "tags": [],
+            },
+        ]
+
+    async with get_async_remote_runnable(
+        app, raise_app_exceptions=False, path="/get_pets"
+    ) as runnable:
+        # Test good requests
+        events = [event async for event in runnable.astream_events("foo", version="v1")]
+        _clean_up_events(events)
+        assert events == [
+            {
+                "data": {"input": "foo"},
+                "event": "on_chain_start",
+                "name": "/get_pets",
+                "tags": [],
+            },
+            {
+                "data": {
+                    "chunk": [{"age": 1, "name": "foo"}, {"age": 2, "name": "bar"}]
+                },
+                "event": "on_chain_stream",
+                "name": "/get_pets",
+                "tags": [],
+            },
+            {
+                "data": {
+                    "output": [{"age": 1, "name": "foo"}, {"age": 2, "name": "bar"}]
+                },
+                "event": "on_chain_end",
+                "name": "/get_pets",
+                "tags": [],
+            },
+        ]
+
+    async with get_async_remote_runnable(
+        app, raise_app_exceptions=False, path="/break"
+    ) as runnable:
+        # Test good requests
+        with pytest.raises(httpx.HTTPStatusError) as cb:
+            async for event in runnable.astream_events("foo", version="v1"):
+                pass
+        assert cb.value.response.status_code == 500
+
+
+async def test_astream_events_with_prompt_model_parser_chain(
+    async_remote_runnable: RemoteRunnable,
+) -> None:
+    """Test prompt + model + parser chain"""
+
+    app = FastAPI()
+
+    messages = cycle([AIMessage(content="Hello World!")])
+
+    model = GenericFakeChatModel(messages=messages)
+
+    prompt = ChatPromptTemplate.from_messages(
+        [("system", "You are a cat."), ("user", "{question}")]
+    )
+
+    chain = prompt | model | StrOutputParser()
+    add_routes(app, chain)
+
+    async with get_async_remote_runnable(app) as runnable:
+        # Test good requests
+        events = [
+            event
+            async for event in runnable.astream_events(
+                {"question": "hello"}, version="v1"
+            )
+        ]
+        _clean_up_events(events)
+        assert events == [
+            {
+                "data": {"input": {"question": "hello"}},
+                "event": "on_chain_start",
+                "name": "RunnableSequence",
+                "tags": [],
+            },
+            {
+                "data": {"input": {"question": "hello"}},
+                "event": "on_prompt_start",
+                "name": "ChatPromptTemplate",
+                "tags": ["seq:step:1"],
+            },
+            {
+                "data": {
+                    "input": {"question": "hello"},
+                    "output": {
+                        "messages": [
+                            SystemMessage(content="You are a cat."),
+                            HumanMessage(content="hello"),
+                        ]
+                    },
+                },
+                "event": "on_prompt_end",
+                "name": "ChatPromptTemplate",
+                "tags": ["seq:step:1"],
+            },
+            {
+                "data": {
+                    "input": {
+                        "messages": [
+                            [
+                                SystemMessage(content="You are a cat."),
+                                HumanMessage(content="hello"),
+                            ]
+                        ]
+                    }
+                },
+                "event": "on_chat_model_start",
+                "name": "GenericFakeChatModel",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {},
+                "event": "on_parser_start",
+                "name": "StrOutputParser",
+                "tags": ["seq:step:3"],
+            },
+            {
+                "data": {"chunk": "Hello"},
+                "event": "on_parser_stream",
+                "name": "StrOutputParser",
+                "tags": ["seq:step:3"],
+            },
+            {
+                "data": {"chunk": "Hello"},
+                "event": "on_chain_stream",
+                "name": "RunnableSequence",
+                "tags": [],
+            },
+            {
+                "data": {"chunk": AIMessageChunk(content="Hello")},
+                "event": "on_chat_model_stream",
+                "name": "GenericFakeChatModel",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {"chunk": " "},
+                "event": "on_parser_stream",
+                "name": "StrOutputParser",
+                "tags": ["seq:step:3"],
+            },
+            {
+                "data": {"chunk": " "},
+                "event": "on_chain_stream",
+                "name": "RunnableSequence",
+                "tags": [],
+            },
+            {
+                "data": {"chunk": AIMessageChunk(content=" ")},
+                "event": "on_chat_model_stream",
+                "name": "GenericFakeChatModel",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {"chunk": "World!"},
+                "event": "on_parser_stream",
+                "name": "StrOutputParser",
+                "tags": ["seq:step:3"],
+            },
+            {
+                "data": {"chunk": "World!"},
+                "event": "on_chain_stream",
+                "name": "RunnableSequence",
+                "tags": [],
+            },
+            {
+                "data": {"chunk": AIMessageChunk(content="World!")},
+                "event": "on_chat_model_stream",
+                "name": "GenericFakeChatModel",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {
+                    "input": {
+                        "messages": [
+                            [
+                                SystemMessage(content="You are a cat."),
+                                HumanMessage(content="hello"),
+                            ]
+                        ]
+                    },
+                    "output": LLMResult(
+                        generations=[
+                            [
+                                ChatGenerationChunk(
+                                    text="Hello World!",
+                                    message=AIMessageChunk(content="Hello World!"),
+                                )
+                            ]
+                        ],
+                        llm_output=None,
+                        run=None,
+                    ),
+                },
+                "event": "on_chat_model_end",
+                "name": "GenericFakeChatModel",
+                "tags": ["seq:step:2"],
+            },
+            {
+                "data": {
+                    "input": AIMessageChunk(content="Hello World!"),
+                    "output": "Hello World!",
+                },
+                "event": "on_parser_end",
+                "name": "StrOutputParser",
+                "tags": ["seq:step:3"],
+            },
+            {
+                "data": {"output": "Hello World!"},
+                "event": "on_chain_end",
+                "name": "RunnableSequence",
+                "tags": [],
+            },
+        ]
+
+
 async def test_path_dependencies() -> None:
    """Test path dependencies."""

@@ -2111,6 +2721,7 @@ async def test_path_dependencies() -> None:
        ("POST", "/batch", {"inputs": [1, 2]}),
        ("POST", "/stream", {"input": 1}),
        ("POST", "/stream_log", {"input": 1}),
+        ("POST", "/stream_events", {"input": 1}),
        ("GET", "/input_schema", {}),
        ("GET", "/output_schema", {}),
        ("GET", "/config_schema", {}),
@@ -2122,6 +2733,7 @@ async def test_path_dependencies() -> None:
        ("POST", "/c/1234/batch", {"inputs": [1, 2]}),
        ("POST", "/c/1234/stream", {"input": 1}),
        ("POST", "/c/1234/stream_log", {"input": 1}),
+        ("POST", "/c/1234/stream_events", {"input": 1}),
        ("GET", "/c/1234/input_schema", {}),
        ("GET", "/c/1234/output_schema", {}),
        ("GET", "/c/1234/config_schema", {}),
@@ -2145,3 +2757,68 @@ async def test_path_dependencies() -> None:
                f"with {response.text}. "
                f"Should not return 422 status code since we are passing the header."
            )
+
+
+async def test_remote_configurable_remote_runnable() -> None:
+    """Test that a configurable a client runnable that's configurable works.
+
+    Here, we wrap the client runnable in a RunnableWithMessageHistory.
+
+    The test verifies that the extra information populated by RunnableWithMessageHistory
+    does not interfere with the serialization logic.
+    """
+    app = FastAPI()
+
+    class InMemoryHistory(BaseChatMessageHistory, BaseModel):
+        """In memory implementation of chat message history."""
+
+        messages: List[BaseMessage] = Field(default_factory=list)
+
+        def add_message(self, message: BaseMessage) -> None:
+            """Add a self-created message to the store"""
+            self.messages.append(message)
+
+        def clear(self) -> None:
+            self.messages = []
+
+    # Here we use a global variable to store the chat message history.
+    # This will make it easier to inspect it to see the underlying results.
+    store = {}
+
+    def get_by_session_id(session_id: str) -> BaseChatMessageHistory:
+        if session_id not in store:
+            store[session_id] = InMemoryHistory()
+        return store[session_id]
+
+    prompt = ChatPromptTemplate.from_messages(
+        [
+            ("system", "You're an assistant who's good at {ability}"),
+            MessagesPlaceholder(variable_name="history"),
+            ("human", "{question}"),
+        ]
+    )
+
+    model = GenericFakeChatModel(messages=cycle([AIMessage(content="Hello World!")]))
+    chain = prompt | model
+
+    add_routes(app, chain)
+
+    # Invoke request
+    async with get_async_remote_runnable(app, raise_app_exceptions=False) as client:
+        chain_with_history = RunnableWithMessageHistory(
+            client,
+            # Uses the get_by_session_id function defined in the example
+            # above.
+            get_by_session_id,
+            input_messages_key="question",
+            history_messages_key="history",
+        )
+        result = await chain_with_history.ainvoke(
+            {"question": "hi"}, {"configurable": {"session_id": "1"}}
+        )
+        assert result == AIMessage(content="Hello World!")
+        assert store == {
+            "1": InMemoryHistory(
+                messages=[HumanMessage(content="hi"), AIMessage(content="Hello World!")]
+            )
+        }
@@ -0,0 +1,244 @@
+"""Fake Chat Model wrapper for testing purposes."""
+import asyncio
+import re
+import time
+from typing import (
+    Any,
+    AsyncIterator,
+    Iterator,
+    List,
+    Mapping,
+    Optional,
+    cast,
+)
+
+from langchain_core.callbacks.manager import (
+    AsyncCallbackManagerForLLMRun,
+    CallbackManagerForLLMRun,
+)
+from langchain_core.language_models import LLM
+from langchain_core.language_models.chat_models import BaseChatModel
+from langchain_core.messages import (
+    AIMessage,
+    AIMessageChunk,
+    BaseMessage,
+)
+from langchain_core.outputs import ChatGeneration, ChatGenerationChunk, ChatResult
+from langchain_core.runnables import run_in_executor
+
+
+class GenericFakeChatModel(BaseChatModel):
+    """A generic fake chat model that can be used to test the chat model interface.
+
+    * Chat model should be usable in both sync and async tests
+    * Invokes on_llm_new_token to allow for testing of callback related code for new
+      tokens.
+    * Includes logic to break messages into message chunk to facilitate testing of
+      streaming.
+    """
+
+    messages: Iterator[AIMessage]
+    """Get an iterator over messages.
+
+    This can be expanded to accept other types like Callables / dicts / strings
+    to make the interface more generic if needed.
+
+    Note: if you want to pass a list, you can use `iter` to convert it to an iterator.
+
+    Please note that streaming is not implemented yet. We should try to implement it
+    in the future by delegating to invoke and then breaking the resulting output
+    into message chunks.
+    """
+    delay_per_token: Optional[float] = None
+    """Delay per token in seconds.
+    
+    If specified, we will sleep for this amount of time after each token when
+    streaming.
+    """
+
+    def _generate(
+        self,
+        messages: List[BaseMessage],
+        stop: Optional[List[str]] = None,
+        run_manager: Optional[CallbackManagerForLLMRun] = None,
+        **kwargs: Any,
+    ) -> ChatResult:
+        """Top Level call"""
+        message = next(self.messages)
+        generation = ChatGeneration(message=message)
+        return ChatResult(generations=[generation])
+
+    def _stream(
+        self,
+        messages: List[BaseMessage],
+        stop: Optional[List[str]] = None,
+        run_manager: Optional[CallbackManagerForLLMRun] = None,
+        **kwargs: Any,
+    ) -> Iterator[ChatGenerationChunk]:
+        for chunk in self._stream_implementation(
+            messages, stop=stop, run_manager=run_manager, **kwargs
+        ):
+            if self.delay_per_token:
+                time.sleep(self.delay_per_token)
+            yield chunk
+
+    def _stream_implementation(
+        self,
+        messages: List[BaseMessage],
+        stop: Optional[List[str]] = None,
+        run_manager: Optional[CallbackManagerForLLMRun] = None,
+        **kwargs: Any,
+    ) -> Iterator[ChatGenerationChunk]:
+        """Stream the output of the model."""
+        chat_result = self._generate(
+            messages, stop=stop, run_manager=run_manager, **kwargs
+        )
+        if not isinstance(chat_result, ChatResult):
+            raise ValueError(
+                f"Expected generate to return a ChatResult, "
+                f"but got {type(chat_result)} instead."
+            )
+
+        message = chat_result.generations[0].message
+
+        if not isinstance(message, AIMessage):
+            raise ValueError(
+                f"Expected invoke to return an AIMessage, "
+                f"but got {type(message)} instead."
+            )
+
+        content = message.content
+
+        if content:
+            # Use a regular expression to split on whitespace with a capture group
+            # so that we can preserve the whitespace in the output.
+            assert isinstance(content, str)
+            content_chunks = cast(List[str], re.split(r"(\s)", content))
+
+            for token in content_chunks:
+                chunk = ChatGenerationChunk(message=AIMessageChunk(content=token))
+                yield chunk
+                if run_manager:
+                    run_manager.on_llm_new_token(token, chunk=chunk)
+
+        if message.additional_kwargs:
+            for key, value in message.additional_kwargs.items():
+                # We should further break down the additional kwargs into chunks
+                # Special case for function call
+                if key == "function_call":
+                    for fkey, fvalue in value.items():
+                        if isinstance(fvalue, str):
+                            # Break function call by `,`
+                            fvalue_chunks = cast(List[str], re.split(r"(,)", fvalue))
+                            for fvalue_chunk in fvalue_chunks:
+                                chunk = ChatGenerationChunk(
+                                    message=AIMessageChunk(
+                                        content="",
+                                        additional_kwargs={
+                                            "function_call": {fkey: fvalue_chunk}
+                                        },
+                                    )
+                                )
+                                yield chunk
+                                if run_manager:
+                                    run_manager.on_llm_new_token(
+                                        "",
+                                        chunk=chunk,  # No token for function call
+                                    )
+                        else:
+                            chunk = ChatGenerationChunk(
+                                message=AIMessageChunk(
+                                    content="",
+                                    additional_kwargs={"function_call": {fkey: fvalue}},
+                                )
+                            )
+                            yield chunk
+                            if run_manager:
+                                run_manager.on_llm_new_token(
+                                    "",
+                                    chunk=chunk,  # No token for function call
+                                )
+                else:
+                    chunk = ChatGenerationChunk(
+                        message=AIMessageChunk(
+                            content="", additional_kwargs={key: value}
+                        )
+                    )
+                    yield chunk
+                    if run_manager:
+                        run_manager.on_llm_new_token(
+                            "",
+                            chunk=chunk,  # No token for function call
+                        )
+
+    async def _astream(
+        self,
+        messages: List[BaseMessage],
+        stop: Optional[List[str]] = None,
+        run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
+        **kwargs: Any,
+    ) -> AsyncIterator[ChatGenerationChunk]:
+        """Stream the output of the model."""
+        result = await run_in_executor(
+            None,
+            self._stream,
+            messages,
+            stop=stop,
+            run_manager=run_manager.get_sync() if run_manager else None,
+            **kwargs,
+        )
+        for chunk in result:
+            if self.delay_per_token:
+                await asyncio.sleep(self.delay_per_token)
+            yield chunk
+
+    @property
+    def _llm_type(self) -> str:
+        return "generic-fake-chat-model"
+
+
+class FakeListLLM(LLM):
+    """Fake LLM for testing purposes."""
+
+    responses: List[str]
+    sleep: Optional[float] = None
+    i: int = 0
+
+    @property
+    def _llm_type(self) -> str:
+        """Return type of llm."""
+        return "fake-list"
+
+    def _call(
+        self,
+        prompt: str,
+        stop: Optional[List[str]] = None,
+        run_manager: Optional[CallbackManagerForLLMRun] = None,
+        **kwargs: Any,
+    ) -> str:
+        """Return next response"""
+        response = self.responses[self.i]
+        if self.i < len(self.responses) - 1:
+            self.i += 1
+        else:
+            self.i = 0
+        return response
+
+    async def _acall(
+        self,
+        prompt: str,
+        stop: Optional[List[str]] = None,
+        run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
+        **kwargs: Any,
+    ) -> str:
+        """Return next response"""
+        response = self.responses[self.i]
+        if self.i < len(self.responses) - 1:
+            self.i += 1
+        else:
+            self.i = 0
+        return response
+
+    @property
+    def _identifying_params(self) -> Mapping[str, Any]:
+        return {"responses": self.responses}
@@ -0,0 +1,185 @@
+"""Tests for verifying that testing utility code works as expected."""
+from itertools import cycle
+from typing import Any, Dict, List, Optional, Union
+from uuid import UUID
+
+from langchain_core.callbacks.base import AsyncCallbackHandler
+from langchain_core.messages import AIMessage, AIMessageChunk, BaseMessage
+from langchain_core.outputs import ChatGenerationChunk, GenerationChunk
+
+from tests.unit_tests.utils.llms import GenericFakeChatModel
+
+
+def test_generic_fake_chat_model_invoke() -> None:
+    # Will alternate between responding with hello and goodbye
+    infinite_cycle = cycle([AIMessage(content="hello"), AIMessage(content="goodbye")])
+    model = GenericFakeChatModel(messages=infinite_cycle)
+    response = model.invoke("meow")
+    assert response == AIMessage(content="hello")
+    response = model.invoke("kitty")
+    assert response == AIMessage(content="goodbye")
+    response = model.invoke("meow")
+    assert response == AIMessage(content="hello")
+
+
+async def test_generic_fake_chat_model_ainvoke() -> None:
+    # Will alternate between responding with hello and goodbye
+    infinite_cycle = cycle([AIMessage(content="hello"), AIMessage(content="goodbye")])
+    model = GenericFakeChatModel(messages=infinite_cycle)
+    response = await model.ainvoke("meow")
+    assert response == AIMessage(content="hello")
+    response = await model.ainvoke("kitty")
+    assert response == AIMessage(content="goodbye")
+    response = await model.ainvoke("meow")
+    assert response == AIMessage(content="hello")
+
+
+async def test_generic_fake_chat_model_stream() -> None:
+    """Test streaming."""
+    infinite_cycle = cycle(
+        [
+            AIMessage(content="hello goodbye"),
+        ]
+    )
+    model = GenericFakeChatModel(messages=infinite_cycle)
+    chunks = [chunk async for chunk in model.astream("meow")]
+    assert chunks == [
+        AIMessageChunk(content="hello"),
+        AIMessageChunk(content=" "),
+        AIMessageChunk(content="goodbye"),
+    ]
+
+    chunks = [chunk for chunk in model.stream("meow")]
+    assert chunks == [
+        AIMessageChunk(content="hello"),
+        AIMessageChunk(content=" "),
+        AIMessageChunk(content="goodbye"),
+    ]
+
+    # Test streaming of additional kwargs.
+    # Relying on insertion order of the additional kwargs dict
+    message = AIMessage(content="", additional_kwargs={"foo": 42, "bar": 24})
+    model = GenericFakeChatModel(messages=cycle([message]))
+    chunks = [chunk async for chunk in model.astream("meow")]
+    assert chunks == [
+        AIMessageChunk(content="", additional_kwargs={"foo": 42}),
+        AIMessageChunk(content="", additional_kwargs={"bar": 24}),
+    ]
+
+    message = AIMessage(
+        content="",
+        additional_kwargs={
+            "function_call": {
+                "name": "move_file",
+                "arguments": '{\n  "source_path": "foo",\n  "'
+                'destination_path": "bar"\n}',
+            }
+        },
+    )
+    model = GenericFakeChatModel(messages=cycle([message]))
+    chunks = [chunk async for chunk in model.astream("meow")]
+
+    assert chunks == [
+        AIMessageChunk(
+            content="", additional_kwargs={"function_call": {"name": "move_file"}}
+        ),
+        AIMessageChunk(
+            content="",
+            additional_kwargs={
+                "function_call": {"arguments": '{\n  "source_path": "foo"'}
+            },
+        ),
+        AIMessageChunk(
+            content="", additional_kwargs={"function_call": {"arguments": ","}}
+        ),
+        AIMessageChunk(
+            content="",
+            additional_kwargs={
+                "function_call": {"arguments": '\n  "destination_path": "bar"\n}'}
+            },
+        ),
+    ]
+
+    accumulate_chunks = None
+    for chunk in chunks:
+        if accumulate_chunks is None:
+            accumulate_chunks = chunk
+        else:
+            accumulate_chunks += chunk
+
+    assert accumulate_chunks == AIMessageChunk(
+        content="",
+        additional_kwargs={
+            "function_call": {
+                "name": "move_file",
+                "arguments": '{\n  "source_path": "foo",\n  "'
+                'destination_path": "bar"\n}',
+            }
+        },
+    )
+
+
+async def test_generic_fake_chat_model_astream_log() -> None:
+    """Test streaming."""
+    infinite_cycle = cycle([AIMessage(content="hello goodbye")])
+    model = GenericFakeChatModel(messages=infinite_cycle)
+    log_patches = [
+        log_patch async for log_patch in model.astream_log("meow", diff=False)
+    ]
+    final = log_patches[-1]
+    assert final.state["streamed_output"] == [
+        AIMessageChunk(content="hello"),
+        AIMessageChunk(content=" "),
+        AIMessageChunk(content="goodbye"),
+    ]
+
+
+async def test_callback_handlers() -> None:
+    """Verify that model is implemented correctly with handlers working."""
+
+    class MyCustomAsyncHandler(AsyncCallbackHandler):
+        def __init__(self, store: List[str]) -> None:
+            self.store = store
+
+        async def on_chat_model_start(
+            self,
+            serialized: Dict[str, Any],
+            messages: List[List[BaseMessage]],
+            *,
+            run_id: UUID,
+            parent_run_id: Optional[UUID] = None,
+            tags: Optional[List[str]] = None,
+            metadata: Optional[Dict[str, Any]] = None,
+            **kwargs: Any,
+        ) -> Any:
+            # Do nothing
+            # Required to implement since this is an abstract method
+            pass
+
+        async def on_llm_new_token(
+            self,
+            token: str,
+            *,
+            chunk: Optional[Union[GenerationChunk, ChatGenerationChunk]] = None,
+            run_id: UUID,
+            parent_run_id: Optional[UUID] = None,
+            tags: Optional[List[str]] = None,
+            **kwargs: Any,
+        ) -> None:
+            self.store.append(token)
+
+    infinite_cycle = cycle(
+        [
+            AIMessage(content="hello goodbye"),
+        ]
+    )
+    model = GenericFakeChatModel(messages=infinite_cycle)
+    tokens: List[str] = []
+    # New model
+    results = list(model.stream("meow", {"callbacks": [MyCustomAsyncHandler(tokens)]}))
+    assert results == [
+        AIMessageChunk(content="hello"),
+        AIMessageChunk(content=" "),
+        AIMessageChunk(content="goodbye"),
+    ]
+    assert tokens == ["hello", " ", "goodbye"]
@@ -1,62 +1,10 @@
-from typing import Any, Dict, List, Mapping, Optional
+from typing import Dict, List
 from uuid import UUID

-from langchain.callbacks.manager import (
-    AsyncCallbackManagerForLLMRun,
-    CallbackManagerForLLMRun,
-)
 from langchain.callbacks.tracers.base import BaseTracer
-from langchain.llms.base import LLM
 from langsmith.schemas import Run


-class FakeListLLM(LLM):
-    """Fake LLM for testing purposes."""
-
-    responses: List[str]
-    sleep: Optional[float] = None
-    i: int = 0
-
-    @property
-    def _llm_type(self) -> str:
-        """Return type of llm."""
-        return "fake-list"
-
-    def _call(
-        self,
-        prompt: str,
-        stop: Optional[List[str]] = None,
-        run_manager: Optional[CallbackManagerForLLMRun] = None,
-        **kwargs: Any,
-    ) -> str:
-        """Return next response"""
-        response = self.responses[self.i]
-        if self.i < len(self.responses) - 1:
-            self.i += 1
-        else:
-            self.i = 0
-        return response
-
-    async def _acall(
-        self,
-        prompt: str,
-        stop: Optional[List[str]] = None,
-        run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
-        **kwargs: Any,
-    ) -> str:
-        """Return next response"""
-        response = self.responses[self.i]
-        if self.i < len(self.responses) - 1:
-            self.i += 1
-        else:
-            self.i = 0
-        return response
-
-    @property
-    def _identifying_params(self) -> Mapping[str, Any]:
-        return {"responses": self.responses}
-
-
 class FakeTracer(BaseTracer):
    """Fake tracer that records LangChain execution.
Author	SHA1	Message	Date
Eugene Yurtsev	b6ec1e86bd	Release 0.0.45 (#498 ) Improvements for playground	2024-02-28 14:03:18 -05:00
Jacob Lee	ce727fcbf4	Adds playground chat widget for message list inputs, additional global callback for chunks (#489 ) Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	2024-02-28 13:39:54 -05:00
Eugene Yurtsev	8ea0cb4b53	Release 0.0.44 (#494 ) See release notes	2024-02-28 12:19:17 -05:00
Engin Diri	af98fed73b	docs: add IaC section to the docs with first examples for Pulumi. (#491 ) This PR introduces a new subsection under the Deployment section titled "Deploy using Infrastructure as Code" In this subsection, we will outline methods for utilizing IaC tools such as Pulumi, OpenTofu, and AWS CDK, among others. I have initiated with examples in Pulumi and AWS. Examples for other cloud providers are expected to be added shortly. Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	2024-02-28 12:12:24 -05:00
Eugene Yurtsev	bcc1cd6d89	Update .clabot (#493 )	2024-02-28 12:11:57 -05:00
Nat Noordanus	827229fd2d	Exclude playground asset paths from OpenAPI docs (#488 )	2024-02-28 11:53:42 -05:00
Jacob Lee	ff07eec3d2	Update JS docs links (#490 ) @eyurtsev	2024-02-28 00:37:26 -08:00
Nat Noordanus	a01ab330ac	Make playground work with apps behind a proxy with root_path (#472 ) Make the playground respect the [root_path](https://fastapi.tiangolo.com/advanced/behind-a-proxy/) from the request scope. Also streamline some related code in APIHandler around normalizing trailing slashes on paths.	2024-02-27 21:51:51 -05:00
jakerachleff	c1d52441f6	Update Hosted LangServe Signup Link (#485 )	2024-02-26 14:31:02 -08:00
Eugene Yurtsev	53f265a0f8	Release 0.0.43 (#484 ) See release notes	2024-02-26 13:19:48 -05:00
Eugene Yurtsev	25ebe634f2	Fix: Pass dependencies to doc endpont for stream (#483 ) This PR fixes a bug that wasn't making dependency information show up for the documentation of the stream endpoint. The dependencies were correctly applied for the endpoint itself, so only the documentation is affected for /stream endpoint.	2024-02-26 13:19:18 -05:00
Eugene Yurtsev	594527e90b	Update .clabot (#480 )	2024-02-22 10:22:16 -05:00
Eugene Yurtsev	ddb6ecbdbf	release version 0.0.42 (#477 ) see release notes	2024-02-22 10:11:45 -05:00
Eugene Yurtsev	e1cec832da	patch: name used for tracing should take into account APIRouter path (#476 ) If there is a router at path /foo, and a runnable is added to that router at path /bar, the name of the runnable should be /foo/bar for logging purposes.	2024-02-22 10:09:02 -05:00
Eugene Yurtsev	54725a91ed	Examples: Add multiple servers example (#469 ) Adds an example of using multiple servers	2024-02-14 13:11:55 -05:00
Eugene Yurtsev	d46017048e	Update Readme to include path information when disabling the playground (#464 ) Clarify how to disable/enable endpoints -- i.e., it's provided as an extra argument when invoking add_routes	2024-02-13 11:56:54 -05:00
Eugene Yurtsev	03183ebda4	Examples: Remove duplicated examples (#455 ) Remove duplicated example	2024-02-06 14:21:05 -05:00
Eugene Yurtsev	1b990368fb	Example: Add custom agent streaming (#453 ) Show how to implement custom agent streaming	2024-02-06 13:53:13 -05:00
Eugene Yurtsev	3bed90105c	Examples: Add chat widget to agent (#452 ) * Adds a chat widget to the agent * Adds chat widget to the chat with history persisted on the user side (but this is not yet supported by the playground)	2024-02-06 12:30:04 -05:00
Eugene Yurtsev	0561555718	Examples: Improve agent with history example to show astream events usage (#451 ) Show how astream events can be used on the server side.	2024-02-06 12:11:13 -05:00
Eugene Yurtsev	fa1951f436	Examples: Update agent example to include astream events (#448 ) This PR shows an example of astream events on the client side.	2024-02-06 11:33:16 -05:00
Eugene Yurtsev	5f1b2dab8d	README.md: fix typos in with_types arguments (#446 )	2024-02-06 10:39:39 -05:00
Eugene Yurtsev	853274c286	Update README.md to include absolute link to langserve (#445 )	2024-02-06 10:38:18 -05:00
Eugene Yurtsev	90bf613711	Example: Add ollama example for a local LLM (#436 )	2024-02-01 19:18:36 -08:00
Eugene Yurtsev	956f0376d8	Release 0.0.41 (#426 ) See release notes	2024-01-27 00:12:27 -05:00
Eugene Yurtsev	1828f91bda	Fix: Drop non json serializable values in the config prior to sending it to the server (#425 ) This PR drops all non json serializable values in the config prior to sending the config to the server. This seems like the correct behavior in general, which is why it's being merged without exposing a way to control it. Any non serializable value seem to be used only locallly by the runnable or something that wraps. Server side configurable runnables are supposed to only exposed configuration that is trivially json serializable because that's how configurable runnables were designed.	2024-01-27 00:07:23 -05:00
Eugene Yurtsev	534009a591	Readme: Add information about astream_events (#424 ) Add information about astream_events	2024-01-26 16:15:19 -05:00
Eugene Yurtsev	6c77437158	Bump version for release 0.0.40 (#423 ) See release notes	2024-01-26 15:52:57 -05:00
Eugene Yurtsev	02319e5a8f	Add StreamEventRequest type, update OpenAPIDocs (#422 ) This PR adds a type for StreamEventsRequest and updates OpenAPI docs.	2024-01-26 15:41:49 -05:00
Eugene Yurtsev	f41fb26daa	Tests: inlclude astream events endpoint in testing enabled/disabled endpoint config (#421 ) This PR will make sure that enabled/disabled config for astream events works correctly.	2024-01-26 14:39:07 -05:00
Eugene Yurtsev	65f50b333d	Tests: test stream events with chat model output (#420 ) Verify that serialization with chat model output works	2024-01-26 14:34:37 -05:00
Eugene Yurtsev	5a9adbbf0b	Tests: re-organize utilities and chat model utility (#419 ) Add utility code to mock a chat model	2024-01-26 14:20:10 -05:00
Eugene Yurtsev	0c6faa37c5	astream events: Add serializtion path tests (#418 ) Add tests on the serialization path for astream events	2024-01-26 14:19:54 -05:00
Eugene Yurtsev	a8cbaf57e2	patch: rename internal function name (#417 ) Rename name of function that cleans up metadata before sending to user	2024-01-26 12:06:01 -05:00
Eugene Yurtsev	e7315e3ee1	Strip internal keys from metadata (#416 ) This PR will make sure that we strip internal information from the event metadata.	2024-01-26 12:02:52 -05:00
Eugene Yurtsev	16eb5247bd	Add astream_events support to Remote Runnable (#415 ) Add astream events to Remote Runnable	2024-01-26 11:42:56 -05:00
Eugene Yurtsev	3dc83cc5a2	Add astream_events support to langserve (#412 ) This PR adds initial support to langserve for the new astream events Runnable method.	2024-01-26 11:09:37 -05:00
Eugene Yurtsev	a7c2a94bb7	Update README with new example (#400 ) Include new example in readme	2024-01-18 14:42:52 -05:00
Eugene Yurtsev	18b35f8e30	Example: Add agent with history (#399 ) Add an example showing an agent with history	2024-01-18 14:36:50 -05:00
Bagatur	d5e6a7cd38	docs: fix links README (#391 )	2024-01-10 17:33:14 -05:00
Eugene Yurtsev	8ab404d15d	Release 0.0.39 (#390 ) See release notes	2024-01-10 12:18:10 -05:00
Eugene Yurtsev	047a47e706	Playground: Use 1 and 0 for feedback score instead of 1 and -1 (#388 ) Use 1 and 0 for feedback scores instead of 1 and -1	2024-01-10 11:39:34 -05:00
Eugene Yurtsev	d3fabc6f0f	Example: Add example of configurable runnable used in agent executor (#382 ) Adds an example that shows how to create a custom runnable for a more complex use case. Shows how to create a workaround for this issue: https://github.com/langchain-ai/langserve/issues/314 while the underlying bug is being fixed in langchain.	2024-01-05 22:16:06 -05:00
Eugene Yurtsev	23472f7c5d	Update agent client example with streaming/stream log (#379 ) This PR updates the notebook and server to show how to stream and stream log responses from the agent. The stream log response will contain individual llm tokens after turning streaming=True on the underlying LLM.	2024-01-05 17:28:03 -05:00