langchain-ai/langsmith-model-server
1
0
Fork 0
mirror of https://github.com/langchain-ai/langsmith-model-server.git synced 2026-07-01 11:38:21 -04:00
Code Issues Packages Projects Releases Wiki Activity

docs: add README for LangSmith model server

Browse Source
This commit is contained in:
infra
2024-06-04 13:47:49 -07:00
parent 062f716845
commit f4f0e1b311
2 changed files with 35 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+33
View File
@@ -34,6 +34,34 @@ After running through the setup, this server should work out of the box with the
poetry run uvicorn app.server:app --host 0.0.0.0 --port <port>
```
The server should now be running
```
INFO: Started server process [55114]
INFO: Waiting for application startup.
__ ___ .__ __. _______ _______. _______ .______ ____ ____ _______
| | / \ | \ | | / _____| / || ____|| _ \ \ \ / / | ____|
| | / ^ \ | \| | | | __ | (----`| |__ | |_) | \ \/ / | |__
| | / /_\ \ | . ` | | | |_ | \ \ | __| | / \ / | __|
| `----./ _____ \ | |\ | | |__| | .----) | | |____ | |\ \----. \ / | |____
|_______/__/ \__\ |__| \__| \______| |_______/ |_______|| _| `._____| \__/ |_______|
LANGSERVE: Playground for chain "/chat/" is live at:
LANGSERVE: │
LANGSERVE: └──> /chat/playground/
LANGSERVE:
LANGSERVE: Playground for chain "/" is live at:
LANGSERVE: │
LANGSERVE: └──> /playground/
LANGSERVE:
LANGSERVE: See all available routes at /docs/
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
```
By default, we expose the chat model at `/chat` and the instruct model at `/`.
## Testing
To test that your server is running correctly, we have a helpful `test_server.py` script that leverages [RemoteRunnable](https://github.com/langchain-ai/langserve/blob/main/langserve/client.py#L259). To run the test, run the following command:
@@ -44,6 +72,11 @@ poetry run python test_server.py
You should see a response from the server indicating that the server is running correctly.
```bash
Pinging chat model: help
Pinging instruct model: help
```
## Running in Docker
This project folder includes a Dockerfile that allows you to easily build and host your model server. This will be needed
test_server.py
+2 -2
View File
@@ -2,7 +2,7 @@ from langserve import RemoteRunnable
chat_model = RemoteRunnable("http://localhost:8080/chat")
print(chat_model.invoke("help"))
print(f"Pinging chat model: {chat_model.invoke('help').content}")
instruct_model = RemoteRunnable("http://localhost:8080")
print(instruct_model.invoke("help")
print(f"Pinging instruct model: {instruct_model.invoke('help')}")