langchain-ai/langserve
1
0
Fork 0
mirror of https://github.com/langchain-ai/langserve.git synced 2026-07-16 09:34:30 -04:00
Code Issues 136 Packages Projects Releases 10 Wiki Activity

how can i use local model #247

New Issue
Open
opened 2026-02-16 00:19:47 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 00:19:47 -05:00
Owner
Copy Link
Copy Source

Originally created by @GXKIM on GitHub (Mar 25, 2025).

I have started using a model with vllm, and I want to use the web-based Langsmith in conjunction with my local model. The code is as follows. I’m curious about how I should customize the model and then use it on the web page.

###code
llm = ChatOpenAI(model_name = 'Qwen2.5-14B-Instruct', base_url = "http://xxx:9009/v1", api_key = "EMPTY", temperature=0).bind(response_format={"type": "json_object"})

Image

Originally created by @GXKIM on GitHub (Mar 25, 2025). I have started using a model with vllm, and I want to use the web-based Langsmith in conjunction with my local model. The code is as follows. I’m curious about how I should customize the model and then use it on the web page. ###code llm = ChatOpenAI(model_name = 'Qwen2.5-14B-Instruct', base_url = "http://xxx:9009/v1", api_key = "EMPTY", temperature=0).bind(response_format={"type": "json_object"}) ![Image](https://github.com/user-attachments/assets/2616b4ab-ad6a-45ae-91dd-e2736dcafff9)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
duplicate
enhancement
help wanted
in-progress
investigate
not langserve
playground
pull-request
Mirrored from GitHub Pull Request
 question
stale
streaming
unable-to-reproduce
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langserve#247
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?