Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2974] [BUG]: Can't get model list in LiteLLM #1899

New Issue
Closed
opened 2026-02-22 18:27:08 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:27:08 -05:00
Owner
Copy Link
Copy Source

Originally created by @voidance13 on GitHub (Jan 14, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2974

How are you running AnythingLLM?

Docker (remote machine)

What happened?

I have started LiteLLM and test in both python and url successfully. I'm using docker version anythingllm:1.2.0, linux/amd64

client = openai.OpenAI(
api_key="sk-1234", # pass litellm proxy key, if you're using virtual keys
base_url="http://0.0.0.0:4000" # litellm-proxy-base url
)

response = client.chat.completions.create(
model="qwen72b",
messages = [
{
"role": "user",
"content": "what llm are you"
}
],
)

print(response)

image

However, in the LLMPreference setting page, LiteLLM can't get the model list.
image
image

I've tried adding /v1 after the base url but not working.

Are there known steps to reproduce?

No response

Originally created by @voidance13 on GitHub (Jan 14, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2974 ### How are you running AnythingLLM? Docker (remote machine) ### What happened? I have started LiteLLM and test in both python and url successfully. I'm using docker version anythingllm:1.2.0, linux/amd64 client = openai.OpenAI( api_key="sk-1234", # pass litellm proxy key, if you're using virtual keys base_url="http://0.0.0.0:4000" # litellm-proxy-base url ) response = client.chat.completions.create( model="qwen72b", messages = [ { "role": "user", "content": "what llm are you" } ], ) print(response) ![image](https://github.com/user-attachments/assets/03012f18-b90d-4696-8737-1d306e6df888) However, in the LLMPreference setting page, LiteLLM can't get the model list. ![image](https://github.com/user-attachments/assets/84902a2c-a537-47c6-9370-031ff6123270) ![image](https://github.com/user-attachments/assets/9f57b2d9-865b-482e-9703-37ddc1c81c36) I've tried adding /v1 after the base url but not working. ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:27:08 -05:00
yindo closed this issue 2026-02-22 18:27:08 -05:00
yindo commented 2026-02-22 18:27:08 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 15, 2025):

Since you are in docker, 0.0.0.0 will result in local loopback inside of the container - not the host.

https://docs.anythingllm.com/installation-docker/local-docker#1-cannot-connect-to-service-running-on-localhost

So for your case, http://host.docker.internal:4000 if on windows/mac. On linux/Ubuntu it is 172.17.0.1

@timothycarambat commented on GitHub (Jan 15, 2025): Since you are in docker, 0.0.0.0 will result in local loopback _inside of the container_ - not the host. https://docs.anythingllm.com/installation-docker/local-docker#1-cannot-connect-to-service-running-on-localhost So for your case, `http://host.docker.internal:4000` if on windows/mac. On linux/Ubuntu it is `172.17.0.1`
yindo changed title from [BUG]: Can't get model list in LiteLLM to [GH-ISSUE #2974] [BUG]: Can't get model list in LiteLLM 2026-06-05 14:43:20 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1899
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?