Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #952] [BUG]: [LMStudio 0.2.17] Model with key 'model-placeholder' not loaded. #578

New Issue
Closed
opened 2026-02-22 18:20:16 -05:00 by yindo · 7 comments
yindo commented 2026-02-22 18:20:16 -05:00
Owner
Copy Link
Copy Source

Originally created by @mictadlo on GitHub (Mar 21, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/952

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

In LLM Studio I got:

[2024-03-22 11:47:57.602] [INFO] [LM STUDIO SERVER] Stopping server..
[2024-03-22 11:47:57.603] [INFO] [LM STUDIO SERVER] Server stopped
[2024-03-22 11:48:03.340] [INFO] [LM STUDIO SERVER] Verbose server logs are ENABLED
[2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] Success! HTTP server listening on port 1234
[2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] Supported endpoints:
[2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] ->	GET  http://localhost:1234/v1/models
[2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] ->	POST http://localhost:1234/v1/chat/completions
[2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] ->	POST http://localhost:1234/v1/completions
[2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] Logs are saved into /tmp/lmstudio-server-log.txt
[2024-03-22 11:55:12.655] [INFO] [LM STUDIO SERVER] Processing queued request...
[2024-03-22 11:55:12.656] [INFO] Received POST request to /v1/chat/completions with body: {
  "model": "model-placeholder",
  "stream": true,
  "messages": [
    {
      "role": "system",
      "content": "Given the following conversation, relevant context, and a follow up question, reply with an answer to the current question the user is asking. Return only your response to the question given the above information following the users instructions as needed."
    },
    {
      "role": "user",
      "content": "Could you please summerise upladed document?"
    }
  ],
  "temperature": 0.7
}
[2024-03-22 11:55:12.657] [ERROR] Model with key 'model-placeholder' not loaded.
[2024-03-22 11:55:21.448] [INFO] [LM STUDIO SERVER] Processing queued request...
[2024-03-22 11:55:21.449] [INFO] Received POST request to /v1/chat/completions with body: {
  "model": "model-placeholder",
  "stream": true,
  "messages": [
    {
      "role": "system",
      "content": "Given the following conversation, relevant context, and a follow up question, reply with an answer to the current question the user is asking. Return only your response to the question given the above information following the users instructions as needed."
    },
    {
      "role": "user",
      "content": "hi"
    }
  ],
  "temperature": 0.7
}
[2024-03-22 11:55:21.452] [ERROR] Model with key 'model-placeholder' not loaded.

In AnythingLLM I got:

Could not respond to message.
Request failed with status code 500

Are there known steps to reproduce?

I followed this video

Originally created by @mictadlo on GitHub (Mar 21, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/952 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? In LLM Studio I got: ``` [2024-03-22 11:47:57.602] [INFO] [LM STUDIO SERVER] Stopping server.. [2024-03-22 11:47:57.603] [INFO] [LM STUDIO SERVER] Server stopped [2024-03-22 11:48:03.340] [INFO] [LM STUDIO SERVER] Verbose server logs are ENABLED [2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] Success! HTTP server listening on port 1234 [2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] Supported endpoints: [2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] -> GET http://localhost:1234/v1/models [2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] -> POST http://localhost:1234/v1/chat/completions [2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] -> POST http://localhost:1234/v1/completions [2024-03-22 11:48:03.343] [INFO] [LM STUDIO SERVER] Logs are saved into /tmp/lmstudio-server-log.txt [2024-03-22 11:55:12.655] [INFO] [LM STUDIO SERVER] Processing queued request... [2024-03-22 11:55:12.656] [INFO] Received POST request to /v1/chat/completions with body: { "model": "model-placeholder", "stream": true, "messages": [ { "role": "system", "content": "Given the following conversation, relevant context, and a follow up question, reply with an answer to the current question the user is asking. Return only your response to the question given the above information following the users instructions as needed." }, { "role": "user", "content": "Could you please summerise upladed document?" } ], "temperature": 0.7 } [2024-03-22 11:55:12.657] [ERROR] Model with key 'model-placeholder' not loaded. [2024-03-22 11:55:21.448] [INFO] [LM STUDIO SERVER] Processing queued request... [2024-03-22 11:55:21.449] [INFO] Received POST request to /v1/chat/completions with body: { "model": "model-placeholder", "stream": true, "messages": [ { "role": "system", "content": "Given the following conversation, relevant context, and a follow up question, reply with an answer to the current question the user is asking. Return only your response to the question given the above information following the users instructions as needed." }, { "role": "user", "content": "hi" } ], "temperature": 0.7 } [2024-03-22 11:55:21.452] [ERROR] Model with key 'model-placeholder' not loaded. ``` In AnythingLLM I got: ``` Could not respond to message. Request failed with status code 500 ``` ### Are there known steps to reproduce? I followed this [video](https://www.youtube.com/watch?v=-Rs8-M-xBFI)
yindo added the possible bug label 2026-02-22 18:20:16 -05:00
yindo closed this issue 2026-02-22 18:20:17 -05:00
yindo commented 2026-02-22 18:20:17 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 21, 2024):

Is this the latest version of LMStudio? I know they have a multi-model mode now so that very well could have resulted in a breaking change for their inference server

@timothycarambat commented on GitHub (Mar 21, 2024): Is this the latest version of LMStudio? I know they have a multi-model mode now so that very well could have resulted in a breaking change for their inference server
yindo commented 2026-02-22 18:20:17 -05:00
Author
Owner
Copy Link
Copy Source

@mictadlo commented on GitHub (Mar 21, 2024):

I am using LM Studio Version 0.2.17 (0.2.17)

@mictadlo commented on GitHub (Mar 21, 2024): I am using LM Studio Version 0.2.17 (0.2.17)
yindo commented 2026-02-22 18:20:17 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 22, 2024):

So looking through their Discord they have a bug currently in 0.2.17 that has not been patched yet where the inference server is requiring now the model key during chats but it is unable to be set or ignored when using the inference server.

Will make a patch now as the bug does allow the /models endpoint to work but it returns whatever model you are running on their inference server to be called Loaded from Chat UI.

Im sure at some point they will patch, but this version of LMStudio is broken currently for external API access.

Unfortunately, we cannot create a quick patch for desktop app as soon as the code fix is merged here and it will be present in the next desktop app release next week.

@timothycarambat commented on GitHub (Mar 22, 2024): So looking through their Discord they have a bug currently in 0.2.17 that has not been patched yet where the inference server is requiring now the `model` key during chats **but** it is unable to be set or ignored when using the inference server. Will make a patch now as the bug does allow the `/models` endpoint to work but it returns whatever model you are running on their inference server to be called `Loaded from Chat UI`. Im sure at some point they will patch, but this version of LMStudio is broken currently for external API access. Unfortunately, we cannot create a quick patch for desktop app as soon as the code fix is merged here and it will be present in the next desktop app release next week.
yindo commented 2026-02-22 18:20:17 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 22, 2024):

@mictadlo while the desktop app patch is underway - here is a workaround for using LMStudio 0.2.17.

  1. Go to playground. Start multi-model chat
  2. Click "load model" in the top bar - this will be your desired model.
  3. A popup modal will appear that asks for a "model identifier" put model-placeholder in this field. Spelled exactly like that and case-sensitive.
  4. Click start server with whatever port you desired
  5. Use the endpoint in AnythingLLM as you did previous and model-placeholder will map to your preferred model that was working prior to LMStudio 0.2.17.

Until their team patches this bug, this is the most we can do to offset that.

@timothycarambat commented on GitHub (Mar 22, 2024): @mictadlo while the desktop app patch is underway - here is a workaround for using LMStudio 0.2.17. 1. Go to playground. Start multi-model chat 2. Click "load model" in the top bar - this will be your desired model. 3. A popup modal will appear that asks for a "model identifier" put `model-placeholder` in this field. Spelled exactly like that and case-sensitive. 4. Click start server with whatever port you desired 5. Use the endpoint in AnythingLLM as you did previous and `model-placeholder` will map to your preferred model that was working prior to LMStudio 0.2.17. Until their team patches this bug, this is the most we can do to offset that.
yindo commented 2026-02-22 18:20:17 -05:00
Author
Owner
Copy Link
Copy Source

@mictadlo commented on GitHub (Mar 23, 2024):

I will try it out. Could you please allow to change the Token Size after the first setup in anything llm ?

@mictadlo commented on GitHub (Mar 23, 2024): I will try it out. Could you please allow to change the Token Size after the first setup in anything llm ?
yindo commented 2026-02-22 18:20:17 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 25, 2024):

#969

@timothycarambat commented on GitHub (Mar 25, 2024): #969
yindo commented 2026-02-22 18:20:17 -05:00
Author
Owner
Copy Link
Copy Source

@TufelmalikPQC commented on GitHub (Mar 3, 2025):

@mictadlo while the desktop app patch is underway - here is a workaround for using LMStudio 0.2.17.

  1. Go to playground. Start multi-model chat
  2. Click "load model" in the top bar - this will be your desired model.
  3. A popup modal will appear that asks for a "model identifier" put model-placeholder in this field. Spelled exactly like that and case-sensitive.
  4. Click start server with whatever port you desired
  5. Use the endpoint in AnythingLLM as you did previous and model-placeholder will map to your preferred model that was working prior to LMStudio 0.2.17.

Until their team patches this bug, this is the most we can do to offset that.

There is no option in 0.3.10v of LM Studio

@TufelmalikPQC commented on GitHub (Mar 3, 2025): > [@mictadlo](https://github.com/mictadlo) while the desktop app patch is underway - here is a workaround for using LMStudio 0.2.17. > > 1. Go to playground. Start multi-model chat > 2. Click "load model" in the top bar - this will be your desired model. > 3. A popup modal will appear that asks for a "model identifier" put `model-placeholder` in this field. Spelled exactly like that and case-sensitive. > 4. Click start server with whatever port you desired > 5. Use the endpoint in AnythingLLM as you did previous and `model-placeholder` will map to your preferred model that was working prior to LMStudio 0.2.17. > > Until their team patches this bug, this is the most we can do to offset that. There is no option in 0.3.10v of LM Studio
yindo changed title from [BUG]: [LMStudio 0.2.17] Model with key 'model-placeholder' not loaded. to [GH-ISSUE #952] [BUG]: [LMStudio 0.2.17] Model with key 'model-placeholder' not loaded. 2026-06-05 14:36:06 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#578
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?