Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5757] [BUG]: LM Studio context length setting not persisted — resets to 8192 on container restart #5279

New Issue
Closed
opened 2026-06-05 14:53:05 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 14:53:05 -04:00
Owner
Copy Link
Copy Source

Originally created by @aikidovation on GitHub (Jun 4, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5757

How are you running AnythingLLM?

Docker (local)

What happened?

the env var LMSTUDIO_MODEL_TOKEN_LIMIT seems to be the root cause --the UI neither reads from nor writes to the env var, resulting in confusion.

Are there known steps to reproduce?

  1. Configure AnythingLLM with LM Studio as the LLM provider
  2. Set context length to a non-default value (e.g. 81920)
  3. Restart the AnythingLLM container

Root cause (confirmed via database inspection):
The workspaces table has no context length column. The system_settings table has no context length entry. The value is held purely in server memory (LMStudioLLM.modelContextWindows) and is never written to the database.
The auto-detection queries /api/v0/models correctly and the endpoint returns the right value (loaded_context_length: 81920). However if the cache population fails at startup (e.g. model not yet loaded), the code falls back to 16384 — and separately the UI displays 8192, suggesting a second fallback somewhere in the UI layer.

Expected behavior:
The context length value set by the user should be persisted to system_settings and restored on startup, with auto-detection used only as a fallback when no persisted value exists.

Environment:
--AnythingLLM running in Docker (bare docker run, not Compose)
--LM Studio on macOS host, accessed via host.docker.internal:1234
--LM Studio model: google/gemma-4-26b-a4b, loaded at 81920 context
-- /api/v0/models correctly reports loaded_context_length: 81920
-- /v1/models returns no max_context_length field

LLM Provider & Model (if applicable)

google/gemma-4-26b-a4b running in LM Studio

Embedder Provider & Model (if applicable)

text-embedding-nomic-embed-text-v1.5 as embedder, also in LM Studio

Originally created by @aikidovation on GitHub (Jun 4, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5757 ### How are you running AnythingLLM? Docker (local) ### What happened? the env var LMSTUDIO_MODEL_TOKEN_LIMIT seems to be the root cause --the UI neither reads from nor writes to the env var, resulting in confusion. ### Are there known steps to reproduce? 1) Configure AnythingLLM with LM Studio as the LLM provider 2) Set context length to a non-default value (e.g. 81920) 3) Restart the AnythingLLM container Root cause (confirmed via database inspection): The workspaces table has no context length column. The system_settings table has no context length entry. The value is held purely in server memory (LMStudioLLM.modelContextWindows) and is never written to the database. The auto-detection queries /api/v0/models correctly and the endpoint returns the right value (loaded_context_length: 81920). However if the cache population fails at startup (e.g. model not yet loaded), the code falls back to 16384 — and separately the UI displays 8192, suggesting a second fallback somewhere in the UI layer. Expected behavior: The context length value set by the user should be persisted to system_settings and restored on startup, with auto-detection used only as a fallback when no persisted value exists. Environment: --AnythingLLM running in Docker (bare docker run, not Compose) --LM Studio on macOS host, accessed via host.docker.internal:1234 --LM Studio model: google/gemma-4-26b-a4b, loaded at 81920 context -- /api/v0/models correctly reports loaded_context_length: 81920 -- /v1/models returns no max_context_length field ### LLM Provider & Model (if applicable) google/gemma-4-26b-a4b running in LM Studio ### Embedder Provider & Model (if applicable) text-embedding-nomic-embed-text-v1.5 as embedder, also in LM Studio
yindo added the possible bugneeds info / can't replicateinvestigating labels 2026-06-05 14:53:05 -04:00
yindo closed this issue 2026-06-05 14:53:05 -04:00
yindo commented 2026-06-05 14:53:06 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 5, 2026):

Trying to replicate this issue and cannot so far. Can you help me with understanding where the issue lies?

  • I start LMStudio (no loaded model)
  • AnythingLLM i make connection and set context to something easy to spot like 33333
  • I send chats and see that context size being loaded in logs
  • I restart the container and this ENV is visible in the UI and subsequent chats are at this context.

Is there a step I am missing trying to reproduce this bug?

<!-- gh-comment-id:4632772937 --> @timothycarambat commented on GitHub (Jun 5, 2026): Trying to replicate this issue and cannot so far. Can you help me with understanding where the issue lies? - I start LMStudio (no loaded model) - AnythingLLM i make connection and set context to something easy to spot like `33333` - I send chats and see that context size being loaded in logs - I restart the container and this ENV is visible in the UI and subsequent chats are at this context. Is there a step I am missing trying to reproduce this bug?
yindo commented 2026-06-05 14:53:06 -04:00
Author
Owner
Copy Link
Copy Source

@aikidovation commented on GitHub (Jun 5, 2026):

Thank you for the quick response. I think I can clarify the reproduction steps.

The issue in my case was specific to a Docker setup using docker run with environment variables passed explicitly via -e. I was having trouble getting some settings to persist, hence I adopted this approach.

My launch.sh contained:
-e LMSTUDIO_MODEL_TOKEN_LIMIT=8192

This environment variable was overriding the UI setting on every container restart. The UI would show 8192 regardless of what I set, because the env var takes precedence over the auto-detected or manually entered value.

The behavioral issue is twofold:

  1. The UI does not seem to consistently reflect the active value when an env var is set — it should either display the env var value and indicate it's locked, or warn the user that the field is being overridden by an environment variable.

  2. When no env var is set and the UI field is left blank, the value is not persisted to the database and resets to a fallback on container restart. This is a separate but related issue.

I've since fixed my launch.sh to use LMSTUDIO_MODEL_TOKEN_LIMIT=81920 and the correct value now persists. But a user who doesn't know to check their env vars would have no way to diagnose why their context length keeps resetting.

Having said all that, I'm having a grand time with AnythingLLM. It helps me in my work multiple times a day. Thank you for a great product!

<!-- gh-comment-id:4633417102 --> @aikidovation commented on GitHub (Jun 5, 2026): Thank you for the quick response. I think I can clarify the reproduction steps. The issue in my case was specific to a Docker setup using docker run with environment variables passed explicitly via -e. I was having trouble getting some settings to persist, hence I adopted this approach. My launch.sh contained: -e LMSTUDIO_MODEL_TOKEN_LIMIT=8192 This environment variable was overriding the UI setting on every container restart. The UI would show 8192 regardless of what I set, because the env var takes precedence over the auto-detected or manually entered value. The behavioral issue is twofold: 1) The UI does not seem to consistently reflect the active value when an env var is set — it should either display the env var value and indicate it's locked, or warn the user that the field is being overridden by an environment variable. 2) When no env var is set and the UI field is left blank, the value is not persisted to the database and resets to a fallback on container restart. This is a separate but related issue. I've since fixed my launch.sh to use LMSTUDIO_MODEL_TOKEN_LIMIT=81920 and the correct value now persists. But a user who doesn't know to check their env vars would have no way to diagnose why their context length keeps resetting. Having said all that, I'm having a grand time with AnythingLLM. It helps me in my work multiple times a day. Thank you for a great product!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating needs info / can't replicate possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5279
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?