Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #4468] [MERGED] Model context limit auto-detection for LM Studio and Ollama LLM Providers #4637

New Issue
Closed
opened 2026-02-22 18:36:14 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:36:14 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/4468
Author: @shatfield4
Created: 10/1/2025
Status: ✅ Merged
Merged: 10/2/2025
Merged by: @timothycarambat

Base: master ← Head: lmstudio-ollama-auto-detect-model-context-limits

📝 Commits (7)

  • 8c64af4 auto model context limit detection for ollama llm provider
  • 19a82b5 auto model context limit detection for lmstudio llm provider
  • ae97534 Merge branch 'master' into lmstudio-ollama-auto-detect-model-context-limits
  • 8f0b19b Merge branch 'master' into lmstudio-ollama-auto-detect-model-context-limits
  • 15be972 Patch Ollama to function and sync context windows like Foundry
  • 1d5b9fe normalize how model context windows are cached from endpoint service
  • 2bd93b2 remove debug log

📊 Changes

10 files changed (+281 additions, -90 deletions)

View changed files

📝 frontend/src/components/LLMSelection/LMStudioOptions/index.jsx (+25 -25)
📝 frontend/src/components/LLMSelection/OllamaLLMOptions/index.jsx (+28 -24)
📝 server/models/systemSettings.js (+2 -2)
📝 server/utils/AiProviders/lmStudio/index.js (+83 -17)
📝 server/utils/AiProviders/ollama/index.js (+83 -20)
📝 server/utils/agents/aibitat/providers/lmstudio.js (+3 -0)
📝 server/utils/agents/aibitat/providers/ollama.js (+17 -0)
➕ server/utils/boot/eagerLoadContextWindows.js (+35 -0)
📝 server/utils/boot/index.js (+3 -0)
📝 server/utils/helpers/updateENV.js (+2 -2)

📄 Description

Pull Request Type

  • ✨ feat
  • 🐛 fix
  • ♻️ refactor
  • 💄 style
  • 🔨 chore
  • 📝 docs

Relevant Issues

resolves #4469

What is in this change?

  • Utilizes Ollama's /api/show rest endpoint for getting context window directly from Ollama
  • Utilizes LM Studio's /api/v0/models rest endpoint for getting context window directly from LM Studio
  • Has fallbacks for models that cannot be automatically detected (4096)
  • Caching system inside each provider ensuring we do not hit the endpoint multiple times if already cached
  • Move Max Tokens option in frontend to advanced options and allow users to override auto detected limits for both Ollama and LM Studio

Additional Information

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/4468 **Author:** [@shatfield4](https://github.com/shatfield4) **Created:** 10/1/2025 **Status:** ✅ Merged **Merged:** 10/2/2025 **Merged by:** [@timothycarambat](https://github.com/timothycarambat) **Base:** `master` ← **Head:** `lmstudio-ollama-auto-detect-model-context-limits` --- ### 📝 Commits (7) - [`8c64af4`](https://github.com/Mintplex-Labs/anything-llm/commit/8c64af43417b87886a1f101eacefd6934a2f13f0) auto model context limit detection for ollama llm provider - [`19a82b5`](https://github.com/Mintplex-Labs/anything-llm/commit/19a82b51f47d0e99915344469f8e2272361eea85) auto model context limit detection for lmstudio llm provider - [`ae97534`](https://github.com/Mintplex-Labs/anything-llm/commit/ae97534ee8407acf5cce8748a8a3b9cc70498370) Merge branch 'master' into lmstudio-ollama-auto-detect-model-context-limits - [`8f0b19b`](https://github.com/Mintplex-Labs/anything-llm/commit/8f0b19b05e4f5ac7ab6808b7f3e2d8d11681c033) Merge branch 'master' into lmstudio-ollama-auto-detect-model-context-limits - [`15be972`](https://github.com/Mintplex-Labs/anything-llm/commit/15be972ae200cb3d21af62c52dafe46c1d4eb1af) Patch Ollama to function and sync context windows like Foundry - [`1d5b9fe`](https://github.com/Mintplex-Labs/anything-llm/commit/1d5b9fef4da2bc25076f9554db122ae9f82104a0) normalize how model context windows are cached from endpoint service - [`2bd93b2`](https://github.com/Mintplex-Labs/anything-llm/commit/2bd93b29bda72dd9ab9c20c11a68660ab93943e6) remove debug log ### 📊 Changes **10 files changed** (+281 additions, -90 deletions) <details> <summary>View changed files</summary> 📝 `frontend/src/components/LLMSelection/LMStudioOptions/index.jsx` (+25 -25) 📝 `frontend/src/components/LLMSelection/OllamaLLMOptions/index.jsx` (+28 -24) 📝 `server/models/systemSettings.js` (+2 -2) 📝 `server/utils/AiProviders/lmStudio/index.js` (+83 -17) 📝 `server/utils/AiProviders/ollama/index.js` (+83 -20) 📝 `server/utils/agents/aibitat/providers/lmstudio.js` (+3 -0) 📝 `server/utils/agents/aibitat/providers/ollama.js` (+17 -0) ➕ `server/utils/boot/eagerLoadContextWindows.js` (+35 -0) 📝 `server/utils/boot/index.js` (+3 -0) 📝 `server/utils/helpers/updateENV.js` (+2 -2) </details> ### 📄 Description ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [x] ✨ feat - [ ] 🐛 fix - [ ] ♻️ refactor - [ ] 💄 style - [ ] 🔨 chore - [ ] 📝 docs ### Relevant Issues <!-- Use "resolves #xxx" to auto resolve on merge. Otherwise, please use "connect #xxx" --> resolves #4469 ### What is in this change? <!-- Describe the changes in this PR that are impactful to the repo. --> - Utilizes Ollama's `/api/show` rest endpoint for getting context window directly from Ollama - Utilizes LM Studio's `/api/v0/models` rest endpoint for getting context window directly from LM Studio - Has fallbacks for models that cannot be automatically detected (4096) - Caching system inside each provider ensuring we do not hit the endpoint multiple times if already cached - Move `Max Tokens` option in frontend to advanced options and allow users to override auto detected limits for both Ollama and LM Studio ### Additional Information <!-- Add any other context about the Pull Request here that was not captured above. --> ### Developer Validations <!-- All of the applicable items should be checked. --> - [x] I ran `yarn lint` from the root of the repo & committed changes - [x] Relevant documentation has been updated - [x] I have tested my code functionality - [ ] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-22 18:36:14 -05:00
yindo closed this issue 2026-02-22 18:36:14 -05:00
yindo changed title from [PR #4468] Model context limit auto-detection for LM Studio and Ollama LLM Providers to [PR #4468] [MERGED] Model context limit auto-detection for LM Studio and Ollama LLM Providers 2026-06-05 15:19:34 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#4637
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?