Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5620] [CLOSED] Fix agent context window overflow and provider tag leak #5502

New Issue
Closed
opened 2026-06-05 15:21:33 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 15:21:33 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5620
Author: @shatfield4
Created: 5/12/2026
Status: ❌ Closed

Base: master ← Head: fix/aibitat-context-overflow

📝 Commits (4)

  • ac2bdc1 fix agent context window overflow and provider tag leak
  • d7dbabe clarify tool token handling in agent context budget comment
  • 7625df5 Merge branch 'master' into fix/aibitat-context-overflow
  • 1ffbd29 sync master

📊 Changes

2 files changed (+98 additions, -6 deletions)

View changed files

📝 server/utils/agents/aibitat/index.js (+19 -6)
➕ server/utils/agents/aibitat/utils/messageWindow.js (+79 -0)

📄 Description

Pull Request Type

  • ✨ feat (New feature)
  • 🐛 fix (Bug fix)
  • ♻️ refactor (Code refactoring without changing behavior)
  • 💄 style (UI style changes)
  • 🔨 chore (Build, CI, maintenance)
  • 📝 docs (Documentation updates)

Relevant Issues

resolves #

Description

  • Agents were crashing on follow-up chats with a Converting circular structure to JSON error.
  • Root cause: aibitat.provider was being overwritten from the string "openai" to the provider class instance after each turn, so Provider.contextLimit silently fell back to an 8k window for any tool that called it.
  • Fixed by storing the active provider instance on aibitat.activeProvider so the string tag stays intact.
  • Added a defensive warning in Provider.contextLimit so this kind of regression can't sneak back in silently.
  • Added a trim that pops the oldest user/assistant pairs from agent history when the conversation grows past the model's context window — keeps the system prompt and current user message intact.
  • Trim reserves 15% of the context window for the model's response and tools.
  • Logs a warning and an introspect line whenever trimming happens so we can see it in user reports.

Visuals (if applicable)

Additional Information

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5620 **Author:** [@shatfield4](https://github.com/shatfield4) **Created:** 5/12/2026 **Status:** ❌ Closed **Base:** `master` ← **Head:** `fix/aibitat-context-overflow` --- ### 📝 Commits (4) - [`ac2bdc1`](https://github.com/Mintplex-Labs/anything-llm/commit/ac2bdc172fcb53d2602df3662c5eeec9b8951a5d) fix agent context window overflow and provider tag leak - [`d7dbabe`](https://github.com/Mintplex-Labs/anything-llm/commit/d7dbabed796194496affbdf0ab6ec9a1a05bddd6) clarify tool token handling in agent context budget comment - [`7625df5`](https://github.com/Mintplex-Labs/anything-llm/commit/7625df59e35f2f7176c0152b6e10dad9d19ac70d) Merge branch 'master' into fix/aibitat-context-overflow - [`1ffbd29`](https://github.com/Mintplex-Labs/anything-llm/commit/1ffbd29215f57158b53b8803227ddb383061f4c4) sync master ### 📊 Changes **2 files changed** (+98 additions, -6 deletions) <details> <summary>View changed files</summary> 📝 `server/utils/agents/aibitat/index.js` (+19 -6) ➕ `server/utils/agents/aibitat/utils/messageWindow.js` (+79 -0) </details> ### 📄 Description ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [ ] ✨ feat (New feature) - [x] 🐛 fix (Bug fix) - [ ] ♻️ refactor (Code refactoring without changing behavior) - [ ] 💄 style (UI style changes) - [ ] 🔨 chore (Build, CI, maintenance) - [ ] 📝 docs (Documentation updates) ### Relevant Issues <!-- Use "resolves #xxx" to auto resolve on merge. Otherwise, please use "connect #xxx" --> resolves # ### Description <!-- Describe the changes in this PR that are impactful to the repo. What problem does it solve? --> - Agents were crashing on follow-up chats with a `Converting circular structure to JSON` error. - Root cause: `aibitat.provider` was being overwritten from the string `"openai"` to the provider class instance after each turn, so `Provider.contextLimit` silently fell back to an 8k window for any tool that called it. - Fixed by storing the active provider instance on `aibitat.activeProvider` so the string tag stays intact. - Added a defensive warning in `Provider.contextLimit` so this kind of regression can't sneak back in silently. - Added a trim that pops the oldest user/assistant pairs from agent history when the conversation grows past the model's context window — keeps the system prompt and current user message intact. - Trim reserves 15% of the context window for the model's response and tools. - Logs a warning and an introspect line whenever trimming happens so we can see it in user reports. ### Visuals (if applicable) <!-- Add screenshots or screen recordings to demonstrate the changes, especially for UI updates. --> ### Additional Information <!-- Add any other context about the Pull Request here that was not captured above. --> - Bug introduced in this PR: https://github.com/Mintplex-Labs/anything-llm/pull/5197 ### Developer Validations <!-- All of the applicable items should be checked. --> - [x] I ran `yarn lint` from the root of the repo & committed changes - [x] Relevant documentation has been updated (if applicable) - [x] I have tested my code functionality - [x] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 15:21:33 -04:00
yindo closed this issue 2026-06-05 15:21:33 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5502
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?