Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5719] [MERGED] Improve agent summarizer tool #5543

New Issue
Closed
opened 2026-06-05 15:21:40 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 15:21:40 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5719
Author: @shatfield4
Created: 5/28/2026
Status: ✅ Merged
Merged: 6/2/2026
Merged by: @timothycarambat

Base: master ← Head: 5718-summary-tool-chunked-rewrite

📝 Commits (6)

  • 31dd6bb rewrite agent summarizer to chunk by context window and drop langchain
  • 441fc92 pass aibitat into summarizer for introspection and tool approval
  • 9b0d88c drop unused caller param from summarizer introspect
  • ad01b1e Merge branch 'master' into 5718-summary-tool-chunked-rewrite
  • c6b6db6 minor nits
  • 5c0d89c Merge branch 'master' into 5718-summary-tool-chunked-rewrite

📊 Changes

5 files changed (+158 additions, -45 deletions)

View changed files

📝 server/utils/agentFlows/executors/web-scraping.js (+1 -0)
📝 server/utils/agents/aibitat/plugins/summarize.js (+1 -0)
📝 server/utils/agents/aibitat/plugins/web-scraping.js (+1 -0)
📝 server/utils/agents/aibitat/plugins/websocket.js (+5 -0)
📝 server/utils/agents/aibitat/utils/summarize.js (+150 -45)

📄 Description

Pull Request Type

  • ✨ feat (New feature)
  • 🐛 fix (Bug fix)
  • ♻️ refactor (Code refactoring without changing behavior)
  • 💄 style (UI style changes)
  • 🔨 chore (Build, CI, maintenance)
  • 📝 docs (Documentation updates)

Relevant Issues

resolves #5718

Description

  • Greatly improves UX of summarizing with small context window models
  • Removes LangChain deps from the summarize agent tool
  • Rewrites summarization to walk content one chunk at a time instead of the old map_reduce chain that never converged on small windows
  • Chunks content to 45% of the model's context window using the model tokenizer (see note in additional information section about other percentages tested)
  • Asks the model for only the key points of each section and accumulates them
  • Carries prior key points into the next chunk for continuity, capped at 5% of the window (truncated from the start so the latest details are kept)
  • Asks the user to continue after the first 3 sections so a long document never silently locks up the chat
  • Approving continues to the end without asking again, denying returns the key points gathered so far
  • Reports progress to the UI as each section is summarized

Visuals (if applicable)

Additional Information

  • I also tested CHUNK_CONTEXT_RATIO @ 0.5, 0.55, and 0.6 with no issues at all and would allow local models to be more performant (since less overall chunks) but we do run the risk of getting close to the actual context window limit the higher we make this percentage
  • Side note: tested a local version of https://github.com/Mintplex-Labs/anything-llm/pull/5548 merged with this PR (#5719) and it works much better than the current HTML scraper version we have
  • If combined with the markdown scraper PR, it removes all the css and js artifacts and the text is much cleaner to parse resulting is much less overall chunks (and more accurate links)
  • Chunks are much smaller and using a local model is still slow but much more bearable than the current implementation we have (no more infinite looping issue due to small context windows)

Developer Validations

Tested on:

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5719 **Author:** [@shatfield4](https://github.com/shatfield4) **Created:** 5/28/2026 **Status:** ✅ Merged **Merged:** 6/2/2026 **Merged by:** [@timothycarambat](https://github.com/timothycarambat) **Base:** `master` ← **Head:** `5718-summary-tool-chunked-rewrite` --- ### 📝 Commits (6) - [`31dd6bb`](https://github.com/Mintplex-Labs/anything-llm/commit/31dd6bbdcb194745de63a323123b6fb2b9964e3c) rewrite agent summarizer to chunk by context window and drop langchain - [`441fc92`](https://github.com/Mintplex-Labs/anything-llm/commit/441fc92b8d508cdc90f5a5335d905dcf30a54e58) pass aibitat into summarizer for introspection and tool approval - [`9b0d88c`](https://github.com/Mintplex-Labs/anything-llm/commit/9b0d88cc7b501dceb2f98a7f5e94889bbafa0f8d) drop unused caller param from summarizer introspect - [`ad01b1e`](https://github.com/Mintplex-Labs/anything-llm/commit/ad01b1e1f5a898d3a91a5df6b22400f94f29bdf4) Merge branch 'master' into 5718-summary-tool-chunked-rewrite - [`c6b6db6`](https://github.com/Mintplex-Labs/anything-llm/commit/c6b6db6c91d92dd1337fc5fb8b7bd8c99a7be7ba) minor nits - [`5c0d89c`](https://github.com/Mintplex-Labs/anything-llm/commit/5c0d89caaafb34527ad36616c5125bc7124b7d23) Merge branch 'master' into 5718-summary-tool-chunked-rewrite ### 📊 Changes **5 files changed** (+158 additions, -45 deletions) <details> <summary>View changed files</summary> 📝 `server/utils/agentFlows/executors/web-scraping.js` (+1 -0) 📝 `server/utils/agents/aibitat/plugins/summarize.js` (+1 -0) 📝 `server/utils/agents/aibitat/plugins/web-scraping.js` (+1 -0) 📝 `server/utils/agents/aibitat/plugins/websocket.js` (+5 -0) 📝 `server/utils/agents/aibitat/utils/summarize.js` (+150 -45) </details> ### 📄 Description ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [x] ✨ feat (New feature) - [ ] 🐛 fix (Bug fix) - [ ] ♻️ refactor (Code refactoring without changing behavior) - [ ] 💄 style (UI style changes) - [ ] 🔨 chore (Build, CI, maintenance) - [ ] 📝 docs (Documentation updates) ### Relevant Issues <!-- Use "resolves #xxx" to auto resolve on merge. Otherwise, please use "connect #xxx" --> resolves #5718 ### Description <!-- Describe the changes in this PR that are impactful to the repo. What problem does it solve? --> - Greatly improves UX of summarizing with small context window models - Removes LangChain deps from the summarize agent tool - Rewrites summarization to walk content one chunk at a time instead of the old map_reduce chain that never converged on small windows - Chunks content to 45% of the model's context window using the model tokenizer _(see note in additional information section about other percentages tested)_ - Asks the model for only the key points of each section and accumulates them - Carries prior key points into the next chunk for continuity, capped at 5% of the window (truncated from the start so the latest details are kept) - Asks the user to continue after the first 3 sections so a long document never silently locks up the chat - Approving continues to the end without asking again, denying returns the key points gathered so far - Reports progress to the UI as each section is summarized ### Visuals (if applicable) <!-- Add screenshots or screen recordings to demonstrate the changes, especially for UI updates. --> ### Additional Information <!-- Add any other context about the Pull Request here that was not captured above. --> - I also tested `CHUNK_CONTEXT_RATIO` @ `0.5`, `0.55`, and `0.6` with no issues at all and would allow local models to be more performant (since less overall chunks) but we do run the risk of getting close to the actual context window limit the higher we make this percentage - Side note: tested a local version of https://github.com/Mintplex-Labs/anything-llm/pull/5548 merged with this PR (#5719) and it works much better than the current HTML scraper version we have - If combined with the markdown scraper PR, it removes all the css and js artifacts and the text is much cleaner to parse resulting is much less overall chunks (and more accurate links) - Chunks are much smaller and using a local model is still slow but much more bearable than the current implementation we have (no more infinite looping issue due to small context windows) ### Developer Validations Tested on: - https://en.wikipedia.org/wiki/Artificial_intelligence - https://anythingllm.com/ - https://github.com/Mintplex-Labs/anything-llm <!-- All of the applicable items should be checked. --> - [x] I ran `yarn lint` from the root of the repo & committed changes - [x] Relevant documentation has been updated (if applicable) - [x] I have tested my code functionality - [x] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 15:21:40 -04:00
yindo closed this issue 2026-06-05 15:21:41 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5543
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?