Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3700] [BUG]: Memory Management in Multi-User Environments with @agent #2385

New Issue
Closed
opened 2026-02-22 18:29:26 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:29:26 -05:00
Owner
Copy Link
Copy Source

Originally created by @roller100 on GitHub (Apr 22, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3700

How are you running AnythingLLM?

Docker (remote machine)

Our Experience

We wanted to share our experience deploying AnythingLLM on AWS EC2 instances in a multi-user environment, where we encountered some memory management challenges. We're sharing these findings in the spirit of collaboration, hoping it might help others and contribute to the continued improvement of this excellent platform.

What We Observed

When running AnythingLLM with multiple concurrent users, we noticed:

  • Memory usage would gradually increase, especially when users were actively using @agent features
  • In environments with limited memory (2GB RAM), the Node.js process would eventually be terminated
  • The application log showed: /usr/local/bin/docker-entrypoint.sh: line 7: 103 Killed node /app/server/index.js

This seemed to happen more frequently when:

  • Multiple users were logged in simultaneously
  • Several users were making @agent requests concurrently
  • RAG queries were being processed alongside @agent operations

Our Environment

For context, our setup was:

  • AWS EC2 t3.small instance (2GB RAM)
  • Docker container running AnythingLLM latest
  • Multiple concurrent users
  • Claude models (claude-3-sonnet and claude-3-5-sonnet)
  • LanceDB for vector storage
  • Several MCP servers for extended functionality

Relevant Logs

We noticed these log entries appeared shortly before the memory issues:

[2025-04-22T14:37:11Z WARN  lance_core::utils::tokio] Number of CPUs is less than or equal to the number of IO core reservations. This is not a supported configuration. using 1 CPU for compute intensive tasks.
[backend] info: [AgentHandler] Attached rag-memory plugin to Agent cluster
[backend] info: [AgentHandler] Attached document-summarizer plugin to Agent cluster
[backend] info: [AgentHandler] Attached web-scraping plugin to Agent cluster
[backend] info: [AgentHandler] Attached create-chart plugin to Agent cluster
[backend] info: [AgentHandler] Attached save-file-to-browser plugin to Agent cluster
[backend] info: [MCPCompatibilityLayer] Failed to start MCP server: BNKG {"error":"Connection timeout"...}

How We Addressed It

We found these adjustments helped stabilize our deployment:

  1. Upgrading to a t3.medium instance (4GB RAM)
  2. Adding Docker memory limits to provide better resource boundaries: 
    --memory=3g --memory-swap=3g
  3. Setting up CloudWatch monitoring to keep an eye on memory patterns

Ideas for Discussion

We'd love to collaborate on potential enhancements that might help others:

  1. Memory Usage Patterns: Has anyone else observed how memory is handled when users log off or sessions become inactive? We're curious if there are opportunities for more aggressive cleanup.

  2. Multi-User Considerations: Perhaps the docs could include some guidance on sizing for multi-user deployments? Our experience suggests t3.medium or larger for production multi-user environments.

  3. Observability: We found CloudWatch monitoring invaluable - would built-in memory metrics or warnings be helpful for others?

  4. Resource Efficiency: We wonder if there might be ways to make plugin loading more efficient, perhaps loading them on-demand or unloading them when not in use?

Community Benefit

We're sharing this in hopes that:

  • Others might avoid similar issues in their deployments
  • We can learn from others who may have found different solutions
  • These insights might contribute to making AnythingLLM even more robust in multi-user scenarios

We're happy to provide any additional information that might be helpful, and we're grateful for this excellent platform that has become a key part of our workflow!

Originally created by @roller100 on GitHub (Apr 22, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3700 ### How are you running AnythingLLM? Docker (remote machine) ## Our Experience We wanted to share our experience deploying AnythingLLM on AWS EC2 instances in a multi-user environment, where we encountered some memory management challenges. We're sharing these findings in the spirit of collaboration, hoping it might help others and contribute to the continued improvement of this excellent platform. ## What We Observed When running AnythingLLM with multiple concurrent users, we noticed: - Memory usage would gradually increase, especially when users were actively using @agent features - In environments with limited memory (2GB RAM), the Node.js process would eventually be terminated - The application log showed: `/usr/local/bin/docker-entrypoint.sh: line 7: 103 Killed node /app/server/index.js` This seemed to happen more frequently when: - Multiple users were logged in simultaneously - Several users were making @agent requests concurrently - RAG queries were being processed alongside @agent operations ## Our Environment For context, our setup was: - AWS EC2 t3.small instance (2GB RAM) - Docker container running AnythingLLM latest - Multiple concurrent users - Claude models (claude-3-sonnet and claude-3-5-sonnet) - LanceDB for vector storage - Several MCP servers for extended functionality ## Relevant Logs We noticed these log entries appeared shortly before the memory issues: ``` [2025-04-22T14:37:11Z WARN lance_core::utils::tokio] Number of CPUs is less than or equal to the number of IO core reservations. This is not a supported configuration. using 1 CPU for compute intensive tasks. [backend] info: [AgentHandler] Attached rag-memory plugin to Agent cluster [backend] info: [AgentHandler] Attached document-summarizer plugin to Agent cluster [backend] info: [AgentHandler] Attached web-scraping plugin to Agent cluster [backend] info: [AgentHandler] Attached create-chart plugin to Agent cluster [backend] info: [AgentHandler] Attached save-file-to-browser plugin to Agent cluster [backend] info: [MCPCompatibilityLayer] Failed to start MCP server: BNKG {"error":"Connection timeout"...} ``` ## How We Addressed It We found these adjustments helped stabilize our deployment: 1. Upgrading to a t3.medium instance (4GB RAM) 2. Adding Docker memory limits to provide better resource boundaries: ``` --memory=3g --memory-swap=3g ``` 3. Setting up CloudWatch monitoring to keep an eye on memory patterns ## Ideas for Discussion We'd love to collaborate on potential enhancements that might help others: 1. **Memory Usage Patterns**: Has anyone else observed how memory is handled when users log off or sessions become inactive? We're curious if there are opportunities for more aggressive cleanup. 2. **Multi-User Considerations**: Perhaps the docs could include some guidance on sizing for multi-user deployments? Our experience suggests t3.medium or larger for production multi-user environments. 3. **Observability**: We found CloudWatch monitoring invaluable - would built-in memory metrics or warnings be helpful for others? 4. **Resource Efficiency**: We wonder if there might be ways to make plugin loading more efficient, perhaps loading them on-demand or unloading them when not in use? ## Community Benefit We're sharing this in hopes that: - Others might avoid similar issues in their deployments - We can learn from others who may have found different solutions - These insights might contribute to making AnythingLLM even more robust in multi-user scenarios We're happy to provide any additional information that might be helpful, and we're grateful for this excellent platform that has become a key part of our workflow!
yindo added the possible bug label 2026-02-22 18:29:26 -05:00
yindo closed this issue 2026-02-22 18:29:26 -05:00
yindo commented 2026-02-22 18:29:27 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 23, 2025):

So i would not categorize this as a bug, i guess more of a question but allow me to shed some insights here that would explain basically everything you have run into.

So first, as you found, 2GB is not a lot to work with given

  • Assuming embedder preference is using default (every user query is embedded for RAG search!)
  • You could also be seeing load on requests if also using Accuracy Optimized search (reranking)
  • MCP services could be impacting memory allocation since MCP will spawn them, allocating memory for a new NodeJS process

Ultimately, given your review - your major bottleneck on such an instance will be the built-in embedder model running on CPU. The default embedder runs on two occasions:

  • You embed a document that has never been embedded before
  • A user sends a query on a workspace that has a non-zero amount of vectors.

As you can imagine, a lot of queries at the same time, the CPU and memory will gain dramatically or at least spike since each req it being handled independently. The actual search done via LanceDB would not impact anything here. If you moved to an off-instance embedder I would bet the majority of your memory problems resolve

We even have a warning about this on our own hosted services
https://docs.anythingllm.com/cloud/limitations#limited-capacity-for-built-in-embedder

@timothycarambat commented on GitHub (Apr 23, 2025): So i would not categorize this as a bug, i guess more of a question but allow me to shed some insights here that would explain basically everything you have run into. So first, as you found, 2GB is not a lot to work with given - Assuming embedder preference is using default (every user query is embedded for RAG search!) - You _could_ also be seeing load on requests if also using [Accuracy Optimized search (reranking)](https://docs.anythingllm.com/llm-not-using-my-docs#vector-database-settings--search-preference) - MCP services _could_ be impacting memory allocation since MCP will `spawn` them, allocating memory for a new NodeJS process Ultimately, given your review - your major bottleneck on such an instance will be the built-in embedder model running on CPU. The default embedder runs on two occasions: - You embed a document that has never been embedded before - A user sends a query on a workspace that has a non-zero amount of vectors. As you can imagine, a lot of queries at the same time, the CPU and memory will gain dramatically or at least spike since each req it being handled independently. The actual search done via LanceDB would not impact anything here. If you moved to an off-instance embedder I would bet the majority of your memory problems resolve We even have a warning about this on our own hosted services https://docs.anythingllm.com/cloud/limitations#limited-capacity-for-built-in-embedder
yindo changed title from [BUG]: Memory Management in Multi-User Environments with @agent to [GH-ISSUE #3700] [BUG]: Memory Management in Multi-User Environments with @agent 2026-06-05 14:46:15 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2385
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?