Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4783] [BUG]: Unexpected PC Workstation denial of service due to large corpus document vectorization while Updating workspace #3011

New Issue
Closed
opened 2026-02-22 18:32:15 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:32:15 -05:00
Owner
Copy Link
Copy Source

Originally created by @uruiamme on GitHub (Dec 14, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4783

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

After files are uploaded to the Document window and the user chooses to Embed, the vectorization begins to process files on the backend while a message is displayed, "Updating workspace..." This process can take days, and it can be very resource intensive. It uses the CPU and main memory for vectorization. If a large number of PDFs of differing size and complexity are part of the corpus, the process can get out of hand.

There's a PDF size that, once exceeded, the background vectorization will undergo crippling, excessive resource usage. For example, a large PDF over 70 MB could be too large to fit into main memory. If it does, the OS will begin to use virtual memory, a.k.a. swap. But unlike other processes (like a browser or a photo editor) that are swapped to disk, the vectorization process will need to utilize the entirety of the data it expects to be in DRAM. This forces the OS to swap the process's data in and out of the swap file at several hundred MB per second. (e.g. 250 MB/s read, 750 MB/s write followed by 1 GB/s read, 300 MB/s write)

This effectively serves as a denial of service for the PC user while the vectorization continues. I define crippling resource usage as: CPU usage in the 30 to 70 percent range, 100% memory usage, and 100% NVMe usage on the system (swap) drive.

Image

This will be accompanied by the tail of the output log mentioning that it is "Caching vectorized results of xxx.pdf-hex_code.json" and "Snippets created" from such document.

Subsequent to the denial of service due to the vectorization of a large file, PC resource usage may return to normal. This is due to the processing of smaller files in the corpus which fit comfortably in main memory.

Meanwhile, the Documents window continues to say "Updating workspace..." as its only notice on AnythingLLM desktop, no matter the resources being used.

Except for the fact that vectorization is processed in alphabetical order from the uploaded files, the user will likely be unaware if and when the process will exhibit crippling resource usage (as above) or modest usage, e.g. 47% CPU usage, 61% memory usage, and 1% NVMe usage. The switch between crippling and moderate resource usage will be seemingly random, with 10 minutes of normal followed by 2 hours of crippling usage and back to normal for several hours.

Image

Suggestion: A calculation for the necessary DRAM size should be made prior to vectorization. If a file will exceed a certain threshold, the file should be pre-processed into smaller pieces that can be later combined.

Suggestion: If the size at which a corpus document becomes too large can be predetermined, inform the user to either remove them from the corpus as a mitigation or beware that crippling resource usage will occur each time a large file is vectorized.

Are there known steps to reproduce?

Include book-length PDFs within a corpus of uploaded files, for example those from the Internet Archive or Google Books. Examples for testing purposes would include several types of nonfiction: dictionaries, encyclopedias, textbooks, collected works volumes, and government documents. Even a large fiction book could exceed the threshold.

Originally created by @uruiamme on GitHub (Dec 14, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4783 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? After files are uploaded to the Document window and the user chooses to Embed, the vectorization begins to process files on the backend while a message is displayed, "Updating workspace..." This process can take days, and it can be very resource intensive. It uses the CPU and main memory for vectorization. If a large number of PDFs of differing size and complexity are part of the corpus, the process can get out of hand. There's a PDF size that, once exceeded, the background vectorization will undergo **crippling, excessive resource usage.** For example, a large PDF over 70 MB could be too large to fit into main memory. If it does, the OS will begin to use virtual memory, a.k.a. swap. But unlike other processes (like a browser or a photo editor) that are swapped to disk, the vectorization process will need to utilize the entirety of the data it expects to be in DRAM. This forces the OS to swap the process's data in and out of the swap file at several hundred MB per second. (e.g. 250 MB/s read, 750 MB/s write followed by 1 GB/s read, 300 MB/s write) **This effectively serves as a denial of service for the PC user while the vectorization continues. I define crippling resource usage as: CPU usage in the 30 to 70 percent range, 100% memory usage, and 100% NVMe usage on the system (swap) drive.** <img width="1342" height="603" alt="Image" src="https://github.com/user-attachments/assets/2361e1f2-6298-425d-b387-dc79c0494513" /> This will be accompanied by the tail of the output log mentioning that it is "Caching vectorized results of xxx.pdf-hex_code.json" and "Snippets created" from such document. Subsequent to the denial of service due to the vectorization of a large file, PC resource usage may return to normal. This is due to the processing of smaller files in the corpus which fit comfortably in main memory. Meanwhile, the Documents window continues to say "Updating workspace..." as its only notice on AnythingLLM desktop, no matter the resources being used. Except for the fact that vectorization is processed in alphabetical order from the uploaded files, the user will likely be unaware if and when the process will exhibit crippling resource usage (as above) or modest usage, e.g. 47% CPU usage, 61% memory usage, and 1% NVMe usage. The switch between crippling and moderate resource usage will be seemingly random, with 10 minutes of normal followed by 2 hours of crippling usage and back to normal for several hours. <img width="1349" height="603" alt="Image" src="https://github.com/user-attachments/assets/87c9538e-21aa-4549-9812-c831bf15f46b" /> Suggestion: A calculation for the necessary DRAM size should be made prior to vectorization. If a file will exceed a certain threshold, the file should be pre-processed into smaller pieces that can be later combined. Suggestion: If the size at which a corpus document becomes too large can be predetermined, inform the user to either remove them from the corpus as a mitigation or beware that crippling resource usage will occur each time a large file is vectorized. ### Are there known steps to reproduce? Include book-length PDFs within a corpus of uploaded files, for example those from the Internet Archive or Google Books. Examples for testing purposes would include several types of nonfiction: dictionaries, encyclopedias, textbooks, collected works volumes, and government documents. Even a large fiction book could exceed the threshold.
yindo added the possible bug label 2026-02-22 18:32:15 -05:00
yindo closed this issue 2026-02-22 18:32:15 -05:00
yindo commented 2026-02-22 18:32:15 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 15, 2025):

This is not really a bug, but more of a side effect of an area of sub-optimization of the embedder. That being said, the default embedder model (which can be changed to run on GPU or cloud) would, of course, be hardware-limited/constrained since if we are intending to embed a corpus of documents. In situations like this, offloading embedding to a cloud provider would be best.

The parsing of the document does not seem to be the issue - just the much more intensive embedding action. We have the embedding already not run in concurrency and instead run sequentially while also encouraging GC every loop, simply to try to work around this.

That being said, if the content is millions of words, at some point, we need to run that process and generate embeddings. If the embedding model output dimensions are large this is a lot of data to write every chunk of content with the addition of upsert into the vector database.

For optimizations, there are two items we are already aware of for optimizations/refactors:

  • Sending process to the background to unblock the main thread via a forked process
  • Writing to vector db during the embed process so we do not have to keep copies of embeddings for later upsertion, which can be massive with a large document, large embedder output dimensions, or both.

The second point is the bottleneck from the issue you described above. All this to say, we are aware of these performance considerations specifically with the default CPU/based embedder and especially with the in-process memory for large files or embedder outputs.

@timothycarambat commented on GitHub (Dec 15, 2025): This is not really a bug, but more of a side effect of an area of sub-optimization of the embedder. That being said, the default embedder model (which can be changed to run on GPU or cloud) would, of course, be hardware-limited/constrained since if we are intending to embed a corpus of documents. In situations like this, offloading embedding to a cloud provider would be best. The _parsing_ of the document does not seem to be the issue - just the much more intensive embedding action. We have the embedding already not run in concurrency and instead run sequentially while also encouraging GC every loop, simply to try to work around this. That being said, if the content is millions of words, at some point, we need to run that process and generate embeddings. If the embedding model output dimensions are large this is a lot of data to write every chunk of content with the addition of upsert into the vector database. For optimizations, there are two items we are already aware of for optimizations/refactors: - Sending process to the background to unblock the main thread via a forked process - Writing to vector db during the embed process so we do not have to keep copies of embeddings for later upsertion, which can be massive with a large document, large embedder output dimensions, or both. The second point is the bottleneck from the issue you described above. All this to say, we are aware of these performance considerations _specifically_ with the default CPU/based embedder and especially with the in-process memory for large files or embedder outputs.
yindo changed title from [BUG]: Unexpected PC Workstation denial of service due to large corpus document vectorization while Updating workspace to [GH-ISSUE #4783] [BUG]: Unexpected PC Workstation denial of service due to large corpus document vectorization while Updating workspace 2026-06-05 14:49:48 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3011
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?