Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3035] [BUG]: " --prompt truncated for brevity--".When Uploading Long Files, #1944

New Issue
Closed
opened 2026-02-22 18:27:21 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:27:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @LeanOps101 on GitHub (Jan 27, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3035

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

When I uploading a txt file or pdf file (a report or a chapter of book), use the default setting (Text Chunk Size=512, Text Chunk Overlap=38), the default AnythingLLM Embedder, with LanceDB Vector Database.

I see the Vector Count is around 100 -300.it can successfully embedded,

When I ask the following questions, it always answers wrong.

  • what is the key context about, how many words in the context?

  • How many truncate labels in the context? where are they?

  • Can you find "XXX" in the context?

I guesses this maybe the reason the LLM can't find certain information obviously listed in the context.

How can I solve this "--prompt truncated for brevity--" problem? Please Help~~ Appreciated!!!

Are there known steps to reproduce?

with AnythingLLM 1.7.2 , Upload any "long" files, about 8000 characters (for example , Ray Dalio's .

Ask:

  • what is the key context about, how many words in the context?

  • How many truncate labels in the context? where are they?

  • Can you find "XXX" in the context?

Originally created by @LeanOps101 on GitHub (Jan 27, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3035 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? When I uploading a txt file or pdf file (a report or a chapter of book), use the default setting (Text Chunk Size=512, Text Chunk Overlap=38), the default AnythingLLM Embedder, with LanceDB Vector Database. I see the Vector Count is around 100 -300.it can successfully embedded, When I ask the following questions, it always answers wrong. - what is the key context about, how many words in the context? - How many truncate labels in the context? where are they? - Can you find "XXX" in the context? I guesses this maybe the reason the LLM can't find certain information obviously listed in the context. How can I solve this "--prompt truncated for brevity--" problem? Please Help~~ Appreciated!!! ### Are there known steps to reproduce? with AnythingLLM 1.7.2 , Upload any "long" files, about 8000 characters (for example , Ray Dalio's <Principles>. Ask: - what is the key context about, how many words in the context? - How many truncate labels in the context? where are they? - Can you find "XXX" in the context?
yindo added the possible bug label 2026-02-22 18:27:21 -05:00
yindo closed this issue 2026-02-22 18:27:21 -05:00
yindo commented 2026-02-22 18:27:21 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 27, 2025):

When I uploading a txt file or pdf file (a report or a chapter of book), use the default setting (Text Chunk Size=512, Text Chunk Overlap=38), the default AnythingLLM Embedder, with LanceDB Vector Database.
I see the Vector Count is around 100 -300.it can successfully embedded,

When I ask the following questions, it always answers wrong.
what is the key context about, how many words in the context?

LLMs are not calculators and also you are asking for full context comprehension in a RAG system. RAG, by its very nature is pieces of relevant content. Not the entire text

How many truncate labels in the context? where are they?

Again, would require the full text to know

Can you find "XXX" in the context?

This also would require the full text, but could work in RAG, but unlikely.

You should read this doc about how all RAG systems work: https://docs.anythingllm.com/llm-not-using-my-docs#llms-do-not-introspect

Additionally, the --prompt truncated for brevity-- is not a problem, it's actually helping you from having a problem. Your LLM does not have a large enough context window to fit all of your documents into it. If we just let that happen your computer would either lock up totally from RAM and CPU going to 100% or the LLM would just fail.

You cannot fit an infinite amount of text into an LLM, not even the largest models can accomplish this - there is always a limit. RAG (which is what we use) enables us to chunk the document and then ask retrieve only the bits and pieces the make sense for your question and use that in the context window.

This makes larger documents easier to use, but it is at the expense of these types of "whole document" understandings. Instead you should ask questions about the content not the document itself.

The LLM has no idea what a document is, what it looks like, how many pages, and not the entire text. The only way to even accomplish this is using those very large context models like Gemini (2M) or Anthropic - again they still have limits and ingest that many tokens at once is expensive. Thus why RAG exists.

@timothycarambat commented on GitHub (Jan 27, 2025): > When I uploading a txt file or pdf file (a report or a chapter of book), use the default setting (Text Chunk Size=512, Text Chunk Overlap=38), the default AnythingLLM Embedder, with LanceDB Vector Database. > I see the Vector Count is around 100 -300.it can successfully embedded, >When I ask the following questions, it always answers wrong. > what is the key context about, how many words in the context? LLMs are not calculators and also you are asking for _full context_ comprehension in a RAG system. RAG, by its very nature is _pieces_ of relevant content. Not the entire text > How many truncate labels in the context? where are they? Again, would require the full text to know > Can you find "XXX" in the context? This also would require the full text, but _could_ work in RAG, but unlikely. You should read this doc about how _all_ RAG systems work: https://docs.anythingllm.com/llm-not-using-my-docs#llms-do-not-introspect Additionally, the `--prompt truncated for brevity--` is not a problem, it's actually helping you from having a problem. Your LLM does not have a large enough context window to fit all of your documents into it. If we just let that happen your computer would either lock up totally from RAM and CPU going to 100% or the LLM would just fail. You cannot fit an infinite amount of text into an LLM, not even the largest models can accomplish this - there is always a limit. RAG (which is what we use) enables us to chunk the document and then ask retrieve only the bits and pieces the make sense for your question and use that in the context window. This makes larger documents easier to use, but it is at the expense of these types of "whole document" understandings. Instead you should ask questions about the _content_ not the _document itself_. The LLM has no idea what a document is, what it looks like, how many pages, and not the entire text. The only way to even accomplish this is using those very large context models like Gemini (2M) or Anthropic - again they still have limits and ingest that many tokens at once is _expensive_. Thus why RAG exists.
yindo changed title from [BUG]: " --prompt truncated for brevity--".When Uploading Long Files, to [GH-ISSUE #3035] [BUG]: " --prompt truncated for brevity--".When Uploading Long Files, 2026-06-05 14:43:36 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1944
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?