Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4116] [BUG]: Embedded chunk is lesser than the chunk created from documents #2622

New Issue
Closed
opened 2026-02-22 18:30:29 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:30:29 -05:00
Owner
Copy Link
Copy Source

Originally created by @vikashrajgupta on GitHub (Jul 10, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4116

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

Hello there,

I started using Anything LLM a few weeks ago, and I must say — it’s very easy to use and set up.

I’m working with different types of documents like PPTs, DOCX, and PDFs. To ensure I upload only clean, structured text, I’ve been converting all these files to Markdown using Docking. Then, I upload the Markdown files into the Anything LLM workspace.

However, when I tested it by asking questions related to the uploaded content, I didn’t get the expected results — especially compared to other LLMs like Google’s Notebook LLM.

To improve the output, I started tweaking the default configuration settings. I increased the chunk size to 10,000 and the overlap size to 400. I also lowered the temperature and increased the context size — but none of these changes seemed to help.

While reviewing the logs, I noticed something strange: the number of chunks created from the document was significantly higher than the number of embeddings. I’m not completely sure, but this might be one of the reasons why the output lacks context.

Would it be possible to look into this and see if it can be fixed? I’d really like to continue using Anything LLM if I can get the desired results

Are there known steps to reproduce?

To reproduce this issue:

Try uploading a few files here and also on Notebook LLM for comparison.

Use the default config:

    Vector DB: LanceDB

    Embedder: Anything LLM Embedder

    Text chunk size: 1000

    Overlap size: 300–400

I’m using GPT-4 as the LLM

I tried both query and chat mode with the temperature set to 0.5–0.6.
Originally created by @vikashrajgupta on GitHub (Jul 10, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4116 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? Hello there, I started using Anything LLM a few weeks ago, and I must say — it’s very easy to use and set up. I’m working with different types of documents like PPTs, DOCX, and PDFs. To ensure I upload only clean, structured text, I’ve been converting all these files to Markdown using Docking. Then, I upload the Markdown files into the Anything LLM workspace. However, when I tested it by asking questions related to the uploaded content, I didn’t get the expected results — especially compared to other LLMs like Google’s Notebook LLM. To improve the output, I started tweaking the default configuration settings. I increased the chunk size to 10,000 and the overlap size to 400. I also lowered the temperature and increased the context size — but none of these changes seemed to help. While reviewing the logs, I noticed something strange: the number of chunks created from the document was significantly higher than the number of embeddings. I’m not completely sure, but this might be one of the reasons why the output lacks context. Would it be possible to look into this and see if it can be fixed? I’d really like to continue using Anything LLM if I can get the desired results ### Are there known steps to reproduce? To reproduce this issue: Try uploading a few files here and also on Notebook LLM for comparison. Use the default config: Vector DB: LanceDB Embedder: Anything LLM Embedder Text chunk size: 1000 Overlap size: 300–400 I’m using GPT-4 as the LLM I tried both query and chat mode with the temperature set to 0.5–0.6.
yindo added the possible buginvestigating labels 2026-02-22 18:30:29 -05:00
yindo closed this issue 2026-02-22 18:30:29 -05:00
yindo commented 2026-02-22 18:30:30 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Jul 10, 2025):

After investigating this, I can confirm that everything is working as intended here. It was actually just a misleading console log message displaying this incorrectly and I did confirm that there is a 1:1 relationship for chunks to embeddings.

If you still are not getting good RAG results using our default embedding model I would suggest you test out some other embedding models to see if those improve your results. You can download some local ones online and run them locally using LMStudio or you can also try out some hosted embedding models like the ones that Voyage AI offer. I would also suggest testing this out with less documents at a time to see if it improves your results too.

I'll be patching the misleading console message in a PR so this isn't confusing anymore.

@shatfield4 commented on GitHub (Jul 10, 2025): After investigating this, I can confirm that everything is working as intended here. It was actually just a misleading console log message displaying this incorrectly and I did confirm that there is a 1:1 relationship for chunks to embeddings. If you still are not getting good RAG results using our default embedding model I would suggest you test out some other embedding models to see if those improve your results. You can download some local ones online and run them locally using LMStudio or you can also try out some hosted embedding models like the ones that Voyage AI offer. I would also suggest testing this out with less documents at a time to see if it improves your results too. I'll be patching the misleading console message in a PR so this isn't confusing anymore.
yindo commented 2026-02-22 18:30:30 -05:00
Author
Owner
Copy Link
Copy Source

@vikashrajgupta commented on GitHub (Jul 11, 2025):

Okay, great then. Thanks for the prompt action @shatfield4

@vikashrajgupta commented on GitHub (Jul 11, 2025): Okay, great then. Thanks for the prompt action @shatfield4
yindo changed title from [BUG]: Embedded chunk is lesser than the chunk created from documents to [GH-ISSUE #4116] [BUG]: Embedded chunk is lesser than the chunk created from documents 2026-06-05 14:47:35 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2622
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?