Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1406] [BUG]: Bad result from RAG #899

New Issue
Closed
opened 2026-02-22 18:22:05 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:22:05 -05:00
Owner
Copy Link
Copy Source

Originally created by @CharlesBdg on GitHub (May 15, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1406

Originally assigned to: @timothycarambat on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

Hello,
Since 3 weeks, I get really bad result at RAG.
Before around 18th April, I was able to get accurate information from my files (around 150 txt files). Now, the LLM always said he didn't find any relevant context. When I click on "Show citation", the file where the information is, is not showing up.
For example :
I have a file called "How to enable Warp / Zero Trust" (from Cloudflare).
Inside the file it is written "How to enable Warp / Zero Trust".
I ask the question to the LLM "How to enable Warp / Zero Trust", Output : "Sorry I didn't find any relevant context" and the file is not in "Show citation".

LLM : Ollama local / llama3, phi3, openchat, mistral, same output
Embedding : Ollama / mxbai-embed-large
Vector database : LanceDB or Milvus (I've already tried a hard reset of the DB).

I even tested with the Desktop version (v1.5.4) (in case it was a Docker issue) but same issue.

I've already tried tweaking in "Vector Database" section of the Workspace the "Max Context Snippets" and "Document similarity threshold" but no result.

I don't know if something break with the merged of Agent or the bump version in lancedb deps or bump langchain deps or the code that do the RAG.

Maybe add the ability to completely disable the Agent and get back the old RAG.

PS : Great project, thank you

Are there known steps to reproduce?

No response

Originally created by @CharlesBdg on GitHub (May 15, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1406 Originally assigned to: @timothycarambat on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? Hello, Since 3 weeks, I get really bad result at RAG. Before around 18th April, I was able to get accurate information from my files (around 150 txt files). Now, the LLM always said he didn't find any relevant context. When I click on "Show citation", the file where the information is, is not showing up. For example : I have a file called "How to enable Warp / Zero Trust" (from [Cloudflare](https://one.one.one.one/)). Inside the file it is written "How to enable Warp / Zero Trust". I ask the question to the LLM "How to enable Warp / Zero Trust", Output : "Sorry I didn't find any relevant context" and the file is not in "Show citation". LLM : [Ollama](https://ollama.com/) local / [llama3](https://ollama.com/library/llama3), [phi3](https://ollama.com/library/phi3), [openchat](https://ollama.com/library/openchat), [mistral](https://ollama.com/library/mistral), same output Embedding : [Ollama](https://ollama.com/) / [mxbai-embed-large](https://ollama.com/library/mxbai-embed-large) Vector database : LanceDB or [Milvus](https://milvus.io/) (I've already tried a hard reset of the DB). I even tested with the Desktop version (v1.5.4) (in case it was a Docker issue) but same issue. I've already tried tweaking in "Vector Database" section of the Workspace the "Max Context Snippets" and "Document similarity threshold" but no result. I don't know if something break with the merged of [Agent](https://github.com/Mintplex-Labs/anything-llm/pull/1093) or the [bump version in lancedb deps](https://github.com/Mintplex-Labs/anything-llm/pull/1229) or [bump langchain deps](https://github.com/Mintplex-Labs/anything-llm/pull/1231) or the code that do the RAG. Maybe add the ability to completely disable the Agent and get back the old RAG. PS : Great project, thank you ### Are there known steps to reproduce? _No response_
yindo added the needs info / can't replicateinvestigating labels 2026-02-22 18:22:05 -05:00
yindo closed this issue 2026-02-22 18:22:05 -05:00
yindo commented 2026-02-22 18:22:06 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 15, 2024):

Agents do not run unless you invoke them, they have no impact on regular RAG chats

@timothycarambat commented on GitHub (May 15, 2024): Agents do not run unless you invoke them, they have no impact on regular RAG chats
yindo commented 2026-02-22 18:22:06 -05:00
Author
Owner
Copy Link
Copy Source

@CharlesBdg commented on GitHub (May 15, 2024):

Thanks for the information.

Here what I've found, I have embed the Wikipedia page of OpenAI
Default settings on the workspace.
Question : "How much Microsoft invested ?"
Answer : "Microsoft invested $1 billion in OpenAI, a non-profit artificial intelligence research organization, in 2019. This investment was part of a multi-year partnership aimed at advancing AI capabilities and promoting responsible AI development."
Real answer should be : "Microsoft provided OpenAI Global LLC with a $1 billion investment in 2019 and a $10 billion investment in 2023"
When I look at the "Citation", I don't even see the answer the LLM gave to me.
And every chunk of text have at least "60% match" but in reality, it's totally wrong (see pictures)
Capture d’écran du 2024-05-16 00-09-46
Capture d’écran du 2024-05-16 00-08-51
Capture d’écran du 2024-05-16 00-06-07

@CharlesBdg commented on GitHub (May 15, 2024): Thanks for the information. Here what I've found, I have embed the Wikipedia page of [OpenAI](https://en.wikipedia.org/wiki/OpenAI) Default settings on the workspace. Question : "How much Microsoft invested ?" Answer : "Microsoft invested $1 billion in OpenAI, a non-profit artificial intelligence research organization, in 2019. This investment was part of a multi-year partnership aimed at advancing AI capabilities and promoting responsible AI development." Real answer should be : "Microsoft provided OpenAI Global LLC with a $1 billion investment in 2019 and a $10 billion investment in 2023" When I look at the "Citation", I don't even see the answer the LLM gave to me. And every chunk of text have at least "60% match" but in reality, it's totally wrong (see pictures) ![Capture d’écran du 2024-05-16 00-09-46](https://github.com/Mintplex-Labs/anything-llm/assets/73182771/aef0cf76-4a7a-4ee1-acdf-6d96947e3a83) ![Capture d’écran du 2024-05-16 00-08-51](https://github.com/Mintplex-Labs/anything-llm/assets/73182771/8bf67c67-9fa8-4db4-8853-d019549b606b) ![Capture d’écran du 2024-05-16 00-06-07](https://github.com/Mintplex-Labs/anything-llm/assets/73182771/f8e7d981-7632-4a53-92ca-9c89674ed3e6)
yindo commented 2026-02-22 18:22:06 -05:00
Author
Owner
Copy Link
Copy Source

@man2004 commented on GitHub (May 15, 2024):

I am using gpt-4o and text-embedding-ada-002 in LLM and embedding respectively. Below is my result after adding the Wikipedia page of OpenAI:

Q: How much has Microsoft invested in OpenAI?
A: Microsoft has invested a total of $11 billion in OpenAI, with an initial $1 billion investment in 2019 and an additional $10 billion investment in 2023.

@man2004 commented on GitHub (May 15, 2024): I am using gpt-4o and text-embedding-ada-002 in LLM and embedding respectively. Below is my result after adding the Wikipedia page of OpenAI: Q: How much has Microsoft invested in OpenAI? A: Microsoft has invested a total of $11 billion in OpenAI, with an initial $1 billion investment in 2019 and an additional $10 billion investment in 2023.
yindo commented 2026-02-22 18:22:06 -05:00
Author
Owner
Copy Link
Copy Source

@CharlesBdg commented on GitHub (May 16, 2024):

It seems I had two issues :

  1. The embedding model I used has a context size too small (512 tokens), thanks to @man2004 who use text-embedding-ada-002 and has a context of 8192 tokens. I now use nomic-embed-text 8192 tokens with Ollama. It solved 50% of my issue. This website was helpful to understand the embedding size and chunk overlap. https://langchain-text-splitter.streamlit.app/
  2. It seems my AnythingLLM docker and storage/lanceDB was messy, so I have deleted everything and done a clean installation. Which solve the other 50%.

Now everything is back to normal and work has expected.
Thank you for your help.

@CharlesBdg commented on GitHub (May 16, 2024): It seems I had two issues : 1. The embedding model I used has a context size too small (512 tokens), thanks to @man2004 who use text-embedding-ada-002 and has a context of 8192 tokens. I now use [nomic-embed-text](https://ollama.com/library/nomic-embed-text) 8192 tokens with Ollama. It solved 50% of my issue. This website was helpful to understand the embedding size and chunk overlap. https://langchain-text-splitter.streamlit.app/ 2. It seems my AnythingLLM docker and storage/lanceDB was messy, so I have deleted everything and done a clean installation. Which solve the other 50%. Now everything is back to normal and work has expected. Thank you for your help.
yindo commented 2026-02-22 18:22:06 -05:00
Author
Owner
Copy Link
Copy Source

@jinwater88 commented on GitHub (Apr 16, 2025):

@CharlesBdg I also encountered the poor effect of using anythingllm+RAG. Can you provide your experience and suggestions?

@jinwater88 commented on GitHub (Apr 16, 2025): @CharlesBdg I also encountered the poor effect of using anythingllm+RAG. Can you provide your experience and suggestions?
yindo commented 2026-02-22 18:22:06 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 16, 2025):

There is a dedicated page about how to get better results. https://docs.anythingllm.com/llm-not-using-my-docs

@timothycarambat commented on GitHub (Apr 16, 2025): There is a dedicated page about how to get better results. https://docs.anythingllm.com/llm-not-using-my-docs
yindo changed title from [BUG]: Bad result from RAG to [GH-ISSUE #1406] [BUG]: Bad result from RAG 2026-06-05 14:37:49 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating needs info / can't replicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#899
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?