Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #994] [BUG]: When using Ollama I never have relevant context provided in the chat or the query mode #607

New Issue
Closed
opened 2026-02-22 18:20:26 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:20:26 -05:00
Owner
Copy Link
Copy Source

Originally created by @maxime1992 on GitHub (Mar 29, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/994

How are you running AnythingLLM?

Docker (local)

What happened?

I'm able to upload documents and put them in my workspace.

But whether I am in chat mode or query mode, there's never anything relevant about my documents.

In query mode, it even says "There is no relevant information in this workspace to answer your query."

Are there known steps to reproduce?

  • Installed the app and set it up to use Ollama. Then I just picked the defaults for the other prompts
  • Use a model like llama2
  • Upload document and put it in the workspace
  • Ask questions in chat and query mode

See there's nothing relevant

Originally created by @maxime1992 on GitHub (Mar 29, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/994 ### How are you running AnythingLLM? Docker (local) ### What happened? I'm able to upload documents and put them in my workspace. But whether I am in chat mode or query mode, there's never anything relevant about my documents. In query mode, it even says "There is no relevant information in this workspace to answer your query." ### Are there known steps to reproduce? - Installed the app and set it up to use Ollama. Then I just picked the defaults for the other prompts - Use a model like llama2 - Upload document and put it in the workspace - Ask questions in chat and query mode See there's nothing relevant
yindo added the possible bug label 2026-02-22 18:20:26 -05:00
yindo closed this issue 2026-02-22 18:20:26 -05:00
yindo commented 2026-02-22 18:20:27 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 29, 2024):

This is a configuration problem, not a bug. Pasting this response i have used prior which covers everything you need to know to know how to help yourself resolve the issue.

the reasoning for "mixed results" is going to be a configuration issue that you will have to configure for your specific use case or document set. The defaults we provide work most of the time for most people, but not all of the time for everything - because that is not how the system works since it is not omnipotent! That being said, the main controls you want to be messing with that usually alleviate any hallucinations or logical misses are:

  1. Document Similarity Threshold: This is the minimum relevancy score, by vector distance, a text chunk has. Depending on the # of vectors in the workspace, the embedder used, and even the prompt, this value can change. The default is 20% relevant, but this is not exact and is based on vector similarity, which is a mathematic value, not a semantic one. Changing this to "No Restriction" often helps.

  2. Context Snippet Count: This is how many maximum pieces of context can be used for any given prompt response. The default is 4, however, you may help to widen the search by increasing this to 6, context window permitting. This in conjunction with the above can really get rid of a lot of issues the default config may give you.

  3. Chat/Query Mode: Chat mode will allow the LLM's general knowledge to attempt to fill in gaps in logic the context dont fill - this is often the root cause of a hallucination since most document sets tend to be out of the domain the LLM was trained on. Query mode has the LLM only responding when context is found which keeps its general knowledge from overwhelming the response.

  4. Document Pinning: The last measure, but fully complete, is Document pinning. This method is only wisely used on large-context models like Claude-3 or GPT-32k. This does a full-text insertion of the pinned document into the prompt window for 100% coherence. This should be used very sparingly, if at all.

@timothycarambat commented on GitHub (Mar 29, 2024): This is a configuration problem, not a bug. Pasting this response i have used prior which covers everything you need to know to know how to help yourself resolve the issue. > the reasoning for "mixed results" is going to be a configuration issue that you will have to configure for your specific use case or document set. The defaults we provide work most of the time for most people, but not all of the time for everything - because that is not how the system works since it is not omnipotent! That being said, the main controls you want to be messing with that usually alleviate any hallucinations or logical misses are: > > 1. Document Similarity Threshold: This is the minimum relevancy score, by vector distance, a text chunk has. Depending on the # of vectors in the workspace, the embedder used, and even the prompt, this value can change. The default is 20% relevant, but this is not exact and is based on vector similarity, which is a mathematic value, not a semantic one. Changing this to "No Restriction" often helps. > > 2. Context Snippet Count: This is how many maximum pieces of context can be used for any given prompt response. The default is 4, however, you may help to widen the search by increasing this to 6, context window permitting. This in conjunction with the above can really get rid of a lot of issues the default config may give you. > > 3. Chat/Query Mode: Chat mode will allow the LLM's general knowledge to attempt to fill in gaps in logic the context dont fill - this is often the root cause of a hallucination since most document sets tend to be out of the domain the LLM was trained on. Query mode has the LLM only responding when context is found which keeps its general knowledge from overwhelming the response. > > 4. Document Pinning: The last measure, but fully complete, is Document pinning. This method is only wisely used on large-context models like Claude-3 or GPT-32k. This does a full-text insertion of the pinned document into the prompt window for 100% coherence. This should be used very sparingly, if at all.
yindo changed title from [BUG]: When using Ollama I never have relevant context provided in the chat or the query mode to [GH-ISSUE #994] [BUG]: When using Ollama I never have relevant context provided in the chat or the query mode 2026-06-05 14:36:15 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#607
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?