Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4422] [FEAT]: Chat query strict mode #2821

New Issue
Closed
opened 2026-02-22 18:31:24 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:31:24 -05:00
Owner
Copy Link
Copy Source

Originally created by @cbk0313 on GitHub (Sep 20, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4422

How are you running AnythingLLM?

Docker (local)

What happened?

Bug Description

In query mode, when no relevant documents are found for the current question, the system should return a refusal
message. However, the backfill logic incorrectly uses documents from previous chat history, causing query mode to
be bypassed and return responses based on irrelevant context.

Root Cause Analysis

If the embedding vector search results are fewer than the value set in Max Context Snippets (even if it is set to the minimum of 1, when the vector search results are 0 = no results, this is still less than 1), backfilling occurs and the previous context is added to 'contextTexts'.

After backfilling, the query mode refusal response condition is incorrect, so the question always passes through and query mode effectively gets bypassed.

##Current code example:

if (chatMode === "query" && contextTexts.length === 0)

At the beginning, the reason it successfully did not answer about Google is that there was no previous conversation history, so even though backfilling occurred, contextTexts.length was still 0.

Proposed restrictive fix:

if (chatMode === "query" && vectorSearchResults.sources.length === 0) {

Files Affected

  • server/utils/chats/stream.js:208
  • server/utils/chats/apiChatHandler.js:263, 606
  • server/utils/chats/openaiCompatible.js:125, 359
  • server/utils/chats/embed.js:137

With this change, if there are no vector search results, the system will not answer the query.

In reality, since I don’t have exact knowledge of the query response internals, I’m not sure if this is the fully correct implementation, but this modification seems to work as expected.

Are there known steps to reproduce?

  1. Create a workspace and set chat mode to "query"
  2. Upload a document (e.g., apple.txt about Apple Inc.)
  3. Ask an unrelated question: "Tell me about Google" → ✅ Works correctly(response: "There is no relevant information in this workspace to answer your query.")
  4. Ask a related question: "Tell me about Apple" → ✅ Works correctly(response: "Description of Apple")
  5. Ask an unrelated question: "Tell me about Google" → ❌ Bug occurs(response: "Description of Google")
  6. Ask an unrelated question: "everything" → ❌ Bug occurs (response: "answer everything")
Originally created by @cbk0313 on GitHub (Sep 20, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4422 ### How are you running AnythingLLM? Docker (local) ### What happened? ## Bug Description In query mode, when no relevant documents are found for the current question, the system should return a refusal message. However, the backfill logic incorrectly uses documents from previous chat history, causing query mode to be bypassed and return responses based on irrelevant context. ## Root Cause Analysis If the embedding vector search results are fewer than the value set in Max Context Snippets (even if it is set to the minimum of 1, when the vector search results are 0 = no results, this is still less than 1), backfilling occurs and the previous context is added to 'contextTexts'. After backfilling, the query mode refusal response condition is incorrect, so the question always passes through and query mode effectively gets bypassed. ##Current code example: if (chatMode === "query" && contextTexts.length === 0) At the beginning, the reason it successfully did not answer about Google is that there was no previous conversation history, so even though backfilling occurred, contextTexts.length was still 0. Proposed restrictive fix: if (chatMode === "query" && vectorSearchResults.sources.length === 0) { ## Files Affected - `server/utils/chats/stream.js:208` - `server/utils/chats/apiChatHandler.js:263, 606` - `server/utils/chats/openaiCompatible.js:125, 359` - `server/utils/chats/embed.js:137` With this change, if there are no vector search results, the system will not answer the query. In reality, since I don’t have exact knowledge of the query response internals, I’m not sure if this is the fully correct implementation, but this modification seems to work as expected. ### Are there known steps to reproduce? 1. Create a workspace and set chat mode to "query" 2. Upload a document (e.g., apple.txt about Apple Inc.) 3. Ask an unrelated question: "Tell me about Google" → ✅ Works correctly(response: "There is no relevant information in this workspace to answer your query.") 4. Ask a related question: "Tell me about Apple" → ✅ Works correctly(response: "Description of Apple") 5. Ask an unrelated question: "Tell me about Google" → ❌ Bug occurs(response: "Description of Google") 6. Ask an unrelated question: "everything" → ❌ Bug occurs (response: "answer everything")
yindo added the enhancementfeature request labels 2026-02-22 18:31:24 -05:00
yindo closed this issue 2026-02-22 18:31:25 -05:00
yindo commented 2026-02-22 18:31:25 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 22, 2025):

This is intentional because you are not considering the use case of a follow-up question. The way you outline this request is the way that query used to work, but what wound up happening in people would ask a semantically searchable question, then because the data is chunked, would ask something like "tell me more about that" and the query fails because on it's own 'Tell me more about that" produces no good vector search results.

The tradeoff is that what you showed above can occur. There is an opportunity for maybe a "strict" mode for query that functions the way you want it and how it used to work, but this is not a bug

@timothycarambat commented on GitHub (Sep 22, 2025): This is intentional because you are not considering the use case of a follow-up question. The way you outline this request is the way that `query` used to work, but what wound up happening in people would ask a semantically searchable question, then because the data is chunked, would ask something like "tell me more about that" and the query fails because on it's own 'Tell me more about that" produces no good vector search results. The tradeoff is that what you showed above can occur. There is an opportunity for maybe a "strict" mode for query that functions the way you want it and how it used to work, but this is not a bug
yindo commented 2026-02-22 18:31:25 -05:00
Author
Owner
Copy Link
Copy Source

@cbk0313 commented on GitHub (Sep 22, 2025):

I understand your intention. There's a follow-up question after searching the documentation.
For example, I hadn't considered a case like "Compare it to B" after asking A. This would require a system prompt to block it or a custom fix like strict mode.
It's more complex than I thought.
The code I presented would likely return a valid answer if A and B are mixed, so setting the system prompt seems like the right approach.

@cbk0313 commented on GitHub (Sep 22, 2025): I understand your intention. There's a follow-up question after searching the documentation. For example, I hadn't considered a case like "Compare it to B" after asking A. This would require a system prompt to block it or a custom fix like strict mode. It's more complex than I thought. The code I presented would likely return a valid answer if A and B are mixed, so setting the system prompt seems like the right approach.
yindo changed title from [FEAT]: Chat `query` strict mode to [GH-ISSUE #4422] [FEAT]: Chat `query` strict mode 2026-06-05 14:48:44 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2821
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?