Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #911] [BUG]: Chat settings prompt [agent prompt] is being ignored #546

New Issue
Closed
opened 2026-02-22 18:20:06 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:20:06 -05:00
Owner
Copy Link
Copy Source

Originally created by @chrisribe on GitHub (Mar 15, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/911

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

Hello,

Chat settings prompt is being ignored after "Update Workspace" button is clicked.
It is only correcly applied "after" /reset "clear your chat and begin a new chat" is clicked or a new thread is created.

There is no way for the user to know what prompt is being feed along with the input text.
Using local LLM Mistral-ins-7b-Q4

Are there known steps to reproduce?

Steps

  1. Create a new Workspace

  2. Go to default thread and ask:
    "what is the weather ?"

  3. Go to chat settings
    Add this prompt to the chat settings

You are an AI assistant who ALWAYS starts the response with [AI START] and ends with [AI END]

  1. Hit Update Workspace

  2. Go to default thread and ask:
    "what is the weather ?"

Result

Response will not contain [AI START] [AI END] until you reset the chat or create a new thread.

Originally created by @chrisribe on GitHub (Mar 15, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/911 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? Hello, Chat settings prompt is being ignored after "Update Workspace" button is clicked. It is only correcly applied "after" /reset "clear your chat and begin a new chat" is clicked or a new thread is created. There is no way for the user to know what prompt is being feed along with the input text. Using local LLM Mistral-ins-7b-Q4 ### Are there known steps to reproduce? # Steps 1. Create a new Workspace 1. Go to default thread and ask: "what is the weather ?" 1. Go to chat settings Add this prompt to the chat settings ``` You are an AI assistant who ALWAYS starts the response with [AI START] and ends with [AI END] ``` 1. Hit Update Workspace 1. Go to default thread and ask: "what is the weather ?" # Result Response will not contain [AI START] [AI END] until you reset the chat or create a new thread.
yindo added the possible bug label 2026-02-22 18:20:06 -05:00
yindo closed this issue 2026-02-22 18:20:06 -05:00
yindo commented 2026-02-22 18:20:06 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Mar 15, 2024):

I just tried to replicate this but this is actually an issue with the model unfortunately. This typically happens to Mistral models for some reason but we would suggest you not use any models that have a quantization of less that 5. In the model you are using it has a quantization of 4 noted by the Q4 here: Mistral-ins-7b-Q4.

This is happening because since the quantization is so low it ignores the system prompt and we have noticed it will actually ignore the system prompt baked into the model sometimes as well. Closing due to this being a model issue.

@shatfield4 commented on GitHub (Mar 15, 2024): I just tried to replicate this but this is actually an issue with the model unfortunately. This typically happens to Mistral models for some reason but we would suggest you not use any models that have a quantization of less that 5. In the model you are using it has a quantization of 4 noted by the Q4 here: Mistral-ins-7b-**Q4**. This is happening because since the quantization is so low it ignores the system prompt and we have noticed it will actually ignore the system prompt baked into the model sometimes as well. Closing due to this being a model issue.
yindo commented 2026-02-22 18:20:06 -05:00
Author
Owner
Copy Link
Copy Source

@chrisribe commented on GitHub (Mar 17, 2024):

Thanks for explaining the details as to why this was occuring.
I learned something today :)

@chrisribe commented on GitHub (Mar 17, 2024): Thanks for explaining the details as to why this was occuring. I learned something today :)
yindo commented 2026-02-22 18:20:06 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 18, 2024):

@chrisribe Yeah, LLMs can be tricky - especially with quantized models since they are basically operating with less precision. The model was basically "over-indexing" on the format of the first messages and ignoring the instruction even in the system prompt and your next prompt. You dont see this in more "powerful" models because they can "reason" around it and understand what output you really are likely looking for. Since there was no conflict context when the history was empty, there was nothing to fight against the system prompt or your current query - so it worked.

Just a quirk of LLMs, especially running locally. We are all learning these things together 🚀

@timothycarambat commented on GitHub (Mar 18, 2024): @chrisribe Yeah, LLMs can be tricky - especially with quantized models since they are basically operating with less precision. The model was basically "over-indexing" on the format of the first messages and ignoring the instruction even in the system prompt and your next prompt. You dont see this in more "powerful" models because they can "reason" around it and understand what output you really are likely looking for. Since there was no conflict context when the history was empty, there was nothing to fight against the system prompt or your current query - so it worked. Just a quirk of LLMs, especially running locally. We are all learning these things together 🚀
yindo changed title from [BUG]: Chat settings prompt [agent prompt] is being ignored to [GH-ISSUE #911] [BUG]: Chat settings prompt [agent prompt] is being ignored 2026-06-05 14:35:54 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#546
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?