Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3661] [BUG]: Roles and synonyms not working #2360

New Issue
Closed
opened 2026-02-22 18:29:21 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:29:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @johnec2 on GitHub (Apr 16, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3661

How are you running AnythingLLM?

Local development

What happened?

We have uploaded a JSON file where roles are specified in the metadata section. However, during testing, the role-based prompts are not functioning as expected.
{
"id": "01",
"question": [ "How to access schedules on the public website?", "How to view schedules on the public website?" ],
"answer": "Schedules can be accessed on the public website by following the below steps",
"metadata": {
"category": "Website Schedules", "role": ["all"], "module":"Scheduling"
},
"synonyms": {
"schedules": ["timetable", "agenda", "fixtures"],
"public website": ["site", "webpage", "online portal", "website"],
"view": ["see", "check", "access", "find"]
}
},
{
"id": "02",
"questions": [ "How can players sign up or register to volunteer for a game?", "How do players register to volunteer for a game?" ],
"answer": "Players can sign up as volunteers for a game through their team portal. Here are the steps to register:...",
"metadata": {
"category": "Player Register to Volunteer", "role": ["admin"], "module":"Scheduling"
},
"synonyms": {
"volunteer": ["helper", "assistant", "contributor"],
"register": ["enroll", "sign up", "join"],
"game": ["match", "fixture", "contest"],
"players": ["athletes", "participants", "roster members", "team members"]
}
}
Issues:

  1. Role-based Filtering Not Working:
    For example, the role is set to "admin" in the second dataset. However, the system still returns the answer when we ask a related question with a "team staff" role. This is incorrect — since the roles do not match, the system should return a fallback message instead of the actual answer.

Also, the first dataset has the role set to "all, " meaning it should be accessible to any role without restriction. How can we ensure that this behavior is correctly implemented?

  1. Synonyms Support:
    We have also included a synonyms field in the dataset. For example:

We want the system to recognize and match these synonyms during query processing.
Is this the correct format for defining synonyms? Also, how can we ensure that the synonyms work during response matching?

Can you help us configure this correctly?

Are there known steps to reproduce?

No response

Originally created by @johnec2 on GitHub (Apr 16, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3661 ### How are you running AnythingLLM? Local development ### What happened? We have uploaded a JSON file where roles are specified in the metadata section. However, during testing, the role-based prompts are not functioning as expected. { "id": "01", "question": [ "How to access schedules on the public website?", "How to view schedules on the public website?" ], "answer": "Schedules can be accessed on the public website by following the below steps", "metadata": { "category": "Website Schedules", "role": ["all"], "module":"Scheduling" }, "synonyms": { "schedules": ["timetable", "agenda", "fixtures"], "public website": ["site", "webpage", "online portal", "website"], "view": ["see", "check", "access", "find"] } }, { "id": "02", "questions": [ "How can players sign up or register to volunteer for a game?", "How do players register to volunteer for a game?" ], "answer": "Players can sign up as volunteers for a game through their team portal. Here are the steps to register:...", "metadata": { "category": "Player Register to Volunteer", "role": ["admin"], "module":"Scheduling" }, "synonyms": { "volunteer": ["helper", "assistant", "contributor"], "register": ["enroll", "sign up", "join"], "game": ["match", "fixture", "contest"], "players": ["athletes", "participants", "roster members", "team members"] } } Issues: 1. **Role-based Filtering Not Working:** For example, the role is set to "admin" in the second dataset. However, the system still returns the answer when we ask a related question with a "team staff" role. This is incorrect — since the roles do not match, the system should return a fallback message instead of the actual answer. Also, the first dataset has the role set to "all, " meaning it should be accessible to any role without restriction. How can we ensure that this behavior is correctly implemented? 2. **Synonyms Support:** We have also included a synonyms field in the dataset. For example: We want the system to recognize and match these synonyms during query processing. Is this the correct format for defining synonyms? Also, how can we ensure that the synonyms work during response matching? Can you help us configure this correctly? ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:29:21 -05:00
yindo closed this issue 2026-02-22 18:29:21 -05:00
yindo commented 2026-02-22 18:29:21 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 16, 2025):

This is not how roles work in AnythingLLM. What you are getting is a model hallucination since you are trying to apply roles to your RAG documents and responses. Basically the model is just not following the prompt - very typical dependingon the model, how you are running it, and other configs.

There is nothing for us to do here, you need a stronger model (likely bigger) and you'll find its responses "listen" more to your instructions.

@timothycarambat commented on GitHub (Apr 16, 2025): This is not how roles work in AnythingLLM. What you are getting is a model hallucination since you are trying to apply roles to your RAG documents and responses. Basically the model is just not following the prompt - very typical dependingon the model, how you are running it, and other configs. There is nothing for us to do here, you need a stronger model (likely bigger) and you'll find its responses "listen" more to your instructions.
yindo changed title from [BUG]: Roles and synonyms not working to [GH-ISSUE #3661] [BUG]: Roles and synonyms not working 2026-06-05 14:46:07 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2360
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?