Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2837] [BUG]: Mistral Role template #1818

New Issue
Closed
opened 2026-02-22 18:26:39 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:26:39 -05:00
Owner
Copy Link
Copy Source

Originally created by @marsidorowicz on GitHub (Dec 15, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2837

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

Only user and assistant roles are supported!. Error Data: n/a, Additional Data: n/a after connecting to LM studio mistral model, model was automatically found on automatic search on localhost

Are there known steps to reproduce?

just regular instalation following https://www.youtube.com/watch?v=-Rs8-M-xBFI&t=29s

Originally created by @marsidorowicz on GitHub (Dec 15, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2837 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? Only user and assistant roles are supported!. Error Data: n/a, Additional Data: n/a after connecting to LM studio mistral model, model was automatically found on automatic search on localhost ### Are there known steps to reproduce? just regular instalation following https://www.youtube.com/watch?v=-Rs8-M-xBFI&t=29s
yindo added the possible bug label 2026-02-22 18:26:39 -05:00
yindo closed this issue 2026-02-22 18:26:39 -05:00
yindo commented 2026-02-22 18:26:39 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Dec 17, 2024):

I was able to replicate this with the mistral-7b-instruct-v0.2 model. It seems that LMStudio is not pairing an inference template with this model so you will need to select one in the settings. After selecting a preset, everything works as expected.

Screenshot 2024-12-17 at 3 55 29 PM
@shatfield4 commented on GitHub (Dec 17, 2024): I was able to replicate this with the `mistral-7b-instruct-v0.2` model. It seems that LMStudio is not pairing an inference template with this model so you will need to select one in the settings. After selecting a preset, everything works as expected. <img width="1395" alt="Screenshot 2024-12-17 at 3 55 29 PM" src="https://github.com/user-attachments/assets/c715e772-215e-4793-a342-c2855a384380" />
yindo commented 2026-02-22 18:26:40 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 17, 2024):

Adding on, you probably specifically want the Mistral template, obviously - using the llama template might work but its not designed for Mistral. We cannot set templates via the API, so closing

@timothycarambat commented on GitHub (Dec 17, 2024): Adding on, you probably specifically want the `Mistral` template, obviously - using the llama template _might_ work but its not designed for Mistral. We cannot set templates via the API, so closing
yindo commented 2026-02-22 18:26:40 -05:00
Author
Owner
Copy Link
Copy Source

@JSM684 commented on GitHub (Jan 1, 2025):

Adding on, you probably specifically want the Mistral template, obviously - using the llama template might work but its not designed for Mistral. We cannot set templates via the API, so closing

I am encountering the same error as #2914 (comment)

I am sure I am just being daft, but how does one obtain the Mistral template?

@JSM684 commented on GitHub (Jan 1, 2025): > Adding on, you probably specifically want the `Mistral` template, obviously - using the llama template _might_ work but its not designed for Mistral. We cannot set templates via the API, so closing I am encountering the same error as [#2914 (comment)](https://github.com/Mintplex-Labs/anything-llm/issues/2914) I am sure I am just being daft, but how does one obtain the `Mistral` template?
yindo commented 2026-02-22 18:26:40 -05:00
Author
Owner
Copy Link
Copy Source

@Rysreg7 commented on GitHub (Jul 21, 2025):

{%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages[0].role == 'system' %}
{{- messages[0].content + '\n\n' }}
{%- endif %}
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{"name": , "arguments": }\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages[0].role == 'system' %}
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for forward_message in messages %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- set message = messages[index] %}
{%- set current_content = message.content if message.content is defined and message.content is not none else '' %}
{%- set tool_start = '<tool_response>' %}
{%- set tool_start_length = tool_start|length %}
{%- set start_of_message = current_content[:tool_start_length] %}
{%- set tool_end = '</tool_response>' %}
{%- set tool_end_length = tool_end|length %}
{%- set start_pos = (current_content|length) - tool_end_length %}
{%- if start_pos < 0 %}
{%- set start_pos = 0 %}
{%- endif %}
{%- set end_of_message = current_content[start_pos:] %}
{%- if ns.multi_step_tool and message.role == "user" and not(start_of_message == tool_start and end_of_message == tool_end) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endfor %}
{%- for message in messages %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set m_content = message.content if message.content is defined and message.content is not none else '' %}
{%- set content = m_content %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '' in m_content %}
{%- set content = (m_content.split('')|last).lstrip('\n') %}
{%- set reasoning_content = (m_content.split('')|first).rstrip('\n') %}
{%- set reasoning_content = (reasoning_content.split('')|last).lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- if loop.index0 > ns.last_query_index %}
{%- if loop.last or (not loop.last and (not reasoning_content.strip() == '')) %}
{{- '<|im_start|>' + message.role + '\n\n' + reasoning_content.strip('\n') + '\n\n\n' + content.lstrip('\n') }}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- else %}
{{- '<|im_start|>' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}
{%- if (loop.first and content) or (not loop.first) %}
{{- '\n' }}
{%- endif %}
{%- if tool_call.function %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{%- if tool_call.arguments is string %}
{{- tool_call.arguments }}
{%- else %}
{{- tool_call.arguments | tojson }}
{%- endif %}
{{- '}\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- message.content }}
{{- '\n</tool_response>' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '\n\n\n\n' }}
{%- endif %}
{%- endif %}

@Rysreg7 commented on GitHub (Jul 21, 2025): {%- if tools %} {{- '<|im_start|>system\n' }} {%- if messages[0].role == 'system' %} {{- messages[0].content + '\n\n' }} {%- endif %} {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }} {%- for tool in tools %} {{- "\n" }} {{- tool | tojson }} {%- endfor %} {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }} {%- else %} {%- if messages[0].role == 'system' %} {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }} {%- endif %} {%- endif %} {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %} {%- for forward_message in messages %} {%- set index = (messages|length - 1) - loop.index0 %} {%- set message = messages[index] %} {%- set current_content = message.content if message.content is defined and message.content is not none else '' %} {%- set tool_start = '<tool_response>' %} {%- set tool_start_length = tool_start|length %} {%- set start_of_message = current_content[:tool_start_length] %} {%- set tool_end = '</tool_response>' %} {%- set tool_end_length = tool_end|length %} {%- set start_pos = (current_content|length) - tool_end_length %} {%- if start_pos < 0 %} {%- set start_pos = 0 %} {%- endif %} {%- set end_of_message = current_content[start_pos:] %} {%- if ns.multi_step_tool and message.role == "user" and not(start_of_message == tool_start and end_of_message == tool_end) %} {%- set ns.multi_step_tool = false %} {%- set ns.last_query_index = index %} {%- endif %} {%- endfor %} {%- for message in messages %} {%- if (message.role == "user") or (message.role == "system" and not loop.first) %} {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }} {%- elif message.role == "assistant" %} {%- set m_content = message.content if message.content is defined and message.content is not none else '' %} {%- set content = m_content %} {%- set reasoning_content = '' %} {%- if message.reasoning_content is defined and message.reasoning_content is not none %} {%- set reasoning_content = message.reasoning_content %} {%- else %} {%- if '</think>' in m_content %} {%- set content = (m_content.split('</think>')|last).lstrip('\n') %} {%- set reasoning_content = (m_content.split('</think>')|first).rstrip('\n') %} {%- set reasoning_content = (reasoning_content.split('<think>')|last).lstrip('\n') %} {%- endif %} {%- endif %} {%- if loop.index0 > ns.last_query_index %} {%- if loop.last or (not loop.last and (not reasoning_content.strip() == '')) %} {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }} {%- else %} {{- '<|im_start|>' + message.role + '\n' + content }} {%- endif %} {%- else %} {{- '<|im_start|>' + message.role + '\n' + content }} {%- endif %} {%- if message.tool_calls %} {%- for tool_call in message.tool_calls %} {%- if (loop.first and content) or (not loop.first) %} {{- '\n' }} {%- endif %} {%- if tool_call.function %} {%- set tool_call = tool_call.function %} {%- endif %} {{- '<tool_call>\n{"name": "' }} {{- tool_call.name }} {{- '", "arguments": ' }} {%- if tool_call.arguments is string %} {{- tool_call.arguments }} {%- else %} {{- tool_call.arguments | tojson }} {%- endif %} {{- '}\n</tool_call>' }} {%- endfor %} {%- endif %} {{- '<|im_end|>\n' }} {%- elif message.role == "tool" %} {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %} {{- '<|im_start|>user' }} {%- endif %} {{- '\n<tool_response>\n' }} {{- message.content }} {{- '\n</tool_response>' }} {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %} {{- '<|im_end|>\n' }} {%- endif %} {%- endif %} {%- endfor %} {%- if add_generation_prompt %} {{- '<|im_start|>assistant\n' }} {%- if enable_thinking is defined and enable_thinking is false %} {{- '<think>\n\n</think>\n\n' }} {%- endif %} {%- endif %}
yindo changed title from [BUG]: Mistral Role template to [GH-ISSUE #2837] [BUG]: Mistral Role template 2026-06-05 14:42:50 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1818
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?