Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4681] [FEAT]: Global System Prompt needs to be truly global and layered (current implementation causes confusion and does not solve multi-agent or enterprise use cases). #2965

New Issue
Closed
opened 2026-02-22 18:32:04 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:32:04 -05:00
Owner
Copy Link
Copy Source

Originally created by @HanJammer on GitHub (Nov 24, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4681

What would you like to see?

I’m reopening the core problem from #3906 , because the recently closed PR #4487 does not address the requested functionality.
It introduces a “Default System Prompt for new workspaces”, which is essentially a template initializer, not a global prompt layer.

In its current form, the new feature actually increases confusion and adds no real value for multi-agent or complex deployments.

What was expected (from #3906 )

A true global system prompt, analogous to how ChatGPT / Claude / enterprise LLM deployments structure their instruction pipeline:

1. Global System Prompt (hidden, always applied)

  • Applied to every request
  • Prepending everything in the pipeline
  • Not replaced when a workspace defines its own prompt
  • Setting foundational identity, tone, safety, and core behavioral constraints
  • This is how a multi-tenant or multi-agent architecture maintains coherence.

2. Workspace Prompt (layered on top of global)

  • Refining the behavior for a specific workspace/project without overriding global instructions.

3. User Personalization Prompt

  • Optional per-user fine-tuning (bio, preferences, language).
  • AnythingLLM already partially allows this via {user.bio} prompt variable, which is good and actually kind of clever.

4. Per-Tool / Per-Skill Prompt

Every tool/skill should have:

  • a local prompt describing what the tool is (even if MCP provides this functionality),
  • usage constraints (ability to turn on/off MCP server tools one by one - just like in LM Studio!),
  • relations between tools,
  • and workspace-specific overrides.

Right now tools are global, which makes them borderline unusable in serious multi-agent setups.

What was delivered in #4487

A single setting:

“Default System Prompt is copied into new workspaces when they are created.”

This does not provide:

  • global instruction coherence,
  • layered prompting,
  • runtime concatenation,
  • multi-agent isolation,
  • backward propagation to existing workspaces,
  • or any alignment with how modern LLM systems handle multi-level prompting.

It simply clones a text field into the workspace’s openAiPrompt.
After that, the “global prompt” disappears entirely and cannot influence anything.

This does not solve the problem described in #3906 .

Why this matters in real deployments

In serious, multi-agent systems (AnythingLLM + n8n + MCP + toolchains) we need:

[GLOBAL SYSTEM PROMPT]
    + [WORKSPACE PROMPT]
    + [USER PERSONALIZATION]
    + [TOOL PROMPTS]
    + [CONTEXT]
    + USER MESSAGE

Not:

[WORKSPACE PROMPT only]

The current system forces admins to generate large monolithic prompts for each workspace manually (in my case via Python + Jinja2 + YAML modules). This works, but it defeats the purpose of a framework intended for composable agents and forces to use API to update prompts which doesn't seem to leverage built-in prompt versioning unfortunately (I had to code my own prompt versioning system which is fine as well.

What needs to be implemented

I propose the following actual global prompt architecture:

1. Global System Prompt (hidden, always applied)

  • Injected automatically into every conversation, before any workspace prompt.
  • Not replaceable.
  • Hidden from users.

This is the "You are a helpful assistant disguised as a cat and you reply only with meowing..." part (similar to the hidden kernel prompts used by ChatGPT).

2. Workspace Prompt (visible)

  • Concatenated after global.

This is the workspace prompt - just like we have already or similar to the "Project instructions" in the ChatGPT.

3. User Personalization

  • Concatenated after workspace.

This is the fine-tuning of the agent style - precautions should be taken to avoid the prompt injection leading to the override of the guardrails/global system prompt/workspace prompt/tool prompts (!)

4. Tool / Skill Prompts (per-tool, per-workspace)

  • Concatenated per tool.

5. Final runtime prompt = concatenation of all layers

This mirrors state-of-the-art LLM instruction handling and enables AnythingLLM to operate in serious multi-agent deployments.

Summary

PR #4487 is a useful addition for template initialization, but it does not fulfill the purpose of #3906 .
A true global prompt layer, applied at runtime and concatenated with workspace/user/tool instructions, is required for AnythingLLM to scale beyond simple single-workspace setups.

I can provide architectural diagrams and a reference implementation if needed.

Originally created by @HanJammer on GitHub (Nov 24, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4681 ### What would you like to see? I’m reopening the core problem from #3906 , because the recently closed PR #4487 does not address the requested functionality. It introduces a “Default System Prompt for new workspaces”, which is essentially a template initializer, not a global prompt layer. In its current form, the new feature actually increases confusion and adds no real value for multi-agent or complex deployments. ### What was expected (from #3906 ) A true global system prompt, analogous to how ChatGPT / Claude / enterprise LLM deployments structure their instruction pipeline: **1. Global System Prompt (hidden, always applied)** - Applied to every request - Prepending everything in the pipeline - Not replaced when a workspace defines its own prompt - Setting foundational identity, tone, safety, and core behavioral constraints - This is how a multi-tenant or multi-agent architecture maintains coherence. **2. Workspace Prompt (layered on top of global)** - Refining the behavior for a specific workspace/project without overriding global instructions. **3. User Personalization Prompt** - Optional per-user fine-tuning (bio, preferences, language). - AnythingLLM already partially allows this via {user.bio} prompt variable, which is good and actually kind of clever. **4. Per-Tool / Per-Skill Prompt** Every tool/skill should have: - a local prompt describing what the tool is (even if MCP provides this functionality), - usage constraints (ability to turn on/off MCP server tools one by one - just like in LM Studio!), - relations between tools, - and workspace-specific overrides. Right now **tools are global, which makes them borderline unusable** in serious multi-agent setups. --- ### What was delivered in #4487 A single setting: “Default System Prompt is copied into new workspaces when they are created.” This does not provide: - global instruction coherence, - layered prompting, - runtime concatenation, - multi-agent isolation, - backward propagation to existing workspaces, - or any alignment with how modern LLM systems handle multi-level prompting. It simply clones a text field into the workspace’s openAiPrompt. After that, the “global prompt” disappears entirely and cannot influence anything. This does not solve the problem described in #3906 . --- ### Why this matters in real deployments In serious, multi-agent systems (AnythingLLM + n8n + MCP + toolchains) we need: ``` [GLOBAL SYSTEM PROMPT] + [WORKSPACE PROMPT] + [USER PERSONALIZATION] + [TOOL PROMPTS] + [CONTEXT] + USER MESSAGE ``` Not: `[WORKSPACE PROMPT only]` The current system forces admins to generate large monolithic prompts for each workspace manually (in my case via Python + Jinja2 + YAML modules). This works, but it defeats the purpose of a framework intended for composable agents and forces to use API to update prompts which doesn't seem to leverage built-in prompt versioning unfortunately (I had to code my own prompt versioning system which is fine as well. --- ### What needs to be implemented I propose the following actual global prompt architecture: **1. Global System Prompt (hidden, always applied)** - Injected automatically into every conversation, before any workspace prompt. - Not replaceable. - Hidden from users. This is the "You are a helpful assistant disguised as a cat and you reply only with meowing..." part (similar to the hidden kernel prompts used by ChatGPT). **2. Workspace Prompt (visible)** - Concatenated after global. This is the workspace prompt - just like we have already or similar to the "Project instructions" in the ChatGPT. **3. User Personalization** - Concatenated after workspace. This is the fine-tuning of the agent style - precautions should be taken to avoid the prompt injection leading to the override of the guardrails/global system prompt/workspace prompt/tool prompts (!) **4. Tool / Skill Prompts (per-tool, per-workspace)** - Concatenated per tool. **5. Final runtime prompt = concatenation of all layers** This mirrors state-of-the-art LLM instruction handling and enables AnythingLLM to operate in serious multi-agent deployments. Summary PR #4487 is a useful addition for template initialization, but it does not fulfill the purpose of #3906 . A true global prompt layer, applied at runtime and concatenated with workspace/user/tool instructions, is required for AnythingLLM to scale beyond simple single-workspace setups. I can provide architectural diagrams and a reference implementation if needed.
yindo added the enhancementfeature request labels 2026-02-22 18:32:04 -05:00
yindo closed this issue 2026-02-22 18:32:04 -05:00
yindo commented 2026-02-22 18:32:04 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Nov 24, 2025):

We added the feature as it stands because it is an annoyance to retype a prompt for every new workspace when you have a common prompt. That is what it should have solved and does.

That being said

[GLOBAL SYSTEM PROMPT]
    + [WORKSPACE PROMPT]
    + [USER PERSONALIZATION]
    + [TOOL PROMPTS]
    + [CONTEXT]
    + USER MESSAGE

is, quite literally, what is occurring now works fully within the current structure of how chats are managed now.

You can set a custom system variable called globalPrompt and inject that into every workspace's system prompt + use more system vars for user personalization (to be improved more by #4677). Every time we run a tool-call chat, we also inject tool definitions and examples as well. Lastly, we also send the current conversation history and user prompt + RAG/attachments.

This is what you describe - it just is done in a process that is continuously appended as opposed to a singular massive prompt. As for the "hidden" aspect of this prompt, I do not see the benefit of hiding this information from those who can edit the system prompt for a workspace, as it is highly likely to confuse people more when information is injected and is not easily viewable or auditable by the user.

@timothycarambat commented on GitHub (Nov 24, 2025): We added the feature as it stands because it is an annoyance to retype a prompt for every new workspace when you have a common prompt. That is what it should have solved and does. That being said ``` [GLOBAL SYSTEM PROMPT] + [WORKSPACE PROMPT] + [USER PERSONALIZATION] + [TOOL PROMPTS] + [CONTEXT] + USER MESSAGE ``` is, quite literally, what is occurring now works fully within the current structure of how chats are managed now. You can set a custom system variable called `globalPrompt` and inject that into every workspace's system prompt + use more system vars for user personalization (to be improved more by #4677). Every time we run a tool-call chat, we also inject tool definitions and `examples` as well. Lastly, we also send the current conversation history and user prompt + RAG/attachments. This is what you describe - it just is done in a process that is continuously appended as opposed to a singular massive prompt. As for the "hidden" aspect of this prompt, I do not see the benefit of hiding this information from those who can edit the system prompt for a workspace, as it is highly likely to confuse people more when information is injected and is not easily viewable or auditable by the user.
yindo commented 2026-02-22 18:32:04 -05:00
Author
Owner
Copy Link
Copy Source

@HanJammer commented on GitHub (Nov 27, 2025):

Thanks for the explanation.
To avoid prolonging the discussion, I'll just leave one clarification that may help others reading the thread:
What you describe (variable expansion + runtime concatenation of tools/history/context) is not equivalent to a layered prompt architecture as implemented in ChatGPT, Claude, or enterprise LLM stacks.

Your approach works functionally, but:

  • it is procedural prompt assembly, not structural layering,
  • global variables are string injections, not a guaranteed system-level instruction layer,
  • tools remain globally scoped rather than workspace-scoped,
  • and workspace prompts cannot inherit updates to global logic.

This is perfectly fine for lightweight setups, but not optimal for multi-agent or high-isolation deployments. I understand it’s not planned, that’s fair enough. I'll possibly consider handling proper layering in my own fork.

No further input needed.

@HanJammer commented on GitHub (Nov 27, 2025): Thanks for the explanation. To avoid prolonging the discussion, I'll just leave one clarification that may help others reading the thread: What you describe (variable expansion + runtime concatenation of tools/history/context) is not equivalent to a layered prompt architecture as implemented in ChatGPT, Claude, or enterprise LLM stacks. Your approach works functionally, but: - it is procedural prompt assembly, not structural layering, - global variables are string injections, not a guaranteed system-level instruction layer, - tools remain globally scoped rather than workspace-scoped, - and workspace prompts cannot inherit updates to global logic. This is perfectly fine for lightweight setups, but not optimal for multi-agent or high-isolation deployments. I understand it’s not planned, that’s fair enough. I'll possibly consider handling proper layering in my own fork. No further input needed.
yindo changed title from [FEAT]: Global System Prompt needs to be truly global and layered (current implementation causes confusion and does not solve multi-agent or enterprise use cases). to [GH-ISSUE #4681] [FEAT]: Global System Prompt needs to be truly global and layered (current implementation causes confusion and does not solve multi-agent or enterprise use cases). 2026-06-05 14:49:34 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2965
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?