Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4602] DummyMCP – A Safe, Minimal MCP Test Server for AnythingLLM in docker #2926

New Issue
Closed
opened 2026-02-22 18:31:52 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:31:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @danny0094 on GitHub (Oct 31, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4602

Update / Fix proposal: Working Docker-compatible MCP Bridge + DummyMCP

Hey team

I was made aware a few days ago that Anythingllm cannot influence other Docker instances via Docker.
This means that MCP support via Docker is currently not possible.
So I looked for a solution to fix it.
Primarily for myself. But I also wanted to share it with you.

I’ve been experimenting with a safe way to get AnythingLLM to talk to MCP servers from inside Docker, without triggering the spawn docker ENOENT limitation.

So I thought, well, if Anythingllm isn't allowed to make outgoing calls, it needs a new solution. A bridge. So I designed a mini HTTP bridge and a dummy MCP server for testing.

The bridge runs in the same container as Anythingllm, and the "receiver," which acts as a dummy MCP server, also runs in its own container.
Completely isolated and secure.
Requests and responses are sent.

Key points:

  • Works inside Docker (no Docker-in-Docker needed)
  • JSON-RPC 2.0 compliant (tested against AnythingLLM’s MCP SDK)
  • Fully isolated and safe – the MCP containers can’t affect the host
  • Shows correct handshake and tools/list, ping, etc.
  • Verified with mcp-http-bridge pattern

overview:

anythingllm (container)
│
├─ mini-bridge (container, port 4100)
│ ↳ forwards MCP JSON-RPC requests
│
└─ dummy-mcp (container, port 4200)
↳ responds: “MCP is alive ”

### Example compose setup
(abridged)
```yaml
services:
  anythingllm:
    image: mintplexlabs/anythingllm
    networks: [ danny_ai-net ]

  mini-bridge:
    build: ./mini_bridge
    networks: [ danny_ai-net ]
    depends_on: [ dummy-mcp ]

  dummy-mcp:
    build: ./dummy_MCP
    networks: [ danny_ai-net ]

mini_bridge.py

dummy_mcp.py

If there's interest, I'll continue to update this; otherwise, I'll just keep doing it for myself.
I have no idea when or how the Docker update will be released by the development team.

Image Image Image

repository
https://github.com/danny0094/mcp-bridge-stack

Originally created by @danny0094 on GitHub (Oct 31, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4602 **Update / Fix proposal: Working Docker-compatible MCP Bridge + DummyMCP** Hey team I was made aware a few days ago that Anythingllm cannot influence other Docker instances via Docker. This means that MCP support via Docker is currently not possible. So I looked for a solution to fix it. Primarily for myself. But I also wanted to share it with you. I’ve been experimenting with a safe way to get AnythingLLM to talk to MCP servers from *inside Docker*, without triggering the `spawn docker ENOENT` limitation. So I thought, well, if Anythingllm isn't allowed to make outgoing calls, it needs a new solution. A bridge. So I designed a mini HTTP bridge and a dummy MCP server for testing. The bridge runs in the same container as Anythingllm, and the "receiver," which acts as a dummy MCP server, also runs in its own container. Completely isolated and secure. Requests and responses are sent. Key points: - Works inside Docker (no Docker-in-Docker needed) - JSON-RPC 2.0 compliant (tested against AnythingLLM’s MCP SDK) - Fully isolated and safe – the MCP containers can’t affect the host - Shows correct handshake and `tools/list`, `ping`, etc. - Verified with `mcp-http-bridge` pattern overview: anythingllm (container) │ ├─ mini-bridge (container, port 4100) │ ↳ forwards MCP JSON-RPC requests │ └─ dummy-mcp (container, port 4200) ↳ responds: “MCP is alive ” --- ``` ### Example compose setup (abridged) ```yaml services: anythingllm: image: mintplexlabs/anythingllm networks: [ danny_ai-net ] mini-bridge: build: ./mini_bridge networks: [ danny_ai-net ] depends_on: [ dummy-mcp ] dummy-mcp: build: ./dummy_MCP networks: [ danny_ai-net ] ``` [mini_bridge.py](https://github.com/user-attachments/files/23279597/mini_bridge.py) [dummy_mcp.py](https://github.com/user-attachments/files/23279598/dummy_mcp.py) If there's interest, I'll continue to update this; otherwise, I'll just keep doing it for myself. I have no idea when or how the Docker update will be released by the development team. <img width="1459" height="854" alt="Image" src="https://github.com/user-attachments/assets/4572de6d-1d8e-4036-8f04-49e67f0909a0" /> <img width="847" height="406" alt="Image" src="https://github.com/user-attachments/assets/b0fbedc7-bea0-4c53-9dd9-85fbf6a7135a" /> <img width="862" height="35" alt="Image" src="https://github.com/user-attachments/assets/75ff986c-72ea-4744-9097-8c353fe456a2" /> repository https://github.com/danny0094/mcp-bridge-stack
yindo closed this issue 2026-02-22 18:31:52 -05:00
yindo commented 2026-02-22 18:31:53 -05:00
Author
Owner
Copy Link
Copy Source

@sonusainidevops commented on GitHub (Nov 1, 2025):

Nice idea to continue for development

@sonusainidevops commented on GitHub (Nov 1, 2025): Nice idea to continue for development
yindo commented 2026-02-22 18:31:53 -05:00
Author
Owner
Copy Link
Copy Source

@danny0094 commented on GitHub (Nov 1, 2025):

I am already testing the function with a convenient MCP registration. mcp_registry.json
The reason is that MCP Docker can be implemented as easily as possible.

like Claude : mcp_registry.json

{
  "autoReload": true,
  "servers": [
    {
      "id": "dummy",
      "name": "Dummy MCP",
      "url": "http://dummy-mcp:4200",
      "type": "streamable",
      "enabled": true
    },
    {
      "id": "docs",
      "name": "Docs MCP",
      "url": "http://dummy-mcp:4200",
      "type": "streamable",
      "enabled": true
    }
  ]
}

The goal is to make it as easy as possible for everyone to implement an MCP server in AnythingLMM.

The folder structure in the test environment currently looks like this:

├── docker-compose.yml
├── mini_bridge/
│ ├── mini_bridge.py
│ ├── Dockerfile
│ └── config/
│ └── mcp_registry.json
└── dummy_MCP/
└── Dockerfile

The goal of the project is to serve as a clean template for the development and connection of custom MCP modules (e.g., Docs, Time, Weather, etc.) with AnythingLLM.

@danny0094 commented on GitHub (Nov 1, 2025): I am already testing the function with a convenient MCP registration. `mcp_registry.json ` The reason is that MCP Docker can be implemented as easily as possible. like Claude : mcp_registry.json ``` { "autoReload": true, "servers": [ { "id": "dummy", "name": "Dummy MCP", "url": "http://dummy-mcp:4200", "type": "streamable", "enabled": true }, { "id": "docs", "name": "Docs MCP", "url": "http://dummy-mcp:4200", "type": "streamable", "enabled": true } ] } ``` The goal is to make it as easy as possible for everyone to implement an MCP server in AnythingLMM. The folder structure in the test environment currently looks like this: ├── docker-compose.yml ├── mini_bridge/ │ ├── mini_bridge.py │ ├── Dockerfile │ └── config/ │ └── mcp_registry.json └── dummy_MCP/ └── Dockerfile The goal of the project is to serve as a clean template for the development and connection of custom MCP modules (e.g., Docs, Time, Weather, etc.) with AnythingLLM.
yindo commented 2026-02-22 18:31:53 -05:00
Author
Owner
Copy Link
Copy Source

@danny0094 commented on GitHub (Nov 1, 2025):

Okay, at the moment it looks very good.

Image

The bridge is running inside the container and is successfully sending and receiving from the container.
curl -X POST http://localhost:4100/time \ also confirms everything correctly:

Image Image

I would like to point this out again:

Each MCP server runs in its own isolated Docker container, completely separated from AnythingLLM and from each other.
The Mini-Bridge acts as a single, controlled communication gateway — it forwards only validated JSON-RPC requests and returns responses without exposing internal Docker or system access.
This design provides:

  • Container isolation – every MCP has its own runtime, dependencies, and network namespace.
  • No Docker-in-Docker or host access – AnythingLLM never touches the MCP containers directly.
  • Controlled traffic flow – all communication passes through the Bridge on a private Docker network.
  • JSON-RPC only – no shell commands, filesystem access, or unsafe exec calls.
  • Dynamic registration – new MCPs can be added safely via mcp_registry.json without modifying the codebase.
Image

I’ll keep improving this project as my own independent solution.
If the AnythingLLM team finds the idea useful, feel free to build on it or integrate parts of it.
I plan to publish the full setup on GitHub in the coming days.
As you can see, the tool output is still returned a bit unnaturally instead of a fully formatted chat response — once that’s refined, the public release will follow.

@danny0094 commented on GitHub (Nov 1, 2025): Okay, at the moment it looks very good. <img width="847" height="406" alt="Image" src="https://github.com/user-attachments/assets/be7e9c1f-9077-4ca1-8b4f-aaa5e32e532d" /> The bridge is running inside the container and is successfully sending and receiving from the container. curl -X POST http://localhost:4100/time \ also confirms everything correctly: <img width="847" height="205" alt="Image" src="https://github.com/user-attachments/assets/7b8cbf36-a18f-4aa2-b6b2-7c7ce2e7aa8c" /> <img width="847" height="277" alt="Image" src="https://github.com/user-attachments/assets/e129f6de-d3f7-4f25-9519-e064c0b1a99f" /> I would like to point this out again: Each MCP server runs in its own isolated Docker container, completely separated from AnythingLLM and from each other. The Mini-Bridge acts as a single, controlled communication gateway — it forwards only validated JSON-RPC requests and returns responses without exposing internal Docker or system access. This design provides: - Container isolation – every MCP has its own runtime, dependencies, and network namespace. - No Docker-in-Docker or host access – AnythingLLM never touches the MCP containers directly. - Controlled traffic flow – all communication passes through the Bridge on a private Docker network. - JSON-RPC only – no shell commands, filesystem access, or unsafe exec calls. - Dynamic registration – new MCPs can be added safely via mcp_registry.json without modifying the codebase. <img width="1220" height="97" alt="Image" src="https://github.com/user-attachments/assets/be210d2a-056c-4011-a0e1-dd3ae2a90f85" /> I’ll keep improving this project as my own independent solution. If the AnythingLLM team finds the idea useful, feel free to build on it or integrate parts of it. I plan to publish the full setup on GitHub in the coming days. As you can see, the tool output is still returned a bit unnaturally instead of a fully formatted chat response — once that’s refined, the public release will follow.
yindo commented 2026-02-22 18:31:53 -05:00
Author
Owner
Copy Link
Copy Source

@danny0094 commented on GitHub (Nov 2, 2025):

Feel free to look at the code:

https://github.com/danny0094/mcp-bridge-stack/tree/main

news:

prompt injektor:
A prompt-wrapper intercepts outgoing AnythingLLM requests, appends a short system prompt that instructs the model to decide if the query requires an MCP tool call, and forwards the augmented input to the model. The model must return a structured decision (tool id + confidence). The bridge then routes the request to the chosen MCP only if the decision meets configured checks (allowed tools, minimum confidence, timeout). This keeps AnythingLLM unmodified while enabling model-driven routing.

MCP Hub
The MCP-Hub acts as the central registry and coordination layer for all local MCP modules.
It maintains a live list of available MCP servers, checks their status, and routes requests from the Mini-Bridge or the Decision-Agent to the correct module.
Main features:
Central registry of all MCP endpoints (with auto-reload)
/list and /status APIs for monitoring
/route endpoint for direct routing control
Optional authentication and access rules
This service is designed to scale — multiple bridges or decision agents can connect to the same hub to share MCP state and routing information.

Dual-Model Architecture
The system uses two models working together:

Decision Model – a lightweight local model (e.g. Gemma 3 270M)
It analyzes every user query and decides whether the request should trigger an MCP tool (like time, docs, or weather).
It does not generate text for the user — it only outputs a structured JSON decision, such as:
{"tool": "time", "confidence": 0.92}

Main Model – a larger reasoning model (e.g. Gemma 3 12B or DeepSeek R1 8B)
This one handles the actual conversation, reasoning, and responses to the user.
It receives data from the MCP results (if the Decision Model triggered one) and integrates it naturally into its response.
Workflow:
AnythingLLM sends user input → Mini-Bridge
Mini-Bridge passes it to the Decision Model
Decision Model decides which MCP tool to use
Mini-Bridge forwards to that MCP → gets result
Result goes to the Main Model → final answer to user
This separation allows lightweight models to handle tool routing quickly,
while the larger model focuses purely on reasoning and creativity.

Benefits: lower latency, less resource use, modular scaling, clearer logic flow.

For the devs or curious people, if you want to test the code, make sure you set the correct paths in docker-compose and in the prompt_injector folder in Python:

Environment Variables

OLLAMA_URL = os.getenv("OLLAMA_URL", "http://ollama:11434/api/chat")
MCP_HUB_URL = os.getenv("MCP_HUB_URL", "http://host.docker.internal:4000")

DECISION_MODEL = "gemma3:270m"
ANSWER_MODEL = "gemma3:12b"

sets your models

This test environment does not need to be on the same computer as ollama. The important thing is that you run everything on the same network.

The mcp_hub currently contains the static tool register:

Statische Tool-Registry (dynamically expandable later)

TOOLS = {
    "time": os.getenv("MCP_TIME_URL", "http://host.docker.internal:4210/"),
    "weather": os.getenv("MCP_WEATHER_URL", "http://host.docker.internal:4220/"),
    "docs": os.getenv("MCP_DOCS_URL", "http://host.docker.internal:4230/")

Enter your test MCP server there.

You will find the following in minibridge.py:

REGISTRY_PATH = "/app/config/mcp_registry.json"
CHECK_INTERVAL = 3  # Sekunden für Reload-Check
@danny0094 commented on GitHub (Nov 2, 2025): Feel free to look at the code: https://github.com/danny0094/mcp-bridge-stack/tree/main news: prompt injektor: A prompt-wrapper intercepts outgoing AnythingLLM requests, appends a short system prompt that instructs the model to decide if the query requires an MCP tool call, and forwards the augmented input to the model. The model must return a structured decision (tool id + confidence). The bridge then routes the request to the chosen MCP only if the decision meets configured checks (allowed tools, minimum confidence, timeout). This keeps AnythingLLM unmodified while enabling model-driven routing. MCP Hub The MCP-Hub acts as the central registry and coordination layer for all local MCP modules. It maintains a live list of available MCP servers, checks their status, and routes requests from the Mini-Bridge or the Decision-Agent to the correct module. Main features: Central registry of all MCP endpoints (with auto-reload) /list and /status APIs for monitoring /route endpoint for direct routing control Optional authentication and access rules This service is designed to scale — multiple bridges or decision agents can connect to the same hub to share MCP state and routing information. Dual-Model Architecture The system uses two models working together: Decision Model – a lightweight local model (e.g. Gemma 3 270M) It analyzes every user query and decides whether the request should trigger an MCP tool (like time, docs, or weather). It does not generate text for the user — it only outputs a structured JSON decision, such as: {"tool": "time", "confidence": 0.92} Main Model – a larger reasoning model (e.g. Gemma 3 12B or DeepSeek R1 8B) This one handles the actual conversation, reasoning, and responses to the user. It receives data from the MCP results (if the Decision Model triggered one) and integrates it naturally into its response. Workflow: AnythingLLM sends user input → Mini-Bridge Mini-Bridge passes it to the Decision Model Decision Model decides which MCP tool to use Mini-Bridge forwards to that MCP → gets result Result goes to the Main Model → final answer to user This separation allows lightweight models to handle tool routing quickly, while the larger model focuses purely on reasoning and creativity. Benefits: lower latency, less resource use, modular scaling, clearer logic flow. For the devs or curious people, if you want to test the code, make sure you set the correct paths in docker-compose and in the prompt_injector folder in Python: # Environment Variables ``` OLLAMA_URL = os.getenv("OLLAMA_URL", "http://ollama:11434/api/chat") MCP_HUB_URL = os.getenv("MCP_HUB_URL", "http://host.docker.internal:4000") DECISION_MODEL = "gemma3:270m" ANSWER_MODEL = "gemma3:12b" ``` sets your models This test environment does not need to be on the same computer as ollama. The important thing is that you run everything on the same network. The mcp_hub currently contains the static tool register: # Statische Tool-Registry (dynamically expandable later) ``` TOOLS = { "time": os.getenv("MCP_TIME_URL", "http://host.docker.internal:4210/"), "weather": os.getenv("MCP_WEATHER_URL", "http://host.docker.internal:4220/"), "docs": os.getenv("MCP_DOCS_URL", "http://host.docker.internal:4230/") ``` Enter your test MCP server there. You will find the following in minibridge.py: ``` REGISTRY_PATH = "/app/config/mcp_registry.json" CHECK_INTERVAL = 3 # Sekunden für Reload-Check ```
yindo commented 2026-02-22 18:31:53 -05:00
Author
Owner
Copy Link
Copy Source

@Suranoviremblastiran commented on GitHub (Nov 4, 2025):

I can't get AnythingLLM to even know about MCP servers, everything is configured but somehow the prompt part where the tools are explained to the LLM is missing... I did change my system prompts - did I remove anything by accident - or where is it included?

@Suranoviremblastiran commented on GitHub (Nov 4, 2025): I can't get AnythingLLM to even know about MCP servers, everything is configured but somehow the prompt part where the tools are explained to the LLM is missing... I did change my system prompts - did I remove anything by accident - or where is it included?
yindo commented 2026-02-22 18:31:53 -05:00
Author
Owner
Copy Link
Copy Source

@danny0094 commented on GitHub (Nov 4, 2025):

Which file did you change exactly?

Otherwise, try this:
docker logs -n 100 anythingllm | grep MCP
and show me the log

or:
Docker compose down
and:
docker compose up
The live log is important.

If you have deleted a line in the code, the easiest thing to do is of course to reinstall Docker Compose.

Did it work before the change?
I'll be releasing an update in the next few days that includes automatic MCP usage. The prompt settings will be simplified there anyway.

Did you use the linked PYS files, or my Git repository?

@danny0094 commented on GitHub (Nov 4, 2025): Which file did you change exactly? Otherwise, try this: docker logs -n 100 anythingllm | grep MCP and show me the log or: Docker compose down and: docker compose up The live log is important. If you have deleted a line in the code, the easiest thing to do is of course to reinstall Docker Compose. Did it work before the change? I'll be releasing an update in the next few days that includes automatic MCP usage. The prompt settings will be simplified there anyway. Did you use the linked PYS files, or my Git repository?
yindo changed title from DummyMCP – A Safe, Minimal MCP Test Server for AnythingLLM in docker to [GH-ISSUE #4602] DummyMCP – A Safe, Minimal MCP Test Server for AnythingLLM in docker 2026-06-05 14:49:19 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2926
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?