Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4655] [FEAT]: Upload a markdown file with mermaid #2955

New Issue
Closed
opened 2026-02-22 18:32:00 -05:00 by yindo · 5 comments
yindo commented 2026-02-22 18:32:00 -05:00
Owner
Copy Link
Copy Source

Originally created by @brness on GitHub (Nov 17, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4655

What would you like to see?

When I upload a file with mermaid graph in it, it works OK.
But when I ask the question about the picture (like what's the architecture graph of the process), the response never find the hit the right spot. I'm not sure if it's because of the RAG system or currently anythingLLM don't support this feature

Originally created by @brness on GitHub (Nov 17, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4655 ### What would you like to see? When I upload a file with mermaid graph in it, it works OK. But when I ask the question about the picture (like what's the architecture graph of the process), the response never find the hit the right spot. I'm not sure if it's because of the RAG system or currently anythingLLM don't support this feature
yindo added the enhancementfeature request labels 2026-02-22 18:32:00 -05:00
yindo closed this issue 2026-02-22 18:32:00 -05:00
yindo commented 2026-02-22 18:32:00 -05:00
Author
Owner
Copy Link
Copy Source

@angelplusultra commented on GitHub (Nov 17, 2025):

Just to clarify, are you uploading the markdown file through the document uploader for embeddings or directly in chat as context?

@angelplusultra commented on GitHub (Nov 17, 2025): Just to clarify, are you uploading the markdown file through the document uploader for embeddings or directly in chat as context?
yindo commented 2026-02-22 18:32:01 -05:00
Author
Owner
Copy Link
Copy Source

@brness commented on GitHub (Nov 17, 2025):

the file are uploaded through the embedder

@brness commented on GitHub (Nov 17, 2025): the file are uploaded through the embedder
yindo commented 2026-02-22 18:32:01 -05:00
Author
Owner
Copy Link
Copy Source

@brness commented on GitHub (Nov 17, 2025):

And may I ask one more thing about RAG?
If I upload the docs with picture in it, it will not display the picture in the response. It's because of picture are store in base64 in RAG, and when LLM calculate the cosine distance similarity, it will hit some unrelative content. That's the reason why I want to change the picture to mermaid.
Or I think RAG system are not suitable for the picture docs, I have to change all the docs with picture to txt only?

@brness commented on GitHub (Nov 17, 2025): And may I ask one more thing about RAG? If I upload the docs with picture in it, it will not display the picture in the response. It's because of picture are store in base64 in RAG, and when LLM calculate the cosine distance similarity, it will hit some unrelative content. That's the reason why I want to change the picture to mermaid. Or I think RAG system are not suitable for the picture docs, I have to change all the docs with picture to txt only?
yindo commented 2026-02-22 18:32:01 -05:00
Author
Owner
Copy Link
Copy Source

@angelplusultra commented on GitHub (Nov 18, 2025):

RAG isn’t well-suited for interpreting or reasoning over Markdown-based graph definitions (like Mermaid). These diagram blocks are essentially source code describing a visual structure, and embeddings are optimized for natural language, not diagram syntax or node/edge relationships. As a result, retrieval won’t reliably match queries about the structure or meaning of these graphs.

RAG works best when the user is asking semantically grounded questions about natural-language content in the documents. Tasks like interpreting media, understanding diagram code, or summarizing an entire document are outside the strengths of a retrieval-based architecture.

If you want the model to reason directly about the diagram, the best approach is to upload the document direct in chat so it’s part of the immediate context. This gives the LLM full visibility and leads to significantly better results.

@angelplusultra commented on GitHub (Nov 18, 2025): RAG isn’t well-suited for interpreting or reasoning over Markdown-based graph definitions (like Mermaid). These diagram blocks are essentially source code describing a visual structure, and embeddings are optimized for natural language, not diagram syntax or node/edge relationships. As a result, retrieval won’t reliably match queries about the structure or meaning of these graphs. RAG works best when the user is asking semantically grounded questions about natural-language content in the documents. Tasks like interpreting media, understanding diagram code, or summarizing an entire document are outside the strengths of a retrieval-based architecture. If you want the model to reason directly about the diagram, the best approach is to upload the document direct in chat so it’s part of the immediate context. This gives the LLM full visibility and leads to significantly better results.
yindo commented 2026-02-22 18:32:01 -05:00
Author
Owner
Copy Link
Copy Source

@brness commented on GitHub (Nov 18, 2025):

That explains~~ Thanks for your patient reply

@brness commented on GitHub (Nov 18, 2025): That explains~~ Thanks for your patient reply
yindo changed title from [FEAT]: Upload a markdown file with mermaid to [GH-ISSUE #4655] [FEAT]: Upload a markdown file with mermaid 2026-06-05 14:49:28 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2955
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?