Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2235] [BUG]: How to upload a folder with subfolders with files to AnythingLLM? #1456

New Issue
Open
opened 2026-02-22 18:24:53 -05:00 by yindo · 12 comments
yindo commented 2026-02-22 18:24:53 -05:00
Owner
Copy Link
Copy Source

Originally created by @venturaEffect on GitHub (Sep 6, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2235

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

Hi, I want to prepare all the folders with its files for another person on his AnythingLLM. I'm trying to find a folder where to upload this on my computer as it seems the AnythingLLM just doesn't allow to upload folders just file by file what is tedious and not very useful if trying to use a RAG in its real power. So I would appreciate if somebody knows where (PATH) to go to upload a folder with all the files for the workspace. And also not clear what type of files it accepts, the info modal where to updates it says "Click to upload or drag and drop. supports text files, csv's, spreadsheets, audio files, and more!", well what does this even mean? I try to upload a small .rtf file and it holds forever without success.

BTS, docs doesn't explain much. Would appreciate if you give a list of the files it accepts because it would be very useful to a lot of time.

Appreciate your time.

Are there known steps to reproduce?

No response

Originally created by @venturaEffect on GitHub (Sep 6, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2235 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? Hi, I want to prepare all the folders with its files for another person on his AnythingLLM. I'm trying to find a folder where to upload this on my computer as it seems the AnythingLLM just doesn't allow to upload folders just file by file what is tedious and not very useful if trying to use a RAG in its real power. So I would appreciate if somebody knows where (PATH) to go to upload a folder with all the files for the workspace. And also not clear what type of files it accepts, the info modal where to updates it says "Click to upload or drag and drop. supports text files, csv's, spreadsheets, audio files, and more!", well what does this even mean? I try to upload a small .rtf file and it holds forever without success. BTS, docs doesn't explain much. Would appreciate if you give a list of the files it accepts because it would be very useful to a lot of time. Appreciate your time. ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:24:53 -05:00
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@redkit75 commented on GitHub (Sep 11, 2024):

Subdirectory support is definitely needed. Especially when working on code, the path is crucial due to the MVC structure. For example, even though both files are loaded, when asked to print the method of a route in the controller, the model cannot perform this task because the path of the file added to the vector database does not match the predicted path the model knows. Additionally, there are files with the same name. Therefore, adding the subdirectory feature would be great.

@redkit75 commented on GitHub (Sep 11, 2024): Subdirectory support is definitely needed. Especially when working on code, the path is crucial due to the MVC structure. For example, even though both files are loaded, when asked to print the method of a route in the controller, the model cannot perform this task because the path of the file added to the vector database does not match the predicted path the model knows. Additionally, there are files with the same name. Therefore, adding the subdirectory feature would be great.
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@findyourexit commented on GitHub (Jan 4, 2025):

I'm glad I'm not the only one looking for a way to achieve this! 😅

I'd really love the ability to select a folder/directory to embed, rather than the existing file-by-file approach. My use-case is that I'd like to RAG on an Obsidian Vault, but I'd also like to do the same kind of thing with source code - basically any arbitrarily deep structure of subdirectories.

Please forgive my naivety - I'm a newcomer to AnythingLLM (and really liking it, thus far). Would the feature/improvement be as simple as implementing the following logic?

  1. Accept a Folder as an Upload target. (Alongside the existing File support.)
  2. Once a Folder is specified as an Upload target, traverse the descendant subdirectories recursively to discover all Files.
  3. Add all discovered Files to the My Documents collection (within AnythingLLM - perhaps as an appropriately named New Folder).

I imagine that this would then enable the user to select any/all of the discovered Files with a single click, and execute a single "Move to Workspace" operation, to have the Files added to the Workspace?

Again, total shot in the dark here - I don't mean to trivialise something that could be extraordinarily more complex to implement, in practice. 🙏🏻

Edit:
I'm aware of the drag-drop approach for achieving much of what I've described above (it's referenced in this issue thread). The distinction I'd like to highlight is adding support via the File/Folder Picker interface, rather than via the drag-drop mechanism.

The ultimate outcome would be one that preserves the File paths on Upload - i.e. all the subdirectories are carried forward into the My Documents collection within AnythingLLM, rather than flattening out the whole file structure by placing all discovered Files together.

@findyourexit commented on GitHub (Jan 4, 2025): I'm glad I'm not the only one looking for a way to achieve this! 😅 I'd really love the ability to select a folder/directory to embed, rather than the existing file-by-file approach. My use-case is that I'd like to RAG on an Obsidian Vault, but I'd also like to do the same kind of thing with source code - basically any arbitrarily deep structure of subdirectories. Please forgive my naivety - I'm a newcomer to AnythingLLM (and _**really**_ liking it, thus far). Would the feature/improvement be as simple as implementing the following logic? 1. Accept a **Folder** as an **Upload** target. _(Alongside the existing **File** support.)_ 2. Once a **Folder** is specified as an **Upload** target, traverse the descendant subdirectories recursively to discover all **Files**. 3. Add all discovered **Files** to the **My Documents** collection _(within AnythingLLM - perhaps as an appropriately named **New Folder**)_. I imagine that this would then enable the user to select any/all of the discovered **Files** with a single click, and execute a single **"Move to Workspace"** operation, to have the **Files** added to the **Workspace**? _Again, total shot in the dark here - I don't mean to trivialise something that could be extraordinarily more complex to implement, in practice._ 🙏🏻 **Edit:** I'm aware of the **drag-drop** approach for achieving much of what I've described above (it's referenced in this [issue thread](https://github.com/Mintplex-Labs/anything-llm/issues/1366)). The distinction I'd like to highlight is adding support via the **File/Folder Picker** interface, rather than via the **drag-drop** mechanism. The ultimate outcome would be one that preserves the **File** paths on **Upload** - i.e. all the subdirectories are carried forward into the **My Documents** collection within AnythingLLM, rather than flattening out the whole file structure by placing all discovered **Files** together.
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 6, 2025):

@findyourexit Well put. This issue has been open for so long simply because it isn't exactly trivial but it's by no means an impossible feat or splitting the atom. It's just something that has the opportunity to really cause a mess in the file picker since it only supports 1 level deep folders right now.

The file picker is really the limitation and if we mess that up it causes a ton of problems downstream. If anyone is wondering why we are sitting on this, its not that it is not important or we don't care. Its just a lot more work than a cursory look would suggest.

@timothycarambat commented on GitHub (Jan 6, 2025): @findyourexit Well put. This issue has been open for so long simply because it isn't _exactly_ trivial but it's by no means an impossible feat or splitting the atom. It's just something that has the opportunity to really cause a mess in the file picker since it only supports 1 level deep folders right now. The file picker is really the limitation and if we mess that up it causes a ton of problems downstream. If anyone is wondering why we are sitting on this, its not that it is not important or we don't care. Its just a **lot** more work than a cursory look would suggest.
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@Yuki001 commented on GitHub (Jan 19, 2025):

A new Data Connector “Local folder” is needed, it collects files recursive in the folder like the "GitHub Repo" Data Connector does. "GitHub Repo" Connector can create new folder for all documents and has relative path in the collected document file name.
If is not possible to visit local folder in docker/browser, it can be added only into desktop version application.

@Yuki001 commented on GitHub (Jan 19, 2025): A new Data Connector “Local folder” is needed, it collects files recursive in the folder like the "GitHub Repo" Data Connector does. "GitHub Repo" Connector can create new folder for all documents and has relative path in the collected document file name. If is not possible to visit local folder in docker/browser, it can be added only into desktop version application.
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@workingmagic commented on GitHub (Jan 20, 2025):

I am sorry to ask, as a non-programmer. I am setting up my own Ultima Online Server and within creating new scripts and altering scripts with several LLMs like Claude and ChatGPT. But to give them the whole picture, which are the ServUO source files (280 MB, over 8,000 files), the ClassicUO source files (33 MB, 787 files) and the ServUO HTML documentation (67 MB, over 14,000 files). Yep, Big Data in small size. I am looking for a way to process this. Claude told me, use AnythingLLM.

TLDR: I searched for subfolders in AnythingLLM, that led me to this page here.

What is the current state of things? Would I be able to process a bigger dataset like this, or are there any other tools I need to consider. I actually want to finetune/teach the AI's (over API) to understand what each script is doing and what relationships they have to other scripts. It worked on a small base through the knowledge database, but the limited space and token context window limit is counterproductive at times.

@workingmagic commented on GitHub (Jan 20, 2025): I am sorry to ask, as a non-programmer. I am setting up my own Ultima Online Server and within creating new scripts and altering scripts with several LLMs like Claude and ChatGPT. But to give them the whole picture, which are the ServUO source files (280 MB, over 8,000 files), the ClassicUO source files (33 MB, 787 files) and the ServUO HTML documentation (67 MB, over 14,000 files). Yep, Big Data in small size. I am looking for a way to process this. Claude told me, use AnythingLLM. **TLDR: I searched for subfolders in AnythingLLM, that led me to this page here.** What is the current state of things? Would I be able to process a bigger dataset like this, or are there any other tools I need to consider. I actually want to finetune/teach the AI's (over API) to understand what each script is doing and what relationships they have to other scripts. It worked on a small base through the knowledge database, but the limited space and token context window limit is counterproductive at times.
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@GraniLuk commented on GitHub (Feb 27, 2025):

I would really like to see this feature, I need to include folder with multiple markdown files to my LLM.

@GraniLuk commented on GitHub (Feb 27, 2025): I would really like to see this feature, I need to include folder with multiple markdown files to my LLM.
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@Y-Sari commented on GitHub (May 8, 2025):

This would be a great feature to add to AnythingLLM!

While you are investigating and prioritizing this, is there a good workaround for the time being?

Does AnythingLLM (Desktop) have an API that we could use to upload documents?
Than we could create our own scripts that use that API until this feature becomes part of the product.

Helpful links are appreciated.

Thanks!

@Y-Sari commented on GitHub (May 8, 2025): This would be a great feature to add to AnythingLLM! While you are investigating and prioritizing this, is there a good workaround for the time being? Does AnythingLLM (Desktop) have an API that we could use to upload documents? Than we could create our own scripts that use that API until this feature becomes part of the product. Helpful links are appreciated. Thanks!
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@chadananda commented on GitHub (Jun 7, 2025):

The important thing nowadays is to be able to give a good demo, not support actual user workflows.

That seems like the flaw of many projects nowadays. You only need a few flat files for a demo but real data lives in nested folders.

@chadananda commented on GitHub (Jun 7, 2025): The important thing nowadays is to be able to give a good demo, not support actual user workflows. That seems like the flaw of many projects nowadays. You only need a few flat files for a demo but real data lives in nested folders.
yindo commented 2026-02-22 18:24:54 -05:00
Author
Owner
Copy Link
Copy Source

@raoulg commented on GitHub (Jun 9, 2025):

The important thing nowadays is to be able to give a good demo, not support actual user workflows.

That seems like the flaw of many projects nowadays. You only need a few flat files for a demo but real data lives in nested folders.

The important thing nowadays is that people rather complain about open source software and point to how other people should work harder, instead of picking up the task by themselves.

This seems like a character flaw in many people nowadays. You only need so much intelligence to add a comment but the real effort is in starting to code.

@raoulg commented on GitHub (Jun 9, 2025): > The important thing nowadays is to be able to give a good demo, not support actual user workflows. > > That seems like the flaw of many projects nowadays. You only need a few flat files for a demo but real data lives in nested folders. The important thing nowadays is that people rather complain about open source software and point to how other people should work harder, instead of picking up the task by themselves. This seems like a character flaw in many people nowadays. You only need so much intelligence to add a comment but the real effort is in starting to code.
yindo commented 2026-02-22 18:24:55 -05:00
Author
Owner
Copy Link
Copy Source

@thiromi commented on GitHub (Jun 16, 2025):

As we don't have this right now, we can leverage that AnythingLLM supports MCP and use mcp/filesystem. The only stubborn thing is having to use @agent when prompting

@thiromi commented on GitHub (Jun 16, 2025): As we don't have this right now, we can leverage that [AnythingLLM supports MCP](https://docs.anythingllm.com/mcp-compatibility/overview) and use [mcp/filesystem](https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem#usage-with-claude-desktop). The only stubborn thing is having to use `@agent` when prompting
yindo commented 2026-02-22 18:24:55 -05:00
Author
Owner
Copy Link
Copy Source

@bogart99j commented on GitHub (Nov 30, 2025):

Did this capability get added? In my project, Docker can see all subfolders and files.

However, the AnythingLLM web UI does not display subfolders when I click the arrow or double-click on the mounted folder.

I can only see files at the root level of the mount, not files inside subfolders

@bogart99j commented on GitHub (Nov 30, 2025): Did this capability get added? In my project, Docker can see all subfolders and files. However, the AnythingLLM web UI does not display subfolders when I click the arrow or double-click on the mounted folder. I can only see files at the root level of the mount, not files inside subfolders
yindo changed title from [BUG]: How to upload a folder with subfolders with files to AnythingLLM? to [GH-ISSUE #2235] [BUG]: How to upload a folder with subfolders with files to AnythingLLM? 2026-06-05 14:40:52 -04:00
yindo commented 2026-06-05 14:40:53 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 16, 2026):

Linking #3561 #5000

<!-- gh-comment-id:4068684482 --> @timothycarambat commented on GitHub (Mar 16, 2026): Linking #3561 #5000
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1456
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?