Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5204] [CLOSED] File storage_dir fix & refactor #5350

New Issue
Closed
opened 2026-06-05 15:21:03 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 15:21:03 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5204
Author: @timothycarambat
Created: 3/13/2026
Status: ❌ Closed

Base: master ← Head: 5152-file-fix-and-refactor

📝 Commits (9)

  • ce842f0 File storage_dir fix & refactor
  • bc77ebb remove legacy placeholder files
  • b9b9f2a patch env resolution for tests
  • a063982 Merge branch 'master' into 5152-file-fix-and-refactor
  • 25acdbb improve addDocumentToWorkspace logging
  • ddca814 move paths into own module to prevent circular deps
  • 801ee39 fix stupid job
  • 570974d fix stupid job
  • cb50483 Merge branch 'master' into 5152-file-fix-and-refactor

📊 Changes

65 files changed (+1821 additions, -343 deletions)

View changed files

📝 .github/workflows/check-package-versions.yaml (+14 -9)
📝 collector/.gitignore (+1 -2)
📝 collector/eslint.config.mjs (+51 -4)
➖ collector/hotdir/__HOTDIR__.md (+0 -3)
📝 collector/index.js (+7 -3)
📝 collector/package.json (+1 -0)
📝 collector/processSingleFile/index.js (+2 -11)
➖ collector/storage/.gitignore (+0 -2)
➖ collector/storage/tmp/.placeholder (+0 -0)
📝 collector/utils/OCRLoader/index.js (+2 -5)
📝 collector/utils/WhisperProviders/localWhisper.js (+2 -6)
📝 collector/utils/comKey/index.js (+2 -8)
📝 collector/utils/constants.js (+0 -3)
📝 collector/utils/downloadURIToFile/index.js (+23 -2)
📝 collector/utils/extensions/Confluence/index.js (+6 -9)
📝 collector/utils/extensions/DrupalWiki/DrupalWiki/index.js (+2 -4)
📝 collector/utils/extensions/RepoLoader/GithubRepo/index.js (+2 -9)
📝 collector/utils/extensions/RepoLoader/GitlabRepo/index.js (+6 -9)
📝 collector/utils/files/index.js (+69 -44)
📝 collector/yarn.lock (+779 -9)

...and 45 more files

📄 Description

Pull Request Type

  • ✨ feat (New feature)
  • 🐛 fix (Bug fix)
  • ♻️ refactor (Code refactoring without changing behavior)
  • 💄 style (UI style changes)
  • 🔨 chore (Build, CI, maintenance)
  • 📝 docs (Documentation updates)

Relevant Issues

resolves #5152

Description

Patches issue with container hotdir resolution.

Visuals (if applicable)

Additional Information

Refactored a ton of the use of file paths across the application.
Normalize storage path resolution across server and collector
Centralizes all storage path constants into single source-of-truth exports:

  • server: baseStoragePath, documentsPath, directUploadsPath, vectorCachePath, hotdirPath
  • collector: basePrimaryStoragePath, WATCH_DIRECTORY, tmpStorage, documentsFolder

This eliminates scattered NODE_ENV/STORAGE_DIR ternaries throughout the codebase and ensures consistent path resolution between server and collector.

Other changes:

  • Fix hotdirPath to correctly point to collector/storage/hotdir in development
  • Add ensureRequiredDirectoriesExist() to create storage dirs on collector startup
  • Add path traversal protection in downloadURIToFile
  • Add eslint-plugin-import for unresolved import detection
  • Remove static placeholder files (now created dynamically)

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5204 **Author:** [@timothycarambat](https://github.com/timothycarambat) **Created:** 3/13/2026 **Status:** ❌ Closed **Base:** `master` ← **Head:** `5152-file-fix-and-refactor` --- ### 📝 Commits (9) - [`ce842f0`](https://github.com/Mintplex-Labs/anything-llm/commit/ce842f0cb95b6ac3906755435c74728c3f20d55f) File storage_dir fix & refactor - [`bc77ebb`](https://github.com/Mintplex-Labs/anything-llm/commit/bc77ebb4864750c0dc988a03888e5b61b15c9445) remove legacy placeholder files - [`b9b9f2a`](https://github.com/Mintplex-Labs/anything-llm/commit/b9b9f2aec3703729bdd5c9ace39111498eac9c7f) patch env resolution for tests - [`a063982`](https://github.com/Mintplex-Labs/anything-llm/commit/a063982573db8f5b4a4de5117c5cc36e0e4a4711) Merge branch 'master' into 5152-file-fix-and-refactor - [`25acdbb`](https://github.com/Mintplex-Labs/anything-llm/commit/25acdbb84609b90462fab577dc8268bec1b24868) improve addDocumentToWorkspace logging - [`ddca814`](https://github.com/Mintplex-Labs/anything-llm/commit/ddca814b88be144cf0376f181d8fe5b3c2866f40) move paths into own module to prevent circular deps - [`801ee39`](https://github.com/Mintplex-Labs/anything-llm/commit/801ee39fd9d68983b7746ecc1f0cf9d4b8cdb9fe) fix stupid job - [`570974d`](https://github.com/Mintplex-Labs/anything-llm/commit/570974dc6ad4e74721fe32e223fc55aa333a01b9) fix stupid job - [`cb50483`](https://github.com/Mintplex-Labs/anything-llm/commit/cb50483bf71bd08cf2c6b909fd9f0d5a16f9cf21) Merge branch 'master' into 5152-file-fix-and-refactor ### 📊 Changes **65 files changed** (+1821 additions, -343 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/check-package-versions.yaml` (+14 -9) 📝 `collector/.gitignore` (+1 -2) 📝 `collector/eslint.config.mjs` (+51 -4) ➖ `collector/hotdir/__HOTDIR__.md` (+0 -3) 📝 `collector/index.js` (+7 -3) 📝 `collector/package.json` (+1 -0) 📝 `collector/processSingleFile/index.js` (+2 -11) ➖ `collector/storage/.gitignore` (+0 -2) ➖ `collector/storage/tmp/.placeholder` (+0 -0) 📝 `collector/utils/OCRLoader/index.js` (+2 -5) 📝 `collector/utils/WhisperProviders/localWhisper.js` (+2 -6) 📝 `collector/utils/comKey/index.js` (+2 -8) 📝 `collector/utils/constants.js` (+0 -3) 📝 `collector/utils/downloadURIToFile/index.js` (+23 -2) 📝 `collector/utils/extensions/Confluence/index.js` (+6 -9) 📝 `collector/utils/extensions/DrupalWiki/DrupalWiki/index.js` (+2 -4) 📝 `collector/utils/extensions/RepoLoader/GithubRepo/index.js` (+2 -9) 📝 `collector/utils/extensions/RepoLoader/GitlabRepo/index.js` (+6 -9) 📝 `collector/utils/files/index.js` (+69 -44) 📝 `collector/yarn.lock` (+779 -9) _...and 45 more files_ </details> ### 📄 Description ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [ ] ✨ feat (New feature) - [x] 🐛 fix (Bug fix) - [x] ♻️ refactor (Code refactoring without changing behavior) - [ ] 💄 style (UI style changes) - [ ] 🔨 chore (Build, CI, maintenance) - [ ] 📝 docs (Documentation updates) ### Relevant Issues <!-- Use "resolves #xxx" to auto resolve on merge. Otherwise, please use "connect #xxx" --> resolves #5152 ### Description Patches issue with container `hotdir` resolution. <!-- Describe the changes in this PR that are impactful to the repo. What problem does it solve? --> ### Visuals (if applicable) <!-- Add screenshots or screen recordings to demonstrate the changes, especially for UI updates. --> ### Additional Information Refactored a ton of the use of file paths across the application. Normalize storage path resolution across server and collector Centralizes all storage path constants into single source-of-truth exports: - server: `baseStoragePath`, `documentsPath`, `directUploadsPath`, `vectorCachePath`, `hotdirPath` - collector: `basePrimaryStoragePath`, `WATCH_DIRECTORY`, `tmpStorage`, `documentsFolder` This eliminates scattered NODE_ENV/STORAGE_DIR ternaries throughout the codebase and ensures consistent path resolution between server and collector. Other changes: - Fix hotdirPath to correctly point to collector/storage/hotdir in development - Add ensureRequiredDirectoriesExist() to create storage dirs on collector startup - Add path traversal protection in downloadURIToFile - Add eslint-plugin-import for unresolved import detection - Remove static placeholder files (now created dynamically) <!-- Add any other context about the Pull Request here that was not captured above. --> ### Developer Validations <!-- All of the applicable items should be checked. --> - [ ] I ran `yarn lint` from the root of the repo & committed changes - [ ] Relevant documentation has been updated (if applicable) - [ ] I have tested my code functionality - [ ] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 15:21:03 -04:00
yindo closed this issue 2026-06-05 15:21:03 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5350
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?