Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2065] Failed to vectorize PDF file and addDocumentToNamespace lance error (Permission denied (os error 13)) #1346

New Issue
Closed
opened 2026-02-22 18:24:22 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:24:22 -05:00
Owner
Copy Link
Copy Source

Originally created by @bennwei on GitHub (Aug 7, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2065

How are you running AnythingLLM?

Docker (local)

What happened?

Current setup:

  • Using Docker Desktop in Win 11
  • AnythingLLM:latest run in docker with port - http://localhost:3001, following the setup guide.
  • Ollama run in docker with port - http://localhost:11434
  • Ollama is serving phi3-8b LLM model and I was able to sent chat and get response though AnythingLLM workspace UI.

When adding a PDF file to the same Workspace, I encounter Permission denied (os error 13) when Inserting vectorized chunks into LanceDB collection with the following log:

  2024-08-07 14:40:17 [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
  2024-08-07 14:40:17 [backend] info: prisma:info Starting a sqlite pool with 13 connections.
  2024-08-07 14:40:17 [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"4123f43c-190a-4234-9700-c2d604c4d24f","properties":{"commit":"--","runtime":"docker"}}
  2024-08-07 14:40:17 [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
  2024-08-07 14:40:17 [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
  2024-08-07 14:40:17 [backend] info: Primary server in HTTP mode listening on port 3001
  2024-08-07 14:40:17 [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started.
  2024-08-07 14:40:59 [backend] info: [MetaGenerator] fetching custome meta tag settings...
  2024-08-07 14:41:44 [backend] info: [TELEMETRY SENT] {"event":"workspace_created","distinctId":"4123f43c-190a-4234-9700-c2d604c4d24f::1","properties":{"multiUserMode":true,"LLMSelection":"ollama","Embedder":"native","VectorDbSelection":"lancedb","runtime":"docker"}}
  2024-08-07 14:41:44 [backend] info: [Event Logged] - workspace_created
  2024-08-07 14:42:03 [backend] info: [NativeEmbedder] Initialized
  2024-08-07 14:42:11 [backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"4123f43c-190a-4234-9700-c2d604c4d24f","properties":{"multiUserMode":true,"LLMSelection":"ollama","Embedder":"native","VectorDbSelection":"lancedb","multiModal":false,"runtime":"docker"}}
  2024-08-07 14:42:11 [backend] info: [Event Logged] - sent_chat
  2024-08-07 14:44:54 [backend] info: Adding new vectorized document into namespace ragtest1
  2024-08-07 14:44:54 [backend] info: [NativeEmbedder] Initialized
  2024-08-07 14:44:54 [backend] info: [RecursiveSplitter] Will split with {"chunkSize":1000,"chunkOverlap":20}
  2024-08-07 14:44:54 [backend] info: Chunks created from document: 209
  2024-08-07 14:44:58 [backend] info: [NativeEmbedder] Embedded Chunk 1 of 9
  2024-08-07 14:45:01 [backend] info: [NativeEmbedder] Embedded Chunk 2 of 9
  2024-08-07 14:45:05 [backend] info: [NativeEmbedder] Embedded Chunk 3 of 9
  2024-08-07 14:45:09 [backend] info: [NativeEmbedder] Embedded Chunk 4 of 9
  2024-08-07 14:45:12 [backend] info: [NativeEmbedder] Embedded Chunk 5 of 9
  2024-08-07 14:45:16 [backend] info: [NativeEmbedder] Embedded Chunk 6 of 9
  2024-08-07 14:45:20 [backend] info: [NativeEmbedder] Embedded Chunk 7 of 9
  2024-08-07 14:45:24 [backend] info: [NativeEmbedder] Embedded Chunk 8 of 9
  2024-08-07 14:45:26 [backend] info: [NativeEmbedder] Embedded Chunk 9 of 9
  2024-08-07 14:45:27 [backend] info: Inserting vectorized chunks into LanceDB collection.
  2024-08-07 14:45:28 [backend] error: addDocumentToNamespace lance error: LanceError(IO): Generic LocalFileSystem error: Unable to copy file from /app/server/storage/lancedb/ragtest1.lance/_versions/.tmp_1.manifest_c912e595-55b6-4b94-9e20-88efd350657a to /app/server/storage/lancedb/ragtest1.lance/_versions/1.manifest: Permission denied (os error 13), /home/build_user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-table-0.12.1/src/io/commit.rs:692:54
  2024-08-07 14:45:28 [backend] error: Failed to vectorize 2019_Mortality_Report_FINAL_052022.pdf
  2024-08-07 14:45:28 [backend] info: [TELEMETRY SENT] {"event":"documents_embedded_in_workspace","distinctId":"4123f43c-190a-4234-9700-c2d604c4d24f","properties":{"LLMSelection":"ollama","Embedder":"native","VectorDbSelection":"lancedb","runtime":"docker"}}
  2024-08-07 14:45:28 [backend] info: [Event Logged] - workspace_documents_added

To trouble shoot, I did check the following:

  • Bind mounts correctly between my local folder to Docker
    image

  • LanceDB folder appear to contain necessary files for the document embed:
    image
    image

  • In AnythingLLM docker shell, try change permission of app/server/storage/lancedb folder using:
    chmod -R 777 /app/server/storage/lancedb, but this does not seem to work as it only change permission for existing folder/files, and does not change permission levels when creating a new workspace (see below "ragtest1.lance" folder) after running 'chmod':

image

If anyone encounter similar issues, I would like to hear how you can resolve this?

Thank you!

Are there known steps to reproduce?

No response

Originally created by @bennwei on GitHub (Aug 7, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2065 ### How are you running AnythingLLM? Docker (local) ### What happened? Current setup: - Using Docker Desktop in Win 11 - AnythingLLM:latest run in docker with port - http://localhost:3001, following the [setup guide](https://github.com/Mintplex-Labs/anything-llm/blob/master/docker/HOW_TO_USE_DOCKER.md). - Ollama run in docker with port - http://localhost:11434 - Ollama is serving phi3-8b LLM model and I was able to sent chat and get response though AnythingLLM workspace UI. When adding a PDF file to the same Workspace, I encounter Permission denied (os error 13) when Inserting vectorized chunks into LanceDB collection with the following log: ``` 2024-08-07 14:40:17 [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. 2024-08-07 14:40:17 [backend] info: prisma:info Starting a sqlite pool with 13 connections. 2024-08-07 14:40:17 [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"4123f43c-190a-4234-9700-c2d604c4d24f","properties":{"commit":"--","runtime":"docker"}} 2024-08-07 14:40:17 [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. 2024-08-07 14:40:17 [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. 2024-08-07 14:40:17 [backend] info: Primary server in HTTP mode listening on port 3001 2024-08-07 14:40:17 [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started. 2024-08-07 14:40:59 [backend] info: [MetaGenerator] fetching custome meta tag settings... 2024-08-07 14:41:44 [backend] info: [TELEMETRY SENT] {"event":"workspace_created","distinctId":"4123f43c-190a-4234-9700-c2d604c4d24f::1","properties":{"multiUserMode":true,"LLMSelection":"ollama","Embedder":"native","VectorDbSelection":"lancedb","runtime":"docker"}} 2024-08-07 14:41:44 [backend] info: [Event Logged] - workspace_created 2024-08-07 14:42:03 [backend] info: [NativeEmbedder] Initialized 2024-08-07 14:42:11 [backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"4123f43c-190a-4234-9700-c2d604c4d24f","properties":{"multiUserMode":true,"LLMSelection":"ollama","Embedder":"native","VectorDbSelection":"lancedb","multiModal":false,"runtime":"docker"}} 2024-08-07 14:42:11 [backend] info: [Event Logged] - sent_chat 2024-08-07 14:44:54 [backend] info: Adding new vectorized document into namespace ragtest1 2024-08-07 14:44:54 [backend] info: [NativeEmbedder] Initialized 2024-08-07 14:44:54 [backend] info: [RecursiveSplitter] Will split with {"chunkSize":1000,"chunkOverlap":20} 2024-08-07 14:44:54 [backend] info: Chunks created from document: 209 2024-08-07 14:44:58 [backend] info: [NativeEmbedder] Embedded Chunk 1 of 9 2024-08-07 14:45:01 [backend] info: [NativeEmbedder] Embedded Chunk 2 of 9 2024-08-07 14:45:05 [backend] info: [NativeEmbedder] Embedded Chunk 3 of 9 2024-08-07 14:45:09 [backend] info: [NativeEmbedder] Embedded Chunk 4 of 9 2024-08-07 14:45:12 [backend] info: [NativeEmbedder] Embedded Chunk 5 of 9 2024-08-07 14:45:16 [backend] info: [NativeEmbedder] Embedded Chunk 6 of 9 2024-08-07 14:45:20 [backend] info: [NativeEmbedder] Embedded Chunk 7 of 9 2024-08-07 14:45:24 [backend] info: [NativeEmbedder] Embedded Chunk 8 of 9 2024-08-07 14:45:26 [backend] info: [NativeEmbedder] Embedded Chunk 9 of 9 2024-08-07 14:45:27 [backend] info: Inserting vectorized chunks into LanceDB collection. 2024-08-07 14:45:28 [backend] error: addDocumentToNamespace lance error: LanceError(IO): Generic LocalFileSystem error: Unable to copy file from /app/server/storage/lancedb/ragtest1.lance/_versions/.tmp_1.manifest_c912e595-55b6-4b94-9e20-88efd350657a to /app/server/storage/lancedb/ragtest1.lance/_versions/1.manifest: Permission denied (os error 13), /home/build_user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-table-0.12.1/src/io/commit.rs:692:54 2024-08-07 14:45:28 [backend] error: Failed to vectorize 2019_Mortality_Report_FINAL_052022.pdf 2024-08-07 14:45:28 [backend] info: [TELEMETRY SENT] {"event":"documents_embedded_in_workspace","distinctId":"4123f43c-190a-4234-9700-c2d604c4d24f","properties":{"LLMSelection":"ollama","Embedder":"native","VectorDbSelection":"lancedb","runtime":"docker"}} 2024-08-07 14:45:28 [backend] info: [Event Logged] - workspace_documents_added ``` To trouble shoot, I did check the following: - Bind mounts correctly between my local folder to Docker ![image](https://github.com/user-attachments/assets/cf6e078e-3279-4c80-9c2f-6da2cdca4e98) - LanceDB folder appear to contain necessary files for the document embed: ![image](https://github.com/user-attachments/assets/81690216-133d-4a26-b456-c9e1fb8d07ac) ![image](https://github.com/user-attachments/assets/190c5d68-36c3-462a-afe5-4cb942ed6982) - In AnythingLLM docker shell, try change permission of app/server/storage/lancedb folder using: `chmod -R 777 /app/server/storage/lancedb`, but this does not seem to work as it only change permission for existing folder/files, and does not change permission levels when creating a new workspace (see below "ragtest1.lance" folder) after running 'chmod': ![image](https://github.com/user-attachments/assets/39ae56ac-9d3d-4c3d-8fe7-9ca24dc78580) If anyone encounter similar issues, I would like to hear how you can resolve this? Thank you! ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:24:22 -05:00
yindo closed this issue 2026-02-22 18:24:22 -05:00
yindo commented 2026-02-22 18:24:22 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 7, 2024):

You need to open the permissions to your host directory not the permissions in the docker container 👍

@timothycarambat commented on GitHub (Aug 7, 2024): You need to open the permissions to your _host directory_ not the permissions in the docker container 👍
yindo changed title from Failed to vectorize PDF file and addDocumentToNamespace lance error (Permission denied (os error 13)) to [GH-ISSUE #2065] Failed to vectorize PDF file and addDocumentToNamespace lance error (Permission denied (os error 13)) 2026-06-05 14:40:16 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1346
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?