Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3564] [BUG]: Documents are failing to vectorize. #2297

New Issue
Closed
opened 2026-02-22 18:29:03 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:29:03 -05:00
Owner
Copy Link
Copy Source

Originally created by @DangerousBerries on GitHub (Mar 30, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3564

How are you running AnythingLLM?

Docker (local)

What happened?

[backend] error: addDocumentToNamespace lance error: LanceError(IO): failed to shutdown object writer for app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance: Generic LocalFileSystem error: Unable to access metadata for /app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance#1: No such file or directory (os error 2), /home/build_user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-io-0.22.0/src/object_writer.rs:294:17 Caused by: LanceError(IO): failed to shutdown object writer for app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance: Generic LocalFileSystem error: Unable to access metadata for /app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance#1: No such file or directory (os error 2), /home/build_user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-io-0.22.0/src/object_writer.rs:294:17 Caused by: failed to shutdown object writer for app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance: Generic LocalFileSystem error: Unable to access metadata for /app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance#1: No such file or directory (os error 2)``

Are there known steps to reproduce?

No response

Originally created by @DangerousBerries on GitHub (Mar 30, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3564 ### How are you running AnythingLLM? Docker (local) ### What happened? `[backend] error: addDocumentToNamespace lance error: LanceError(IO): failed to shutdown object writer for app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance: Generic LocalFileSystem error: Unable to access metadata for /app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance#1: No such file or directory (os error 2), /home/build_user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-io-0.22.0/src/object_writer.rs:294:17 Caused by: LanceError(IO): failed to shutdown object writer for app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance: Generic LocalFileSystem error: Unable to access metadata for /app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance#1: No such file or directory (os error 2), /home/build_user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-io-0.22.0/src/object_writer.rs:294:17 Caused by: failed to shutdown object writer for app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance: Generic LocalFileSystem error: Unable to access metadata for /app/server/storage/lancedb/dandd.lance/data/63e76822-ae91-4d70-96fa-ba45cdef0957.lance#1: No such file or directory (os error `2)`` ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:29:03 -05:00
yindo closed this issue 2026-02-22 18:29:03 -05:00
yindo commented 2026-02-22 18:29:03 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 31, 2025):

No such file or directory (os error 2)

Whatever database/storage folder you are using for storage was expected a lancedb table for dandd that was deleted and in browser it was still open. I am not sure if you have other users or mess with the storage folder directly, but that is why. Since you are on Docker local the storage folder is whatever you defined on container boot - which if you did not do would be ephemeral storage so if you had a browser open, then restarted the container with the same invalid start command your storage was deleted and would explain this issue.

https://docs.anythingllm.com/installation-docker/local-docker

I dont know what command you use to start the container, but that is a good place to start.

@timothycarambat commented on GitHub (Mar 31, 2025): >No such file or directory (os error 2) Whatever database/storage folder you are using for storage was expected a lancedb table for `dandd` that was deleted and in browser it was still open. I am not sure if you have other users or mess with the storage folder directly, but that is why. Since you are on Docker local the storage folder is whatever you defined on container boot - which if you did not do would be ephemeral storage so if you had a browser open, then restarted the container with the same invalid start command your storage was deleted and would explain this issue. https://docs.anythingllm.com/installation-docker/local-docker I dont know what command you use to start the container, but that is a good place to start.
yindo commented 2026-02-22 18:29:04 -05:00
Author
Owner
Copy Link
Copy Source

@DangerousBerries commented on GitHub (Mar 31, 2025):

No such file or directory (os error 2)

Whatever database/storage folder you are using for storage was expected a lancedb table for dandd that was deleted and in browser it was still open. I am not sure if you have other users or mess with the storage folder directly, but that is why. Since you are on Docker local the storage folder is whatever you defined on container boot - which if you did not do would be ephemeral storage so if you had a browser open, then restarted the container with the same invalid start command your storage was deleted and would explain this issue.

https://docs.anythingllm.com/installation-docker/local-docker

I dont know what command you use to start the container, but that is a good place to start.

I use the exact command from that link for Windows, i tried to delete the container and use that command again and this error still happens. This does not happen to every document, I think it only happens with long documents. This also happens when making new workspaces and trying to add the same documents. Do you have any other ideas?

Edit: This still happens if I delete the anythingllm storage folder and change the embedder, I use Ollama.
It's fixed if I change the Vector Database Provider from LanceDB to something else.

Edit 2: Increasing the text chunk size also fixes it, it seems to only fail above a number of chunks around 1000 and above?

@DangerousBerries commented on GitHub (Mar 31, 2025): > > No such file or directory (os error 2) > > Whatever database/storage folder you are using for storage was expected a lancedb table for `dandd` that was deleted and in browser it was still open. I am not sure if you have other users or mess with the storage folder directly, but that is why. Since you are on Docker local the storage folder is whatever you defined on container boot - which if you did not do would be ephemeral storage so if you had a browser open, then restarted the container with the same invalid start command your storage was deleted and would explain this issue. > > https://docs.anythingllm.com/installation-docker/local-docker > > I dont know what command you use to start the container, but that is a good place to start. I use the exact command from that link for Windows, i tried to delete the container and use that command again and this error still happens. This does not happen to every document, I think it only happens with long documents. This also happens when making new workspaces and trying to add the same documents. Do you have any other ideas? Edit: This still happens if I delete the anythingllm storage folder and change the embedder, I use Ollama. It's fixed if I change the Vector Database Provider from LanceDB to something else. Edit 2: Increasing the text chunk size also fixes it, it seems to only fail above a number of chunks around 1000 and above?
yindo commented 2026-02-22 18:29:04 -05:00
Author
Owner
Copy Link
Copy Source

@DangerousBerries commented on GitHub (Jun 28, 2025):

No such file or directory (os error 2)

Whatever database/storage folder you are using for storage was expected a lancedb table for dandd that was deleted and in browser it was still open. I am not sure if you have other users or mess with the storage folder directly, but that is why. Since you are on Docker local the storage folder is whatever you defined on container boot - which if you did not do would be ephemeral storage so if you had a browser open, then restarted the container with the same invalid start command your storage was deleted and would explain this issue.

https://docs.anythingllm.com/installation-docker/local-docker

I dont know what command you use to start the container, but that is a good place to start.

This problem is still here by the way, I'm using persistent storage and this doesn't happen with smaller documents or when increasing text chunk size.

This is my command:

$env:STORAGE_LOCATION="$HOME\Documents\anythingllm"; `
If(!(Test-Path $env:STORAGE_LOCATION)) {New-Item $env:STORAGE_LOCATION -ItemType Directory}; `
If(!(Test-Path "$env:STORAGE_LOCATION\.env")) {New-Item "$env:STORAGE_LOCATION\.env" -ItemType File}; `
docker run --name anythingllm --restart always -d -p 3001:3001 `
--cap-add SYS_ADMIN `
-v "$env:STORAGE_LOCATION`:/app/server/storage" `
-v "$env:STORAGE_LOCATION\.env:/app/server/.env" `
-e STORAGE_DIR="/app/server/storage" `
mintplexlabs/anythingllm;
@DangerousBerries commented on GitHub (Jun 28, 2025): > > No such file or directory (os error 2) > > Whatever database/storage folder you are using for storage was expected a lancedb table for `dandd` that was deleted and in browser it was still open. I am not sure if you have other users or mess with the storage folder directly, but that is why. Since you are on Docker local the storage folder is whatever you defined on container boot - which if you did not do would be ephemeral storage so if you had a browser open, then restarted the container with the same invalid start command your storage was deleted and would explain this issue. > > https://docs.anythingllm.com/installation-docker/local-docker > > I dont know what command you use to start the container, but that is a good place to start. This problem is still here by the way, I'm using persistent storage and this doesn't happen with smaller documents or when increasing text chunk size. This is my command: ```powershell $env:STORAGE_LOCATION="$HOME\Documents\anythingllm"; ` If(!(Test-Path $env:STORAGE_LOCATION)) {New-Item $env:STORAGE_LOCATION -ItemType Directory}; ` If(!(Test-Path "$env:STORAGE_LOCATION\.env")) {New-Item "$env:STORAGE_LOCATION\.env" -ItemType File}; ` docker run --name anythingllm --restart always -d -p 3001:3001 ` --cap-add SYS_ADMIN ` -v "$env:STORAGE_LOCATION`:/app/server/storage" ` -v "$env:STORAGE_LOCATION\.env:/app/server/.env" ` -e STORAGE_DIR="/app/server/storage" ` mintplexlabs/anythingllm; ```
yindo changed title from [BUG]: Documents are failing to vectorize. to [GH-ISSUE #3564] [BUG]: Documents are failing to vectorize. 2026-06-05 14:45:44 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2297
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?