Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5256] [BUG]: Data loss and breaking changes introduced by latest image update - encryption key lost, port changed without migration path #4988

New Issue
Closed
opened 2026-06-05 14:51:20 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 14:51:20 -04:00
Owner
Copy Link
Copy Source

Originally created by @monksofzero on GitHub (Mar 23, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5256

How are you running AnythingLLM?

Docker (local)

What happened?

(Description from Claude Sonnet 4.6 which was used for troubleshooting the related docker, anythingllm, ollama config problems)

Environment:

AnythingLLM Docker image: mintplexlabs/anythingllm:latest
Host OS: Windows 11
Docker Desktop on Windows
Previous working version: unknown (was using :latest)
Date of breakage: March 23, 2026

Description:
After restarting the AnythingLLM Docker container today, the :latest image silently pulled a new version that introduced several breaking changes with no migration path, resulting in complete loss of access to all previous chat history and workspaces.
Breaking changes observed:

Internal port changed from 8080 to 3001 with no warning or documentation, causing the container to appear broken when accessed at the previously working URL.
Encryption key not persisted reliably — on restart with the new image, the EncryptionManager logged Self-assigning key & salt instead of loading the existing key, making all previously stored chat data unreadable even though the database file (anythingllm.db) was intact and present.
Volume permissions changed — the new image introduced an EACCES: permission denied error on /app/data/models/context-windows that did not exist with the previous image, requiring user: "0:0" workaround to run at all.
Silent data reset — logs showed EmbeddingEngine changed from undefined to native and Vector configuration changed from undefined to lancedb, treating a previously configured installation as a fresh setup.

Steps to reproduce:

Run AnythingLLM using mintplexlabs/anythingllm:latest with a named Docker volume for storage
Stop the container with docker compose down
Restart with docker compose up — Docker pulls the new :latest image
Container fails to start due to permissions error
After resolving permissions, container starts but generates new encryption key, making all previous data unreadable

Expected behavior:

Breaking changes should never be introduced silently via a :latest tag update
The encryption key must be persisted reliably in the storage volume so data survives container restarts and image updates
If a port changes, there should be clear release notes and a migration path
The app should warn loudly if it cannot find an existing encryption key rather than silently generating a new one

Actual behavior:

All previous chat history and workspaces are permanently inaccessible
No warnings or errors clearly indicated data was at risk
Multiple hours of troubleshooting required just to get the container running again

Suggestion:

Store the encryption key explicitly in the /app/data storage directory so it survives image updates
Treat port changes as major version bumps with clear changelogs
Add a startup check that detects existing data and warns if encryption key is missing rather than silently regenerating it

Are there known steps to reproduce?

Steps to reproduce:

Run AnythingLLM using mintplexlabs/anythingllm:latest with a named Docker volume for storage
Stop the container with docker compose down
Restart with docker compose up — Docker pulls the new :latest image
Container fails to start due to permissions error
After resolving permissions, container starts but generates new encryption key, making all previous data unreadable

Reference issue #3813 (data loss on abnormal shutdown) and #4522 (encryption key self-assigning on Docker) as related issues

Originally created by @monksofzero on GitHub (Mar 23, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5256 ### How are you running AnythingLLM? Docker (local) ### What happened? (Description from Claude Sonnet 4.6 which was used for troubleshooting the related docker, anythingllm, ollama config problems) Environment: AnythingLLM Docker image: mintplexlabs/anythingllm:latest Host OS: Windows 11 Docker Desktop on Windows Previous working version: unknown (was using :latest) Date of breakage: March 23, 2026 Description: After restarting the AnythingLLM Docker container today, the :latest image silently pulled a new version that introduced several breaking changes with no migration path, resulting in complete loss of access to all previous chat history and workspaces. Breaking changes observed: Internal port changed from 8080 to 3001 with no warning or documentation, causing the container to appear broken when accessed at the previously working URL. Encryption key not persisted reliably — on restart with the new image, the EncryptionManager logged Self-assigning key & salt instead of loading the existing key, making all previously stored chat data unreadable even though the database file (anythingllm.db) was intact and present. Volume permissions changed — the new image introduced an EACCES: permission denied error on /app/data/models/context-windows that did not exist with the previous image, requiring user: "0:0" workaround to run at all. Silent data reset — logs showed EmbeddingEngine changed from undefined to native and Vector configuration changed from undefined to lancedb, treating a previously configured installation as a fresh setup. Steps to reproduce: Run AnythingLLM using mintplexlabs/anythingllm:latest with a named Docker volume for storage Stop the container with docker compose down Restart with docker compose up — Docker pulls the new :latest image Container fails to start due to permissions error After resolving permissions, container starts but generates new encryption key, making all previous data unreadable Expected behavior: Breaking changes should never be introduced silently via a :latest tag update The encryption key must be persisted reliably in the storage volume so data survives container restarts and image updates If a port changes, there should be clear release notes and a migration path The app should warn loudly if it cannot find an existing encryption key rather than silently generating a new one Actual behavior: All previous chat history and workspaces are permanently inaccessible No warnings or errors clearly indicated data was at risk Multiple hours of troubleshooting required just to get the container running again Suggestion: Store the encryption key explicitly in the /app/data storage directory so it survives image updates Treat port changes as major version bumps with clear changelogs Add a startup check that detects existing data and warns if encryption key is missing rather than silently regenerating it ### Are there known steps to reproduce? Steps to reproduce: Run AnythingLLM using mintplexlabs/anythingllm:latest with a named Docker volume for storage Stop the container with docker compose down Restart with docker compose up — Docker pulls the new :latest image Container fails to start due to permissions error After resolving permissions, container starts but generates new encryption key, making all previous data unreadable Reference issue #3813 (data loss on abnormal shutdown) and #4522 (encryption key self-assigning on Docker) as related issues
yindo added the possible bug label 2026-06-05 14:51:20 -04:00
yindo closed this issue 2026-06-05 14:51:20 -04:00
yindo commented 2026-06-05 14:51:21 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 23, 2026):

Internal port changed from 8080 to 3001 with no warning or documentation, causing the container to appear broken when accessed at the previously working URL.

Not true, and in fact is a docker config you can set yourself. Not sure where you got that information.

Encryption key not persisted reliably — on restart with the new image, the EncryptionManager logged Self-assigning key & salt instead of loading the existing key, making all previously stored chat data unreadable even though the database file (anythingllm.db) was intact and present.

The chat data is not encrypted and rolling the key is normal. It is only used for communicate payload signing between collector<>server so the collector only responds to real requests.

Volume permissions changed — the new image introduced an EACCES: permission denied error on /app/data/models/context-windows that did not exist with the previous image, requiring user: "0:0" workaround to run at all.

Again, a docker permission. Sounds like you are trying to run two containers at the same time or something - not somehting with the app because we dont control that. Also the context-window finder folder has been a thing for about 10 months now - so it is not new.

Silent data reset — logs showed EmbeddingEngine changed from undefined to native and Vector configuration changed from undefined to lancedb, treating a previously configured installation as a fresh setup.

The default is lancedb - there is no undefined option. If you were on a fresh setup it would still be lance anyway, not undefined.

How are you running the container? Something is clearly wrong on the user side and whatever that issue is, is the cause of your issues because it seems like the container or user trying to run the container does not have access to the storage folder thus resulting in all of your issues.

  • Ensure your user can run docker containers
  • Make sure you have read/write permissions on whatever your are binding the storage folder too for the container

Any logs would also help as well. All of this is just a docker config issue though - not code related.

<!-- gh-comment-id:4115197785 --> @timothycarambat commented on GitHub (Mar 23, 2026): > Internal port changed from 8080 to 3001 with no warning or documentation, causing the container to appear broken when accessed at the previously working URL. Not true, and in fact is a docker config you can set yourself. Not sure where you got that information. > Encryption key not persisted reliably — on restart with the new image, the EncryptionManager logged Self-assigning key & salt instead of loading the existing key, making all previously stored chat data unreadable even though the database file (anythingllm.db) was intact and present. The chat data is not encrypted and rolling the key is normal. It is only used for communicate payload signing between collector<>server so the collector only responds to real requests. > Volume permissions changed — the new image introduced an EACCES: permission denied error on /app/data/models/context-windows that did not exist with the previous image, requiring user: "0:0" workaround to run at all. Again, a docker permission. Sounds like you are trying to run two containers at the same time or something - not somehting with the app because we dont control that. Also the context-window finder folder has been a thing for about 10 months now - so it is not new. > Silent data reset — logs showed EmbeddingEngine changed from undefined to native and Vector configuration changed from undefined to lancedb, treating a previously configured installation as a fresh setup. The default **is** lancedb - there is no undefined option. If you were on a fresh setup it would still be lance anyway, not undefined. How are you running the container? Something is clearly wrong on the user side and whatever that issue is, is the cause of your issues because it seems like the container or user trying to run the container does not have access to the storage folder thus resulting in all of your issues. - Ensure your user can run docker containers - Make sure you have read/write permissions on whatever your are binding the storage folder too for the container Any logs would also help as well. All of this is just a docker config issue though - not code related.
yindo commented 2026-06-05 14:51:21 -04:00
Author
Owner
Copy Link
Copy Source

@monksofzero commented on GitHub (Apr 7, 2026):

Thanks. I agree: I think it was a multitude of issues but mostly with
docker container configuration. I reconfigured the docker compose file and
have re-connected to the database that I thought I'd lost.

On Mon, Mar 23, 2026 at 8:45 PM Timothy Carambat @.***>
wrote:

timothycarambat left a comment (Mintplex-Labs/anything-llm#5256)
https://github.com/Mintplex-Labs/anything-llm/issues/5256#issuecomment-4115197785

Internal port changed from 8080 to 3001 with no warning or documentation,
causing the container to appear broken when accessed at the previously
working URL.

Not true, and in fact is a docker config you can set yourself. Not sure
where you got that information.

Encryption key not persisted reliably — on restart with the new image, the
EncryptionManager logged Self-assigning key & salt instead of loading the
existing key, making all previously stored chat data unreadable even though
the database file (anythingllm.db) was intact and present.

The chat data is not encrypted and rolling the key is normal. It is only
used for communicate payload signing between collector<>server so the
collector only responds to real requests.

Volume permissions changed — the new image introduced an EACCES:
permission denied error on /app/data/models/context-windows that did not
exist with the previous image, requiring user: "0:0" workaround to run at
all.

Again, a docker permission. Sounds like you are trying to run two
containers at the same time or something - not somehting with the app
because we dont control that. Also the context-window finder folder has
been a thing for about 10 months now - so it is not new.

Silent data reset — logs showed EmbeddingEngine changed from undefined to
native and Vector configuration changed from undefined to lancedb, treating
a previously configured installation as a fresh setup.

The default is lancedb - there is no undefined option. If you were on a
fresh setup it would still be lance anyway, not undefined.

How are you running the container? Something is clearly wrong on the user
side and whatever that issue is, is the cause of your issues because it
seems like the container or user trying to run the container does not have
access to the storage folder thus resulting in all of your issues.

  • Ensure your user can run docker containers
  • Make sure you have read/write permissions on whatever your are
    binding the storage folder too for the container

Any logs would also help as well. All of this is just a docker config
issue though - not code related.

—
Reply to this email directly, view it on GitHub
https://github.com/Mintplex-Labs/anything-llm/issues/5256?email_source=notifications&email_token=BFK3CZAYAPOI3YTQACZN7L34SIAGFA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJRGUYTSNZXHA22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4115197785,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BFK3CZB5TPNES2GB6RT74VT4SIAGFAVCNFSM6AAAAACW4XGF3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCMJVGE4TONZYGU
.
You are receiving this because you authored the thread.Message ID:
@.***>

<!-- gh-comment-id:4202780443 --> @monksofzero commented on GitHub (Apr 7, 2026): Thanks. I agree: I think it was a multitude of issues but mostly with docker container configuration. I reconfigured the docker compose file and have re-connected to the database that I thought I'd lost. On Mon, Mar 23, 2026 at 8:45 PM Timothy Carambat ***@***.***> wrote: > *timothycarambat* left a comment (Mintplex-Labs/anything-llm#5256) > <https://github.com/Mintplex-Labs/anything-llm/issues/5256#issuecomment-4115197785> > > Internal port changed from 8080 to 3001 with no warning or documentation, > causing the container to appear broken when accessed at the previously > working URL. > > Not true, and in fact is a docker config you can set yourself. Not sure > where you got that information. > > Encryption key not persisted reliably — on restart with the new image, the > EncryptionManager logged Self-assigning key & salt instead of loading the > existing key, making all previously stored chat data unreadable even though > the database file (anythingllm.db) was intact and present. > > The chat data is not encrypted and rolling the key is normal. It is only > used for communicate payload signing between collector<>server so the > collector only responds to real requests. > > Volume permissions changed — the new image introduced an EACCES: > permission denied error on /app/data/models/context-windows that did not > exist with the previous image, requiring user: "0:0" workaround to run at > all. > > Again, a docker permission. Sounds like you are trying to run two > containers at the same time or something - not somehting with the app > because we dont control that. Also the context-window finder folder has > been a thing for about 10 months now - so it is not new. > > Silent data reset — logs showed EmbeddingEngine changed from undefined to > native and Vector configuration changed from undefined to lancedb, treating > a previously configured installation as a fresh setup. > > The default *is* lancedb - there is no undefined option. If you were on a > fresh setup it would still be lance anyway, not undefined. > > How are you running the container? Something is clearly wrong on the user > side and whatever that issue is, is the cause of your issues because it > seems like the container or user trying to run the container does not have > access to the storage folder thus resulting in all of your issues. > > - Ensure your user can run docker containers > - Make sure you have read/write permissions on whatever your are > binding the storage folder too for the container > > Any logs would also help as well. All of this is just a docker config > issue though - not code related. > > — > Reply to this email directly, view it on GitHub > <https://github.com/Mintplex-Labs/anything-llm/issues/5256?email_source=notifications&email_token=BFK3CZAYAPOI3YTQACZN7L34SIAGFA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJRGUYTSNZXHA22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4115197785>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/BFK3CZB5TPNES2GB6RT74VT4SIAGFAVCNFSM6AAAAACW4XGF3GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCMJVGE4TONZYGU> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#4988
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?