Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2189] [BUG]: Download instability >10 GB model. Repeat fail to complete. Desktop app lock up but not frozen until download completed.. #1427

New Issue
Closed
opened 2026-02-22 18:24:44 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:24:44 -05:00
Owner
Copy Link
Copy Source

Originally created by @CamAust on GitHub (Aug 28, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2189

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

I have been successfully trailing using the Desktop Anything LLM v1.6.4, (OSX version).

  • On Macbook Pro intel 32GB Ram, OSX 14.5 (Sonoma).
  • In a rural area with more limited 3 to 5MB/sec wireless internet connection, so downloads of large files takes time. (This may be significant to this issue.)

Have down loaded and used successfully two smaller LLM's under 5GB. (Mistral7B 4.1GB, and Codellama7B 3.8GB)

  • Mistral7B with an initial trail of pdf docs. Success.
  • Codellama with an initial code trial. Success.

Then, a few days later, I tried to download, and activate a larger code focused LLM model - Dolphin Mixtral 26 GB.

Two issues arise:

Issue 1:

Several attempts to download failed. The download would slow at around 10% to 25%, then cease to download. About three attempts.

Various causes were tested, eg:

  • Laptop sleep interrupting the download, or screen saver, (change OSX settings).
  • VPN issue? (Turn off - seemed to improve situation).

Watching the process, the % counter as download slowed, would slow, then % counter would decrease, go backwards. This to me suggesting of either a corruption of the download packets and having to reload them, or a coding issue with the downloading component of the app in the case of slower more unreliable data connections.

With each failure, I would quit Anything LLM app and restart it. With this, the download would again restart to load that LLM.

With the final step of turning off the VPN, and hence more direct download route, and OSX settings re sleep etc on max (should not matter I know...), I managed to get to 38% download of this larger LLM. But again then this same behaviour.

Screenshot 2024-08-27 at 6 29 24 PM

Unfortunately these repeat partial downloads was maxing out my data for the month. So have had to stop for now. Can do a retrial if I pay $5 for unlimited data for a day. So if solution proposed in advance, can test implementation.

Issue 2:

I have sought to return to use Anything LLM with the two successfully down loaded models. However, now, even after re installing the Anything LLM app, I can not use any of the other models. I get an error. Status code 404, insisting on this larger LLM download that needs to be completed. .... "dolphin-mixtral:latest" not found, try pulling it first.

Screenshot 2024-08-28 at 9 53 28 PM

Now neither of the two originally set up and working workspaces, with the previous two LLM model downloads assigned can be used or activated, as the app seems locked into this download being completed.

This occurs despite ensuring a fresh set up to these workspaces, with the original smaller LLM (not this dolphin-mixtral LLM) assigned. And other settings checked.

I have tried deleting both the original working workspaces, quitting the app and restarting, and setting up a new workspace seeking to use one of the original LLM's successfully downloaded. Also reinstalling the app in the application folder. These actions do not resolve the problem. So currently I can not use the desktop app at all. The app is not frozen, but it is 'locked up'.

I hesitate to do a full uninstall of Anything LLM, including deleting the .ollama file in the user home account. The reason is to avoid needing to down load the two original smaller LLM's.

Fortunately, I do not have significant work to lose as I was doing only an initial explore of the app and functionality, and the LLM options, and trialing use. So that is good re working from here to resolve this issue.

Problem solving my end:

I have played with settings in the app in the process of learning how to use Anything LLM. Perhaps one combination has caused this issue. Setting were not changed while the attempted downloads were happening of the dolphin LLM.

However, I note a full delete of current workspaces and resetting up and full restart of the app does not resolve this lock up. This includes checking the previous successful LLM's were selected. Hence, I think the issue/bug may be dependent on the User account library or .ollma files associated with this app.

I note with this download issue, the .ollama file does not show any download of the larger dolphin-mixtral LLM or any file of the date and time when the download was active. Even when up to 36%.

I had downloaded the two successful models in a previous use in days before, and those large files are evidently stored here, with dates reflecting the time of successful download. There is no indication of this larger LLM being stored, even incompletely. No new files with the day date of the failed download attempts.

Screenshot 2024-08-28 at 10 42 22 PM

Perhaps the downloading LLM does not save until the LLM is fully downloaded and confirmed in RAM, then written into this folder. With no other major RAM use and 32 GB RAM, I do not see a load limit issue likely given the download did not get past 36%. Further I checked OSX Activity Monitor, CPU load and RAM usage, during later failed downloads and there is no indication of any limits being reached. So this seems excluded.

The Anything LLM folder in Library Application Support is complex. I could remove this folder, having deleted all current work spaces, and quit the application, and then re-open the app.

I have registered an account on discord, reviewed comments, and looked here at past bugs lodged. It seems best to raise this issue here.

Are there known steps to reproduce?

This issue may be specific to this dolphin LLM model source file from the download server. Trial a download.

If the issue is rather poor handling of the download function of the anything LLM desktop app with poorer slower internet connections, then it is more difficult to reproduce. I can only suggest, download a larger >20 GB LLM (or specifically the the dolphin-mixtral LLM) and interrupt the download by disconnecting from the internet for a short moment during the process a few times, to see how the app manages this interruption.

Originally created by @CamAust on GitHub (Aug 28, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2189 ### How are you running AnythingLLM? AnythingLLM desktop app ### What happened? I have been successfully trailing using the Desktop Anything LLM v1.6.4, (OSX version). - On Macbook Pro intel 32GB Ram, OSX 14.5 (Sonoma). - In a rural area with more limited 3 to 5MB/sec wireless internet connection, so downloads of large files takes time. (This may be significant to this issue.) Have down loaded and used successfully two smaller LLM's under 5GB. (Mistral7B 4.1GB, and Codellama7B 3.8GB) - Mistral7B with an initial trail of pdf docs. Success. - Codellama with an initial code trial. Success. Then, a few days later, I tried to download, and activate a larger code focused LLM model - Dolphin Mixtral 26 GB. **Two issues arise:** **Issue 1:** Several attempts to download failed. The download would slow at around 10% to 25%, then cease to download. About three attempts. Various causes were tested, eg: - Laptop sleep interrupting the download, or screen saver, (change OSX settings). - VPN issue? (Turn off - seemed to improve situation). Watching the process, the % counter as download slowed, would slow, then % counter would decrease, go backwards. This to me suggesting of either a corruption of the download packets and having to reload them, or a coding issue with the downloading component of the app in the case of slower more unreliable data connections. With each failure, I would quit Anything LLM app and restart it. With this, the download would again restart to load that LLM. With the final step of turning off the VPN, and hence more direct download route, and OSX settings re sleep etc on max (should not matter I know...), I managed to get to 38% download of this larger LLM. But again then this same behaviour. <img width="292" alt="Screenshot 2024-08-27 at 6 29 24 PM" src="https://github.com/user-attachments/assets/d1944951-a255-4ade-92a0-b60f704fe949"> Unfortunately these repeat partial downloads was maxing out my data for the month. So have had to stop for now. Can do a retrial if I pay $5 for unlimited data for a day. So if solution proposed in advance, can test implementation. **Issue 2:** I have sought to return to use Anything LLM with the two successfully down loaded models. However, now, even after re installing the Anything LLM app, I can not use any of the other models. I get an error. Status code 404, insisting on this larger LLM download that needs to be completed. .... "dolphin-mixtral:latest" not found, try pulling it first. <img width="1189" alt="Screenshot 2024-08-28 at 9 53 28 PM" src="https://github.com/user-attachments/assets/e0fb3325-686c-4b21-b29b-e02b4fcfd908"> Now neither of the two originally set up and working workspaces, with the previous two LLM model downloads assigned can be used or activated, as the app seems locked into this download being completed. This occurs despite ensuring a fresh set up to these workspaces, with the original smaller LLM (not this dolphin-mixtral LLM) assigned. And other settings checked. I have tried deleting both the original working workspaces, quitting the app and restarting, and setting up a new workspace seeking to use one of the original LLM's successfully downloaded. Also reinstalling the app in the application folder. These actions do not resolve the problem. So currently I can not use the desktop app at all. The app is not frozen, but it is 'locked up'. I hesitate to do a full uninstall of Anything LLM, including deleting the .ollama file in the user home account. The reason is to avoid needing to down load the two original smaller LLM's. Fortunately, I do not have significant work to lose as I was doing only an initial explore of the app and functionality, and the LLM options, and trialing use. So that is good re working from here to resolve this issue. **Problem solving my end:** I have played with settings in the app in the process of learning how to use Anything LLM. Perhaps one combination has caused this issue. Setting were not changed while the attempted downloads were happening of the dolphin LLM. However, I note a full delete of current workspaces and resetting up and full restart of the app does not resolve this lock up. This includes checking the previous successful LLM's were selected. Hence, I think the issue/bug may be dependent on the User account library or .ollma files associated with this app. I note with this download issue, the .ollama file does not show any download of the larger dolphin-mixtral LLM or any file of the date and time when the download was active. Even when up to 36%. I had downloaded the two successful models in a previous use in days before, and those large files are evidently stored here, with dates reflecting the time of successful download. There is no indication of this larger LLM being stored, even incompletely. No new files with the day date of the failed download attempts. <img width="773" alt="Screenshot 2024-08-28 at 10 42 22 PM" src="https://github.com/user-attachments/assets/4ace486b-ae40-4144-9a4b-4783106977a0"> Perhaps the downloading LLM does not save until the LLM is fully downloaded and confirmed in RAM, then written into this folder. With no other major RAM use and 32 GB RAM, I do not see a load limit issue likely given the download did not get past 36%. Further I checked OSX Activity Monitor, CPU load and RAM usage, during later failed downloads and there is no indication of any limits being reached. So this seems excluded. The Anything LLM folder in Library Application Support is complex. I could remove this folder, having deleted all current work spaces, and quit the application, and then re-open the app. I have registered an account on discord, reviewed comments, and looked here at past bugs lodged. It seems best to raise this issue here. ### Are there known steps to reproduce? This issue may be specific to this dolphin LLM model source file from the download server. Trial a download. If the issue is rather poor handling of the download function of the anything LLM desktop app with poorer slower internet connections, then it is more difficult to reproduce. I can only suggest, download a larger >20 GB LLM (or specifically the the dolphin-mixtral LLM) and interrupt the download by disconnecting from the internet for a short moment during the process a few times, to see how the app manages this interruption.
yindo added the possible bug label 2026-02-22 18:24:44 -05:00
yindo closed this issue 2026-02-22 18:24:44 -05:00
yindo commented 2026-02-22 18:24:45 -05:00
Author
Owner
Copy Link
Copy Source

@CamAust commented on GitHub (Aug 28, 2024):

Add: The .llama folder may, on reflection, be from downloading a terminal use version of ollama, and not be associated with the AnythingLLM app. I had done this in the early days while also trialing this GUI app option. However I do not think this caused a conflict with the app version. I could be wrong here.

@CamAust commented on GitHub (Aug 28, 2024): Add: The .llama folder may, on reflection, be from downloading a terminal use version of ollama, and not be associated with the AnythingLLM app. I had done this in the early days while also trialing this GUI app option. However I do not think this caused a conflict with the app version. I could be wrong here.
yindo commented 2026-02-22 18:24:45 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 28, 2024):

Correct, so for clarification:
.ollama in $HOME is the models downloaded from the terminal ollama you have installed.
the ollama folder in AnythingLLM storage is where our models and ollama version live.

However, we pull models from the same place as ollama at registry.ollama.com, which can be very slow under load and also on slow connections. The way ollama installs models is by layer and I'm willing to bet the slow connection/disconnect lead to a layerfile corruption on the download of Dolphin.

That being said, if you want to use the models you did successfully download all you need to do is swap your model selection back to a working model. All downloaded models have a button on them that says "uninstall" - indicating they are installed. Select a model and click "save changes". Now you can chat again.

Screenshot 2024-08-28 at 10 36 34 AM

The error you have is because your model is still selected as Dolphin, but failed to download for any number of reasons. its not ready, so it can't be used.

A possible solution for all of this is instead of using the built-in Ollama in AnythingLLM, just connect your terminal Ollama running on your computer (screenshot shows it is alive in top taskbar). Then you can ollama pull whatever you need and have the full controls available for you.

Let me know in this issue if you have more questions. The limited connection bandwidth for sure is a big hurdle. You dont need to blow away your whole already downloaded models, just the problematic files - which ollama does a great job of obfuscating due to only using SHAs 😵

@timothycarambat commented on GitHub (Aug 28, 2024): Correct, so for clarification: `.ollama` in `$HOME` is the models downloaded from the terminal ollama you have installed. the `ollama` folder in [AnythingLLM storage](https://docs.anythingllm.com/installation/desktop/general#where-is-my-data-located) is where our models and ollama version live. However, we pull models from the same place as ollama at registry.ollama.com, which can be _very slow_ under load and also on slow connections. The way ollama installs models is by layer and I'm willing to bet the slow connection/disconnect lead to a layerfile corruption on the download of Dolphin. That being said, if you want to use the models you did successfully download all you need to do is swap your model selection back to a working model. All downloaded models have a button on them that says "uninstall" - indicating they are installed. Select a model and click "save changes". Now you can chat again. <img width="1485" alt="Screenshot 2024-08-28 at 10 36 34 AM" src="https://github.com/user-attachments/assets/7c0d7288-7ed7-4cb5-8a62-d8fedb8dc7e6"> The error you have is because your model is still selected as Dolphin, but failed to download for any number of reasons. its not ready, so it can't be used. A possible solution for all of this is instead of using the built-in Ollama in AnythingLLM, just connect your terminal Ollama running on your computer (screenshot shows it is alive in top taskbar). Then you can `ollama pull` whatever you need and have the full controls available for you. Let me know in this issue if you have more questions. The limited connection bandwidth for sure is a big hurdle. You dont need to blow away your whole already downloaded models, just the problematic files - which ollama does a great job of obfuscating due to only using SHAs 😵
yindo changed title from [BUG]: Download instability >10 GB model. Repeat fail to complete. Desktop app lock up but not frozen until download completed.. to [GH-ISSUE #2189] [BUG]: Download instability >10 GB model. Repeat fail to complete. Desktop app lock up but not frozen until download completed.. 2026-06-05 14:40:42 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1427
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?