mirror of
https://github.com/ollama/ollama-python.git
synced 2026-07-21 09:05:23 -04:00
API Method to get model loading progress? #219
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @BrainSlugs83 on GitHub (Feb 7, 2025).
Some models can take a substantial amount of time to load into memory.
Can you give us a method that will load a model into memory, and return a stream of progress results, so that we can show a progress bar of the model loading? (similar to what
/api/pulldoes today, but instead of being a status report of how much has been downloaded, it's a status report of what percentage of the model has been loaded into memory?)Right now we have to just do a text completion, and wait for a response, it could be minutes away, with no feedback to the user of how much progress has been made (if any) of the model being loaded into memory.
@ParthSareen commented on GitHub (Feb 13, 2025):
I don't think this is something we can do just yet as the model loading is dependent on many different factors like hardware, loading params, etc. vs just network and known chunks of the model.
Depending on the use case you can have strategies to pre-load the model or estimate with your hardware how long it takes
@Marcussacapuces91 commented on GitHub (Feb 19, 2025):
Hi,
Maybe a "python"
resp = ollama.generate("deepseek-r1:70b", stream=True)can produce arespindicating the among of bytes loaded and the full size of the model ?@leobenkel commented on GitHub (Aug 21, 2025):
I'd really love an endpoint to get the % of the model loaded in memory too
@BrainSlugs83 commented on GitHub (Sep 10, 2025):
@ParthSareen -- that's the point though... like you don't know how LONG it will take, but you can tell me how much has been loaded so far right? Give me back a % or a total number of bytes, or something that I can show the user so they don't think the app is hanging.