Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5436] Desktop: bundled Ollama model pull hangs silently — stale bundled + pullModel drops all non-progress responses #5092

New Issue
Closed
opened 2026-06-05 14:51:57 -04:00 by yindo · 2 comments
yindo commented 2026-06-05 14:51:57 -04:00
Owner
Copy Link
Copy Source

Originally created by @shdjfensjfsns on GitHub (Apr 14, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5436

TL;DR

On AnythingLLM Desktop, pulling any modern model via the built-in "AnythingLLM" (bundled Ollama) provider hangs forever with no progress, no error, no timeout. Two stacked bugs:

  1. Bundled Ollama is stale (v0.13-era) and the registry rejects modern manifests with HTTP 412.
  2. AnythingLLMOllama.pullModel silently swallows the 412 (and every other non-progress response) because of four separate issues in the stream handler.

Locally patched both. Pull now works end-to-end. Full diagnosis, repro, and proposed source fix below.

Environment

  • AnythingLLM Desktop v1.12.0
  • macOS (Apple Silicon), but the bug is OS-independent — it's in JS + bundled binary
  • Provider: AnythingLLM (bundled Ollama, not user-installed Ollama)

Reproduction

  1. Open AnythingLLM Desktop → Settings → AI Providers → LLM
  2. Select the AnythingLLM provider
  3. In the import box, type any modern model tag (example used: gemma4:26b)
  4. Click Import

Expected: progress bar, download, "model ready".
Actual: UI sits on "Starting pull of model tag…" indefinitely. Backend log shows exactly one line and nothing after it:

{"level":"info","message":"[AnythingLLMOllama] Starting pull of model tag 
\"gemma4:26b\".","service":"backend"}

No progress events. No error. Waiting 30+ minutes changes nothing.

Root cause

Bug 1 — bundled Ollama binary is too old

$ /Applications/AnythingLLM.app/Contents/Resources/ollama/llm --version ollama version is 0.20.3                                                      
Warning: client version is 0.13.0

Current Ollama release is v0.20.7. The 0.13.0 client inside the Desktop bundle predates modern model manifest formats. When asked to pull a current model, the registry returns:

HTTP 412
pull model manifest: 412: The model you are attempting to pull requires a newer version of Ollama. Please download the latest version at: https://ollama.com/download

Bug 2 — AnythingLLMOllama.pullModel drops every non-progress response

Found in the minified Desktop bundle at Contents/Resources/backend/server.js. De-minified, the current
implementation is:

async pullModel(tag, onProgress, onSuccess, onError) {
  await this.bootOrContinue();                                                
  this.log(`Starting pull of model tag "${tag}".`);
  const reader = (await fetch(`${this.basePath()}/api/pull`, {                
    method: "POST",                                                           
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ name: tag, stream: true }),                        
  })).body.getReader();                                                     
                                                                              
  for (;;) {
    const { done, value } = await reader.read();                              
    if (done) break;                                                        
    const chunk = new TextDecoder("utf-8").decode(value);
    try {                                                                     
      const m = safeJsonParse(chunk);
      if (m?.status && m?.total && m?.completed) {                            
        const p = Math.round(m.completed / m.total * 100);                  
        onProgress?.(p, m.status);                                            
      }
      if (m?.status === "success") onSuccess?.();                             
    } catch (e) {                                                           
      console.error("Error parsing JSON", e);                                 
      onError?.(e.message);
    }                                                                         
  }                                                                         
}

Four issues, any one of which is sufficient to produce the hang:

  1. No response.ok check. HTTP 4xx/5xx bodies are consumed as if they were success streams. A 412 with a single JSON error line reaches the parse block, doesn't match the progress guard, the reader hits done=true, and the loop exits without calling any callback. UI is left on its initial "Starting pull…" state forever.
  2. No NDJSON line buffering. /api/pull streams newline-delimited JSON. A raw decoded chunk commonly contains multiple objects or a split object, so JSON.parse(chunk) throws and the entire chunk (including any progress info inside it) is discarded. Compare with createModel a few lines below in the
    same file, which at least handles Array.isArray(parsed) — pullModel does not.
  3. Progress guard is too strict. m?.status && m?.total && m?.completed skips early events like {"status":"pulling manifest"}, {"status":"verifying sha256 digest"}, {"status":"writing manifest"}. None of these have total/completed, so the UI sees nothing for the first several seconds even when parsing succeeds.
  4. No m.error handling. Ollama surfaces errors as {"error": "..."} lines with no status field. The current code has no branch for this — errors are silently dropped.

Net effect: on any non-progress response (HTTP error, error line, or multi-object chunk), pullModel runs to stream completion without firing onSuccess, onProgress, or onError. The frontend sits forever on the initial log line with no signal that anything went wrong.

Proposed fix

(a) Bump bundled Ollama to current stable

Update whatever build script downloads the Ollama binary during Desktop packaging. Current stable is v0.20.7 (https://github.com/ollama/ollama/releases/download/v0.20.7/ollama-darwin.tgz for macOS). v0.20.7 layout is a drop-in replacement for v0.20.3 — same libggml-* files, plus new mlx_metal_v3/ and mlx_metal_v4/ subdirectories that modern
Metal-accelerated models require.

(b) Rewrite pullModel to handle the Ollama stream correctly

async pullModel(modelTag = "llama2", onProgress, onSuccess, onError) {      
  await this.bootOrContinue();               
  this.log(`Starting pull of model tag "${modelTag}".`);
                                                                              
  const response = await fetch(`${this.basePath()}/api/pull`, {
    method: "POST",                                                           
    headers: { "Content-Type": "application/json" },                        
    body: JSON.stringify({ name: modelTag, stream: true }),                   
  });
                                                                              
  if (!response.ok) {                                                       
    let text = "";                           
    try { text = await response.text(); } catch {}
    this.log(`Pull failed with HTTP ${response.status}: ${text}`);            
    onError?.(`HTTP ${response.status}: ${text || response.statusText}`);
    return;                                                                   
  }                                                                         
                                                                              
  const reader = response.body.getReader();                                 
  const decoder = new TextDecoder("utf-8");  
  let buffer = "";

  for (;;) {                                                                  
    const { done, value } = await reader.read();
    if (done) {                                                               
      if (buffer.trim()) {                                                  
        try {                                
          const msg = JSON.parse(buffer);
          if (msg?.error) onError?.(msg.error);
          else if (msg?.status === "success") onSuccess?.();                  
        } catch {}
      }                                                                       
      break;                                                                
    }                                        

    buffer += decoder.decode(value, { stream: true });                        
    const lines = buffer.split("\n");
    buffer = lines.pop();                                                     
                                                                            
    for (const line of lines) {              
      if (!line.trim()) continue;
      let msg;                                                                
      try {
        msg = JSON.parse(line);                                               
      } catch (err) {                                                       
        console.error("Failed to parse Ollama pull line:", line, err);
        continue;
      }                                                                       

      if (msg?.error) {                                                       
        this.log(`Pull error: ${msg.error}`);                               
        onError?.(msg.error);                
        return;
      }

      if (msg?.status) {                                                      
        if (msg.total && msg.completed != null) {
          const percent = Math.round((msg.completed / msg.total) * 100);      
          this.log(`Pull ${msg.status} ${percent}%`);                       
          onProgress?.(percent, msg.status);                                  
        } else {
          this.log(`Pull ${msg.status}`);                                     
          onProgress?.(0, msg.status);                                      
        }                                                                     
      }
                                                                              
      if (msg?.status === "success") onSuccess?.();                         
    }                                        
  }
}

Key differences vs. the current implementation:

  • Checks response.ok and surfaces HTTP errors via onError.
  • Accumulates decoded bytes in buffer, splits on \n, parses each complete line individually, preserves trailing partial line for next chunk — correct NDJSON handling.
  • Relaxes the progress guard to any msg.status, so early lifecycle events appear in the UI.
  • Handles {"error": "..."} lines from Ollama explicitly.
  • decoder.decode(value, { stream: true }) to correctly handle multibyte characters split across chunks.

Verification

I applied both fixes locally (replaced Contents/Resources/ollama/ with v0.20.7 and rewrote pullModel in the installed Desktop bundle). Same repro as above now produces live progress events and the pull completes successfully:

Backend log after the patch shows the full lifecycle:

[AnythingLLMOllama] Starting pull of model tag "gemma4:26b".                
[AnythingLLMOllama] Pull pulling manifest                                     
[AnythingLLMOllama] Pull pulling <digest> 1%                                  
[AnythingLLMOllama] Pull pulling <digest> 2%                                  
...                                                                           
[AnythingLLMOllama] Pull verifying sha256 digest                              
[AnythingLLMOllama] Pull writing manifest                                     
[AnythingLLMOllama] Pull success

Happy to open a PR against the Desktop source repo if a maintainer can point me at it — the AnythingLLMOllama class and the bundled-binary build step don't appear to be in this repo, so I'm filing here as an issue.

Severity note

This isn't just about gemma4:26b — it's any model the bundled Ollama can't fetch (stale client, typo'd tag, network failure, registry 5xx, etc.). All of them hit the same silent-hang path with zero error surfaced to the user. That's a pretty bad first-run experience for anyone trying the bundled provider.

Image
Originally created by @shdjfensjfsns on GitHub (Apr 14, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5436 ## TL;DR On AnythingLLM Desktop, pulling any modern model via the built-in "AnythingLLM" (bundled Ollama) provider hangs forever with no progress, no error, no timeout. Two stacked bugs: 1. **Bundled Ollama is stale** (v0.13-era) and the registry rejects modern manifests with HTTP 412. 2. **`AnythingLLMOllama.pullModel` silently swallows the 412** (and every other non-progress response) because of four separate issues in the stream handler. Locally patched both. Pull now works end-to-end. Full diagnosis, repro, and proposed source fix below. --- ## Environment - AnythingLLM Desktop **v1.12.0** - macOS (Apple Silicon), but the bug is OS-independent — it's in JS + bundled binary - Provider: **AnythingLLM** (bundled Ollama, not user-installed Ollama) --- ## Reproduction 1. Open AnythingLLM Desktop → **Settings → AI Providers → LLM** 2. Select the **AnythingLLM** provider 3. In the import box, type any modern model tag (example used: `gemma4:26b`) 4. Click **Import** **Expected:** progress bar, download, "model ready". **Actual:** UI sits on "Starting pull of model tag…" indefinitely. Backend log shows exactly one line and nothing after it: ``` {"level":"info","message":"[AnythingLLMOllama] Starting pull of model tag \"gemma4:26b\".","service":"backend"} ``` No progress events. No error. Waiting 30+ minutes changes nothing. --- ## Root cause ### Bug 1 — bundled Ollama binary is too old ``` $ /Applications/AnythingLLM.app/Contents/Resources/ollama/llm --version ollama version is 0.20.3 Warning: client version is 0.13.0 ``` Current Ollama release is **v0.20.7**. The `0.13.0` client inside the Desktop bundle predates modern model manifest formats. When asked to pull a current model, the registry returns: ``` HTTP 412 pull model manifest: 412: The model you are attempting to pull requires a newer version of Ollama. Please download the latest version at: https://ollama.com/download ``` ### Bug 2 — `AnythingLLMOllama.pullModel` drops every non-progress response Found in the minified Desktop bundle at `Contents/Resources/backend/server.js`. De-minified, the current implementation is: ```js async pullModel(tag, onProgress, onSuccess, onError) { await this.bootOrContinue(); this.log(`Starting pull of model tag "${tag}".`); const reader = (await fetch(`${this.basePath()}/api/pull`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ name: tag, stream: true }), })).body.getReader(); for (;;) { const { done, value } = await reader.read(); if (done) break; const chunk = new TextDecoder("utf-8").decode(value); try { const m = safeJsonParse(chunk); if (m?.status && m?.total && m?.completed) { const p = Math.round(m.completed / m.total * 100); onProgress?.(p, m.status); } if (m?.status === "success") onSuccess?.(); } catch (e) { console.error("Error parsing JSON", e); onError?.(e.message); } } } ``` Four issues, any one of which is sufficient to produce the hang: 1. **No `response.ok` check.** HTTP 4xx/5xx bodies are consumed as if they were success streams. A 412 with a single JSON error line reaches the parse block, doesn't match the progress guard, the reader hits `done=true`, and the loop exits without calling any callback. UI is left on its initial "Starting pull…" state forever. 2. **No NDJSON line buffering.** `/api/pull` streams newline-delimited JSON. A raw decoded chunk commonly contains multiple objects or a split object, so `JSON.parse(chunk)` throws and the entire chunk (including any progress info inside it) is discarded. Compare with `createModel` a few lines below in the same file, which at least handles `Array.isArray(parsed)` — `pullModel` does not. 3. **Progress guard is too strict.** `m?.status && m?.total && m?.completed` skips early events like `{"status":"pulling manifest"}`, `{"status":"verifying sha256 digest"}`, `{"status":"writing manifest"}`. None of these have `total`/`completed`, so the UI sees nothing for the first several seconds even when parsing succeeds. 4. **No `m.error` handling.** Ollama surfaces errors as `{"error": "..."}` lines with no `status` field. The current code has no branch for this — errors are silently dropped. **Net effect:** on any non-progress response (HTTP error, error line, or multi-object chunk), `pullModel` runs to stream completion without firing `onSuccess`, `onProgress`, or `onError`. The frontend sits forever on the initial log line with no signal that anything went wrong. --- ## Proposed fix ### (a) Bump bundled Ollama to current stable Update whatever build script downloads the Ollama binary during Desktop packaging. Current stable is **v0.20.7** (`https://github.com/ollama/ollama/releases/download/v0.20.7/ollama-darwin.tgz` for macOS). v0.20.7 layout is a drop-in replacement for v0.20.3 — same `libggml-*` files, plus new `mlx_metal_v3/` and `mlx_metal_v4/` subdirectories that modern Metal-accelerated models require. ### (b) Rewrite `pullModel` to handle the Ollama stream correctly ```js async pullModel(modelTag = "llama2", onProgress, onSuccess, onError) { await this.bootOrContinue(); this.log(`Starting pull of model tag "${modelTag}".`); const response = await fetch(`${this.basePath()}/api/pull`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ name: modelTag, stream: true }), }); if (!response.ok) { let text = ""; try { text = await response.text(); } catch {} this.log(`Pull failed with HTTP ${response.status}: ${text}`); onError?.(`HTTP ${response.status}: ${text || response.statusText}`); return; } const reader = response.body.getReader(); const decoder = new TextDecoder("utf-8"); let buffer = ""; for (;;) { const { done, value } = await reader.read(); if (done) { if (buffer.trim()) { try { const msg = JSON.parse(buffer); if (msg?.error) onError?.(msg.error); else if (msg?.status === "success") onSuccess?.(); } catch {} } break; } buffer += decoder.decode(value, { stream: true }); const lines = buffer.split("\n"); buffer = lines.pop(); for (const line of lines) { if (!line.trim()) continue; let msg; try { msg = JSON.parse(line); } catch (err) { console.error("Failed to parse Ollama pull line:", line, err); continue; } if (msg?.error) { this.log(`Pull error: ${msg.error}`); onError?.(msg.error); return; } if (msg?.status) { if (msg.total && msg.completed != null) { const percent = Math.round((msg.completed / msg.total) * 100); this.log(`Pull ${msg.status} ${percent}%`); onProgress?.(percent, msg.status); } else { this.log(`Pull ${msg.status}`); onProgress?.(0, msg.status); } } if (msg?.status === "success") onSuccess?.(); } } } ``` Key differences vs. the current implementation: - Checks `response.ok` and surfaces HTTP errors via `onError`. - Accumulates decoded bytes in `buffer`, splits on `\n`, parses each complete line individually, preserves trailing partial line for next chunk — correct NDJSON handling. - Relaxes the progress guard to any `msg.status`, so early lifecycle events appear in the UI. - Handles `{"error": "..."}` lines from Ollama explicitly. - `decoder.decode(value, { stream: true })` to correctly handle multibyte characters split across chunks. --- ## Verification I applied both fixes locally (replaced `Contents/Resources/ollama/` with v0.20.7 and rewrote `pullModel` in the installed Desktop bundle). Same repro as above now produces live progress events and the pull completes successfully: Backend log after the patch shows the full lifecycle: ``` [AnythingLLMOllama] Starting pull of model tag "gemma4:26b". [AnythingLLMOllama] Pull pulling manifest [AnythingLLMOllama] Pull pulling <digest> 1% [AnythingLLMOllama] Pull pulling <digest> 2% ... [AnythingLLMOllama] Pull verifying sha256 digest [AnythingLLMOllama] Pull writing manifest [AnythingLLMOllama] Pull success ``` Happy to open a PR against the Desktop source repo if a maintainer can point me at it — the `AnythingLLMOllama` class and the bundled-binary build step don't appear to be in this repo, so I'm filing here as an issue. --- ## Severity note This isn't just about `gemma4:26b` — it's any model the bundled Ollama can't fetch (stale client, typo'd tag, network failure, registry 5xx, etc.). All of them hit the same silent-hang path with zero error surfaced to the user. That's a pretty bad first-run experience for anyone trying the bundled provider. <img width="628" height="432" alt="Image" src="https://github.com/user-attachments/assets/5ad294dc-b022-4478-8ed1-af48b904ec4f" />
yindo closed this issue 2026-06-05 14:51:57 -04:00
yindo commented 2026-06-05 14:51:58 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 14, 2026):

Current Ollama release is v0.20.7. The 0.13.0 client inside the Desktop bundle predates modern model manifest formats. When asked to pull a current model, the registry returns:

Yeah, because the latest is basically always broken and needs to be patched within a few days. We have the ability to pull tags but some newer models need the latest binary. However the models we suggest/recommend all work as designed

<!-- gh-comment-id:4246244841 --> @timothycarambat commented on GitHub (Apr 14, 2026): > Current Ollama release is v0.20.7. The 0.13.0 client inside the Desktop bundle predates modern model manifest formats. When asked to pull a current model, the registry returns: Yeah, because the latest is basically always broken and needs to be patched within a few days. We have the ability to pull tags but some newer models need the latest binary. However the models we suggest/recommend all work as designed
yindo commented 2026-06-05 14:51:58 -04:00
Author
Owner
Copy Link
Copy Source

@shdjfensjfsns commented on GitHub (Apr 14, 2026):

understood, it would just be nice to be able to pull the latest models without having to run a separate ollama instance since anythingllm makes everything so easy within a single pane of glass :)

<!-- gh-comment-id:4246297764 --> @shdjfensjfsns commented on GitHub (Apr 14, 2026): understood, it would just be nice to be able to pull the latest models without having to run a separate ollama instance since anythingllm makes everything so easy within a single pane of glass :)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5092
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?