anomalyco/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-22 02:05:26 -04:00
Code Issues 5.9k Packages Projects Releases 19 Wiki Activity

Fix keep_alive support / persistent model sessions for Ollama provider to prevent model reloads between requests #1961

New Issue
Open
opened 2026-02-16 17:33:27 -05:00 by yindo · 5 comments
yindo commented 2026-02-16 17:33:27 -05:00
Owner
Copy Link
Copy Source

Originally created by @gregory-chatelier on GitHub (Oct 5, 2025).

Originally assigned to: @rekram1-node on GitHub.

When using Ollama as a provider in opencode, the model appears to reload into memory for each request.

This significantly increases latency (especially for large models such as qwen3-coder:30b) and consumes unnecessary system resources.

Example configuration:

{
    "$schema": "https://opencode.ai/config.json",
    "provider": {
        "ollama": {
            "npm": "@ai-sdk/openai-compatible",
            "name": "Ollama (local)",
            "options": {
                "baseURL": "http://localhost:11434/v1"
            },
            "models": {
                "qwen3-coder:30b": {
                    "name": "qwen3-coder:30b",
                    "options": {
                        "keep_alive": -1
                    }
                }
            }
        }
    }
}

Behavior:

Each time opencode sends a new request to the Ollama endpoint, the model is unloaded and reloaded by the Ollama server.

This can add up to 60 seconds of overhead on every call for large models.

Direct requests to Ollama using its API or CLI (with keep_alive or OLLAMA_KEEP_ALIVE) can keep the model resident in memory — but this flag seems not to be propagated or respected through the opencode integration.

Expected behavior

When configured, models should remain loaded in memory for a configurable duration (e.g., keep_alive: 300) between requests.

The keep_alive value should be passed through to Ollama’s /v1/chat/completions or /v1/completions API payload, if supported.

Optionally, a global default could be set in the provider’s options.

The following didn't worked :

      "options": {
        "baseURL": "http://localhost:11434/v1",
        "keep_alive": 300
      }
Originally created by @gregory-chatelier on GitHub (Oct 5, 2025). Originally assigned to: @rekram1-node on GitHub. When using Ollama as a provider in opencode, the model appears to reload into memory for each request. This significantly increases latency (especially for large models such as qwen3-coder:30b) and consumes unnecessary system resources. Example configuration: ```json { "$schema": "https://opencode.ai/config.json", "provider": { "ollama": { "npm": "@ai-sdk/openai-compatible", "name": "Ollama (local)", "options": { "baseURL": "http://localhost:11434/v1" }, "models": { "qwen3-coder:30b": { "name": "qwen3-coder:30b", "options": { "keep_alive": -1 } } } } } } ``` Behavior: Each time opencode sends a new request to the Ollama endpoint, the model is unloaded and reloaded by the Ollama server. This can add up to 60 seconds of overhead on every call for large models. Direct requests to Ollama using its API or CLI (with keep_alive or OLLAMA_KEEP_ALIVE) can keep the model resident in memory — but this flag seems not to be propagated or respected through the opencode integration. Expected behavior When configured, models should remain loaded in memory for a configurable duration (e.g., keep_alive: 300) between requests. The keep_alive value should be passed through to Ollama’s /v1/chat/completions or /v1/completions API payload, if supported. Optionally, a global default could be set in the provider’s options. The following didn't worked : ```json "options": { "baseURL": "http://localhost:11434/v1", "keep_alive": 300 } ```
yindo commented 2026-02-16 17:33:27 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Oct 5, 2025):

@gregory-chatelier try this:

    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://localhost:11434/v1",
        "headers": {
          "keep_alive": -1
        }
      },
      "models": {
        "qwen3-coder:30b": {
          "name": "qwen3-coder:30b"
        }
      }
    },
@rekram1-node commented on GitHub (Oct 5, 2025): @gregory-chatelier try this: ``` "ollama": { "npm": "@ai-sdk/openai-compatible", "name": "Ollama (local)", "options": { "baseURL": "http://localhost:11434/v1", "headers": { "keep_alive": -1 } }, "models": { "qwen3-coder:30b": { "name": "qwen3-coder:30b" } } }, ```
yindo commented 2026-02-16 17:33:27 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Oct 5, 2025):

I tested, the headers do get sent if you do it this way^^

@rekram1-node commented on GitHub (Oct 5, 2025): I tested, the headers do get sent if you do it this way^^
yindo commented 2026-02-16 17:33:27 -05:00
Author
Owner
Copy Link
Copy Source

@gregory-chatelier commented on GitHub (Oct 6, 2025):

Nice try, but without any success.

I already set an OLLAMA_KEEP_ALIVE env var to -1 and I'm pretty sure now the issue is not this keep alive directive. I guess there is an additional caller authentification that isn't consistent between requests. I use Ollama under Docker by the way.

@gregory-chatelier commented on GitHub (Oct 6, 2025): Nice try, but without any success. I already set an OLLAMA_KEEP_ALIVE env var to -1 and I'm pretty sure now the issue is not this keep alive directive. I guess there is an additional caller authentification that isn't consistent between requests. I use Ollama under Docker by the way.
yindo commented 2026-02-16 17:33:27 -05:00
Author
Owner
Copy Link
Copy Source

@rekram1-node commented on GitHub (Oct 6, 2025):

@gregory-chatelier I don't have any ollama experience but someone made a simple guide of using opencode + ollama + docker compose, have given people some suggestions from this: https://github.com/p-lemonish/ollama-x-opencode

Not sure if this is helpful at all but a lot of people have told me they found it helpful, it'd be worth adding certain parts to our docs but let me know if anythign there is helpful or not :)

@rekram1-node commented on GitHub (Oct 6, 2025): @gregory-chatelier I don't have any ollama experience but someone made a simple guide of using opencode + ollama + docker compose, have given people some suggestions from this: https://github.com/p-lemonish/ollama-x-opencode Not sure if this is helpful at all but a lot of people have told me they found it helpful, it'd be worth adding certain parts to our docs but let me know if anythign there is helpful or not :)
yindo commented 2026-02-16 17:33:27 -05:00
Author
Owner
Copy Link
Copy Source

@gregory-chatelier commented on GitHub (Oct 7, 2025):

It didn't solve my issue but it's indeed a valuable resource I would recommend

@gregory-chatelier commented on GitHub (Oct 7, 2025): It didn't solve my issue but it's indeed a valuable resource I would recommend
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
acp
bad
bug
discussion
docs
documentation
enhancement
extension
good first issue
growth
help-wanted
mcp
model-problem
needs:compliance
nix
opentui
perf
pull-request
Mirrored from GitHub Pull Request
 tui
web
windows
zen
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: anomalyco/opencode#1961
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?