Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1902] [FEAT]: Add "keep_alive" Parameter Configuration in UI for Ollama Models #1236

New Issue
Closed
opened 2026-02-22 18:23:52 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:23:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @MrSimonC on GitHub (Jul 19, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1902

Originally assigned to: @timothycarambat on GitHub.

What would you like to see?

Dear Maintainers,

I hope this message finds you well. First and foremost, I would like to express my gratitude for your continuous efforts in maintaining and improving Anything LLM. Your work is highly appreciated by the community.

I am writing to request the addition of a feature that would greatly enhance the usability and flexibility of the application, specifically when using Ollama models. Currently, the Ollama models support a "keep_alive" parameter, as documented here on ollama and here in langchain. (I've seen in the source code that you use langchain as a wrapper around the call to ollama which is why I'm providing that link.)

The "keep_alive" parameter is useful for keeping ollama models in memory longer - and as of now, there is no option to configure this parameter directly from the UI.

Problem:
The model always defaults to a 5-minute load in memory only. This behavior can be observed using the "ollama ps" command once you start a chat in Anything LLM. This default setting leads to the model unloading from memory after 5 minutes, which is not ideal for scenarios requiring persistent connections. (Additionally, I've tested the OLLAMA_KEEP_ALIVE environment variable implemented by Ollama does not get respected when using the API/chat endpoint from Anything LLM). I've also tried the curl http://localhost:11434/api/generate -d '{"model": "llama3", "keep_alive": -1}' command (docs) but a new call from AnythingLLM seems to override this setting and default back to 5 mins.

Feature Request:

  1. Add "keep_alive" Configuration in UI:
    • Introduce ani nput field in the ollama-specific UI that allows users to enter a value into the "keep_alive" parameter when configuring Ollama models.
    • Optionally, provide a brief description or tooltip explaining the purpose and impact of the "keep_alive" parameter?

Benefits:

  • Improved Performance: Allowing users to configure the "keep_alive" parameter can help maintain persistent connections, reducing latency and improving overall performance when making requests with a >5 min break in between.
  • Enhanced Flexibility: Users can tailor the behavior of the Ollama models to better suit their specific use cases.
  • Resolve Current Limitation: Address the current limitation where the model defaults to a 5-minute memory load.

I believe this feature will be a valuable addition to Anything LLM and will enhance the user experience for many. Thank you for considering this request. If you need any further information or assistance, please do not hesitate to contact me.

Best regards,
Simon

Originally created by @MrSimonC on GitHub (Jul 19, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1902 Originally assigned to: @timothycarambat on GitHub. ### What would you like to see? Dear Maintainers, I hope this message finds you well. First and foremost, I would like to express my gratitude for your continuous efforts in maintaining and improving Anything LLM. Your work is highly appreciated by the community. I am writing to request the addition of a feature that would greatly enhance the usability and flexibility of the application, specifically when using Ollama models. Currently, the Ollama models support a "keep_alive" parameter, as documented [here on ollama](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-keep-a-model-loaded-in-memory-or-make-it-unload-immediately) and [here in langchain](https://api.python.langchain.com/en/latest/chat_models/langchain_community.chat_models.ollama.ChatOllama.html#langchain_community.chat_models.ollama.ChatOllama.keep_alive). (I've seen in the source code that you use langchain as a wrapper around the call to ollama which is why I'm providing that link.) The "keep_alive" parameter is useful for keeping ollama models in memory longer - and as of now, there is no option to configure this parameter directly from the UI. **Problem:** The model always defaults to a 5-minute load in memory only. This behavior can be observed using the "ollama ps" command once you start a chat in Anything LLM. This default setting leads to the model unloading from memory after 5 minutes, which is not ideal for scenarios requiring persistent connections. (Additionally, I've tested the [OLLAMA_KEEP_ALIVE environment variable](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-keep-a-model-loaded-in-memory-or-make-it-unload-immediately) implemented by Ollama does not get respected when using the API/chat endpoint from Anything LLM). I've also tried the `curl http://localhost:11434/api/generate -d '{"model": "llama3", "keep_alive": -1}'` command ([docs](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-keep-a-model-loaded-in-memory-or-make-it-unload-immediately)) but a new call from AnythingLLM seems to override this setting and default back to 5 mins. **Feature Request:** 1. **Add "keep_alive" Configuration in UI:** - Introduce ani nput field in the ollama-specific UI that allows users to enter a value into the "keep_alive" parameter when configuring Ollama models. - Optionally, provide a brief description or tooltip explaining the purpose and impact of the "keep_alive" parameter? **Benefits:** - **Improved Performance:** Allowing users to configure the "keep_alive" parameter can help maintain persistent connections, reducing latency and improving overall performance when making requests with a >5 min break in between. - **Enhanced Flexibility:** Users can tailor the behavior of the Ollama models to better suit their specific use cases. - **Resolve Current Limitation:** Address the current limitation where the model defaults to a 5-minute memory load. I believe this feature will be a valuable addition to Anything LLM and will enhance the user experience for many. Thank you for considering this request. If you need any further information or assistance, please do not hesitate to contact me. Best regards, Simon
yindo added the enhancementfeature request labels 2026-02-22 18:23:52 -05:00
yindo closed this issue 2026-02-22 18:23:52 -05:00
yindo commented 2026-02-22 18:23:52 -05:00
Author
Owner
Copy Link
Copy Source

@MrSimonC commented on GitHub (Jul 19, 2024):

Also, if it helps, I'm currently running Anything LLM in a Docker container in WSL2 and accompanied by a direct native install of Ollama in WSL 2 on Windows (due to my company restrictions on installing files).

@MrSimonC commented on GitHub (Jul 19, 2024): Also, if it helps, I'm currently running Anything LLM in a Docker container in WSL2 and accompanied by a direct native install of Ollama in WSL 2 on Windows _(due to my company restrictions on installing files)._
yindo commented 2026-02-22 18:23:52 -05:00
Author
Owner
Copy Link
Copy Source

@MrSimonC commented on GitHub (Jul 19, 2024):

Update: I've been searching and found this is a dup of #1588 which has been moved to #1585

@MrSimonC commented on GitHub (Jul 19, 2024): Update: I've been searching and found this is a dup of #1588 which has been moved to #1585
yindo commented 2026-02-22 18:23:52 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 19, 2024):

Very well written, thank you. Will move the conversation to this issue since its more action-oriented.

@timothycarambat commented on GitHub (Jul 19, 2024): Very well written, thank you. Will move the conversation to this issue since its more action-oriented.
yindo changed title from [FEAT]: Add "keep_alive" Parameter Configuration in UI for Ollama Models to [GH-ISSUE #1902] [FEAT]: Add "keep_alive" Parameter Configuration in UI for Ollama Models 2026-06-05 14:39:41 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1236
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?