langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Dify with OIlama keeps restarting #5674

New Issue
Closed
opened 2026-02-21 18:12:07 -05:00 by yindo · 8 comments
yindo commented 2026-02-21 18:12:07 -05:00
Owner
Copy Link
Copy Source

Originally created by @dougpiv on GitHub (Sep 18, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

8.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I am experiencing a strange behavior when I run the workflow in Dify. It makes the call to Ollama, but it is restarting the model at each iteration. It didn’t do this in the previous version 7.3. The model is llama3.1.

image

image

✔️ Expected Behavior

I expect that the memory on gpu keeps full and dont restart de model every iteration

❌ Actual Behavior

Every iteration the model restarts on ollama

Originally created by @dougpiv on GitHub (Sep 18, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 8.2 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce I am experiencing a strange behavior when I run the workflow in Dify. It makes the call to Ollama, but it is restarting the model at each iteration. It didn’t do this in the previous version 7.3. The model is llama3.1. ![image](https://github.com/user-attachments/assets/a8045139-32d1-4e65-ab99-1ea20d6e8605) ![image](https://github.com/user-attachments/assets/3820ca3a-6fa5-4e2d-9d40-e90342131350) ### ✔️ Expected Behavior I expect that the memory on gpu keeps full and dont restart de model every iteration ### ❌ Actual Behavior Every iteration the model restarts on ollama
yindo added the 🐞 bug label 2026-02-21 18:12:07 -05:00
yindo closed this issue 2026-02-21 18:12:07 -05:00
yindo commented 2026-02-21 18:12:08 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Sep 18, 2024):

Hey @dougpiv! 👋 I'm here to help you with any bugs, questions, or contributions you have for Dify with OIlama! I'm an expert bot and I'll assist you while we wait for a human maintainer to jump in.

I found a similar discussion regarding the Ollama model restarting at each iteration in Dify version 8.2. The issue is due to a connection problem between Docker containers and the Ollama service. The Docker container cannot access the Ollama service using localhost. This can be resolved by configuring the service to use host.docker.internal instead of localhost:

http://host.docker.internal:11434

This change ensures that the Docker container can correctly access the Ollama service, preventing the model from restarting at each iteration. Additionally, if you are deploying Dify and Ollama using Docker, consider using the local network IP address, e.g., http://192.168.1.100:11434 or the Docker host machine IP address, e.g., http://172.17.0.1:11434 [1][2].

For more details, you can refer to the discussion on this issue [3].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Sep 18, 2024): <!-- Greeting --> Hey @dougpiv! 👋 I'm here to help you with any bugs, questions, or contributions you have for Dify with OIlama! I'm an expert bot and I'll assist you while we wait for a human maintainer to jump in. <!-- Answer --> I found a similar discussion regarding the Ollama model restarting at each iteration in Dify version 8.2. The issue is due to a connection problem between Docker containers and the Ollama service. The Docker container cannot access the Ollama service using `localhost`. This can be resolved by configuring the service to use `host.docker.internal` instead of `localhost`: ```bash http://host.docker.internal:11434 ``` This change ensures that the Docker container can correctly access the Ollama service, preventing the model from restarting at each iteration. Additionally, if you are deploying Dify and Ollama using Docker, consider using the local network IP address, e.g., `http://192.168.1.100:11434` or the Docker host machine IP address, e.g., `http://172.17.0.1:11434` <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/jp/guides/model-configuration/ollama.md)</sup><sup>[[2]](https://github.com/langgenius/dify-docs/blob/main/en/guides/model-configuration/ollama.md)</sup>. For more details, you can refer to the [discussion on this issue](https://github.com/langgenius/dify/discussions/7218) <sup>[[3]](https://github.com/langgenius/dify/discussions/7218)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:12:08 -05:00
Author
Owner
Copy Link
Copy Source

@dougpiv commented on GitHub (Sep 18, 2024):

I am already using host.docker.internal and the connection is working. What is happening is that it seems to be restarting the model on the GPU with every request in a cycle in Dify.

@dougpiv commented on GitHub (Sep 18, 2024): I am already using host.docker.internal and the connection is working. What is happening is that it seems to be restarting the model on the GPU with every request in a cycle in Dify.
yindo commented 2026-02-21 18:12:08 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Sep 18, 2024):

The model on the GPU is likely restarting with every request in a cycle because the Ollama service is not properly exposed to the network. Ensure that the Ollama service is accessible from the Docker container by setting the OLLAMA_HOST environment variable to 0.0.0.0 to bind the service to all network interfaces.

For macOS:

launchctl setenv OLLAMA_HOST "0.0.0.0"

For Linux:

[Service]
Environment="OLLAMA_HOST=0.0.0.0"

For Windows:

  1. Quit Ollama by clicking on it in the taskbar.
  2. Edit system environment variables from the control panel.
  3. Edit or create a new variable for your user account: OLLAMA_HOST=0.0.0.0.
  4. Click OK/Apply to save.
  5. Run ollama from a new terminal window.

Additionally, if you are deploying Dify using Docker, use the local network IP address or the Docker host machine's IP address instead of localhost:

http://192.168.1.100:11434

or

http://172.17.0.1:11434

Ensure that the Dify container can access the endpoint of Xinference if you are using it for local model deployment. Replace localhost with your machine's IP address in the Dify settings:

http://<Machine_IP>:9997

Start Xinference with the -H 0.0.0.0 option to allow access from any non-local client:

xinference-local -H 0.0.0.0

Finally, ensure that the DIFY_BIND_ADDRESS environment variable in your docker-compose.yaml is set to 0.0.0.0:

DIFY_BIND_ADDRESS: ${DIFY_BIND_ADDRESS:-0.0.0.0}

This configuration should prevent the model from restarting with every request [1][2][3][4].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Sep 18, 2024): <!-- Answer --> The model on the GPU is likely restarting with every request in a cycle because the Ollama service is not properly exposed to the network. Ensure that the Ollama service is accessible from the Docker container by setting the `OLLAMA_HOST` environment variable to `0.0.0.0` to bind the service to all network interfaces. For macOS: ```bash launchctl setenv OLLAMA_HOST "0.0.0.0" ``` For Linux: ```ini [Service] Environment="OLLAMA_HOST=0.0.0.0" ``` For Windows: 1. Quit Ollama by clicking on it in the taskbar. 2. Edit system environment variables from the control panel. 3. Edit or create a new variable for your user account: `OLLAMA_HOST=0.0.0.0`. 4. Click OK/Apply to save. 5. Run `ollama` from a new terminal window. Additionally, if you are deploying Dify using Docker, use the local network IP address or the Docker host machine's IP address instead of `localhost`: ```bash http://192.168.1.100:11434 ``` or ```bash http://172.17.0.1:11434 ``` Ensure that the Dify container can access the endpoint of Xinference if you are using it for local model deployment. Replace `localhost` with your machine's IP address in the Dify settings: ```bash http://<Machine_IP>:9997 ``` Start Xinference with the `-H 0.0.0.0` option to allow access from any non-local client: ```bash xinference-local -H 0.0.0.0 ``` Finally, ensure that the `DIFY_BIND_ADDRESS` environment variable in your `docker-compose.yaml` is set to `0.0.0.0`: ```yaml DIFY_BIND_ADDRESS: ${DIFY_BIND_ADDRESS:-0.0.0.0} ``` This configuration should prevent the model from restarting with every request <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/en/guides/model-configuration/ollama.md)</sup><sup>[[2]](https://github.com/langgenius/dify-docs/blob/main/jp/guides/model-configuration/ollama.md)</sup><sup>[[3]](https://github.com/langgenius/dify-docs/blob/main/en/guides/model-configuration/xinference.md)</sup><sup>[[4]](https://github.com/langgenius/dify/blob/main/docker/docker-compose.yaml)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:12:08 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Sep 23, 2024):

I am already using host.docker.internal and the connection is working. What is happening is that it seems to be restarting the model on the GPU with every request in a cycle in Dify.

Do you have other information regarding this behavior, I can not reproduce this.

@crazywoola commented on GitHub (Sep 23, 2024): > I am already using host.docker.internal and the connection is working. What is happening is that it seems to be restarting the model on the GPU with every request in a cycle in Dify. Do you have other information regarding this behavior, I can not reproduce this.
yindo commented 2026-02-21 18:12:08 -05:00
Author
Owner
Copy Link
Copy Source

@Howe829 commented on GitHub (Sep 25, 2024):

I think dify can add 'keep_live' parameter while making request to Ollama server to keep the model loaded in the memory.
See the Ollama docs:
https://github.com/ollama/ollama/blob/450acb71a6366ca22eada96ab08c0527a6914c5e/docs/faq.md?plain=1#L246

@Howe829 commented on GitHub (Sep 25, 2024): I think dify can add 'keep_live' parameter while making request to Ollama server to keep the model loaded in the memory. See the Ollama docs: https://github.com/ollama/ollama/blob/450acb71a6366ca22eada96ab08c0527a6914c5e/docs/faq.md?plain=1#L246
yindo commented 2026-02-21 18:12:08 -05:00
Author
Owner
Copy Link
Copy Source

@Howe829 commented on GitHub (Sep 25, 2024):

I think dify can add 'keep_live' parameter while making request to Ollama server to keep the model loaded in the memory. See the Ollama docs: https://github.com/ollama/ollama/blob/450acb71a6366ca22eada96ab08c0527a6914c5e/docs/faq.md?plain=1#L246

Ok,I think dify already supports the 'keep_alive' parameter. see #4655

@Howe829 commented on GitHub (Sep 25, 2024): > I think dify can add 'keep_live' parameter while making request to Ollama server to keep the model loaded in the memory. See the Ollama docs: https://github.com/ollama/ollama/blob/450acb71a6366ca22eada96ab08c0527a6914c5e/docs/faq.md?plain=1#L246 Ok,I think dify already supports the 'keep_alive' parameter. see #4655
yindo commented 2026-02-21 18:12:08 -05:00
Author
Owner
Copy Link
Copy Source

@dougpiv commented on GitHub (Oct 30, 2024):

Sorry for the long time without an answer. Here is what is happening: I am using a simple flow:
image

And here is my model config:
image

In early versions of Dify, it didn't keep reloading the model in Ollama. Now, it seems that it loads the embedding model and after that loads the LLM model (I am using an 8GB GPU).
Now I update the version to 0.10.2.

@dougpiv commented on GitHub (Oct 30, 2024): Sorry for the long time without an answer. Here is what is happening: I am using a simple flow: ![image](https://github.com/user-attachments/assets/78f7cf8e-0cb3-4e41-bf7b-5ca0e780d938) And here is my model config: ![image](https://github.com/user-attachments/assets/379684b1-4d36-40b6-a432-3e234f681298) In early versions of Dify, it didn't keep reloading the model in Ollama. Now, it seems that it loads the embedding model and after that loads the LLM model (I am using an 8GB GPU). Now I update the version to 0.10.2.
yindo commented 2026-02-21 18:12:08 -05:00
Author
Owner
Copy Link
Copy Source

@Lanticee commented on GitHub (Nov 25, 2024):

Hi
I am using 0.11.2 , and this status is still happens. event I set keep a live in setting.
1

@Lanticee commented on GitHub (Nov 25, 2024): Hi I am using 0.11.2 , and this status is still happens. event I set keep a live in setting. <img width="937" alt="1" src="https://github.com/user-attachments/assets/cf33080c-26fd-4b04-b6e0-924dd55ad5ed">
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#5674
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?