API proxy leads to sandbox 502 and http request time out in self-hosted docker-compose environment #6385

Closed
opened 2026-02-21 18:15:30 -05:00 by yindo · 13 comments
Owner

Originally created by @Derican on GitHub (Oct 28, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

newest from docker image

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

  1. start dify from docker compose as written in docs.

  2. add proxy to api in docker-compose.yaml and restart dify
    image

  3. test code runner in dify
    image

  4. test http request, GET www.google.com
    image

The LLM Module and network on the server is tested ok. And Suggestion in #3079 is already adopted.

✔️ Expected Behavior

for 3, execute code in python script
for 4, got status 200 from response

Actual Behavior

for 3, got the 502 error as the same in #9737
image
for 4, got reached max retries
image

Originally created by @Derican on GitHub (Oct 28, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:) - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version newest from docker image ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce 1. start dify from docker compose as written in docs. 2. add proxy to api in docker-compose.yaml and restart dify ![image](https://github.com/user-attachments/assets/bcc6b8dc-704a-4e9f-9be9-c0f1ce17f6cb) 3. test code runner in dify ![image](https://github.com/user-attachments/assets/5f33a70e-3137-46b3-952e-5b63dff9bee7) 4. test http request, GET www.google.com ![image](https://github.com/user-attachments/assets/d901eb60-9cad-44d3-ba45-3ef7e5f67968) **The LLM Module and network on the server is tested ok. And Suggestion in #3079 is already adopted.** ### ✔️ Expected Behavior for 3, execute code in python script for 4, got status 200 from response ### ❌ Actual Behavior for 3, got the 502 error as the same in #9737 ![image](https://github.com/user-attachments/assets/3a6de184-52dd-475f-92fc-c4b625713cbe) for 4, got reached max retries ![image](https://github.com/user-attachments/assets/4fc5e82d-0c6c-4899-9043-8e3f92b0616f)
yindo added the 🐞 bug🤔 cant-reproduce labels 2026-02-21 18:15:30 -05:00
yindo closed this issue 2026-02-21 18:15:30 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Oct 28, 2024):

Hello, @Derican! I'm here to help you with bug reports, answer questions, and assist you as a contributor. Let's work together to resolve the issue you're facing.

I found some similar issues and discussions that might help you resolve the 502 error and HTTP request timeout in your self-hosted Docker Compose environment:

  1. 502 Error Related to Sandbox Service: A common cause of a 502 error in the Dify self-hosted environment is Nginx forwarding the service to the wrong location. Ensure that the Nginx configuration is updated with the correct IP addresses of the running containers. You can find the IP addresses using the following command:

    docker ps -q | xargs -n 1 docker inspect --format '{{ .Name }}: {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'
    

    Update the Nginx configuration by replacing http://api:5001 and http://web:3000 with the correct IP addresses and ports. After making these changes, restart the Nginx container or reload the configuration [1].

  2. HTTP Request Timeout: The HTTP request utility timing out can be addressed by removing the SSRF_PROXY_HTTP_URL and SSRF_PROXY_HTTPS_URL environment variables from the API container. Additionally, adjusting the GUNICORN_TIMEOUT environment variable to 360 seconds may help prevent timeouts for longer HTTP requests [2].

  3. Proxy Configuration: Ensure that the proxy settings in your Docker Compose file are correctly configured. The default proxy settings are:

    environment:
      HTTP_PROXY: ${SANDBOX_HTTP_PROXY:-http://ssrf_proxy:3128}
      HTTPS_PROXY: ${SANDBOX_HTTPS_PROXY:-http://ssrf_proxy:3128}
    

    These settings indicate that the default HTTP and HTTPS proxy is http://ssrf_proxy:3128 [3].

By following these steps, you should be able to resolve the 502 error and HTTP request timeout issues. If the problem persists, further investigation into the specific error messages in the logs will be necessary.

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Oct 28, 2024): <!-- Greeting --> Hello, @Derican! I'm here to help you with bug reports, answer questions, and assist you as a contributor. Let's work together to resolve the issue you're facing. <!-- Answer --> I found some similar issues and discussions that might help you resolve the 502 error and HTTP request timeout in your self-hosted Docker Compose environment: 1. **502 Error Related to Sandbox Service**: A common cause of a 502 error in the Dify self-hosted environment is Nginx forwarding the service to the wrong location. Ensure that the Nginx configuration is updated with the correct IP addresses of the running containers. You can find the IP addresses using the following command: ```bash docker ps -q | xargs -n 1 docker inspect --format '{{ .Name }}: {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' ``` Update the Nginx configuration by replacing `http://api:5001` and `http://web:3000` with the correct IP addresses and ports. After making these changes, restart the Nginx container or reload the configuration <sup>[[1]](https://github.com/langgenius/dify-docs/blob/main/en/guides/application-orchestrate/llms-use-faq.md)</sup>. 2. **HTTP Request Timeout**: The HTTP request utility timing out can be addressed by removing the `SSRF_PROXY_HTTP_URL` and `SSRF_PROXY_HTTPS_URL` environment variables from the API container. Additionally, adjusting the `GUNICORN_TIMEOUT` environment variable to 360 seconds may help prevent timeouts for longer HTTP requests <sup>[[2]](https://github.com/langgenius/dify/issues/6244)</sup>. 3. **Proxy Configuration**: Ensure that the proxy settings in your Docker Compose file are correctly configured. The default proxy settings are: ```yaml environment: HTTP_PROXY: ${SANDBOX_HTTP_PROXY:-http://ssrf_proxy:3128} HTTPS_PROXY: ${SANDBOX_HTTPS_PROXY:-http://ssrf_proxy:3128} ``` These settings indicate that the default HTTP and HTTPS proxy is `http://ssrf_proxy:3128` <sup>[[3]](https://github.com/langgenius/dify/blob/main/docker/docker-compose.yaml)</sup>. By following these steps, you should be able to resolve the 502 error and HTTP request timeout issues. If the problem persists, further investigation into the specific error messages in the logs will be necessary. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
Author
Owner
@crazywoola commented on GitHub (Oct 28, 2024): https://github.com/langgenius/dify/blob/488e3c3d565434bb6988d3d2120f0dfa9cb7fa4b/docker/ssrf_proxy/squid.conf.template Check the settings and the docs https://docs.dify.ai/learn-more/faq/install-faq#id-18.-why-is-ssrf_proxy-needed
Author
Owner

@Derican commented on GitHub (Oct 29, 2024):

For anyone who may possibly look into this issue, I provide a PART of solution here. For me, I added http proxy to SSRF in api and worker, and add rules in squid.conf.template to let ssrf access the proxy of the host.
image
image
HOWEVER, while this modification solved the sandbox connection error brought by api proxy, the http request error still persists. I tried replace http_access allow all with rules in the squid config, but still can't make it. Hope anyone can find out the correct way.

@Derican commented on GitHub (Oct 29, 2024): For anyone who may possibly look into this issue, I provide a **PART** of solution here. For me, I added http proxy to **SSRF** in api and worker, and add rules in `squid.conf.template` to let ssrf access the proxy of the host. ![image](https://github.com/user-attachments/assets/97e4132c-0e61-46b8-bae2-ea75a2895791) ![image](https://github.com/user-attachments/assets/dcf54436-6cac-454c-96e1-2056280a4818) **HOWEVER**, while this modification solved the sandbox connection error brought by api proxy, the http request error still persists. I tried replace `http_access allow all` with rules in the squid config, but still can't make it. Hope anyone can find out the correct way.
Author
Owner

@hzhstar commented on GitHub (Nov 25, 2024):

出现这个 大概率是给容器配置了代理了,要把代理去掉

@hzhstar commented on GitHub (Nov 25, 2024): 出现这个 大概率是给容器配置了代理了,要把代理去掉
Author
Owner

@Derican commented on GitHub (Nov 26, 2024):

出现这个 大概率是给容器配置了代理了,要把代理去掉

但是代理是必须的,无论说的是拉取镜像的代理还是自部署的dify连接OpenAI等服务。所以这个Issue以及相关的大部分Issue讨论都是建立在如何正确地给容器添加代理。可以看到上面我给的让ssrf走代理(收集自CSDN和Discord日本组的相关讨论)一定程度上解决了这个问题(只有http请求节点无法连接)。
But the proxy is necessary, whether it is the proxy that pulls the image or the self-hosted dify to connect to services such as OpenAI. So this issue and most of the related issue discussions are based on how to correctly add a proxy to the container. You can see that the above I gave to let ssrf take the proxy (collected from the relevant discussions of CSDN and Discord Japan group) solves this problem to a certain extent (only the http requesting node cannot connect).

@Derican commented on GitHub (Nov 26, 2024): > 出现这个 大概率是给容器配置了代理了,要把代理去掉 但是代理是必须的,无论说的是拉取镜像的代理还是自部署的dify连接OpenAI等服务。所以这个Issue以及相关的大部分Issue讨论都是建立在如何正确地给容器添加代理。可以看到上面我给的让ssrf走代理(收集自CSDN和Discord日本组的相关讨论)一定程度上解决了这个问题(只有http请求节点无法连接)。 But the proxy is necessary, whether it is the proxy that pulls the image or the self-hosted dify to connect to services such as OpenAI. So this issue and most of the related issue discussions are based on **how to correctly add a proxy to the container**. You can see that the above I gave to let ssrf take the proxy (collected from the relevant discussions of CSDN and Discord Japan group) solves this problem to a certain extent (only the http requesting node cannot connect).
Author
Owner

@NiushanDong commented on GitHub (Dec 17, 2024):

@Derican
I got succeed by adding upstream proxy server and acl rules in ssrf_proxy/squid.comf.template !!!
image

@NiushanDong commented on GitHub (Dec 17, 2024): @Derican I got succeed by adding upstream proxy server and acl rules in ssrf_proxy/squid.comf.template !!! ![image](https://github.com/user-attachments/assets/356afcad-8126-4c38-ac55-27c27f02caa0)
Author
Owner

@Derican commented on GitHub (Dec 17, 2024):

@Derican I got succeed by adding upstream proxy server and acl rules in ssrf_proxy/squid.comf.template !!! image

Thanks for the reply and I'll try it out ASAP! :)

@Derican commented on GitHub (Dec 17, 2024): > @Derican I got succeed by adding upstream proxy server and acl rules in ssrf_proxy/squid.comf.template !!! ![image](https://private-user-images.githubusercontent.com/34929731/396419730-356afcad-8126-4c38-ac55-27c27f02caa0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzQ0NDc1ODMsIm5iZiI6MTczNDQ0NzI4MywicGF0aCI6Ii8zNDkyOTczMS8zOTY0MTk3MzAtMzU2YWZjYWQtODEyNi00YzM4LWFjNTUtMjdjMjdmMDJjYWEwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDEyMTclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQxMjE3VDE0NTQ0M1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTI5MGIyNTQ0MGFjMGFkNWNjZjgzOTA3MzMwNjliMTU4MmVjNjlhYTRkZWEwMjU4NWY1OTk4MWIzOGIyOTZhMmQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.UPtgvkJEtW3mM6q8Cy71jsKDNvdfSqxag_CQtHL5BQA) Thanks for the reply and I'll try it out ASAP! :)
Author
Owner

@Axaxin commented on GitHub (Dec 31, 2024):

出现这个 大概率是给容器配置了代理了,要把代理去掉

但是代理是必须的,无论说的是拉取镜像的代理还是自部署的dify连接OpenAI等服务。所以这个Issue以及相关的大部分Issue讨论都是建立在如何正确地给容器添加代理。可以看到上面我给的让ssrf走代理(收集自CSDN和Discord日本组的相关讨论)一定程度上解决了这个问题(只有http请求节点无法连接)。 But the proxy is necessary, whether it is the proxy that pulls the image or the self-hosted dify to connect to services such as OpenAI. So this issue and most of the related issue discussions are based on how to correctly add a proxy to the container. You can see that the above I gave to let ssrf take the proxy (collected from the relevant discussions of CSDN and Discord Japan group) solves this problem to a certain extent (only the http requesting node cannot connect).

  1. LLM平台服务代理问题可以用one-api这类openai compatible凑合解决;
  2. ssrf_proxy配置里面好像有白名单,可以尝试添加sandbox的容器ip到白名单解决;
  3. 我是在docker-compose.yml一开始一大串环境变量里面,注释掉SSRF_PROXY_HTTP_URL就解决本地网络通信问题,HTTPS可以不注释,一些本地http请求有问题也可以在ssrf_proxy配置里加白名单来解决;
  4. docker拉镜像,在终端配置关于代理环境变量就行吧,与容器内的代理无关;
@Axaxin commented on GitHub (Dec 31, 2024): > > 出现这个 大概率是给容器配置了代理了,要把代理去掉 > > 但是代理是必须的,无论说的是拉取镜像的代理还是自部署的dify连接OpenAI等服务。所以这个Issue以及相关的大部分Issue讨论都是建立在如何正确地给容器添加代理。可以看到上面我给的让ssrf走代理(收集自CSDN和Discord日本组的相关讨论)一定程度上解决了这个问题(只有http请求节点无法连接)。 But the proxy is necessary, whether it is the proxy that pulls the image or the self-hosted dify to connect to services such as OpenAI. So this issue and most of the related issue discussions are based on **how to correctly add a proxy to the container**. You can see that the above I gave to let ssrf take the proxy (collected from the relevant discussions of CSDN and Discord Japan group) solves this problem to a certain extent (only the http requesting node cannot connect). 1. LLM平台服务代理问题可以用one-api这类openai compatible凑合解决; 2. ssrf_proxy配置里面好像有白名单,可以尝试添加sandbox的容器ip到白名单解决; 3. 我是在docker-compose.yml一开始一大串环境变量里面,注释掉SSRF_PROXY_HTTP_URL就解决本地网络通信问题,HTTPS可以不注释,一些本地http请求有问题也可以在ssrf_proxy配置里加白名单来解决; 4. docker拉镜像,在终端配置关于代理环境变量就行吧,与容器内的代理无关;
Author
Owner

@Derican commented on GitHub (Dec 31, 2024):

@Axaxin 感谢回复!

  1. 如果从中转API角度,确实可以解决个人需求,但是作为业务开发不能这么简单处理,况且可能还有像Google等国外服务请求的需求,不能寄希望于都在本地另开服务再调用;
  2. 是的 @NiushanDong 上面给出的配置可以解决这个问题
  3. 我忘记在一开始尝试的时候这个方法是否有用了(当时也是老版本),也是想官方既然给了ssrf的框架,想知道能否在这个框架内一劳永逸地解决;
  4. 嗯这个是我写多余了,抱歉
@Derican commented on GitHub (Dec 31, 2024): @Axaxin 感谢回复! 1. 如果从中转API角度,确实可以解决个人需求,但是作为业务开发不能这么简单处理,况且可能还有像Google等国外服务请求的需求,不能寄希望于都在本地另开服务再调用; 2. 是的 @NiushanDong 上面给出的配置可以解决这个问题 3. 我忘记在一开始尝试的时候这个方法是否有用了(当时也是老版本),也是想官方既然给了ssrf的框架,想知道能否在这个框架内一劳永逸地解决; 4. 嗯这个是我写多余了,抱歉
Author
Owner

@Hzrr97 commented on GitHub (Jan 13, 2025):

@Derican I got succeed by adding upstream proxy server and acl rules in ssrf_proxy/squid.comf.template !!! image

this config solved my problem. thanks

@Hzrr97 commented on GitHub (Jan 13, 2025): > @Derican I got succeed by adding upstream proxy server and acl rules in ssrf_proxy/squid.comf.template !!! ![image](https://private-user-images.githubusercontent.com/34929731/396419730-356afcad-8126-4c38-ac55-27c27f02caa0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzY4MjM4MTgsIm5iZiI6MTczNjgyMzUxOCwicGF0aCI6Ii8zNDkyOTczMS8zOTY0MTk3MzAtMzU2YWZjYWQtODEyNi00YzM4LWFjNTUtMjdjMjdmMDJjYWEwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAxMTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMTE0VDAyNTgzOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTI1ZjQxOGU3ODBiYmUwYmNkYmI1YzVhZjMxZjM5MTI4M2UyYTZhNWE4ZDg0OTk3Mzg1MzA1NmI0Njk3N2JkYzcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.AIu_Wf2jBy7nMBQDRxER2xDqVV0Nj3p0BVY0fl3uGpQ) this config solved my problem. thanks
Author
Owner

@BGMer7 commented on GitHub (Mar 31, 2025):

add 'sandbox' to NO_PROXY in specific service or shared config

kibana:
    image: docker.elastic.co/kibana/kibana:8.14.3
    container_name: kibana
    profiles:
      - elasticsearch
    depends_on:
      - elasticsearch
    restart: always
    environment:
      XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY: d1a66dfd-c4d3-4a0a-8290-2abcb83ab3aa
      NO_PROXY: localhost,127.0.0.1,elasticsearch,kibana

or

x-shared-env: &shared-api-worker-env
  SSRF_PROXY_HTTP_URL: "http://localhost:8888"
  SSRF_PROXY_HTTPS_URL: "http://localhost:8888"
  HTTP_PROXY: "http://localhost:8888"
  HTTPS_PROXY: "http://localhost:8888"
  NO_PROXY: "localhost,127.0.0.1,weaviate,qdrand,db,redis,web,worker,plugin_daemon,sandbox"
@BGMer7 commented on GitHub (Mar 31, 2025): add 'sandbox' to NO_PROXY in specific service or shared config ```yaml kibana: image: docker.elastic.co/kibana/kibana:8.14.3 container_name: kibana profiles: - elasticsearch depends_on: - elasticsearch restart: always environment: XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY: d1a66dfd-c4d3-4a0a-8290-2abcb83ab3aa NO_PROXY: localhost,127.0.0.1,elasticsearch,kibana ``` or ```yaml x-shared-env: &shared-api-worker-env SSRF_PROXY_HTTP_URL: "http://localhost:8888" SSRF_PROXY_HTTPS_URL: "http://localhost:8888" HTTP_PROXY: "http://localhost:8888" HTTPS_PROXY: "http://localhost:8888" NO_PROXY: "localhost,127.0.0.1,weaviate,qdrand,db,redis,web,worker,plugin_daemon,sandbox" ```
Author
Owner

@mirhet commented on GitHub (Aug 11, 2025):

add 'sandbox' to NO_PROXY in specific service or shared config

NO_PROXY: "localhost,127.0.0.1,weaviate,qdrand,db,redis,web,worker,plugin_daemon,sandbox"

This didn't work for me. Any ideas?

@mirhet commented on GitHub (Aug 11, 2025): > add 'sandbox' to NO_PROXY in specific service or shared config ` NO_PROXY: "localhost,127.0.0.1,weaviate,qdrand,db,redis,web,worker,plugin_daemon,sandbox"` This didn't work for me. Any ideas?
Author
Owner

@zgldh commented on GitHub (Aug 23, 2025):

add 'sandbox' to NO_PROXY in specific service or shared config

NO_PROXY: "localhost,127.0.0.1,weaviate,qdrand,db,redis,web,worker,plugin_daemon,sandbox"

This didn't work for me. Any ideas?

Try no_proxy: .... The lower case.

@zgldh commented on GitHub (Aug 23, 2025): > > add 'sandbox' to NO_PROXY in specific service or shared config > > ` NO_PROXY: "localhost,127.0.0.1,weaviate,qdrand,db,redis,web,worker,plugin_daemon,sandbox"` > > This didn't work for me. Any ideas? Try `no_proxy: ....` The lower case.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#6385