langgenius/dify-plugin-daemon
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-daemon.git synced 2026-07-21 17:25:23 -04:00
Code Issues 111 Packages Projects Releases 10 Wiki Activity

Plugin installation fails in corporate proxy environment due to untrusted CA #131

New Issue
Closed
opened 2026-02-16 00:20:01 -05:00 by yindo · 6 comments
yindo commented 2026-02-16 00:20:01 -05:00
Owner
Copy Link
Copy Source

Originally created by @Yukawa111 on GitHub (Jun 5, 2025).

Originally assigned to: @Yeuoly on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

api: 1.4.1 plugin_daemon: 0.1.1-local

Cloud or Self Hosted

Self Hosted (Source), Self Hosted (Docker)

Steps to reproduce

1. Environment

  • OS: macOS
  • Docker: Rancher Desktop
  • Network: Behind a corporate proxy (Netskope) that intercepts all HTTPS traffic using a custom root CA.

2. Problem Description

When trying to install any plugin from the marketplace (e.g., Azure OpenAI), the installation fails with the error: Reached maximum retries (3) for URL https://marketplace.dify.ai/api/v1/plugins/download?...

This is caused by an SSL certificate verification error because the container does not trust the corporate proxy's root CA.

3. Extensive Troubleshooting Steps Taken

We have tried extensive steps to resolve this by installing the custom CA certificate into the containers. The key steps include:

  • Creating a custom entrypoint script for the api, worker, and plugin_daemon services in docker-compose.yaml.
  • The script successfully copies the custom CA (.pem file) to /usr/local/share/ca-certificates/custom_certs/ and renames it with a .crt extension.
  • The script then runs update-ca-certificates.
  • The logs for all three services (api, worker, plugin_daemon) confirm that the certificate was successfully added (1 added, 0 removed; done.).
  • We also configured the following environment variables for the plugin_daemon service, pointing to the newly installed certificate:
    • PIP_CERT
    • REQUESTS_CA_BUNDLE
    • CURL_CA_BUNDLE
    • SSL_CERT_FILE
  • We also resolved internal networking and database issues (dify_plugin database creation).

4. Final Diagnosis: The Container Environment is Correctly Configured

After all the above fixes, the Dify application still fails with the same error.

However, a manual test from within the plugin_daemon container proves that the container's OS and networking stack are correctly configured and trust the custom CA.

The following command, executed inside the plugin_daemon container, succeeds perfectly:

docker exec -it <plugin_daemon_container> curl -v https://marketplace.dify.ai

Output of successful curl -v command:

Host marketplace.dify.ai:443 was resolved.
IPv4: 104.26.9.156, 104.26.8.156, 172.67.71.187
Trying 104.26.9.156:443...
Connected to marketplace.dify.ai (104.26.9.156) port 443
ALPN: curl offers h2,http/1.1
CAfile: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt
CApath: /etc/ssl/certs ...
issuer: emailAddress=certadmin@netskope.com; CN=ca.ppcd-netskope-tenant.goskope.com; O=PayPay Card Corporation; L=Chiyoda; ST=TYO; C=JP
SSL certificate verify ok. ... < HTTP/1.1 200 OK ... (HTML content follows)

Conclusion

The curl test proves that the container's system-level trust store is working correctly. The issue seems to be that the plugin_daemon Go application is not using the system's CA certificate store and is therefore unable to verify the HTTPS connection through our corporate proxy.

Could you please advise if there is a specific, undocumented environment variable or another method to make the Dify Go application trust a custom CA certificate? Thank you.

✔️ Expected Behavior

After correctly configuring the container environment to trust our corporate proxy's custom CA certificate, I expected to be able to successfully install plugins (e.g., Azure OpenAI) from the Dify marketplace. The download should complete without any network or SSL/TLS errors.

❌ Actual Behavior

The plugin installation fails with a timeout error. The UI shows an error toast with the message: "Reached maximum retries (3) for URL https://marketplace.dify.ai/api/v1/plugins/download?..."

This failure occurs even though the container's OS-level tools (like curl) can connect to this URL successfully with proper certificate verification, as proven in the "Steps to reproduce" section above.

Originally created by @Yukawa111 on GitHub (Jun 5, 2025). Originally assigned to: @Yeuoly on GitHub. ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version api: 1.4.1 plugin_daemon: 0.1.1-local ### Cloud or Self Hosted Self Hosted (Source), Self Hosted (Docker) ### Steps to reproduce ### 1. Environment - **OS:** macOS - **Docker:** Rancher Desktop - **Network:** Behind a corporate proxy (Netskope) that intercepts all HTTPS traffic using a custom root CA. ### 2. Problem Description When trying to install any plugin from the marketplace (e.g., Azure OpenAI), the installation fails with the error: `Reached maximum retries (3) for URL https://marketplace.dify.ai/api/v1/plugins/download?...` This is caused by an SSL certificate verification error because the container does not trust the corporate proxy's root CA. ### 3. Extensive Troubleshooting Steps Taken We have tried extensive steps to resolve this by installing the custom CA certificate into the containers. The key steps include: - Creating a custom `entrypoint` script for the `api`, `worker`, and `plugin_daemon` services in `docker-compose.yaml`. - The script successfully copies the custom CA (`.pem` file) to `/usr/local/share/ca-certificates/custom_certs/` and renames it with a `.crt` extension. - The script then runs `update-ca-certificates`. - The logs for all three services (`api`, `worker`, `plugin_daemon`) confirm that the certificate was successfully added (`1 added, 0 removed; done.`). - We also configured the following environment variables for the `plugin_daemon` service, pointing to the newly installed certificate: - `PIP_CERT` - `REQUESTS_CA_BUNDLE` - `CURL_CA_BUNDLE` - `SSL_CERT_FILE` - We also resolved internal networking and database issues (`dify_plugin` database creation). ### 4. Final Diagnosis: The Container Environment is Correctly Configured After all the above fixes, the Dify application still fails with the same error. However, a manual test from within the `plugin_daemon` container proves that the container's OS and networking stack **are correctly configured and trust the custom CA**. The following command, executed inside the `plugin_daemon` container, **succeeds perfectly**: `docker exec -it <plugin_daemon_container> curl -v https://marketplace.dify.ai` **Output of successful `curl -v` command:** Host marketplace.dify.ai:443 was resolved. IPv4: 104.26.9.156, 104.26.8.156, 172.67.71.187 Trying 104.26.9.156:443... Connected to marketplace.dify.ai (104.26.9.156) port 443 ALPN: curl offers h2,http/1.1 CAfile: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt CApath: /etc/ssl/certs ... issuer: emailAddress=certadmin@netskope.com; CN=ca.ppcd-netskope-tenant.goskope.com; O=PayPay Card Corporation; L=Chiyoda; ST=TYO; C=JP SSL certificate verify ok. ... &lt; HTTP/1.1 200 OK ... (HTML content follows) <!-- end list --> ### Conclusion The `curl` test proves that the container's system-level trust store is working correctly. The issue seems to be that the `plugin_daemon` Go application is **not using the system's CA certificate store** and is therefore unable to verify the HTTPS connection through our corporate proxy. Could you please advise if there is a specific, undocumented environment variable or another method to make the Dify Go application trust a custom CA certificate? Thank you. ### ✔️ Expected Behavior After correctly configuring the container environment to trust our corporate proxy's custom CA certificate, I expected to be able to successfully install plugins (e.g., Azure OpenAI) from the Dify marketplace. The download should complete without any network or SSL/TLS errors. ### ❌ Actual Behavior The plugin installation fails with a timeout error. The UI shows an error toast with the message: "Reached maximum retries (3) for URL https://marketplace.dify.ai/api/v1/plugins/download?..." This failure occurs even though the container's OS-level tools (like `curl`) can connect to this URL successfully with proper certificate verification, as proven in the "Steps to reproduce" section above.
yindo closed this issue 2026-02-16 00:20:01 -05:00
yindo commented 2026-02-16 00:20:02 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jun 5, 2025):

Hi @Yukawa111! I'm Dosu and I’m helping the dify team.

Based on what I found, there isn't a documented or undocumented environment variable in Dify to make the Go-based plugin_daemon trust a custom CA certificate. The Dify codebase does not expose Go source or provide a way to override or inject custom CA certs for the Go application; it relies on the system CA store bundled with the Docker image used for plugin_daemon and sandbox services. All SSL/TLS configuration in the repo is for NGINX as a reverse proxy, not for the Go containers themselves, and the plugin_daemon service's environment variables do not include any for custom CA handling reference.

If your custom CA is present in /etc/ssl/certs/ca-certificates.crt inside the container, but the Go application still doesn't trust it, it's likely because the base image for langgenius/dify-plugin-daemon was built without your CA. In this case, you may need to build a custom image for plugin_daemon that includes your CA at build time, ensuring the Go runtime picks it up. There is no Dify-specific override for this; it's a limitation of how Go statically loads CA certs from the system at process start.

If you need more detailed steps on how to build a custom Docker image for plugin_daemon with your CA, let me know!

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Join Discord Share on X

@dosubot[bot] commented on GitHub (Jun 5, 2025): <!-- Greeting --> Hi @Yukawa111! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the dify team. <!-- Answer --> Based on what I found, there isn't a documented or undocumented environment variable in Dify to make the Go-based plugin_daemon trust a custom CA certificate. The Dify codebase does not expose Go source or provide a way to override or inject custom CA certs for the Go application; it relies on the system CA store bundled with the Docker image used for plugin_daemon and sandbox services. All SSL/TLS configuration in the repo is for NGINX as a reverse proxy, not for the Go containers themselves, and the plugin_daemon service's environment variables do not include any for custom CA handling [reference](https://github.com/langgenius/dify/blob/85859b67237c28dbec67f73b7c8e7e8484c62800/docker/docker-compose.yaml). If your custom CA is present in `/etc/ssl/certs/ca-certificates.crt` inside the container, but the Go application still doesn't trust it, it's likely because the base image for `langgenius/dify-plugin-daemon` was built without your CA. In this case, you may need to build a custom image for plugin_daemon that includes your CA at build time, ensuring the Go runtime picks it up. There is no Dify-specific override for this; it's a limitation of how Go statically loads CA certs from the system at process start. If you need more detailed steps on how to build a custom Docker image for plugin_daemon with your CA, let me know! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/92826ae8-b73c-40f9-8c27-29e9f4c7eb95?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/92826ae8-b73c-40f9-8c27-29e9f4c7eb95?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/92826ae8-b73c-40f9-8c27-29e9f4c7eb95?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/92826ae8-b73c-40f9-8c27-29e9f4c7eb95?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/92826ae8-b73c-40f9-8c27-29e9f4c7eb95?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/92826ae8-b73c-40f9-8c27-29e9f4c7eb95?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/92826ae8-b73c-40f9-8c27-29e9f4c7eb95?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/20722)
yindo commented 2026-02-16 00:20:02 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Jun 6, 2025):

https://github.com/langgenius/dify/blob/92614765ff280bbc5e7958abef2ff9c9e9e05cb4/docker/docker-compose.yaml#L660

Can you provide the env settings regarding the plugin_daemon.

@crazywoola commented on GitHub (Jun 6, 2025): https://github.com/langgenius/dify/blob/92614765ff280bbc5e7958abef2ff9c9e9e05cb4/docker/docker-compose.yaml#L660 Can you provide the env settings regarding the `plugin_daemon`.
yindo commented 2026-02-16 00:20:02 -05:00
Author
Owner
Copy Link
Copy Source

@Yukawa111 commented on GitHub (Jun 6, 2025):

Hello, thank you for your previous suggestions.

I have now tried every possible solution, including building a custom Docker image for the plugin_daemon service as you recommended.

Final Attempt: Building a Custom Image

  1. I created a Dockerfile for plugin_daemon that uses langgenius/dify-plugin-daemon:0.1.1-local as a base.
  2. The Dockerfile COPYs our corporate root CA (.pem file) into the image and then RUNs update-ca-certificates.
  3. The docker compose up -d --build command completes successfully. The build log confirms that the COPY and RUN update-ca-certificates steps are executed flawlessly.
  4. The docker-compose.yaml for the plugin_daemon was updated to use build: ... instead of image:.
  5. All other services (api, worker) were also configured with a runtime entrypoint script to install the certificate, and their logs confirm 1 added, 0 removed; done.

The Result: Still Failing

Even with the certificate baked into the plugin_daemon image at build time, the application still fails with the exact same error: Reached maximum retries (3) for URL https://marketplace.dify.ai/api/v1/plugins/download?...

The logs for the plugin_daemon container now show a clean startup, with no certificate script running (as expected), but the UI error persists.

Final Conclusion

This, combined with the successful curl -v test from my original report, is definitive proof that the Dify plugin_daemon Go application is fundamentally ignoring the operating system's certificate trust store, no matter how or when the custom CA is installed.

This appears to be a hard limitation or a bug in the application that makes it unusable in any environment that requires a custom root CA for HTTPS inspection.

Could the development team please acknowledge this issue and suggest a workaround or a plan for a fix? Without a solution, we cannot use Dify's plugin features.

I have a major update. This is the final piece of evidence.

I temporarily disabled my corporate proxy (Netskope), and the plugin installation SUCCEEDED.

This definitively proves that the root cause of the Reached maximum retries error is the SSL certificate interception by the proxy.

The current situation is:

  • With Netskope ON: The plugin installation fails due to a timeout, even though the container's OS has the custom CA installed and curl -v works perfectly.
  • With Netskope OFF: The plugin installation succeeds. (A new, secondary error No such file or directory appears when configuring the API key, but this is a separate issue that only occurs after a successful installation).

This confirms that the plugin_daemon Go application is ignoring the system's CA trust store and is unable to function behind a corporate proxy that performs HTTPS inspection.

Could the development team please investigate this and provide a solution? Without a fix, it seems Dify cannot be used in such network environments. Thank you.

@Yukawa111 commented on GitHub (Jun 6, 2025): Hello, thank you for your previous suggestions. I have now tried every possible solution, including building a custom Docker image for the `plugin_daemon` service as you recommended. **Final Attempt: Building a Custom Image** 1. I created a `Dockerfile` for `plugin_daemon` that uses `langgenius/dify-plugin-daemon:0.1.1-local` as a base. 2. The `Dockerfile` `COPY`s our corporate root CA (`.pem` file) into the image and then `RUN`s `update-ca-certificates`. 3. The `docker compose up -d --build` command completes successfully. The build log confirms that the `COPY` and `RUN update-ca-certificates` steps are executed flawlessly. 4. The `docker-compose.yaml` for the `plugin_daemon` was updated to use `build: ...` instead of `image:`. 5. All other services (`api`, `worker`) were also configured with a runtime `entrypoint` script to install the certificate, and their logs confirm `1 added, 0 removed; done.` **The Result: Still Failing** Even with the certificate baked into the `plugin_daemon` image at build time, the application **still fails** with the exact same error: `Reached maximum retries (3) for URL https://marketplace.dify.ai/api/v1/plugins/download?...` The logs for the `plugin_daemon` container now show a clean startup, with no certificate script running (as expected), but the UI error persists. **Final Conclusion** This, combined with the successful `curl -v` test from my original report, is definitive proof that **the Dify `plugin_daemon` Go application is fundamentally ignoring the operating system's certificate trust store**, no matter how or when the custom CA is installed. This appears to be a hard limitation or a bug in the application that makes it unusable in any environment that requires a custom root CA for HTTPS inspection. Could the development team please acknowledge this issue and suggest a workaround or a plan for a fix? Without a solution, we cannot use Dify's plugin features. I have a major update. This is the final piece of evidence. **I temporarily disabled my corporate proxy (Netskope), and the plugin installation SUCCEEDED.** This definitively proves that the root cause of the `Reached maximum retries` error is the SSL certificate interception by the proxy. The current situation is: - **With Netskope ON:** The plugin installation fails due to a timeout, even though the container's OS has the custom CA installed and `curl -v` works perfectly. - **With Netskope OFF:** The plugin installation succeeds. (A new, secondary error `No such file or directory` appears when configuring the API key, but this is a separate issue that only occurs after a successful installation). This confirms that the `plugin_daemon` Go application is ignoring the system's CA trust store and is unable to function behind a corporate proxy that performs HTTPS inspection. Could the development team please investigate this and provide a solution? Without a fix, it seems Dify cannot be used in such network environments. Thank you.
yindo commented 2026-02-16 00:20:02 -05:00
Author
Owner
Copy Link
Copy Source

@Yukawa111 commented on GitHub (Jun 6, 2025):

dify/docker/docker-compose.yaml

Line 660 in 9261476

PPROF_ENABLED: ${PLUGIN_PPROF_ENABLED:-false}
Can you provide the env settings regarding the plugin_daemon.

Hi @crazywoola,

Thank you for looking into my issue. Here are the final environment settings for the plugin_daemon service from my docker-compose.yaml, including the variables we added to try and fix the CA issue.

YAML

environment:
  <<: *shared-api-worker-env
  DB_DATABASE: ${DB_PLUGIN_DATABASE:-dify_plugin}
  SERVER_PORT: ${PLUGIN_DAEMON_PORT:-5002}
  # ... (some variables are omitted for brevity) ...
  PYTHON_ENV_INIT_TIMEOUT: ${PLUGIN_PYTHON_ENV_INIT_TIMEOUT:-120}
  PLUGIN_MAX_EXECUTION_TIMEOUT: ${PLUGIN_MAX_EXECUTION_TIMEOUT:-600}
  PIP_MIRROR_URL: ${PIP_MIRROR_URL:-}
  # --- Custom CA Configuration ---
  PIP_CERT: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt
  REQUESTS_CA_BUNDLE: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt
  CURL_CA_BUNDLE: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt
  SSL_CERT_FILE: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt

However, I have a major update which I posted in a separate comment just now. I believe this new information is the key to solving this issue.

In summary, I was able to prove that:

A manual curl -v from inside the plugin_daemon container works perfectly and verifies the custom CA.
Temporarily disabling my corporate proxy allows the plugin installation to succeed.
This strongly suggests the issue is not with my environment variables, but with the plugin_daemon Go application itself ignoring the system's CA trust store.

Could you please take a look at my latest comment with the full details?

Thank you again for your help!

@Yukawa111 commented on GitHub (Jun 6, 2025): > [dify/docker/docker-compose.yaml](https://github.com/langgenius/dify/blob/92614765ff280bbc5e7958abef2ff9c9e9e05cb4/docker/docker-compose.yaml#L660) > > Line 660 in [9261476](/langgenius/dify/commit/92614765ff280bbc5e7958abef2ff9c9e9e05cb4) > > PPROF_ENABLED: ${PLUGIN_PPROF_ENABLED:-false} > Can you provide the env settings regarding the `plugin_daemon`. Hi @crazywoola, Thank you for looking into my issue. Here are the final environment settings for the plugin_daemon service from my docker-compose.yaml, including the variables we added to try and fix the CA issue. YAML environment: <<: *shared-api-worker-env DB_DATABASE: ${DB_PLUGIN_DATABASE:-dify_plugin} SERVER_PORT: ${PLUGIN_DAEMON_PORT:-5002} # ... (some variables are omitted for brevity) ... PYTHON_ENV_INIT_TIMEOUT: ${PLUGIN_PYTHON_ENV_INIT_TIMEOUT:-120} PLUGIN_MAX_EXECUTION_TIMEOUT: ${PLUGIN_MAX_EXECUTION_TIMEOUT:-600} PIP_MIRROR_URL: ${PIP_MIRROR_URL:-} # --- Custom CA Configuration --- PIP_CERT: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt REQUESTS_CA_BUNDLE: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt CURL_CA_BUNDLE: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt SSL_CERT_FILE: /usr/local/share/ca-certificates/custom_certs/netskope-cert-bundle.crt However, I have a major update which I posted in a separate comment just now. I believe this new information is the key to solving this issue. In summary, I was able to prove that: A manual curl -v from inside the plugin_daemon container works perfectly and verifies the custom CA. Temporarily disabling my corporate proxy allows the plugin installation to succeed. This strongly suggests the issue is not with my environment variables, but with the plugin_daemon Go application itself ignoring the system's CA trust store. Could you please take a look at my latest comment with the full details? Thank you again for your help!
yindo commented 2026-02-16 00:20:02 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Jun 6, 2025):

@Yukawa111 Thanks for the detailed investigation.

I will transfer this issue to this repo instead.

I think we might need some insights from @Yeuoly

@crazywoola commented on GitHub (Jun 6, 2025): @Yukawa111 Thanks for the detailed investigation. I will transfer this issue to this repo instead. I think we might need some insights from @Yeuoly
yindo commented 2026-02-16 00:20:02 -05:00
Author
Owner
Copy Link
Copy Source

@Yukawa111 commented on GitHub (Jun 6, 2025):

Hello @dosu ,@crazywoola and @Yeuoly

Success! I have finally resolved all the issues.

Thank you so much for your time and suggestions. I was able to find the final pieces of the puzzle and get Dify working perfectly behind our corporate proxy.

For anyone else facing this issue in the future, here is the complete, working solution. The key was that there were two separate problems that required two different solutions: one for the Go-based plugin_daemon and another for the Python-based api and worker services.

Final Solution
Problem 1: plugin_daemon (Go service) ignores runtime CA updates.
As @dosu correctly pointed out, this service seems to load its CAs at build time. The solution was to build a custom image.

  1. Create custom-plugin-daemon/Dockerfile:
    This file bakes the custom CA certificate into the image.

Dockerfile

Base image

FROM langgenius/dify-plugin-daemon:0.1.1-local

Copy the real certificate PEM file into the image as a .crt file

COPY ./netskope-cert-bundle.pem /usr/local/share/ca-certificates/netskope-cert-bundle.crt

Run update-ca-certificates during the image build

RUN update-ca-certificates
(You also need to place netskope-cert-bundle.pem inside the custom-plugin-daemon folder.)

  1. Modify docker-compose.yaml for plugin_daemon:
    This uses the new Dockerfile to build the image and removes the now-unnecessary entrypoint and certificate volume.

YAML

  plugin_daemon:
    build:
      context: ./custom-plugin-daemon
    restart: always
    environment:
      # ... (environment variables, including the ones for cert paths, remain) ...
    volumes:
      - ./volumes/plugin_daemon:/app/storage
    depends_on:
      db:
        condition: service_healthy
    networks:
      - default

Problem 2: api & worker (Python services) ignore the OS trust store and use certifi.
My logs revealed that these services were ignoring the system CA store (even after update-ca-certificates succeeded) and were exclusively using the CA bundle provided by the Python certifi package.

  1. Create custom-api-entrypoint/entrypoint-wrapper.sh:
    This script updates both the OS store AND appends the custom CA to the certifi bundle at container startup.

Bash

#!/bin/bash
set -e

echo "--- [Dify Custom Entrypoint Wrapper] Starting..."
CUSTOM_CERT_SRC="/tmp/my_host_certs/netskope-cert-bundle.crt"

echo "Waiting 5 seconds for volume mount..."
sleep 5

if [ -f "$CUSTOM_CERT_SRC" ]; then
    echo "Custom CA found. Updating stores..."
    
    # 1. Update OS store (for general tools like curl)
    cp "$CUSTOM_CERT_SRC" /usr/local/share/ca-certificates/
    update-ca-certificates
    
    # 2. Update Python's certifi store (CRITICAL STEP)
    PYTHON_CERT_BUNDLE=$(python -c 'import certifi; print(certifi.where())')
    if [ -f "$PYTHON_CERT_BUNDLE" ]; then
        if ! grep -qF "PayPay Card Corporation" "$PYTHON_CERT_BUNDLE"; then
             echo "Appending CA to Python's certifi store: $PYTHON_CERT_BUNDLE"
             cat "$CUSTOM_CERT_SRC" >> "$PYTHON_CERT_BUNDLE"
        fi
    fi
else
    echo "WARNING: Custom certificate not found. Skipping setup."
fi

echo "Executing original application command: $@"
exec /bin/bash /entrypoint.sh "$@"
  1. Modify docker-compose.yaml for api and worker:
    This uses the wrapper script as the new entrypoint.

YAML

  api: # and 'worker' service
    image: langgenius/dify-api:1.4.1
    # ...
    volumes:
      - ./certs:/tmp/my_host_certs:ro
      - ./custom-api-entrypoint/entrypoint-wrapper.sh:/entrypoint-wrapper.sh
      - app-storage:/app/api/storage
    entrypoint: ["/bin/bash", "/entrypoint-wrapper.sh"]
    # ...

With these combined solutions, Dify is now fully functional behind our corporate proxy. Plugin installation and model provider configuration both work flawlessly.

Thank you again for your guidance, which pointed me in the right direction. I believe this issue can now be closed.

@Yukawa111 commented on GitHub (Jun 6, 2025): Hello @dosu ,@crazywoola and @Yeuoly Success! I have finally resolved all the issues. Thank you so much for your time and suggestions. I was able to find the final pieces of the puzzle and get Dify working perfectly behind our corporate proxy. For anyone else facing this issue in the future, here is the complete, working solution. The key was that there were two separate problems that required two different solutions: one for the Go-based plugin_daemon and another for the Python-based api and worker services. Final Solution Problem 1: plugin_daemon (Go service) ignores runtime CA updates. As @dosu correctly pointed out, this service seems to load its CAs at build time. The solution was to build a custom image. 1. Create custom-plugin-daemon/Dockerfile: This file bakes the custom CA certificate into the image. Dockerfile # Base image FROM langgenius/dify-plugin-daemon:0.1.1-local # Copy the real certificate PEM file into the image as a .crt file COPY ./netskope-cert-bundle.pem /usr/local/share/ca-certificates/netskope-cert-bundle.crt # Run update-ca-certificates during the image build RUN update-ca-certificates (You also need to place netskope-cert-bundle.pem inside the custom-plugin-daemon folder.) 2. Modify docker-compose.yaml for plugin_daemon: This uses the new Dockerfile to build the image and removes the now-unnecessary entrypoint and certificate volume. YAML ``` plugin_daemon: build: context: ./custom-plugin-daemon restart: always environment: # ... (environment variables, including the ones for cert paths, remain) ... volumes: - ./volumes/plugin_daemon:/app/storage depends_on: db: condition: service_healthy networks: - default ``` Problem 2: api & worker (Python services) ignore the OS trust store and use certifi. My logs revealed that these services were ignoring the system CA store (even after update-ca-certificates succeeded) and were exclusively using the CA bundle provided by the Python certifi package. 1. Create custom-api-entrypoint/entrypoint-wrapper.sh: This script updates both the OS store AND appends the custom CA to the certifi bundle at container startup. Bash ``` #!/bin/bash set -e echo "--- [Dify Custom Entrypoint Wrapper] Starting..." CUSTOM_CERT_SRC="/tmp/my_host_certs/netskope-cert-bundle.crt" echo "Waiting 5 seconds for volume mount..." sleep 5 if [ -f "$CUSTOM_CERT_SRC" ]; then echo "Custom CA found. Updating stores..." # 1. Update OS store (for general tools like curl) cp "$CUSTOM_CERT_SRC" /usr/local/share/ca-certificates/ update-ca-certificates # 2. Update Python's certifi store (CRITICAL STEP) PYTHON_CERT_BUNDLE=$(python -c 'import certifi; print(certifi.where())') if [ -f "$PYTHON_CERT_BUNDLE" ]; then if ! grep -qF "PayPay Card Corporation" "$PYTHON_CERT_BUNDLE"; then echo "Appending CA to Python's certifi store: $PYTHON_CERT_BUNDLE" cat "$CUSTOM_CERT_SRC" >> "$PYTHON_CERT_BUNDLE" fi fi else echo "WARNING: Custom certificate not found. Skipping setup." fi echo "Executing original application command: $@" exec /bin/bash /entrypoint.sh "$@" ``` 2. Modify docker-compose.yaml for api and worker: This uses the wrapper script as the new entrypoint. YAML ``` api: # and 'worker' service image: langgenius/dify-api:1.4.1 # ... volumes: - ./certs:/tmp/my_host_certs:ro - ./custom-api-entrypoint/entrypoint-wrapper.sh:/entrypoint-wrapper.sh - app-storage:/app/api/storage entrypoint: ["/bin/bash", "/entrypoint-wrapper.sh"] # ... ``` With these combined solutions, Dify is now fully functional behind our corporate proxy. Plugin installation and model provider configuration both work flawlessly. Thank you again for your guidance, which pointed me in the right direction. I believe this issue can now be closed.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Windows
bug
cmd
documentation
enhancement
go
good first issue
plugin-daemon
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-daemon#131
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?