langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

How does plugin-daemon use my own CA certificates? #360

New Issue
Closed
opened 2026-02-16 10:19:07 -05:00 by yindo · 3 comments
yindo commented 2026-02-16 10:19:07 -05:00
Owner
Copy Link
Copy Source

Originally created by @skylinenzo on GitHub (Jun 11, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.3.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Hi, I have tried to deploy an air-gapped Dify behind proxy.

When I add Volcengine models with plugin, I encountered constantly 400 error from api server, as it says:

plugin_daemon-1  | [GIN] 2025/06/11 - 02:12:10 | 200 |  2.764494159s |      172.18.0.8 | POST     "/plugin/9bb4614f-6392-4b2a-914b-64c95af47c16/dispatch/model/validate_model_credentials"
api-1            | 2025-06-11 02:12:10.696 DEBUG [Thread-56 (process_request_thread)] [connectionpool.py:544] - http://plugin_daemon:5002 "POST /plugin/9bb4614f-6392-4b2a-914b-64c95af47c16/dispatch/model/validate_model_credentials HTTP/1.1" 200 None
api-1            | 2025-06-11 02:12:10.701 ERROR [Thread-56 (process_request_thread)] [models.py:162] - Failed to save model credentials, tenant_id: 9bb4614f-6392-4b2a-914b-64c95af47c16, model: doubao-1.5-pro, model_type: llm
api-1            | Traceback (most recent call last):
api-1            |   File "/app/api/controllers/console/workspace/models.py", line 154, in post
api-1            |     model_provider_service.save_model_credentials(
api-1            |   File "/app/api/services/model_provider_service.py", line 237, in save_model_credentials
api-1            |     provider_configuration.add_or_update_custom_model_credentials(
api-1            |   File "/app/api/core/entities/provider_configuration.py", line 422, in add_or_update_custom_model_credentials
api-1            |     provider_model_record, credentials = self.custom_model_credentials_validate(model_type, model, credentials)
api-1            |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/app/api/core/entities/provider_configuration.py", line 402, in custom_model_credentials_validate
api-1            |     credentials = model_provider_factory.model_credentials_validate(
api-1            |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
api-1            |   File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 191, in model_credentials_validate
api-1            |     self.plugin_model_manager.validate_model_credentials(
api-1            |   File "/app/api/core/plugin/impl/model.py", line 137, in validate_model_credentials
api-1            |     for resp in response:
api-1            |                 ^^^^^^^^
api-1            |   File "/app/api/core/plugin/impl/base.py", line 191, in _request_with_plugin_daemon_response_stream
api-1            |     self._handle_plugin_daemon_error(error.error_type, error.message)
api-1            |   File "/app/api/core/plugin/impl/base.py", line 221, in _handle_plugin_daemon_error
api-1            |     raise CredentialsValidateFailedError(error_object.get("message"))
api-1            | core.model_runtime.errors.validate.CredentialsValidateFailedError: Connection error., request_id: 20250611021210uqmHYMZyBJ6jnPTZGiMu
api-1            | 2025-06-11 02:12:10.705 INFO [Thread-56 (process_request_thread)] [_internal.py:97] - 172.18.0.10 - - [11/Jun/2025 02:12:10] "POST /console/api/workspaces/current/model-providers/langgenius/volcengine_maas/volcengine_maas/models HTTP/1.1" 400 -

I set both extra env, volumes in docker-compose.yaml to apply the proxy settings and mount the CA certificates for the proxy:

      HTTP_PROXY: http://proxy:port
      HTTPS_PROXY: http://proxy:port
      NO_PROXY: ssrf_proxy,api,weaviate,gdrant,db,redis,web,worker,sandbox,plugin_daemon,localhost,127.0.0.1,172.19.*.*
    volumes:
      - /etc/pki/tls/certs/ca-bundle.crt:/usr/local/share/ca-certificates/root_cert.crt
      - ./volumes/plugin_daemon:/app/storage

I run update-ca-certificates and commit the API and plugin-daemon container locally, so next time I run this local container, it should has all my own CA certificates loaded.

I am pretty sure the proxy is well set in both API and plugin-daemon container, as I can access to Internet and both API of volcengine with curl.

However, when I click on the Dify front end to set the model, it still return 400 error, so I use tcpdump and it says still unknown CA and reset the connection. And with Wireshark, I found the plugin-daemon tries to access the Internet with this HTTP packet:

Hypertext Transfer Protocol
    CONNECT ark.cn-beijing.volces.com:443 HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): CONNECT ark.cn-beijing.volces.com:443 HTTP/1.1\r\n]
            [CONNECT ark.cn-beijing.volces.com:443 HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: CONNECT
        Request URI: ark.cn-beijing.volces.com:443
        Request Version: HTTP/1.1
    Host: ark.cn-beijing.volces.com:443\r\n
    Accept: */*\r\n
    \r\n
    [Full request URI: ark.cn-beijing.volces.com:443]
    [HTTP request 1/1]
    [Response in frame: 43]

My guess on the reason caused the problem is the plugin-daemon is accessing the internet with HTTP at 443 rather than HTTPS? That caused the CA certificates didn't mounted properly

✔️ Expected Behavior

The volcengine model should be properly added to Dify by plugin.

❌ Actual Behavior

See above.

Originally created by @skylinenzo on GitHub (Jun 11, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.3.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Hi, I have tried to deploy an air-gapped Dify behind proxy. When I add Volcengine models with plugin, I encountered constantly 400 error from api server, as it says: ``` plugin_daemon-1 | [GIN] 2025/06/11 - 02:12:10 | 200 | 2.764494159s | 172.18.0.8 | POST "/plugin/9bb4614f-6392-4b2a-914b-64c95af47c16/dispatch/model/validate_model_credentials" api-1 | 2025-06-11 02:12:10.696 DEBUG [Thread-56 (process_request_thread)] [connectionpool.py:544] - http://plugin_daemon:5002 "POST /plugin/9bb4614f-6392-4b2a-914b-64c95af47c16/dispatch/model/validate_model_credentials HTTP/1.1" 200 None api-1 | 2025-06-11 02:12:10.701 ERROR [Thread-56 (process_request_thread)] [models.py:162] - Failed to save model credentials, tenant_id: 9bb4614f-6392-4b2a-914b-64c95af47c16, model: doubao-1.5-pro, model_type: llm api-1 | Traceback (most recent call last): api-1 | File "/app/api/controllers/console/workspace/models.py", line 154, in post api-1 | model_provider_service.save_model_credentials( api-1 | File "/app/api/services/model_provider_service.py", line 237, in save_model_credentials api-1 | provider_configuration.add_or_update_custom_model_credentials( api-1 | File "/app/api/core/entities/provider_configuration.py", line 422, in add_or_update_custom_model_credentials api-1 | provider_model_record, credentials = self.custom_model_credentials_validate(model_type, model, credentials) api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-1 | File "/app/api/core/entities/provider_configuration.py", line 402, in custom_model_credentials_validate api-1 | credentials = model_provider_factory.model_credentials_validate( api-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-1 | File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 191, in model_credentials_validate api-1 | self.plugin_model_manager.validate_model_credentials( api-1 | File "/app/api/core/plugin/impl/model.py", line 137, in validate_model_credentials api-1 | for resp in response: api-1 | ^^^^^^^^ api-1 | File "/app/api/core/plugin/impl/base.py", line 191, in _request_with_plugin_daemon_response_stream api-1 | self._handle_plugin_daemon_error(error.error_type, error.message) api-1 | File "/app/api/core/plugin/impl/base.py", line 221, in _handle_plugin_daemon_error api-1 | raise CredentialsValidateFailedError(error_object.get("message")) api-1 | core.model_runtime.errors.validate.CredentialsValidateFailedError: Connection error., request_id: 20250611021210uqmHYMZyBJ6jnPTZGiMu api-1 | 2025-06-11 02:12:10.705 INFO [Thread-56 (process_request_thread)] [_internal.py:97] - 172.18.0.10 - - [11/Jun/2025 02:12:10] "POST /console/api/workspaces/current/model-providers/langgenius/volcengine_maas/volcengine_maas/models HTTP/1.1" 400 - ``` I set both extra env, volumes in docker-compose.yaml to apply the proxy settings and mount the CA certificates for the proxy: ``` HTTP_PROXY: http://proxy:port HTTPS_PROXY: http://proxy:port NO_PROXY: ssrf_proxy,api,weaviate,gdrant,db,redis,web,worker,sandbox,plugin_daemon,localhost,127.0.0.1,172.19.*.* volumes: - /etc/pki/tls/certs/ca-bundle.crt:/usr/local/share/ca-certificates/root_cert.crt - ./volumes/plugin_daemon:/app/storage ``` I run ``` update-ca-certificates``` and commit the API and plugin-daemon container locally, so next time I run this local container, it should has all my own CA certificates loaded. I am pretty sure the proxy is well set in both API and plugin-daemon container, as I can access to Internet and both API of volcengine with curl. However, when I click on the Dify front end to set the model, it still return 400 error, so I use tcpdump and it says still unknown CA and reset the connection. And with Wireshark, I found the plugin-daemon tries to access the Internet with this HTTP packet: ``` Hypertext Transfer Protocol CONNECT ark.cn-beijing.volces.com:443 HTTP/1.1\r\n [Expert Info (Chat/Sequence): CONNECT ark.cn-beijing.volces.com:443 HTTP/1.1\r\n] [CONNECT ark.cn-beijing.volces.com:443 HTTP/1.1\r\n] [Severity level: Chat] [Group: Sequence] Request Method: CONNECT Request URI: ark.cn-beijing.volces.com:443 Request Version: HTTP/1.1 Host: ark.cn-beijing.volces.com:443\r\n Accept: */*\r\n \r\n [Full request URI: ark.cn-beijing.volces.com:443] [HTTP request 1/1] [Response in frame: 43] ``` My guess on the reason caused the problem is the plugin-daemon is accessing the internet with HTTP at 443 rather than HTTPS? That caused the CA certificates didn't mounted properly ### ✔️ Expected Behavior The volcengine model should be properly added to Dify by plugin. ### ❌ Actual Behavior See above.
yindo closed this issue 2026-02-16 10:19:07 -05:00
yindo commented 2026-02-16 10:19:07 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jun 11, 2025):

The plugin-daemon uses the Python requests library for outbound HTTP(S) connections, which by default relies on the system CA certificates inside the container. There is no explicit code or configuration in Dify to customize CA certificate handling for the plugin-daemon—so it will only trust CAs that are available to the Python environment at runtime. If you want the plugin-daemon to use your own CA certificates, you need to ensure:

  1. Your CA bundle is present in the container and recognized by the OS.
  2. The environment variable REQUESTS_CA_BUNDLE is set to the path of your CA bundle (e.g., /usr/local/share/ca-certificates/root_cert.crt) in the plugin-daemon container. This tells requests to use your custom CA bundle for all HTTPS requests.
  3. After mounting the CA file, run update-ca-certificates in the container to update the system trust store.

Your Docker Compose should include the volume mount and the REQUESTS_CA_BUNDLE environment variable. Example:

    environment:
      REQUESTS_CA_BUNDLE: /usr/local/share/ca-certificates/root_cert.crt
    volumes:
      - /etc/pki/tls/certs/ca-bundle.crt:/usr/local/share/ca-certificates/root_cert.crt

If you’ve already done this and still see unknown CA errors, double-check that the CA file is readable inside the running container and that the path matches exactly. You can also test with a simple Python script using requests inside the container to verify SSL connections.

The HTTP CONNECT method you see is normal for HTTPS over a proxy; it’s not an indication that the connection is downgraded to plain HTTP. The error likely means the CA bundle isn’t being picked up by the Python process, or the proxy is intercepting SSL and presenting a certificate not signed by your CA.

There is no Dify-specific configuration for CA certificates in the plugin-daemon; it relies entirely on the Python and OS-level trust settings. Any customization must be done via environment variables or by updating the container’s trust store.
Reference: plugin-daemon uses requests library and system CA certificates

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Join Discord Share on X

@dosubot[bot] commented on GitHub (Jun 11, 2025): <!-- Answer --> The plugin-daemon uses the Python `requests` library for outbound HTTP(S) connections, which by default relies on the system CA certificates inside the container. There is no explicit code or configuration in Dify to customize CA certificate handling for the plugin-daemon—so it will only trust CAs that are available to the Python environment at runtime. If you want the plugin-daemon to use your own CA certificates, you need to ensure: 1. Your CA bundle is present in the container and recognized by the OS. 2. The environment variable `REQUESTS_CA_BUNDLE` is set to the path of your CA bundle (e.g., `/usr/local/share/ca-certificates/root_cert.crt`) in the plugin-daemon container. This tells `requests` to use your custom CA bundle for all HTTPS requests. 3. After mounting the CA file, run `update-ca-certificates` in the container to update the system trust store. Your Docker Compose should include the volume mount and the `REQUESTS_CA_BUNDLE` environment variable. Example: ```yaml environment: REQUESTS_CA_BUNDLE: /usr/local/share/ca-certificates/root_cert.crt volumes: - /etc/pki/tls/certs/ca-bundle.crt:/usr/local/share/ca-certificates/root_cert.crt ``` If you’ve already done this and still see unknown CA errors, double-check that the CA file is readable inside the running container and that the path matches exactly. You can also test with a simple Python script using `requests` inside the container to verify SSL connections. The HTTP CONNECT method you see is normal for HTTPS over a proxy; it’s not an indication that the connection is downgraded to plain HTTP. The error likely means the CA bundle isn’t being picked up by the Python process, or the proxy is intercepting SSL and presenting a certificate not signed by your CA. There is no Dify-specific configuration for CA certificates in the plugin-daemon; it relies entirely on the Python and OS-level trust settings. Any customization must be done via environment variables or by updating the container’s trust store. [Reference: plugin-daemon uses requests library and system CA certificates](https://github.com/langgenius/dify/blob/0f3d4d0b6e65bcf1892316f1b6e4274e24f80e71/api/core/plugin/impl/base.py) <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/71deb944-6b6d-4556-877f-89919c12ede3?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/71deb944-6b6d-4556-877f-89919c12ede3?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/71deb944-6b6d-4556-877f-89919c12ede3?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/71deb944-6b6d-4556-877f-89919c12ede3?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/71deb944-6b6d-4556-877f-89919c12ede3?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/71deb944-6b6d-4556-877f-89919c12ede3?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/71deb944-6b6d-4556-877f-89919c12ede3?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/20901)
yindo commented 2026-02-16 10:19:07 -05:00
Author
Owner
Copy Link
Copy Source

@skylinenzo commented on GitHub (Jun 11, 2025):

@dosu, still persist, "Connection error., request_id: 20250611072056BLXwWKNLvdJSaZwUwvbr"

@skylinenzo commented on GitHub (Jun 11, 2025): @dosu, still persist, "Connection error., request_id: 20250611072056BLXwWKNLvdJSaZwUwvbr"
yindo commented 2026-02-16 10:19:07 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jun 11, 2025):

The persistent connection error suggests the plugin-daemon still isn't trusting your custom CA, or there's a proxy-related issue. Here are some focused troubleshooting steps:

  1. Double-check that the CA file is present and readable inside the running plugin-daemon container at the exact path set in REQUESTS_CA_BUNDLE. You can exec into the container and run ls -l /usr/local/share/ca-certificates/root_cert.crt and try a simple Python requests call to an HTTPS endpoint using your proxy.
  2. Make sure both uppercase and lowercase proxy environment variables (HTTP_PROXY, http_proxy, HTTPS_PROXY, https_proxy, and NO_PROXY, no_proxy) are set in your Docker Compose. Some Python libraries only pick up lowercase variants, especially in multi-layered Docker setups.
  3. If your proxy is doing SSL interception, ensure its CA is included in your bundle. If not, the requests library will reject the proxy's certificate.
  4. Try unsetting proxy variables (unset HTTP_PROXY and unset HTTPS_PROXY) inside the container and see if the error changes—sometimes proxy settings can interfere with internal service calls or loopback addresses.
  5. If you’re using a custom or older plugin-daemon image, consider updating to the latest version, as some proxy handling bugs were fixed in recent releases.

If you’ve confirmed all of the above and still see the error, capturing the full error traceback from the plugin-daemon logs (not just the API) may reveal more about whether it’s a CA trust or proxy routing issue. The Dify codebase does not provide any plugin-daemon-specific CA or proxy configuration—everything is handled at the OS and environment variable level, so the fix will be in the container setup, not in Dify config files.
Reference: plugin-daemon uses requests library and system CA certificates

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Join Discord Share on X

@dosubot[bot] commented on GitHub (Jun 11, 2025): <!-- Answer --> The persistent connection error suggests the plugin-daemon still isn't trusting your custom CA, or there's a proxy-related issue. Here are some focused troubleshooting steps: 1. Double-check that the CA file is present and readable inside the running plugin-daemon container at the exact path set in `REQUESTS_CA_BUNDLE`. You can exec into the container and run `ls -l /usr/local/share/ca-certificates/root_cert.crt` and try a simple Python `requests` call to an HTTPS endpoint using your proxy. 2. Make sure both uppercase and lowercase proxy environment variables (`HTTP_PROXY`, `http_proxy`, `HTTPS_PROXY`, `https_proxy`, and `NO_PROXY`, `no_proxy`) are set in your Docker Compose. Some Python libraries only pick up lowercase variants, especially in multi-layered Docker setups. 3. If your proxy is doing SSL interception, ensure its CA is included in your bundle. If not, the requests library will reject the proxy's certificate. 4. Try unsetting proxy variables (`unset HTTP_PROXY` and `unset HTTPS_PROXY`) inside the container and see if the error changes—sometimes proxy settings can interfere with internal service calls or loopback addresses. 5. If you’re using a custom or older plugin-daemon image, consider updating to the latest version, as some proxy handling bugs were fixed in recent releases. If you’ve confirmed all of the above and still see the error, capturing the full error traceback from the plugin-daemon logs (not just the API) may reveal more about whether it’s a CA trust or proxy routing issue. The Dify codebase does not provide any plugin-daemon-specific CA or proxy configuration—everything is handled at the OS and environment variable level, so the fix will be in the container setup, not in Dify config files. [Reference: plugin-daemon uses requests library and system CA certificates](https://github.com/langgenius/dify/blob/0f3d4d0b6e65bcf1892316f1b6e4274e24f80e71/api/core/plugin/impl/base.py) <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/78c3743b-ddf4-478c-904c-fa46185f199b?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/78c3743b-ddf4-478c-904c-fa46185f199b?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/78c3743b-ddf4-478c-904c-fa46185f199b?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/78c3743b-ddf4-478c-904c-fa46185f199b?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/78c3743b-ddf4-478c-904c-fa46185f199b?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/78c3743b-ddf4-478c-904c-fa46185f199b?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/78c3743b-ddf4-478c-904c-fa46185f199b?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/20901)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#360
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?