403 Client Error: Forbidden for url: http://dify-plugin-daemon:5002/plugin/3b61e0ad-96ff-4179 #20340

Closed
opened 2026-02-21 20:06:55 -05:00 by yindo · 2 comments
Owner

Originally created by @superwangnan on GitHub (Nov 14, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.9.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

after deploy the dify in the K8S cluster, got error below in the dify-api pod


requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://dify-plugin-daemon:5002/plugin/3b61e0ad-96ff-4179-a0a8-2beb8f1e1ec4/management/install/tasks?page=1&page_size=100
2025-11-14 15:47:27,710.710 ERROR [Dummy-16] [base.py:144] - Failed to request plugin daemon, status: 403, url: plugin/3b61e0ad-96ff-4179-a0a8-2beb8f1e1ec4/management/install/tasks
Traceback (most recent call last):
File "/app/api/core/plugin/impl/base.py", line 141, in _request_with_plugin_daemon_response
response.raise_for_status()
File "/app/api/.venv/lib/python3.12/site-packages/requests/models.py", line 1026, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://dify-plugin-daemon:5002/plugin/3b61e0ad-96ff-4179-a0a8-2beb8f1e1ec4/management/install/tasks?page=1&page_size=100
2025-11-14 15:47:27,711.711 ERROR [Dummy-16] [app.py:875] - Exception on /console/api/workspaces/current/plugin/tasks [GET]
Traceback (most recent call last):
File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/flask_restx/api.py", line 402, in wrapper
resp = resource(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/flask/views.py", line 110, in view
return current_app.ensure_sync(self.dispatch_request)(**kwargs) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/flask_restx/re
line 41, in dispatch_request
resp = meth(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/controllers/console/wraps.py", line 215, in decorated
return view(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/libs/login.py", line 62, in decorated_view
return current_app.ensure_sync(func)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/controllers/console/wraps.py", line 37, in decorated
return view(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/controllers/console/workspace/__init__.py", line 37, in decorated
return view(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/controllers/console/workspace/plugin.py", line 322, in get
{"tasks": PluginService.fetch_install_tasks(tenant_id, args["page"], args["page_size"])}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/services/plugin/plugin_service.py", line 221, in fetch_install_tasks
return manager.fetch_plugin_installation_tasks(tenant_id, page, page_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/plugin/impl/plugin.py", line 120, in fetch_plugin_installation_tasks
return self._request_with_plugin_daemon_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/plugin/impl/base.py", line 145, in _request_with_plugin_daemon_response
raise e
File "/app/api/core/plugin/impl/base.py", line 141, in _request_with_plugin_daemon_response
response.raise_for_status()
File "/app/api/.venv/lib/python3.12/site-packages/requests/models.py", line 1026, in raise_for_status
raise HTTPError(http_error
response=self)
requests.exceptions.HTTPErro

✔️ Expected Behavior

access the api in plugin-daemon

Actual Behavior

got 403 error

Originally created by @superwangnan on GitHub (Nov 14, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.9.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce after deploy the dify in the K8S cluster, got error below in the dify-api pod -------- requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://dify-plugin-daemon:5002/plugin/3b61e0ad-96ff-4179-a0a8-2beb8f1e1ec4/management/install/tasks?page=1&page_size=100 2025-11-14 15:47:27,710.710 ERROR [Dummy-16] [base.py:144] - Failed to request plugin daemon, status: 403, url: plugin/3b61e0ad-96ff-4179-a0a8-2beb8f1e1ec4/management/install/tasks Traceback (most recent call last): File "/app/api/core/plugin/impl/base.py", line 141, in _request_with_plugin_daemon_response response.raise_for_status() File "/app/api/.venv/lib/python3.12/site-packages/requests/models.py", line 1026, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://dify-plugin-daemon:5002/plugin/3b61e0ad-96ff-4179-a0a8-2beb8f1e1ec4/management/install/tasks?page=1&page_size=100 2025-11-14 15:47:27,711.711 ERROR [Dummy-16] [app.py:875] - Exception on /console/api/workspaces/current/plugin/tasks [GET] Traceback (most recent call last): File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/.venv/lib/python3.12/site-packages/flask_restx/api.py", line 402, in wrapper resp = resource(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/.venv/lib/python3.12/site-packages/flask/views.py", line 110, in view return current_app.ensure_sync(self.dispatch_request)(**kwargs) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/.venv/lib/python3.12/site-packages/flask_restx/re line 41, in dispatch_request resp = meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/app/api/controllers/console/wraps.py", line 215, in decorated return view(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/app/api/libs/login.py", line 62, in decorated_view return current_app.ensure_sync(func)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/controllers/console/wraps.py", line 37, in decorated return view(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/app/api/controllers/console/workspace/__init__.py", line 37, in decorated return view(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/app/api/controllers/console/workspace/plugin.py", line 322, in get {"tasks": PluginService.fetch_install_tasks(tenant_id, args["page"], args["page_size"])} ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/services/plugin/plugin_service.py", line 221, in fetch_install_tasks return manager.fetch_plugin_installation_tasks(tenant_id, page, page_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/plugin/impl/plugin.py", line 120, in fetch_plugin_installation_tasks return self._request_with_plugin_daemon_response( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/plugin/impl/base.py", line 145, in _request_with_plugin_daemon_response raise e File "/app/api/core/plugin/impl/base.py", line 141, in _request_with_plugin_daemon_response response.raise_for_status() File "/app/api/.venv/lib/python3.12/site-packages/requests/models.py", line 1026, in raise_for_status raise HTTPError(http_error response=self) requests.exceptions.HTTPErro ### ✔️ Expected Behavior access the api in plugin-daemon ### ❌ Actual Behavior got 403 error
yindo added the 🐞 bug label 2026-02-21 20:06:55 -05:00
yindo closed this issue 2026-02-21 20:06:55 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Nov 14, 2025):

Hi @superwangnan! I'm Dosu and I’m helping the dify team.

The 403 error when dify-api tries to access the plugin daemon is almost always caused by an authentication or proxy misconfiguration. The dify-api sends an X-Api-Key header with each request, using the value from the PLUGIN_DAEMON_KEY environment variable. If this key is missing, incorrect, or not matching what the plugin daemon expects, you'll get a 403 Forbidden response. Double-check that both dify-api and dify-plugin-daemon are using the same PLUGIN_DAEMON_KEY value in their environment variables.

Another common cause is proxy settings: if you have HTTP_PROXY or HTTPS_PROXY set, make sure your NO_PROXY (and no_proxy) environment variable includes all internal service names (like plugin_daemon, api, db, redis, etc.) so internal requests bypass the proxy. Otherwise, the proxy may block or alter internal traffic, resulting in a 403 error. Restart all services after making changes to environment variables to ensure they take effect.

If the issue persists, check the plugin daemon logs for more details on why the request was forbidden, and verify that the service is running and accessible from the dify-api pod.

References:

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Nov 14, 2025): <!-- Greeting --> Hi @superwangnan! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the dify team. <!-- Answer --> The 403 error when dify-api tries to access the plugin daemon is almost always caused by an authentication or proxy misconfiguration. The dify-api sends an `X-Api-Key` header with each request, using the value from the `PLUGIN_DAEMON_KEY` environment variable. If this key is missing, incorrect, or not matching what the plugin daemon expects, you'll get a 403 Forbidden response. Double-check that both dify-api and dify-plugin-daemon are using the same `PLUGIN_DAEMON_KEY` value in their environment variables. Another common cause is proxy settings: if you have `HTTP_PROXY` or `HTTPS_PROXY` set, make sure your `NO_PROXY` (and `no_proxy`) environment variable includes all internal service names (like `plugin_daemon`, `api`, `db`, `redis`, etc.) so internal requests bypass the proxy. Otherwise, the proxy may block or alter internal traffic, resulting in a 403 error. Restart all services after making changes to environment variables to ensure they take effect. If the issue persists, check the plugin daemon logs for more details on why the request was forbidden, and verify that the service is running and accessible from the dify-api pod. References: - [Proxy and NO_PROXY troubleshooting](https://github.com/langgenius/dify/issues/26177) - [API key header handling in code](https://github.com/langgenius/dify/blob/4a89403566c85eff76a959bc62be76f1e8af5ce6/api/core/plugin/impl/base.py#L39-L210) <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/9e0459d4-eba3-4c93-8274-bf3e7b70cd7e?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/9e0459d4-eba3-4c93-8274-bf3e7b70cd7e?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/9e0459d4-eba3-4c93-8274-bf3e7b70cd7e?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/9e0459d4-eba3-4c93-8274-bf3e7b70cd7e?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/9e0459d4-eba3-4c93-8274-bf3e7b70cd7e?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/9e0459d4-eba3-4c93-8274-bf3e7b70cd7e?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/9e0459d4-eba3-4c93-8274-bf3e7b70cd7e?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/28240)
Author
Owner

@superwangnan commented on GitHub (Nov 14, 2025):

I have no proxy setting in pods.

by default, in dify-api, there is PLUGIN_DAEMON_KEY environment variable, but this no PLUGIN_DAEMON_KEY in dify-plugin-daemon.

Then I add the PLUGIN_DAEMON_KEY in dify-plugin-daemon and set the same value as PLUGIN_DAEMON_KEY in dify-api. but it doesn't work yet.

@superwangnan commented on GitHub (Nov 14, 2025): I have no proxy setting in pods. by default, in dify-api, there is PLUGIN_DAEMON_KEY environment variable, but this no PLUGIN_DAEMON_KEY in dify-plugin-daemon. Then I add the PLUGIN_DAEMON_KEY in dify-plugin-daemon and set the same value as PLUGIN_DAEMON_KEY in dify-api. but it doesn't work yet.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#20340