mirror of
https://github.com/langgenius/dify-official-plugins.git
synced 2026-07-21 17:45:23 -04:00
Issues related to the minerU plugin #849
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @ooodwbooo on GitHub (Dec 2, 2025).
Self Checks
Dify version
1.10.1
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
minerU plugin 0.5.1
https://github.com/langgenius/dify/discussions/28790
https://github.com/langgenius/dify/discussions/29024
When using the minerU plugin, small files are processed without any problems; however, large files fail if the processing time exceeds 300 seconds. Additionally, the relevant configuration options cannot be found in the env file. What should be done? Moreover, the task ID changes with each run, making it impossible to reuse the cache from a previous processing session.
✔️ Expected Behavior
large files consistently success
❌ Actual Behavior
large files consistently fail after 300 seconds
@dosubot[bot] commented on GitHub (Dec 2, 2025):
The 300-second timeout for large files in the minerU plugin is caused by a hardcoded polling mechanism: the plugin retries up to 50 times with a 5-second interval, totaling about 250 seconds before it fails with a timeout error. These values (MAX_RETRIES and the sleep interval) are not configurable via environment variables or plugin settings—they are set directly in the plugin code and confirmed unchanged in recent updates. To increase the timeout, you would need to manually edit these values in tools/mineru/tools/parse.py, rebuild the plugin Docker image, and redeploy it.
The task ID (batch_id) is generated by the remote MinerU API each time a file is processed, so it changes with every run. This design prevents cache reuse based on task ID, and there is no way to persist or configure batch IDs from the plugin side without changes to the upstream API or plugin logic itself.
Workarounds include modifying the plugin code to increase MAX_RETRIES or reduce the polling interval, or splitting large files into smaller chunks. Environment variables like HTTP_REQUEST_MAX_CONNECT_TIMEOUT or PLUGIN_DAEMON_TIMEOUT do not affect this polling timeout. For unrelated connection issues, ensure all containers are on the same Docker network and use a Docker-accessible FILES_URL (e.g., http://nginx) for file access, but this does not resolve the polling timeout for large files.
Manual code changes may be overwritten by future plugin updates, so monitor the repository for any official fixes or new configuration options.
Source 1 | Source 2 | Source 3
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@ooodwbooo commented on GitHub (Dec 2, 2025):
i found it .
How can I build a plugin for Dify?
@dosubot[bot] commented on GitHub (Dec 2, 2025):
To build a plugin for Dify, you'll need to:
dify_plugin>=0.3.0,<0.6.0for compatibility) — you can add more dependencies as needed in requirements.txt.python -m main. The plugin will show up in your Dify instance as "debugging" mode.dify-plugin plugin package ./YOUR_PLUGIN_DIRto generate a .difypkg file. You can then install it in your Dify instance via the UI ("install from local file") or automate publishing with GitHub Actions if you want to contribute to the marketplace.If you're modifying an existing plugin (like minerU), make your code changes, then rebuild the Docker image and redeploy it. For custom plugins, after packaging, upload the .difypkg file to your self-hosted Dify instance.
Detailed guides and manifest examples can be found in these plugin guides:
For offline installs or more advanced packaging, see the offline install instructions.
Let me know if you want step-by-step instructions for a specific type of plugin or for rebuilding an existing one!
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@ooodwbooo commented on GitHub (Dec 2, 2025):
I modified the MinerU code, but encountered an error when installing the packaged version.
@dosubot[bot] commented on GitHub (Dec 2, 2025):
This error happens because Dify's plugin signature verification is enabled, and modifying the MinerU code invalidates the original signature. There are two ways to proceed:
For development/testing:
You can disable signature verification by setting
FORCE_VERIFYING_SIGNATURE=falsein your.envfile (usually indify/docker), then restart your Dify containers. This will let you install modified plugins without signature checks.Reference: GitHub Issue langgenius/dify#1629
For production or if verification must remain enabled:
You need to sign your modified plugin with your own key pair and configure Dify to trust your public key.
Full details and CLI commands are in the official docs:
Third-party signature verification guide
Disabling verification is fine for local development, but for production, signing your plugin and configuring trusted keys is recommended for security.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@ooodwbooo commented on GitHub (Dec 2, 2025):
FORCE_VERIFYING_SIGNATURE=false
ENFORCE_LANGGENIUS_PLUGIN_SIGNATURES=false
MAX_RETRIES=500
I have revised the content above, packaged and installed the plugin, but it still reports an error after exceeding 300 seconds.
@dosu
@dosu.
@dosubot[bot] commented on GitHub (Dec 2, 2025):
If the plugin still fails after 300 seconds even with MAX_RETRIES increased, the timeout is almost certainly enforced by your infrastructure—most commonly nginx, traefik, or Docker network/proxy settings—which often default to 300 seconds and override application-level changes. To resolve this, increase these timeouts in your Docker
.envanddocker-compose.yaml(e.g., GUNICORN_TIMEOUT, NGINX_PROXY_READ_TIMEOUT, NGINX_PROXY_SEND_TIMEOUT, PLUGIN_MAX_EXECUTION_TIMEOUT, DIFY_BACKWARDS_INVOCATION_READ_TIMEOUT), and in your nginx config (setproxy_read_timeoutandproxy_send_timeoutto 3600s,proxy_buffering off,proxy_http_version 1.1,proxy_set_header Connection ""). Restart all related containers and nginx after making these changes. If you’re running Uvicorn directly, increase--timeout-keep-aliveas well.If you’ve already done all this and still see the 300s cutoff, please share the exact error message or stack trace you get after the timeout—this will help pinpoint which layer is enforcing the limit.
Reference | Reference
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@ooodwbooo commented on GitHub (Dec 2, 2025):
https://github.com/langgenius/dify/discussions/28790
For further details, please refer to the link provided above.
@dosu
@dosu.
@dosubot[bot] commented on GitHub (Dec 2, 2025):
The discussion you linked is highly relevant: persistent 300s timeouts are almost always enforced by infrastructure, not just plugin code. In Dify’s Docker setup, you need to raise all relevant timeouts above 300s in your
.envanddocker-compose.yaml—specifically:GUNICORN_TIMEOUT,PLUGIN_MAX_EXECUTION_TIMEOUT,NGINX_PROXY_READ_TIMEOUT,NGINX_PROXY_SEND_TIMEOUT, and (if present)PLUGIN_DAEMON_TIMEOUT(set to e.g. 1200). In your nginx config, make sureproxy_read_timeoutandproxy_send_timeoutare set to 3600s,proxy_buffering off,proxy_http_version 1.1, andproxy_set_header Connection ""source.After changing these, restart all related containers and nginx. Also, confirm that all containers are healthy, on the same Docker network, and that
PLUGIN_DAEMON_URLandPLUGIN_DAEMON_KEYare correct. If you still hit a 300s cutoff, check the plugin_daemon and nginx logs for timeout errors—this will help pinpoint which layer is enforcing the limit source.Don’t set both
DEBUGandFLASK_DEBUGto true at the same time, and keepFLASK_DEBUGoff in production for stability. If the issue persists after all config changes, try splitting large files, using a different plugin, or running Dify outside Docker to rule out proxy/network issues source.To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@dosubot[bot] commented on GitHub (Dec 2, 2025):
The minerU plugin's timeout for large files is hardcoded in its code: it polls for results up to 50 times with a 5-second interval, so the maximum wait is about 250 seconds. This is not configurable via environment variables or the env file, which is why you can't find relevant options to adjust the timeout. To increase the timeout, you would need to modify the plugin code directly by raising the
MAX_RETRIESvalue or reducing the sleep interval in the polling loop. There are currently no official configuration options for this.Regarding cache reuse, the plugin generates a new task ID (
batch_id) for each run, as the batch ID is created by the server when a new parsing task is started. There is no mechanism in the code to persist or reuse batch IDs, so cache reuse is not possible with the current implementation.Workarounds include modifying the plugin code to increase retries, splitting large files into smaller chunks, or adjusting infrastructure-level timeouts, but these all require manual intervention. For more details, see the relevant code and discussion here.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other