mirror of
https://github.com/langgenius/dify-official-plugins.git
synced 2026-07-22 01:55:27 -04:00
Error when using a vision model compatible with the OpenAI-API-compatible #676
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @jbb892543969 on GitHub (Sep 28, 2025).
Self Checks
Dify version
1.9.0
Plugin version
0.0.22
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
There is an error when running the visual model in Dify, but it works fine when I directly call it via the /v1/chat/completions endpoint.
✔️ Error log
2025-09-29 01:05:27.007 ERROR [GraphWorker-0] [base.py:211] - Error in stream reponse for plugin {'code': -500, 'message': '{"message":"{\"args\":{},\"error_type\":\"ChunkedEncodingError\",\"message\":\"Response ended prematurely\"}","error_type":"PluginInvokeError","args":null}', 'data': None}
2025-09-29 01:05:27.009 ERROR [GraphDispatcher] [error_handler.py:101] - Node 1758786549398 failed with ABORT strategy: [openai_api_compatible] Error: req_id: 3dcdae0e0b PluginInvokeError: {"args":{},"error_type":"ChunkedEncodingError","message":"Response ended prematurely"}
2025-09-29 01:05:27.217 ERROR [Thread-160 (_generate_worker)] [workflow_entry.py:110] - Unknown Error when workflow entry running
Traceback (most recent call last):
File "/app/api/core/workflow/workflow_entry.py", line 106, in run
yield from generator
File "/app/api/core/workflow/graph_engine/graph_engine.py", line 262, in run
raise self._graph_execution.error
RuntimeError: [openai_api_compatible] Error: req_id: 3dcdae0e0b PluginInvokeError: {"args":{},"error_type":"ChunkedEncodingError","message":"Response ended prematurely"}
@dosubot[bot] commented on GitHub (Sep 28, 2025):
Hi @jbb892543969! I'm Dosu and I’m helping the dify-official-plugins team.
This error is likely caused by how Dify's workflow engine handles streaming responses from the OpenAI-API-compatible plugin, especially for vision models. The workflow engine uses generators to process streaming data, and any interruption or premature closure of the HTTP stream (like a ChunkedEncodingError) will abort the workflow and result in empty output. Direct API calls to
/v1/chat/completionsbypass these layers and typically handle streaming more robustly, which is why they work as expected.Recent updates added a configuration option (
enable_stream) for OpenAI-compatible models and vision models in the plugin. Disabling streaming for the problematic model in your workflow may help avoid this error—look for theenable_streamparameter in your plugin or model configuration and set it tofalseif available. This workaround has been used to bypass streaming-related issues and KYC requirements for other models as well example.If you continue to see this error, check your Docker and server environment for HTTP timeout, proxy buffering, or Uvicorn server settings that might close streams prematurely. These can impact chunked/streaming responses in self-hosted setups details.
Let me know if disabling streaming resolves the issue, or if you need help locating the configuration!
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@jbb892543969 commented on GitHub (Sep 28, 2025):
@dosu
I didn't find any options compatible with OpenAI-API-compatible
@dosubot[bot] commented on GitHub (Sep 28, 2025):
There isn't a universal UI or config option in Dify workflows to disable streaming for OpenAI-API-compatible plugins. The plugin decides whether to stream based on the 'stream' parameter in the request body, but this isn't exposed as a toggle in the workflow UI by default. If you want to force non-streaming mode, you would need to modify the plugin code to always set
"stream": falsein the endpoint handler (source).As a workaround, you can try adjusting your Docker/server HTTP settings (timeouts, proxy buffering, Uvicorn settings) since these can cause premature stream closure and ChunkedEncodingError. If you prefer not to edit code, using direct API calls or switching to a different plugin/model are also options. Let me know if you want step-by-step instructions for any of these approaches.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@jbb892543969 commented on GitHub (Sep 28, 2025):
@dosu
How to set Docker/server HTTP settings (timeouts, proxy buffering, Uvicorn settings)
@dosubot[bot] commented on GitHub (Sep 28, 2025):
To adjust Docker/server HTTP settings and help prevent premature stream closure errors in Dify, here’s what you can do:
1. Increase timeouts in Docker/.env:
Edit your
docker/.envfile and set higher values for these variables (add them if missing):These control backend, plugin, and SSRF timeouts. After editing, restart your containers with:
Reference | Reference
2. Check and update NGINX (or other proxy) config:
If you use NGINX as a reverse proxy (common in Docker setups), make sure these are set in your proxy config (often at
docker/nginx/proxy.conf.template):This disables buffering and ensures proper streaming. Reload or restart NGINX after changes.
Reference
3. Uvicorn server settings:
If you run Uvicorn directly (for plugins or model servers), you can set timeouts with command-line flags, e.g.:
Increase
--timeout-keep-aliveas needed.4. Networking tips:
Avoid using
127.0.0.1for inter-container calls; use the host IP or Docker service name. Make sure no firewall or Docker network misconfiguration is blocking traffic.Reference
5. Hardcoded plugin timeouts:
For OpenAI-API-compatible plugins, some timeouts (like 300s) are hardcoded in the plugin code. To change these, you must edit the relevant Python file on the host, rebuild the Docker image, and restart the container, or mount the file into the container.
Reference
If you’re unsure which proxy you’re using or need help with a specific config file, let me know your setup details!
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@jbb892543969 commented on GitHub (Sep 28, 2025):
@dosu
The configuration is still invalid after modification. Is there any other plug-in that can support openAI compatibility?
@dosubot[bot] commented on GitHub (Sep 28, 2025):
Yes, there are other plugins that support OpenAI API compatibility in Dify. The most notable alternative is the Regolo.ai plugin, which was recently added and supports multiple LLMs—including vision-capable models like Qwen2.5-VL-32B-Instruct. It is designed for OpenAI API schema compatibility and handles streaming, with workarounds for known issues in some models. You can set it up in Dify under Model Provider → Regolo.ai by providing your Regolo API key and (optionally) a custom endpoint URL. More details and supported models are in the plugin's documentation here.
Other plugins like Grok (x.ai) also offer OpenAI API compatibility, but may require extra proxy configuration in Docker to work reliably example. Azure OpenAI, OpenRouter, and vLLM plugins are also options, but for vision and robust streaming, Regolo.ai is currently the most promising alternative.
Let me know if you need setup steps or run into issues with Regolo or any of these alternatives!
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@jbb892543969 commented on GitHub (Sep 28, 2025):
@dosu
I deployed the visual model locally using lmdeploy. I tried all the above plugins but still got this error.
@jbb892543969 commented on GitHub (Sep 29, 2025):
Dify the generated JSON structure
OpenAI's JSON structure
dify the adjusted configuration
I found that the model adjustment error was caused by a problem with the json structure, and had nothing to do with streaming or non-streaming. After I added the User message, it can be used normally. Is this a bug or a usage problem?
@dosubot[bot] commented on GitHub (Oct 16, 2025):
Hi, @jbb892543969. I'm Dosu, and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale.
Issue Summary:
Next Steps:
Thank you for your understanding and contribution!