langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

improve oaicompat_dify_model and oaicompat_dify_app #15

New Issue
Closed
opened 2026-02-16 10:17:45 -05:00 by yindo · 4 comments
yindo commented 2026-02-16 10:17:45 -05:00
Owner
Copy Link
Copy Source

Originally created by @mumutu66 on GitHub (Feb 20, 2025).

  • add /model endpoint
    most app use openai-compatibility endpoint will access /model to get available models for further usage
  • /completions returns response but third-party client not treat well

here is example from oaicompat_dify_model


{"id": "chatcmpl-1ba42808-5ed6-4c48-b5ee-ad8dffbf7260", "object": "chat.completion.chunk", "created": 1740037148, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u4f60\u597d"}, "finish_reason": null}]}

{"id": "chatcmpl-756edf1f-9a68-4260-aa98-2a7e01b1a4a6", "object": "chat.completion.chunk", "created": 1740037148, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\uff01"}, "finish_reason": null}]}

{"id": "chatcmpl-f5420f8e-5c48-445d-9f75-2cc786c058b9", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u6709\u4ec0\u4e48"}, "finish_reason": null}]}

{"id": "chatcmpl-2fe112b6-aa93-4738-89a4-867335b3945b", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u53ef\u4ee5\u5e2e\u52a9"}, "finish_reason": null}]}

{"id": "chatcmpl-7643a1a1-e3cb-4345-b830-278b7263ca06", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u4f60\u7684"}, "finish_reason": null}]}

{"id": "chatcmpl-426cc443-ecd7-412a-85b2-81be7576422e", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u5417"}, "finish_reason": null}]}

{"id": "chatcmpl-39a5a99e-90d6-4861-8f4e-085d1fbfae82", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\uff1f"}, "finish_reason": null}]}

{"id": "chatcmpl-0651e7b9-397b-413e-9dc6-4d61b763f6da", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]}

client not response

and got a config bug if change the llm return format to json, get following error


{\"message\":\"{\\\"args\\\":{},\\\"error_type\\\":\\\"TypeError\\\",\\\"message\\\":\\\"dify_plugin.entities.model.llm.LLMModelConfig() got multiple values for keyword argument 'completion_params'\\\"}\",\"error_type\":\"PluginDaemonInternalServerError\",\"args\":null}",
Originally created by @mumutu66 on GitHub (Feb 20, 2025). * add /model endpoint most app use openai-compatibility endpoint will access /model to get available models for further usage * /completions returns response but third-party client not treat well here is example from oaicompat_dify_model ``` {"id": "chatcmpl-1ba42808-5ed6-4c48-b5ee-ad8dffbf7260", "object": "chat.completion.chunk", "created": 1740037148, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u4f60\u597d"}, "finish_reason": null}]} {"id": "chatcmpl-756edf1f-9a68-4260-aa98-2a7e01b1a4a6", "object": "chat.completion.chunk", "created": 1740037148, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\uff01"}, "finish_reason": null}]} {"id": "chatcmpl-f5420f8e-5c48-445d-9f75-2cc786c058b9", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u6709\u4ec0\u4e48"}, "finish_reason": null}]} {"id": "chatcmpl-2fe112b6-aa93-4738-89a4-867335b3945b", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u53ef\u4ee5\u5e2e\u52a9"}, "finish_reason": null}]} {"id": "chatcmpl-7643a1a1-e3cb-4345-b830-278b7263ca06", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u4f60\u7684"}, "finish_reason": null}]} {"id": "chatcmpl-426cc443-ecd7-412a-85b2-81be7576422e", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\u5417"}, "finish_reason": null}]} {"id": "chatcmpl-39a5a99e-90d6-4861-8f4e-085d1fbfae82", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": "\uff1f"}, "finish_reason": null}]} {"id": "chatcmpl-0651e7b9-397b-413e-9dc6-4d61b763f6da", "object": "chat.completion.chunk", "created": 1740037149, "model": "qwen2.5:32b", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": null}]} ``` client not response and got a config bug if change the llm return format to json, get following error ``` {\"message\":\"{\\\"args\\\":{},\\\"error_type\\\":\\\"TypeError\\\",\\\"message\\\":\\\"dify_plugin.entities.model.llm.LLMModelConfig() got multiple values for keyword argument 'completion_params'\\\"}\",\"error_type\":\"PluginDaemonInternalServerError\",\"args\":null}", ```
yindo closed this issue 2026-02-16 10:17:45 -05:00
yindo commented 2026-02-16 10:17:45 -05:00
Author
Owner
Copy Link
Copy Source

@mumutu66 commented on GitHub (Feb 20, 2025):

simply add “data” and “data: [Done]” solve problem 2



                for chunk in llm_invoke_response:
                    yield (
                        "data: " + json.dumps(
                            {
                                "id": "chatcmpl-" + str(uuid.uuid4()),
                                "object": "chat.completion.chunk",
                                "created": int(time.time()),
                                "model": llm.get("model"),
                                "choices": [
                                    {
                                        "index": 0,
                                        "delta": {
                                            "content": chunk.delta.message.content
                                        },
                                        "finish_reason": None,
                                    }
                                ],
                            }
                        )
                        + "\n\n"
                    )
                yield("data: [DONE]\n\n")
@mumutu66 commented on GitHub (Feb 20, 2025): simply add “data” and “data: [Done]” solve problem 2 ``` for chunk in llm_invoke_response: yield ( "data: " + json.dumps( { "id": "chatcmpl-" + str(uuid.uuid4()), "object": "chat.completion.chunk", "created": int(time.time()), "model": llm.get("model"), "choices": [ { "index": 0, "delta": { "content": chunk.delta.message.content }, "finish_reason": None, } ], } ) + "\n\n" ) yield("data: [DONE]\n\n") ```
yindo commented 2026-02-16 10:17:45 -05:00
Author
Owner
Copy Link
Copy Source

@caolicaoli commented on GitHub (Mar 13, 2025):

also error:
org.springframework.ai.retry.TransientAiException: 500 - {"code":-500,"message":"{"message":"{\"args\":{},\"error_type\":\"TypeError\",\"message\":\"dify_plugin.entities.model.llm.LLMModelConfig() got multiple values for keyword argument 'completion_params'\"}","error_type":"PluginDaemonInternalServerError","args":null}","data":null}

改一下代码就行了：
把
model_config=LLMModelConfig(
completion_params=llm.get("completion_params", {}), **llm
)
改成
model_config=LLMModelConfig(**llm)

@caolicaoli commented on GitHub (Mar 13, 2025): also error: org.springframework.ai.retry.TransientAiException: 500 - {"code":-500,"message":"{\"message\":\"{\\\"args\\\":{},\\\"error_type\\\":\\\"TypeError\\\",\\\"message\\\":\\\"dify_plugin.entities.model.llm.LLMModelConfig() got multiple values for keyword argument 'completion_params'\\\"}\",\"error_type\":\"PluginDaemonInternalServerError\",\"args\":null}","data":null} 改一下代码就行了： 把 model_config=LLMModelConfig( completion_params=llm.get("completion_params", {}), **llm ) 改成 model_config=LLMModelConfig(**llm)
yindo commented 2026-02-16 10:17:45 -05:00
Author
Owner
Copy Link
Copy Source

@caolicaoli commented on GitHub (Mar 13, 2025):

还有问题：
content_type="event-stream" if stream else "application/json",
变成
content_type="text/event-stream" if stream else "application/json",

@caolicaoli commented on GitHub (Mar 13, 2025): 还有问题： content_type="event-stream" if stream else "application/json", 变成 content_type="text/event-stream" if stream else "application/json",
yindo commented 2026-02-16 10:17:46 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 16, 2025):

Hi, @mumutu66. I'm Dosu, and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You requested adding a /model endpoint to list available models and fixing the /completions endpoint to return properly chunked streamed responses.
  • You suggested adding "data" and "data: [DONE]" prefixes to streamed chunks for compatibility with third-party clients.
  • Contributor caolicaoli identified and fixed a TypeError related to LLMModelConfig keyword arguments.
  • The content_type header for streaming responses was corrected from "event-stream" to "text/event-stream".
  • These fixes improved compatibility and error handling in the OpenAI-compatible API implementation.

Next Steps:

  • Please confirm if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting below.
  • If no further updates are provided, I will automatically close this issue in 5 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Aug 16, 2025): Hi, @mumutu66. I'm [Dosu](https://dosu.dev), and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale. **Issue Summary:** - You requested adding a /model endpoint to list available models and fixing the /completions endpoint to return properly chunked streamed responses. - You suggested adding "data" and "data: [DONE]" prefixes to streamed chunks for compatibility with third-party clients. - Contributor caolicaoli identified and fixed a TypeError related to LLMModelConfig keyword arguments. - The content_type header for streaming responses was corrected from "event-stream" to "text/event-stream". - These fixes improved compatibility and error handling in the OpenAI-compatible API implementation. **Next Steps:** - Please confirm if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting below. - If no further updates are provided, I will automatically close this issue in 5 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#15
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?