langgenius/dify-plugin-sdks
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-sdks.git synced 2026-07-22 02:15:22 -04:00
Code Issues 58 Packages Projects Releases 10 Wiki Activity

usage info ignored by _handle_generate_stream_response #73

New Issue
Open
opened 2026-02-15 21:15:34 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 21:15:34 -05:00
Owner
Copy Link
Copy Source

Originally created by @smalon on GitHub (Feb 9, 2026).

it seems to be a bug in
python/dify_plugin/interfaces/model/openai_compatible/llm.py
key code:

 def _handle_generate_stream_response(
     ...
     if chunk_json:  # noqa: SIM102
         if u := chunk_json.get("usage"):
             usage = u
         if not chunk_json or len(chunk_json["choices"]) == 0:
             continue
     ...

if we got chunks like this:

...
data: {"model":"qwen3-max","id":"chatcmpl-2c88ceee-85d2-9a51-8326-53b61ac5bf48","choices":[{"delta":{"content":"","role":null},"index":0,"finish_reason":"stop"}],"created":1770648705,"object":"chat.completion.chunk","usage":null}

data: {"model":"qwen3-max","id":"chatcmpl-2c88ceee-85d2-9a51-8326-53b61ac5bf48","choices":[],"created":1770648705,"object":"chat.completion.chunk","usage":{"total_tokens":95,"completion_tokens":84,"prompt_tokens":11,"prompt_tokens_details":{"cached_tokens":0}}}

data: [DONE]

usage Info will not be read because of the continue logic.
i have fixed this in our local dify server, yet i'm not sure if this is a bug.

Originally created by @smalon on GitHub (Feb 9, 2026). it seems to be a bug in python/dify_plugin/interfaces/model/openai_compatible/llm.py key code: ``` def _handle_generate_stream_response( ... if chunk_json: # noqa: SIM102 if u := chunk_json.get("usage"): usage = u if not chunk_json or len(chunk_json["choices"]) == 0: continue ... ``` if we got chunks like this: ``` ... data: {"model":"qwen3-max","id":"chatcmpl-2c88ceee-85d2-9a51-8326-53b61ac5bf48","choices":[{"delta":{"content":"","role":null},"index":0,"finish_reason":"stop"}],"created":1770648705,"object":"chat.completion.chunk","usage":null} data: {"model":"qwen3-max","id":"chatcmpl-2c88ceee-85d2-9a51-8326-53b61ac5bf48","choices":[],"created":1770648705,"object":"chat.completion.chunk","usage":{"total_tokens":95,"completion_tokens":84,"prompt_tokens":11,"prompt_tokens_details":{"cached_tokens":0}}} data: [DONE] ``` usage Info will not be read because of the continue logic. i have fixed this in our local dify server, yet i'm not sure if this is a bug.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-sdks#73
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?