langgenius/dify-plugin-sdks
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-sdks.git synced 2026-07-22 02:15:22 -04:00
Code Issues 58 Packages Projects Releases 10 Wiki Activity

Model Invocation is there a non-blocking Socket? #68

New Issue
Open
opened 2026-02-15 21:15:32 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 21:15:32 -05:00
Owner
Copy Link
Copy Source

Originally created by @SugarFatFree on GitHub (Dec 23, 2025).

Although this code is a streaming request, the internal implementation is a blocking Socket. If many large model requests are initiated at the same time, the performance will be very poor. Is there any request that supports non-blocking Socket? Aiohttp, for example

self.session.model.llm.invoke(
                model_config=LLMModelConfig(
                    provider=llm_info.get('provider'),
                    model=llm_info.get('model'),
                    mode=llm_info.get('mode'),
                    completion_params=llm_info.get('completion_params')
                ),
                prompt_messages=[
                    SystemPromptMessage(content=user_prompt),
                ],
                stream=True
            )

I found that httpx.Client() is used, can I use httpx.AsyncClient instead?

Originally created by @SugarFatFree on GitHub (Dec 23, 2025). Although this code is a streaming request, the internal implementation is a blocking Socket. If many large model requests are initiated at the same time, the performance will be very poor. Is there any request that supports non-blocking Socket? Aiohttp, for example ``` self.session.model.llm.invoke( model_config=LLMModelConfig( provider=llm_info.get('provider'), model=llm_info.get('model'), mode=llm_info.get('mode'), completion_params=llm_info.get('completion_params') ), prompt_messages=[ SystemPromptMessage(content=user_prompt), ], stream=True ) ``` I found that httpx.Client() is used, can I use httpx.AsyncClient instead?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
pull-request
Mirrored from GitHub Pull Request
 question
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-sdks#68
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?