langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 20:45:26 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

api-token use performance enhance #6380

New Issue
Closed
opened 2026-02-21 18:15:29 -05:00 by yindo · 7 comments
yindo commented 2026-02-21 18:15:29 -05:00
Owner
Copy Link
Copy Source

Originally created by @aconeshana on GitHub (Oct 28, 2024).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Our team frequently relies on the API for batch usage, but we’ve encountered performance issues. Specifically, the API authentication performs poorly under high QPS cauz too many database io.

I will contribute a pr for this issue.

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @aconeshana on GitHub (Oct 28, 2024). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. Our team frequently relies on the API for batch usage, but we’ve encountered performance issues. Specifically, the API authentication performs poorly under high QPS cauz too many database io. I will contribute a pr for this issue. ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [X] I am interested in contributing to this feature.
yindo added the 💪 enhancement label 2026-02-21 18:15:29 -05:00
yindo closed this issue 2026-02-21 18:15:29 -05:00
yindo commented 2026-02-21 18:15:29 -05:00
Author
Owner
Copy Link
Copy Source

@aconeshana commented on GitHub (Oct 28, 2024):

In a single API call, there are typically 3 or more database IO. This issue is merely proposing a direction for optimization. We aim to contribute further or look forward to more work about it.

@aconeshana commented on GitHub (Oct 28, 2024): In a single API call, there are typically 3 or more database IO. This issue is merely proposing a direction for optimization. We aim to contribute further or look forward to more work about it.
yindo commented 2026-02-21 18:15:29 -05:00
Author
Owner
Copy Link
Copy Source

@aconeshana commented on GitHub (Oct 31, 2024):

btw, why main thread blocking wait for name? This logic is a disaster when calling the API. We commented this out in production but I want to know why did this. @takatost
if self._conversation_name_generate_thread: self._conversation_name_generate_thread.join()

@aconeshana commented on GitHub (Oct 31, 2024): btw, why main thread blocking wait for name? This logic is a disaster when calling the API. We commented this out in production but I want to know why did this. @takatost ` if self._conversation_name_generate_thread: self._conversation_name_generate_thread.join()`
yindo commented 2026-02-21 18:15:29 -05:00
Author
Owner
Copy Link
Copy Source

@aconeshana commented on GitHub (Oct 31, 2024):

btw, Another question is why conversation reload from db when AppRunner#run such as AgentChatAppRunner.

def run(
        self,
        application_generate_entity: AgentChatAppGenerateEntity,
        queue_manager: AppQueueManager,
        conversation: Conversation,
        message: Message,
    ) -> None:
    
        if {ModelFeature.MULTI_TOOL_CALL, ModelFeature.TOOL_CALL}.intersection(model_schema.features or []):
            agent_entity.strategy = AgentEntity.Strategy.FUNCTION_CALLING

        conversation = db.session.query(Conversation).filter(Conversation.id == conversation.id).first()
        message = db.session.query(Message).filter(Message.id == message.id).first()
        db.session.close()

This method receives a conversation parameter, but reads it from the database again before runer_cls is instantiated. However, I think there is almost no non-real-time data in the same conversation, and it is meaningless to change the configuration or concurrency of the ongoing session.There is no consistency issue here, any suggestion? @takatost

@aconeshana commented on GitHub (Oct 31, 2024): btw, Another question is why conversation reload from db when AppRunner#run such as AgentChatAppRunner. ``` def run( self, application_generate_entity: AgentChatAppGenerateEntity, queue_manager: AppQueueManager, conversation: Conversation, message: Message, ) -> None: if {ModelFeature.MULTI_TOOL_CALL, ModelFeature.TOOL_CALL}.intersection(model_schema.features or []): agent_entity.strategy = AgentEntity.Strategy.FUNCTION_CALLING conversation = db.session.query(Conversation).filter(Conversation.id == conversation.id).first() message = db.session.query(Message).filter(Message.id == message.id).first() db.session.close() ``` This method receives a conversation parameter, but reads it from the database again before runer_cls is instantiated. However, I think there is almost no non-real-time data in the same conversation, and it is meaningless to change the configuration or concurrency of the ongoing session.There is no consistency issue here, any suggestion? @takatost
yindo commented 2026-02-21 18:15:29 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Oct 31, 2024):

btw, why main thread blocking wait for name? This logic is a disaster when calling the API. We commented this out in production but I want to know why did this. @takatost if self._conversation_name_generate_thread: self._conversation_name_generate_thread.join()

I checked with @takatost

You can decide wether you turn it on or off for generating the conversation name

@crazywoola commented on GitHub (Oct 31, 2024): > btw, why main thread blocking wait for name? This logic is a disaster when calling the API. We commented this out in production but I want to know why did this. @takatost ` if self._conversation_name_generate_thread: self._conversation_name_generate_thread.join()` I checked with @takatost > You can decide wether you turn it on or off for generating the conversation name
yindo commented 2026-02-21 18:15:29 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Oct 31, 2024):

Please add my wechat: crazyphage. I can setup a group chat with our backend guys.

@crazywoola commented on GitHub (Oct 31, 2024): Please add my wechat: crazyphage. I can setup a group chat with our backend guys.
yindo commented 2026-02-21 18:15:29 -05:00
Author
Owner
Copy Link
Copy Source

@dajianguo commented on GitHub (Nov 13, 2024):

Yes the database is a limit,I have a test，100/s can accomplish。
You can refer to the following, but I haven't tried it for higher performance

postgre:
ALTER SYSTEM SET max_parallel_workers = 60;
ALTER SYSTEM SET shared_buffers='80GB';
ALTER SYSTEM SET log_min_duration_statement = 2000;
ALTER SYSTEM SET max_connections = 10000;
SELECT pg_reload_conf();
SHOW shared_buffers
SHOW max_connections

dify:
SQLALCHEMY_POOL_SIZE: 1000
SQLALCHEMY_POOL_TIMEOUT: 20
SQLALCHEMY_POOL_RECYCLE: 240
SERVER_WORKER_AMOUNT: 12

the appache jmeter test
image
image
image

@dajianguo commented on GitHub (Nov 13, 2024): Yes the database is a limit,I have a test，100/s can accomplish。 You can refer to the following, but I haven't tried it for higher performance postgre: ALTER SYSTEM SET max_parallel_workers = 60; ALTER SYSTEM SET shared_buffers='80GB'; ALTER SYSTEM SET log_min_duration_statement = 2000; ALTER SYSTEM SET max_connections = 10000; SELECT pg_reload_conf(); SHOW shared_buffers SHOW max_connections dify: SQLALCHEMY_POOL_SIZE: 1000 SQLALCHEMY_POOL_TIMEOUT: 20 SQLALCHEMY_POOL_RECYCLE: 240 SERVER_WORKER_AMOUNT: 12 the appache jmeter test ![image](https://github.com/user-attachments/assets/8e7b464d-c842-4258-a945-13dcd63c3fe7) ![image](https://github.com/user-attachments/assets/fc1f59b5-71ac-4a7a-a418-810869f18778) ![image](https://github.com/user-attachments/assets/663546e6-f143-4f74-a0b0-d5b4f23693b7)
yindo commented 2026-02-21 18:15:30 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Dec 14, 2024):

Hi, @aconeshana. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary

  • You reported performance issues with API authentication due to excessive database I/O under high QPS conditions.
  • You plan to contribute a pull request to address these issues and have raised questions about main thread blocking and redundant database queries.
  • @crazywoola suggested that the blocking logic can be toggled and offered to facilitate further discussion with backend developers.
  • @dajianguo provided configuration suggestions for PostgreSQL and SQLAlchemy, sharing test results from Apache JMeter.

Next Steps

  • Please let us know if this issue is still relevant to the latest version of the Dify repository. If so, you can keep the discussion open by commenting on this issue.
  • Otherwise, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Dec 14, 2024): Hi, @aconeshana. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary** - You reported performance issues with API authentication due to excessive database I/O under high QPS conditions. - You plan to contribute a pull request to address these issues and have raised questions about main thread blocking and redundant database queries. - @crazywoola suggested that the blocking logic can be toggled and offered to facilitate further discussion with backend developers. - @dajianguo provided configuration suggestions for PostgreSQL and SQLAlchemy, sharing test results from Apache JMeter. **Next Steps** - Please let us know if this issue is still relevant to the latest version of the Dify repository. If so, you can keep the discussion open by commenting on this issue. - Otherwise, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#6380
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?