langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #21131] fix: shorten connection timeout to pypi.org for deprecation check for weaviate client #29572

New Issue
Closed
opened 2026-02-21 20:45:49 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:45:49 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/21131

State: closed
Merged: Yes

Summary

With PR #19003, the weaviate-client was upgraded to 3.24.0, and now every time Dify retrieves knowledge from Weaviate, it accesses pypi.org to perform a version check.

As a result, if the network is slow, has no external access, or if there are issues on pypi.org side, knowledge retrieval from Weaviate will have to wait for the version check timeout (1 second), which leads to a consistent decrease in performance.

This PR minimizes the performance impact of the version check by setting the timeout as short as possible (0.001 seconds).

Closes #20972,
Related #19839, #20693, #21006

Tests

Setup

For testing, prepare the following docker-compose.override.yaml file.
By resolving pypi.org to a non-existent IP address, we can intentionally reproduce a state where Dify cannot connect to pypi.org (and must wait for a timeout).

services:
  api:
    extra_hosts:
      - "pypi.org:192.168.192.192"

Also, create a minimal app with three nodes:

image

Dify 1.3.1: ✅ No Issues

The weaviate-client bundled with Dify 1.3.1 does not have version checking implemented.
Therefore, it is always fast enough.

Trial Knowledge retrieval time
1 3.233 s
2 148.450 ms
3 99.577 ms
4 110.847 ms
5 100.596 ms
6 106.840 ms
7 93.164 ms
8 111.231 ms
9 107.721 ms
10 105.303 ms

Dify 1.4.3: 🚨 Affected

Every time knowledge retrieval is performed, a version check is made by connecting to pypi.org.
In this testing environment, since pypi.org cannot be reached, there is always a 1-second timeout.
As a result, compared to 1.3.1, knowledge retrieval is noticeably slower (always +1 second).

Trial Knowledge retrieval time
1 5.014 s
2 1.162 s
3 1.161 s
4 1.109 s
5 1.165 s
6 1.149 s
7 1.152 s
8 1.121 s
9 1.141 s
10 1.099 s

Dify 1.4.3 with this PR: ✅ No Issues

Each time knowledge retrieval is performed, a version check is attempted by connecting to pypi.org, but since the timeout is set to 0.001 seconds, the overhead is negligible.
Performance is similar to that of 1.3.1.

Trial Knowledge retrieval time
1 2.989 s
2 141.955 ms
3 121.295 ms
4 119.567 ms
5 116.361 ms
6 104.549 ms
7 104.705 ms
8 129.797 ms
9 104.393 ms
10 103.211 ms

image

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
**Original Pull Request:** https://github.com/langgenius/dify/pull/21131 **State:** closed **Merged:** Yes --- ## Summary With PR #19003, the `weaviate-client` was upgraded to `3.24.0`, and now every time Dify retrieves knowledge from Weaviate, it accesses pypi.org to perform a version check. As a result, if the network is slow, has no external access, or if there are issues on pypi.org side, knowledge retrieval from Weaviate will have to wait for the version check timeout (1 second), which leads to a consistent decrease in performance. This PR minimizes the performance impact of the version check by setting the timeout as short as possible (0.001 seconds). Closes #20972, Related #19839, #20693, #21006 ## Tests ### Setup For testing, prepare the following `docker-compose.override.yaml` file. By resolving pypi.org to a non-existent IP address, we can intentionally reproduce a state where Dify cannot connect to pypi.org (and must wait for a timeout). ```yaml services: api: extra_hosts: - "pypi.org:192.168.192.192" ``` Also, create a minimal app with three nodes: ![image](https://github.com/user-attachments/assets/0d8c2552-40ee-4398-9923-7d1c154bdb61) ### Dify 1.3.1: ✅ No Issues The `weaviate-client` bundled with Dify 1.3.1 does not have version checking implemented. Therefore, it is always fast enough. | Trial | Knowledge retrieval time | | - | - | | 1 | 3.233 s | | 2 | 148.450 ms | | 3 | 99.577 ms | | 4 | 110.847 ms | | 5 | 100.596 ms | | 6 | 106.840 ms | | 7 | 93.164 ms | | 8 | 111.231 ms | | 9 | 107.721 ms | | 10 | 105.303 ms | ### Dify 1.4.3: 🚨 Affected Every time knowledge retrieval is performed, a version check is made by connecting to pypi.org. In this testing environment, since pypi.org cannot be reached, there is always a 1-second timeout. As a result, compared to 1.3.1, knowledge retrieval is noticeably slower (always +1 second). | Trial | Knowledge retrieval time | | - | - | | 1 | 5.014 s | | 2 | 1.162 s | | 3 | 1.161 s | | 4 | 1.109 s | | 5 | 1.165 s | | 6 | 1.149 s | | 7 | 1.152 s | | 8 | 1.121 s | | 9 | 1.141 s | | 10 | 1.099 s | ### Dify 1.4.3 with this PR: ✅ No Issues Each time knowledge retrieval is performed, a version check is attempted by connecting to pypi.org, but **since the timeout is set to 0.001 seconds, the overhead is negligible.** Performance is similar to that of 1.3.1. | Trial | Knowledge retrieval time | | - | - | | 1 | 2.989 s | | 2 | 141.955 ms | | 3 | 121.295 ms | | 4 | 119.567 ms | | 5 | 116.361 ms | | 6 | 104.549 ms | | 7 | 104.705 ms | | 8 | 129.797 ms | | 9 | 104.393 ms | | 10 | 103.211 ms | ![image](https://github.com/user-attachments/assets/22de6984-0372-4670-ba57-3917b6f877fc) ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [ ] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [ ] I've updated the documentation accordingly. - [x] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods
yindo added the pull-request label 2026-02-21 20:45:49 -05:00
yindo closed this issue 2026-02-21 20:45:49 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#29572
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?