langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

[PR #1152] [MERGED] fix(vertex-ai): switch to REST transport to fix concurrency hang #1740

New Issue
Closed
opened 2026-02-16 10:23:50 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 10:23:50 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langgenius/dify-official-plugins/pull/1152
Author: @adrianfinantyo
Created: 6/24/2025
Status: ✅ Merged
Merged: 6/24/2025
Merged by: @crazywoola

Base: main ← Head: main

📝 Commits (1)

  • 2ba7e7c fix(vertex-ai): switch to REST transport to fix concurrency hang

📊 Changes

4 files changed (+7 additions, -10 deletions)

View changed files

📝 models/vertex_ai/main.py (+0 -3)
📝 models/vertex_ai/manifest.yaml (+1 -1)
📝 models/vertex_ai/models/llm/llm.py (+2 -2)
📝 models/vertex_ai/models/text_embedding/text_embedding.py (+4 -4)

📄 Description

Related Issues or Context

This PR fixes an issue where the Vertex AI plugin hangs during document ingestion when handling multiple tasks. The problem was caused by gRPC transport not working well with the Gevent-based runtime. Switching to REST transport solves the issue.

Root Cause: gRPC transport hangs under concurrency due to incompatibility with Gevent.
Fix: Use REST transport in the Vertex AI SDK.

Root Cause (5 Whys)

  • Why did the Dify plugin daemon get stuck?
    It was waiting for responses from the Python plugin runtime, which never completed the embedding tasks.
  • Why were the tasks never completed?
    The Python plugin runtime got stuck during the first call to Vertex AI SDK’s embedding method.
  • Why did it get stuck on that call?
    The SDK used gRPC, which isn’t compatible with the Gevent/greenlets concurrency model.
  • Why was gRPC incompatible?
    The runtime used grpc.experimental.gevent.init_gevent() which causes silent hangs under load.
  • Why wasn’t this caught earlier?
    Heartbeat ran in a separate thread and continued normally, masking the deadlock.

Ingestion Process

Task Execution Details

Start → Task starts in greenlet → Call Vertex AI SDK → SDK makes gRPC call → Monkey-patched gRPC triggers C-core call →
[Decision] Is gRPC call non-blocking? → ❌ No → gRPC performs blocking I/O → Greenlet does not yield → Event loop frozen
[Decision] Does gRPC need callback or IO to finish? → ✅ Yes → Needs scheduler to run → 🧨 Deadlock - circular wait

Fixing Approach: Use Vertex AI REST Transport

image

Result: With REST transport, concurrent embedding tasks complete successfully.

Related GitHub Issues

Closes #1132

This PR contains Changes to Non-Plugin

  • Documentation
  • Other

This PR contains Changes to Non-LLM Models Plugin

  • I have Run Comprehensive Tests Relevant to My Changes
    📷 Tested locally under concurrent embedding load. Plugin no longer hangs.

This PR contains Changes to LLM Models Plugin

  • My Changes Affect Tool Interaction Flow (Multi-Round Usage and Output Handling)
    📷 Before: plugin hangs after first request. After: runs normally with multiple concurrent tasks.
  • My Changes Affect Message Flow Handling
  • My Changes Affect Multimodal Input Handling
  • My Changes Affect Multimodal Output Generation
  • My Changes Affect Structured Output Format
  • My Changes Affect Token Consumption Metrics
  • My Changes Affect Other LLM Functionalities
  • Other Changes (Changed Vertex AI SDK transport from gRPC to REST)

Version Control (Any Changes to the Plugin Will Require Bumping the Version)

  • I have Bumped Up the Version in manifest.yaml to 0.0.22

Dify Plugin SDK Version

  • I have Ensured dify_plugin>=0.3.0,<0.4.0 is in requirements.txt

Environment Verification (If Any Code Changes)

Local Deployment Environment

  • Dify Version is: 1.3.1
    Tested locally with clean setup. Plugin runs with REST transport and handles concurrent tasks correctly.

SaaS Environment

  • I have Tested My Changes on cloud.dify.ai with a Clean Environment That Matches the Production Configuration

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langgenius/dify-official-plugins/pull/1152 **Author:** [@adrianfinantyo](https://github.com/adrianfinantyo) **Created:** 6/24/2025 **Status:** ✅ Merged **Merged:** 6/24/2025 **Merged by:** [@crazywoola](https://github.com/crazywoola) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (1) - [`2ba7e7c`](https://github.com/langgenius/dify-official-plugins/commit/2ba7e7c212676ab5a388f9ef36bc46f2eb0e93e7) fix(vertex-ai): switch to REST transport to fix concurrency hang ### 📊 Changes **4 files changed** (+7 additions, -10 deletions) <details> <summary>View changed files</summary> 📝 `models/vertex_ai/main.py` (+0 -3) 📝 `models/vertex_ai/manifest.yaml` (+1 -1) 📝 `models/vertex_ai/models/llm/llm.py` (+2 -2) 📝 `models/vertex_ai/models/text_embedding/text_embedding.py` (+4 -4) </details> ### 📄 Description ## Related Issues or Context This PR fixes an issue where the Vertex AI plugin hangs during document ingestion when handling multiple tasks. The problem was caused by gRPC transport not working well with the Gevent-based runtime. Switching to REST transport solves the issue. Root Cause: gRPC transport hangs under concurrency due to incompatibility with Gevent. Fix: Use REST transport in the Vertex AI SDK. ### Root Cause (5 Whys) - Why did the Dify plugin daemon get stuck? It was waiting for responses from the Python plugin runtime, which never completed the embedding tasks. - Why were the tasks never completed? The Python plugin runtime got stuck during the first call to Vertex AI SDK’s embedding method. - Why did it get stuck on that call? The SDK used gRPC, which isn’t compatible with the Gevent/greenlets concurrency model. - Why was gRPC incompatible? The runtime used `grpc.experimental.gevent.init_gevent()` which causes silent hangs under load. - Why wasn’t this caught earlier? Heartbeat ran in a separate thread and continued normally, masking the deadlock. ### Ingestion Process [![](https://mermaid.ink/img/pako:eNp1VO1q2zAUfZWLfmWQdGmyZq0ZhZKUtrSspS0bjMCQ7RtbRJY8SU7jlf7cv8H-jw32GHuevsBeYddW4rgNDZjo456je-490j2LdIwsYBa_FKginAieGJ5NFdAv58aJSORcOZiIWflRmzma7b0rWSRCTThmWvndUC9Xq3BdKCcyhM5V6VKtXvmALYryeIkRcEujNaIHt9zOodoonDbQuU0N8hh2X-ZYIP0_I_GLF8I6VLhhGbzIcoo0DZFvMW02blDFLa7hiotWt6vzAY3D5dFZxebHQJObyTl0kuurMUE9ZlPh3uFhu6RBfRzEOiqySgpmIcaxUAk4qo8Ht-MruC9FQKQ25y5K61BYCA5nl5CLHFcwH-cRVGc6Kud36tkRkBhEJdE1IAolzFpZAGMuZQuUIbU6Jn3oPjertlH6XjsEI5LUgZ7BhqUqBxQWLSjuxALB1eUl7LvQvD7sKO0g0hkJEqFEuBMuhROsBDTUa7JeS9Lj7-9wyikBsEJSsCyBqMCgzbWy-EJaa_TJSjuEUkdzjOGOC1eJnJElq4zr3MZcVdmVAmVMOSpntISQR_M1ectWW-19_PXNtzhtDNaxTlBJdY6G1GrFZZPmk1b3lhvbENHPP__-_iAhjThw2rewU3nYgi7cU716QT5uU3bbqQYtz_tm1OKEKtB2a-EZn1fFsCVdrwx4nlN8pUO6tNw-6EhK34_J8dHk4nJ8DvRC1PlhfctJKWgly5r6xnMK668TWawELskXXQgLB8ors7UlqEFkadZliRExC5wpsMsyNBmvpuy-SmTKXIoZTllAw5ib-ZRN1QNh6Jp-0jpbw4wukpQFMy4tzYo85m79Ljarpn4AxpreBhaMBjUHC-7ZkgW7w_7O3mAwHPRHw93RcLhPuyUF7e2MBm8ORgej_gF9_dFDl32tT-3v7L_de_gPkx7mOA?type=png)](https://mermaid.live/edit#pako:eNp1VO1q2zAUfZWLfmWQdGmyZq0ZhZKUtrSspS0bjMCQ7RtbRJY8SU7jlf7cv8H-jw32GHuevsBeYddW4rgNDZjo456je-490j2LdIwsYBa_FKginAieGJ5NFdAv58aJSORcOZiIWflRmzma7b0rWSRCTThmWvndUC9Xq3BdKCcyhM5V6VKtXvmALYryeIkRcEujNaIHt9zOodoonDbQuU0N8hh2X-ZYIP0_I_GLF8I6VLhhGbzIcoo0DZFvMW02blDFLa7hiotWt6vzAY3D5dFZxebHQJObyTl0kuurMUE9ZlPh3uFhu6RBfRzEOiqySgpmIcaxUAk4qo8Ht-MruC9FQKQ25y5K61BYCA5nl5CLHFcwH-cRVGc6Kud36tkRkBhEJdE1IAolzFpZAGMuZQuUIbU6Jn3oPjertlH6XjsEI5LUgZ7BhqUqBxQWLSjuxALB1eUl7LvQvD7sKO0g0hkJEqFEuBMuhROsBDTUa7JeS9Lj7-9wyikBsEJSsCyBqMCgzbWy-EJaa_TJSjuEUkdzjOGOC1eJnJElq4zr3MZcVdmVAmVMOSpntISQR_M1ectWW-19_PXNtzhtDNaxTlBJdY6G1GrFZZPmk1b3lhvbENHPP__-_iAhjThw2rewU3nYgi7cU716QT5uU3bbqQYtz_tm1OKEKtB2a-EZn1fFsCVdrwx4nlN8pUO6tNw-6EhK34_J8dHk4nJ8DvRC1PlhfctJKWgly5r6xnMK668TWawELskXXQgLB8ors7UlqEFkadZliRExC5wpsMsyNBmvpuy-SmTKXIoZTllAw5ib-ZRN1QNh6Jp-0jpbw4wukpQFMy4tzYo85m79Ljarpn4AxpreBhaMBjUHC-7ZkgW7w_7O3mAwHPRHw93RcLhPuyUF7e2MBm8ORgej_gF9_dFDl32tT-3v7L_de_gPkx7mOA) ### Task Execution Details [![](https://mermaid.ink/img/pako:eNqdlE1u2zAQha8yYDcJYCcmKau2Fi38UxRBm6RIigKt7AUtjWTBEimQVBLH8Fm6bI_XI5SSLCfedFFtRBLvm5n3KGhHIhUjCUiSq8doLbSFz3cLCe65t253Fjav5Tn0--9gQsOvwmzA1GcGMgmpRpQ52mXLTGirY-FM5Dl8Q23xCSZXcD__1ElYK-GhO4NCbNBAevdlBpEjOg1vNV54reQGt_1S2GiNcSu0OktT1AZm_UhpPAG9BpzT3dWrqiCV7K9yFW0ymb7fL2QrntfDwo1qkCkNG32JOlG6MNDp4ery9lB92rqbsvDjwTbEyo0vlYVthnnc6VqLUx5-eEBpIVeqhESrZ5TQd2owtZkqd9U7ojU8Z7u56vKQ6AzX469EtAGl4eoWrIIkk5lZ1y4aD3Ur-O6YpqMX3jjKdA1Q14SuZNfmEA-KuHYXhn9-_vp93LrZokxHVS40PIrMLpfHpNjrpIbum8hcqrWmDkhJkMJmDwjGVUF7adfalex6Dk96nobfDT4Ztuk3KRqIlLRa5VBq5e4j33a325aa-C_5azRV4YqIxDqzblcqabDT-43-g4zP2u82UkXpKDTL824OY7c51jeWOEvBm2S16hnXe4Nuzflh3X_MYrsOWPl0AvH_gbwOiuMXyPf_CR1v6IAK8YIOBqcoP0VdBi0UJdER4mO-kKRHUp3FJLC6wh4pUBei3pJdzS-IXWOBCxK4ZSz0ZkEWcu-YUsgfShUdplWVrkmQiNy4XVXGwuI8E6kWxfFUo4xRz1QlLQmG1GuKkGBHnkhAR-yCUuZTn9MhZW8p75EtCZh3wdmI-t5gwOlgPOJ83yPPTd_Bxcij3PPY2B9zxthg2CMYZ1bp6_ZX1vzR9n8BaCCFqQ?type=png)](https://mermaid.live/edit#pako:eNqdlE1u2zAQha8yYDcJYCcmKau2Fi38UxRBm6RIigKt7AUtjWTBEimQVBLH8Fm6bI_XI5SSLCfedFFtRBLvm5n3KGhHIhUjCUiSq8doLbSFz3cLCe65t253Fjav5Tn0--9gQsOvwmzA1GcGMgmpRpQ52mXLTGirY-FM5Dl8Q23xCSZXcD__1ElYK-GhO4NCbNBAevdlBpEjOg1vNV54reQGt_1S2GiNcSu0OktT1AZm_UhpPAG9BpzT3dWrqiCV7K9yFW0ymb7fL2QrntfDwo1qkCkNG32JOlG6MNDp4ery9lB92rqbsvDjwTbEyo0vlYVthnnc6VqLUx5-eEBpIVeqhESrZ5TQd2owtZkqd9U7ojU8Z7u56vKQ6AzX469EtAGl4eoWrIIkk5lZ1y4aD3Ur-O6YpqMX3jjKdA1Q14SuZNfmEA-KuHYXhn9-_vp93LrZokxHVS40PIrMLpfHpNjrpIbum8hcqrWmDkhJkMJmDwjGVUF7adfalex6Dk96nobfDT4Ztuk3KRqIlLRa5VBq5e4j33a325aa-C_5azRV4YqIxDqzblcqabDT-43-g4zP2u82UkXpKDTL824OY7c51jeWOEvBm2S16hnXe4Nuzflh3X_MYrsOWPl0AvH_gbwOiuMXyPf_CR1v6IAK8YIOBqcoP0VdBi0UJdER4mO-kKRHUp3FJLC6wh4pUBei3pJdzS-IXWOBCxK4ZSz0ZkEWcu-YUsgfShUdplWVrkmQiNy4XVXGwuI8E6kWxfFUo4xRz1QlLQmG1GuKkGBHnkhAR-yCUuZTn9MhZW8p75EtCZh3wdmI-t5gwOlgPOJ83yPPTd_Bxcij3PPY2B9zxthg2CMYZ1bp6_ZX1vzR9n8BaCCFqQ) Start → Task starts in greenlet → Call Vertex AI SDK → SDK makes gRPC call → Monkey-patched gRPC triggers C-core call → [Decision] Is gRPC call non-blocking? → ❌ No → gRPC performs blocking I/O → Greenlet does not yield → Event loop frozen [Decision] Does gRPC need callback or IO to finish? → ✅ Yes → Needs scheduler to run → 🧨 Deadlock - circular wait ### Fixing Approach: Use Vertex AI REST Transport <img width="347" alt="image" src="https://github.com/user-attachments/assets/7b77ad75-8031-46de-98a1-d66093380d6b" /> Result: With REST transport, concurrent embedding tasks complete successfully. ### Related GitHub Issues Closes #1132 --- ## This PR contains Changes to *Non-Plugin* * [ ] Documentation * [x] Other --- ## This PR contains Changes to *Non-LLM Models Plugin* * [x] I have Run Comprehensive Tests Relevant to My Changes 📷 *Tested locally under concurrent embedding load. Plugin no longer hangs.* --- ## This PR contains Changes to *LLM Models Plugin* * [x] My Changes Affect Tool Interaction Flow (Multi-Round Usage and Output Handling) 📷 *Before: plugin hangs after first request. After: runs normally with multiple concurrent tasks.* * [ ] My Changes Affect Message Flow Handling * [ ] My Changes Affect Multimodal Input Handling * [ ] My Changes Affect Multimodal Output Generation * [ ] My Changes Affect Structured Output Format * [ ] My Changes Affect Token Consumption Metrics * [ ] My Changes Affect Other LLM Functionalities * [x] Other Changes (Changed Vertex AI SDK transport from gRPC to REST) --- ## Version Control (Any Changes to the Plugin Will Require Bumping the Version) * [x] I have Bumped Up the Version in `manifest.yaml` to `0.0.22` --- ## Dify Plugin SDK Version * [x] I have Ensured `dify_plugin>=0.3.0,<0.4.0` is in requirements.txt --- ## Environment Verification (If Any Code Changes) ### Local Deployment Environment * [x] Dify Version is: 1.3.1 Tested locally with clean setup. Plugin runs with REST transport and handles concurrent tasks correctly. ### SaaS Environment * [ ] I have Tested My Changes on cloud.dify.ai with a Clean Environment That Matches the Production Configuration --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-16 10:23:50 -05:00
yindo closed this issue 2026-02-16 10:23:50 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#1740
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?