langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

Performance: Replace json with orjson in Function_Calling Agent Strategies for Faster Tool Invocation #882

New Issue
Closed
opened 2026-02-16 10:20:50 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 10:20:50 -05:00
Owner
Copy Link
Copy Source

Originally created by @real-akimoto-akira on GitHub (Dec 16, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Yes, when building an AI assistant that integrates multiple MCP tools (weather, calendar, email, database queries, etc.) for enterprise customers. During production deployment, we noticed that agent (a agent node with Dify Official Agent Strategies function_calling.py ) response times were higher than expected.

We tested the same workflow with a third-party MCP plugin that uses orjson, and the performance difference was dramatic.

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @real-akimoto-akira on GitHub (Dec 16, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. Yes, when building an AI assistant that integrates multiple MCP tools (weather, calendar, email, database queries, etc.) for enterprise customers. During production deployment, we noticed that agent (a agent node with Dify Official Agent Strategies [function_calling.py](https://github.com/langgenius/dify-official-plugins/blob/main/agent-strategies/cot_agent/strategies/function_calling.py) ) response times were higher than expected. We tested the same workflow with a [third-party MCP plugin](https://github.com/langgenius/dify-official-plugins/blob/main/agent-strategies/cot_agent/strategies/function_calling.py) that uses `orjson`, and the performance difference was dramatic. ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [x] I am interested in contributing to this feature.
yindo added the enhancement label 2026-02-16 10:20:50 -05:00
yindo closed this issue 2026-02-16 10:20:50 -05:00
yindo commented 2026-02-16 10:20:50 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Dec 16, 2025):

Hi @real-akimoto-akira! I'm Dosu and I’m helping the dify-official-plugins team.

You can improve tool invocation performance in function_calling.py by replacing the built-in json module with orjson for serialization and deserialization. The key locations for this change are where json.dumps and json.loads are used to handle tool call arguments and responses—these are hot paths for agent workflows and directly impact latency when many tool calls are involved. For example, see usages like:

arguments=json.dumps(tool_call_args, ensure_ascii=False)
args = json.loads(prompt_message.function.arguments)
text = json.dumps(cast(ToolInvokeMessage.JsonMessage, tool_invoke_response.message).json_object, ensure_ascii=False)

Source

How to update:

  • Replace import json with import orjson.
  • Change json.dumps(obj, ensure_ascii=False) to orjson.dumps(obj).decode("utf-8") (since orjson.dumps returns bytes, not a string).
  • Change json.loads(s) to orjson.loads(s) (drop-in replacement).
  • Preserve error handling for JSON decoding, as orjson raises orjson.JSONDecodeError (compatible with json.JSONDecodeError).
  • orjson always outputs UTF-8, so you don’t need ensure_ascii=False.

No existing PRs or issues address this optimization in agent strategies, so this would be a new contribution. Make sure to add orjson to your dependencies (e.g., requirements.txt or pyproject.toml) if it’s not already present.

Let me know if you want a more detailed code diff or have questions about edge cases!

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Dec 16, 2025): <!-- Greeting --> Hi @real-akimoto-akira! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the dify-official-plugins team. <!-- Answer --> You can improve tool invocation performance in `function_calling.py` by replacing the built-in `json` module with `orjson` for serialization and deserialization. The key locations for this change are where `json.dumps` and `json.loads` are used to handle tool call arguments and responses—these are hot paths for agent workflows and directly impact latency when many tool calls are involved. For example, see usages like: ```python arguments=json.dumps(tool_call_args, ensure_ascii=False) args = json.loads(prompt_message.function.arguments) text = json.dumps(cast(ToolInvokeMessage.JsonMessage, tool_invoke_response.message).json_object, ensure_ascii=False) ``` [Source](https://github.com/langgenius/dify-official-plugins/blob/c426921b2fc9a84857741790431e5b9113723ae2/agent-strategies/cot_agent/strategies/function_calling.py#L1-L604) **How to update:** - Replace `import json` with `import orjson`. - Change `json.dumps(obj, ensure_ascii=False)` to `orjson.dumps(obj).decode("utf-8")` (since `orjson.dumps` returns bytes, not a string). - Change `json.loads(s)` to `orjson.loads(s)` (drop-in replacement). - Preserve error handling for JSON decoding, as `orjson` raises `orjson.JSONDecodeError` (compatible with `json.JSONDecodeError`). - `orjson` always outputs UTF-8, so you don’t need `ensure_ascii=False`. No existing PRs or issues address this optimization in agent strategies, so this would be a new contribution. Make sure to add `orjson` to your dependencies (e.g., `requirements.txt` or `pyproject.toml`) if it’s not already present. Let me know if you want a more detailed code diff or have questions about edge cases! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/c49a91aa-624e-492b-95b5-aa5183e358eb?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/c49a91aa-624e-492b-95b5-aa5183e358eb?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/c49a91aa-624e-492b-95b5-aa5183e358eb?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/c49a91aa-624e-492b-95b5-aa5183e358eb?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c49a91aa-624e-492b-95b5-aa5183e358eb?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/c49a91aa-624e-492b-95b5-aa5183e358eb?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c49a91aa-624e-492b-95b5-aa5183e358eb?feedback_type=other)</sup> [![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=You%20can%20improve%20tool%20invocation%20performance%20in%20%60function_calling.py%60%20by%20replacing%20the%20built-in%20%60json%60%20module%20with%20%60orjson%60%20for%20serialization%20and%20deserialization.%20The%20key%20locations%20for%20this%20change%20are%20where%20%60json.dumps%60%20and%20%60json.loads%60%20are%20used%20to%20handle%20tool%20call%20arguments%20and%20responses%E2%80%94these%20are%20hot%20paths%20for%20agent%20workflows%20and%20directly%20impact%20latency%20when%20many%20tool%20calls%20are%20involved.%20For%20example%2C%20see%20usages%20like%3A%0A%0A%60%60%60python%0Aarguments%3Djson.dumps%28tool_call_args%2C%20ensure_ascii%3DFalse%29%0Aargs%20%3D%20json.loads%28prompt_message.function.arguments%29%0Atext%20%3D%20json.dumps%28cast%28ToolInvokeMessage.JsonMessage%2C%20tool_invoke_response.message%29.json_object%2C%20ensure_ascii%3DFalse%29%0A%60%60%60%0A%5BSource%5D%28https%3A//github.com/langgenius/dify-official-plugins/blob/c426921b2fc9a84857741790431e5b9113723ae2/agent-strategies/cot_agent/strategies/function_calling.py%23L1-L604%29%0A%0A%2A%2AHow%20to%20update%3A%2A%2A%0A-%20Replace%20%60import%20json%60%20with%20%60import%20orjson%60.%0A-%20Change%20%60json.dumps%28obj%2C%20ensure_ascii%3DFalse%29%60%20to%20%60orjson.dumps%28obj%29.decode%28%22utf-8%22%29%60%20%28since%20%60orjson.dumps%60%20returns%20bytes%2C%20not%20a%20string%29.%0A-%20Change%20%60json.loads%28s%29%60%20to%20%60orjson.loads%28s%29%60%20%28drop-in%20replacement%29.%0A-%20Preserve%20error%20handling%20for%20JSON%20decoding%2C%20as%20%60orjson%60%20raises%20%60orjson.JSONDecodeError%60%20%28compatible%20with%20%60json.JSONDecodeError%60%29.%0A-%20%60orjson%60%20always%20outputs%20UTF-8%2C%20so%20you%20don%E2%80%99t%20need%20%60ensure_ascii%3DFalse%60.%0A%0ANo%20existing%20PRs%20or%20issues%20address%20this%20optimization%20in%20agent%20strategies%2C%20so%20this%20would%20be%20a%20new%20contribution.%20Make%20sure%20to%20add%20%60orjson%60%20to%20your%20dependencies%20%28e.g.%2C%20%60requirements.txt%60%20or%20%60pyproject.toml%60%29%20if%20it%E2%80%99s%20not%20already%20present.%0A%0ALet%20me%20know%20if%20you%20want%20a%20more%20detailed%20code%20diff%20or%20have%20questions%20about%20edge%20cases%21)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/2267)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#882
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?