langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #6773] Feat: Add MongoDB Query Tool #25424

New Issue
Closed
opened 2026-02-21 20:24:54 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:24:54 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/6773

State: closed
Merged: No

Checklist:

Important

Please review the checklist below before submitting your pull request.

  • Please open an issue before creating a PR or link to an existing issue
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

Description

Notes: This feature is currently in POC, I will appreciate it if anybody can share insights

graph TD
    style A fill:#ffffb3,stroke:#333,stroke-width:2px
    A[POC Completed] --> B[More Test Cases and Polishing WIP]
    B --> C[Formal PR Submission]

MongoDB is an non-relational database that is highly compatible with Python. We are developing a tool that introduces a MongoDB to dify, enabling it to run aggregation pipelines. This enhancement significantly boosts the productivity of script-based (as opposed to conversational) large model tasks. For instance, in our team's use case, we can query a range of data from MongoDB, run batch inference tasks through iterative nodes, and then store the results back into MongoDB.

  • Known-Issue: String length and array length limitation in code execution

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update, included: Dify Document
  • Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
  • Dependency upgrade

Testing Instructions

  • Aggregation Pipeline Query - Read
    In our testing enviroment setup, we use a mongo atlas as DB source. The tool requires mongodb_uri to be verfied, just like "api_key" in other tools. The test result is in the attached video.

Input:

  • (secret) mongodb_uri: mongodb+srv://myDatabaseUser:D1fficultP%40ssw0rd@mongodb0.example.com/?- authSource=admin&replicaSet=myRepl
  • database_name: xml
  • collection_name: cnnvd
  • 'query_pipeline` : [{"$search":{"index":"cnnvd_search_index","text":{"query":"nginx","path":"name","fuzzy":{"maxEdits":2,"prefixLength":0,"maxExpansions":50}}}},{"$project":{"_id":0}},{"$sort":{"modified":-1}}] (which means to search any documents that match 'nginx', this could also be generated by LLM )

Output:
The tool will return this:

        results = {
            "query_result": List, #the returned results
            "number_of_queried": Number, #shows how many records are returned
            "total_number":Number # shows how many records are in the target collection
        }

The output can be parsed by a following code execution node, for example:

# Node input is `result` (MongoDB/json Array[Object] from MongoDB node
def main(result) -> dict:
    def format_result(result):
        formatted_result = str(result) #your own format function here
        return formatted_result[:200] #we currently get error if the length is set higher than 200, working on it

    if result[0]['number_of_queried'] > 1000: #default value is 30
        query_result = ['CODE_MAX_STRING_ARRAY_LENGTH exceeded']
    else:
        query_result = [format_result(i) for i in result[0]['query_result']]

    return {
        "number_of_queried": result[0]['number_of_queried'],
        "query_result": query_result,
        "total_number":result[0]['total_number']
    }

For more info about the testing database, please visit https://github.com/sorphwer/cnnvd_on_mongodb, or feel free to get in touch with me to get mongodb uri for testing, any comment is welcome.

https://github.com/user-attachments/assets/a68950a8-4e45-4e1c-9616-ec9896a5cdc1

**Original Pull Request:** https://github.com/langgenius/dify/pull/6773 **State:** closed **Merged:** No --- # Checklist: > [!IMPORTANT] > Please review the checklist below before submitting your pull request. - [X] Please open an issue before creating a PR or link to an existing issue - [X] I have performed a self-review of my own code - [X] I have commented my code, particularly in hard-to-understand areas - [X] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods # Description > Notes: This feature is currently in POC, I will appreciate it if anybody can share insights ```mermaid graph TD style A fill:#ffffb3,stroke:#333,stroke-width:2px A[POC Completed] --> B[More Test Cases and Polishing WIP] B --> C[Formal PR Submission] ``` MongoDB is an non-relational database that is highly compatible with Python. We are developing a tool that introduces a MongoDB to dify, enabling it to run [aggregation pipelines](https://www.mongodb.com/docs/manual/tutorial/update-documents-with-aggregation-pipeline/). This enhancement significantly boosts the productivity of script-based (as opposed to conversational) large model tasks. **For instance, in our team's use case, we can query a range of data from MongoDB, run batch inference tasks through iterative nodes, and then store the results back into MongoDB.** - Known-Issue: String length and array length limitation in code execution ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [ ] Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement - [X] Dependency upgrade # Testing Instructions - [X] Aggregation Pipeline Query - Read In our testing enviroment setup, we use a mongo atlas as DB source. The tool requires `mongodb_uri` to be verfied, just like "api_key" in other tools. The test result is in the attached video. **Input:** - (secret) `mongodb_uri`: mongodb+srv://myDatabaseUser:D1fficultP%40ssw0rd@mongodb0.example.com/?- authSource=admin&replicaSet=myRepl - `database_name`: xml - `collection_name`: cnnvd - 'query_pipeline` : [{"$search":{"index":"cnnvd_search_index","text":{"query":"nginx","path":"name","fuzzy":{"maxEdits":2,"prefixLength":0,"maxExpansions":50}}}},{"$project":{"_id":0}},{"$sort":{"modified":-1}}] (**which means to search any documents that match 'nginx'**, this could also be generated by LLM ) **Output:** The tool will return this: ```python results = { "query_result": List, #the returned results "number_of_queried": Number, #shows how many records are returned "total_number":Number # shows how many records are in the target collection } ``` The output can be parsed by a following code execution node, for example: ```python # Node input is `result` (MongoDB/json Array[Object] from MongoDB node def main(result) -> dict: def format_result(result): formatted_result = str(result) #your own format function here return formatted_result[:200] #we currently get error if the length is set higher than 200, working on it if result[0]['number_of_queried'] > 1000: #default value is 30 query_result = ['CODE_MAX_STRING_ARRAY_LENGTH exceeded'] else: query_result = [format_result(i) for i in result[0]['query_result']] return { "number_of_queried": result[0]['number_of_queried'], "query_result": query_result, "total_number":result[0]['total_number'] } ``` **For more info about the testing database, please visit https://github.com/sorphwer/cnnvd_on_mongodb, or feel free to get in touch with me to get mongodb uri for testing, any comment is welcome.** https://github.com/user-attachments/assets/a68950a8-4e45-4e1c-9616-ec9896a5cdc1
yindo added the pull-request label 2026-02-21 20:24:54 -05:00
yindo closed this issue 2026-02-21 20:24:54 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#25424
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?