langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #28716] test: add comprehensive unit tests for ExternalDatasetService (external knowledge API integration) #32148

New Issue
Closed
opened 2026-02-21 20:50:50 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:50:50 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/28716

State: closed
Merged: Yes

Summary

Add a new unit test suite for ExternalDatasetService to cover external knowledge API integration, dataset creation, and retrieval flows.

Fix: #28715

Test Coverage

External knowledge API templates

  • get_external_knowledge_apis
    • Verifies pagination behavior via db.paginate
    • Covers optional search keyword usage
  • validate_api_list
    • Valid configuration (endpoint + api_key)
    • Error cases for empty config, missing endpoint, missing api_key
  • create_external_knowledge_api
    • Successful creation with valid settings (endpoint + api_key)
    • Error when settings is missing
  • get_external_knowledge_api
    • Returns existing template
    • Raises ValueError("api template not found") when missing
  • update_external_knowledge_api
    • Updates name/description/settings and keeps stored api_key when HIDDEN_VALUE is provided
    • Raises ValueError("api template not found") when missing
  • delete_external_knowledge_api
    • Deletes existing template and commits
    • Raises ValueError("api template not found") when missing
  • external_knowledge_api_use_check
    • Returns (True, count) when bindings exist
    • Returns (False, 0) when there are no bindings
  • get_external_knowledge_binding_with_dataset_id
    • Returns binding for given dataset/tenant
    • Raises ValueError("external knowledge binding not found") when missing

Document processing and HTTP integration

  • document_create_args_validate
    • Passes when all required custom parameters are present
    • Raises ValueError("api template not found") if template/settings are missing
    • Raises ValueError("<name> is required") when required document parameters are absent
  • process_external_api
    • Correctly maps supported HTTP methods (e.g. POST) to ssrf_proxy with URL, headers, JSON body, and follow_redirects=True
    • Raises InvalidHttpMethodError for unsupported HTTP methods
  • assembling_headers
    • Builds bearer headers (Authorization: Bearer <api_key>) with default header
    • Builds basic headers with custom header name
    • Builds custom headers with raw API key
    • Raises when authorization.config is missing
    • Raises when authorization.config.api_key is None
    • Ensures no-auth leaves headers unchanged (returns a copied mapping)

Settings, dataset creation, and external retrieval

  • get_external_knowledge_api_settings
    • Validates mapping into ExternalKnowledgeApiSetting (url, method, headers, params)
  • create_external_dataset
    • Creates a new external dataset and associated ExternalKnowledgeBindings row when:
      • Dataset name is unique per tenant
      • External knowledge API exists
      • external_knowledge_api_id and external_knowledge_id are provided
    • Raises DatasetNameDuplicateError if dataset name already exists
    • Raises ValueError("api template not found") if the external API template does not exist
    • Raises ValueError("external_knowledge_id is required") / ValueError("external_knowledge_api_id is required") when required IDs are missing
  • fetch_external_knowledge_retrieval
    • Returns external records when:
      • Binding exists
      • API template exists and has valid settings (endpoint/api_key)
      • process_external_api returns a 200 response with records payload
    • Raises ValueError("external knowledge binding not found") when binding is absent
    • Raises ValueError("external api template not found") when API template is missing or settings is None
    • Returns empty list for non-200 responses (e.g. 500) from external API

Implementation Notes

  • New test suite file: api/tests/unit_tests/services/external_dataset_service.py
  • Uses rich docstrings, comments, and spacing to make scenarios easy to read and review
  • All external calls (db.session, db.paginate, ssrf_proxy, process_external_api) are fully mocked
  • No changes to production service logic; this PR only adds tests

Testing

  • pytest api/tests/unit_tests/services/external_dataset_service.py

Contribution by Gittensor, learn more at https://gittensor.io/

**Original Pull Request:** https://github.com/langgenius/dify/pull/28716 **State:** closed **Merged:** Yes --- ## Summary Add a new unit test suite for `ExternalDatasetService` to cover external knowledge API integration, dataset creation, and retrieval flows. Fix: #28715 ## Test Coverage ### External knowledge API templates - **get_external_knowledge_apis** - Verifies pagination behavior via `db.paginate` - Covers optional search keyword usage - **validate_api_list** - Valid configuration (endpoint + api_key) - Error cases for empty config, missing `endpoint`, missing `api_key` - **create_external_knowledge_api** - Successful creation with valid settings (endpoint + api_key) - Error when `settings` is missing - **get_external_knowledge_api** - Returns existing template - Raises `ValueError("api template not found")` when missing - **update_external_knowledge_api** - Updates name/description/settings and keeps stored api_key when `HIDDEN_VALUE` is provided - Raises `ValueError("api template not found")` when missing - **delete_external_knowledge_api** - Deletes existing template and commits - Raises `ValueError("api template not found")` when missing - **external_knowledge_api_use_check** - Returns `(True, count)` when bindings exist - Returns `(False, 0)` when there are no bindings - **get_external_knowledge_binding_with_dataset_id** - Returns binding for given dataset/tenant - Raises `ValueError("external knowledge binding not found")` when missing ### Document processing and HTTP integration - **document_create_args_validate** - Passes when all required custom parameters are present - Raises `ValueError("api template not found")` if template/settings are missing - Raises `ValueError("<name> is required")` when required document parameters are absent - **process_external_api** - Correctly maps supported HTTP methods (e.g. POST) to `ssrf_proxy` with URL, headers, JSON body, and `follow_redirects=True` - Raises `InvalidHttpMethodError` for unsupported HTTP methods - **assembling_headers** - Builds bearer headers (`Authorization: Bearer <api_key>`) with default header - Builds basic headers with custom header name - Builds custom headers with raw API key - Raises when `authorization.config` is missing - Raises when `authorization.config.api_key` is `None` - Ensures `no-auth` leaves headers unchanged (returns a copied mapping) ### Settings, dataset creation, and external retrieval - **get_external_knowledge_api_settings** - Validates mapping into `ExternalKnowledgeApiSetting` (url, method, headers, params) - **create_external_dataset** - Creates a new external dataset and associated `ExternalKnowledgeBindings` row when: - Dataset name is unique per tenant - External knowledge API exists - `external_knowledge_api_id` and `external_knowledge_id` are provided - Raises `DatasetNameDuplicateError` if dataset name already exists - Raises `ValueError("api template not found")` if the external API template does not exist - Raises `ValueError("external_knowledge_id is required")` / `ValueError("external_knowledge_api_id is required")` when required IDs are missing - **fetch_external_knowledge_retrieval** - Returns external records when: - Binding exists - API template exists and has valid settings (endpoint/api_key) - `process_external_api` returns a 200 response with `records` payload - Raises `ValueError("external knowledge binding not found")` when binding is absent - Raises `ValueError("external api template not found")` when API template is missing or settings is `None` - Returns empty list for non-200 responses (e.g. 500) from external API ## Implementation Notes - New test suite file: `api/tests/unit_tests/services/external_dataset_service.py` - Uses rich docstrings, comments, and spacing to make scenarios easy to read and review - All external calls (`db.session`, `db.paginate`, `ssrf_proxy`, `process_external_api`) are fully mocked - No changes to production service logic; this PR only adds tests ## Testing - `pytest api/tests/unit_tests/services/external_dataset_service.py` Contribution by Gittensor, learn more at https://gittensor.io/
yindo added the pull-request label 2026-02-21 20:50:50 -05:00
yindo closed this issue 2026-02-21 20:50:50 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#32148
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?