langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #28750] Add comprehensive API/controller tests for dataset endpoints (list, create, update, delete, documents, segments, hit testing, external datasets) #32162

New Issue
Closed
opened 2026-02-21 20:50:52 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:50:52 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/28750

State: closed
Merged: Yes

Summary

Adds comprehensive API/integration test coverage for dataset-related controller endpoints to ensure the HTTP API layer works correctly and integrates properly with underlying services.

Fix: #28749

Test Coverage

Dataset CRUD Endpoints

  • ✅ GET /datasets - List datasets endpoint

    • Successful dataset listing with pagination
    • Search functionality with keyword parameter
    • Tag filtering
    • Pagination parameters (page, limit)
    • Response structure validation

  • ✅ GET /datasets/{id} - Get dataset details endpoint

    • Successful dataset retrieval
    • Dataset not found error handling (404)
    • Permission validation
    • Authentication checks

  • ✅ POST /datasets - Create dataset endpoint

    • Successful dataset creation
    • Request body validation
    • Response structure validation
    • Status code verification (201)

Hit Testing Endpoints

  • ✅ POST /datasets/{id}/hit-testing - Retrieval testing endpoint
    • Successful hit testing operation
    • Query parameter validation
    • Dataset validation
    • Response structure validation
    • Integration with HitTestingService

External Dataset Endpoints

  • ✅ GET /datasets/external-knowledge-api - List external knowledge APIs
    • Successful API list retrieval
    • Pagination support
    • Response structure validation
    • Integration with ExternalDatasetService

Test Structure

  • Test Data Factory: ControllerApiTestDataFactory provides reusable mock objects for Flask app, API instances, datasets, users, and HTTP clients
  • Comprehensive Coverage: Each endpoint tested with success scenarios, error conditions, and edge cases
  • Proper Mocking: All dependencies properly mocked (Flask app, Flask-RESTX API, service methods, authentication)
  • Follows Patterns: Test structure follows existing patterns from other API test files
  • Detailed Documentation: Extensive docstrings and comments (1110+ lines) covering:
    • Architecture overview
    • Testing strategy
    • API testing best practices
    • Flask-RESTX resource testing patterns
    • Request/response validation
    • Authentication and authorization testing
    • Error handling patterns
    • Performance considerations

Implementation Details

HTTP Request/Response Testing

  • GET, POST, PATCH, DELETE methods
  • Query parameters and request body validation
  • Response status codes and body structure
  • Headers and content types

Authentication and Authorization

  • Login required checks
  • Account initialization checks
  • Permission validation
  • Role-based access control
  • Mocked authentication context

Request Validation

  • Required parameter validation
  • Parameter type validation
  • Parameter range validation
  • Custom validation rules

Error Handling

  • 400 Bad Request (validation errors)
  • 401 Unauthorized (authentication errors)
  • 403 Forbidden (authorization errors)
  • 404 Not Found (resource not found)
  • 500 Internal Server Error (unexpected errors)

Service Integration

  • Service method invocation verification
  • Service method parameter passing verification
  • Service method return value handling
  • Service exception handling

Files Changed

  • api/tests/unit_tests/services/controller_api.py - New test file (1110 lines)

Testing

  • All tests follow Arrange-Act-Assert pattern
  • Proper use of pytest fixtures for test setup
  • Flask test client for HTTP request/response testing
  • Mock objects properly configured to simulate real behavior
  • Error conditions and edge cases thoroughly tested
  • HTTP status codes and response structures verified

Additional Notes

This test suite focuses on the controller/API layer, complementing the existing service-layer unit tests. Additional endpoints that could be added in future iterations:

  • Document endpoints (upload, list, get, update, delete, batch)
  • Segment endpoints (list, get, update, delete)
  • Dataset update/delete endpoints (PATCH, DELETE)
  • Advanced scenarios (file uploads, large payloads, concurrent requests)

Related

Part of ongoing effort to improve test coverage for dataset functionality across all layers (services, controllers, models).

Contribution by Gittensor, learn more at https://gittensor.io/

**Original Pull Request:** https://github.com/langgenius/dify/pull/28750 **State:** closed **Merged:** Yes --- ## Summary Adds comprehensive API/integration test coverage for dataset-related controller endpoints to ensure the HTTP API layer works correctly and integrates properly with underlying services. Fix: #28749 ## Test Coverage ### Dataset CRUD Endpoints - ✅ **GET /datasets** - List datasets endpoint - Successful dataset listing with pagination - Search functionality with keyword parameter - Tag filtering - Pagination parameters (page, limit) - Response structure validation - ✅ **GET /datasets/{id}** - Get dataset details endpoint - Successful dataset retrieval - Dataset not found error handling (404) - Permission validation - Authentication checks - ✅ **POST /datasets** - Create dataset endpoint - Successful dataset creation - Request body validation - Response structure validation - Status code verification (201) ### Hit Testing Endpoints - ✅ **POST /datasets/{id}/hit-testing** - Retrieval testing endpoint - Successful hit testing operation - Query parameter validation - Dataset validation - Response structure validation - Integration with HitTestingService ### External Dataset Endpoints - ✅ **GET /datasets/external-knowledge-api** - List external knowledge APIs - Successful API list retrieval - Pagination support - Response structure validation - Integration with ExternalDatasetService ## Test Structure - **Test Data Factory**: `ControllerApiTestDataFactory` provides reusable mock objects for Flask app, API instances, datasets, users, and HTTP clients - **Comprehensive Coverage**: Each endpoint tested with success scenarios, error conditions, and edge cases - **Proper Mocking**: All dependencies properly mocked (Flask app, Flask-RESTX API, service methods, authentication) - **Follows Patterns**: Test structure follows existing patterns from other API test files - **Detailed Documentation**: Extensive docstrings and comments (1110+ lines) covering: - Architecture overview - Testing strategy - API testing best practices - Flask-RESTX resource testing patterns - Request/response validation - Authentication and authorization testing - Error handling patterns - Performance considerations ## Implementation Details ### HTTP Request/Response Testing - GET, POST, PATCH, DELETE methods - Query parameters and request body validation - Response status codes and body structure - Headers and content types ### Authentication and Authorization - Login required checks - Account initialization checks - Permission validation - Role-based access control - Mocked authentication context ### Request Validation - Required parameter validation - Parameter type validation - Parameter range validation - Custom validation rules ### Error Handling - 400 Bad Request (validation errors) - 401 Unauthorized (authentication errors) - 403 Forbidden (authorization errors) - 404 Not Found (resource not found) - 500 Internal Server Error (unexpected errors) ### Service Integration - Service method invocation verification - Service method parameter passing verification - Service method return value handling - Service exception handling ## Files Changed - `api/tests/unit_tests/services/controller_api.py` - New test file (1110 lines) ## Testing - All tests follow Arrange-Act-Assert pattern - Proper use of pytest fixtures for test setup - Flask test client for HTTP request/response testing - Mock objects properly configured to simulate real behavior - Error conditions and edge cases thoroughly tested - HTTP status codes and response structures verified ## Additional Notes This test suite focuses on the controller/API layer, complementing the existing service-layer unit tests. Additional endpoints that could be added in future iterations: - Document endpoints (upload, list, get, update, delete, batch) - Segment endpoints (list, get, update, delete) - Dataset update/delete endpoints (PATCH, DELETE) - Advanced scenarios (file uploads, large payloads, concurrent requests) ## Related Part of ongoing effort to improve test coverage for dataset functionality across all layers (services, controllers, models). Contribution by Gittensor, learn more at https://gittensor.io/
yindo added the pull-request label 2026-02-21 20:50:52 -05:00
yindo closed this issue 2026-02-21 20:50:52 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#32162
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?