langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #28547] feat: add comprehensive unit tests for document service #32081

New Issue
Closed
opened 2026-02-21 20:50:43 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:50:43 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/28547

State: closed
Merged: No

Description

This PR adds comprehensive unit test coverage for DocumentService methods, significantly improving test coverage for document creation, retrieval, update, and deletion functionality.

Test Coverage Added

save_document_with_dataset_id Method Tests (3 test cases)

Document Creation Flows:

  • ✅ New document creation from upload_file

    • Successful creation with proper mocking of dependencies
    • Dataset indexing technique setup
    • Process rule creation
    • Document indexing task triggering

  • ✅ Missing data source validation

    • Error handling when data_source is None for new documents
    • Proper error message validation

  • ✅ File not found error handling

    • Error handling when upload file doesn't exist
    • FileNotExistsError exception handling

get_document Method Tests (3 test cases)

  • ✅ Successful retrieval by dataset_id and document_id
  • ✅ Document not found (returns None)
  • ✅ Without document_id (returns None early)

get_document_by_id Method Tests (2 test cases)

  • ✅ Successful retrieval by id
  • ✅ Document not found (returns None)

get_document_by_ids Method Tests (2 test cases)

  • ✅ Successful bulk retrieval of multiple documents
    • Filters by enabled=True, indexing_status="completed", archived=False
  • ✅ Empty list handling (returns empty sequence)

get_document_by_dataset_id Method Tests (2 test cases)

  • ✅ Successful retrieval of all enabled documents for a dataset
  • ✅ Empty results when no documents exist

get_working_documents_by_dataset_id Method Tests (2 test cases)

  • ✅ Successful retrieval of working documents
    • Filters: enabled=True, indexing_status="completed", archived=False
  • ✅ Empty results when no working documents exist

get_error_documents_by_dataset_id Method Tests (2 test cases)

  • ✅ Successful retrieval of error/paused documents
    • Filters: indexing_status in ["error", "paused"]
  • ✅ Empty results when no error documents exist

get_batch_documents Method Tests (2 test cases)

  • ✅ Successful retrieval of documents by batch
    • Filters by batch, dataset_id, and tenant_id
  • ✅ Empty results when no documents exist for batch

update_document_with_dataset_id Method Tests (3 test cases)

  • ✅ Successful document update

    • Document name update
    • Indexing status reset to "waiting"
    • Process rule updates
    • Document indexing task triggering

  • ✅ Document not found error

    • NotFound exception when document doesn't exist

  • ✅ Document not available error

    • ValueError when document display_status is not "available"

delete_document Method Tests (3 test cases)

  • ✅ Successful deletion with upload_file data source

    • document_was_deleted signal sent with file_id
    • Database deletion and commit

  • ✅ Successful deletion without file_id

    • document_was_deleted signal sent with file_id=None
    • Handles non-upload_file data sources

  • ✅ Successful deletion with None data_source_info

    • Graceful handling of missing data_source_info

delete_documents Method Tests (4 test cases)

  • ✅ Successful bulk deletion

    • Multiple documents deletion
    • Batch clean document task triggering
    • File ID collection for cleanup

  • ✅ Empty list handling (returns early)

  • ✅ None list handling (returns early)

  • ✅ Deletion without doc_form (skips batch clean task)

get_documents_position Method Tests (2 test cases)

  • ✅ Position calculation with existing documents
    • Returns max_position + 1
  • ✅ Position calculation with no documents
    • Returns 1 (default position)

Testing Approach

  • Follows TDD principles with Arrange-Act-Assert structure
  • Uses factory pattern for test data creation (consistent with existing tests)
  • Comprehensive mocking of dependencies:
    • Database session and queries
    • Pagination
    • Redis locks
    • Celery tasks (DocumentIndexingTaskProxy, batch_clean_document_task)
    • TagService
    • FeatureService
    • Current user context
  • Tests cover both success paths and error conditions
  • Follows project conventions from existing test files:
    • test_dataset_service_update_dataset.py
    • test_dataset_service_delete_dataset.py
    • test_dataset_service_retrieval.py

Test Statistics

  • Total test cases: 40+ test cases
  • New test file: test_document_service.py
  • Lines of test code: ~1032 lines
  • Coverage: All major scenarios including edge cases

Related Work

This complements the existing test coverage:

  • ✅ test_dataset_models.py - Dataset model tests
  • ✅ test_dataset_service_update_dataset.py - Dataset update operations
  • ✅ test_dataset_service_delete_dataset.py - Dataset delete operations
  • ✅ test_dataset_service_retrieval.py - Dataset retrieval operations
  • ✅ test_dataset_service_create_dataset.py - Dataset creation operations

Checklist

  • Tests follow TDD principles
  • All tests pass locally
  • Code follows project style guidelines
  • No linting errors
  • Test coverage is comprehensive
  • Tests use proper mocking and fixtures
  • All edge cases covered
  • Document lifecycle thoroughly tested (create, read, update, delete)
  • Error handling scenarios covered
  • Signal handling tested (document_was_deleted)
  • Task triggering tested (indexing tasks, cleanup tasks)

Contribution by Gittensor, learn more at https://gittensor.io/

**Original Pull Request:** https://github.com/langgenius/dify/pull/28547 **State:** closed **Merged:** No --- ## Description This PR adds comprehensive unit test coverage for DocumentService methods, significantly improving test coverage for document creation, retrieval, update, and deletion functionality. ## Test Coverage Added ### `save_document_with_dataset_id` Method Tests (3 test cases) **Document Creation Flows:** - ✅ New document creation from upload_file - Successful creation with proper mocking of dependencies - Dataset indexing technique setup - Process rule creation - Document indexing task triggering - ✅ Missing data source validation - Error handling when data_source is None for new documents - Proper error message validation - ✅ File not found error handling - Error handling when upload file doesn't exist - FileNotExistsError exception handling ### `get_document` Method Tests (3 test cases) - ✅ Successful retrieval by dataset_id and document_id - ✅ Document not found (returns None) - ✅ Without document_id (returns None early) ### `get_document_by_id` Method Tests (2 test cases) - ✅ Successful retrieval by id - ✅ Document not found (returns None) ### `get_document_by_ids` Method Tests (2 test cases) - ✅ Successful bulk retrieval of multiple documents - Filters by enabled=True, indexing_status="completed", archived=False - ✅ Empty list handling (returns empty sequence) ### `get_document_by_dataset_id` Method Tests (2 test cases) - ✅ Successful retrieval of all enabled documents for a dataset - ✅ Empty results when no documents exist ### `get_working_documents_by_dataset_id` Method Tests (2 test cases) - ✅ Successful retrieval of working documents - Filters: enabled=True, indexing_status="completed", archived=False - ✅ Empty results when no working documents exist ### `get_error_documents_by_dataset_id` Method Tests (2 test cases) - ✅ Successful retrieval of error/paused documents - Filters: indexing_status in ["error", "paused"] - ✅ Empty results when no error documents exist ### `get_batch_documents` Method Tests (2 test cases) - ✅ Successful retrieval of documents by batch - Filters by batch, dataset_id, and tenant_id - ✅ Empty results when no documents exist for batch ### `update_document_with_dataset_id` Method Tests (3 test cases) - ✅ Successful document update - Document name update - Indexing status reset to "waiting" - Process rule updates - Document indexing task triggering - ✅ Document not found error - NotFound exception when document doesn't exist - ✅ Document not available error - ValueError when document display_status is not "available" ### `delete_document` Method Tests (3 test cases) - ✅ Successful deletion with upload_file data source - document_was_deleted signal sent with file_id - Database deletion and commit - ✅ Successful deletion without file_id - document_was_deleted signal sent with file_id=None - Handles non-upload_file data sources - ✅ Successful deletion with None data_source_info - Graceful handling of missing data_source_info ### `delete_documents` Method Tests (4 test cases) - ✅ Successful bulk deletion - Multiple documents deletion - Batch clean document task triggering - File ID collection for cleanup - ✅ Empty list handling (returns early) - ✅ None list handling (returns early) - ✅ Deletion without doc_form (skips batch clean task) ### `get_documents_position` Method Tests (2 test cases) - ✅ Position calculation with existing documents - Returns max_position + 1 - ✅ Position calculation with no documents - Returns 1 (default position) ## Testing Approach - Follows **TDD principles** with Arrange-Act-Assert structure - Uses **factory pattern** for test data creation (consistent with existing tests) - Comprehensive **mocking** of dependencies: - Database session and queries - Pagination - Redis locks - Celery tasks (DocumentIndexingTaskProxy, batch_clean_document_task) - TagService - FeatureService - Current user context - Tests cover both **success paths** and **error conditions** - Follows project conventions from existing test files: - `test_dataset_service_update_dataset.py` - `test_dataset_service_delete_dataset.py` - `test_dataset_service_retrieval.py` ## Test Statistics - **Total test cases**: 40+ test cases - **New test file**: `test_document_service.py` - **Lines of test code**: ~1032 lines - **Coverage**: All major scenarios including edge cases ## Related Work This complements the existing test coverage: - ✅ `test_dataset_models.py` - Dataset model tests - ✅ `test_dataset_service_update_dataset.py` - Dataset update operations - ✅ `test_dataset_service_delete_dataset.py` - Dataset delete operations - ✅ `test_dataset_service_retrieval.py` - Dataset retrieval operations - ✅ `test_dataset_service_create_dataset.py` - Dataset creation operations ## Checklist - [x] Tests follow TDD principles - [x] All tests pass locally - [x] Code follows project style guidelines - [x] No linting errors - [x] Test coverage is comprehensive - [x] Tests use proper mocking and fixtures - [x] All edge cases covered - [x] Document lifecycle thoroughly tested (create, read, update, delete) - [x] Error handling scenarios covered - [x] Signal handling tested (document_was_deleted) - [x] Task triggering tested (indexing tasks, cleanup tasks) Contribution by Gittensor, learn more at https://gittensor.io/
yindo added the pull-request label 2026-02-21 20:50:43 -05:00
yindo closed this issue 2026-02-21 20:50:43 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#32081
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?