langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-19 16:44:00 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #28806] feat: complete test script of reranker #32184

New Issue
Closed
opened 2026-02-21 20:50:55 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:50:55 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/28806

State: closed
Merged: Yes

Add Comprehensive Test Suite for Reranker Functionality

Important

  1. Make sure you have read our contribution guidelines
  2. Ensure there is an associated issue and you have been assigned to it
  3. Use the correct syntax to link this PR: Fixes #<issue number>.

Summary

This PR adds a comprehensive test suite for the RAG reranker functionality, covering all aspects of document reranking including cross-encoder models, weighted scoring, and factory patterns.

Fixes

Closes https://github.com/langgenius/dify/issues/28805

What's Added

New Test File: api/tests/unit_tests/core/rag/rerank/test_reranker.py

Test Coverage (37 tests total):

  1. RerankModelRunner Tests (9 tests)

    • Cross-encoder reranking with model-based scoring
    • Score threshold filtering and top-k selection
    • Document deduplication for dify and external providers
    • Metadata preservation and score injection
    • Empty documents and user parameter handling

  2. WeightRerankRunner Tests (8 tests)

    • Weighted scoring combining keyword (TF-IDF/BM25) and vector (cosine similarity) scores
    • Score threshold and top-k filtering for weighted reranking
    • Document deduplication and weight combination verification
    • Existing vector score reuse optimization

  3. RerankRunnerFactory Tests (4 tests)

    • Factory pattern for creating reranker instances
    • Parameter forwarding and error handling
    • Support for different runner types (RERANKING_MODEL, WEIGHTED_SCORE)

  4. Integration Tests (2 tests)

    • End-to-end reranking workflows
    • Score normalization across documents

  5. Edge Cases Tests (8 tests)

    • Empty/missing metadata handling
    • Boundary conditions (zero threshold, perfect scores)
    • Special characters and Unicode support
    • Large document sets (100+ documents)
    • Very long content (30K+ characters)
    • Zero weights and empty queries

  6. Performance Tests (2 tests)

    • Batch processing efficiency
    • Keyword extraction caching

  7. Error Handling Tests (4 tests)

    • Model invocation failures
    • Invalid indices and missing parameters
    • Missing vector error handling

Key Features

  • ✅ Comprehensive documentation: Every test includes detailed docstrings and inline comments
  • ✅ Follows TDD principles: Arrange-Act-Assert pattern throughout
  • ✅ Mocking strategy: No external dependencies, fast and reliable execution
  • ✅ Linting compliant: Passes all Ruff checks
  • ✅ Project conventions: Matches existing test patterns and style

Testing

All 37 tests pass successfully:

pytest api/tests/unit_tests/core/rag/rerank/test_reranker.py -v
# ======================================================= 37 passed in 37.20s ========================================================

Linting passes:

uv run --project api ruff check api/tests/unit_tests/core/rag/rerank/test_reranker.py
# All checks passed!

Screenshots

Not applicable - this is a test suite addition with no UI changes.

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

Additional Notes

This test suite provides comprehensive coverage for the reranker functionality, ensuring:

  • Reliability: All core reranking features are tested
  • Maintainability: Clear documentation helps future contributors
  • Quality: Edge cases and error conditions are properly handled
  • Performance: Batch processing and caching behaviors are verified

The tests serve as both validation and documentation for how the reranker system works, making it easier for developers to understand and modify the codebase with confidence.

Contribution by Gittensor, learn more at https://gittensor.io/

**Original Pull Request:** https://github.com/langgenius/dify/pull/28806 **State:** closed **Merged:** Yes --- # Add Comprehensive Test Suite for Reranker Functionality > [!IMPORTANT] > > 1. Make sure you have read our [contribution guidelines](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) > 1. Ensure there is an associated issue and you have been assigned to it > 1. Use the correct syntax to link this PR: `Fixes #<issue number>`. ## Summary This PR adds a comprehensive test suite for the RAG reranker functionality, covering all aspects of document reranking including cross-encoder models, weighted scoring, and factory patterns. ### Fixes Closes https://github.com/langgenius/dify/issues/28805 ### What's Added **New Test File**: `api/tests/unit_tests/core/rag/rerank/test_reranker.py` **Test Coverage (37 tests total)**: 1. **RerankModelRunner Tests (9 tests)** - Cross-encoder reranking with model-based scoring - Score threshold filtering and top-k selection - Document deduplication for dify and external providers - Metadata preservation and score injection - Empty documents and user parameter handling 2. **WeightRerankRunner Tests (8 tests)** - Weighted scoring combining keyword (TF-IDF/BM25) and vector (cosine similarity) scores - Score threshold and top-k filtering for weighted reranking - Document deduplication and weight combination verification - Existing vector score reuse optimization 3. **RerankRunnerFactory Tests (4 tests)** - Factory pattern for creating reranker instances - Parameter forwarding and error handling - Support for different runner types (RERANKING_MODEL, WEIGHTED_SCORE) 4. **Integration Tests (2 tests)** - End-to-end reranking workflows - Score normalization across documents 5. **Edge Cases Tests (8 tests)** - Empty/missing metadata handling - Boundary conditions (zero threshold, perfect scores) - Special characters and Unicode support - Large document sets (100+ documents) - Very long content (30K+ characters) - Zero weights and empty queries 6. **Performance Tests (2 tests)** - Batch processing efficiency - Keyword extraction caching 7. **Error Handling Tests (4 tests)** - Model invocation failures - Invalid indices and missing parameters - Missing vector error handling ### Key Features - ✅ **Comprehensive documentation**: Every test includes detailed docstrings and inline comments - ✅ **Follows TDD principles**: Arrange-Act-Assert pattern throughout - ✅ **Mocking strategy**: No external dependencies, fast and reliable execution - ✅ **Linting compliant**: Passes all Ruff checks - ✅ **Project conventions**: Matches existing test patterns and style ### Testing All 37 tests pass successfully: ```bash pytest api/tests/unit_tests/core/rag/rerank/test_reranker.py -v # ======================================================= 37 passed in 37.20s ======================================================== ``` Linting passes: ```bash uv run --project api ruff check api/tests/unit_tests/core/rag/rerank/test_reranker.py # All checks passed! ``` ## Screenshots Not applicable - this is a test suite addition with no UI changes. ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [x] I've updated the documentation accordingly. - [x] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods ## Additional Notes This test suite provides comprehensive coverage for the reranker functionality, ensuring: - **Reliability**: All core reranking features are tested - **Maintainability**: Clear documentation helps future contributors - **Quality**: Edge cases and error conditions are properly handled - **Performance**: Batch processing and caching behaviors are verified The tests serve as both validation and documentation for how the reranker system works, making it easier for developers to understand and modify the codebase with confidence. Contribution by Gittensor, learn more at https://gittensor.io/
yindo added the pull-request label 2026-02-21 20:50:55 -05:00
yindo closed this issue 2026-02-21 20:50:55 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#32184
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?