langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #28951] feat: enhance OceanBase vector database with SQL injection fixes, unified processing, and improved error handling #32250

New Issue
Closed
opened 2026-02-21 20:51:02 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:51:02 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/28951

State: closed
Merged: Yes

Summary

This PR enhances the OceanBase vector database implementation by fixing security vulnerabilities, improving code quality, and enhancing user experience.

Changes

1. SQL Injection Prevention

  • Modified: get_ids_by_metadata_field() method (oceanbase_vector.py:265-297)
    • Added regex validation for metadata keys to prevent injection in JSON path
    • Used parameterized queries with SQLAlchemy text() to safely handle values
  • Modified: search_by_full_text() method (oceanbase_vector.py:331-375)
    • Used parameterized placeholders for document IDs in WHERE clause
  • Modified: search_by_vector() method (oceanbase_vector.py:377-430)
    • Added validation for document IDs before query construction
  • Security Impact: Critical - prevents SQL injection attacks in metadata queries

2. Unified Search Result Processing

  • Added: _process_search_results() method (oceanbase_vector.py:299-329)
  • Refactored search_by_vector() and search_by_full_text() to use unified processing
  • Benefits: Reduced code duplication, consistent result handling

3. Field Management

  • Added: field_exists() method (oceanbase_vector.py:85-92)
  • Added: _load_collection_fields() method (oceanbase_vector.py:69-83)
  • Added: _fields list to track collection schema
  • Auto-loads fields on initialization and after collection creation

4. Improved Error Handling

  • Added try-except blocks to all major methods
  • Using logger.exception() for detailed error context
  • Exception chaining with from e to preserve original errors
  • Added debug logging for successful operations
  • Affected methods: add_texts(), text_exists(), delete_by_ids(), get_ids_by_metadata_field(), delete_by_metadata_field(), delete(), search_by_vector(), search_by_full_text()

5. User-Friendly Warnings

  • Modified: search_by_full_text() (oceanbase_vector.py:331-375)
  • Added warning when hybrid search is disabled with configuration hint
  • Added warning when text field is missing with solution steps

Files Modified

  • api/core/rag/datasource/vdb/oceanbase/oceanbase_vector.py

Testing

  • ✅ All linting checks pass (make lint)
  • ✅ All type checks pass (make type-check)
  • ✅ Code formatting applied

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran make lint and make type-check to appease the lint gods

Breaking Changes

None. All changes are backward compatible.

Related Issues

Fixes #28950

**Original Pull Request:** https://github.com/langgenius/dify/pull/28951 **State:** closed **Merged:** Yes --- ## Summary This PR enhances the OceanBase vector database implementation by fixing security vulnerabilities, improving code quality, and enhancing user experience. ## Changes ### 1. SQL Injection Prevention - **Modified**: `get_ids_by_metadata_field()` method (oceanbase_vector.py:265-297) - Added regex validation for metadata keys to prevent injection in JSON path - Used parameterized queries with SQLAlchemy `text()` to safely handle values - **Modified**: `search_by_full_text()` method (oceanbase_vector.py:331-375) - Used parameterized placeholders for document IDs in WHERE clause - **Modified**: `search_by_vector()` method (oceanbase_vector.py:377-430) - Added validation for document IDs before query construction - **Security Impact**: Critical - prevents SQL injection attacks in metadata queries ### 2. Unified Search Result Processing - **Added**: `_process_search_results()` method (oceanbase_vector.py:299-329) - Refactored `search_by_vector()` and `search_by_full_text()` to use unified processing - **Benefits**: Reduced code duplication, consistent result handling ### 3. Field Management - **Added**: `field_exists()` method (oceanbase_vector.py:85-92) - **Added**: `_load_collection_fields()` method (oceanbase_vector.py:69-83) - **Added**: `_fields` list to track collection schema - Auto-loads fields on initialization and after collection creation ### 4. Improved Error Handling - Added try-except blocks to all major methods - Using `logger.exception()` for detailed error context - Exception chaining with `from e` to preserve original errors - Added debug logging for successful operations - **Affected methods**: `add_texts()`, `text_exists()`, `delete_by_ids()`, `get_ids_by_metadata_field()`, `delete_by_metadata_field()`, `delete()`, `search_by_vector()`, `search_by_full_text()` ### 5. User-Friendly Warnings - **Modified**: `search_by_full_text()` (oceanbase_vector.py:331-375) - Added warning when hybrid search is disabled with configuration hint - Added warning when text field is missing with solution steps ## Files Modified - `api/core/rag/datasource/vdb/oceanbase/oceanbase_vector.py` ## Testing - ✅ All linting checks pass (`make lint`) - ✅ All type checks pass (`make type-check`) - ✅ Code formatting applied ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [x] I've updated the documentation accordingly. - [x] I ran `make lint` and `make type-check` to appease the lint gods ## Breaking Changes None. All changes are backward compatible. ## Related Issues Fixes #28950
yindo added the pull-request label 2026-02-21 20:51:02 -05:00
yindo closed this issue 2026-02-21 20:51:02 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#32250
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?