langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #22673] fix: prevent stale dataset cache in workflow knowledge retrieval #30014

New Issue
Closed
opened 2026-02-21 20:46:41 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:46:41 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/22673

State: closed
Merged: No

Fix workflow knowledge retrieval cache bug

🐛 Problem Description

Issue: After deleting a dataset and adding a new one, workflow knowledge retrieval nodes showed both a blank option (ghost data) and the correct new dataset option.

User Impact:

  • Confusing UX with unexplained blank options
  • Users had to refresh the page to see correct state
  • Workflow configuration became unreliable

Root Cause Analysis:

  1. SelectDataset component created 'ghost' objects with only IDs during initialization
  2. SWR cache was not properly invalidated when datasets were deleted
  3. Race conditions in component state management

✅ Solution

Core Fixes:

  1. Fixed SelectDataset initialization logic

    • Removed creation of ghost objects with incomplete data
    • Added proper initialization flow after data loads
    • Prevented unwanted re-initialization with hasInitialized flag

  2. Enhanced SWR cache invalidation

    • Added comprehensive cache clearing when datasets are deleted
    • Robust cache key matching for both string and object formats
    • Immediate cache revalidation to ensure consistency

Files Changed:

  • web/app/components/app/configuration/dataset-config/select-dataset/index.tsx
  • web/app/(commonLayout)/datasets/DatasetCard.tsx

🧪 Testing

Test Scenarios Completed ���:

  1. Basic bug reproduction: Delete dataset → Add new dataset → Check workflow
  2. Fast consecutive operations: Rapid delete→add→check cycles
  3. Concurrent multi-tab scenarios: Multiple workflow tabs with simultaneous dataset operations
  4. Network interruption handling: Operations during network issues
  5. User interaction boundaries: Manual selection/deselection behavior
  6. Pagination scenarios: Large dataset lists with paging

Test Environment:

  • Docker-based local deployment
  • Built custom test image with fixes
  • Verified in realistic production-like conditions

📊 Technical Details

Before Fix:

// Created ghost objects with incomplete data
const [selected, setSelected] = React.useState<DataSet[]>(
  selectedIds.map(id => ({ id }) as any)
)

After Fix:

// Clean initialization with proper data flow
const [selected, setSelected] = React.useState<DataSet[]>([])
const [hasInitialized, setHasInitialized] = React.useState(false)

// Proper initialization after data loads
if (!hasInitialized && selectedIds.length > 0) {
  const validSelectedDatasets = selectedIds
    .map(id => newList.find(item => item.id === id))
    .filter(Boolean) as DataSet[]
  setSelected(validSelectedDatasets)
  setHasInitialized(true)
}

Cache Invalidation:

// Comprehensive SWR cache clearing
mutate(
  key => {
    if (typeof key === 'string') return key.includes('/datasets')
    if (typeof key === 'object' && key !== null) {
      return key.url === '/datasets' || key.url?.includes('/datasets')
    }
    return false
  },
  undefined,
  { revalidate: true }
)

🔒 Risk Assessment

Low Risk Changes:

  • ✅ Minimal code changes (only 2 files)
  • ✅ Targeted fixes addressing specific root causes
  • ✅ No breaking API changes
  • ✅ Backward compatible
  • ✅ Extensive edge case testing completed

Safeguards:

  • Added race condition protection
  • Robust error handling
  • Graceful fallback behavior
  • Comprehensive cache key matching

📝 Checklist

  • Bug fix tested and verified
  • No new dependencies added
  • Code follows existing patterns
  • Edge cases tested thoroughly
  • No performance regressions
  • Documentation updated (inline comments)
  • Backward compatibility maintained

🎯 Verification Steps

To verify this fix:

  1. Basic verification:

    1. Create a workflow with knowledge retrieval node
2. Add a dataset to the node
3. Delete that dataset from /datasets page
4. Add a new dataset
5. Return to workflow node configuration
6. ✅ Should only show the new dataset, no blank options

  2. Edge case verification:

    • Rapid consecutive operations
    • Multi-tab scenarios
    • Network interruption recovery

🚀 Impact

Positive Impact:

  • ✅ Eliminates confusing ghost options in workflow UI
  • ✅ Improves workflow configuration reliability
  • ✅ Better user experience with consistent state
  • ✅ Reduces support requests related to this issue

No Negative Impact:

  • ✅ No breaking changes
  • ✅ No performance degradation
  • ✅ No new dependencies

Ready for Review 🔍

This PR addresses a user-facing bug with a clean, minimal solution that has been thoroughly tested in multiple scenarios.

**Original Pull Request:** https://github.com/langgenius/dify/pull/22673 **State:** closed **Merged:** No --- # Fix workflow knowledge retrieval cache bug ## 🐛 Problem Description **Issue**: After deleting a dataset and adding a new one, workflow knowledge retrieval nodes showed both a blank option (ghost data) and the correct new dataset option. **User Impact**: - Confusing UX with unexplained blank options - Users had to refresh the page to see correct state - Workflow configuration became unreliable **Root Cause Analysis**: 1. `SelectDataset` component created 'ghost' objects with only IDs during initialization 2. SWR cache was not properly invalidated when datasets were deleted 3. Race conditions in component state management ## ✅ Solution ### **Core Fixes**: 1. **Fixed SelectDataset initialization logic** - Removed creation of ghost objects with incomplete data - Added proper initialization flow after data loads - Prevented unwanted re-initialization with `hasInitialized` flag 2. **Enhanced SWR cache invalidation** - Added comprehensive cache clearing when datasets are deleted - Robust cache key matching for both string and object formats - Immediate cache revalidation to ensure consistency ### **Files Changed**: - `web/app/components/app/configuration/dataset-config/select-dataset/index.tsx` - `web/app/(commonLayout)/datasets/DatasetCard.tsx` ## 🧪 Testing ### **Test Scenarios Completed** ���: 1. **Basic bug reproduction**: Delete dataset → Add new dataset → Check workflow 2. **Fast consecutive operations**: Rapid delete→add→check cycles 3. **Concurrent multi-tab scenarios**: Multiple workflow tabs with simultaneous dataset operations 4. **Network interruption handling**: Operations during network issues 5. **User interaction boundaries**: Manual selection/deselection behavior 6. **Pagination scenarios**: Large dataset lists with paging ### **Test Environment**: - Docker-based local deployment - Built custom test image with fixes - Verified in realistic production-like conditions ## 📊 Technical Details ### **Before Fix**: ```typescript // Created ghost objects with incomplete data const [selected, setSelected] = React.useState<DataSet[]>( selectedIds.map(id => ({ id }) as any) ) ``` ### **After Fix**: ```typescript // Clean initialization with proper data flow const [selected, setSelected] = React.useState<DataSet[]>([]) const [hasInitialized, setHasInitialized] = React.useState(false) // Proper initialization after data loads if (!hasInitialized && selectedIds.length > 0) { const validSelectedDatasets = selectedIds .map(id => newList.find(item => item.id === id)) .filter(Boolean) as DataSet[] setSelected(validSelectedDatasets) setHasInitialized(true) } ``` ### **Cache Invalidation**: ```typescript // Comprehensive SWR cache clearing mutate( key => { if (typeof key === 'string') return key.includes('/datasets') if (typeof key === 'object' && key !== null) { return key.url === '/datasets' || key.url?.includes('/datasets') } return false }, undefined, { revalidate: true } ) ``` ## 🔒 Risk Assessment ### **Low Risk Changes**: - ✅ Minimal code changes (only 2 files) - ✅ Targeted fixes addressing specific root causes - ✅ No breaking API changes - ✅ Backward compatible - ✅ Extensive edge case testing completed ### **Safeguards**: - Added race condition protection - Robust error handling - Graceful fallback behavior - Comprehensive cache key matching ## 📝 Checklist - [x] Bug fix tested and verified - [x] No new dependencies added - [x] Code follows existing patterns - [x] Edge cases tested thoroughly - [x] No performance regressions - [x] Documentation updated (inline comments) - [x] Backward compatibility maintained ## 🎯 Verification Steps To verify this fix: 1. **Basic verification**: ``` 1. Create a workflow with knowledge retrieval node 2. Add a dataset to the node 3. Delete that dataset from /datasets page 4. Add a new dataset 5. Return to workflow node configuration 6. ✅ Should only show the new dataset, no blank options ``` 2. **Edge case verification**: - Rapid consecutive operations - Multi-tab scenarios - Network interruption recovery ## 🚀 Impact **Positive Impact**: - ✅ Eliminates confusing ghost options in workflow UI - ✅ Improves workflow configuration reliability - ✅ Better user experience with consistent state - ✅ Reduces support requests related to this issue **No Negative Impact**: - ✅ No breaking changes - ✅ No performance degradation - ✅ No new dependencies --- **Ready for Review** 🔍 This PR addresses a user-facing bug with a clean, minimal solution that has been thoroughly tested in multiple scenarios.
yindo added the pull-request label 2026-02-21 20:46:41 -05:00
yindo closed this issue 2026-02-21 20:46:41 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#30014
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?