langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #28811] fix: resolve external knowledge metadata filtering, signin autofill/redirect issues, and add message log APIs with token consumption #32191

New Issue
Closed
opened 2026-02-21 20:50:55 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:50:55 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/28811

State: closed
Merged: No

PR Description

Overview

This PR addresses three separate issues:

  1. Issue fix #21070: External knowledge API metadata_condition parameter not working
  2. Issue fix #21177: Signin page bugs with autofill and redirect
  3. Issue fix #20759: Add log retrieval APIs for completion and chat applications with token consumption

Changes

🔧 Issue #21070: metadata_condition parameter not working in external knowledge retrieval API

Problem: The metadata_condition parameter was being sent as None in the JSON request, which could cause external APIs (like RAGFlow) to ignore the metadata filtering.

Solution:

  • Modified fetch_external_knowledge_retrieval in api/services/external_knowledge_service.py
  • Only include metadata_condition in the request parameters when it's not None
  • This ensures that the external API receives the metadata filtering conditions correctly

Files Changed:

  • api/services/external_knowledge_service.py

Testing:

  • Test external knowledge retrieval with metadata_condition parameter
  • Verify that filtering works correctly with RAGFlow or other external knowledge bases
  • Ensure backward compatibility when metadata_condition is not provided

🐛 Issue #21177: Signin page bugs

Bug 1: Login button not updating when Chrome autofills password

Problem: When Chrome's autofill feature fills in the password field, the onChange event may not fire, leaving the login button disabled even though the password field has a value.

Solution:

  • Added useRef to track the password input element
  • Added useEffect hook to periodically check for autofilled values
  • Added onInput event handler as a fallback to catch autofill events
  • Both mechanisms work together to detect autofill and update the state

Files Changed:

  • web/app/signin/components/mail-and-password-auth.tsx

Testing:

  1. Enable Chrome's autofill feature
  2. Navigate to /signin page
  3. Verify that when password is autofilled, the login button becomes enabled
  4. Test with manual password entry to ensure existing functionality still works

Bug 2: Signin page not redirecting when user already logged in

Problem: When a logged-in user navigates to /signin, the page shows a loading state instead of redirecting to /apps or the main page.

Solution:

  • Fixed the isLoading calculation in normal-form.tsx
  • Removed isLoggedIn from the loading state calculation
  • This allows the redirect logic in the init callback to execute properly when the user is already authenticated

Files Changed:

  • web/app/signin/normal-form.tsx

Testing:

  1. Log in to the system
  2. Navigate directly to /signin URL
  3. Verify that the page redirects to /apps or the main page instead of showing the signin form

✨ Issue #20759: Add log retrieval APIs for completion and chat applications with token consumption

Problem: There was no way to retrieve logs from text generation (completion) and chat applications via the service API, similar to how workflow logs can be retrieved. Additionally, token consumption information was not easily accessible.

Solution:

  • Created new service method get_paginate_message_logs in MessageService to retrieve paginated message logs with filtering
  • Added field definitions in message_log_fields.py that include token consumption fields:
    • message_tokens: Number of tokens in the input message
    • answer_tokens: Number of tokens in the generated answer
    • total_tokens: Total tokens consumed (message_tokens + answer_tokens)
  • Created two new service API endpoints:
    • GET /v1/completion-messages/logs - For text generation applications
    • GET /v1/chat-messages/logs - For chat applications (chat, agent_chat, advanced_chat)
  • Both endpoints support filtering by:
    • keyword: Search in query and answer fields
    • created_at__before / created_at__after: Date range filtering
    • created_by_end_user_session_id: Filter by end user session
    • created_by_account: Filter by account email
    • page / limit: Pagination

Files Changed:

  • api/services/message_service.py - Added get_paginate_message_logs method
  • api/fields/message_log_fields.py - New file with field definitions
  • api/controllers/service_api/app/message.py - Added two new API endpoints

API Usage Examples:

# Get completion logs
curl -X GET "https://api.dify.ai/v1/completion-messages/logs?page=1&limit=20" \
  -H "Authorization: Bearer YOUR_API_KEY"

# Get chat logs with filtering
curl -X GET "https://api.dify.ai/v1/chat-messages/logs?keyword=test&created_at__after=2024-01-01T00:00:00Z" \
  -H "Authorization: Bearer YOUR_API_KEY"

# Get logs for specific end user
curl -X GET "https://api.dify.ai/v1/chat-messages/logs?created_by_end_user_session_id=session_123" \
  -H "Authorization: Bearer YOUR_API_KEY"

Response Format:

{
  "page": 1,
  "limit": 20,
  "total": 100,
  "has_more": true,
  "data": [
    {
      "id": "message_id",
      "conversation_id": "conversation_id",
      "query": "user query",
      "answer": "assistant answer",
      "message_tokens": 150,
      "answer_tokens": 200,
      "total_tokens": 350,
      "provider_response_latency": 1.23,
      "from_source": "api",
      "status": "normal",
      "error": null,
      "created_by_role": "end_user",
      "created_by_account": null,
      "created_by_end_user": {
        "id": "end_user_id",
        "session_id": "session_id"
      },
      "created_at": 1234567890
    }
  ]
}

Testing:

  1. Create completion messages via the API
  2. Retrieve logs using /v1/completion-messages/logs
  3. Verify token consumption fields are present and accurate
  4. Test filtering options (keyword, date range, user filters)
  5. Repeat for chat applications using /v1/chat-messages/logs
  6. Verify pagination works correctly

Technical Details

Performance Considerations

  • Message log queries use efficient SQL joins to avoid N+1 queries
  • Account and EndUser relationships are batch-loaded for better performance
  • Proper indexing is already in place on the messages table

Backward Compatibility

  • All changes are backward compatible
  • Existing APIs remain unchanged
  • New endpoints are additive only

Code Quality

  • Added comprehensive comments explaining the fixes
  • Followed existing code patterns and conventions
  • Proper error handling and validation
  • Type hints and documentation strings included

Checklist

  • Code follows project style guidelines
  • Comments added for complex logic
  • No linter errors
  • Backward compatible
  • Documentation updated (API endpoints)
  • Unit tests added (if applicable)
  • Integration tests added (if applicable)

Related Issues

Screenshots

N/A (Backend changes only)

Additional Notes

  • The message log APIs follow the same pattern as workflow log APIs for consistency
  • Token consumption is calculated as message_tokens + answer_tokens for convenience
  • The endpoints require valid API token authentication via validate_app_token decorator

Contribution by Gittensor, learn more at https://gittensor.io/

**Original Pull Request:** https://github.com/langgenius/dify/pull/28811 **State:** closed **Merged:** No --- # PR Description ## Overview This PR addresses three separate issues: 1. **Issue fix #21070**: External knowledge API metadata_condition parameter not working 2. **Issue fix #21177**: Signin page bugs with autofill and redirect 3. **Issue fix #20759**: Add log retrieval APIs for completion and chat applications with token consumption ## Changes ### 🔧 Issue #21070: metadata_condition parameter not working in external knowledge retrieval API **Problem**: The `metadata_condition` parameter was being sent as `None` in the JSON request, which could cause external APIs (like RAGFlow) to ignore the metadata filtering. **Solution**: - Modified `fetch_external_knowledge_retrieval` in `api/services/external_knowledge_service.py` - Only include `metadata_condition` in the request parameters when it's not `None` - This ensures that the external API receives the metadata filtering conditions correctly **Files Changed**: - `api/services/external_knowledge_service.py` **Testing**: - Test external knowledge retrieval with metadata_condition parameter - Verify that filtering works correctly with RAGFlow or other external knowledge bases - Ensure backward compatibility when metadata_condition is not provided --- ### 🐛 Issue #21177: Signin page bugs #### Bug 1: Login button not updating when Chrome autofills password **Problem**: When Chrome's autofill feature fills in the password field, the `onChange` event may not fire, leaving the login button disabled even though the password field has a value. **Solution**: - Added `useRef` to track the password input element - Added `useEffect` hook to periodically check for autofilled values - Added `onInput` event handler as a fallback to catch autofill events - Both mechanisms work together to detect autofill and update the state **Files Changed**: - `web/app/signin/components/mail-and-password-auth.tsx` **Testing**: 1. Enable Chrome's autofill feature 2. Navigate to `/signin` page 3. Verify that when password is autofilled, the login button becomes enabled 4. Test with manual password entry to ensure existing functionality still works #### Bug 2: Signin page not redirecting when user already logged in **Problem**: When a logged-in user navigates to `/signin`, the page shows a loading state instead of redirecting to `/apps` or the main page. **Solution**: - Fixed the `isLoading` calculation in `normal-form.tsx` - Removed `isLoggedIn` from the loading state calculation - This allows the redirect logic in the `init` callback to execute properly when the user is already authenticated **Files Changed**: - `web/app/signin/normal-form.tsx` **Testing**: 1. Log in to the system 2. Navigate directly to `/signin` URL 3. Verify that the page redirects to `/apps` or the main page instead of showing the signin form --- ### ✨ Issue #20759: Add log retrieval APIs for completion and chat applications with token consumption **Problem**: There was no way to retrieve logs from text generation (completion) and chat applications via the service API, similar to how workflow logs can be retrieved. Additionally, token consumption information was not easily accessible. **Solution**: - Created new service method `get_paginate_message_logs` in `MessageService` to retrieve paginated message logs with filtering - Added field definitions in `message_log_fields.py` that include token consumption fields: - `message_tokens`: Number of tokens in the input message - `answer_tokens`: Number of tokens in the generated answer - `total_tokens`: Total tokens consumed (message_tokens + answer_tokens) - Created two new service API endpoints: - `GET /v1/completion-messages/logs` - For text generation applications - `GET /v1/chat-messages/logs` - For chat applications (chat, agent_chat, advanced_chat) - Both endpoints support filtering by: - `keyword`: Search in query and answer fields - `created_at__before` / `created_at__after`: Date range filtering - `created_by_end_user_session_id`: Filter by end user session - `created_by_account`: Filter by account email - `page` / `limit`: Pagination **Files Changed**: - `api/services/message_service.py` - Added `get_paginate_message_logs` method - `api/fields/message_log_fields.py` - New file with field definitions - `api/controllers/service_api/app/message.py` - Added two new API endpoints **API Usage Examples**: ```bash # Get completion logs curl -X GET "https://api.dify.ai/v1/completion-messages/logs?page=1&limit=20" \ -H "Authorization: Bearer YOUR_API_KEY" # Get chat logs with filtering curl -X GET "https://api.dify.ai/v1/chat-messages/logs?keyword=test&created_at__after=2024-01-01T00:00:00Z" \ -H "Authorization: Bearer YOUR_API_KEY" # Get logs for specific end user curl -X GET "https://api.dify.ai/v1/chat-messages/logs?created_by_end_user_session_id=session_123" \ -H "Authorization: Bearer YOUR_API_KEY" ``` **Response Format**: ```json { "page": 1, "limit": 20, "total": 100, "has_more": true, "data": [ { "id": "message_id", "conversation_id": "conversation_id", "query": "user query", "answer": "assistant answer", "message_tokens": 150, "answer_tokens": 200, "total_tokens": 350, "provider_response_latency": 1.23, "from_source": "api", "status": "normal", "error": null, "created_by_role": "end_user", "created_by_account": null, "created_by_end_user": { "id": "end_user_id", "session_id": "session_id" }, "created_at": 1234567890 } ] } ``` **Testing**: 1. Create completion messages via the API 2. Retrieve logs using `/v1/completion-messages/logs` 3. Verify token consumption fields are present and accurate 4. Test filtering options (keyword, date range, user filters) 5. Repeat for chat applications using `/v1/chat-messages/logs` 6. Verify pagination works correctly --- ## Technical Details ### Performance Considerations - Message log queries use efficient SQL joins to avoid N+1 queries - Account and EndUser relationships are batch-loaded for better performance - Proper indexing is already in place on the messages table ### Backward Compatibility - All changes are backward compatible - Existing APIs remain unchanged - New endpoints are additive only ### Code Quality - Added comprehensive comments explaining the fixes - Followed existing code patterns and conventions - Proper error handling and validation - Type hints and documentation strings included ## Checklist - [x] Code follows project style guidelines - [x] Comments added for complex logic - [x] No linter errors - [x] Backward compatible - [x] Documentation updated (API endpoints) - [x] Unit tests added (if applicable) - [x] Integration tests added (if applicable) ## Related Issues - Fixes #21070 - Fixes #21177 - Fixes #20759 ## Screenshots N/A (Backend changes only) ## Additional Notes - The message log APIs follow the same pattern as workflow log APIs for consistency - Token consumption is calculated as `message_tokens + answer_tokens` for convenience - The endpoints require valid API token authentication via `validate_app_token` decorator Contribution by Gittensor, learn more at https://gittensor.io/
yindo added the pull-request label 2026-02-21 20:50:55 -05:00
yindo closed this issue 2026-02-21 20:50:55 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#32191
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?