langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

[PR #1422] [MERGED] feat(bedrock): Enhanced inference profile support for all model types #1880

New Issue
Closed
opened 2026-02-16 10:24:16 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 10:24:16 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langgenius/dify-official-plugins/pull/1422
Author: @akshaysinghas
Created: 8/5/2025
Status: ✅ Merged
Merged: 8/8/2025
Merged by: @crazywoola

Base: main ← Head: model-option-inference-profile

📝 Commits (6)

  • 365652c feat(bedrock): Enhanced inference profile support for all model types
  • 9e3241e fix(bedrock): Hide Bedrock Model parameter for inference profiles
  • 3b2a034 chore(bedrock): Bump version to 0.0.28
  • 6b39f1b resolve: Merge conflicts with main branch
  • 4f134dc chore(bedrock): Bump version to 0.0.29
  • 856ff33 refactor(bedrock): Address PR feedback - remove manual model selection and deduplicate code

📊 Changes

9 files changed (+532 additions, -89 deletions)

View changed files

📝 models/bedrock/manifest.yaml (+1 -1)
📝 models/bedrock/models/llm/llm.py (+43 -72)
➕ models/bedrock/models/rerank/model_ids.py (+41 -0)
📝 models/bedrock/models/rerank/rerank.py (+130 -6)
➕ models/bedrock/models/text_embedding/model_ids.py (+43 -0)
📝 models/bedrock/models/text_embedding/text_embedding.py (+143 -5)
📝 models/bedrock/provider/bedrock.yaml (+6 -5)
➕ models/bedrock/utils/__init__.py (+1 -0)
➕ models/bedrock/utils/inference_profile.py (+124 -0)

📄 Description

Summary

This PR implements comprehensive inference profile support for all AWS Bedrock model types (LLM, text embedding, and rerank), addressing the limitations identified in issue #1421.

🎯 Key Improvements

1. Universal Inference Profile Support

  • ✅ LLM models: Enhanced with parameter rules loading
  • ✅ Text embedding models: Full inference profile support added
  • ✅ Rerank models: Full inference profile support added
  • ✅ Consistent validation: Unified credential validation across all model types

2. Enhanced User Experience

  • 🎨 Model provider dropdown: Select from Anthropic Claude, Amazon, Cohere, etc.
  • 🎨 Model name dropdown: Dynamic options based on selected provider
  • 🎨 Better help text: Clear instructions and examples
  • 🎨 User-controlled naming: Model names show exactly what users enter

3. Parameter Rules Integration

  • ⚙️ Dynamic parameter loading: Load parameters from underlying model configurations
  • ⚙️ Model-specific options: Users see relevant parameters for their chosen model
  • ⚙️ Schema matching: Automatic matching of inference profiles to predefined model schemas

📁 Files Changed

New Files:

  • models/bedrock/models/text_embedding/model_ids.py - Text embedding model ID mappings
  • models/bedrock/models/rerank/model_ids.py - Rerank model ID mappings

Modified Files:

  • models/bedrock/provider/bedrock.yaml - Enhanced configuration with dropdowns
  • models/bedrock/models/llm/llm.py - Parameter rules loading and improved schema
  • models/bedrock/models/text_embedding/text_embedding.py - Added inference profile support
  • models/bedrock/models/rerank/rerank.py - Added inference profile support

🧪 Testing

  • LLM inference profiles with parameter rules loading
  • Text embedding inference profiles with validation
  • Rerank inference profiles with validation
  • Dynamic dropdown options based on model type
  • User-provided model names preserved
  • Backward compatibility maintained
  • Syntax validation passed for all files

🔄 Backward Compatibility

  • ✅ Existing configurations continue to work
  • ✅ No breaking API changes
  • ✅ Additional fields are optional
  • ✅ Enhanced functionality is additive

📋 Configuration Example

Before (LLM only):

inference_profile_id: "abc123"
context_length: "4096"

After (All model types):

inference_profile_id: "abc123"
underlying_model_type: "anthropic claude"
underlying_model_name: "Claude 3.5 Sonnet V2"
context_length: "4096"

🎁 Benefits

  1. Complete Coverage: All Bedrock model types can use inference profiles
  2. Better UX: Dropdown selections instead of manual ID entry
  3. Proper Parameters: Users see all available model-specific parameters
  4. Consistent Validation: Uniform error handling across model types
  5. User Control: Model names display exactly as entered by users

🔗 Related

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

Evidences from local testing:
Screenshot 2025-08-07 at 11 59 43 AM
Screenshot 2025-08-07 at 11 59 22 AM
Screenshot 2025-08-07 at 11 59 17 AM
Changed the maxTokens to 50 to test parameter is working
Screenshot 2025-08-07 at 12 01 42 PM
Screenshot 2025-08-07 at 12 02 16 PM

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langgenius/dify-official-plugins/pull/1422 **Author:** [@akshaysinghas](https://github.com/akshaysinghas) **Created:** 8/5/2025 **Status:** ✅ Merged **Merged:** 8/8/2025 **Merged by:** [@crazywoola](https://github.com/crazywoola) **Base:** `main` ← **Head:** `model-option-inference-profile` --- ### 📝 Commits (6) - [`365652c`](https://github.com/langgenius/dify-official-plugins/commit/365652c2d83b6218119242af36976b34bd0799c1) feat(bedrock): Enhanced inference profile support for all model types - [`9e3241e`](https://github.com/langgenius/dify-official-plugins/commit/9e3241ece19d3afb475b4e757d451edad5b7dcee) fix(bedrock): Hide Bedrock Model parameter for inference profiles - [`3b2a034`](https://github.com/langgenius/dify-official-plugins/commit/3b2a0342a9b7e0a89f488cc681fdeabc864d836f) chore(bedrock): Bump version to 0.0.28 - [`6b39f1b`](https://github.com/langgenius/dify-official-plugins/commit/6b39f1b2c9372e77a6ba0d4b8726f7fbf89ce2cb) resolve: Merge conflicts with main branch - [`4f134dc`](https://github.com/langgenius/dify-official-plugins/commit/4f134dc70da8a4f6baad4d55e5574749c67f60cc) chore(bedrock): Bump version to 0.0.29 - [`856ff33`](https://github.com/langgenius/dify-official-plugins/commit/856ff338ca77c83c8a078e47a7eee4aaa39b9e28) refactor(bedrock): Address PR feedback - remove manual model selection and deduplicate code ### 📊 Changes **9 files changed** (+532 additions, -89 deletions) <details> <summary>View changed files</summary> 📝 `models/bedrock/manifest.yaml` (+1 -1) 📝 `models/bedrock/models/llm/llm.py` (+43 -72) ➕ `models/bedrock/models/rerank/model_ids.py` (+41 -0) 📝 `models/bedrock/models/rerank/rerank.py` (+130 -6) ➕ `models/bedrock/models/text_embedding/model_ids.py` (+43 -0) 📝 `models/bedrock/models/text_embedding/text_embedding.py` (+143 -5) 📝 `models/bedrock/provider/bedrock.yaml` (+6 -5) ➕ `models/bedrock/utils/__init__.py` (+1 -0) ➕ `models/bedrock/utils/inference_profile.py` (+124 -0) </details> ### 📄 Description ## Summary This PR implements comprehensive inference profile support for all AWS Bedrock model types (LLM, text embedding, and rerank), addressing the limitations identified in issue #1421. ## 🎯 Key Improvements ### 1. **Universal Inference Profile Support** - ✅ **LLM models**: Enhanced with parameter rules loading - ✅ **Text embedding models**: Full inference profile support added - ✅ **Rerank models**: Full inference profile support added - ✅ **Consistent validation**: Unified credential validation across all model types ### 2. **Enhanced User Experience** - 🎨 **Model provider dropdown**: Select from Anthropic Claude, Amazon, Cohere, etc. - 🎨 **Model name dropdown**: Dynamic options based on selected provider - 🎨 **Better help text**: Clear instructions and examples - 🎨 **User-controlled naming**: Model names show exactly what users enter ### 3. **Parameter Rules Integration** - ⚙️ **Dynamic parameter loading**: Load parameters from underlying model configurations - ⚙️ **Model-specific options**: Users see relevant parameters for their chosen model - ⚙️ **Schema matching**: Automatic matching of inference profiles to predefined model schemas ## 📁 Files Changed ### New Files: - `models/bedrock/models/text_embedding/model_ids.py` - Text embedding model ID mappings - `models/bedrock/models/rerank/model_ids.py` - Rerank model ID mappings ### Modified Files: - `models/bedrock/provider/bedrock.yaml` - Enhanced configuration with dropdowns - `models/bedrock/models/llm/llm.py` - Parameter rules loading and improved schema - `models/bedrock/models/text_embedding/text_embedding.py` - Added inference profile support - `models/bedrock/models/rerank/rerank.py` - Added inference profile support ## 🧪 Testing - [x] LLM inference profiles with parameter rules loading - [x] Text embedding inference profiles with validation - [x] Rerank inference profiles with validation - [x] Dynamic dropdown options based on model type - [x] User-provided model names preserved - [x] Backward compatibility maintained - [x] Syntax validation passed for all files ## 🔄 Backward Compatibility - ✅ **Existing configurations continue to work** - ✅ **No breaking API changes** - ✅ **Additional fields are optional** - ✅ **Enhanced functionality is additive** ## 📋 Configuration Example Before (LLM only): ```yaml inference_profile_id: "abc123" context_length: "4096" ``` After (All model types): ```yaml inference_profile_id: "abc123" underlying_model_type: "anthropic claude" underlying_model_name: "Claude 3.5 Sonnet V2" context_length: "4096" ``` ## 🎁 Benefits 1. **Complete Coverage**: All Bedrock model types can use inference profiles 2. **Better UX**: Dropdown selections instead of manual ID entry 3. **Proper Parameters**: Users see all available model-specific parameters 4. **Consistent Validation**: Uniform error handling across model types 5. **User Control**: Model names display exactly as entered by users ## 🔗 Related - Fixes #1421 - Addresses all issues mentioned in the original issue - Implements the complete solution as proposed --- 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> Evidences from local testing: <img width="282" height="543" alt="Screenshot 2025-08-07 at 11 59 43 AM" src="https://github.com/user-attachments/assets/1a6e2e22-bd7f-4b73-9f17-1210337cd20b" /> <img width="591" height="548" alt="Screenshot 2025-08-07 at 11 59 22 AM" src="https://github.com/user-attachments/assets/08304422-2ea9-425d-a00c-e717ecf1f44a" /> <img width="562" height="522" alt="Screenshot 2025-08-07 at 11 59 17 AM" src="https://github.com/user-attachments/assets/a2e7b5c2-77a5-4ebd-ae8c-6fa343172dae" /> Changed the maxTokens to 50 to test parameter is working <img width="664" height="792" alt="Screenshot 2025-08-07 at 12 01 42 PM" src="https://github.com/user-attachments/assets/57601d63-ca2f-44e8-8bc2-71a5c60e9f9f" /> <img width="989" height="773" alt="Screenshot 2025-08-07 at 12 02 16 PM" src="https://github.com/user-attachments/assets/f30eeffb-a3de-4392-bb9f-30db2b9956a8" /> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-16 10:24:16 -05:00
yindo closed this issue 2026-02-16 10:24:16 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#1880
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?