langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-22 01:55:27 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

Enhanced Bedrock Inference Profile Support for All Model Types #520

New Issue
Closed
opened 2026-02-16 10:19:38 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 10:19:38 -05:00
Owner
Copy Link
Copy Source

Originally created by @akshaysinghas on GitHub (Aug 5, 2025).

Summary

The current Bedrock plugin has limited inference profile support that only works for LLM models. This enhancement request proposes extending inference profile functionality to all model types (LLM, text embedding, and rerank) with improved user experience and proper parameter rule loading.

Current Issues

1. Limited Model Type Support

  • Inference profiles only work for LLM models
  • Text embedding and rerank models cannot use inference profiles
  • Configuration in bedrock.yaml restricts inference_profile_id to LLM only

2. Poor User Experience

  • Users must manually enter inference profile IDs without guidance
  • No dropdown selection of underlying models from known Bedrock models
  • Missing model-specific parameter rules when using inference profiles

3. Missing Infrastructure

  • No model ID mapping files for text embedding and rerank models
  • Parameter rules not loaded from underlying model configurations
  • Validation logic doesn't support inference profiles for non-LLM models

Proposed Solution

1. Universal Inference Profile Support

  • Remove show_on restrictions for inference_profile_id field
  • Add support in text_embedding.py and rerank.py for inference profiles
  • Implement get_customizable_model_schema() for all model types

2. Enhanced Configuration UI

  • Add underlying_model_type dropdown (e.g., "Anthropic Claude", "Amazon", "Cohere")
  • Add underlying_model_name dropdown with model-specific options
  • Create model ID mapping files for text embedding and rerank models

3. Parameter Rules Integration

  • Load parameter rules from underlying model YAML files
  • Display model-specific parameters in the UI when using inference profiles
  • Ensure proper validation and schema generation

Implementation Details

New Files Needed:

  • models/bedrock/models/text_embedding/model_ids.py
  • models/bedrock/models/rerank/model_ids.py

Modified Files:

  • models/bedrock/provider/bedrock.yaml - Enhanced configuration schema
  • models/bedrock/models/llm/llm.py - Parameter rules loading
  • models/bedrock/models/text_embedding/text_embedding.py - Inference profile support
  • models/bedrock/models/rerank/rerank.py - Inference profile support

Configuration Schema:

model_credential_schema:
  credential_form_schemas:
    - variable: inference_profile_id
      # No longer restricted to LLM only
      required: true
    - variable: underlying_model_type
      type: select
      options: # Dynamic based on model type
        - anthropic claude (for LLM)
        - amazon (for text-embedding/rerank)
        - cohere (for text-embedding/rerank)
    - variable: underlying_model_name
      type: select
      options: # Dynamic based on underlying_model_type

Expected Benefits

  1. Complete Inference Profile Coverage - All Bedrock model types can use inference profiles
  2. Better User Experience - Dropdown selections instead of manual entry
  3. Proper Parameter Display - Users see all available model parameters
  4. Consistent Validation - Uniform credential validation across model types
  5. Improved Error Handling - Better error messages and validation feedback

Testing Requirements

  • LLM inference profiles with parameter rules loading
  • Text embedding inference profiles with proper validation
  • Rerank inference profiles with proper validation
  • UI displays correct dropdown options based on model type
  • Parameter rules appear correctly for different underlying models
  • Credential validation works for all model types

Backward Compatibility

This enhancement maintains full backward compatibility:

  • Existing LLM inference profile configurations continue to work
  • No breaking changes to existing API or configuration
  • Additional fields are optional and enhance functionality

Priority: High

This addresses a significant gap in Bedrock plugin functionality and greatly improves the user experience for AWS Bedrock inference profiles across all supported model types.

Originally created by @akshaysinghas on GitHub (Aug 5, 2025). ## Summary The current Bedrock plugin has limited inference profile support that only works for LLM models. This enhancement request proposes extending inference profile functionality to all model types (LLM, text embedding, and rerank) with improved user experience and proper parameter rule loading. ## Current Issues ### 1. **Limited Model Type Support** - Inference profiles only work for LLM models - Text embedding and rerank models cannot use inference profiles - Configuration in `bedrock.yaml` restricts `inference_profile_id` to LLM only ### 2. **Poor User Experience** - Users must manually enter inference profile IDs without guidance - No dropdown selection of underlying models from known Bedrock models - Missing model-specific parameter rules when using inference profiles ### 3. **Missing Infrastructure** - No model ID mapping files for text embedding and rerank models - Parameter rules not loaded from underlying model configurations - Validation logic doesn't support inference profiles for non-LLM models ## Proposed Solution ### 1. **Universal Inference Profile Support** - Remove `show_on` restrictions for `inference_profile_id` field - Add support in `text_embedding.py` and `rerank.py` for inference profiles - Implement `get_customizable_model_schema()` for all model types ### 2. **Enhanced Configuration UI** - Add `underlying_model_type` dropdown (e.g., "Anthropic Claude", "Amazon", "Cohere") - Add `underlying_model_name` dropdown with model-specific options - Create model ID mapping files for text embedding and rerank models ### 3. **Parameter Rules Integration** - Load parameter rules from underlying model YAML files - Display model-specific parameters in the UI when using inference profiles - Ensure proper validation and schema generation ## Implementation Details ### New Files Needed: - `models/bedrock/models/text_embedding/model_ids.py` - `models/bedrock/models/rerank/model_ids.py` ### Modified Files: - `models/bedrock/provider/bedrock.yaml` - Enhanced configuration schema - `models/bedrock/models/llm/llm.py` - Parameter rules loading - `models/bedrock/models/text_embedding/text_embedding.py` - Inference profile support - `models/bedrock/models/rerank/rerank.py` - Inference profile support ### Configuration Schema: ```yaml model_credential_schema: credential_form_schemas: - variable: inference_profile_id # No longer restricted to LLM only required: true - variable: underlying_model_type type: select options: # Dynamic based on model type - anthropic claude (for LLM) - amazon (for text-embedding/rerank) - cohere (for text-embedding/rerank) - variable: underlying_model_name type: select options: # Dynamic based on underlying_model_type ``` ## Expected Benefits 1. **Complete Inference Profile Coverage** - All Bedrock model types can use inference profiles 2. **Better User Experience** - Dropdown selections instead of manual entry 3. **Proper Parameter Display** - Users see all available model parameters 4. **Consistent Validation** - Uniform credential validation across model types 5. **Improved Error Handling** - Better error messages and validation feedback ## Testing Requirements - [ ] LLM inference profiles with parameter rules loading - [ ] Text embedding inference profiles with proper validation - [ ] Rerank inference profiles with proper validation - [ ] UI displays correct dropdown options based on model type - [ ] Parameter rules appear correctly for different underlying models - [ ] Credential validation works for all model types ## Backward Compatibility This enhancement maintains full backward compatibility: - Existing LLM inference profile configurations continue to work - No breaking changes to existing API or configuration - Additional fields are optional and enhance functionality ## Priority: High This addresses a significant gap in Bedrock plugin functionality and greatly improves the user experience for AWS Bedrock inference profiles across all supported model types.
yindo closed this issue 2026-02-16 10:19:39 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#520
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?