langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-22 01:55:27 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

[PR #1619] [MERGED] Fix/bedrock inference profile cache support #1991

New Issue
Closed
opened 2026-02-16 11:15:47 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 11:15:47 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langgenius/dify-official-plugins/pull/1619
Author: @akshaysinghas
Created: 8/30/2025
Status: ✅ Merged
Merged: 9/1/2025
Merged by: @crazywoola

Base: main ← Head: fix/bedrock-inference-profile-cache-support

📝 Commits (10+)

  • e354aa1 fix(bedrock): improve cache support detection for inference profiles
  • eaf982e fix(bedrock): change cache checkpoint defaults to respect UI settings
  • 359a399 fix(bedrock): clean up and apply only essential cache support fixes
  • 0dca1cf Address PR review feedback: Replace print statements with logger calls and sanitize logging
  • a5223e0 Add back cache debug logs with appropriate logger.debug() level
  • a8fe744 Remove [CACHE DEBUG] markers and use logger.debug()
  • ae1dcac Remove dead code comment and improve cache checkpoint default explanation
  • c7d8768 Consolidate duplicate stream warning messages
  • 1fa707f Remove obsolete debug comment
  • efeb140 Remove unnecessary cache params debug logging

📊 Changes

3 files changed (+26 additions, -36 deletions)

View changed files

📝 models/bedrock/manifest.yaml (+1 -1)
📝 models/bedrock/models/llm/cache_config.py (+0 -2)
📝 models/bedrock/models/llm/llm.py (+25 -33)

📄 Description

Related Issues or Context

Context: Fixed cache support detection for AWS Bedrock inference profiles. Previously, cache support was incorrectly evaluated as false for inference profiles even when the underlying model supported caching (e.g., Nova Pro, Claude 3.7 Sonnet).

This PR contains Changes to Non-Plugin

  • Documentation
  • Other

This PR contains Changes to Non-LLM Models Plugin

  • I have Run Comprehensive Tests Relevant to My Changes

This PR contains Changes to LLM Models Plugin

  • My Changes Affect Message Flow Handling (System Messages and User→Assistant Turn-Taking)
  • My Changes Affect Tool Interaction Flow (Multi-Round Usage and Output Handling, for both Agent App and Agent Node)
  • My Changes Affect Multimodal Input Handling (Images, PDFs, Audio, Video, etc.)
  • My Changes Affect Multimodal Output Generation (Images, Audio, Video, etc.)
  • My Changes Affect Structured Output Format (JSON, XML, etc.)
  • My Changes Affect Token Consumption Metrics
  • My Changes Affect Other LLM Functionalities (Reasoning Process, Grounding, Prompt Caching, etc.)

Changes to Prompt Caching for Inference Profiles:

  • Fixed cache support detection for AWS Bedrock inference profiles
  • Cache support now correctly uses underlying model ID instead of inference profile ARN
  • Cache settings from UI are properly respected (defaults changed from True to False)

Testing Results:

Before Fix:
[CACHE DEBUG] Model: arn:aws:bedrock:XXX:application-inference-profile/XXX, Cache supported: False
[CACHE METRICS] Read: 0 tokens, Write: 0 tokens (even with cache-supported models)

After Fix:
[CACHE DEBUG] Model: arn:aws:bedrock:XXX:application-inference-profile/XXX, Underlying: amazon.nova-pro-v1:0, Cache supported: True
When cache enabled in UI: [CACHE METRICS] Read: 0 tokens, Write: 1128 tokens
When cache disabled in UI: [CACHE METRICS] Read: 0 tokens, Write: 0 tokens
  • Other Changes (Add New Models, Fix Model Parameters etc.)

Bug Fix: Corrected cache parameter defaults to respect UI settings

Version Control (Any Changes to the Plugin Will Require Bumping the Version)

  • I have Bumped Up the Version in Manifest.yaml (Top-Level Version Field, Not in Meta Section)

Version bumped from 0.0.33 to 0.0.34 (PATCH - bug fix)

Dify Plugin SDK Version

  • I have Ensured dify_plugin>=0.3.0,<0.5.0 is in requirements.txt (SDK docs)

Environment Verification (If Any Code Changes)

Local Deployment Environment

  • Dify Version is: 1.2.0+, I have Tested My Changes on Local Deployment Dify with a Clean Environment That Matches the Production Configuration.

Tested with both Nova Pro and Claude 3.5 Sonnet inference profiles on local Dify deployment.

SaaS Environment

  • I have Tested My Changes on cloud.dify.ai with a Clean Environment That Matches the Production Configuration

These images reflect logs when enabled the cache from UI and disabled from UI for system prompt. Observe the diff in Cache checkpoints - System: True/False
Screenshot 2025-08-30 at 5 09 55 PM
0-4c5e22c17025" />
Screenshot 2025-08-30 at 5 10 09 PM
When true is reads from cache

Screenshot 2025-08-30 at 5 07 56 PM Screenshot 2025-08-30 at 5 07 42 PM

When false nothing read from cache
Screenshot 2025-08-30 at 5 11 55 PM

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langgenius/dify-official-plugins/pull/1619 **Author:** [@akshaysinghas](https://github.com/akshaysinghas) **Created:** 8/30/2025 **Status:** ✅ Merged **Merged:** 9/1/2025 **Merged by:** [@crazywoola](https://github.com/crazywoola) **Base:** `main` ← **Head:** `fix/bedrock-inference-profile-cache-support` --- ### 📝 Commits (10+) - [`e354aa1`](https://github.com/langgenius/dify-official-plugins/commit/e354aa11af4ec08563c4d31530ad0baaf4a787ed) fix(bedrock): improve cache support detection for inference profiles - [`eaf982e`](https://github.com/langgenius/dify-official-plugins/commit/eaf982ecc9e51edaea39b0879d892bb6ca1ab685) fix(bedrock): change cache checkpoint defaults to respect UI settings - [`359a399`](https://github.com/langgenius/dify-official-plugins/commit/359a399a368f893fd4091060dd38cefc1850e9c9) fix(bedrock): clean up and apply only essential cache support fixes - [`0dca1cf`](https://github.com/langgenius/dify-official-plugins/commit/0dca1cfa0ef012346c95c703181d7f4d9eba1680) Address PR review feedback: Replace print statements with logger calls and sanitize logging - [`a5223e0`](https://github.com/langgenius/dify-official-plugins/commit/a5223e0f51939f2d3b6cf2d4d98e91740f93f6db) Add back cache debug logs with appropriate logger.debug() level - [`a8fe744`](https://github.com/langgenius/dify-official-plugins/commit/a8fe7445937307b572ad28816982f8bed0b7494b) Remove [CACHE DEBUG] markers and use logger.debug() - [`ae1dcac`](https://github.com/langgenius/dify-official-plugins/commit/ae1dcac4a0b9c453c8653a3929231ae6ee3c001b) Remove dead code comment and improve cache checkpoint default explanation - [`c7d8768`](https://github.com/langgenius/dify-official-plugins/commit/c7d87689d3c20c8240ede1fb7dbe012b4c0d59bb) Consolidate duplicate stream warning messages - [`1fa707f`](https://github.com/langgenius/dify-official-plugins/commit/1fa707f22ed88a97d0a4dc2bbd6dff4e8ee19522) Remove obsolete debug comment - [`efeb140`](https://github.com/langgenius/dify-official-plugins/commit/efeb140d7ab4458d16baf4269c78d072c1783935) Remove unnecessary cache params debug logging ### 📊 Changes **3 files changed** (+26 additions, -36 deletions) <details> <summary>View changed files</summary> 📝 `models/bedrock/manifest.yaml` (+1 -1) 📝 `models/bedrock/models/llm/cache_config.py` (+0 -2) 📝 `models/bedrock/models/llm/llm.py` (+25 -33) </details> ### 📄 Description ## Related Issues or Context <!-- ⚠️ NOTE: This repository is for Dify Official Plugins only. For community contributions, please submit to https://github.com/langgenius/dify-plugins instead. - Link Related Issues if Applicable: #issue_number - Or Provide Context about Why this Change is Needed --> Context: Fixed cache support detection for AWS Bedrock inference profiles. Previously, cache support was incorrectly evaluated as false for inference profiles even when the underlying model supported caching (e.g., Nova Pro, Claude 3.7 Sonnet). ## This PR contains Changes to *Non-Plugin* <!-- Put an `x` in all the boxes that apply by replacing [ ] with [x] For example: - [x] Documentation --> - [ ] Documentation - [ ] Other ## This PR contains Changes to *Non-LLM Models Plugin* - [ ] I have Run Comprehensive Tests Relevant to My Changes <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> ## This PR contains Changes to *LLM Models Plugin* <!-- LLM Models Test Example: --> <!-- https://github.com/langgenius/dify-official-plugins/blob/main/.assets/test-examples/llm-plugin-tests/llm_test_example.md --> - [ ] My Changes Affect Message Flow Handling (System Messages and User→Assistant Turn-Taking) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Tool Interaction Flow (Multi-Round Usage and Output Handling, for both Agent App and Agent Node) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Multimodal Input Handling (Images, PDFs, Audio, Video, etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Multimodal Output Generation (Images, Audio, Video, etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Structured Output Format (JSON, XML, etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Token Consumption Metrics <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [x] My Changes Affect Other LLM Functionalities (Reasoning Process, Grounding, Prompt Caching, etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> **Changes to Prompt Caching for Inference Profiles:** - Fixed cache support detection for AWS Bedrock inference profiles - Cache support now correctly uses underlying model ID instead of inference profile ARN - Cache settings from UI are properly respected (defaults changed from True to False) **Testing Results:** ``` Before Fix: [CACHE DEBUG] Model: arn:aws:bedrock:XXX:application-inference-profile/XXX, Cache supported: False [CACHE METRICS] Read: 0 tokens, Write: 0 tokens (even with cache-supported models) After Fix: [CACHE DEBUG] Model: arn:aws:bedrock:XXX:application-inference-profile/XXX, Underlying: amazon.nova-pro-v1:0, Cache supported: True When cache enabled in UI: [CACHE METRICS] Read: 0 tokens, Write: 1128 tokens When cache disabled in UI: [CACHE METRICS] Read: 0 tokens, Write: 0 tokens ``` - [x] Other Changes (Add New Models, Fix Model Parameters etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> **Bug Fix:** Corrected cache parameter defaults to respect UI settings ## Version Control (Any Changes to the Plugin Will Require Bumping the Version) - [x] I have Bumped Up the Version in Manifest.yaml (Top-Level `Version` Field, Not in Meta Section) <!-- ⚠️ NOTE: Version Format: MAJOR.MINOR.PATCH - MAJOR (0.x.x): Reserved for Significant architectural changes or incompatible API modifications - MINOR (x.0.x): For New feature additions while maintaining backward compatibility - PATCH (x.x.0): For Backward-compatible bug fixes and minor improvements - Note: Each Version Component (MAJOR, MINOR, PATCH) Can Be 2 Digits, e.g., 10.11.22 --> Version bumped from 0.0.33 to 0.0.34 (PATCH - bug fix) ## Dify Plugin SDK Version - [x] I have Ensured `dify_plugin>=0.3.0,<0.5.0` is in requirements.txt ([SDK docs](https://github.com/langgenius/dify-plugin-sdks/blob/main/python/README.md)) ## Environment Verification (If Any Code Changes) <!-- ⚠️ NOTE: At Least One Environment Must Be Tested. --> ### Local Deployment Environment - [x] Dify Version is: 1.2.0+, I have Tested My Changes on Local Deployment Dify with a Clean Environment That Matches the Production Configuration. <!-- - Python Virtual Env Matching Manifest.yaml & requirements.txt - No Breaking Changes in Dify That May Affect the Testing Result --> Tested with both Nova Pro and Claude 3.5 Sonnet inference profiles on local Dify deployment. ### SaaS Environment - [ ] I have Tested My Changes on cloud.dify.ai with a Clean Environment That Matches the Production Configuration <!-- - Python Virtual Env Matching Manifest.yaml & requirements.txt --> These images reflect logs when enabled the cache from UI and disabled from UI for system prompt. Observe the diff in Cache checkpoints - System: True/False <img width="1605" height="49" alt="Screenshot 2025-08-30 at 5 09 55 PM" src="https://github.com/user-attachments/assets/aab6980a-434b-49f3-885e-65c46758d069" /> 0-4c5e22c17025" /> <img width="1655" height="55" alt="Screenshot 2025-08-30 at 5 10 09 PM" src="https://github.com/user-attachments/assets/859e0c8a-61d3-42d3-9caf-1301a7be610d" /> When true is reads from cache <img width="1542" height="47" alt="Screenshot 2025-08-30 at 5 07 56 PM" src="https://github.com/user-attachments/assets/7a252016-3241-4e73-b4a0-c645bb448132" /> <img width="1563" height="82" alt="Screenshot 2025-08-30 at 5 07 42 PM" src="https://github.com/user-attachments/assets/3d787d24-7985-403f-807f-12c526f4d3ef" /> When false nothing read from cache <img width="1577" height="64" alt="Screenshot 2025-08-30 at 5 11 55 PM" src="https://github.com/user-attachments/assets/5235a5e8-e4d5-4fd8-8670-fe75fc024b49" /> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-16 11:15:47 -05:00
yindo closed this issue 2026-02-16 11:15:47 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#1991
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?