Claude Sonnet 3.7 Model Not Supported in aws_bedrock_claude_pipeline.py Due to Inference Profile Requirement #220

New Issue

2026-02-15T19:17:03-05:00

yindo commented

2026-02-15 19:17:03 -05:00

Originally created by @samuelpetermoshi on GitHub (Apr 6, 2025).

hello @g453030291,

Issue Summary

OpenWebUI is currently unable to use the Claude Sonnet 3.7 model via aws_bedrock_claude_pipeline.py. The model does not appear in the available model list due to a filtering condition and fails invocation due to a missing inference profile.

https://github.com/open-webui/pipelines/blob/main/examples/pipelines/providers/aws_bedrock_claude_pipeline.py

Root Cause Analysis (RCA)

1. Model Filtering Excludes Claude Sonnet 3.7

The pipeline script retrieves available models using the following line:

https://github.com/propertyguru/ds-openwebui/blob/bbd5eba50ea32a3e7e4e2e526a5794679d401b2b/pg_openwebui_pipelines/pipelines/aws_bedrock_claude_pipeline.py#L150

response = self.bedrock.list_foundation_models(byProvider='Anthropic', byInferenceType='ON_DEMAND')

However, Claude Sonnet 3.7 is not an on-demand model. As a result, it is excluded from the model list due to this filtering.

2. Invocation Requires an Inference Profile

Even when the model is manually added or the filtering condition is removed, invocation fails with the following error:

Invocation of model ID anthropic.claude-3-7-sonnet-20250219-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.

This confirms that Claude Sonnet 3.7 must be invoked using an inference profile ARN, rather than via the default on-demand method.

Suggested Fix

Update the model listing logic to include models that are not on-demand (e.g., by removing or making byInferenceType optional).
Support invocation using an inference profile ARN for models that require it, like Claude Sonnet 3.7.

Reference video explaining the inference profile flow:
📹 https://youtu.be/7-BIpBn9ao4?si=HZksyMVVR__eKfhv&t=343

Impact

Claude Sonnet 3.7 is currently inaccessible from OpenWebUI.
Users cannot leverage the latest capabilities from the Claude 3.x series unless the above is addressed.

cc: @tjbck

Originally created by @samuelpetermoshi on GitHub (Apr 6, 2025). hello @g453030291, **Issue Summary** OpenWebUI is currently unable to use the Claude Sonnet 3.7 model via `aws_bedrock_claude_pipeline.py`. The model does not appear in the available model list due to a filtering condition and fails invocation due to a missing inference profile. - https://github.com/open-webui/pipelines/blob/main/examples/pipelines/providers/aws_bedrock_claude_pipeline.py --- **Root Cause Analysis (RCA)** ### 1. **Model Filtering Excludes Claude Sonnet 3.7** The pipeline script retrieves available models using the following line: https://github.com/propertyguru/ds-openwebui/blob/bbd5eba50ea32a3e7e4e2e526a5794679d401b2b/pg_openwebui_pipelines/pipelines/aws_bedrock_claude_pipeline.py#L150 ```python response = self.bedrock.list_foundation_models(byProvider='Anthropic', byInferenceType='ON_DEMAND') ``` However, Claude Sonnet 3.7 **is not an on-demand model**. As a result, it is **excluded** from the model list due to this filtering. --- ### 2. **Invocation Requires an Inference Profile** Even when the model is manually added or the filtering condition is removed, invocation fails with the following error: > **Invocation of model ID anthropic.claude-3-7-sonnet-20250219-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.** This confirms that **Claude Sonnet 3.7 must be invoked using an inference profile ARN**, rather than via the default on-demand method. --- **Suggested Fix** - **Update the model listing logic** to include models that are not on-demand (e.g., by removing or making `byInferenceType` optional). - **Support invocation using an inference profile ARN** for models that require it, like Claude Sonnet 3.7. Reference video explaining the inference profile flow: 📹 https://youtu.be/7-BIpBn9ao4?si=HZksyMVVR__eKfhv&t=343 --- **Impact** - Claude Sonnet 3.7 is currently **inaccessible** from OpenWebUI. - Users cannot leverage the latest capabilities from the Claude 3.x series unless the above is addressed. cc: @tjbck

yindo closed this issue

2026-02-15 19:17:03 -05:00

yindo referenced this issue

2026-02-15 20:15:13 -05:00

[PR #220] [MERGED] Made env vars more generic, added Ollama port to dev-docker.sh #350

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: open-webui/pipelines#220