Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3553] [BUG/FEAT]: AWS Bedrock reasoning models with @agent #2290

New Issue
Open
opened 2026-02-22 18:29:01 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:29:01 -05:00
Owner
Copy Link
Copy Source

Originally created by @timothycarambat on GitHub (Mar 27, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3553

Originally assigned to: @timothycarambat on GitHub.

How are you running AnythingLLM?

All versions

What happened?

With the refactor of AWS Bedrock to move away from Langchain in https://github.com/Mintplex-Labs/anything-llm/pull/3537

This needs to be expanded to the agent execution provider, as the use of Langchain to use reasoning models for agent execution is not possible with the current implementation.

Current workaround
Do not use reasoning models for AWS Bedrock agent execution as the content response is an array as opposed to a string - like all other models.

Are there known steps to reproduce?

Use a reasoning model for AWS bedrock and send a single agent chat. This error will manifest as a jsonString?.startsWith error - which is a red herring as the real error is the output formats from the response being mishandled by Langchain.

Originally created by @timothycarambat on GitHub (Mar 27, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3553 Originally assigned to: @timothycarambat on GitHub. ### How are you running AnythingLLM? All versions ### What happened? With the refactor of AWS Bedrock to move away from Langchain in https://github.com/Mintplex-Labs/anything-llm/pull/3537 This needs to be expanded to the `agent` execution provider, as the use of Langchain to use reasoning models for agent execution is not possible with the current implementation. **Current workaround** Do not use reasoning models for AWS Bedrock agent execution as the `content` response is an array as opposed to a string - like all other models. ### Are there known steps to reproduce? Use a reasoning model for AWS bedrock and send a single agent chat. This error will manifest as a `jsonString?.startsWith` error - which is a red herring as the real error is the output formats from the response being mishandled by Langchain.
yindo added the enhancementbugIntegration Request labels 2026-02-22 18:29:01 -05:00
yindo commented 2026-02-22 18:29:01 -05:00
Author
Owner
Copy Link
Copy Source

@tristan-stahnke-GPS commented on GitHub (Apr 24, 2025):

@timothycarambat I think my PR #3714 resolved this, can you check?

@tristan-stahnke-GPS commented on GitHub (Apr 24, 2025): @timothycarambat I think my PR #3714 resolved this, can you check?
yindo commented 2026-02-22 18:29:02 -05:00
Author
Owner
Copy Link
Copy Source

@tristan-stahnke-GPS commented on GitHub (Apr 25, 2025):

I was able to create agents using bedrock with my PR applied for the bedrock provider, parsed the content of a google search and sent it back to the LLM; not sure if there's another part of the @agent functionality that's needed, but would be cool to test it out, definitely want to get agents working fully with bedrock 💯

@tristan-stahnke-GPS commented on GitHub (Apr 25, 2025): I was able to create agents using bedrock with my PR applied for the bedrock provider, parsed the content of a google search and sent it back to the LLM; not sure if there's another part of the @agent functionality that's needed, but would be cool to test it out, definitely want to get agents working fully with bedrock 💯
yindo commented 2026-02-22 18:29:02 -05:00
Author
Owner
Copy Link
Copy Source

@0xbadshah commented on GitHub (Jun 25, 2025):

@tristan-stahnke Amazon Bedrock has multiple models and not every model works correctly despite right IAM permissions to access. I tried agent with Deepseek and it errors out. Which Bedrock model did you this against?

@0xbadshah commented on GitHub (Jun 25, 2025): @tristan-stahnke Amazon Bedrock has multiple models and not every model works correctly despite right IAM permissions to access. I tried agent with Deepseek and it errors out. Which Bedrock model did you this against?
yindo commented 2026-02-22 18:29:02 -05:00
Author
Owner
Copy Link
Copy Source

@tristan-stahnke-GPS commented on GitHub (Jun 25, 2025):

@Chan9390 I primarily use Claude Sonnet / Opus models (as well as the amazon Nova etc. Models). I haven't had a chance to look at Deepseek; that would definitely be something to chase down! And making the provider more model agnostic would be ideal as well, so we can leave room for additional functionality down the road (maybe allowing capabilities to thought process tokens?) I'll take a look!

@tristan-stahnke-GPS commented on GitHub (Jun 25, 2025): @Chan9390 I primarily use Claude Sonnet / Opus models (as well as the amazon Nova etc. Models). I haven't had a chance to look at Deepseek; that would definitely be something to chase down! And making the provider more model agnostic would be ideal as well, so we can leave room for additional functionality down the road (maybe allowing capabilities to thought process tokens?) I'll take a look!
yindo commented 2026-02-22 18:29:02 -05:00
Author
Owner
Copy Link
Copy Source

@bhasmang-tri commented on GitHub (Oct 7, 2025):

I also get Invalid message content: empty string. 'ai' must contain non-empty content. when i invoke the @agent with Bedrock (Claude model) using IAM role (i hosted it on AWS via docker)

@bhasmang-tri commented on GitHub (Oct 7, 2025): I also get Invalid message content: empty string. 'ai' must contain non-empty content. when i invoke the @agent with Bedrock (Claude model) using IAM role (i hosted it on AWS via docker)
yindo commented 2026-02-22 18:29:02 -05:00
Author
Owner
Copy Link
Copy Source

@SquadUpSquid commented on GitHub (Nov 3, 2025):

Hello, Is there any update to this issue. I'm having similar issues when trying to call @agent (MCP Server) using Bedrock models.
Hosted on an AWS EC2 using Docker.

Input: @agent blah blah blah

Output: (it's not always the same)

  1. nothing, just a blank response
  2. AWSBedrock::streamGetChatCompletion failed during setup. Bedrock is unable to process your request.
  3. "ai" must contain non-empty content
  4. Bedrock is unable to process your request.

I have tried this with Bedrock Claude 3.5, Claude 3.7, Nova Pro, Llama 3 70B. I'm using IAM Roles and gave it "bedrock:*" just to rule out any permissions problems with bedrock.

I have used the free Grok models, as a "control", and have gotten a proper response from it.

@SquadUpSquid commented on GitHub (Nov 3, 2025): Hello, Is there any update to this issue. I'm having similar issues when trying to call @agent (MCP Server) using Bedrock models. Hosted on an AWS EC2 using Docker. Input: @agent blah blah blah Output: (it's not always the same) 1. *nothing*, just a blank response 2. AWSBedrock::streamGetChatCompletion failed during setup. Bedrock is unable to process your request. 3. "ai" must contain non-empty content 4. Bedrock is unable to process your request. I have tried this with Bedrock Claude 3.5, Claude 3.7, Nova Pro, Llama 3 70B. I'm using IAM Roles and gave it "bedrock:*" just to rule out any permissions problems with bedrock. I have used the free Grok models, as a "control", and have gotten a proper response from it.
yindo changed title from [BUG/FEAT]: AWS Bedrock reasoning models with `@agent` to [GH-ISSUE #3553] [BUG/FEAT]: AWS Bedrock reasoning models with `@agent` 2026-06-05 14:45:40 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Integration Request bug enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2290
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?