langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-22 01:55:27 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

Using Agent with LiteLLM Bedrock Deepseek: The stop sequence you provided at index 0 is longer than the model limit of 10 characters #299

New Issue
Closed
opened 2026-02-16 10:18:52 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 10:18:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @micology-code on GitHub (May 20, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.4.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Create a Agent, select AWS Bedrock serverless deepseek model provided via LiteLLM, using OpenAI compatible API in dify.

Say Hi,
LiteLLM receive request:

{
  "stop": [
    "Observation"
  ],
  "user": "96d78806-a6e3-48e7-bf5a-ecb0cd101606",
  "model": "Deepseek R1",
  "stream": true,
  "messages": [
    {
      "role": "system",
      "content": "Respond to the human as helpfully and accurately as possible.\n\n\n\nYou have access to the following tools:\n\n[]\n\nUse a json blob to specify a tool by providing an action key (tool name) and an action_input key (tool input).\nValid \"action\" values: \"Final Answer\" or \n\nProvide only ONE action per $JSON_BLOB, as shown:\n\n```\n{\n  \"action\": $TOOL_NAME,\n  \"action_input\": $ACTION_INPUT\n}\n```\n\nFollow this format:\n\nQuestion: input question to answer\nThought: consider previous and subsequent steps\nAction:\n```\n$JSON_BLOB\n```\nObservation: action result\n... (repeat Thought/Action/Observation N times)\nThought: I know what to respond\nAction:\n```\n{\n  \"action\": \"Final Answer\",\n  \"action_input\": \"Final response to human\"\n}\n```\n\nBegin! Reminder to ALWAYS respond with a valid json blob of a single action. Use tools if necessary. Respond directly if appropriate. Format is Action:```$JSON_BLOB```then Observation:.\n"
    },
    {
      "role": "user",
      "content": "hi'"
    }
  ]
}

The deep seek model works properly in normal Chat/Chat flow LLM, but agent seems to use the stop sequence of Observation which breaks it

✔️ Expected Behavior

Works

❌ Actual Behavior

litellm.BadRequestError: BedrockException - {"message":"The stop sequence you provided at index 0 is longer than the model limit of 10 characters. Try again with a stop sequence that is shorter than 10 characters."}. Received Model Group=Deepseek R1
Originally created by @micology-code on GitHub (May 20, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.4.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Create a Agent, select AWS Bedrock serverless deepseek model provided via LiteLLM, using OpenAI compatible API in dify. Say Hi, LiteLLM receive request: ```json { "stop": [ "Observation" ], "user": "96d78806-a6e3-48e7-bf5a-ecb0cd101606", "model": "Deepseek R1", "stream": true, "messages": [ { "role": "system", "content": "Respond to the human as helpfully and accurately as possible.\n\n\n\nYou have access to the following tools:\n\n[]\n\nUse a json blob to specify a tool by providing an action key (tool name) and an action_input key (tool input).\nValid \"action\" values: \"Final Answer\" or \n\nProvide only ONE action per $JSON_BLOB, as shown:\n\n```\n{\n \"action\": $TOOL_NAME,\n \"action_input\": $ACTION_INPUT\n}\n```\n\nFollow this format:\n\nQuestion: input question to answer\nThought: consider previous and subsequent steps\nAction:\n```\n$JSON_BLOB\n```\nObservation: action result\n... (repeat Thought/Action/Observation N times)\nThought: I know what to respond\nAction:\n```\n{\n \"action\": \"Final Answer\",\n \"action_input\": \"Final response to human\"\n}\n```\n\nBegin! Reminder to ALWAYS respond with a valid json blob of a single action. Use tools if necessary. Respond directly if appropriate. Format is Action:```$JSON_BLOB```then Observation:.\n" }, { "role": "user", "content": "hi'" } ] } ``` The deep seek model works properly in normal Chat/Chat flow LLM, but agent seems to use the stop sequence of `Observation` which breaks it ### ✔️ Expected Behavior Works ### ❌ Actual Behavior ``` litellm.BadRequestError: BedrockException - {"message":"The stop sequence you provided at index 0 is longer than the model limit of 10 characters. Try again with a stop sequence that is shorter than 10 characters."}. Received Model Group=Deepseek R1 ```
yindo closed this issue 2026-02-16 10:18:52 -05:00
yindo commented 2026-02-16 10:18:53 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (May 20, 2025):

Hi @micology-code! I'm Dosu, a bot here to help you with bugs, answer your questions, and assist you in becoming a contributor. I'm here while you wait for a human maintainer. Let's tackle your issue together!

The error you're encountering is due to the stop sequence "Observation" being longer than the model's character limit of 10 characters. To resolve this issue, you need to use a stop sequence that is 10 characters or fewer.

In the Dify codebase, the stop sequence is configured in the completion_params dictionary, which is part of the agent's setup. You can modify this configuration to use a shorter stop sequence. Here's a general approach to address this:

  1. Locate the configuration for the stop sequence in your setup. This is typically found in the api/core/workflow/nodes/llm/node.py file, where the stop sequences are extracted from the completion_params dictionary [1].

  2. Change the stop sequence from "Observation" to a shorter string that fits within the 10-character limit. For example, you could use "Obs" or any other suitable abbreviation.

  3. Test the agent again with the updated stop sequence to ensure it works without triggering the character limit error.

By making these adjustments, you should be able to use the LiteLLM Bedrock Deepseek model without encountering the stop sequence length issue.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (May 20, 2025): <!-- Greeting --> Hi @micology-code! I'm [Dosu](https://dosu.dev), a bot here to help you with bugs, answer your questions, and assist you in becoming a contributor. I'm here while you wait for a human maintainer. Let's tackle your issue together! <!-- Answer --> The error you're encountering is due to the stop sequence "Observation" being longer than the model's character limit of 10 characters. To resolve this issue, you need to use a stop sequence that is 10 characters or fewer. In the Dify codebase, the stop sequence is configured in the `completion_params` dictionary, which is part of the agent's setup. You can modify this configuration to use a shorter stop sequence. Here's a general approach to address this: 1. Locate the configuration for the stop sequence in your setup. This is typically found in the `api/core/workflow/nodes/llm/node.py` file, where the stop sequences are extracted from the `completion_params` dictionary <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/workflow/nodes/llm/node.py)</sup>. 2. Change the stop sequence from "Observation" to a shorter string that fits within the 10-character limit. For example, you could use "Obs" or any other suitable abbreviation. 3. Test the agent again with the updated stop sequence to ensure it works without triggering the character limit error. By making these adjustments, you should be able to use the LiteLLM Bedrock Deepseek model without encountering the stop sequence length issue. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/c7ebd9eb-0042-43fe-ad90-24845c955b05?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/c7ebd9eb-0042-43fe-ad90-24845c955b05?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/c7ebd9eb-0042-43fe-ad90-24845c955b05?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/c7ebd9eb-0042-43fe-ad90-24845c955b05?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c7ebd9eb-0042-43fe-ad90-24845c955b05?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/c7ebd9eb-0042-43fe-ad90-24845c955b05?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c7ebd9eb-0042-43fe-ad90-24845c955b05?feedback_type=other)</sup>
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#299
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?