langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 12:36:14 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

anthropic-sonnet3.5 max_token wrong #4767

New Issue
Closed
opened 2026-02-21 18:07:55 -05:00 by yindo · 4 comments
yindo commented 2026-02-21 18:07:55 -05:00
Owner
Copy Link
Copy Source

Originally created by @wxsunwzx on GitHub (Jul 24, 2024).

Originally assigned to: @crazywoola on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.15

Cloud or Self Hosted

Cloud, Self Hosted (Docker)

Steps to reproduce

the eewest version Dify and cloud version 0.6.15 have bug that while using anthropic-sonnet3.5 model will report "[anthropic]Bad Request Error,Error code:400-[type' invalid request error [ message : max tokens 8192> 4096,which is the maximum allowed number of output tokens for claude-3-5-sonnet-20240620)]",i found that the file claude-3-5-sonnet-20240620.yaml in 0.6.15 have wrong max token set and the previous version 0.6.14 was correct,please fix it

✔️ Expected Behavior

fix the problem that can use claude3.5 sonnet in cloud

❌ Actual Behavior

No response

Originally created by @wxsunwzx on GitHub (Jul 24, 2024). Originally assigned to: @crazywoola on GitHub. ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.6.15 ### Cloud or Self Hosted Cloud, Self Hosted (Docker) ### Steps to reproduce the eewest version Dify and cloud version 0.6.15 have bug that while using anthropic-sonnet3.5 model will report "[anthropic]Bad Request Error,Error code:400-[type' invalid request error [ message : max tokens 8192> 4096,which is the maximum allowed number of output tokens for claude-3-5-sonnet-20240620)]",i found that the file claude-3-5-sonnet-20240620.yaml in 0.6.15 have wrong max token set and the previous version 0.6.14 was correct,please fix it ### ✔️ Expected Behavior fix the problem that can use claude3.5 sonnet in cloud ### ❌ Actual Behavior _No response_
yindo added the 🐞 bug label 2026-02-21 18:07:55 -05:00
yindo closed this issue 2026-02-21 18:07:55 -05:00
yindo commented 2026-02-21 18:07:55 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jul 24, 2024):

@wxsunwzx Hello there! I'm here to assist you with bug solving, answering questions, and becoming a contributor. Let's work together to resolve any issues you're encountering.

To fix the incorrect max_tokens setting in the claude-3-5-sonnet-20240620.yaml file, you need to adjust the context_size under model_properties to a more appropriate value. The current setting of 200000 is likely incorrect. Based on the max_tokens parameter, a more reasonable context_size would be 4096.

Here is the corrected YAML configuration:

model: claude-3-5-sonnet-20240620
label:
  en_US: claude-3-5-sonnet-20240620
model_type: llm
features:
  - agent-thought
  - vision
  - tool-call
  - stream-tool-call
model_properties:
  mode: chat
  context_size: 4096
parameter_rules:
  - name: temperature
    use_template: temperature
  - name: top_p
    use_template: top_p
  - name: top_k
    label:
      zh_Hans: 取样数量
      en_US: Top k
    type: int
    help:
      zh_Hans: 仅从每个后续标记的前 K 个选项中采样。
      en_US: Only sample from the top K options for each subsequent token.
    required: false
  - name: max_tokens
    use_template: max_tokens
    required: true
    default: 4096
    min: 1
    max: 4096
  - name: response_format
    use_template: response_format
pricing:
  input: '3.00'
  output: '15.00'
  unit: '0.000001'
  currency: USD

This should resolve the issue and allow the claude3.5 sonnet model to be used correctly in the cloud [1][2].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Jul 24, 2024): <!-- Greeting --> @wxsunwzx Hello there! I'm here to assist you with bug solving, answering questions, and becoming a contributor. Let's work together to resolve any issues you're encountering. <!-- Answer --> To fix the incorrect `max_tokens` setting in the `claude-3-5-sonnet-20240620.yaml` file, you need to adjust the `context_size` under `model_properties` to a more appropriate value. The current setting of `200000` is likely incorrect. Based on the `max_tokens` parameter, a more reasonable `context_size` would be `4096`. Here is the corrected YAML configuration: ```yaml model: claude-3-5-sonnet-20240620 label: en_US: claude-3-5-sonnet-20240620 model_type: llm features: - agent-thought - vision - tool-call - stream-tool-call model_properties: mode: chat context_size: 4096 parameter_rules: - name: temperature use_template: temperature - name: top_p use_template: top_p - name: top_k label: zh_Hans: 取样数量 en_US: Top k type: int help: zh_Hans: 仅从每个后续标记的前 K 个选项中采样。 en_US: Only sample from the top K options for each subsequent token. required: false - name: max_tokens use_template: max_tokens required: true default: 4096 min: 1 max: 4096 - name: response_format use_template: response_format pricing: input: '3.00' output: '15.00' unit: '0.000001' currency: USD ``` This should resolve the issue and allow the `claude3.5 sonnet` model to be used correctly in the cloud <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/vertex_ai/llm/anthropic.claude-3.5-sonnet.yaml)</sup><sup>[[2]](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/anthropic/llm/claude-3-5-sonnet-20240620.yaml)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:07:56 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Jul 24, 2024):

Hello this is a beta feature. We introduced this one in this pr https://github.com/langgenius/dify/issues/6335 by @richards199999 .

I guess we still need some time to use 8k tokens in applications.

In the mean time, you could adjust the numbers down to 4096 in order to get things up and running.

@crazywoola commented on GitHub (Jul 24, 2024): Hello this is a beta feature. We introduced this one in this pr https://github.com/langgenius/dify/issues/6335 by @richards199999 . I guess we still need some time to use 8k tokens in applications. In the mean time, you could adjust the numbers down to 4096 in order to get things up and running.
yindo commented 2026-02-21 18:07:56 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Jul 31, 2024):

Fixed in https://github.com/langgenius/dify/pull/6843. Will release it tonight.

@crazywoola commented on GitHub (Jul 31, 2024): Fixed in https://github.com/langgenius/dify/pull/6843. Will release it tonight.
yindo commented 2026-02-21 18:07:56 -05:00
Author
Owner
Copy Link
Copy Source

@ivanberry commented on GitHub (Aug 2, 2024):

on 0.6.15 dify version of self host with docker. the problems is still.

and I check through lobeChat with same model and it worked, but not dify

@ivanberry commented on GitHub (Aug 2, 2024): on 0.6.15 dify version of self host with docker. the problems is still. and I check through lobeChat with same model and it worked, but not dify
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#4767
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?