langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

issue when add model from hugging face. #910

New Issue
Closed
opened 2026-02-21 17:28:59 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 17:28:59 -05:00
Owner
Copy Link
Copy Source

Originally created by @heatingp on GitHub (Jan 16, 2024).

Originally assigned to: @GarfieldDai on GitHub.

Self Checks

Dify version

0.4.6

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

1, Successful deployed model on hugging face, with Inference Endpoints, like the pic blow:
image
2. And I connected to Dify successfully by following the steps from:
https://docs.dify.ai/v/zh-hans/guides/model-configuration/hugging-face
3. But meet error when I tried to talk to the model, it said:
Error: TextGenerationStreamResponse.init() missing 1 required positional argument: 'token'
image
Looking forward to fix it, thanks!

✔️ Expected Behavior

Can generate text smoothy

❌ Actual Behavior

Error: TextGenerationStreamResponse.init() missing 1 required positional argument: 'token'

Originally created by @heatingp on GitHub (Jan 16, 2024). Originally assigned to: @GarfieldDai on GitHub. ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to file this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). ### Dify version 0.4.6 ### Cloud or Self Hosted Self Hosted (Source) ### Steps to reproduce 1, Successful deployed model on hugging face, with Inference Endpoints, like the pic blow: ![image](https://github.com/langgenius/dify/assets/128213969/7bfcefb3-6e48-45b0-b432-79cc6c594c97) 2. And I connected to Dify successfully by following the steps from: https://docs.dify.ai/v/zh-hans/guides/model-configuration/hugging-face 3. But meet error when I tried to talk to the model, it said: Error: TextGenerationStreamResponse.init() missing 1 required positional argument: 'token' <img width="703" alt="image" src="https://github.com/langgenius/dify/assets/128213969/7207044e-8093-4a0a-97b8-b49763cda50d"> Looking forward to fix it, thanks! ### ✔️ Expected Behavior Can generate text smoothy ### ❌ Actual Behavior Error: TextGenerationStreamResponse.init() missing 1 required positional argument: 'token'
yindo added the 🐞 bug label 2026-02-21 17:28:59 -05:00
yindo closed this issue 2026-02-21 17:28:59 -05:00
yindo commented 2026-02-21 17:28:59 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 16, 2024):

🤖

Hello @heatingp! 👋

I'm Dosu, a friendly bot here to help you with bugs, answer your queries, and guide you on becoming a better contributor while you wait for a human maintainer to assist you. 🤖🔧

Feel free to ask me anything about dify. I'm available 24/7 to help you out! 😄🌙

The error you're encountering, "TextGenerationStreamResponse.init() missing 1 required positional argument: 'token'", is likely due to the response from the HuggingFace InferenceClient's text_generation method not conforming to the expected format. The response is expected to be an iterable of TextGenerationStreamResponse objects, each of which should have a 'token' attribute.

In the current implementation of Dify, there is no explicit handling for the case where the 'token' attribute might be missing from the response. This could occur if the model's response does not conform to the expected format, or if there is an issue with the HuggingFace InferenceClient or the model itself.

To address this issue, you could add error handling to check if the 'token' attribute is present in the response. Here's an example of how you could modify the _stream_response method to handle this:

def _stream_response(
        self,
        response: Iterable,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
) -> Iterator[str]:
    for r in response:
        # Check if 'token' attribute is present
        if not hasattr(r, 'token'):
            raise ValueError("Response object does not have 'token' attribute")

        # skip special tokens
        if r.token.special:
            continue

        token = r.token.text
        if run_manager:
            run_manager.on_llm_new_token(
                token=token, verbose=self.verbose, log_probs=None
            )

        # yield the generated token
        yield token

This will raise a more informative error when the 'token' attribute is missing, which should help you debug the issue. You can find the relevant code in the HuggingFaceEndpointLLM class in the Dify repository.

Please note that the expected format of the response from the HuggingFace InferenceClient's text_generation method in the Dify codebase is a string, which is the text generated by the model. If the 'streaming' attribute of the HuggingFaceEndpointLLM instance is True, the response is expected to be an iterable of tokens. These tokens are combined into a single string, which is then returned.

I hope this helps! If you have any further questions or need additional clarification, please don't hesitate to ask.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@dosubot[bot] commented on GitHub (Jan 16, 2024): _🤖_ <!-- Greeting --> Hello @heatingp! 👋 I'm [Dosu](https://dosu.dev), a friendly bot here to help you with bugs, answer your queries, and guide you on becoming a better contributor while you wait for a human maintainer to assist you. 🤖🔧 Feel free to ask me anything about dify. I'm available 24/7 to help you out! 😄🌙 <!-- Answer --> The error you're encountering, "TextGenerationStreamResponse.init() missing 1 required positional argument: 'token'", is likely due to the response from the HuggingFace InferenceClient's text_generation method not conforming to the expected format. The response is expected to be an iterable of TextGenerationStreamResponse objects, each of which should have a 'token' attribute. In the current implementation of Dify, there is no explicit handling for the case where the 'token' attribute might be missing from the response. This could occur if the model's response does not conform to the expected format, or if there is an issue with the HuggingFace InferenceClient or the model itself. To address this issue, you could add error handling to check if the 'token' attribute is present in the response. Here's an example of how you could modify the _stream_response method to handle this: ```py def _stream_response( self, response: Iterable, run_manager: Optional[CallbackManagerForLLMRun] = None, ) -> Iterator[str]: for r in response: # Check if 'token' attribute is present if not hasattr(r, 'token'): raise ValueError("Response object does not have 'token' attribute") # skip special tokens if r.token.special: continue token = r.token.text if run_manager: run_manager.on_llm_new_token( token=token, verbose=self.verbose, log_probs=None ) # yield the generated token yield token ``` This will raise a more informative error when the 'token' attribute is missing, which should help you debug the issue. You can find the relevant code in the [HuggingFaceEndpointLLM class](https://github.com/langgenius/dify/blob/main/api/core/third_party/langchain/llms/huggingface_endpoint_llm.py) in the Dify repository. Please note that the expected format of the response from the HuggingFace InferenceClient's text_generation method in the Dify codebase is a string, which is the text generated by the model. If the 'streaming' attribute of the HuggingFaceEndpointLLM instance is `True`, the response is expected to be an iterable of tokens. These tokens are combined into a single string, which is then returned. I hope this helps! If you have any further questions or need additional clarification, please don't hesitate to ask. ## Sources - [api/core/third_party/langchain/llms/huggingface_endpoint_llm.py](https://github.com/langgenius/dify/blob/main/api/core/third_party/langchain/llms/huggingface_endpoint_llm.py) --- This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.
yindo commented 2026-02-21 17:28:59 -05:00
Author
Owner
Copy Link
Copy Source

@GarfieldDai commented on GitHub (Jan 17, 2024):

image It seems that the return of your model API don't have the parameter of 'token'. This problem is not at Dify.
@GarfieldDai commented on GitHub (Jan 17, 2024): <img width="800" alt="image" src="https://github.com/langgenius/dify/assets/28395549/0d4836f3-6d4a-4f9d-8e67-9b94873082d0"> It seems that the return of your model API don't have the parameter of 'token'. This problem is not at Dify. - Please check the Huggingface Docs of TGI project. To learn more about the TGI project, please refer to https://github.com/huggingface/text-generation-inference. - You also can contact the author of this model to fix this parameter. @heatingp
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#910
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?