langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

xinference plugin: I can't use a model like Kokoro、F5-TTS #181

New Issue
Closed
opened 2026-02-16 10:18:22 -05:00 by yindo · 2 comments
yindo commented 2026-02-16 10:18:22 -05:00
Owner
Copy Link
Copy Source

Originally created by @937089773 on GitHub (Apr 9, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues Dify issues & Dify Official Plugins, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

all

Plugin version

0.0.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I can't use a model like Kokoro、F5-TTS
error: please check model type, the model you want to invoke is not a text-to-audio model
but The Kokoro model is actually text-to-audio

This is caused by a problem with the code：

xinference_helper.py

 @staticmethod
  def _get_xinference_extra_parameter(
      server_url: str, model_uid: str, api_key: str
  ) -> XinferenceModelExtraParameter:

    if (
        not model_uid
        or not model_uid.strip()
        or not server_url
        or not server_url.strip()
    ):
        raise RuntimeError("model_uid is empty")

    url = str(URL(server_url) / "v1" / "models" / model_uid)

    # this method is surrounded by a lock, and default requests may hang forever,
    # so we just set a Adapter with max_retries=3
    session = Session()
    session.mount("http://", HTTPAdapter(max_retries=3))
    session.mount("https://", HTTPAdapter(max_retries=3))
    headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}

    try:
        response = session.get(url, headers=headers, timeout=10)
    except (MissingSchema, ConnectionError, Timeout) as e:
        raise RuntimeError(
            f"get xinference model extra parameter failed, url: {url}, error: {e}"
        )
    if response.status_code != 200:
        raise RuntimeError(
            f"get xinference model extra parameter failed, status code: {response.status_code},"
            f" response: {response.text}"
        )

    response_json = response.json()

    model_format = response_json.get("model_format", "ggmlv3")
    model_ability = response_json.get("model_ability", [])
    model_family = response_json.get("model_family", None)

    if response_json.get("model_type") == "embedding":
        model_handle_type = "embedding"
    elif response_json.get("model_type") == "audio":
        model_handle_type = "audio"
        if model_family and model_family in {"ChatTTS", "CosyVoice", "FishAudio"}:
            model_ability.append("text-to-audio")
        else:
            model_ability.append("audio-to-text")
    elif model_format == "ggmlv3" and "chatglm" in response_json["model_name"]:
        model_handle_type = "chatglm"
    elif "generate" in model_ability:
        model_handle_type = "generate"
    elif "chat" in model_ability:
        model_handle_type = "chat"
    else:
        raise NotImplementedError("xinference model handle type is not supported")

    support_function_call = "tools" in model_ability
    support_vision = "vision" in model_ability
    max_tokens = response_json.get("max_tokens", 512)

    context_length = response_json.get("context_length", 2048)

    return XinferenceModelExtraParameter(
        model_format=model_format,
        model_handle_type=model_handle_type,
        model_ability=model_ability,
        support_function_call=support_function_call,
        support_vision=support_vision,
        max_tokens=max_tokens,
        context_length=context_length,
        model_family=model_family,
    )

Here, line 123：“if model_family and model_family in {"ChatTTS", "CosyVoice", "FishAudio"}:”

✔️ Error log

No response

Originally created by @937089773 on GitHub (Apr 9, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [Dify issues](https://github.com/langgenius/dify/issues) & [Dify Official Plugins](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version all ### Plugin version 0.0.2 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce I can't use a model like Kokoro、F5-TTS error: please check model type, the model you want to invoke is not a text-to-audio model but The Kokoro model is actually text-to-audio This is caused by a problem with the code： xinference_helper.py @staticmethod def _get_xinference_extra_parameter( server_url: str, model_uid: str, api_key: str ) -> XinferenceModelExtraParameter: if ( not model_uid or not model_uid.strip() or not server_url or not server_url.strip() ): raise RuntimeError("model_uid is empty") url = str(URL(server_url) / "v1" / "models" / model_uid) # this method is surrounded by a lock, and default requests may hang forever, # so we just set a Adapter with max_retries=3 session = Session() session.mount("http://", HTTPAdapter(max_retries=3)) session.mount("https://", HTTPAdapter(max_retries=3)) headers = {"Authorization": f"Bearer {api_key}"} if api_key else {} try: response = session.get(url, headers=headers, timeout=10) except (MissingSchema, ConnectionError, Timeout) as e: raise RuntimeError( f"get xinference model extra parameter failed, url: {url}, error: {e}" ) if response.status_code != 200: raise RuntimeError( f"get xinference model extra parameter failed, status code: {response.status_code}," f" response: {response.text}" ) response_json = response.json() model_format = response_json.get("model_format", "ggmlv3") model_ability = response_json.get("model_ability", []) model_family = response_json.get("model_family", None) if response_json.get("model_type") == "embedding": model_handle_type = "embedding" elif response_json.get("model_type") == "audio": model_handle_type = "audio" if model_family and model_family in {"ChatTTS", "CosyVoice", "FishAudio"}: model_ability.append("text-to-audio") else: model_ability.append("audio-to-text") elif model_format == "ggmlv3" and "chatglm" in response_json["model_name"]: model_handle_type = "chatglm" elif "generate" in model_ability: model_handle_type = "generate" elif "chat" in model_ability: model_handle_type = "chat" else: raise NotImplementedError("xinference model handle type is not supported") support_function_call = "tools" in model_ability support_vision = "vision" in model_ability max_tokens = response_json.get("max_tokens", 512) context_length = response_json.get("context_length", 2048) return XinferenceModelExtraParameter( model_format=model_format, model_handle_type=model_handle_type, model_ability=model_ability, support_function_call=support_function_call, support_vision=support_vision, max_tokens=max_tokens, context_length=context_length, model_family=model_family, ) Here, line 123：“if model_family and model_family in {"ChatTTS", "CosyVoice", "FishAudio"}:” ### ✔️ Error log _No response_
yindo added the bug label 2026-02-16 10:18:22 -05:00
yindo closed this issue 2026-02-16 10:18:22 -05:00
yindo commented 2026-02-16 10:18:22 -05:00
Author
Owner
Copy Link
Copy Source

@lnxyz commented on GitHub (Jun 6, 2025):

me too in dify 1.4.1 and xinference plugin 0.03

@lnxyz commented on GitHub (Jun 6, 2025): me too in dify 1.4.1 and xinference plugin 0.03
yindo commented 2026-02-16 10:18:22 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 18, 2025):

Hi, @937089773. I'm Dosu, and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale.

Issue Summary:

  • The xinference plugin does not recognize text-to-audio models like Kokoro and F5-TTS due to a problem in xinference_helper.py.
  • This issue causes incorrect model type checks and API request failures.
  • Another user has confirmed experiencing the same problem with dify 1.4.1 and xinference plugin 0.03.
  • The problem affects multiple users and relates to incorrect model classification in the plugin.

Next Steps:

  • Please let me know if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting here.
  • If I do not hear back within 5 days, I will automatically close this issue.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Aug 18, 2025): Hi, @937089773. I'm [Dosu](https://dosu.dev), and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale. **Issue Summary:** - The xinference plugin does not recognize text-to-audio models like Kokoro and F5-TTS due to a problem in xinference_helper.py. - This issue causes incorrect model type checks and API request failures. - Another user has confirmed experiencing the same problem with dify 1.4.1 and xinference plugin 0.03. - The problem affects multiple users and relates to incorrect model classification in the plugin. **Next Steps:** - Please let me know if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting here. - If I do not hear back within 5 days, I will automatically close this issue. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#181
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?