langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[feature] Support changing language parameter in OpenAI Whisper (speech-to-text) #12314

New Issue
Closed
opened 2026-02-21 19:06:52 -05:00 by yindo · 6 comments
yindo commented 2026-02-21 19:06:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @sungreong on GitHub (Mar 28, 2025).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Hi Dify team,

First of all, thank you for the amazing work on this project!

I'm using Dify's speech-to-text feature with OpenAI's Whisper model. It seems like the language parameter is currently hardcoded or defaults to "english", and there is no way to override it through the UI or configuration.

✨ Feature Request
I’d like to request a feature to allow users to specify the language parameter when using OpenAI Whisper. This is essential for using speech-to-text with non-English languages like Korean ("ko"), Japanese ("ja"), etc.

🛠 Expected Change
Please consider:

Allowing language to be passed from the frontend or YAML config

Making "english" the default value if not specified (to keep current behavior)

🔍 Additional Help
If possible, could you also point me to the file or code snippet where the OpenAI Whisper API is called? I’d be happy to contribute or override it locally in the meantime.

Thank you again for this fantastic platform!

Best regards,

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @sungreong on GitHub (Mar 28, 2025). ### Self Checks - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. Hi Dify team, First of all, thank you for the amazing work on this project! I'm using Dify's speech-to-text feature with OpenAI's Whisper model. It seems like the language parameter is currently hardcoded or defaults to "english", and there is no way to override it through the UI or configuration. ✨ Feature Request I’d like to request a feature to allow users to specify the language parameter when using OpenAI Whisper. This is essential for using speech-to-text with non-English languages like Korean ("ko"), Japanese ("ja"), etc. 🛠 Expected Change Please consider: Allowing language to be passed from the frontend or YAML config Making "english" the default value if not specified (to keep current behavior) 🔍 Additional Help If possible, could you also point me to the file or code snippet where the OpenAI Whisper API is called? I’d be happy to contribute or override it locally in the meantime. Thank you again for this fantastic platform! Best regards, ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [ ] I am interested in contributing to this feature.
yindo added the 💪 enhancementstt & tts labels 2026-02-21 19:06:52 -05:00
yindo closed this issue 2026-02-21 19:06:52 -05:00
yindo commented 2026-02-21 19:06:52 -05:00
Author
Owner
Copy Link
Copy Source

@gstrat88 commented on GitHub (Mar 30, 2025):

As a starter, you can take a look from tts model
https://github.com/langgenius/dify-official-plugins/blob/main/models/openai/models/tts/tts-1.yaml

Actual call:
https://github.com/langgenius/dify-official-plugins/blob/main/models/openai/models/speech2text/speech2text.py

Languages should be added as properties:
https://github.com/langgenius/dify-official-plugins/blob/main/models/openai/models/speech2text/whisper-1.yaml

Frontend should change to handle the selection:
https://github.com/langgenius/dify/blob/46d235bca06d6b7b32f40072b728012eca3dc5dd/web/app/components/base/features/new-feature-panel/speech-to-text.tsx#L49
Again tts could be an example
https://github.com/langgenius/dify/blob/46d235bca06d6b7b32f40072b728012eca3dc5dd/web/app/components/base/features/new-feature-panel/text-to-speech/index.tsx

@gstrat88 commented on GitHub (Mar 30, 2025): As a starter, you can take a look from tts model https://github.com/langgenius/dify-official-plugins/blob/main/models/openai/models/tts/tts-1.yaml Actual call: https://github.com/langgenius/dify-official-plugins/blob/main/models/openai/models/speech2text/speech2text.py Languages should be added as properties: https://github.com/langgenius/dify-official-plugins/blob/main/models/openai/models/speech2text/whisper-1.yaml Frontend should change to handle the selection: https://github.com/langgenius/dify/blob/46d235bca06d6b7b32f40072b728012eca3dc5dd/web/app/components/base/features/new-feature-panel/speech-to-text.tsx#L49 Again tts could be an example https://github.com/langgenius/dify/blob/46d235bca06d6b7b32f40072b728012eca3dc5dd/web/app/components/base/features/new-feature-panel/text-to-speech/index.tsx
yindo commented 2026-02-21 19:06:52 -05:00
Author
Owner
Copy Link
Copy Source

@sungreong commented on GitHub (Mar 30, 2025):

You showed me several links, but what I want is not TTS.
What I'm looking for refers to the parameters that can be used when translating speech in STT.

@sungreong commented on GitHub (Mar 30, 2025): You showed me several links, but what I want is not TTS. What I'm looking for refers to the parameters that can be used when translating speech in STT.
yindo commented 2026-02-21 19:06:52 -05:00
Author
Owner
Copy Link
Copy Source

@gstrat88 commented on GitHub (Mar 30, 2025):

I am referencing tts as an example as they are handling other languages there!
Take a closer look at the links provided, the "actual call" is what you want to alter for a quick and dirty hack

@gstrat88 commented on GitHub (Mar 30, 2025): I am referencing tts as an example as they are handling other languages there! Take a closer look at the links provided, the "actual call" is what you want to alter for a quick and dirty hack
yindo commented 2026-02-21 19:06:52 -05:00
Author
Owner
Copy Link
Copy Source

@sungreong commented on GitHub (Mar 31, 2025):

Oh, thank you!
After checking, I realized that there are some parts that need to be changed.
However, what I'm curious about is how these parts are actually connected behind the scenes—I couldn't quite figure that out. Would you mind explaining it to me?

In the dify GitHub repository, there's this line in api/core/plugin/manager/model.py:

path = f"plugin/{tenant_id}/dispatch/speech2text/invoke"
This is how the API is being called.

Then, in the dify-plugin-daemon repository, there's:

group.POST("/speech2text/invoke", controllers.InvokeSpeech2Text(config))

And in dify-plugin-daemon/internal/core/dify_invocation/real/http_request:

func (i *RealBackwardsInvocation) InvokeSpeech2Text(payload *dify_invocation.InvokeSpeech2TextRequest) (*model_entities.Speech2TextResult, error) {
	return Request[model_entities.Speech2TextResult](i, "POST", "invoke/speech2text", http_requests.HttpPayloadJson(payload))
}

I was able to trace the flow up to this point, but couldn't track what happens next.

Then I found this in the dify-official-plugins repository under
dify-official-plugins/models/openai_api_compatible/models/speech2text/speech2text.py.

Although it's slightly different from what you mentioned, I was able to locate the part where the API is actually used.

What I really want to understand is:
Since I need to run this as a service using Docker, could you help me understand how these pieces are organically connected?
Also, if I modify the internal source code in the dify-official-plugins repository and then build it with Docker, will it work as expected?

Sorry for asking in such detail—I'm still learning and really appreciate your help.

@sungreong commented on GitHub (Mar 31, 2025): Oh, thank you! After checking, I realized that there are some parts that need to be changed. However, what I'm curious about is how these parts are actually connected behind the scenes—I couldn't quite figure that out. Would you mind explaining it to me? In the dify GitHub repository, there's this line in api/core/plugin/manager/model.py: path = f"plugin/{tenant_id}/dispatch/speech2text/invoke" This is how the API is being called. Then, in the dify-plugin-daemon repository, there's: ``` group.POST("/speech2text/invoke", controllers.InvokeSpeech2Text(config)) ``` And in dify-plugin-daemon/internal/core/dify_invocation/real/http_request: ``` func (i *RealBackwardsInvocation) InvokeSpeech2Text(payload *dify_invocation.InvokeSpeech2TextRequest) (*model_entities.Speech2TextResult, error) { return Request[model_entities.Speech2TextResult](i, "POST", "invoke/speech2text", http_requests.HttpPayloadJson(payload)) } ``` I was able to trace the flow up to this point, but couldn't track what happens next. Then I found this in the dify-official-plugins repository under dify-official-plugins/models/openai_api_compatible/models/speech2text/speech2text.py. Although it's slightly different from what you mentioned, I was able to locate the part where the API is actually used. What I really want to understand is: Since I need to run this as a service using Docker, could you help me understand how these pieces are organically connected? Also, if I modify the internal source code in the dify-official-plugins repository and then build it with Docker, will it work as expected? Sorry for asking in such detail—I'm still learning and really appreciate your help.
yindo commented 2026-02-21 19:06:52 -05:00
Author
Owner
Copy Link
Copy Source

@gstrat88 commented on GitHub (Mar 31, 2025):

After the repository splits I am not sure how it works, but i believe that by digging the volumes you should probably find the installed openai plugin and make the alternations there.
Another approach is to package it an install as a local plugin
https://docs.dify.ai/plugins/publish-plugins/package-plugin-file-and-publish

@gstrat88 commented on GitHub (Mar 31, 2025): After the repository splits I am not sure how it works, but i believe that by digging the volumes you should probably find the installed openai plugin and make the alternations there. Another approach is to package it an install as a local plugin https://docs.dify.ai/plugins/publish-plugins/package-plugin-file-and-publish
yindo commented 2026-02-21 19:06:52 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (May 1, 2025):

Hi, @sungreong. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

  • You requested a feature enhancement for Dify's speech-to-text functionality to support non-English languages using OpenAI's Whisper model.
  • You expressed willingness to contribute to this development.
  • Guidance was provided by gstrat88 on modifying the codebase, using TTS as an example for handling multiple languages.
  • You sought further clarification on component connections and running the service with Docker.

Next Steps:

  • Please let me know if this issue is still relevant to the latest version of the Dify repository. If so, you can keep the discussion open by commenting here.
  • If there is no further activity, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (May 1, 2025): Hi, @sungreong. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary:** - You requested a feature enhancement for Dify's speech-to-text functionality to support non-English languages using OpenAI's Whisper model. - You expressed willingness to contribute to this development. - Guidance was provided by gstrat88 on modifying the codebase, using TTS as an example for handling multiple languages. - You sought further clarification on component connections and running the service with Docker. **Next Steps:** - Please let me know if this issue is still relevant to the latest version of the Dify repository. If so, you can keep the discussion open by commenting here. - If there is no further activity, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label stt & tts 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#12314
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?