langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

When requesting /v1/text-to-audio with the streaming parameter set to true, get the TypeError: 'function' object is not iterable #4107

New Issue
Closed
opened 2026-02-21 18:04:46 -05:00 by yindo · 10 comments
yindo commented 2026-02-21 18:04:46 -05:00
Owner
Copy Link
Copy Source

Originally created by @yanyu510 on GitHub (Jun 15, 2024).

Originally assigned to: @yanyu510 on GitHub.

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.0-fix1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

  1. Configure Tongyi Qianwen TTS model
    image

  2. Request /v1/text-to-audio url, set the parameter 'streaming' to true
    curl --location 'http://localhost/v1/text-to-audio'
    --header 'accept: /'
    --header 'Authorization: Bearer app-xxx'
    --data '{"text":"hello dify","streaming":true}'
    1718440357716

✔️ Expected Behavior

Returns 200 with audio binary data

❌ Actual Behavior

Return code 500
image

Originally created by @yanyu510 on GitHub (Jun 15, 2024). Originally assigned to: @yanyu510 on GitHub. ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.6.0-fix1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce 1. Configure Tongyi Qianwen TTS model ![image](https://github.com/langgenius/dify/assets/13358679/d76d1904-eaf1-4876-985d-202c236ad416) 2. Request /v1/text-to-audio url, set the parameter 'streaming' to true curl --location 'http://localhost/v1/text-to-audio' \ --header 'accept: */*' \ --header 'Authorization: Bearer app-xxx' \ --data '{"text":"hello dify","streaming":true}' ![1718440357716](https://github.com/langgenius/dify/assets/13358679/90002ae1-046f-4a2a-aeea-57f6766e4bed) ### ✔️ Expected Behavior Returns 200 with audio binary data ### ❌ Actual Behavior Return code 500 ![image](https://github.com/langgenius/dify/assets/13358679/39685c52-7769-41ea-8d30-ba3a361a4629)
yindo added the 🐞 bug label 2026-02-21 18:04:46 -05:00
yindo closed this issue 2026-02-21 18:04:46 -05:00
yindo commented 2026-02-21 18:04:46 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jun 15, 2024):

Hello, @yanyu510! I'm here to help you with any bugs, questions, or contributions related to the repository. Let's work together to resolve your issue.

To resolve the TypeError: 'function' object is not iterable error when requesting the /v1/text-to-audio endpoint with the streaming parameter set to true, you need to ensure that the AudioService.transcript_tts method is correctly implemented to handle the streaming parameter. The error suggests that somewhere in the code, a function is being treated as an iterable, which is not allowed.

Here is a possible fix:

  1. Check the implementation of AudioService.transcript_tts to ensure it correctly handles the streaming parameter.
  2. Ensure that the transcript_tts method does not return a function or any non-iterable object when streaming is set to true.

Here is an example of how you might modify the transcript_tts method to handle streaming correctly:

class AudioService:
    @staticmethod
    def transcript_tts(app_model, text, end_user, voice=None, streaming=False):
        if streaming:
            # Ensure that the response is an iterable, such as a generator
            def generate_audio_stream():
                # Your logic to generate audio stream
                yield b'audio_chunk_1'
                yield b'audio_chunk_2'
                # Continue yielding audio chunks

            return generate_audio_stream()
        else:
            # Your logic to generate a complete audio file
            audio_data = b'complete_audio_file'
            return audio_data

In this example, when streaming is true, the method returns a generator that yields audio chunks, which is an iterable. When streaming is false, it returns a complete audio file, which is also an iterable (a bytes object) [1].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Jun 15, 2024): <!-- Greeting --> Hello, @yanyu510! I'm here to help you with any bugs, questions, or contributions related to the repository. Let's work together to resolve your issue. <!-- Answer --> To resolve the `TypeError: 'function' object is not iterable` error when requesting the `/v1/text-to-audio` endpoint with the `streaming` parameter set to `true`, you need to ensure that the `AudioService.transcript_tts` method is correctly implemented to handle the `streaming` parameter. The error suggests that somewhere in the code, a function is being treated as an iterable, which is not allowed. Here is a possible fix: 1. Check the implementation of `AudioService.transcript_tts` to ensure it correctly handles the `streaming` parameter. 2. Ensure that the `transcript_tts` method does not return a function or any non-iterable object when `streaming` is set to `true`. Here is an example of how you might modify the `transcript_tts` method to handle streaming correctly: ```python class AudioService: @staticmethod def transcript_tts(app_model, text, end_user, voice=None, streaming=False): if streaming: # Ensure that the response is an iterable, such as a generator def generate_audio_stream(): # Your logic to generate audio stream yield b'audio_chunk_1' yield b'audio_chunk_2' # Continue yielding audio chunks return generate_audio_stream() else: # Your logic to generate a complete audio file audio_data = b'complete_audio_file' return audio_data ``` In this example, when `streaming` is `true`, the method returns a generator that yields audio chunks, which is an iterable. When `streaming` is `false`, it returns a complete audio file, which is also an iterable (a bytes object) <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/controllers/service_api/app/audio.py)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:04:46 -05:00
Author
Owner
Copy Link
Copy Source

@yanyu510 commented on GitHub (Jun 15, 2024):

I looked at the code and it may be the problem of the following line of code

# api/core/model_runtime/model_providers/tongyi/tts/tts.py
if streaming:
            return Response(stream_with_context(self._tts_invoke_streaming(model=model,
                                                                           credentials=credentials,
                                                                           content_text=content_text,
                                                                           voice=voice,
                                                                           tenant_id=tenant_id)),
                            status=200, mimetype=f'audio/{audio_type}')

# should return response.get_audio_data() ? 
def _tts_invoke_streaming(self, model: str, tenant_id: str, credentials: dict, content_text: str,
                              voice: str) -> any:
        """
        _tts_invoke_streaming text2speech model

        :param model: model name
        :param tenant_id: user tenant id
        :param credentials: model credentials
        :param voice: model timbre
        :param content_text: text content to be translated
        :return: text translated to audio file
        """
        word_limit = self._get_model_word_limit(model, credentials)
        audio_type = self._get_model_audio_type(model, credentials)
        tts_file_id = self._get_file_name(content_text)
        file_path = f'generate_files/audio/{tenant_id}/{tts_file_id}.{audio_type}'
        try:
            sentences = list(self._split_text_into_sentences(text=content_text, limit=word_limit))
            for sentence in sentences:
                response = dashscope.audio.tts.SpeechSynthesizer.call(model=voice, sample_rate=48000,
                                                                      api_key=credentials.get('dashscope_api_key'),
                                                                      text=sentence.strip(),
                                                                      format=audio_type, word_timestamp_enabled=True,
                                                                      phoneme_timestamp_enabled=True)
                if isinstance(response.get_audio_data(), bytes):
                    storage.save(file_path, response.get_audio_data())
        except Exception as ex:
            raise InvokeBadRequestError(str(ex))
@yanyu510 commented on GitHub (Jun 15, 2024): I looked at the code and it may be the problem of the following line of code ```python # api/core/model_runtime/model_providers/tongyi/tts/tts.py if streaming: return Response(stream_with_context(self._tts_invoke_streaming(model=model, credentials=credentials, content_text=content_text, voice=voice, tenant_id=tenant_id)), status=200, mimetype=f'audio/{audio_type}') # should return response.get_audio_data() ? def _tts_invoke_streaming(self, model: str, tenant_id: str, credentials: dict, content_text: str, voice: str) -> any: """ _tts_invoke_streaming text2speech model :param model: model name :param tenant_id: user tenant id :param credentials: model credentials :param voice: model timbre :param content_text: text content to be translated :return: text translated to audio file """ word_limit = self._get_model_word_limit(model, credentials) audio_type = self._get_model_audio_type(model, credentials) tts_file_id = self._get_file_name(content_text) file_path = f'generate_files/audio/{tenant_id}/{tts_file_id}.{audio_type}' try: sentences = list(self._split_text_into_sentences(text=content_text, limit=word_limit)) for sentence in sentences: response = dashscope.audio.tts.SpeechSynthesizer.call(model=voice, sample_rate=48000, api_key=credentials.get('dashscope_api_key'), text=sentence.strip(), format=audio_type, word_timestamp_enabled=True, phoneme_timestamp_enabled=True) if isinstance(response.get_audio_data(), bytes): storage.save(file_path, response.get_audio_data()) except Exception as ex: raise InvokeBadRequestError(str(ex)) ```
yindo commented 2026-02-21 18:04:46 -05:00
Author
Owner
Copy Link
Copy Source

@laipz8200 commented on GitHub (Jun 15, 2024):

@yanyu510 Hi! Thanks for submitting the issue. Would you be interested in helping us fix it?

@laipz8200 commented on GitHub (Jun 15, 2024): @yanyu510 Hi! Thanks for submitting the issue. Would you be interested in helping us fix it?
yindo commented 2026-02-21 18:04:46 -05:00
Author
Owner
Copy Link
Copy Source

@yanyu510 commented on GitHub (Jun 15, 2024):

ok, i'm interested. let me understand the code first. Why did they make these code changes in commit 300d989? maybe need help!
image

@yanyu510 commented on GitHub (Jun 15, 2024): ok, i'm interested. let me understand the code first. Why did they make these code changes in commit 300d989? maybe need help! ![image](https://github.com/langgenius/dify/assets/13358679/0049226e-65b9-486b-a8ee-87a822e21c5a)
yindo commented 2026-02-21 18:04:47 -05:00
Author
Owner
Copy Link
Copy Source

@laipz8200 commented on GitHub (Jun 15, 2024):

@yanyu510 If other TTS providers are working, you might just need to check api/core/model_runtime/model_providers/tongyi/tts/tts.py.

Feel free to try and test it out. If you run into any issues, just let us know.

@laipz8200 commented on GitHub (Jun 15, 2024): @yanyu510 If other TTS providers are working, you might just need to check `api/core/model_runtime/model_providers/tongyi/tts/tts.py`. Feel free to try and test it out. If you run into any issues, just let us know.
yindo commented 2026-02-21 18:04:47 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Jun 16, 2024):

@charli117

@crazywoola commented on GitHub (Jun 16, 2024): @charli117
yindo commented 2026-02-21 18:04:47 -05:00
Author
Owner
Copy Link
Copy Source

@laipz8200 commented on GitHub (Jun 23, 2024):

Hi @yanyu510, it's been a week since we last communicated. Have you made any progress or run into any issues with this?

@laipz8200 commented on GitHub (Jun 23, 2024): Hi @yanyu510, it's been a week since we last communicated. Have you made any progress or run into any issues with this?
yindo commented 2026-02-21 18:04:47 -05:00
Author
Owner
Copy Link
Copy Source

@yanyu510 commented on GitHub (Jun 23, 2024):

I requested a PR. #5302 @laipz8200

@yanyu510 commented on GitHub (Jun 23, 2024): I requested a PR. #5302 @laipz8200
yindo commented 2026-02-21 18:04:47 -05:00
Author
Owner
Copy Link
Copy Source

@charli117 commented on GitHub (Jun 24, 2024):

Streaming tts has not been developed before, it will be processed together in this pr #5492 , involving more changes, and it will be verified together after improvement

@charli117 commented on GitHub (Jun 24, 2024): Streaming tts has not been developed before, it will be processed together in this pr #5492 , involving more changes, and it will be verified together after improvement
yindo commented 2026-02-21 18:04:47 -05:00
Author
Owner
Copy Link
Copy Source

@yanyu510 commented on GitHub (Jun 24, 2024):

@charli117 thx.
I think streaming is developed based on Flask streaming context and http chunked. I've implemented real-time buffered playback of long text-to-speech using fetch and mediasource api, why do you say it hasn't been developed yet?

@yanyu510 commented on GitHub (Jun 24, 2024): @charli117 thx. I think streaming is developed based on Flask streaming context and http chunked. I've implemented real-time buffered playback of long text-to-speech using fetch and mediasource api, why do you say it hasn't been developed yet?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#4107
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?