langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Add comprehensive unit tests for AudioService #20648

New Issue
Closed
opened 2026-02-21 20:08:18 -05:00 by yindo · 1 comment
yindo commented 2026-02-21 20:08:18 -05:00
Owner
Copy Link
Copy Source

Originally created by @jworth008 on GitHub (Nov 28, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

While working on improving test coverage for the Dify backend API, I noticed that the AudioService class in /api/services/audio_service.py lacks any unit tests. This service is critical for handling audio processing operations across the platform, enabling users to convert speech to text (ASR) and text to speech (TTS).

The service handles three main areas of functionality:

  • Speech-to-Text (ASR): Transcribing audio files to text with validation
  • Text-to-Speech (TTS): Converting text to audio with voice selection
  • TTS Voice Listing: Retrieving available voices for different languages

Without proper test coverage, it's difficult to:

  1. Ensure file validation works correctly (30MB size limit, supported audio formats)
  2. Verify feature flag validation (speech_to_text/text_to_speech enabled)
  3. Validate proper error handling for various failure scenarios
  4. Test voice selection logic (explicit, default, auto-discovery)
  5. Confirm app mode handling (CHAT vs ADVANCED_CHAT/WORKFLOW)
  6. Verify message-based TTS works correctly (UUID validation, empty answers)
  7. Test ModelManager integration and model instance availability
  8. Prevent regressions when making changes to audio functionality

This gap in test coverage poses a risk for maintaining code quality and user experience, especially as audio features are increasingly important for accessibility and user interaction.

2. Additional context or comments

Test Coverage for Speech-to-Text (ASR) - 8 tests:

  • Successful transcription in CHAT mode
  • Successful transcription in ADVANCED_CHAT/WORKFLOW mode
  • Feature disabled error (CHAT mode)
  • Feature disabled error (WORKFLOW mode)
  • Missing workflow error
  • No file uploaded error (NoAudioUploadedServiceError)
  • Unsupported audio type error (UnsupportedAudioTypeServiceError)
  • File too large error (AudioTooLargeServiceError - 30MB limit)
  • No model instance error (ProviderNotSupportSpeechToTextServiceError)

Test Coverage for Text-to-Speech (TTS) - 11 tests:

  • TTS with text input (explicit voice)
  • TTS with message ID (database lookup)
  • TTS with default voice from config
  • TTS with auto-discovered voice (first available)
  • TTS in WORKFLOW mode with draft workflow
  • Missing text error (ValueError)
  • Invalid message ID format (returns None)
  • Non-existent message (returns None)
  • Empty message answer (returns None)
  • No voices available error (ValueError)
  • Feature validation for different app modes

Test Coverage for TTS Voice Listing - 3 tests:

  • Successful voice retrieval with language filter
  • No model instance error (ProviderNotSupportTextToSpeechServiceError)
  • Exception propagation from model instance

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @jworth008 on GitHub (Nov 28, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. While working on improving test coverage for the Dify backend API, I noticed that the `AudioService` class in `/api/services/audio_service.py` lacks any unit tests. This service is critical for handling audio processing operations across the platform, enabling users to convert speech to text (ASR) and text to speech (TTS). The service handles three main areas of functionality: - **Speech-to-Text (ASR)**: Transcribing audio files to text with validation - **Text-to-Speech (TTS)**: Converting text to audio with voice selection - **TTS Voice Listing**: Retrieving available voices for different languages Without proper test coverage, it's difficult to: 1. Ensure file validation works correctly (30MB size limit, supported audio formats) 2. Verify feature flag validation (speech_to_text/text_to_speech enabled) 3. Validate proper error handling for various failure scenarios 4. Test voice selection logic (explicit, default, auto-discovery) 5. Confirm app mode handling (CHAT vs ADVANCED_CHAT/WORKFLOW) 6. Verify message-based TTS works correctly (UUID validation, empty answers) 7. Test ModelManager integration and model instance availability 8. Prevent regressions when making changes to audio functionality This gap in test coverage poses a risk for maintaining code quality and user experience, especially as audio features are increasingly important for accessibility and user interaction. ### 2. Additional context or comments ### Test Coverage for Speech-to-Text (ASR) - 8 tests: - Successful transcription in CHAT mode - Successful transcription in ADVANCED_CHAT/WORKFLOW mode - Feature disabled error (CHAT mode) - Feature disabled error (WORKFLOW mode) - Missing workflow error - No file uploaded error (NoAudioUploadedServiceError) - Unsupported audio type error (UnsupportedAudioTypeServiceError) - File too large error (AudioTooLargeServiceError - 30MB limit) - No model instance error (ProviderNotSupportSpeechToTextServiceError) ### Test Coverage for Text-to-Speech (TTS) - 11 tests: - TTS with text input (explicit voice) - TTS with message ID (database lookup) - TTS with default voice from config - TTS with auto-discovered voice (first available) - TTS in WORKFLOW mode with draft workflow - Missing text error (ValueError) - Invalid message ID format (returns None) - Non-existent message (returns None) - Empty message answer (returns None) - No voices available error (ValueError) - Feature validation for different app modes ### Test Coverage for TTS Voice Listing - 3 tests: - Successful voice retrieval with language filter - No model instance error (ProviderNotSupportTextToSpeechServiceError) - Exception propagation from model instance ### 3. Can you help us with this feature? - [x] I am interested in contributing to this feature.
yindo added the 💪 enhancement label 2026-02-21 20:08:18 -05:00
yindo closed this issue 2026-02-21 20:08:18 -05:00
yindo commented 2026-02-21 20:08:19 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 28, 2025):

Here's a practical guide for adding comprehensive unit tests for AudioService:

  • Key methods to test:

    • transcript_asr (speech-to-text): validates app mode, feature flags, file presence/type/size, and uses ModelManager for transcription.
    • transcript_tts (text-to-speech): supports text or message ID input, handles voice selection (explicit, config, auto-discovery), app modes, and returns either a Flask streaming response or direct model output.
    • transcript_tts_voices: retrieves available voices, raises errors if no model instance, propagates exceptions from the model instance
      (source).

  • Dependencies to mock:

    • ModelManager and its model instances
    • Database queries for Message (for message-based TTS)
    • File uploads (FileStorage)
    • Flask Response and streaming (for TTS audio output)
    • Custom exceptions from services.errors.audio (source)

  • Testing framework:

    • Use pytest with unittest.mock for mocking dependencies.
    • The test suite sets up a Flask app fixture and global Redis mock in conftest.py (source).

  • Test organization:

    • Place your tests in api/tests/unit_tests/services/test_audio_service.py.

  • Best practices:

    • Cover all success and error branches for each method, as outlined in your scenario list.
    • Use parameterized tests for different app modes and feature flag states.
    • Mock file uploads with various mimetypes and sizes to trigger validation errors.
    • For TTS, mock both direct text input and message ID lookups, including edge cases (invalid UUID, missing/empty message).
    • For streaming responses, use Flask's test client or mock stream_with_context to verify audio output handling.

If you want example test code or help with specific mocking patterns, let me know!

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Nov 28, 2025): <!-- Answer --> Here's a practical guide for adding comprehensive unit tests for `AudioService`: - **Key methods to test:** - `transcript_asr` (speech-to-text): validates app mode, feature flags, file presence/type/size, and uses `ModelManager` for transcription. - `transcript_tts` (text-to-speech): supports text or message ID input, handles voice selection (explicit, config, auto-discovery), app modes, and returns either a Flask streaming response or direct model output. - `transcript_tts_voices`: retrieves available voices, raises errors if no model instance, propagates exceptions from the model instance ([source](https://github.com/langgenius/dify/blob/c51ab6ec3722338ad619f079ab235a2435be7f6c/api/services/audio_service.py#L10-L166)). - **Dependencies to mock:** - `ModelManager` and its model instances - Database queries for `Message` (for message-based TTS) - File uploads (`FileStorage`) - Flask `Response` and streaming (for TTS audio output) - Custom exceptions from `services.errors.audio` ([source](https://github.com/langgenius/dify/blob/c51ab6ec3722338ad619f079ab235a2435be7f6c/api/services/errors/audio.py#L1-L22)) - **Testing framework:** - Use `pytest` with `unittest.mock` for mocking dependencies. - The test suite sets up a Flask app fixture and global Redis mock in `conftest.py` ([source](https://github.com/langgenius/dify/blob/c51ab6ec3722338ad619f079ab235a2435be7f6c/api/tests/unit_tests/conftest.py#L4-L38)). - **Test organization:** - Place your tests in `api/tests/unit_tests/services/test_audio_service.py`. - **Best practices:** - Cover all success and error branches for each method, as outlined in your scenario list. - Use parameterized tests for different app modes and feature flag states. - Mock file uploads with various mimetypes and sizes to trigger validation errors. - For TTS, mock both direct text input and message ID lookups, including edge cases (invalid UUID, missing/empty message). - For streaming responses, use Flask's test client or mock `stream_with_context` to verify audio output handling. If you want example test code or help with specific mocking patterns, let me know! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/9a0db67a-bdff-48c5-8cc4-0ceb7f0b8eaf?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/9a0db67a-bdff-48c5-8cc4-0ceb7f0b8eaf?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/9a0db67a-bdff-48c5-8cc4-0ceb7f0b8eaf?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/9a0db67a-bdff-48c5-8cc4-0ceb7f0b8eaf?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/9a0db67a-bdff-48c5-8cc4-0ceb7f0b8eaf?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/9a0db67a-bdff-48c5-8cc4-0ceb7f0b8eaf?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/9a0db67a-bdff-48c5-8cc4-0ceb7f0b8eaf?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=Here%27s%20a%20practical%20guide%20for%20adding%20comprehensive%20unit%20tests%20for%20%60AudioService%60%3A%0A%0A-%20%2A%2AKey%20methods%20to%20test%3A%2A%2A%20%20%0A%20%20-%20%60transcript_asr%60%20%28speech-to-text%29%3A%20validates%20app%20mode%2C%20feature%20flags%2C%20file%20presence/type/size%2C%20and%20uses%20%60ModelManager%60%20for%20transcription.%20%20%0A%20%20-%20%60transcript_tts%60%20%28text-to-speech%29%3A%20supports%20text%20or%20message%20ID%20input%2C%20handles%20voice%20selection%20%28explicit%2C%20config%2C%20auto-discovery%29%2C%20app%20modes%2C%20and%20returns%20either%20a%20Flask%20streaming%20response%20or%20direct%20model%20output.%20%20%0A%20%20-%20%60transcript_tts_voices%60%3A%20retrieves%20available%20voices%2C%20raises%20errors%20if%20no%20model%20instance%2C%20propagates%20exceptions%20from%20the%20model%20instance%20%20%0A%20%20%28%5Bsource%5D%28https%3A//github.com/langgenius/dify/blob/c51ab6ec3722338ad619f079ab235a2435be7f6c/api/services/audio_service.py%23L10-L166%29%29.%0A%0A-%20%2A%2ADependencies%20to%20mock%3A%2A%2A%20%20%0A%20%20-%20%60ModelManager%60%20and%20its%20model%20instances%20%20%0A%20%20-%20Database%20queries%20for%20%60Message%60%20%28for%20message-based%20TTS%29%20%20%0A%20%20-%20File%20uploads%20%28%60FileStorage%60%29%20%20%0A%20%20-%20Flask%20%60Response%60%20and%20streaming%20%28for%20TTS%20audio%20output%29%20%20%0A%20%20-%20Custom%20exceptions%20from%20%60services.errors.audio%60%20%28%5Bsource%5D%28https%3A//github.com/langgenius/dify/blob/c51ab6ec3722338ad619f079ab235a2435be7f6c/api/services/errors/audio.py%23L1-L22%29%29%0A%0A-%20%2A%2ATesting%20framework%3A%2A%2A%20%20%0A%20%20-%20Use%20%60pytest%60%20with%20%60unittest.mock%60%20for%20mocking%20dependencies.%20%20%0A%20%20-%20The%20test%20suite%20sets%20up%20a%20Flask%20app%20fixture%20and%20global%20Redis%20mock%20in%20%60conftest.py%60%20%28%5Bsource%5D%28https%3A//github.com/langgenius/dify/blob/c51ab6ec3722338ad619f079ab235a2435be7f6c/api/tests/unit_tests/conftest.py%23L4-L38%29%29.%0A%0A-%20%2A%2ATest%20organization%3A%2A%2A%20%20%0A%20%20-%20Place%20your%20tests%20in%20%60api/tests/unit_tests/services/test_audio_service.py%60.%0A%0A-%20%2A%2ABest%20practices%3A%2A%2A%20%20%0A%20%20-%20Cover%20all%20success%20and%20error%20branches%20for%20each%20method%2C%20as%20outlined%20in%20your%20scenario%20list.%0A%20%20-%20Use%20parameterized%20tests%20for%20different%20app%20modes%20and%20feature%20flag%20states.%0A%20%20-%20Mock%20file%20uploads%20with%20various%20mimetypes%20and%20sizes%20to%20trigger%20validation%20errors.%0A%20%20-%20For%20TTS%2C%20mock%20both%20direct%20text%20input%20and%20message%20ID%20lookups%2C%20including%20edge%20cases%20%28invalid%20UUID%2C%20missing/empty%20message%29.%0A%20%20-%20For%20streaming%20responses%2C%20use%20Flask%27s%20test%20client%20or%20mock%20%60stream_with_context%60%20to%20verify%20audio%20output%20handling.%0A%0AIf%20you%20want%20example%20test%20code%20or%20help%20with%20specific%20mocking%20patterns%2C%20let%20me%20know%21)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/28859)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#20648
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?