langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

OpenAI Audio Plugin Fail #327

New Issue
Closed
opened 2026-02-16 10:19:00 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 10:19:00 -05:00
Owner
Copy Link
Copy Source

Originally created by @tanreinama on GitHub (May 30, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.4.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

openai_audio plugin (https://marketplace.dify.ai/plugins/lysonober/openai_audio) fails in local uploaded file.
plugin version is 0.0.4

Image

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Originally created by @tanreinama on GitHub (May 30, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.4.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce openai_audio plugin (https://marketplace.dify.ai/plugins/lysonober/openai_audio) fails in local uploaded file. plugin version is 0.0.4 ![Image](https://github.com/user-attachments/assets/1981577e-5ee6-46f8-8bc6-6ffd44e2d904) ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior _No response_
yindo closed this issue 2026-02-16 10:19:00 -05:00
yindo commented 2026-02-16 10:19:01 -05:00
Author
Owner
Copy Link
Copy Source

@Olemi-llm-apprentice commented on GitHub (Jun 4, 2025):

Hi @tanreinama,

Thank you for reporting this issue. I've analyzed the problem and identified the root cause of the OpenAI audio plugin failure.

Root Cause Analysis

The issue occurs because the current OpenAI plugin implementation is missing support for AudioPromptMessageContent in the message processing logic. Specifically:

  1. Missing import: The AudioPromptMessageContent class is not imported in /models/openai/models/llm/llm.py
  2. Missing audio processing logic: The _convert_prompt_message_to_dict method doesn't handle audio content conversion
  3. Inadequate token limits: The current max_tokens setting (4096) is too low for audio processing scenarios

Technical Details

The expected audio message processing should convert audio content to the OpenAI API format:

{
    "type": "input_audio",
    "input_audio": {
        "data": "base64_encoded_audio_data",
        "format": "wav"
    }
}

However, this conversion is currently not implemented.

Proposed Solution

I'm planning to contribute a fix that includes:

  1. ✅ Add AudioPromptMessageContent import and processing logic
  2. ✅ Update token limits for audio models (16384 instead of 4096)
  3. ✅ Add support for the new gpt-4o-audio-preview-0603 model (released yesterday)

Timeline

I'll start working on this fix and submit a PR within the next few days. This will make the audio functionality work as expected.

@Olemi-llm-apprentice commented on GitHub (Jun 4, 2025): Hi @tanreinama, Thank you for reporting this issue. I've analyzed the problem and identified the root cause of the OpenAI audio plugin failure. ## Root Cause Analysis The issue occurs because the current OpenAI plugin implementation is **missing support for `AudioPromptMessageContent`** in the message processing logic. Specifically: 1. **Missing import**: The `AudioPromptMessageContent` class is not imported in `/models/openai/models/llm/llm.py` 2. **Missing audio processing logic**: The `_convert_prompt_message_to_dict` method doesn't handle audio content conversion 3. **Inadequate token limits**: The current max_tokens setting (4096) is too low for audio processing scenarios ## Technical Details The expected audio message processing should convert audio content to the OpenAI API format: ```python { "type": "input_audio", "input_audio": { "data": "base64_encoded_audio_data", "format": "wav" } } ``` However, this conversion is currently not implemented. ## Proposed Solution I'm planning to contribute a fix that includes: 1. ✅ Add `AudioPromptMessageContent` import and processing logic 2. ✅ Update token limits for audio models (16384 instead of 4096) 3. ✅ Add support for the new `gpt-4o-audio-preview-0603` model (released yesterday) ## Timeline I'll start working on this fix and submit a PR within the next few days. This will make the audio functionality work as expected.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#327
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?