Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 09:35:22 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3885] [FEAT]: OpenAI Generic for Transcription, STT & TTS #2479

New Issue
Closed
opened 2026-02-22 18:29:52 -05:00 by yindo · 8 comments
yindo commented 2026-02-22 18:29:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @laweschan on GitHub (May 26, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3885

Originally assigned to: @shatfield4 on GitHub.

What would you like to see?

Hi,

 currently my region (hong kong) can't access openai and therefore we can't select openai as an option for transcription setting. also "voice & speech" setting has no too much setting option in order to point to external provider.

 can anythingllm provide the general setting so that we can point to prefer llm like vector and workspace one ?

thanks a lot

Regards,
Lawes

Originally created by @laweschan on GitHub (May 26, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3885 Originally assigned to: @shatfield4 on GitHub. ### What would you like to see? Hi, currently my region (hong kong) can't access openai and therefore we can't select openai as an option for transcription setting. also "voice & speech" setting has no too much setting option in order to point to external provider. can anythingllm provide the general setting so that we can point to prefer llm like vector and workspace one ? thanks a lot Regards, Lawes
yindo added the enhancementfeature request labels 2026-02-22 18:29:52 -05:00
yindo commented 2026-02-22 18:29:52 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 26, 2025):

We can ceratinly expand the trasnscription provider beyond the default built-in and OpenAI (using generic connection that is open-ai compatible), as well as doing that for STT & TTS

@timothycarambat commented on GitHub (May 26, 2025): We can ceratinly expand the trasnscription provider beyond the default built-in and OpenAI (using generic connection that is open-ai compatible), as well as doing that for STT & TTS
yindo commented 2026-02-22 18:29:52 -05:00
Author
Owner
Copy Link
Copy Source

@laweschan commented on GitHub (May 26, 2025):

thx, hope can ready to test in upcoming release if possible.
cheers,

@laweschan commented on GitHub (May 26, 2025): thx, hope can ready to test in upcoming release if possible. cheers,
yindo commented 2026-02-22 18:29:52 -05:00
Author
Owner
Copy Link
Copy Source

@learning4life commented on GitHub (May 27, 2025):

Generic connection for STT & TTS is a big plus for enterprise deployments.

@learning4life commented on GitHub (May 27, 2025): Generic connection for STT & TTS is a big plus for enterprise deployments.
yindo commented 2026-02-22 18:29:53 -05:00
Author
Owner
Copy Link
Copy Source

@learning4life commented on GitHub (May 28, 2025):

@shatfield4 if you need a tester for the STT implementation, ping me here 👍

@learning4life commented on GitHub (May 28, 2025): @shatfield4 if you need a tester for the STT implementation, ping me here 👍
yindo commented 2026-02-22 18:29:53 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 3, 2025):

Note Gemini compatibility: https://github.com/Mintplex-Labs/anything-llm/issues/3927

@timothycarambat commented on GitHub (Jun 3, 2025): Note Gemini compatibility: https://github.com/Mintplex-Labs/anything-llm/issues/3927
yindo commented 2026-02-22 18:29:53 -05:00
Author
Owner
Copy Link
Copy Source

@laweschan commented on GitHub (Jul 1, 2025):

Hi,

I would share a voice model that is better to handle STT / TTS / Transcription in chinese / cantonese support , and hope anythingllm can integrated / reuse so that it can better support on chinese env.

https://github.com/FunAudioLLM/SenseVoice
https://github.com/modelscope/FunASR

I note Gemini may support chinese STT / TTS well, but we can't use this LLM in our region .. too bad

Cheers,
Lawes

@laweschan commented on GitHub (Jul 1, 2025): Hi, I would share a voice model that is better to handle STT / TTS / Transcription in chinese / cantonese support , and hope anythingllm can integrated / reuse so that it can better support on chinese env. https://github.com/FunAudioLLM/SenseVoice https://github.com/modelscope/FunASR I note Gemini may support chinese STT / TTS well, but we can't use this LLM in our region .. too bad Cheers, Lawes
yindo commented 2026-02-22 18:29:53 -05:00
Author
Owner
Copy Link
Copy Source

@fkdp06 commented on GitHub (Jan 26, 2026):

For me the whisper doesn't work as expected. I used ollama run karanchopda333/whisper but no lack to make it working. I tried to upload 2025 - unknown album.mp3 which contains my voice, but it fails. Any suggestions?

Image Image

This is a log collector-2026-01-26.log:

{"level":"info","message":"\u001b[35m[TikTokenTokenizer]\u001b[0m Initialized new TikTokenTokenizer instance.","service":"collector"}
{"level":"info","message":"Collector hot directory and tmp storage wiped!","service":"collector"}
{"level":"info","message":"[production] AnythingLLM Standalone Document processor listening on port 8888.","service":"collector"}
{"level":"info","message":"\u001b[35m[TikTokenTokenizer]\u001b[0m Initialized new TikTokenTokenizer instance.","service":"collector"}
{"level":"info","message":"Collector hot directory and tmp storage wiped!","service":"collector"}
{"level":"info","message":"[production] AnythingLLM Standalone Document processor listening on port 8888.","service":"collector"}
{"level":"info","message":"-- Working 2025 - unknown album.mp3 --","service":"collector"}
{"level":"info","message":"\u001b[32m[LocalWhisper]\u001b[0m Initialized.","service":"collector"}
{"level":"error","message":"Error: LocalWhisper Provider for audio transcription is not supported anymore. Use OpenAIWhisper instead.\n    at ve.processFile (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:14:3779)\n    at Fs (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:14:5504)\n    at Si (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:23:3103)\n    at C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:54:1677\n    at Layer.handle [as handle_request] (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\layer.js:95:5)\n    at next (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\route.js:149:13)\n    at ya (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:26:4416)\n    at Layer.handle [as handle_request] (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\layer.js:95:5)\n    at next (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\route.js:149:13)\n    at Route.dispatch (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\route.js:119:3)","service":"collector"}
@fkdp06 commented on GitHub (Jan 26, 2026): For me the whisper doesn't work as expected. I used `ollama run karanchopda333/whisper` but no lack to make it working. I tried to upload `2025 - unknown album.mp3` which contains my voice, but it fails. Any suggestions? <img width="661" height="264" alt="Image" src="https://github.com/user-attachments/assets/1426bb44-f05b-4f37-95aa-25fa04252b45" /> <img width="569" height="587" alt="Image" src="https://github.com/user-attachments/assets/a5bf69ba-3c08-40f1-9c59-395b01934522" /> This is a log `collector-2026-01-26.log`: ``` {"level":"info","message":"\u001b[35m[TikTokenTokenizer]\u001b[0m Initialized new TikTokenTokenizer instance.","service":"collector"} {"level":"info","message":"Collector hot directory and tmp storage wiped!","service":"collector"} {"level":"info","message":"[production] AnythingLLM Standalone Document processor listening on port 8888.","service":"collector"} {"level":"info","message":"\u001b[35m[TikTokenTokenizer]\u001b[0m Initialized new TikTokenTokenizer instance.","service":"collector"} {"level":"info","message":"Collector hot directory and tmp storage wiped!","service":"collector"} {"level":"info","message":"[production] AnythingLLM Standalone Document processor listening on port 8888.","service":"collector"} {"level":"info","message":"-- Working 2025 - unknown album.mp3 --","service":"collector"} {"level":"info","message":"\u001b[32m[LocalWhisper]\u001b[0m Initialized.","service":"collector"} {"level":"error","message":"Error: LocalWhisper Provider for audio transcription is not supported anymore. Use OpenAIWhisper instead.\n at ve.processFile (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:14:3779)\n at Fs (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:14:5504)\n at Si (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:23:3103)\n at C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:54:1677\n at Layer.handle [as handle_request] (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\layer.js:95:5)\n at next (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\route.js:149:13)\n at ya (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\collector.js:26:4416)\n at Layer.handle [as handle_request] (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\layer.js:95:5)\n at next (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\route.js:149:13)\n at Route.dispatch (C:\\Users\\Admin\\AppData\\Local\\Programs\\AnythingLLM\\resources\\backend\\node_modules\\express\\lib\\router\\route.js:119:3)","service":"collector"} ```
yindo changed title from [FEAT]: OpenAI Generic for Transcription, STT & TTS to [GH-ISSUE #3885] [FEAT]: OpenAI Generic for Transcription, STT & TTS 2026-06-05 14:46:48 -04:00
yindo closed this issue 2026-06-05 14:46:49 -04:00
yindo commented 2026-06-05 14:46:49 -04:00
Author
Owner
Copy Link
Copy Source

@LauraGPT commented on GitHub (May 23, 2026):

+1 for OpenAI Generic STT support. FunASR (16K+ stars) already provides an OpenAI-compatible endpoint:

pip install funasr fastapi uvicorn python-multipart
funasr-server --device cuda --port 8000
# Now POST /v1/audio/transcriptions works

If AnythingLLM supports generic OpenAI-format STT endpoints (custom base URL), FunASR becomes a zero-config local alternative:

  • 170x realtime (non-autoregressive, much faster than Whisper)
  • 50+ languages, auto-detection
  • Speaker diarization built-in
  • Runs on CPU too (17x realtime)

Would love to see a "Custom OpenAI STT" provider option.

<!-- gh-comment-id:4526458215 --> @LauraGPT commented on GitHub (May 23, 2026): +1 for OpenAI Generic STT support. [FunASR](https://github.com/modelscope/FunASR) (16K+ stars) already provides an OpenAI-compatible endpoint: ```bash pip install funasr fastapi uvicorn python-multipart funasr-server --device cuda --port 8000 # Now POST /v1/audio/transcriptions works ``` If AnythingLLM supports generic OpenAI-format STT endpoints (custom base URL), FunASR becomes a zero-config local alternative: - **170x realtime** (non-autoregressive, much faster than Whisper) - **50+ languages**, auto-detection - Speaker diarization built-in - Runs on CPU too (17x realtime) Would love to see a "Custom OpenAI STT" provider option.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2479
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?