Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3806] [FEAT]: Lightweight STT Parakeet-tdt-0.6b-v2 #2438

New Issue
Closed
opened 2026-02-22 18:29:40 -05:00 by yindo · 7 comments
yindo commented 2026-02-22 18:29:40 -05:00
Owner
Copy Link
Copy Source

Originally created by @privacy-advo on GitHub (May 11, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3806

What would you like to see?

Lightweight STT Parakeet-tdt-0.6b-v2

Whisper is still resource-intensive. NVIDIA's Parakeet-tdt-0.6b-v2 currently leads the Open ASR Leaderboard and offers a lightweight, high-performing alternative.

https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2

Originally created by @privacy-advo on GitHub (May 11, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3806 ### What would you like to see? **Lightweight STT Parakeet-tdt-0.6b-v2** Whisper is still resource-intensive. [NVIDIA's Parakeet-tdt-0.6b-v2](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2) currently leads the [Open ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) and offers a lightweight, high-performing alternative. https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2
yindo added the enhancementfeature request labels 2026-02-22 18:29:40 -05:00
yindo commented 2026-02-22 18:29:41 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 12, 2025):

Whisper is still resource-intensive

This is true, but there are tiny and large ONNX distributions of Whisper (250mb - 1.5GB). Parakeet is 2.7GB but does not seem to have a version for ONNX at this time, so we cannot use it in the same way. That being said, if you run the model locally, we have should support API based STT providers via a Generic connector to make that easier and more expansive to support with building a whole workflow for every model/provider

@timothycarambat commented on GitHub (May 12, 2025): > Whisper is still resource-intensive This is true, but there are `tiny` and `large` ONNX distributions of Whisper (250mb - 1.5GB). Parakeet is 2.7GB but does not seem to have a version for ONNX at this time, so we cannot use it in the same way. That being said, if you run the model locally, we have should support API based STT providers via a Generic connector to make that easier and more expansive to support with building a whole workflow for every model/provider
yindo commented 2026-02-22 18:29:41 -05:00
Author
Owner
Copy Link
Copy Source

@privacy-advo commented on GitHub (May 18, 2025):

I can only select whisper-tiny in v1.8.1. Base would be a sweet spot as whisper is only running in CPU.

@privacy-advo commented on GitHub (May 18, 2025): I can only select whisper-tiny in v1.8.1. Base would be a sweet spot as whisper is only running in CPU.
yindo commented 2026-02-22 18:29:41 -05:00
Author
Owner
Copy Link
Copy Source

@deepanshu-yadav commented on GitHub (May 23, 2025):

Hello Guys, Someone has released integer converted onnx versions of parakeet here https://github.com/k2-fsa/sherpa-onnx/blob/master/scripts/nemo/parakeet-tdt-0.6b-v2/test_onnx.py I utilized them in my project https://github.com/deepanshu-yadav/voice-form-filler and it works quite decently even without a GPU.

@deepanshu-yadav commented on GitHub (May 23, 2025): Hello Guys, Someone has released integer converted onnx versions of parakeet here https://github.com/k2-fsa/sherpa-onnx/blob/master/scripts/nemo/parakeet-tdt-0.6b-v2/test_onnx.py I utilized them in my project https://github.com/deepanshu-yadav/voice-form-filler and it works quite decently even without a GPU.
yindo commented 2026-02-22 18:29:41 -05:00
Author
Owner
Copy Link
Copy Source

@privacy-advo commented on GitHub (May 23, 2025):

@timothycarambat
Option to reopen the issue?
https://huggingface.co/onnx-community/parakeet-tdt-0.6b-v2-ONNX/tree/main/onnx

@privacy-advo commented on GitHub (May 23, 2025): @timothycarambat Option to reopen the issue? https://huggingface.co/onnx-community/parakeet-tdt-0.6b-v2-ONNX/tree/main/onnx
yindo commented 2026-02-22 18:29:41 -05:00
Author
Owner
Copy Link
Copy Source

@csukuangfj commented on GitHub (May 27, 2025):

I suggest that you have a look at
https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-transducer/nemo-transducer-models.html#sherpa-onnx-nemo-parakeet-tdt-0-6b-v2-int8-english

We have benchmarked its speed on RK3588 using Cortex A76 CPUs. It is super-fast without a GPU.

@csukuangfj commented on GitHub (May 27, 2025): I suggest that you have a look at https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-transducer/nemo-transducer-models.html#sherpa-onnx-nemo-parakeet-tdt-0-6b-v2-int8-english We have benchmarked its speed on RK3588 using Cortex A76 CPUs. It is super-fast without a GPU.
yindo commented 2026-02-22 18:29:41 -05:00
Author
Owner
Copy Link
Copy Source

@csukuangfj commented on GitHub (May 27, 2025):

We also provide Android APKs for this model.

1. Non-streaming speech recognition

https://k2-fsa.github.io/sherpa/onnx/vad/apk-asr.html

Image

2. Streaming speech recognition

https://k2-fsa.github.io/sherpa/onnx/android/apk-simulate-streaming-asr.html

Image
@csukuangfj commented on GitHub (May 27, 2025): We also provide Android APKs for this model. ## 1. Non-streaming speech recognition https://k2-fsa.github.io/sherpa/onnx/vad/apk-asr.html <img width="1087" alt="Image" src="https://github.com/user-attachments/assets/f9759f8d-6007-4775-b4a1-21221132f61b" /> ## 2. Streaming speech recognition https://k2-fsa.github.io/sherpa/onnx/android/apk-simulate-streaming-asr.html <img width="1081" alt="Image" src="https://github.com/user-attachments/assets/3f28b99c-0697-44eb-9007-41363811948a" />
yindo changed title from [FEAT]: Lightweight STT Parakeet-tdt-0.6b-v2 to [GH-ISSUE #3806] [FEAT]: Lightweight STT Parakeet-tdt-0.6b-v2 2026-06-05 14:46:32 -04:00
yindo closed this issue 2026-06-05 14:46:32 -04:00
yindo commented 2026-06-05 14:46:33 -04:00
Author
Owner
Copy Link
Copy Source

@LauraGPT commented on GitHub (May 23, 2026):

Another lightweight option: FunASR SenseVoice-Small — 234MB model, 17x realtime on CPU, 170x on GPU. Smaller than Parakeet and faster.

Now has an OpenAI-compatible server: funasr-server --device cpu → local /v1/audio/transcriptions.

<!-- gh-comment-id:4526458904 --> @LauraGPT commented on GitHub (May 23, 2026): Another lightweight option: [FunASR SenseVoice-Small](https://github.com/modelscope/FunASR) — 234MB model, 17x realtime on CPU, 170x on GPU. Smaller than Parakeet and faster. Now has an OpenAI-compatible server: `funasr-server --device cpu` → local `/v1/audio/transcriptions`.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2438
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?