mirror of
https://github.com/modelscope/FunASR.git
synced 2026-10-12 02:19:18 +00:00
zhifu gaoandLauraGPT
84e049ed48
* docs: design product deployment hub * docs(site): plan deployment product hub implementation * feat(site): add deployment evidence registry * feat(site): add deployment recommendation engine * build(site): track reproducible live-site corpus * feat(site): build bilingual deployment product surface * feat(site): add deployment contracts and benchmarks * feat(site): preserve legacy routes and unify navigation * ci(site): validate reproducible product builds * ops(site): add atomic releases and hardened nginx * test(site): add responsive product browser coverage * fix(site): support managed and manual nginx masters * fix(site): make release smoke routes configurable * docs(site): record product hub rollout --------- Co-authored-by: LauraGPT <[email protected]>
133 lines
14 KiB
HTML
133 lines
14 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||
<title>Run FunASR on CPU: the llama.cpp / GGUF Runtime (a whisper.cpp Alternative) | FunASR</title>
|
||
<meta name="description" content="Download-and-run FunASR llama.cpp runtime: a single binary, no Python or GPU, running SenseVoice/Paraformer/Fun-ASR-Nano on CPU/edge with built-in VAD. Chinese CER ~1/3 of whisper.cpp. Prebuilt binaries for Linux/macOS/Windows.">
|
||
<meta name="keywords" content="FunASR llama.cpp, GGUF speech recognition, CPU speech recognition, offline speech to text, whisper.cpp alternative, on-device ASR, speech to text without GPU, SenseVoice GGUF">
|
||
<link rel="icon" href="/logo.png">
|
||
<meta property="og:title" content="Run FunASR on CPU: the llama.cpp / GGUF Runtime (a whisper.cpp Alternative) | FunASR"><meta property="og:description" content="Download-and-run FunASR llama.cpp runtime: a single binary, no Python or GPU, running SenseVoice/Paraformer/Fun-ASR-Nano on CPU/edge with built-in VAD. Chinese CER ~1/3 of whisper.cpp. Prebuilt binaries for Linux/macOS/Windows.">
|
||
<meta property="og:type" content="website"><meta property="og:url" content="https://www.funasr.com/en/llama-cpp.html">
|
||
<meta property="og:image" content="https://www.funasr.com/logo.png">
|
||
<link rel="canonical" href="https://www.funasr.com/en/llama-cpp.html">
|
||
<link rel="alternate" hreflang="en" href="https://www.funasr.com/en/llama-cpp.html"><link rel="alternate" hreflang="zh" href="https://www.funasr.com/llama-cpp.html">
|
||
<link rel="alternate" hreflang="x-default" href="https://www.funasr.com/en/llama-cpp.html">
|
||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&display=swap" rel="stylesheet">
|
||
<script type="application/ld+json">
|
||
{"@context":"https://schema.org","@type":"SoftwareApplication","name":"FunASR llama.cpp runtime","description":"Download-and-run FunASR llama.cpp runtime: a single binary, no Python or GPU, running SenseVoice/Paraformer/Fun-ASR-Nano on CPU/edge with built-in VAD. Chinese CER ~1/3 of whisper.cpp. Prebuilt binaries for Linux/macOS/Windows.","applicationCategory":"DeveloperApplication","operatingSystem":"Linux, macOS, Windows","offers":{"@type":"Offer","price":"0","priceCurrency":"USD"},"author":{"@type":"Organization","name":"FunASR / Tongyi Lab"},"url":"https://www.funasr.com/en/llama-cpp.html"}
|
||
</script>
|
||
<style>:root{--primary:#2563eb;--primary-dark:#1d4ed8;--bg:#fff;--surface:#f8fafc;--border:#e2e8f0;--text:#0f172a;--text-soft:#475569;--text-muted:#94a3b8;--radius:12px}
|
||
*{margin:0;padding:0;box-sizing:border-box}
|
||
body{font-family:'Inter',-apple-system,sans-serif;color:var(--text);line-height:1.7}
|
||
a{color:var(--primary);text-decoration:none}a:hover{text-decoration:underline}
|
||
.container{max-width:800px;margin:0 auto;padding:0 24px}
|
||
.nav{position:fixed;top:0;left:0;right:0;z-index:100;background:rgba(255,255,255,0.9);backdrop-filter:blur(12px);border-bottom:1px solid var(--border);padding:14px 0}
|
||
.nav .container{max-width:1120px;display:flex;align-items:center;gap:24px}
|
||
.nav-logo{font-size:1.2rem;font-weight:800;color:var(--text)}.nav-logo span{color:var(--primary)}
|
||
.nav-links{display:flex;gap:20px;margin-left:auto;flex-wrap:wrap}
|
||
.nav-links a{font-size:0.85rem;font-weight:500;color:var(--text-soft)}
|
||
.nav-btn{padding:7px 16px;background:var(--primary);color:#fff;border-radius:7px;font-size:0.83rem;font-weight:600}
|
||
article{padding:120px 0 80px}
|
||
h1{font-size:2rem;font-weight:800;line-height:1.3;margin-bottom:14px}
|
||
h2{font-size:1.3rem;font-weight:700;margin:36px 0 14px;padding-top:16px;border-top:1px solid var(--border)}
|
||
h3{font-size:1.05rem;font-weight:700;margin:22px 0 10px}
|
||
p{margin-bottom:14px;color:var(--text-soft)}
|
||
ul,ol{margin:0 0 14px 22px;color:var(--text-soft)}li{margin-bottom:6px}
|
||
table{width:100%;border-collapse:collapse;margin:16px 0;font-size:0.86rem}
|
||
th,td{padding:10px 12px;text-align:left;border-bottom:1px solid var(--border)}
|
||
th{font-weight:600;background:var(--surface);font-size:0.78rem;color:var(--text-muted)}
|
||
pre{background:#1e293b;color:#e2e8f0;padding:18px;border-radius:10px;overflow-x:auto;font-size:0.82rem;line-height:1.7;margin:14px 0}
|
||
code{font-family:ui-monospace,Menlo,monospace}
|
||
p code,li code{background:var(--surface);padding:2px 6px;border-radius:5px;font-size:0.85em;color:var(--primary-dark)}
|
||
.post-meta{color:var(--text-muted);font-size:0.85rem;margin-bottom:24px}
|
||
.cta{background:var(--surface);border:1px solid var(--border);border-radius:var(--radius);padding:24px;margin:28px 0;text-align:center}
|
||
.cta a.btn{display:inline-block;padding:10px 24px;background:var(--primary);color:#fff;border-radius:8px;font-weight:600}
|
||
.post-list{list-style:none;margin:0;padding:0}
|
||
.post-list li{margin:0 0 8px}
|
||
.post-card{display:block;border:1px solid var(--border);border-radius:var(--radius);padding:20px 22px;margin-bottom:14px}
|
||
.post-card:hover{border-color:var(--primary);text-decoration:none}
|
||
.post-card h2{border:none;padding:0;margin:0 0 6px;font-size:1.15rem;color:var(--text)}
|
||
.post-card p{margin:0;color:var(--text-soft);font-size:0.9rem}
|
||
.post-card .date{color:var(--text-muted);font-size:0.8rem}
|
||
footer{background:#0f172a;color:#94a3b8;padding:28px 0;text-align:center;font-size:0.8rem}
|
||
footer a{color:#94a3b8}
|
||
.dl-grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(200px,1fr));gap:12px;margin:18px 0}
|
||
.dl-card{border:1px solid var(--border);border-radius:10px;padding:16px 18px}
|
||
.dl-card b{color:var(--text)}
|
||
.hero{text-align:center;padding:20px 0 8px}
|
||
.hero .tags{color:var(--text-muted);font-size:0.9rem;margin-top:8px}
|
||
@media(max-width:900px){.nav-links{display:none}.nav .container{gap:16px}.nav-logo{margin-right:auto}h1{font-size:1.75rem}table{table-layout:fixed;font-size:0.78rem}th,td{padding:8px 4px;overflow-wrap:anywhere}p code,li code{overflow-wrap:anywhere;word-break:break-word}}
|
||
</style>
|
||
</head>
|
||
<body>
|
||
<nav class="nav"><div class="container"><a href="/en/" class="nav-logo">Fun<span>ASR</span></a><div class="nav-links"><a href="/en/#capabilities">Capabilities</a><a href="/en/#usage">How to Use</a><a href="/en/#benchmark">Benchmark</a><a href="/en/#demo">Demo</a><a href="https://modelscope.github.io/FunASR/" target="_blank">Docs</a><a href="/en/quickstart.html">Quickstart</a><a href="/en/models.html">Models</a><a href="/en/ecosystem.html">Ecosystem</a><a href="/en/blog/">Blog</a><a href="/en/vs-whisper.html">vs Whisper</a><a href="/en/donors.html">Thanks</a></div><a href="https://www.funasr.com/llama-cpp.html" style="font-size:0.83rem;font-weight:500;color:var(--text-soft)">中文</a><a href="https://github.com/modelscope/FunASR" target="_blank" class="nav-btn">GitHub ★</a></div></nav>
|
||
<article><div class="container">
|
||
<div class="hero"><h1>Run FunASR on CPU — the llama.cpp / GGUF Runtime</h1>
|
||
<p class="tags">Single binary · No Python · No GPU · Quantized weights · whisper.cpp-style on-device ASR, strong on Chinese</p></div>
|
||
|
||
<p><strong>FunASR on llama.cpp</strong> is to FunASR what <a href="https://github.com/ggml-org/whisper.cpp" target="_blank">whisper.cpp</a> is to Whisper: it runs SenseVoice / Paraformer / Fun-ASR-Nano on the <a href="https://github.com/ggml-org/llama.cpp" target="_blank">ggml</a> stack, so the models work <strong>where there is no GPU and no Python</strong> (laptops, edge boxes, embedded C/C++ apps), complementing the PyTorch / vLLM paths for GPU serving. FSMN-VAD is built into the binaries.</p>
|
||
|
||
<h2>Download prebuilt binaries (download & run)</h2>
|
||
<p>Linux (x64/arm64), macOS (arm64), Windows (x64) — static, self-contained, zero dependencies. Available in all three repos' Releases:</p>
|
||
<div class="dl-grid">
|
||
<a class="dl-card" href="https://github.com/modelscope/FunASR/releases" target="_blank"><b>⬇ modelscope/FunASR</b><br><span style="color:var(--text-soft);font-size:0.88rem">all three models</span></a>
|
||
<a class="dl-card" href="https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.9" target="_blank"><b>⬇ Current runtime v0.1.8</b><br><span style="color:var(--text-soft);font-size:0.88rem">Linux/Windows Vulkan + Windows CUDA + 9 platform assets</span></a>
|
||
<a class="dl-card" href="https://github.com/modelscope/FunASR/releases/tag/v1.3.26" target="_blank"><b>Python package v1.3.26</b><br><span style="color:var(--text-soft);font-size:0.88rem">--hub ms fix + runtime download table</span></a>
|
||
<a class="dl-card" href="https://github.com/QwenAudio/SenseVoice/releases" target="_blank"><b>⬇ QwenAudio/SenseVoice</b><br><span style="color:var(--text-soft);font-size:0.88rem">multilingual + emotion/events</span></a>
|
||
<a class="dl-card" href="https://github.com/QwenAudio/Fun-ASR/releases" target="_blank"><b>⬇ QwenAudio/Fun-ASR</b><br><span style="color:var(--text-soft);font-size:0.88rem">Fun-ASR-Nano (LLM-ASR)</span></a>
|
||
</div>
|
||
<p>Recommended download entry today: <a href="https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.9" target="_blank">runtime-llamacpp-v0.1.9</a>. Common accelerated builds include <a href="https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.9/funasr-llamacpp-linux-x64-vulkan.tar.gz" target="_blank">funasr-llamacpp-linux-x64-vulkan.tar.gz</a>, <a href="https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.9/funasr-llamacpp-windows-x64-vulkan.zip" target="_blank">funasr-llamacpp-windows-x64-vulkan.zip</a>, and <a href="https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.9/funasr-llamacpp-windows-x64-cuda.zip" target="_blank">funasr-llamacpp-windows-x64-cuda.zip</a>. Python users can install <code>python -m pip install -U "funasr==1.3.26"</code>; the <a href="https://github.com/modelscope/FunASR/releases/tag/v1.3.26" target="_blank">v1.3.26 Latest Release</a> also lists the same runtime download links and SHA-256 checksums.</p>
|
||
|
||
<h2>Three lines to run</h2>
|
||
<pre># 1) Unpack the binaries, fetch a model (downloads GGUF + VAD)
|
||
bash download-funasr-model.sh sensevoice ./gguf
|
||
|
||
# 2) Get text directly (in-binary detok, no Python)
|
||
llama-funasr-sensevoice -m ./gguf/SenseVoiceSmall-f16.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav</pre>
|
||
<p>Other models: <code>download-funasr-model.sh paraformer</code> with <code>llama-funasr-paraformer</code>; <code>download-funasr-model.sh nano</code> with <code>llama-funasr-cli</code> (Fun-ASR-Nano for zh/en/ja + Chinese dialects/accents). The 31-language scope belongs to the separate MLT-Nano checkpoint.</p>
|
||
|
||
<h2>Accuracy: far ahead of whisper.cpp on Chinese</h2>
|
||
<p>Same 184-clip Chinese test set, character error rate (CER, micro-avg, lower is better):</p>
|
||
<table>
|
||
<tr><th>Model</th><th>CER ↓</th><th>Notes</th></tr>
|
||
<tr><td><strong>FunASR SenseVoice</strong></td><td><strong>8.01%</strong></td><td>multilingual + emotion/events</td></tr>
|
||
<tr><td><strong>FunASR Paraformer</strong></td><td><strong>9.85%</strong></td><td>non-autoregressive, industrial Chinese</td></tr>
|
||
<tr><td><strong>FunASR Fun-ASR-Nano</strong></td><td><strong>8.30%</strong></td><td>LLM-ASR, zh/en/ja + Chinese dialects/accents</td></tr>
|
||
<tr><td>whisper.cpp small</td><td>22.12%</td><td></td></tr>
|
||
<tr><td>whisper.cpp large-v3-turbo</td><td>23.15%</td><td></td></tr>
|
||
<tr><td>whisper.cpp base</td><td>31.33%</td><td></td></tr>
|
||
</table>
|
||
<p>FunASR's Chinese CER is roughly a third of whisper.cpp's. Full methodology in each repo's <a href="https://github.com/modelscope/FunASR/blob/main/runtime/llama.cpp/BENCHMARKS.md" target="_blank">runtime/llama.cpp/BENCHMARKS.md</a>.</p>
|
||
|
||
<h2>Model sizes & accuracy (quantized)</h2>
|
||
<p>Quantized builds are smaller and faster with virtually unchanged accuracy (184-file micro-CER):</p>
|
||
<table>
|
||
<tr><th>Model</th><th>Type</th><th>Size</th><th>CER ↓</th><th>Speed</th></tr>
|
||
<tr><td>SenseVoice encoder</td><td>f16</td><td>470 MB</td><td>8.01%</td><td>23×</td></tr>
|
||
<tr><td>SenseVoice encoder</td><td><strong>q8</strong></td><td><strong>254 MB</strong></td><td>7.99%</td><td>27×</td></tr>
|
||
<tr><td>Paraformer encoder</td><td>f16</td><td>435 MB</td><td>9.85%</td><td>22×</td></tr>
|
||
<tr><td>Paraformer encoder</td><td><strong>q8</strong></td><td><strong>237 MB</strong></td><td>9.78%</td><td>27×</td></tr>
|
||
<tr><td>Fun-ASR-Nano LLM*</td><td>q8_0</td><td>805 MB</td><td>8.30%</td><td>6.0×</td></tr>
|
||
<tr><td>Fun-ASR-Nano LLM*</td><td><strong>q5_K_M</strong></td><td>551 MB</td><td><strong>8.25%</strong></td><td>5.7×</td></tr>
|
||
<tr><td>Fun-ASR-Nano LLM*</td><td><strong>q4_K_M</strong></td><td><strong>484 MB</strong></td><td>8.35%</td><td>6.1×</td></tr>
|
||
</table>
|
||
<p style="font-size:0.9rem">* Fun-ASR-Nano also needs the encoder (f16, 470 MB). The q8 encoder halves size while CER even drops slightly; the three Nano LLM tiers are nearly identical — pick <strong>q4_K_M</strong> (smallest) or <strong>q5_K_M</strong> (best).</p>
|
||
|
||
<h2>What's included</h2>
|
||
<ul>
|
||
<li>6 binaries: <code>llama-funasr-{cli,encoder,embd,sensevoice,paraformer,vad}</code>, static, no .so dependencies.</li>
|
||
<li>Built-in FSMN-VAD (<code>--vad</code>), in-binary detokenization (prints text), kaldi-compatible fbank front end.</li>
|
||
<li>GGUF models on Hugging Face: <a href="https://huggingface.co/FunAudioLLM" target="_blank">FunAudioLLM</a> / <a href="https://huggingface.co/funasr" target="_blank">funasr</a> (f16 / f32 / <strong>q8</strong> with embedded vocab; <strong>q8 is ~half the size of f16 with the same accuracy</strong>).</li>
|
||
<li>Source & docs: <a href="https://github.com/modelscope/FunASR/tree/main/runtime/llama.cpp" target="_blank">runtime/llama.cpp/</a> (README / DESIGN / BENCHMARKS).</li>
|
||
</ul>
|
||
|
||
<div class="cta"><p><strong>If it helps, a GitHub Star really supports the project 👇</strong> Fully open-source, commercial-friendly.</p>
|
||
<p><a class="btn" href="https://github.com/modelscope/FunASR" target="_blank">⭐ Star FunASR</a></p>
|
||
<p style="margin-top:10px;font-size:0.85rem">Also star: <a href="https://github.com/QwenAudio/SenseVoice" target="_blank">SenseVoice</a> · <a href="https://github.com/QwenAudio/Fun-ASR" target="_blank">Fun-ASR</a> · <a href="https://github.com/modelscope/FunClip" target="_blank">FunClip</a></p></div>
|
||
<p style="font-size:0.9rem">Further reading: <a href="/en/blog/funasr-llama-cpp-whisper-cpp-alternative.html">FunASR on llama.cpp (a whisper.cpp alternative) — deep dive</a></p>
|
||
</div></article>
|
||
<footer><div class="container">© 2024-2026 Tongyi Lab, Alibaba Group · <a href="https://github.com/modelscope/FunASR/blob/main/LICENSE">MIT License</a></div></footer>
|
||
</body>
|
||
</html>
|