FunASR/tutorial.html

<!DOCTYPE html>
<html lang="en"><head><meta charset="UTF-8"><meta name="viewport" content="width=device-width,initial-scale=1.0">
<title>FunASR Tutorial</title>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
<link rel="stylesheet" href="style.css">
</head><body>
<nav class="nav"><div class="container">
<a href="index.html" class="nav-logo">FunASR</a>
            <div class="nav-links">
                <a href="index.html">Home</a>
                <a href="tutorial.html" class="active">Tutorial</a>
                <a href="training.html">Training</a>
                <a href="model-registration.html">Develop</a>
                <a href="api.html">API</a>
            </div>
<div class="lang-switch">
<a href="tutorial.html" title="English">EN</a>
<a href="zh/tutorial.html" title="中文">中</a>
<a href="ja/tutorial.html" title="日本語">JP</a>
</div>
<a href="https://github.com/modelscope/FunASR" class="nav-github">GitHub</a>
</div></nav>

<div class="content"><div class="container narrow">
<h1>Tutorial</h1>
<p>From zero to production — learn FunASR in 10 minutes.</p>

<div class="toc-grid">
<a href="#quick">30-Second Demo</a>
<a href="#install">Install</a>
<a href="#choose">Choose a Model</a>
<a href="#scenarios">Usage Scenarios</a>
<a href="#offline">Offline ASR</a>
<a href="#streaming">Streaming ASR</a>
<a href="#spk">Speaker Diarization</a>
<a href="#emotion">Emotion Detection</a>
<a href="#vad">VAD</a>
<a href="#punc">Punctuation</a>
<a href="#export">ONNX Export</a>
<a href="#faq">FAQ & Troubleshooting</a>
</div>

<!-- 30-Second Demo -->
<h2 id="quick">30-Second Demo</h2>
<p>Copy-paste this and run. No config needed — FunASR downloads everything automatically:</p>
<pre>pip install funasr

python -c "
from funasr import AutoModel
model = AutoModel(model='paraformer-zh', vad_model='fsmn-vad', punc_model='ct-punc')
res = model.generate(input='https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav')
print(res[0]['text'])
"</pre>
<p><strong>Output:</strong></p>
<div class="output-box">欢迎大家来体验达摩院推出的语音识别模型。</div>

<p>That's it. One model object handles everything: loading audio, VAD segmentation, ASR, and punctuation.</p>

<!-- Install -->
<h2 id="install">Install</h2>
<pre># Stable release
pip install funasr

# Latest (recommended — has newest models and bug fixes)
pip install git+https://github.com/modelscope/FunASR.git</pre>

<div class="tip"><strong>China users:</strong> Models download from ModelScope by default (fast in China). International users: add <code>hub="hf"</code> to download from HuggingFace.</div>

<!-- Choose a Model -->
<h2 id="choose">Which Model Should I Use?</h2>

<table>
<tr><th>Model</th><th>Best For</th><th>Languages</th><th>Speed</th><th>Punctuation</th></tr>
<tr><td><strong>Paraformer</strong></td><td>Chinese production ASR</td><td>Chinese, English</td><td>Fast</td><td>Needs punc_model</td></tr>
<tr><td><strong>Fun-ASR-Nano</strong></td><td>Multi-language, dialects, lyrics</td><td>31 languages</td><td>Medium</td><td>Built-in ✓</td></tr>
<tr><td><strong>SenseVoice</strong></td><td>Emotion + events + ASR</td><td>5 languages</td><td>Ultra-fast (70ms/10s)</td><td>Built-in ✓</td></tr>
<tr><td><strong>Qwen3-ASR</strong></td><td>Highest accuracy, context-aware</td><td>52 languages</td><td>Slow (LLM)</td><td>Built-in ✓</td></tr>
<tr><td><strong>Paraformer-Streaming</strong></td><td>Real-time transcription</td><td>Chinese</td><td>Real-time</td><td>Needs punc_model</td></tr>
</table>

<div class="note">
<strong>Quick recommendation:</strong><br>
• Chinese meeting/call → <code>paraformer-zh</code> + <code>fsmn-vad</code> + <code>ct-punc</code> + <code>cam++</code><br>
• Multi-language → <code>Fun-ASR-Nano</code><br>
• Need emotion/sound events → <code>SenseVoice</code><br>
• Real-time subtitles → <code>paraformer-zh-streaming</code><br>
• Highest quality, no latency concern → <code>Qwen3-ASR</code>
</div>

<!-- Scenarios -->
<h2 id="scenarios">Usage Scenarios</h2>

<div class="scenario-card">
<h4>🎤 "I have a meeting recording and want text + who said what"</h4>
<p>→ Paraformer + VAD + Punctuation + Speaker Diarization → <a href="#spk">Jump to Speaker Diarization</a></p>
</div>

<div class="scenario-card">
<h4>📺 "I want real-time subtitles for a live stream"</h4>
<p>→ Paraformer-Streaming, feed audio chunks every 600ms → <a href="#streaming">Jump to Streaming ASR</a></p>
</div>

<div class="scenario-card">
<h4>😊 "I want to detect user emotions from voice"</h4>
<p>→ SenseVoice (outputs emotion tags: happy, sad, angry, neutral) → <a href="#emotion">Jump to Emotion Detection</a></p>
</div>

<div class="scenario-card">
<h4>🌍 "I have audio in Japanese/Korean/Arabic/etc."</h4>
<p>→ Fun-ASR-Nano (31 languages) or Qwen3-ASR (52 languages) → <a href="#offline">Jump to Offline ASR</a></p>
</div>

<div class="scenario-card">
<h4>✂️ "I want to clip a video by spoken content"</h4>
<p>→ Use <a href="https://github.com/modelscope/FunClip">FunClip</a> which integrates FunASR for smart video editing</p>
</div>

<!-- Offline ASR -->
<h2 id="offline">Offline ASR</h2>

<h3>Paraformer (Chinese)</h3>
<pre>from funasr import AutoModel

model = AutoModel(
    model="paraformer-zh",          # Chinese ASR
    vad_model="fsmn-vad",           # handles any audio length
    vad_kwargs={"max_single_segment_time": 60000},
    punc_model="ct-punc",           # adds punctuation
)
res = model.generate(input="meeting.wav", batch_size_s=300, hotword='达摩院 语音识别')
print(res[0]["text"])       # "欢迎大家来体验达摩院推出的语音识别模型。"
print(res[0]["timestamp"])  # [[880,1120],[1120,1360],...] (ms per character)</pre>

<h3>Fun-ASR-Nano (31 Languages)</h3>
<pre>from funasr import AutoModel

model = AutoModel(
    model="FunAudioLLM/Fun-ASR-Nano-2512",
    trust_remote_code=True,
    remote_code="./model.py",
    vad_model="fsmn-vad",
    vad_kwargs={"max_single_segment_time": 30000},
    device="cuda:0",
    hub="hf",
)
res = model.generate(input=["audio.wav"], cache={}, batch_size=1,
                     hotwords=["keyword"], language="中文")
print(res[0]["text"])        # recognized text with punctuation
print(res[0]["timestamps"])  # [{"token":"开","start_time":0.42,"end_time":0.48}, ...]</pre>

<div class="note"><strong>Fun-ASR-Nano outputs punctuation natively</strong> — no need for <code>punc_model</code>.</div>

<h3>SenseVoice (ASR + Emotion + Events)</h3>
<pre>from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model = AutoModel(
    model="iic/SenseVoiceSmall",
    vad_model="fsmn-vad",
    vad_kwargs={"max_single_segment_time": 30000},
    device="cuda:0",
)
res = model.generate(input="audio.wav", cache={}, language="auto",
                     use_itn=True, batch_size_s=60, merge_vad=True, merge_length_s=15)

# Raw output contains emotion/event tags: <|zh|><|HAPPY|><|Speech|>你好
text = rich_transcription_postprocess(res[0]["text"])
print(text)  # "你好" (clean text)</pre>

<h3>Qwen3-ASR (52 Languages, Highest Accuracy)</h3>
<pre># pip install qwen-asr
from funasr import AutoModel

model = AutoModel(model="Qwen/Qwen3-ASR-1.7B", hub="hf", device="cuda:0")
res = model.generate(input="audio.wav")
print(res[0]["text"])             # recognized text
print(res[0].get("language"))    # auto-detected language</pre>

<!-- Streaming -->
<h2 id="streaming">Streaming ASR (Real-time)</h2>
<p>Process audio chunk-by-chunk for real-time transcription. Each chunk produces partial text immediately.</p>

<pre>from funasr import AutoModel
import soundfile

model = AutoModel(model="paraformer-zh-streaming")

speech, sr = soundfile.read("audio.wav")
chunk_size = [0, 10, 5]          # 600ms display, 300ms lookahead
chunk_stride = chunk_size[1] * 960  # 9600 samples per chunk

cache = {}
total_chunks = int((len(speech) - 1) / chunk_stride + 1)
for i in range(total_chunks):
    chunk = speech[i * chunk_stride:(i + 1) * chunk_stride]
    is_final = (i == total_chunks - 1)

    res = model.generate(input=chunk, cache=cache, is_final=is_final,
                         chunk_size=chunk_size,
                         encoder_chunk_look_back=4,
                         decoder_chunk_look_back=1)
    if res[0]["text"]:
        print(res[0]["text"], end="", flush=True)  # incremental output</pre>

<p><strong>Output (printed incrementally):</strong></p>
<div class="output-box">欢迎大 | 家来 | 体验达 | 摩院推 | 出的语 | 音识 | 别模型</div>

<div class="note">
<strong>Key points:</strong><br>
• <code>cache={}</code> must persist across all chunks (don't recreate it)<br>
• <code>is_final=True</code> on last chunk flushes remaining buffered text<br>
• <code>chunk_size=[0,10,5]</code>: first number unused, second=display granularity (×60ms), third=lookahead (×60ms)
</div>

<!-- Speaker Diarization -->
<h2 id="spk">Speaker Diarization ("Who Said What")</h2>
<p>Works with <strong>all three major models</strong>: Paraformer, Fun-ASR-Nano, and SenseVoice. Add <code>spk_model="cam++"</code> to get speaker labels per sentence.</p>

<h3>Paraformer + Speaker</h3>
<pre>from funasr import AutoModel

model = AutoModel(
    model="paraformer-zh",
    vad_model="fsmn-vad",
    punc_model="ct-punc",      # Paraformer needs this for sentence segmentation
    spk_model="cam++",
)
res = model.generate(input="meeting.wav", batch_size_s=300)

for sent in res[0]["sentence_info"]:
    print(f"[Speaker {sent['spk']}] [{sent['start']}-{sent['end']}ms] {sent['text']}")</pre>

<p><strong>Output:</strong></p>
<div class="output-box">[Speaker 0] [880-5195ms] 欢迎大家来体验达摩院推出的语音识别模型。</div>

<h3>Fun-ASR-Nano + Speaker (no punc needed)</h3>
<pre>model = AutoModel(
    model="FunAudioLLM/Fun-ASR-Nano-2512",
    trust_remote_code=True, remote_code="./model.py",
    vad_model="fsmn-vad", vad_kwargs={"max_single_segment_time": 30000},
    spk_model="cam++",     # no punc_model needed!
    device="cuda:0", hub="hf",
)
res = model.generate(input=["meeting.wav"], cache={}, batch_size=1, language="中文")
for sent in res[0]["sentence_info"]:
    print(f"Speaker {sent['spk']}: {sent.get('text', sent.get('sentence', ''))}")</pre>

<h3>SenseVoice + Speaker (no punc needed)</h3>
<pre>model = AutoModel(
    model="iic/SenseVoiceSmall",
    vad_model="fsmn-vad", vad_kwargs={"max_single_segment_time": 30000},
    spk_model="cam++",    # no punc_model needed!
    device="cuda:0",
)
res = model.generate(input="meeting.wav", cache={}, language="auto",
                     use_itn=True, batch_size_s=60, merge_vad=True, merge_length_s=15)
for sent in res[0]["sentence_info"]:
    print(f"Speaker {sent['spk']}: {rich_transcription_postprocess(sent['text'])}")</pre>

<!-- Emotion -->
<h2 id="emotion">Emotion Detection</h2>

<h3>Dedicated Emotion Model (emotion2vec)</h3>
<pre>from funasr import AutoModel

model = AutoModel(model="iic/emotion2vec_plus_large", device="cuda:0")
res = model.generate(input="audio.wav", granularity="utterance")

print(res[0]["labels"])  # ['angry', 'happy', 'neutral', 'sad', ...]
print(res[0]["scores"])  # [0.01,   0.03,   0.89,      0.05, ...]
# → This audio is "neutral" with 89% confidence</pre>

<h3>SenseVoice (ASR + Emotion in one shot)</h3>
<p>SenseVoice embeds emotion tags directly in the transcription output:</p>
<pre># Raw output: "<|zh|><|HAPPY|><|Speech|><|withitn|>今天真是太开心了。"
# The <|HAPPY|> tag tells you the emotion
# Use rich_transcription_postprocess() to get clean text</pre>

<!-- VAD -->
<h2 id="vad">Voice Activity Detection</h2>

<h3>Offline (full audio)</h3>
<pre>from funasr import AutoModel

model = AutoModel(model="fsmn-vad")
res = model.generate(input="audio.wav")
print(res[0]["value"])  # [[610, 5530], [7200, 12400], ...]
                         # Each pair: [start_ms, end_ms] of speech</pre>

<h3>Streaming (chunk-by-chunk)</h3>
<pre>import soundfile
from funasr import AutoModel

model = AutoModel(model="fsmn-vad")
speech, sr = soundfile.read("audio.wav")
chunk_stride = int(200 * sr / 1000)  # 200ms chunks

cache = {}
for i in range(int((len(speech)-1)/chunk_stride+1)):
    chunk = speech[i*chunk_stride:(i+1)*chunk_stride]
    is_final = i == int((len(speech)-1)/chunk_stride)
    res = model.generate(input=chunk, cache=cache, is_final=is_final, chunk_size=200)
    if res[0]["value"]:
        print(res[0]["value"])
        # [[610, -1]]   → speech started at 610ms
        # [[-1, 5530]]  → speech ended at 5530ms
        # [[610, 5530]] → complete segment</pre>

<!-- Punctuation -->
<h2 id="punc">Punctuation Restoration</h2>
<pre>from funasr import AutoModel

model = AutoModel(model="ct-punc")
res = model.generate(input="那今天的会就到这里吧 happy new year 明年见")
print(res[0]["text"])  # "那今天的会就到这里吧，happy new year，明年见。"</pre>
<div class="note"><strong>When do you need this?</strong> Only with Paraformer (which outputs raw text without punctuation). Fun-ASR-Nano, SenseVoice, and Qwen3-ASR output punctuation natively.</div>

<!-- Export -->
<h2 id="export">ONNX Export</h2>
<pre># Export model to ONNX format
from funasr import AutoModel
model = AutoModel(model="paraformer", device="cpu")
model.export(quantize=False)   # saves to model cache directory

# Use ONNX model (faster, no PyTorch needed)
# pip install funasr-onnx
from funasr_onnx import Paraformer
model = Paraformer("damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch",
                   batch_size=1, quantize=True)
result = model(["audio.wav"])
print(result)</pre>

<!-- FAQ -->
<h2 id="faq">FAQ & Troubleshooting</h2>

<h4>Model download is slow</h4>
<p>Set <code>hub="hf"</code> for HuggingFace (faster outside China), or download once and use local path:</p>
<pre>model = AutoModel(model="/path/to/local/model", disable_update=True)</pre>

<h4>Out of Memory (OOM)</h4>
<p>Three knobs to reduce memory:</p>
<ol>
<li>Reduce <code>batch_size_s</code> (e.g., from 300 to 60)</li>
<li>Reduce <code>max_single_segment_time</code> in vad_kwargs (e.g., 30000 → 15000)</li>
<li>Add <code>batch_size_threshold_s=30</code> to force batch=1 for long segments</li>
</ol>

<h4>Every startup shows "Downloading Model..."</h4>
<p>Not actually re-downloading — just verifying cache. To skip entirely:</p>
<pre>model = AutoModel(model="/local/path/to/model", disable_update=True)</pre>

<h4>"ModelName is not registered"</h4>
<p>Usually means pypi version is outdated. Install from source:</p>
<pre>pip install git+https://github.com/modelscope/FunASR.git</pre>

<h4>Want to suppress all progress bars and logs?</h4>
<pre>model = AutoModel(model="...", disable_update=True, disable_pbar=True, log_level="ERROR")</pre>

<h4>How to pass pre-loaded numpy audio?</h4>
<pre>import soundfile as sf
audio, sr = sf.read("audio.wav")  # numpy array, 16kHz
res = model.generate(input=audio)  # pass directly — no file needed</pre>

<h4>GPU not being used?</h4>
<pre># Check:
import torch
print(torch.cuda.is_available())  # must be True

# Specify GPU explicitly:
model = AutoModel(model="...", device="cuda:0")</pre>

</div></div>

<footer>
<p>FunASR &middot; Tongyi Lab, Alibaba Group</p>
<p><a href="index.html">Home</a> &middot; <a href="tutorial.html">Tutorial</a> &middot; <a href="training.html">Training</a> &middot; <a href="model-registration.html">Develop</a> &middot; <a href="api.html">API</a> &middot; <a href="https://github.com/modelscope/FunASR">GitHub</a></p>
</footer>
</body></html>