modelscope/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR.git synced 2026-10-12 02:19:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

revert: remove incorrect VAD finetune docs and script

Browse Source 
FsmnVADStreaming.forward() is inference-only (no loss computation),
so standard train_ds.py training does not work. Removing the
finetune.sh and docs that were incorrectly added.

The parameter tuning guide (vad_kwargs) remains valid and is
documented in the issue thread (#2961).
This commit is contained in:
游雁
2026-06-10 03:46:57 +00:00
parent 0ccdce52a2
commit 1065d2e5ab
5 changed files with 2 additions and 284 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -21,7 +21,7 @@
</p>
<p align="center">
<a href="#quick-start">Quick Start</a> · <a href="./examples/colab/">Colab</a> · <a href="#benchmark">Benchmark</a> · <a href="./docs/model_selection.md">Model selection</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="./docs/finetune_vad.md">VAD fine-tuning</a> · <a href="#model-zoo">Models</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent Integration</a> · <a href="https://modelscope.github.io/FunASR/">Docs</a> · <a href="./CONTRIBUTING.md">Contribute</a>
<a href="#quick-start">Quick Start</a> · <a href="./examples/colab/">Colab</a> · <a href="#benchmark">Benchmark</a> · <a href="./docs/model_selection.md">Model selection</a> · <a href="./docs/migration_from_whisper.md">Migration guide</a> · <a href="./docs/use_case_showcase.md">Use cases</a> · <a href="./docs/deployment_matrix.md">Deployment matrix</a> · <a href="#model-zoo">Models</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent Integration</a> · <a href="https://modelscope.github.io/FunASR/">Docs</a> · <a href="./CONTRIBUTING.md">Contribute</a>
</p>
---
README_zh.md
+1 -1
View File
@@ -21,7 +21,7 @@
</p>
<p align="center">
<a href="#快速开始">快速开始</a> · <a href="./examples/colab/README_zh.md">Colab</a> · <a href="#性能评测">性能评测</a> · <a href="./docs/model_selection_zh.md">模型选择</a> · <a href="./docs/migration_from_whisper_zh.md">迁移指南</a> · <a href="./docs/use_case_showcase_zh.md">场景速览</a> · <a href="./docs/deployment_matrix_zh.md">部署选型</a> · <a href="./docs/finetune_vad_zh.md">VAD微调</a> · <a href="#模型列表">模型列表</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 集成</a> · <a href="https://modelscope.github.io/FunASR/zh/">文档</a> · <a href="./CONTRIBUTING.md">贡献</a>
<a href="#快速开始">快速开始</a> · <a href="./examples/colab/README_zh.md">Colab</a> · <a href="#性能评测">性能评测</a> · <a href="./docs/model_selection_zh.md">模型选择</a> · <a href="./docs/migration_from_whisper_zh.md">迁移指南</a> · <a href="./docs/use_case_showcase_zh.md">场景速览</a> · <a href="./docs/deployment_matrix_zh.md">部署选型</a> · <a href="#模型列表">模型列表</a> · <a href="https://modelscope.github.io/FunASR/agent.html">Agent 集成</a> · <a href="https://modelscope.github.io/FunASR/zh/">文档</a> · <a href="./CONTRIBUTING.md">贡献</a>
</p>
---
docs/finetune_vad.md
-109
View File
@@ -1,109 +0,0 @@
# Fine-tuning FSMN-VAD
This guide shows how to fine-tune the FSMN-VAD (Voice Activity Detection) model on your own data. Fine-tuning is recommended when the default model does not perform well on your specific audio domain (e.g., noisy environments, walkie-talkie audio, call center recordings).
## When to Fine-tune
- High miss rate (speech segments not detected)
- High false alarm rate (non-speech detected as speech)
- Domain-specific audio that differs significantly from the training data (e.g., 8kHz telephony, high-noise industrial environments)
If you only need minor adjustments, try [parameter tuning](#parameter-tuning-without-fine-tuning) first.
## Data Preparation
### Required files
1. **wav.scp** — Audio file list
2. **vad.txt** — VAD labels (speech segment timestamps)
### wav.scp format
```
utt001 /path/to/audio1.wav
utt002 /path/to/audio2.wav
utt003 /path/to/audio3.wav
```
### vad.txt format
Each line contains the utterance ID followed by a list of speech segments `[start_ms, end_ms]`:
```
utt001 [[320, 2150], [3800, 6420], [8100, 12500]]
utt002 [[0, 5000], [6200, 9800]]
utt003 [[100, 3000]]
```
Prepare separate `train_wav.scp` / `train_vad.txt` and `val_wav.scp` / `val_vad.txt` for training and validation.
### Recommended data size
- **Minimum**: 50-100 labeled audio files
- **Recommended**: 500+ for robust fine-tuning
- Include both positive (speech) and negative (silence/noise) examples representative of your target domain
## Fine-tuning
### Step 1: Place data
```
data/list/
├── train_wav.scp
├── train_vad.txt
├── val_wav.scp
└── val_vad.txt
```
### Step 2: Run fine-tuning
```bash
cd examples/industrial_data_pretraining/fsmn_vad_streaming
bash finetune.sh
```
Key parameters in `finetune.sh`:
| Parameter | Default | Description |
|-----------|---------|-------------|
| `model_name_or_model_dir` | `iic/speech_fsmn_vad_zh-cn-16k-common-pytorch` | Base model (use `*-8k-*` for 8kHz audio) |
| `train_conf.max_epoch` | 20 | Training epochs |
| `optim_conf.lr` | 0.00005 | Learning rate (lower = more conservative) |
| `train_conf.validate_interval` | 1000 | Validation frequency (steps) |
### Step 3: Use the fine-tuned model
```python
from funasr import AutoModel
model = AutoModel(model="/path/to/outputs/model_dir")
result = model.generate(input="test.wav")
```
## Parameter Tuning (without fine-tuning)
For minor adjustments, pass VAD parameters directly:
```python
from funasr import AutoModel
model = AutoModel(
model="paraformer-zh",
vad_model="iic/speech_fsmn_vad_zh-cn-16k-common-pytorch",
vad_kwargs={
"max_end_silence_time": 500, # Max silence within a sentence (ms)
"speech_noise_thres": 0.5, # Speech/noise threshold (lower = more sensitive)
"max_single_segment_time": 60000, # Max segment duration (ms)
}
)
```
### Key parameters
| Parameter | Default | Effect |
|-----------|---------|--------|
| `max_end_silence_time` | 800 | Increase to tolerate longer pauses within sentences |
| `speech_noise_thres` | 0.6 | Decrease to detect more speech (may increase false alarms) |
| `max_single_segment_time` | 60000 | Max duration of a single speech segment |
| `speech_2_noise_ratio` | 1.0 | Decrease to be more sensitive to speech |
| `max_start_silence_time` | 3000 | Max leading silence before speech starts |
docs/finetune_vad_zh.md
-107
View File
@@ -1,107 +0,0 @@
# FSMN-VAD 模型微调指南
本文介绍如何在自定义数据上微调 FSMN-VAD（语音活动检测）模型。当默认模型在特定音频场景（如高噪声环境、对讲机音频、电话录音等）表现不佳时，建议进行微调。
## 何时需要微调
- 漏检率高（语音片段未被检测到）
- 误检率高（非语音被误判为语音）
- 音频域与训练数据差异大（如 8kHz 电话音频、工业噪声环境）
如果只需要小幅调整，可以先尝试[参数调优](#参数调优无需微调)。
## 数据准备
### 所需文件
1. **wav.scp** — 音频文件列表
2. **vad.txt** — VAD 标注（语音段时间戳）
### wav.scp 格式
```
utt001 /path/to/audio1.wav
utt002 /path/to/audio2.wav
```
### vad.txt 格式
每行包含语音 ID 和语音段列表 `[起始毫秒, 结束毫秒]`：
```
utt001 [[320, 2150], [3800, 6420], [8100, 12500]]
utt002 [[0, 5000], [6200, 9800]]
```
分别准备训练集（`train_wav.scp` / `train_vad.txt`）和验证集（`val_wav.scp` / `val_vad.txt`）。
### 建议数据量
- **最少**: 50-100 条标注音频
- **推荐**: 500+ 条，效果更稳定
- 应包含目标场景的典型音频（语音和静音/噪声样本）
## 微调步骤
### 第一步：放置数据
```
data/list/
├── train_wav.scp
├── train_vad.txt
├── val_wav.scp
└── val_vad.txt
```
### 第二步：运行微调
```bash
cd examples/industrial_data_pretraining/fsmn_vad_streaming
bash finetune.sh
```
`finetune.sh` 中的关键参数：
| 参数 | 默认值 | 说明 |
|------|--------|------|
| `model_name_or_model_dir` | `iic/speech_fsmn_vad_zh-cn-16k-common-pytorch` | 基础模型（8kHz 音频用 `*-8k-*`） |
| `train_conf.max_epoch` | 20 | 训练轮数 |
| `optim_conf.lr` | 0.00005 | 学习率（越低越保守） |
| `train_conf.validate_interval` | 1000 | 验证频率（步数） |
### 第三步：使用微调后的模型
```python
from funasr import AutoModel
model = AutoModel(model="/path/to/outputs/model_dir")
result = model.generate(input="test.wav")
```
## 参数调优（无需微调）
对于小幅调整，可以直接传入 VAD 参数：
```python
from funasr import AutoModel
model = AutoModel(
model="paraformer-zh",
vad_model="iic/speech_fsmn_vad_zh-cn-16k-common-pytorch",
vad_kwargs={
"max_end_silence_time": 500, # 句内最大静音时长(ms)
"speech_noise_thres": 0.5, # 语音/噪声阈值（越低越敏感）
"max_single_segment_time": 60000, # 单段最大时长(ms)
}
)
```
### 常用参数说明
| 参数 | 默认值 | 作用 |
|------|--------|------|
| `max_end_silence_time` | 800 | 增大可容忍句内更长的停顿 |
| `speech_noise_thres` | 0.6 | 减小可检测更多语音（可能增加误检） |
| `max_single_segment_time` | 60000 | 单个语音段的最大时长 |
| `speech_2_noise_ratio` | 1.0 | 减小可提高对语音的敏感度 |
| `max_start_silence_time` | 3000 | 起始段最大静音时长 |
examples/industrial_data_pretraining/fsmn_vad_streaming/finetune.sh
-66
View File
@@ -1,66 +0,0 @@
#!/bin/bash
# Copyright FunASR (https://github.com/modelscope/FunASR). All Rights Reserved.
# MIT License (https://opensource.org/licenses/MIT)
#
# Fine-tune FSMN-VAD model on custom data.
# Usage: bash finetune.sh
workspace=`pwd`
export CUDA_VISIBLE_DEVICES="0"
gpu_num=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
# Model: choose 16k or 8k version
model_name_or_model_dir="iic/speech_fsmn_vad_zh-cn-16k-common-pytorch"
# For 8k audio: model_name_or_model_dir="iic/speech_fsmn_vad_zh-cn-8k-common"
# Data directory
data_dir="../../../data/list"
train_data="${data_dir}/train.jsonl"
val_data="${data_dir}/val.jsonl"
# Generate train.jsonl from wav.scp and vad.txt
# wav.scp format: utt_id /path/to/audio.wav
# vad.txt format: utt_id [[start_ms, end_ms], [start_ms, end_ms], ...]
scp2jsonl \
++scp_file_list='["../../../data/list/train_wav.scp", "../../../data/list/train_vad.txt"]' \
++data_type_list='["source", "target"]' \
++jsonl_file_out="${train_data}"
scp2jsonl \
++scp_file_list='["../../../data/list/val_wav.scp", "../../../data/list/val_vad.txt"]' \
++data_type_list='["source", "target"]' \
++jsonl_file_out="${val_data}"
output_dir="./outputs"
log_file="${output_dir}/log.txt"
mkdir -p ${output_dir}
torchrun \
--nnodes 1 \
--nproc_per_node $gpu_num \
../../../funasr/bin/train_ds.py \
++model="${model_name_or_model_dir}" \
++train_data_set_list="${train_data}" \
++valid_data_set_list="${val_data}" \
++dataset="AudioDataset" \
++dataset_conf.index_ds="IndexDSJsonl" \
++dataset_conf.data_split_num=1 \
++dataset_conf.batch_sampler="BatchSampler" \
++dataset_conf.batch_size=6000 \
++dataset_conf.sort_size=1024 \
++dataset_conf.batch_type="token" \
++dataset_conf.num_workers=4 \
++train_conf.max_epoch=20 \
++train_conf.log_interval=50 \
++train_conf.resume=true \
++train_conf.validate_interval=1000 \
++train_conf.save_checkpoint_interval=1000 \
++train_conf.keep_nbest_models=5 \
++train_conf.avg_nbest_model=3 \
++train_conf.use_deepspeed=false \
++optim_conf.lr=0.00005 \
++scheduler="warmuplr" \
++scheduler_conf.warmup_steps=1000 \
++output_dir="${output_dir}" &> ${log_file}