72 Commits
Author SHA1 Message Date
LauraGPT a57cc3f92a fix(sensevoice): keep words separate at standalone boundaries
Signed-off-by: LauraGPT <[email protected]>
2026-09-29 06:12:59 +00:00
LauraGPT 2ddabb2c91 fix(sensevoice): normalize first word before joining subpieces
Signed-off-by: LauraGPT <[email protected]>
2026-09-29 05:58:08 +00:00
谢翊凡 9413d7d768 fix: use torch.amp via compatibility shim (#3518)
Use the non-deprecated torch.amp APIs when available while preserving the torch.cuda.amp fallback and legacy autocast argument semantics on older PyTorch releases.
2026-08-26 20:10:53 +08:00
liuweiqing 57e2f48c2c fix(tokenizer): treat special-token strings as text to prevent batch crashes (#3110)
Fixes #3110.
2026-07-07 14:42:36 +08:00
zhifu gao e79afd551b fix(sense_voice): remove docstring between @triton.jit and def (SyntaxError) (#3027) 2026-06-23 16:09:27 +08:00
游雁 78cc132a51 docs: add docstrings to 1950 functions across 276 files for API documentation 2026-05-21 07:52:52 +00:00
游雁 bff79be1a2 fix: add super().__init__() to SinusoidalPositionEncoder classes; add VAD streaming test
- Fix SinusoidalPositionEncoder and StreamSinusoidalPositionEncoder in transformer/embedding.py
- Fix SinusoidalPositionEncoder in sense_voice/model.py
- Missing super().__init__() caused state_dict() crash with newer PyTorch versions
- Add test_fsmn_vad_streaming.py: chunk-by-chunk VAD test with consistency check
2026-05-19 16:46:37 +00:00
游雁 22bb089e3a fix: correct __int__ typo to __init__ in SinusoidalPositionEncoder
Three classes had __int__ instead of __init__, causing the constructor
to never be called. Affected files:
- funasr/models/sense_voice/model.py (SinusoidalPositionEncoder)
- funasr/models/transformer/embedding.py (SinusoidalPositionEncoder)
- funasr/models/transformer/embedding.py (StreamSinusoidalPositionEncoder)
2026-05-19 15:06:25 +00:00
游雁 a2f3d5483d fix: resolve multiple issues (#2689, #2720, #2755, #2780)
- #2689: Add text_language check before removing spaces for aishell data
- #2720: Fix streaming ASR code example syntax error in README
- #2755: Use log_softmax instead of softmax for CTC forced alignment in SenseVoice
- #2780: Add bounds check in GetFrameState to prevent IndexError with non-default decibel_thres
2026-05-19 14:14:10 +00:00
彭震东and启焚 1bf72e7ff8 add whisper_lib for sensevoice (#2735)
Co-authored-by: 启焚 <[email protected]>
2025-12-09 17:08:08 +08:00
slin000111 88b464edb9 Fix prefix concatenation in timestamp post-processing for sensevoice (#2670)
* fix prefix concatenation in timestamp post-processing for sensevoice

* Remove unnecessary code
2025-09-20 22:40:30 +08:00
chengligen 8b0fb74bde feat: add 'words' key aligned with timestamps in sensevoice model output (#2531) 2025-05-26 14:11:33 +08:00
Isuxiz Slidder 3df109adfc Update model.py to fix "IndexError: index 1 is out of bounds for dimension 1 with size 0" (#2454)
* Update model.py

Avoid exception of "IndexError: index 1 is out of bounds for dimension 1 with size 0"

* Update model.py

Add return word in timestamps

* Revert "Update model.py"

This reverts commit bc736df302.
2025-03-31 17:51:52 +08:00
passerbya 5ee2f382b3 FIX 'NoneType' object has no attribute 'isalpha' (#2440)
Traceback (most recent call last):
  File "/root/miniconda3/envs/sensevoice/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/root/miniconda3/envs/sensevoice/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/data2/workspace/egs_vocal_extractor/data/speech_det.py", line 156, in process_audio_task
    res = model.generate(
  File "/root/miniconda3/envs/sensevoice/lib/python3.10/site-packages/funasr/auto/auto_model.py", line 306, in generate
    return self.inference_with_vad(input, input_len=input_len, **cfg)
  File "/root/miniconda3/envs/sensevoice/lib/python3.10/site-packages/funasr/auto/auto_model.py", line 464, in inference_with_vad
    results = self.inference(
  File "/root/miniconda3/envs/sensevoice/lib/python3.10/site-packages/funasr/auto/auto_model.py", line 345, in inference
    res = model.inference(**batch, **kwargs)
  File "/root/miniconda3/envs/sensevoice/lib/python3.10/site-packages/funasr/models/sense_voice/model.py", line 950, in inference
    timestamp = self.post(timestamp)
  File "/root/miniconda3/envs/sensevoice/lib/python3.10/site-packages/funasr/models/sense_voice/model.py", line 973, in post
    elif prev_word.isalpha() and prev_word.isascii() and word.isalpha() and word.isascii():
AttributeError: 'NoneType' object has no attribute 'isalpha'
2025-03-20 23:01:05 +08:00
Han Zhang 3c349ac053 fix: use converted token_ids for alignment for sensevoice model with timestamp output (#2429)
* fix: use converted token_ids for alignment

BPE doesn't guarantee converted ids (subwords) are revertible. which means `tokens` converted back is not always the same as `token_int`. A easy fix is to directly use the converted ids for alignment. Since they are from the same text, it shouldn't matter.

* fix: handle empty string

to index an empty string is to raise an exception. 这里没有判空。
2025-03-18 11:45:37 +08:00
hohaiuhsx 6fe10a8dbf 修复 当选用SenseVoice模型处理长音频(同时开启vad和output_timestamp)时的异常 (#2413) 2025-03-10 23:16:22 +08:00
zhifu gao 3f8294b9d7 Revert "shfit to shift (#2266)" (#2336)
This reverts commit 1367973f98.
2024-12-25 17:16:11 +08:00
Zhanzhao (Deo) Liang 8c7b7e5feb fix export_meta import of sense voice (#2334) 2024-12-25 16:40:29 +08:00
Rin Arakaki 1367973f98 shfit to shift (#2266) 2024-12-24 17:51:31 +08:00
游雁 bb0017a686 bugfix 2024-12-12 11:35:06 +08:00
shixian 026b8e3fdc update sensevoice small with timestamp 2024-12-05 19:29:19 +08:00
pointerhackerandzhaochaojin 70645e4807 数据并行可能导致的模型训练报错 (#2139)
* fix: 修复数据并行训练中ä¼可能会出现的错误

* fix: 修复数据并行训练中ä¼可能会出现的错误

* fix: 修复数据并行ènot need tensor

---------

Co-authored-by: zhaochaojin <[email protected]>
2024-10-15 17:50:51 +08:00
游雁 fc547e14e8 bugfix memory leaky 2024-09-25 15:26:14 +08:00
北念 8aa01036b6 fix sensevoice loss_rich 2024-07-24 19:09:02 +08:00
维石 2ae59b6ce0 ONNX and torchscript export for sensevoice 2024-07-22 16:58:27 +08:00
gaochangfengand常材 340c55838b EMO_UNK禁用和Merge VAD修复 (#1940)
* 添加富文本解码约束

* special token

* bug fix

* fix

* 增加unk score的参数

* emobaned

* kwargs2cfg

* merge_vad bug fix

---------

Co-authored-by: 常材 <[email protected]>
2024-07-22 15:28:27 +08:00
游雁 0170f534b0 sensevoice 2024-07-05 00:17:06 +08:00
游雁 92b14aaa2a update 2024-07-01 11:25:23 +08:00
游雁 a456ab57a8 update 2024-07-01 11:15:08 +08:00
+8 abb33d6b20 Dev gzf deepspeed (#1844)
* total_time/accum_grad

* fp16

* update with main (#1817)

* add cmakelist

* add paraformer-torch

* add debug for funasr-onnx-offline

* fix redefinition of jieba StdExtension.hpp

* add loading torch models

* update funasr-onnx-offline

* add SwitchArg for wss-server

* add SwitchArg for funasr-onnx-offline

* update cmakelist

* update funasr-onnx-offline-rtf

* add define condition

* add gpu define for offlne-stream

* update com define

* update offline-stream

* update cmakelist

* update func CompileHotwordEmbedding

* add timestamp for paraformer-torch

* add C10_USE_GLOG for paraformer-torch

* update paraformer-torch

* fix func FunASRWfstDecoderInit

* update model.h

* fix func FunASRWfstDecoderInit

* fix tpass_stream

* update paraformer-torch

* add bladedisc for funasr-onnx-offline

* update comdefine

* update funasr-wss-server

* add log for torch

* fix GetValue BLADEDISC

* fix log

* update cmakelist

* update warmup to 10

* update funasrruntime

* add batch_size for wss-server

* add batch for bins

* add batch for offline-stream

* add batch for paraformer

* add batch for offline-stream

* fix func SetBatchSize

* add SetBatchSize for model

* add SetBatchSize for model

* fix func Forward

* fix padding

* update funasrruntime

* add dec reset for batch

* set batch default value

* add argv for CutSplit

* sort frame_queue

* sorted msgs

* fix FunOfflineInfer

* add dynamic batch for fetch

* fix FetchDynamic

* update run_server.sh

* update run_server.sh

* cpp http post server support (#1739)

* add cpp http server

* add some comment

* remove some comments

* del debug infos

* restore run_server.sh

* adapt to new model struct

* 修复了onnxruntime在macos下编译失败的错误 (#1748)

* Add files via upload

增加macos的编译支持

* Add files via upload

增加macos支持

* Add files via upload

target_link_directories(funasr PUBLIC ${ONNXRUNTIME_DIR}/lib)
target_link_directories(funasr PUBLIC ${FFMPEG_DIR}/lib)
添加 if(APPLE) 限制

---------

Co-authored-by: Yabin Li <[email protected]>

* Delete docs/images/wechat.png

* Add files via upload

* fixed the issues about seaco-onnx timestamp

* fix bug (#1764)

当语音识别结果包含 `http` 时,标点符号预测会把它会被当成 url

* fix empty asr result (#1765)

解码结果为空的语音片段,text 用空字符串

* update export

* update export

* docs

* docs

* update export name

* docs

* update

* docs

* docs

* keep empty speech result (#1772)

* docs

* docs

* update wechat QRcode

* Add python funasr api support for websocket srv (#1777)

* add python funasr_api supoort

* change little to README.md

* add core tools stream

* modified a little

* fix bug for timeout

* support for buffer decode

* add ffmpeg decode for buffer

* libtorch demo

* update libtorch infer

* update utils

* update demo

* update demo

* update libtorch inference

* update model class

* update seaco paraformer

* bug fix

* bug fix

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* Dev gzf exp (#1785)

* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding

* wechat

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* dynamic batch

* start_data_split_i=0

* total_time/accum_grad

* total_time/accum_grad

* total_time/accum_grad

* update avg slice

* update avg slice

* sensevoice sanm

* sensevoice sanm

* sensevoice sanm

---------

Co-authored-by: 北念 <[email protected]>

* auto frontend

* update paraformer timestamp

* [Optimization] support bladedisc fp16 optimization (#1790)

* add cif_v1 and cif_export

* Update SDK_advanced_guide_offline_zh.md

* add cif_wo_hidden_v1

* [fix] fix empty asr result (#1794)

* english timestamp for valilla paraformer

* wechat

* [fix] better solution for handling empty result (#1796)

* update scripts

* modify the qformer adaptor (#1804)

Co-authored-by: nichongjia-2007 <[email protected]>

* add ctc inference code (#1806)

Co-authored-by: haoneng.lhn <[email protected]>

* Update auto_model.py

修复空字串进入speaker model时报raw_text变量不存在的bug

* Update auto_model.py

修复识别出空串后spk_model内变量未定义问题

* update model name

* fix paramter 'quantize' unused issue (#1813)

Co-authored-by: ZihanLiao <[email protected]>

* wechat

* Update cif_predictor.py (#1811)

* Update cif_predictor.py

* modify cif_v1_export

under extreme cases, max_label_len calculated by batch_len misaligns with token_num

* Update cif_predictor.py

torch.cumsum precision degradation, using float64 instead

* update code

---------

Co-authored-by: 雾聪 <[email protected]>
Co-authored-by: zhaomingwork <[email protected]>
Co-authored-by: szsteven008 <[email protected]>
Co-authored-by: Ephemeroptera <[email protected]>
Co-authored-by: 彭震东 <[email protected]>
Co-authored-by: Shi Xian <[email protected]>
Co-authored-by: 维石 <[email protected]>
Co-authored-by: 北念 <[email protected]>
Co-authored-by: xiaowan0322 <[email protected]>
Co-authored-by: zhuangzhong <[email protected]>
Co-authored-by: Xingchen Song(宋星辰) <[email protected]>
Co-authored-by: nichongjia-2007 <[email protected]>
Co-authored-by: haoneng.lhn <[email protected]>
Co-authored-by: liugz18 <[email protected]>
Co-authored-by: Marlowe <[email protected]>
Co-authored-by: ZihanLiao <[email protected]>
Co-authored-by: zhong zhuang <[email protected]>

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* v1.0.28 (#1836)

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* sensevoice

* update (#1841)

* v1.0.28

* version checker

* version checker

* rollback cif_v1 for training bug

* fixbug

* fixbug for cif

* fixbug

---------

Co-authored-by: 维石 <[email protected]>

* update (#1842)

* v1.0.28

* version checker

* version checker

* rollback cif_v1 for training bug

* fixbug

* fixbug for cif

* fixbug

---------

Co-authored-by: 维石 <[email protected]>

* inference

---------

Co-authored-by: 雾聪 <[email protected]>
Co-authored-by: zhaomingwork <[email protected]>
Co-authored-by: szsteven008 <[email protected]>
Co-authored-by: Ephemeroptera <[email protected]>
Co-authored-by: 彭震东 <[email protected]>
Co-authored-by: Shi Xian <[email protected]>
Co-authored-by: 维石 <[email protected]>
Co-authored-by: 北念 <[email protected]>
Co-authored-by: xiaowan0322 <[email protected]>
Co-authored-by: zhuangzhong <[email protected]>
Co-authored-by: Xingchen Song(宋星辰) <[email protected]>
Co-authored-by: nichongjia-2007 <[email protected]>
Co-authored-by: haoneng.lhn <[email protected]>
Co-authored-by: liugz18 <[email protected]>
Co-authored-by: Marlowe <[email protected]>
Co-authored-by: ZihanLiao <[email protected]>
Co-authored-by: zhong zhuang <[email protected]>
2024-06-24 17:06:21 +08:00
游雁 45d7aa9004 decoding 2024-06-19 10:26:40 +08:00
Shi Xian 6c467e6f0a Merge pull request #1825 from modelscope/dev_libt
Dev libt
2024-06-18 10:01:56 +08:00
维石 9377eed41e update code 2024-06-17 20:20:00 +08:00
北念 ada76b6312 sensevoice 2024-06-17 13:36:22 +08:00
游雁 59bc02b089 decoding 2024-06-14 13:59:49 +08:00
游雁 67329a74a5 decoding 2024-06-14 11:04:49 +08:00
32e7836645 update with main (#1786)
* add cmakelist

* add paraformer-torch

* add debug for funasr-onnx-offline

* fix redefinition of jieba StdExtension.hpp

* add loading torch models

* update funasr-onnx-offline

* add SwitchArg for wss-server

* add SwitchArg for funasr-onnx-offline

* update cmakelist

* update funasr-onnx-offline-rtf

* add define condition

* add gpu define for offlne-stream

* update com define

* update offline-stream

* update cmakelist

* update func CompileHotwordEmbedding

* add timestamp for paraformer-torch

* add C10_USE_GLOG for paraformer-torch

* update paraformer-torch

* fix func FunASRWfstDecoderInit

* update model.h

* fix func FunASRWfstDecoderInit

* fix tpass_stream

* update paraformer-torch

* add bladedisc for funasr-onnx-offline

* update comdefine

* update funasr-wss-server

* add log for torch

* fix GetValue BLADEDISC

* fix log

* update cmakelist

* update warmup to 10

* update funasrruntime

* add batch_size for wss-server

* add batch for bins

* add batch for offline-stream

* add batch for paraformer

* add batch for offline-stream

* fix func SetBatchSize

* add SetBatchSize for model

* add SetBatchSize for model

* fix func Forward

* fix padding

* update funasrruntime

* add dec reset for batch

* set batch default value

* add argv for CutSplit

* sort frame_queue

* sorted msgs

* fix FunOfflineInfer

* add dynamic batch for fetch

* fix FetchDynamic

* update run_server.sh

* update run_server.sh

* cpp http post server support (#1739)

* add cpp http server

* add some comment

* remove some comments

* del debug infos

* restore run_server.sh

* adapt to new model struct

* 修复了onnxruntime在macos下编译失败的错误 (#1748)

* Add files via upload

增加macos的编译支持

* Add files via upload

增加macos支持

* Add files via upload

target_link_directories(funasr PUBLIC ${ONNXRUNTIME_DIR}/lib)
target_link_directories(funasr PUBLIC ${FFMPEG_DIR}/lib)
添加 if(APPLE) 限制

---------

Co-authored-by: Yabin Li <[email protected]>

* Delete docs/images/wechat.png

* Add files via upload

* fixed the issues about seaco-onnx timestamp

* fix bug (#1764)

当语音识别结果包含 `http` 时,标点符号预测会把它会被当成 url

* fix empty asr result (#1765)

解码结果为空的语音片段,text 用空字符串

* docs

* docs

* docs

* docs

* docs

* keep empty speech result (#1772)

* docs

* docs

* update wechat QRcode

* Add python funasr api support for websocket srv (#1777)

* add python funasr_api supoort

* change little to README.md

* add core tools stream

* modified a little

* fix bug for timeout

* support for buffer decode

* add ffmpeg decode for buffer

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* auto frontend

* Dev gzf exp (#1785)

* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding

* wechat

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* dynamic batch

* start_data_split_i=0

* total_time/accum_grad

* total_time/accum_grad

* total_time/accum_grad

* update avg slice

* update avg slice

* sensevoice sanm

* sensevoice sanm

* sensevoice sanm

---------

Co-authored-by: 北念 <[email protected]>

* auto frontend

---------

Co-authored-by: 雾聪 <[email protected]>
Co-authored-by: zhaomingwork <[email protected]>
Co-authored-by: szsteven008 <[email protected]>
Co-authored-by: Ephemeroptera <[email protected]>
Co-authored-by: 彭震东 <[email protected]>
Co-authored-by: Shi Xian <[email protected]>
Co-authored-by: 维石 <[email protected]>
Co-authored-by: 北念 <[email protected]>
2024-06-06 09:54:35 +08:00
zhifu gaoand北念 db9ec58cb4 Dev gzf exp (#1785)
* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding

* wechat

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* dynamic batch

* start_data_split_i=0

* total_time/accum_grad

* total_time/accum_grad

* total_time/accum_grad

* update avg slice

* update avg slice

* sensevoice sanm

* sensevoice sanm

* sensevoice sanm

---------

Co-authored-by: 北念 <[email protected]>
2024-06-05 17:30:57 +08:00
游雁 79b09f1d67 auto frontend 2024-06-05 17:22:21 +08:00
961ec280af Dev gzf deepspeed (#1736)
* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding

* wechat

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* dynamic batch

* start_data_split_i=0

* total_time/accum_grad

* total_time/accum_grad

* total_time/accum_grad

* update avg slice

* update avg slice

* sensevoice sanm

* sensevoice sanm

* add

* add

* add

* add

* deepspeed

* update with main (#1731)

* c++ runtime adapt to 1.0 (#1724)

* adapt vad runtime to 1.0

* add json

* change yml name

* add func LoadVocabFromJson

* add token file for InitAsr

* add token path for OfflineStream

* add funcOpenYaml

* add token file for InitPunc

* add token file for stream

* update punc-model

* update funasr-wss-server

* update runtime_sdk_download_tool.py

* update docker list

* Delete docs/images/wechat.png

* Add files via upload

* Emo2Vec限定选择的情感类别 (#1730)

* 限定选择的情感类别

* 使用none来禁用情感标签输出

* 修改输出接口

* 使用unuse来禁用token

---------

Co-authored-by: 常材 <[email protected]>

* bugfix

* v1.0.27

* update docs

* hf hub

* Fix incorrect assignment of 'end' attribute to 'start' in sentences list comprehension (#1680)

---------

Co-authored-by: Yabin Li <[email protected]>
Co-authored-by: gaochangfeng <[email protected]>
Co-authored-by: 常材 <[email protected]>
Co-authored-by: nsdou <[email protected]>

* docs

* docs

* deepspeed

* deepspeed

* deepspeed

* deepspeed

* update

* ds

* ds

* ds

* ds

* ds

* ds

* ds

---------

Co-authored-by: Yabin Li <[email protected]>
Co-authored-by: gaochangfeng <[email protected]>
Co-authored-by: 常材 <[email protected]>
Co-authored-by: nsdou <[email protected]>
2024-05-20 15:31:46 +08:00
a0f03bd2a8 Dev gzf deepspeed (#1732)
* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding

* wechat

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* dynamic batch

* start_data_split_i=0

* total_time/accum_grad

* total_time/accum_grad

* total_time/accum_grad

* update avg slice

* update avg slice

* sensevoice sanm

* sensevoice sanm

* add

* add

* add

* add

* deepspeed

* update with main (#1731)

* c++ runtime adapt to 1.0 (#1724)

* adapt vad runtime to 1.0

* add json

* change yml name

* add func LoadVocabFromJson

* add token file for InitAsr

* add token path for OfflineStream

* add funcOpenYaml

* add token file for InitPunc

* add token file for stream

* update punc-model

* update funasr-wss-server

* update runtime_sdk_download_tool.py

* update docker list

* Delete docs/images/wechat.png

* Add files via upload

* Emo2Vec限定选择的情感类别 (#1730)

* 限定选择的情感类别

* 使用none来禁用情感标签输出

* 修改输出接口

* 使用unuse来禁用token

---------

Co-authored-by: 常材 <[email protected]>

* bugfix

* v1.0.27

* update docs

* hf hub

* Fix incorrect assignment of 'end' attribute to 'start' in sentences list comprehension (#1680)

---------

Co-authored-by: Yabin Li <[email protected]>
Co-authored-by: gaochangfeng <[email protected]>
Co-authored-by: 常材 <[email protected]>
Co-authored-by: nsdou <[email protected]>

* docs

---------

Co-authored-by: Yabin Li <[email protected]>
Co-authored-by: gaochangfeng <[email protected]>
Co-authored-by: 常材 <[email protected]>
Co-authored-by: nsdou <[email protected]>
2024-05-15 19:48:50 +08:00
游雁 c7bfd1fdc5 bugfix 2024-05-15 16:40:12 +08:00
zhifu gao 4adb76a6ed Dev gzf exp (#1707)
* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding

* wechat

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* dynamic batch
2024-05-08 19:21:58 +08:00
zhifu gaoand常材 a7bc099548 Dev gzf exp (#1705)
* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding

* wechat

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key

* Gcf (#1704)

* 添加富文本解码约束

* special token

* bug fix

* fix

---------

Co-authored-by: 常材 <[email protected]>

* decoding key

---------

Co-authored-by: 常材 <[email protected]>
2024-05-08 17:32:36 +08:00
zhifu gao b1c186fd00 Dev gzf exp (#1700)
* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding

* wechat

* decoding key

* decoding key

* decoding key

* decoding key

* decoding key
2024-05-08 00:31:29 +08:00
zhifu gao 7be2a1d7e4 Dev gzf exp (#1696)
* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding

* decoding
2024-05-06 22:36:00 +08:00
zhifu gao 00d0df3a10 Dev gzf decoding (#1695)
* resume from step

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* batch

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* train_loss_avg train_acc_avg

* log step

* wav is not exist

* wav is not exist

* decoding

* decoding
2024-05-06 22:17:25 +08:00
zhifu gao a09aba419f Dev gzf exp (#1682)
* resume from step
2024-04-30 12:52:58 +08:00
zhifu gao 11cf10e433 Dev gzf exp (#1678)
* resume from step

* batch

* batch

* batch
2024-04-29 14:52:20 +08:00
zhifu gao 1cdb3cc28d Dev gzf exp (#1665)
* rwkv 5

* rwkv v4

* rwkv v4

* rwkv

* rwkv

* update

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step

* resume from step
2024-04-26 11:27:39 +08:00