modelscope/FunASR

Fork 0

mirror of https://github.com/modelscope/FunASR.git synced 2026-10-12 02:19:18 +00:00

Files

T

History

da12182b6b Fix invalid escape sequences in text normalization and models (#3709 )

Three modules use invalid escapes in non-raw string literals:
- fun_text_processing/text_normalization/normalize.py - \., \w
- funasr/models/fsmn_kws/encoder.py - \[
- funasr/models/fun_asr_nano/tools/format5res.py - \.

On Python 3.12+ each raises SyntaxWarning: invalid escape sequence on
import, and FunASR runs the 3.10-3.12 CI matrix. Escaping the
backslashes leaves every string value identical (ruff W605-style).


Claude-Session: https://claude.ai/code/session_014HBQNzAf5C2E3MiJC48HQw

Co-authored-by: Mehdi Boutayeb <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>

2026-09-28 16:08:01 +08:00

inverse_text_normalization

Update run_evaluate.py (#2175 )

2024-10-28 21:22:39 +08:00

text_normalization

Fix invalid escape sequences in text normalization and models (#3709 )

2026-09-28 16:08:01 +08:00

__init__.py

update version 0.1.6

2023-01-16 18:46:40 +08:00

install_pynini.sh

FunTextProcessing

2022-12-22 13:28:07 +08:00

README.md

update version 0.1.6

2023-01-16 18:46:40 +08:00

README.md

Fundamental Text Processing (FunTextProcessing)

Introduction

FunTextProcessing is a Python toolkit for fundamental text processing in ASR including text processing , inverse text processing, num2words, which is included in the FunASR.

Highlights

FunTextProcessing supports inverse text processing (ITN), text processing (TN), number to words (num2words).
FunTextProcessing supports multilingual, 10+ languages for ITN, 5 languages for TN, 50+ languages for num2words.

Example

Inverse Text Processing (ITN)

Given text inputs, such as speech recognition results, use fun_text_processing/inverse_text_normalization/inverse_normalize.py to output ITN results. You may refer to the following example scripts.

test_file=fun_text_processing/inverse_text_normalization/id/id_itn_test_input.txt

python fun_text_processing/inverse_text_normalization/inverse_normalize.py --input_file $test_file --cache_dir ./itn_model/ --output_file output.txt --language=id

Acknowledge

We borrowed a lot of codes from NeMo.
We refered the codes from WeTextProcessing for Chinese inverse text normalization.
We borrowed a lot of codes from num2words library for convert the number to words function in some languages.

License

This project is licensed under the The MIT License. FunTextProcessing also contains various third-party components and some code modified from other repos under other open source licenses.