mirror of
https://github.com/modelscope/FunASR.git
synced 2026-10-11 22:39:57 +00:00
da12182b6b
Three modules use invalid escapes in non-raw string literals: - fun_text_processing/text_normalization/normalize.py - \., \w - funasr/models/fsmn_kws/encoder.py - \[ - funasr/models/fun_asr_nano/tools/format5res.py - \. On Python 3.12+ each raises SyntaxWarning: invalid escape sequence on import, and FunASR runs the 3.10-3.12 CI matrix. Escaping the backslashes leaves every string value identical (ruff W605-style). Claude-Session: https://claude.ai/code/session_014HBQNzAf5C2E3MiJC48HQw Co-authored-by: Mehdi Boutayeb <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]>
Fundamental Text Processing (FunTextProcessing)
Introduction
FunTextProcessing is a Python toolkit for fundamental text processing in ASR including text processing , inverse text processing, num2words, which is included in the FunASR.
Highlights
- FunTextProcessing supports inverse text processing (ITN), text processing (TN), number to words (num2words).
- FunTextProcessing supports multilingual, 10+ languages for ITN, 5 languages for TN, 50+ languages for num2words.
Example
Inverse Text Processing (ITN)
Given text inputs, such as speech recognition results, use fun_text_processing/inverse_text_normalization/inverse_normalize.py to output ITN results. You may refer to the following example scripts.
test_file=fun_text_processing/inverse_text_normalization/id/id_itn_test_input.txt
python fun_text_processing/inverse_text_normalization/inverse_normalize.py --input_file $test_file --cache_dir ./itn_model/ --output_file output.txt --language=id
Acknowledge
- We borrowed a lot of codes from NeMo.
- We refered the codes from WeTextProcessing for Chinese inverse text normalization.
- We borrowed a lot of codes from num2words library for convert the number to words function in some languages.
License
This project is licensed under the The MIT License. FunTextProcessing also contains various third-party components and some code modified from other repos under other open source licenses.