openharmony/third_party_rust_encoding_rs
1
0
Fork 0
mirror of https://github.com/openharmony/third_party_rust_encoding_rs.git synced 2026-07-19 23:04:37 -04:00
Code Issues Packages Projects Releases Wiki Activity

Document how to minimize unmappables when using the encoders.

Browse Source
This commit is contained in:
Henri Sivonen
2019-05-21 12:06:27 +03:00
parent 05c65ea427
commit 9b14a0bf76
2 changed files with 24 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+12
View File
@@ -86,6 +86,18 @@ one directly. (It wraps this crate and adds UTF-7 decoding.)
For mappings to and from Windows code page identifiers, use the
[`codepage`](https://crates.io/crates/codepage) crate.
## Preparing Text for the Encoders
Normalizing text into Unicode Normalization Form C prior to encoding text into
a legacy encoding minimizes unmappable characters. Text can be normalized to
Unicode Normalization Form C using the
[`unic-normal`](https://crates.io/crates/unic-normal) crate.
The exception is windows-1258, which after normalizing to Unicode Normalization
Form C requires tone marks to be decomposed in order to minimize unmappable
characters. Vietnamese tone marks can be decomposed using the
[`detone`](https://crates.io/crates/detone) crate.
## Licensing
Please see the file named
src/lib.rs
+12
View File
@@ -248,6 +248,18 @@
//! [charset](https://crates.io/crates/charset) wraps encoding_rs and adds
//! UTF-7 decoding for email purposes.
//!
//! # Preparing Text for the Encoders
//!
//! Normalizing text into Unicode Normalization Form C prior to encoding text
//! into a legacy encoding minimizes unmappable characters. Text can be
//! normalized to Unicode Normalization Form C using the
//! [`unic-normal`](https://crates.io/crates/unic-normal) crate.
//!
//! The exception is windows-1258, which after normalizing to Unicode
//! Normalization Form C requires tone marks to be decomposed in order to
//! minimize unmappable characters. Vietnamese tone marks can be decomposed
//! using the [`detone`](https://crates.io/crates/detone) crate.
//!
//! # Streaming & Non-Streaming; Rust & C/C++
//!
//! The API in Rust has two modes of operation: streaming and non-streaming.