Magic-Mirror/gecko-dev
1
0
Fork 0
mirror of https://github.com/mozilla/gecko-dev.git synced 2024-10-18 15:55:36 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Bug 1560038 - Vendor in fluent-rs.

Browse Source 
Differential Revision: https://phabricator.services.mozilla.com/D56641

Depends on D58858

--HG--
extra : rebase_source : e2ffe74759467c613326d8e966c6eb32c05b38b7
This commit is contained in:
Zibi Braniecki 2020-03-10 20:05:06 +02:00
parent 42a7b711f8
commit 646dfccfd9
181 changed files with 26086 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

71
Cargo.lock generated
View File

@ -1278,6 +1278,32 @@ dependencies = [
"num-traits",
]
[[package]]
name = "fluent"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ebe7532e1e5146a909de9e019e31835a84b5dee3eeb234561e525844f3cf3bf"
dependencies = [
"fluent-bundle",
"fluent-pseudo",
"unic-langid",
]
[[package]]
name = "fluent-bundle"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "27ade33328521266c81cc0924523988f43ccd7359f64689a1b6e818afca3a646"
dependencies = [
"fluent-langneg",
"fluent-syntax",
"intl-memoizer",
"intl_pluralrules",
"rental",
"smallvec 1.2.0",
"unic-langid",
]
[[package]]
name = "fluent-langneg"
version = "0.12.1"
@ -1300,6 +1326,21 @@ dependencies = [
"xpcom",
]
[[package]]
name = "fluent-pseudo"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca3a870aefc42d175d11fb1ec089221ced8a160d66ca1e0c64a57b4ae90d2462"
dependencies = [
"regex",
]
[[package]]
name = "fluent-syntax"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9fe937dbd784d0f085f05c15a06f0d5dd06ce31cc823f7ab12ebb3758d948b39"
[[package]]
name = "fnv"
version = "1.0.6"
@ -1618,6 +1659,7 @@ dependencies = [
"cubeb-sys",
"encoding_glue",
"env_logger",
"fluent",
"fluent-langneg",
"fluent-langneg-ffi",
"fog",
@ -1925,6 +1967,26 @@ dependencies = [
"adler32",
]
[[package]]
name = "intl-memoizer"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9867e2d65d82936ef34217ed0f87b639a94384e93a0676158142c861c705391f"
dependencies = [
"type-map",
"unic-langid",
]
[[package]]
name = "intl_pluralrules"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d82c14d8eece42c03353e0ce86a4d3f97b1f1cef401e4d962dca6c6214a85002"
dependencies = [
"tinystr",
"unic-langid",
]
[[package]]
name = "iovec"
version = "0.1.2"
@ -4441,6 +4503,15 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e604eb7b43c06650e854be16a2a03155743d3752dd1c943f6829e26b7a36e382"
[[package]]
name = "type-map"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d2741b1474c327d95c1f1e3b0a2c3977c8e128409c572a33af2914e7d636717"
dependencies = [
"fxhash",
]
[[package]]
name = "typenum"
version = "1.10.0"

1
third_party/rust/fluent-bundle/.cargo-checksum.json vendored Normal file
View File

File diff suppressed because one or more lines are too long

145
third_party/rust/fluent-bundle/CHANGELOG.md vendored Normal file
View File

@ -0,0 +1,145 @@
# Changelog
## Unreleased
- …
## fluent-bundle 0.11.0 (March 10, 2020)
- Separate out `concurrent` version of `FluentBundle`.
- Switch FluentBundle functions to use function pointers.
## fluent-bundle 0.10.2 (February 20, 2020)
- Update to `intl_memoizer` 0.3.0 to allow for Send+Sync on FluentBundle.
## fluent-bundle 0.10.1 (February 15, 2020)
- Switch RefCell in FluentBundle to Mutex.
## fluent-bundle 0.10.0 (February 13, 2020)
- Update `fluent-langneg` to 0.12.
- Update `intl_pluralrules` to 6.0.
- Update `unic-langid` to 0.8.
- Introduce `intl-memoizer`.
- Improve the ergonomics of FluentArgs.
- Add `add_resource_overriding`.
- Remove dependency on `failure`.
- Switch the strategy to mitigate bomb attack to limit the number of placeables.
- Introduce `FluentType` for custom types.
- Improve ergonomics of `FluentNumber` and bring its features closer to ECMA402 Intl.NumberFormat.
## fluent-bundle 0.9.0 (November 26, 2019)
- Update `unic-langid` to 0.7.
- Update `fluent-langneg` to 0.11.
- Update `intl_pluralrules` to 5.0.
## fluent-bundle 0.8.0 (October 3, 2019)
- Update `unic-langid` to 0.6.
- Update `fluent-locale` to 0.10.
## fluent-bundle 0.7.2 (October 1, 2019)
- Update `unic-langid` to 0.5.
- Update `fluent-locale` to 0.9.
- Stop using macros to cut on compilation time and dependencies.
## fluent-bundle 0.7.1 (August 1, 2019)
- Fix FluentBundle::default to use isolating by default.
## fluent-bundle 0.7.0 (August 1, 2019)
- Turn FluentBundle to be a generic over Borrow<FluentResource> (#114)
- Update FluentBundle to the latest API (0.14) (#120)
- Switch to unic_langid for Language Identifier Management
- Refactor FluentArgs (#130)
- Add transform to FluentBundle to enable pseudolocalization (#131)
- Refactor resolver errors to provide better fallbacking and return errors out of formatting (#93)
- Enable FSI/PDI direction isolation (#116)
- Add more convenience From impls for FluentValue (#108)
- Fix `bare_trait_objects` warnings (#110)
## fluent-bundle 0.6.0 (March 26, 2019)
- Update to fluent-syntax 0.9
- Unify benchmark testsuite with fluent.js
## fluent-bundle 0.5.0 (January 31, 2019)
- Update to fluent-syntax 0.8
- Add unicode escaping
- Align with zero-copy parser
## fluent 0.4.3 (October 13, 2018)
- Support Sync+Send in Entry (#70)
## fluent 0.4.2 (October 1, 2018)
- Separate lifetimes of `FluentBundle::new` and return values. (#68)
## fluent 0.4.1 (August 31, 2018)
- Update README to make the example match new API
## fluent 0.4.0 (August 31, 2018)
- Rename MessageContext to FluentBundle
- Update the FluentBundle API to match fluent.js 0.8
- Update intl-pluralrules to 1.0
- Add FluentBundle::format_message
- Add FluentResource for external resource caching
- Update fluent-syntax to 0.1.1
- Update the signature of FluentBundle::format and FluentBundle::format_message
## fluent 0.3.1 (August 6, 2018)
- Update `fluent-locale` to 0.4.1.
- Switch MessageContext::locales to be an owned Vec\<String>
- Switch FluentValue::From\<i8> to FluentValue::From\<isize>
## fluent 0.3.0 (August 3, 2018)
- Add support for custom functions in MessageContext. (#50)
- Switch error handling to `annotate-snippets crate`.
- Separate `fluent` and `fluent-syntax` crates.
- Handle cyclic references. (#55)
- Switch parser binary to use `clap`.
- Switch plural rules handling to `intl_pluralrules`. (#56)
- Add `FluentValue::as_number`
- Move `IntlPluralRules` initialization into `MessageContext::new`
- General cleanups in line with `cargo fmt` and `cargo clippy`
## fluent 0.2.0 (February 11, 2018)
- Support Rust 1.23 stable
- Support Fluent 0.5 syntax
- Dual-license Apache 2.0 and MIT
## fluent 0.1.2 (October 14, 2017)
- Add more complex PluralRules support
## fluent 0.1.0 (October 13, 2017)
- Support parsing Fluent Syntax 0.3.
- Support formatting Messages and Attributes alike.
- Support string- and Number-typed external arguments
- Select expressions:
- without a selector.
- with literal strings and numbers as selector,
- with external arguments as selector,
- with message reference as selector (using tags).
- Support matching numbers in select expression to plural categories.
- Only a single mock plural rule has been implemented for now.
- Support Attribute expressions.
- Support Variant expressions.
- `MessageContext::new` now takes a slice as the `locales` argument.
- Added integration with Travis CI and Coveralls.
- Expanded module documentation.
## fluent 0.0.1 (January 17, 2017)
- This is the first release to be listed in the CHANGELOG.
- Basic parser support for the FTL syntax.
- Message references.

855
third_party/rust/fluent-bundle/Cargo.lock generated vendored Normal file
View File

@ -0,0 +1,855 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
[[package]]
name = "atty"
version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9b39be18770d11421cdb1b9947a45dd3f37e93092cbf377614828a319d5fee8"
dependencies = [
"hermit-abi",
"libc",
"winapi",
]
[[package]]
name = "autocfg"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8aac770f1885fd7e387acedd76065302551364496e46b3dd00860b2f8359b9d"
[[package]]
name = "bitflags"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693"
[[package]]
name = "bstr"
version = "0.2.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "502ae1441a0a5adb8fbd38a5955a6416b9493e92b465de5e4a9bde6a539c2c48"
dependencies = [
"lazy_static",
"memchr",
"regex-automata",
"serde",
]
[[package]]
name = "bumpalo"
version = "3.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1f359dc14ff8911330a51ef78022d376f25ed00248912803b58f00cb1c27f742"
[[package]]
name = "byteorder"
version = "1.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08c48aae112d48ed9f069b33538ea9e3e90aa263cfa3d1c24309612b1f7472de"
[[package]]
name = "c2-chacha"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "214238caa1bf3a496ec3392968969cab8549f96ff30652c9e56885329315f6bb"
dependencies = [
"ppv-lite86",
]
[[package]]
name = "cast"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4b9434b9a5aa1450faa3f9cb14ea0e8c53bb5d2b3c1bfd1ab4fc03e9f33fbfb0"
dependencies = [
"rustc_version",
]
[[package]]
name = "cfg-if"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
[[package]]
name = "clap"
version = "2.33.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5067f5bb2d80ef5d68b4c87db81601f0b75bca627bc2ef76b141d7b846a3c6d9"
dependencies = [
"bitflags",
"textwrap",
"unicode-width",
]
[[package]]
name = "criterion"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fc755679c12bda8e5523a71e4d654b6bf2e14bd838dfc48cde6559a05caf7d1"
dependencies = [
"atty",
"cast",
"clap",
"criterion-plot",
"csv",
"itertools",
"lazy_static",
"num-traits",
"oorandom",
"plotters",
"rayon",
"regex",
"serde",
"serde_derive",
"serde_json",
"tinytemplate",
"walkdir",
]
[[package]]
name = "criterion-plot"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a01e15e0ea58e8234f96146b1f91fa9d0e4dd7a38da93ff7a75d42c0b9d3a545"
dependencies = [
"cast",
"itertools",
]
[[package]]
name = "crossbeam-deque"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f02af974daeee82218205558e51ec8768b48cf524bd01d550abe5573a608285"
dependencies = [
"crossbeam-epoch",
"crossbeam-utils",
"maybe-uninit",
]
[[package]]
name = "crossbeam-epoch"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "058ed274caafc1f60c4997b5fc07bf7dc7cca454af7c6e81edffe5f33f70dace"
dependencies = [
"autocfg",
"cfg-if",
"crossbeam-utils",
"lazy_static",
"maybe-uninit",
"memoffset",
"scopeguard",
]
[[package]]
name = "crossbeam-queue"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c695eeca1e7173472a32221542ae469b3e9aac3a4fc81f7696bcad82029493db"
dependencies = [
"cfg-if",
"crossbeam-utils",
]
[[package]]
name = "crossbeam-utils"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3c7c73a2d1e9fc0886a08b93e98eb643461230d5f1925e4036204d5f2e261a8"
dependencies = [
"autocfg",
"cfg-if",
"lazy_static",
]
[[package]]
name = "csv"
version = "1.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00affe7f6ab566df61b4be3ce8cf16bc2576bca0963ceb0955e45d514bf9a279"
dependencies = [
"bstr",
"csv-core",
"itoa",
"ryu",
"serde",
]
[[package]]
name = "csv-core"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b2466559f260f48ad25fe6317b3c8dac77b5bdb5763ac7d9d6103530663bc90"
dependencies = [
"memchr",
]
[[package]]
name = "dtoa"
version = "0.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4358a9e11b9a09cf52383b451b49a169e8d797b68aa02301ff586d70d9661ea3"
[[package]]
name = "either"
version = "1.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb1f6b1ce1c140482ea30ddd3335fc0024ac7ee112895426e0a629a6c20adfe3"
[[package]]
name = "fluent-bundle"
version = "0.11.0"
dependencies = [
"criterion",
"fluent-langneg",
"fluent-syntax",
"intl-memoizer",
"intl_pluralrules",
"rand",
"rental",
"serde",
"serde_yaml",
"smallvec",
"unic-langid",
]
[[package]]
name = "fluent-langneg"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe5815efd5542e40841cd34ef9003822352b04c67a70c595c6758597c72e1f56"
dependencies = [
"unic-langid",
]
[[package]]
name = "fluent-syntax"
version = "0.9.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac0f7e83d14cccbf26e165d8881dcac5891af0d85a88543c09dd72ebd31d91ba"
[[package]]
name = "fxhash"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c31b6d751ae2c7f11320402d34e41349dd1016f8d5d45e48c4312bc8625af50c"
dependencies = [
"byteorder",
]
[[package]]
name = "getrandom"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7abc8dd8451921606d809ba32e95b6111925cd2906060d2dcc29c070220503eb"
dependencies = [
"cfg-if",
"libc",
"wasi",
]
[[package]]
name = "hermit-abi"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1010591b26bbfe835e9faeabeb11866061cc7dcebffd56ad7d0942d0e61aefd8"
dependencies = [
"libc",
]
[[package]]
name = "intl-memoizer"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9867e2d65d82936ef34217ed0f87b639a94384e93a0676158142c861c705391f"
dependencies = [
"type-map",
"unic-langid",
]
[[package]]
name = "intl_pluralrules"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d82c14d8eece42c03353e0ce86a4d3f97b1f1cef401e4d962dca6c6214a85002"
dependencies = [
"tinystr",
"unic-langid",
]
[[package]]
name = "itertools"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f56a2d0bc861f9165be4eb3442afd3c236d8a98afd426f65d92324ae1091a484"
dependencies = [
"either",
]
[[package]]
name = "itoa"
version = "0.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8b7a7c0c47db5545ed3fef7468ee7bb5b74691498139e4b3f6a20685dc6dd8e"
[[package]]
name = "js-sys"
version = "0.3.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1cb931d43e71f560c81badb0191596562bafad2be06a3f9025b845c847c60df5"
dependencies = [
"wasm-bindgen",
]
[[package]]
name = "lazy_static"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
[[package]]
name = "libc"
version = "0.2.67"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eb147597cdf94ed43ab7a9038716637d2d1bf2bc571da995d0028dec06bd3018"
[[package]]
name = "linked-hash-map"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae91b68aebc4ddb91978b11a1b02ddd8602a05ec19002801c5666000e05e0f83"
[[package]]
name = "log"
version = "0.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "14b6052be84e6b71ab17edffc2eeabf5c2c3ae1fdb464aae35ac50c67a44e1f7"
dependencies = [
"cfg-if",
]
[[package]]
name = "maybe-uninit"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "60302e4db3a61da70c0cb7991976248362f30319e88850c487b9b95bbf059e00"
[[package]]
name = "memchr"
version = "2.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3728d817d99e5ac407411fa471ff9800a778d88a24685968b36824eaf4bee400"
[[package]]
name = "memoffset"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75189eb85871ea5c2e2c15abbdd541185f63b408415e5051f5cac122d8c774b9"
dependencies = [
"rustc_version",
]
[[package]]
name = "num-traits"
version = "0.2.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c62be47e61d1842b9170f0fdeec8eba98e60e90e5446449a0545e5152acd7096"
dependencies = [
"autocfg",
]
[[package]]
name = "num_cpus"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46203554f085ff89c235cd12f7075f3233af9b11ed7c9e16dfe2560d03313ce6"
dependencies = [
"hermit-abi",
"libc",
]
[[package]]
name = "oorandom"
version = "11.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebcec7c9c2a95cacc7cd0ecb89d8a8454eca13906f6deb55258ffff0adeb9405"
[[package]]
name = "plotters"
version = "0.2.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e3bb8da247d27ae212529352020f3e5ee16e83c0c258061d27b08ab92675eeb"
dependencies = [
"js-sys",
"num-traits",
"wasm-bindgen",
"web-sys",
]
[[package]]
name = "ppv-lite86"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74490b50b9fbe561ac330df47c08f3f33073d2d00c150f719147d7c54522fa1b"
[[package]]
name = "proc-macro-hack"
version = "0.5.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ecd45702f76d6d3c75a80564378ae228a85f0b59d2f3ed43c91b4a69eb2ebfc5"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c09721c6781493a2a492a96b5a5bf19b65917fe6728884e7c44dd0c60ca3435"
dependencies = [
"unicode-xid",
]
[[package]]
name = "quote"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bdc6c187c65bca4260c9011c9e3132efe4909da44726bad24cf7572ae338d7f"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rand"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a6b1679d49b24bbfe0c803429aa1874472f50d9b363131f0e89fc356b544d03"
dependencies = [
"getrandom",
"libc",
"rand_chacha",
"rand_core",
"rand_hc",
]
[[package]]
name = "rand_chacha"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03a2a90da8c7523f554344f921aa97283eadf6ac484a6d2a7d0212fa7f8d6853"
dependencies = [
"c2-chacha",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90bde5296fc891b0cef12a6d03ddccc162ce7b2aff54160af9338f8d40df6d19"
dependencies = [
"getrandom",
]
[[package]]
name = "rand_hc"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca3129af7b92a17112d59ad498c6f81eaf463253766b90396d39ea7a39d6613c"
dependencies = [
"rand_core",
]
[[package]]
name = "rayon"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db6ce3297f9c85e16621bb8cca38a06779ffc31bb8184e1be4bed2be4678a098"
dependencies = [
"crossbeam-deque",
"either",
"rayon-core",
]
[[package]]
name = "rayon-core"
version = "1.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08a89b46efaf957e52b18062fb2f4660f8b8a4dde1807ca002690868ef2c85a9"
dependencies = [
"crossbeam-deque",
"crossbeam-queue",
"crossbeam-utils",
"lazy_static",
"num_cpus",
]
[[package]]
name = "regex"
version = "1.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "322cf97724bea3ee221b78fe25ac9c46114ebb51747ad5babd51a2fc6a8235a8"
dependencies = [
"regex-syntax",
]
[[package]]
name = "regex-automata"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae1ded71d66a4a97f5e961fd0cb25a5f366a42a41570d16a763a69c092c26ae4"
dependencies = [
"byteorder",
]
[[package]]
name = "regex-syntax"
version = "0.6.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1132f845907680735a84409c3bebc64d1364a5683ffbce899550cd09d5eaefc1"
[[package]]
name = "rental"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8545debe98b2b139fb04cad8618b530e9b07c152d99a5de83c860b877d67847f"
dependencies = [
"rental-impl",
"stable_deref_trait",
]
[[package]]
name = "rental-impl"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "475e68978dc5b743f2f40d8e0a8fdc83f1c5e78cbf4b8fa5e74e73beebc340de"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "rustc_version"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "138e3e0acb6c9fb258b19b67cb8abd63c00679d2851805ea151465464fe9030a"
dependencies = [
"semver",
]
[[package]]
name = "ryu"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bfa8506c1de11c9c4e4c38863ccbe02a305c8188e85a05a784c9e11e1c3910c8"
[[package]]
name = "same-file"
version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502"
dependencies = [
"winapi-util",
]
[[package]]
name = "scopeguard"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d29ab0c6d3fc0ee92fe66e2d99f700eab17a8d57d1c1d3b748380fb20baa78cd"
[[package]]
name = "semver"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d7eb9ef2c18661902cc47e535f9bc51b78acd254da71d375c2f6720d9a40403"
dependencies = [
"semver-parser",
]
[[package]]
name = "semver-parser"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "388a1df253eca08550bef6c72392cfe7c30914bf41df5269b68cbd6ff8f570a3"
[[package]]
name = "serde"
version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "414115f25f818d7dfccec8ee535d76949ae78584fc4f79a6f45a904bf8ab4449"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "128f9e303a5a29922045a830221b8f78ec74a5f544944f3d5984f8ec3895ef64"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.48"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9371ade75d4c2d6cb154141b9752cf3781ec9c05e0e5cf35060e1e70ee7b9c25"
dependencies = [
"itoa",
"ryu",
"serde",
]
[[package]]
name = "serde_yaml"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "691b17f19fc1ec9d94ec0b5864859290dff279dbd7b03f017afda54eb36c3c35"
dependencies = [
"dtoa",
"linked-hash-map",
"serde",
"yaml-rust",
]
[[package]]
name = "smallvec"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c2fb2ec9bcd216a5b0d0ccf31ab17b5ed1d627960edff65bbe95d3ce221cefc"
[[package]]
name = "stable_deref_trait"
version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dba1a27d3efae4351c8051072d619e3ade2820635c3958d826bfea39d59b54c8"
[[package]]
name = "syn"
version = "1.0.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "123bd9499cfb380418d509322d7a6d52e5315f064fe4b3ad18a53d6b92c07859"
dependencies = [
"proc-macro2",
"quote",
"unicode-xid",
]
[[package]]
name = "textwrap"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d326610f408c7a4eb6f51c37c330e496b08506c9457c9d34287ecc38809fb060"
dependencies = [
"unicode-width",
]
[[package]]
name = "tinystr"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4bac79c4b51eda1b090b1edebfb667821bbb51f713855164dc7cec2cb8ac2ba3"
[[package]]
name = "tinytemplate"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57a3c6667d3e65eb1bc3aed6fd14011c6cbc3a0665218ab7f5daf040b9ec371a"
dependencies = [
"serde",
"serde_json",
]
[[package]]
name = "type-map"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d2741b1474c327d95c1f1e3b0a2c3977c8e128409c572a33af2914e7d636717"
dependencies = [
"fxhash",
]
[[package]]
name = "unic-langid"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "24d81136159f779c35b10655f45210c71cd5ca5a45aadfe9840a61c7071735ed"
dependencies = [
"unic-langid-impl",
"unic-langid-macros",
]
[[package]]
name = "unic-langid-impl"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c43c61e94492eb67f20facc7b025778a904de83d953d8fcb60dd9adfd6e2d0ea"
dependencies = [
"tinystr",
]
[[package]]
name = "unic-langid-macros"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49bd90791278634d57e3ed4a4073108e3f79bfb87ab6a7b8664ba097425703df"
dependencies = [
"proc-macro-hack",
"tinystr",
"unic-langid-impl",
"unic-langid-macros-impl",
]
[[package]]
name = "unic-langid-macros-impl"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e0098f77bd754f8fb7850cdf4ab143aa821898c4ac6dc16bcb2aa3e62ce858d1"
dependencies = [
"proc-macro-hack",
"quote",
"syn",
"unic-langid-impl",
]
[[package]]
name = "unicode-width"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "caaa9d531767d1ff2150b9332433f32a24622147e5ebb1f26409d5da67afd479"
[[package]]
name = "unicode-xid"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "826e7639553986605ec5979c7dd957c7895e93eabed50ab2ffa7f6128a75097c"
[[package]]
name = "walkdir"
version = "2.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "777182bc735b6424e1a57516d35ed72cb8019d85c8c9bf536dccb3445c1a2f7d"
dependencies = [
"same-file",
"winapi",
"winapi-util",
]
[[package]]
name = "wasi"
version = "0.9.0+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"
[[package]]
name = "wasm-bindgen"
version = "0.2.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3557c397ab5a8e347d434782bcd31fc1483d927a6826804cec05cc792ee2519d"
dependencies = [
"cfg-if",
"wasm-bindgen-macro",
]
[[package]]
name = "wasm-bindgen-backend"
version = "0.2.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e0da9c9a19850d3af6df1cb9574970b566d617ecfaf36eb0b706b6f3ef9bd2f8"
dependencies = [
"bumpalo",
"lazy_static",
"log",
"proc-macro2",
"quote",
"syn",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0f6fde1d36e75a714b5fe0cffbb78978f222ea6baebb726af13c78869fdb4205"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "25bda4168030a6412ea8a047e27238cadf56f0e53516e1e83fec0a8b7c786f6d"
dependencies = [
"proc-macro2",
"quote",
"syn",
"wasm-bindgen-backend",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc9f36ad51f25b0219a3d4d13b90eb44cd075dff8b6280cca015775d7acaddd8"
[[package]]
name = "web-sys"
version = "0.3.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "721c6263e2c66fd44501cc5efbfa2b7dfa775d13e4ea38c46299646ed1f9c70a"
dependencies = [
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "winapi"
version = "0.3.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8093091eeb260906a183e6ae1abdba2ef5ef2257a21801128899c3fc699229c6"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]]
name = "winapi-util"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ccfbf554c6ad11084fb7517daca16cfdcaccbdadba4fc336f032a8b12c2ad80"
dependencies = [
"winapi",
]
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "yaml-rust"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65923dd1784f44da1d2c3dbbc5e822045628c590ba72123e1c73d3c230c4434d"
dependencies = [
"linked-hash-map",
]

64
third_party/rust/fluent-bundle/Cargo.toml vendored Normal file
View File

@ -0,0 +1,64 @@
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g., crates.io) dependencies
#
# If you believe there's an error in this file please file an
# issue against the rust-lang/cargo repository. If you're
# editing this file be aware that the upstream Cargo.toml
# will likely look very different (and much more reasonable)
[package]
edition = "2018"
name = "fluent-bundle"
version = "0.11.0"
authors = ["Zibi Braniecki <gandalf@mozilla.com>", "Staś Małolepszy <stas@mozilla.com>"]
description = "A localization system designed to unleash the entire expressive power of\nnatural language translations.\n"
homepage = "http://www.projectfluent.org"
readme = "README.md"
keywords = ["localization", "l10n", "i18n", "intl", "internationalization"]
categories = ["localization", "internationalization"]
license = "Apache-2.0/MIT"
repository = "https://github.com/projectfluent/fluent-rs"
[[bench]]
name = "resolver"
harness = false
[dependencies.fluent-langneg]
version = "0.12"
[dependencies.fluent-syntax]
version = "0.9"
[dependencies.intl-memoizer]
version = "0.4"
[dependencies.intl_pluralrules]
version = "6.0"
[dependencies.rental]
version = "0.5"
[dependencies.smallvec]
version = "1.0"
[dependencies.unic-langid]
version = "0.8"
[dev-dependencies.criterion]
version = "0.3"
[dev-dependencies.rand]
version = "0.7"
[dev-dependencies.serde]
version = "1.0"
features = ["derive"]
[dev-dependencies.serde_yaml]
version = "0.8"
[dev-dependencies.unic-langid]
version = "0.8"
features = ["macros"]

111
third_party/rust/fluent-bundle/README.md vendored Normal file
View File

@ -0,0 +1,111 @@
# Fluent
`fluent-rs` is a Rust implementation of [Project Fluent][], a localization
framework designed to unleash the entire expressive power of natural language
translations.
[![crates.io](http://meritbadge.herokuapp.com/fluent)](https://crates.io/crates/fluent)
[![Build Status](https://travis-ci.org/projectfluent/fluent-rs.svg?branch=master)](https://travis-ci.org/projectfluent/fluent-rs)
[![Coverage Status](https://coveralls.io/repos/github/projectfluent/fluent-rs/badge.svg?branch=master)](https://coveralls.io/github/projectfluent/fluent-rs?branch=master)
Project Fluent keeps simple things simple and makes complex things possible.
The syntax used for describing translations is easy to read and understand. At
the same time it allows, when necessary, to represent complex concepts from
natural languages like gender, plurals, conjugations, and others.
[Documentation][]
[Project Fluent]: http://projectfluent.org
[Documentation]: https://docs.rs/fluent/
Usage
-----
```rust
use fluent_bundle::{FluentBundle, FluentResource};
use unic_langid::langid;
fn main() {
let ftl_string = "hello-world = Hello, world!".to_owned();
let res = FluentResource::try_new(ftl_string)
.expect("Could not parse an FTL string.");
let langid_en = langid!("en");
let mut bundle = FluentBundle::new(&[langid_en]);
bundle.add_resource(&res)
.expect("Failed to add FTL resources to the bundle.");
let msg = bundle.get_message("hello-world")
.expect("Failed to retrieve a message.");
let val = msg.value.expect("Message has no value.");
let mut errors = vec![];
let value = bundle.format_pattern(val, None, &mut errors);
assert_eq!(&value, "Hello, world!");
}
```
Status
------
The implementation is in its early stages and supports only some of the Project
Fluent's spec. Consult the [list of milestones][] for more information about
release planning and scope.
[list of milestones]: https://github.com/projectfluent/fluent-rs/milestones
Local Development
-----------------
cargo build
cargo test
cargo bench
cargo run --example simple-app
When submitting a PR please use [`cargo fmt`][] (nightly).
[`cargo fmt`]: https://github.com/rust-lang-nursery/rustfmt
Learn the FTL syntax
--------------------
FTL is a localization file format used for describing translation resources.
FTL stands for _Fluent Translation List_.
FTL is designed to be simple to read, but at the same time allows to represent
complex concepts from natural languages like gender, plurals, conjugations, and
others.
hello-user = Hello, { $username }!
[Read the Fluent Syntax Guide][] in order to learn more about the syntax. If
you're a tool author you may be interested in the formal [EBNF grammar][].
[Read the Fluent Syntax Guide]: http://projectfluent.org/fluent/guide/
[EBNF grammar]: https://github.com/projectfluent/fluent/tree/master/spec
Get Involved
------------
`fluent-rs` is open-source, licensed under the Apache License, Version 2.0. We
encourage everyone to take a look at our code and we'll listen to your
feedback.
Discuss
-------
We'd love to hear your thoughts on Project Fluent! Whether you're a localizer
looking for a better way to express yourself in your language, or a developer
trying to make your app localizable and multilingual, or a hacker looking for
a project to contribute to, please do get in touch on the mailing list and the
IRC channel.
- Discourse: https://discourse.mozilla.org/c/fluent
- IRC channel: [irc://irc.mozilla.org/l20n](irc://irc.mozilla.org/l20n)

318
third_party/rust/fluent-bundle/benches/menubar.ftl vendored Normal file
View File

@ -0,0 +1,318 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
## browser/locales/en-US/browser/menubar.ftl
## File menu
file-menu =
.label = File
.accesskey = F
tab-cmd =
.label = New Tab
.accesskey = T
new-user-context =
.label = New Container Tab
.accesskey = B
new-navigator-cmd =
.label = New Window
.accesskey = N
new-private-window =
.label = New Private Window
.accesskey = W
# Only displayed on OS X, and only on windows that aren't main browser windows,
# or when there are no windows but Firefox is still running.
open-location-cmd =
.label = Open Location…
open-file-cmd =
.label = Open File…
.accesskey = O
close-cmd =
.label = Close
.accesskey = C
close-window =
.label = Close Window
.accesskey = d
save-page-cmd =
.label = Save Page As…
.accesskey = A
email-page-cmd =
.label = Email Link…
.accesskey = E
print-setup-cmd =
.label = Page Setup…
.accesskey = u
print-preview-cmd =
.label = Print Preview
.accesskey = v
print-cmd =
.label = Print…
.accesskey = P
go-offline-cmd =
.label = Work Offline
.accesskey = k
quit-application-cmd =
.label = Quit
.accesskey = Q
## Edit menu
edit-menu =
.label = Edit
.accesskey = E
undo-cmd =
.label = Undo
.accesskey = U
redo-cmd =
.label = Redo
.accesskey = R
cut-cmd =
.label = Cut
.accesskey = t
copy-cmd =
.label = Copy
.accesskey = C
paste-cmd =
.label = Paste
.accesskey = P
delete-cmd =
.label = Delete
.accesskey = D
select-all-cmd =
.label = Select All
.accesskey = A
find-on-cmd =
.label = Find in This Page…
.accesskey = F
find-again-cmd =
.label = Find Again
.accesskey = g
bidi-switch-text-direction-item =
.label = Switch Text Direction
.accesskey = w
preferences-cmd-unix =
.label = Preferences
.accesskey = n
## View menu
view-menu =
.label = View
.accesskey = V
view-toolbars-menu =
.label = Toolbars
.accesskey = T
view-customize-toolbar =
.label = Customize…
.accesskey = C
view-sidebar-menu =
.label = Sidebar
.accesskey = e
bookmarks-button =
.label = Bookmarks
history-button =
.label = History
synced-tabs =
.label = Synced Tabs
full-zoom =
.label = Zoom
.accesskey = Z
full-zoom-enlarge-cmd =
.label = Zoom In
.accesskey = I
full-zoom-reduce-cmd =
.label = Zoom Out
.accesskey = O
full-zoom-reset-cmd =
.label = Reset
.accesskey = R
full-zoom-toggle-cmd =
.label = Zoom Text Only
.accesskey = T
page-style-menu =
.label = Page Style
.accesskey = y
page-style-no-style =
.label = No Style
.accesskey = n
page-style-persistent-only =
.label = Basic Page Style
.accesskey = b
charset-menu2 =
.label = Text Encoding
.accesskey = c
## Full Screen controls
## Match what Safari and other Apple applications use on OS X Lion.
#
enter-full-screen-cmd =
.accesskey = F
.label = Enter Full Screen
exit-full-screen-cmd =
.accesskey = F
.label = Exit Full Screen
full-screen-cmd =
.accesskey = F
.label = Full Screen
show-all-tabs-cmd =
.accesskey = A
.label = Show All Tabs
bidi-switch-page-direction-item =
.label = Switch Page Direction
.accesskey = D
## History menu
history-menu =
.label = History
.accesskey = s
show-all-history-cmd2 =
.label = Show All History
clear-recent-history =
.label = Clear Recent History…
sync-tabs-menu3 =
.label = Synced Tabs
history-restore-last-session =
.label = Restore Previous Session
hidden-tabs =
.label = Hidden Tabs
history-undo-menu =
.label = Recently Closed Tabs
history-undo-window-menu =
.label = Recently Closed Windows
## Bookmarks menu
bookmarks-menu =
.label = Bookmarks
.accesskey = B
show-all-bookmarks2 =
.label = Show All Bookmarks
add-cur-pages-cmd =
.label = Bookmark All Tabs…
personalbar-cmd =
.label = Bookmarks Toolbar
other-bookmarks-cmd =
.label = Other Bookmarks
mobile-bookmarks-cmd =
.label = Mobile Bookmarks
## Tools menu
tools-menu =
.label = Tools
.accesskey = T
downloads =
.label = Downloads
.accesskey = D
addons =
.label = Add-ons
.accesskey = A
sync-sign-in =
.label = Sign In To { -sync-brand-short-name }…
.accesskey = Y
sync-sync-now-item =
.label = Sync Now
.accesskey = S
sync-re-auth-item =
.label = Reconnect to { -sync-brand-short-name }…
.accesskey = R
web-developer-menu =
.label = Web Developer
.accesskey = W
page-source-cmd =
.label = Page Source
.accesskey = o
page-info-cmd =
.accesskey = I
.label = Page Info
preferences-cmd2 =
.label = Options
.accesskey = O
preferences-ldb-cmd =
.label = Layout Debugger
.accesskey = L
preferences-cmd-mac =
.label = Preferences…
services-menu-mac =
.label = Services
hide-this-app-cmd-mac2 =
.label = Hide { -brand-shorter-name }
hide-other-apps-cmd-mac =
.label = Hide Others
show-all-apps-cmd-mac =
.label = Show All
window-menu =
.label = Window
bring-all-to-front =
.label = Bring All to Front
help-menu =
.label = Help
.accesskey = H
product-help2 =
.label = { -brand-shorter-name } Help
.accesskey = H
help-show-tour2 =
.label = { -brand-shorter-name } Tour
.accesskey = o
help-keyboard-shortcuts =
.label = Keyboard Shortcuts
.accesskey = K
help-troubleshooting-info =
.accesskey = T
.label = Troubleshooting Information
help-feedback-page =
.accesskey = S
.label = Submit Feedback…
help-safe-mode =
.accesskey = R
.label = Restart with Add-ons Disabled…
.stopaccesskey = R
.stoplabel = Restart with Add-ons Enabled
report-deceptive-site-menu =
.label = Report Deceptive Site…
.accesskey = D
safeb =
.label = This isn’t a deceptive site…
.accesskey = d
about-product2 =
.accesskey = A
.label = About { -brand-shorter-name }
# browser/locales/en-US/browser/toolbar.ftl
urlbar-textbox =
.placeholder = Search or enter address
.accesskey = d
## Toolbar items
view-bookmarks-broadcaster =
.label = Bookmarks
view-bookmarks-key =
.key = b
view-bookmarks-key-win =
.key = i
view-history-broadcaster =
.label = History
view-history-key =
.key = h
view-tabs-broadcaster =
.label = Synced Tabs
# browser/branding/official/locales/en-US/brand.ftl
-brand-shorter-name = Firefox
-brand-short-name = Firefox
-brand-full-name = Mozilla Firefox
-vendor-short-name = Mozilla
trademark-info =
Firefox and the Firefox logos are trademarks of the Mozilla Foundation.
-sync-brand-short-name = Sync

1077
third_party/rust/fluent-bundle/benches/preferences.ftl vendored Normal file
View File

File diff suppressed because it is too large Load Diff

113
third_party/rust/fluent-bundle/benches/resolver.rs vendored Normal file
View File

@ -0,0 +1,113 @@
use criterion::criterion_group;
use criterion::criterion_main;
use criterion::Criterion;
use std::collections::HashMap;
use std::fs::File;
use std::io;
use std::io::Read;
use fluent_bundle::{FluentBundle, FluentResource, FluentValue};
use fluent_syntax::ast;
use unic_langid::langid;
fn read_file(path: &str) -> Result<String, io::Error> {
let mut f = File::open(path)?;
let mut s = String::new();
f.read_to_string(&mut s)?;
Ok(s)
}
fn get_strings(tests: &[&'static str]) -> HashMap<&'static str, String> {
let mut ftl_strings = HashMap::new();
for test in tests {
let path = format!("./benches/{}.ftl", test);
ftl_strings.insert(*test, read_file(&path).expect("Couldn't load file"));
}
return ftl_strings;
}
fn get_ids(res: &FluentResource) -> Vec<String> {
res.ast()
.body
.iter()
.filter_map(|entry| match entry {
ast::ResourceEntry::Entry(ast::Entry::Message(ast::Message { id, .. })) => {
Some(id.name.to_owned())
}
_ => None,
})
.collect()
}
fn get_args(name: &str) -> Option<HashMap<&str, FluentValue>> {
match name {
"preferences" => {
let mut prefs_args = HashMap::new();
prefs_args.insert("name", FluentValue::from("John"));
prefs_args.insert("tabCount", FluentValue::from(5));
prefs_args.insert("count", FluentValue::from(3));
prefs_args.insert("version", FluentValue::from("65.0"));
prefs_args.insert("path", FluentValue::from("/tmp"));
prefs_args.insert("num", FluentValue::from(4));
prefs_args.insert("email", FluentValue::from("john@doe.com"));
prefs_args.insert("value", FluentValue::from(4.5));
prefs_args.insert("unit", FluentValue::from("mb"));
prefs_args.insert("service-name", FluentValue::from("Mozilla Disk"));
Some(prefs_args)
}
_ => None,
}
}
fn add_functions<R>(name: &'static str, bundle: &mut FluentBundle<R>) {
match name {
"preferences" => {
bundle
.add_function("PLATFORM", |_args, _named_args| {
return "linux".into();
})
.expect("Failed to add a function to the bundle.");
}
_ => {}
}
}
fn resolver_bench(c: &mut Criterion) {
let tests = &["simple", "preferences", "menubar", "unescape"];
let ftl_strings = get_strings(tests);
c.bench_function_over_inputs(
"resolve",
move |b, &&name| {
let source = &ftl_strings[name];
let res =
FluentResource::try_new(source.to_owned()).expect("Couldn't parse an FTL source");
let ids = get_ids(&res);
let lids = &[langid!("en")];
let mut bundle = FluentBundle::new(lids);
bundle
.add_resource(res)
.expect("Couldn't add FluentResource to the FluentBundle");
add_functions(name, &mut bundle);
let args = get_args(name);
b.iter(|| {
for id in &ids {
let msg = bundle.get_message(id).expect("Message found");
let mut errors = vec![];
if let Some(value) = msg.value {
let _ = bundle.format_pattern(value, args.as_ref(), &mut errors);
}
for (_, value) in msg.attributes {
let _ = bundle.format_pattern(value, args.as_ref(), &mut errors);
}
assert!(errors.len() == 0, "Resolver errors: {:#?}", errors);
}
})
},
tests,
);
}
criterion_group!(benches, resolver_bench);
criterion_main!(benches);

102
third_party/rust/fluent-bundle/benches/simple.ftl vendored Normal file
View File

@ -0,0 +1,102 @@
# Artificial testcase with 100 simple Fluent Messages
key0 = Value 0
key1 = Value 1
key2 = Value 2
key3 = Value 3
key4 = Value 4
key5 = Value 5
key6 = Value 6
key7 = Value 7
key8 = Value 8
key9 = Value 9
key10 = Value 10
key11 = Value 11
key12 = Value 12
key13 = Value 13
key14 = Value 14
key15 = Value 15
key16 = Value 16
key17 = Value 17
key18 = Value 18
key19 = Value 19
key20 = Value 20
key21 = Value 21
key22 = Value 22
key23 = Value 23
key24 = Value 24
key25 = Value 25
key26 = Value 26
key27 = Value 27
key28 = Value 28
key29 = Value 29
key30 = Value 30
key31 = Value 31
key32 = Value 32
key33 = Value 33
key34 = Value 34
key35 = Value 35
key36 = Value 36
key37 = Value 37
key38 = Value 38
key39 = Value 39
key40 = Value 40
key41 = Value 41
key42 = Value 42
key43 = Value 43
key44 = Value 44
key45 = Value 45
key46 = Value 46
key47 = Value 47
key48 = Value 48
key49 = Value 49
key50 = Value 50
key51 = Value 51
key52 = Value 52
key53 = Value 53
key54 = Value 54
key55 = Value 55
key56 = Value 56
key57 = Value 57
key58 = Value 58
key59 = Value 59
key60 = Value 60
key61 = Value 61
key62 = Value 62
key63 = Value 63
key64 = Value 64
key65 = Value 65
key66 = Value 66
key67 = Value 67
key68 = Value 68
key69 = Value 69
key70 = Value 70
key71 = Value 71
key72 = Value 72
key73 = Value 73
key74 = Value 74
key75 = Value 75
key76 = Value 76
key77 = Value 77
key78 = Value 78
key79 = Value 79
key80 = Value 80
key81 = Value 81
key82 = Value 82
key83 = Value 83
key84 = Value 84
key85 = Value 85
key86 = Value 86
key87 = Value 87
key88 = Value 88
key89 = Value 89
key90 = Value 90
key91 = Value 91
key92 = Value 92
key93 = Value 93
key94 = Value 94
key95 = Value 95
key96 = Value 96
key97 = Value 97
key98 = Value 98
key99 = Value 99

9
third_party/rust/fluent-bundle/benches/unescape.ftl vendored Normal file
View File

@ -0,0 +1,9 @@
face-with-tears-of-joy = 😂
tetragram-for-centre = 𝌆
surrogates-in-text = \uD83D\uDE02
surrogates-in-string = {"\uD83D\uDE02"}
surrogates-in-adjacent-strings = {"\uD83D"}{"\uDE02"}
emoji-in-text = A face 😂 with tears of joy.
emoji-in-string = {"A face 😂 with tears of joy."}

6
third_party/rust/fluent-bundle/examples/README.md vendored Normal file
View File

@ -0,0 +1,6 @@
This directory contains a set of examples
of how to use Fluent.
Start with the `simple-app.rs` which is a very
trivial example of a command line application
with localization handled by Fluent.

145
third_party/rust/fluent-bundle/examples/custom_formatter.rs vendored Normal file
View File

@ -0,0 +1,145 @@
// This is an example of an application which uses a custom formatter
// to format selected types of values.
//
// This allows users to plug their own number formatter to Fluent.
use unic_langid::LanguageIdentifier;
use fluent_bundle::memoizer::MemoizerKind;
use fluent_bundle::types::{FluentNumber, FluentNumberOptions};
use fluent_bundle::{FluentArgs, FluentBundle, FluentResource, FluentValue};
fn custom_formatter<M: MemoizerKind>(num: &FluentValue, _intls: &M) -> Option<String> {
match num {
FluentValue::Number(n) => Some(format!("CUSTOM({})", n.value).into()),
_ => None,
}
}
fn main() {
// 1. Bootstrap a FluentBundle with a number of messages which use
// number formatting in different forms.
let ftl_string = String::from(
"
key-implicit = Here is an implicitly encoded number: { 5 }.
key-explicit = Here is an explicitly encoded number: { NUMBER(5) }.
key-var-implicit = Here is an implicitly encoded variable: { $num }.
key-var-explicit = Here is an explicitly encoded variable: { NUMBER($num) }.
key-var-with-arg = Here is a variable formatted with an argument { NUMBER($num, minimumFractionDigits: 5) }.
",
);
let res = FluentResource::try_new(ftl_string).expect("Could not parse an FTL string.");
let lang: LanguageIdentifier = "en".parse().unwrap();
let mut bundle = FluentBundle::new(&[lang]);
bundle
.add_resource(res)
.expect("Failed to add FTL resources to the bundle.");
bundle
.add_function("NUMBER", |positional, named| {
match positional.get(0) {
Some(FluentValue::Number(n)) => {
let mut num = n.clone();
// This allows us to merge the arguments provided
// as arguments to the function into the new FluentNumber.
num.options.merge(named);
FluentValue::Number(num)
}
_ => FluentValue::None,
}
})
.expect("Failed to add a function.");
bundle.set_use_isolating(false);
let mut errors = vec![];
// 2. First, we're going to format the number using the implicit formatter.
// At the moment the number will be formatted in a very dummy way, since
// we do not have a locale aware number formatter available yet.
let msg = bundle
.get_message("key-implicit")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
assert_eq!(value, "Here is an implicitly encoded number: 5.");
println!("{}", value);
// 3. Next, we're going to plug our custom formatter.
bundle.set_formatter(Some(custom_formatter));
// 4. Now, when you attempt to format a number, the custom formatter
// will be used instead of the default one.
let msg = bundle
.get_message("key-implicit")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
assert_eq!(value, "Here is an implicitly encoded number: CUSTOM(5).");
println!("{}", value);
// 5. The same custom formatter will be used for explicitly formatter numbers,
// and variables of type number.
let msg = bundle
.get_message("key-explicit")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
assert_eq!(value, "Here is an explicitly encoded number: CUSTOM(5).");
println!("{}", value);
let msg = bundle
.get_message("key-var-implicit")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let mut args = FluentArgs::new();
args.insert("num", FluentValue::from(-15));
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
assert_eq!(
value,
"Here is an implicitly encoded variable: CUSTOM(-15)."
);
println!("{}", value);
let msg = bundle
.get_message("key-var-explicit")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let mut args = FluentArgs::new();
args.insert("num", FluentValue::from(-15));
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
assert_eq!(
value,
"Here is an explicitly encoded variable: CUSTOM(-15)."
);
println!("{}", value);
// 6. The merging operation on FluentNumber options allows the
// options provided from the localizer to be merged into the
// default ones and ones provided by the developer.
let msg = bundle
.get_message("key-var-explicit")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let mut args = FluentArgs::new();
let num = FluentNumber::new(
25.2,
FluentNumberOptions {
maximum_fraction_digits: Some(8),
minimum_fraction_digits: Some(1),
..Default::default()
},
);
args.insert("num", num.into());
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
// Notice, that since we specificed minimum and maximum fraction digits options
// to be 1 and 8 when construction the argument, and then the minimum fraction
// digits option has been overridden in the localization the formatter
// will received options:
// - minimum_fraction_digits: Some(5)
// - maximum_fraction_digits: Some(8)
assert_eq!(
value,
"Here is an explicitly encoded variable: CUSTOM(25.2)."
);
println!("{}", value);
}

190
third_party/rust/fluent-bundle/examples/custom_type.rs vendored Normal file
View File

@ -0,0 +1,190 @@
// This is an example of an application which adds a custom type of value,
// and a function to format it.
//
// In this example we're going to add a new type - DateTime.
//
// We're also going to add a built-in function DATETIME to produce that type
// out of a number argument (epoch).
//
// Lastly, we'll also create a new formatter which will be memoizable.
//
// The type and its options are modelled after ECMA402 Intl.DateTimeFormat.
use intl_memoizer::Memoizable;
use unic_langid::LanguageIdentifier;
use fluent_bundle::types::FluentType;
use fluent_bundle::{FluentArgs, FluentBundle, FluentResource, FluentValue};
// First we're going to define what options our new type is going to accept.
// For the sake of the example, we're only going to allow two options:
// - dateStyle
// - timeStyle
//
// with an enum of allowed values.
#[derive(Debug, PartialEq, Eq, Clone, Hash)]
enum DateTimeStyleValue {
Full,
Long,
Medium,
Short,
None,
}
impl std::default::Default for DateTimeStyleValue {
fn default() -> Self {
Self::None
}
}
// This is just a helper to make it easier to convert
// a value provided to FluentArgs into an option value.
impl<'l> From<&FluentValue<'l>> for DateTimeStyleValue {
fn from(input: &FluentValue) -> Self {
if let FluentValue::String(s) = input {
match s.as_ref() {
"full" => Self::Full,
"long" => Self::Long,
"medium" => Self::Medium,
"short" => Self::Short,
_ => Self::None,
}
} else {
Self::None
}
}
}
#[derive(Debug, PartialEq, Eq, Default, Clone, Hash)]
struct DateTimeOptions {
pub date_style: DateTimeStyleValue,
pub time_style: DateTimeStyleValue,
}
impl DateTimeOptions {
// The merge function is going to be used by the Fluent Function
// to merge localizer provided options into defaults of values
// provided by the developer.
//
// If you want to limit which options the localizer can override,
// here's the right place to do it.
pub fn merge(&mut self, input: &FluentArgs) {
for (key, value) in input {
match *key {
"dateStyle" => self.date_style = value.into(),
"timeStyle" => self.time_style = value.into(),
_ => {}
}
}
}
}
impl<'l> From<&FluentArgs<'l>> for DateTimeOptions {
fn from(input: &FluentArgs) -> Self {
let mut opts = Self::default();
opts.merge(input);
opts
}
}
// Our new custom type will store a value as an epoch number,
// and the options.
#[derive(Debug, PartialEq, Clone)]
struct DateTime {
epoch: usize,
options: DateTimeOptions,
}
impl DateTime {
pub fn new(epoch: usize, options: DateTimeOptions) -> Self {
Self { epoch, options }
}
}
impl FluentType for DateTime {
fn duplicate(&self) -> Box<dyn FluentType> {
Box::new(DateTime::new(self.epoch, DateTimeOptions::default()))
}
fn as_string(&self, intls: &intl_memoizer::IntlLangMemoizer) -> std::borrow::Cow<'static, str> {
intls
.with_try_get::<DateTimeFormatter, _, _>((self.options.clone(),), |dtf| {
dtf.format(self.epoch).into()
})
.expect("Failed to format a date.")
}
fn as_string_threadsafe(
&self,
_: &intl_memoizer::concurrent::IntlLangMemoizer,
) -> std::borrow::Cow<'static, str> {
format!("2020-01-20 {}:00", self.epoch).into()
}
}
/// Formatter
struct DateTimeFormatter {
lang: LanguageIdentifier,
options: DateTimeOptions,
}
impl DateTimeFormatter {
pub fn new(lang: LanguageIdentifier, options: DateTimeOptions) -> Result<Self, ()> {
Ok(Self { lang, options })
}
pub fn format(&self, epoch: usize) -> String {
format!(
"My Custom Formatted Date from epoch: {}, in locale: {}, using options: {:#?}",
epoch, self.lang, self.options
)
}
}
impl Memoizable for DateTimeFormatter {
type Args = (DateTimeOptions,);
type Error = ();
fn construct(lang: LanguageIdentifier, args: Self::Args) -> Result<Self, Self::Error> {
Self::new(lang, args.0)
}
}
fn main() {
// 1. Bootstrap a FluentBundle with a number of messages which use
// number formatting in different forms.
let ftl_string = String::from(
r#"
key-date = Today is { DATETIME($epoch, dateStyle: "long", timeStyle: "short") }
"#,
);
let res = FluentResource::try_new(ftl_string).expect("Could not parse an FTL string.");
let lang: LanguageIdentifier = "en".parse().unwrap();
let mut bundle = FluentBundle::new(&[lang]);
bundle
.add_resource(res)
.expect("Failed to add FTL resources to the bundle.");
bundle
.add_function("DATETIME", |positional, named| match positional.get(0) {
Some(FluentValue::Number(n)) => {
let epoch = n.value as usize;
let options = named.into();
FluentValue::Custom(Box::new(DateTime::new(epoch, options)))
}
_ => FluentValue::None,
})
.expect("Failed to add a function.");
bundle.set_use_isolating(false);
let msg = bundle
.get_message("key-date")
.expect("Failed to retrieve the message.");
let pattern = msg.value.expect("Message has no value.");
let mut errors = vec![];
let mut args = FluentArgs::new();
let epoch: u64 = 1580127760093;
args.insert("epoch", epoch.into());
let value = bundle.format_pattern(pattern, Some(&args), &mut errors);
println!("{}", value);
}

50
third_party/rust/fluent-bundle/examples/external_arguments.rs vendored Normal file
View File

@ -0,0 +1,50 @@
use fluent_bundle::{FluentArgs, FluentBundle, FluentResource, FluentValue};
use unic_langid::langid;
fn main() {
let ftl_string = String::from(
"
hello-world = Hello { $name }
ref = The previous message says { hello-world }
unread-emails =
{ $emailCount ->
[one] You have { $emailCount } unread email
*[other] You have { $emailCount } unread emails
}
",
);
let res = FluentResource::try_new(ftl_string).expect("Could not parse an FTL string.");
let langid_en = langid!("en");
let mut bundle = FluentBundle::new(&[langid_en]);
bundle
.add_resource(res)
.expect("Failed to add FTL resources to the bundle.");
let mut args = FluentArgs::new();
args.insert("name", FluentValue::from("John"));
let msg = bundle
.get_message("hello-world")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
println!("{}", value);
let msg = bundle.get_message("ref").expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
println!("{}", value);
let mut args = FluentArgs::new();
args.insert("emailCount", 1.into());
let msg = bundle
.get_message("unread-emails")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
println!("{}", value);
}

82
third_party/rust/fluent-bundle/examples/functions.rs vendored Normal file
View File

@ -0,0 +1,82 @@
use fluent_bundle::{FluentBundle, FluentResource, FluentValue};
use unic_langid::langid;
fn main() {
// We define the resources here so that they outlive
// the bundle.
let ftl_string1 = String::from("hello-world = Hey there! { HELLO() }");
let ftl_string2 = String::from("meaning-of-life = { MEANING_OF_LIFE(42) }");
let ftl_string3 = String::from("all-your-base = { BASE_OWNERSHIP(hello, ownership: \"us\") }");
let res1 = FluentResource::try_new(ftl_string1).expect("Could not parse an FTL string.");
let res2 = FluentResource::try_new(ftl_string2).expect("Could not parse an FTL string.");
let res3 = FluentResource::try_new(ftl_string3).expect("Could not parse an FTL string.");
let langid_en_us = langid!("en-US");
let mut bundle = FluentBundle::new(&[langid_en_us]);
// Test for a simple function that returns a string
bundle
.add_function("HELLO", |_args, _named_args| {
return "I'm a function!".into();
})
.expect("Failed to add a function to the bundle.");
// Test for a function that accepts unnamed positional arguments
bundle
.add_function("MEANING_OF_LIFE", |args, _named_args| {
if let Some(arg0) = args.get(0) {
if *arg0 == 42.into() {
return "The answer to life, the universe, and everything".into();
}
}
FluentValue::None
})
.expect("Failed to add a function to the bundle.");
// Test for a function that accepts named arguments
bundle
.add_function("BASE_OWNERSHIP", |_args, named_args| {
return match named_args.get("ownership") {
Some(FluentValue::String(ref string)) => {
format!("All your base belong to {}", string).into()
}
_ => FluentValue::None,
};
})
.expect("Failed to add a function to the bundle.");
bundle
.add_resource(res1)
.expect("Failed to add FTL resources to the bundle.");
bundle
.add_resource(res2)
.expect("Failed to add FTL resources to the bundle.");
bundle
.add_resource(res3)
.expect("Failed to add FTL resources to the bundle.");
let msg = bundle
.get_message("hello-world")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
assert_eq!(&value, "Hey there! \u{2068}I'm a function!\u{2069}");
let msg = bundle
.get_message("meaning-of-life")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
assert_eq!(&value, "The answer to life, the universe, and everything");
let msg = bundle
.get_message("all-your-base")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
assert_eq!(&value, "All your base belong to us");
}

18
third_party/rust/fluent-bundle/examples/hello.rs vendored Normal file
View File

@ -0,0 +1,18 @@
use fluent_bundle::{FluentBundle, FluentResource};
fn main() {
let ftl_string = String::from("hello-world = Hello, world!");
let res = FluentResource::try_new(ftl_string).expect("Could not parse an FTL string.");
let mut bundle = FluentBundle::default();
bundle
.add_resource(&res)
.expect("Failed to add FTL resources to the bundle.");
let msg = bundle
.get_message("hello-world")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
assert_eq!(&value, "Hello, world!");
}

33
third_party/rust/fluent-bundle/examples/message_reference.rs vendored Normal file
View File

@ -0,0 +1,33 @@
use fluent_bundle::{FluentBundle, FluentResource};
fn main() {
let ftl_string = String::from(
"
foo = Foo
foobar = { foo } Bar
bazbar = { baz } Bar
",
);
let res = FluentResource::try_new(ftl_string).expect("Could not parse an FTL string.");
let mut bundle = FluentBundle::default();
bundle
.add_resource(res)
.expect("Failed to add FTL resources to the bundle.");
let msg = bundle
.get_message("foobar")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
println!("{}", value);
let msg = bundle
.get_message("bazbar")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
println!("{}", value);
}

7
third_party/rust/fluent-bundle/examples/resources/en-US/simple.ftl vendored Normal file
View File

@ -0,0 +1,7 @@
missing-arg-error = Error: Please provide a number as argument.
input-parse-error = Error: Could not parse input `{ $input }`. Reason: { $reason }
response-msg =
{ $value ->
[one] "{ $input }" has one Collatz step.
*[other] "{ $input }" has { $value } Collatz steps.
}

7
third_party/rust/fluent-bundle/examples/resources/fr/simple.ftl vendored Normal file
View File

@ -0,0 +1,7 @@
missing-arg-error = Erreur : veuillez saisir un nombre en paramètre.
input-parse-error = Erreur : impossible d'interpréter le paramètre `{ $input }`. Raison : { $reason }
response-msg =
{ $value ->
[one] La suite de Syracuse du nombre "{ $input }" comporte une valeur.
*[other] La suite de Syracuse du nombre "{ $input }" comporte { $value } valeurs.
}

8
third_party/rust/fluent-bundle/examples/resources/pl/simple.ftl vendored Normal file
View File

@ -0,0 +1,8 @@
missing-arg-error = Błąd: Proszę wprowadzić liczbę jako argument.
input-parse-error = Błąd: Nie udało się sparsować `{ $input }`. Powód: { $reason }
response-msg =
{ $value ->
[one] "{ $input }" ma jeden krok Collatza.
[few] "{ $input }" ma { $value } kroki Collatza.
*[many] "{ $input }" ma { $value } kroków Collatza.
}

41
third_party/rust/fluent-bundle/examples/selector.rs vendored Normal file
View File

@ -0,0 +1,41 @@
use fluent_bundle::{FluentArgs, FluentBundle, FluentResource, FluentValue};
fn main() {
let ftl_string = String::from(
"
hello-world = Hello { $missing ->
*[one] World
[two] Moon
}
hello-world2 = Hello { $name ->
*[world] World
[moon] Moon
}
",
);
let res = FluentResource::try_new(ftl_string).expect("Could not parse an FTL string.");
let mut bundle = FluentBundle::default();
bundle
.add_resource(res)
.expect("Failed to add FTL resources to the bundle.");
let msg = bundle
.get_message("hello-world")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
println!("{}", value);
let mut args = FluentArgs::new();
args.insert("name", FluentValue::from("moon"));
let msg = bundle
.get_message("hello-world2")
.expect("Message doesn't exist.");
let mut errors = vec![];
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
println!("{}", value);
}

179
third_party/rust/fluent-bundle/examples/simple-app.rs vendored Normal file
View File

@ -0,0 +1,179 @@
//! This is an example of a simple application
//! which calculates the Collatz conjecture.
//!
//! The function itself is trivial on purpose,
//! so that we can focus on understanding how
//! the application can be made localizable
//! via Fluent.
//!
//! To try the app launch `cargo run --example simple-app NUM (LOCALES)`
//!
//! NUM is a number to be calculated, and LOCALES is an optional
//! parameter with a comma-separated list of locales requested by the user.
//!
//! Example:
//!
//! cargo run --example simple-app 123 de,pl
//!
//! If the second argument is omitted, `en-US` locale is used as the
//! default one.
use fluent_bundle::{FluentArgs, FluentBundle, FluentResource, FluentValue};
use fluent_langneg::{negotiate_languages, NegotiationStrategy};
use std::env;
use std::fs;
use std::fs::File;
use std::io;
use std::io::prelude::*;
use std::path::Path;
use std::str::FromStr;
use unic_langid::{langid, LanguageIdentifier};
/// We need a generic file read helper function to
/// read the localization resource file.
///
/// The resource files are stored in
/// `./examples/resources/{locale}` directory.
fn read_file(path: &Path) -> Result<String, io::Error> {
let mut f = File::open(path)?;
let mut s = String::new();
f.read_to_string(&mut s)?;
Ok(s)
}
/// This helper function allows us to read the list
/// of available locales by reading the list of
/// directories in `./examples/resources`.
///
/// It is expected that every directory inside it
/// has a name that is a valid BCP47 language tag.
fn get_available_locales() -> Result<Vec<LanguageIdentifier>, io::Error> {
let mut locales = vec![];
let mut dir = env::current_dir()?;
if dir.to_string_lossy().ends_with("fluent-rs") {
dir.push("fluent-bundle");
}
dir.push("examples");
dir.push("resources");
let res_dir = fs::read_dir(dir)?;
for entry in res_dir {
if let Ok(entry) = entry {
let path = entry.path();
if path.is_dir() {
if let Some(name) = path.file_name() {
if let Some(name) = name.to_str() {
let langid = name.parse().expect("Parsing failed.");
locales.push(langid);
}
}
}
}
}
return Ok(locales);
}
static L10N_RESOURCES: &[&str] = &["simple.ftl"];
fn main() {
// 1. Get the command line arguments.
let args: Vec<String> = env::args().collect();
// 3. If the argument length is more than 1,
// take the second argument as a comma-separated
// list of requested locales.
let requested = args.get(2).map_or(vec![], |arg| {
arg.split(",")
.map(|s| -> LanguageIdentifier { s.parse().expect("Parsing locale failed.") })
.collect()
});
// 4. Negotiate it against the available ones
let default_locale = langid!("en-US");
let available = get_available_locales().expect("Retrieving available locales failed.");
let resolved_locales = negotiate_languages(
&requested,
&available,
Some(&default_locale),
NegotiationStrategy::Filtering,
);
let current_locale = resolved_locales
.get(0)
.expect("At least one locale should match.");
// 5. Create a new Fluent FluentBundle using the
// resolved locales.
let mut bundle = FluentBundle::new(resolved_locales.clone());
// 6. Load the localization resource
for path in L10N_RESOURCES {
let mut full_path = env::current_dir().expect("Failed to retireve current dir.");
if full_path.to_string_lossy().ends_with("fluent-rs") {
full_path.push("fluent-bundle");
}
full_path.push("examples");
full_path.push("resources");
full_path.push(current_locale.to_string());
full_path.push(path);
let source = read_file(&full_path).expect("Failed to read file.");
let resource = FluentResource::try_new(source).expect("Could not parse an FTL string.");
bundle
.add_resource(resource)
.expect("Failed to add FTL resources to the bundle.");
}
// 7. Check if the input is provided.
match args.get(1) {
Some(input) => {
// 7.1. Cast it to a number.
match isize::from_str(&input) {
Ok(i) => {
// 7.2. Construct a map of arguments
// to format the message.
let mut args = FluentArgs::new();
args.insert("input", FluentValue::from(i));
args.insert("value", FluentValue::from(collatz(i)));
// 7.3. Format the message.
let mut errors = vec![];
let msg = bundle
.get_message("response-msg")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
println!("{}", value);
}
Err(err) => {
let mut args = FluentArgs::new();
args.insert("input", FluentValue::from(input.as_str()));
args.insert("reason", FluentValue::from(err.to_string()));
let mut errors = vec![];
let msg = bundle
.get_message("input-parse-error")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
println!("{}", value);
}
}
}
None => {
let mut errors = vec![];
let msg = bundle
.get_message("missing-arg-error")
.expect("Message doesn't exist.");
let pattern = msg.value.expect("Message has no value.");
let value = bundle.format_pattern(&pattern, None, &mut errors);
println!("{}", value);
}
}
}
/// Collatz conjecture calculating function.
fn collatz(n: isize) -> isize {
match n {
1 => 0,
_ => match n % 2 {
0 => 1 + collatz(n / 2),
_ => 1 + collatz(n * 3 + 1),
},
}
}

524
third_party/rust/fluent-bundle/src/bundle.rs vendored Normal file
View File

@ -0,0 +1,524 @@
//! `FluentBundle` is a collection of localization messages in Fluent.
//!
//! It stores a list of messages in a single locale which can reference one another, use the same
//! internationalization formatters, functions, scopeironmental variables and are expected to be used
//! together.
use std::borrow::Borrow;
use std::borrow::Cow;
use std::collections::hash_map::{Entry as HashEntry, HashMap};
use std::default::Default;
use fluent_syntax::ast;
use unic_langid::LanguageIdentifier;
use crate::entry::Entry;
use crate::entry::GetEntry;
use crate::errors::FluentError;
use crate::memoizer::MemoizerKind;
use crate::resolve::{ResolveValue, Scope};
use crate::resource::FluentResource;
use crate::types::FluentValue;
/// A single localization unit composed of an identifier,
/// value, and attributes.
#[derive(Debug, PartialEq)]
pub struct FluentMessage<'m> {
pub value: Option<&'m ast::Pattern<'m>>,
pub attributes: HashMap<&'m str, &'m ast::Pattern<'m>>,
}
/// A map of arguments passed from the code to
/// the localization to be used for message
/// formatting.
pub type FluentArgs<'args> = HashMap<&'args str, FluentValue<'args>>;
/// A collection of localization messages for a single locale, which are meant
/// to be used together in a single view, widget or any other UI abstraction.
///
/// # Examples
///
/// ```
/// use fluent_bundle::{FluentBundle, FluentResource, FluentValue};
/// use std::collections::HashMap;
/// use unic_langid::langid;
///
/// let ftl_string = String::from("intro = Welcome, { $name }.");
/// let resource = FluentResource::try_new(ftl_string)
/// .expect("Could not parse an FTL string.");
///
/// let langid_en = langid!("en-US");
/// let mut bundle = FluentBundle::new(&[langid_en]);
///
/// bundle.add_resource(&resource)
/// .expect("Failed to add FTL resources to the bundle.");
///
/// let mut args = HashMap::new();
/// args.insert("name", FluentValue::from("Rustacean"));
///
/// let msg = bundle.get_message("intro").expect("Message doesn't exist.");
/// let mut errors = vec![];
/// let pattern = msg.value.expect("Message has no value.");
/// let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
/// assert_eq!(&value, "Welcome, \u{2068}Rustacean\u{2069}.");
///
/// ```
///
/// # `FluentBundle` Life Cycle
///
/// ## Create a bundle
///
/// To create a bundle, call [`FluentBundle::new`] with a locale list that represents the best
/// possible fallback chain for a given locale. The simplest case is a one-locale list.
///
/// Fluent uses [`LanguageIdentifier`] which can be created using `langid!` macro.
///
/// ## Add Resources
///
/// Next, call [`add_resource`] one or more times, supplying translations in the FTL syntax.
///
/// Since [`FluentBundle`] is generic over anything that can borrow a [`FluentResource`],
/// one can use [`FluentBundle`] to own its resources, store references to them,
/// or even [`Rc<FluentResource>`] or [`Arc<FluentResource>`].
///
/// The [`FluentBundle`] instance is now ready to be used for localization.
///
/// ## Format
///
/// To format a translation, call [`get_message`] to retrieve a [`FluentMessage`],
/// and then call [`format_pattern`] on the message value or attribute in order to
/// retrieve the translated string.
///
/// The result of [`format_pattern`] is an [`Cow<str>`]. It is
/// recommended to treat the result as opaque from the perspective of the program and use it only
/// to display localized messages. Do not examine it or alter in any way before displaying. This
/// is a general good practice as far as all internationalization operations are concerned.
///
/// If errors were encountered during formatting, they will be
/// accumulated in the [`Vec<FluentError>`] passed as the third argument.
///
/// While they are not fatal, they usually indicate problems with the translation,
/// and should be logged or reported in a way that allows the developer to notice
/// and fix them.
///
///
/// # Locale Fallback Chain
///
/// [`FluentBundle`] stores messages in a single locale, but keeps a locale fallback chain for the
/// purpose of language negotiation with i18n formatters. For instance, if date and time formatting
/// are not available in the first locale, [`FluentBundle`] will use its `locales` fallback chain
/// to negotiate a sensible fallback for date and time formatting.
///
/// [`add_resource`]: ./struct.FluentBundle.html#method.add_resource
/// [`FluentBundle::new`]: ./struct.FluentBundle.html#method.new
/// [`FluentMessage`]: ./struct.FluentMessage.html
/// [`FluentBundle`]: ./struct.FluentBundle.html
/// [`FluentResource`]: ./struct.FluentResource.html
/// [`get_message`]: ./struct.FluentBundle.html#method.get_message
/// [`format_pattern`]: ./struct.FluentBundle.html#method.format_pattern
/// [`add_resource`]: ./struct.FluentBundle.html#method.add_resource
/// [`Cow<str>`]: http://doc.rust-lang.org/std/borrow/enum.Cow.html
/// [`Rc<FluentResource>`]: https://doc.rust-lang.org/std/rc/struct.Rc.html
/// [`Arc<FluentResource>`]: https://doc.rust-lang.org/std/sync/struct.Arc.html
/// [`LanguageIdentifier`]: https://crates.io/crates/unic-langid
/// [`Vec<FluentError>`]: ./enum.FluentError.html
pub struct FluentBundleBase<R, M> {
pub locales: Vec<LanguageIdentifier>,
pub(crate) resources: Vec<R>,
pub(crate) entries: HashMap<String, Entry>,
pub(crate) intls: M,
pub(crate) use_isolating: bool,
pub(crate) transform: Option<fn(&str) -> Cow<str>>,
pub(crate) formatter: Option<fn(&FluentValue, &M) -> Option<String>>,
}
impl<R, M: MemoizerKind> FluentBundleBase<R, M> {
/// Constructs a FluentBundle. `locales` is the fallback chain of locales
/// to use for formatters like date and time. `locales` does not influence
/// message selection.
///
/// # Examples
///
/// ```
/// use fluent_bundle::FluentBundle;
/// use fluent_bundle::FluentResource;
/// use unic_langid::langid;
///
/// let langid_en = langid!("en-US");
/// let mut bundle: FluentBundle<FluentResource> = FluentBundle::new(&[langid_en]);
/// ```
///
/// # Errors
///
/// This will panic if no formatters can be found for the locales.
pub fn new<'a, L: 'a + Into<LanguageIdentifier> + PartialEq + Clone>(
locales: impl IntoIterator<Item = &'a L>,
) -> Self {
let locales = locales
.into_iter()
.map(|s| s.clone().into())
.collect::<Vec<_>>();
let lang = locales.get(0).cloned().unwrap_or_default();
FluentBundleBase {
locales,
resources: vec![],
entries: HashMap::new(),
intls: M::new(lang),
use_isolating: true,
transform: None,
formatter: None,
}
}
/// Adds a resource to the bundle, returning an empty [`Result<T>`] on success.
///
/// If any entry in the resource uses the same identifier as an already
/// existing key in the bundle, the new entry will be ignored and a
/// `FluentError::Overriding` will be added to the result.
///
/// The method can take any type that can be borrowed to FluentResource:
/// - FluentResource
/// - &FluentResource
/// - Rc<FluentResource>
/// - Arc<FluentResurce>
///
/// This allows the user to introduce custom resource management and share
/// resources between instances of `FluentBundle`.
///
/// # Examples
///
/// ```
/// use fluent_bundle::{FluentBundle, FluentResource};
/// use unic_langid::langid;
///
/// let ftl_string = String::from("
/// hello = Hi!
/// goodbye = Bye!
/// ");
/// let resource = FluentResource::try_new(ftl_string)
/// .expect("Could not parse an FTL string.");
/// let langid_en = langid!("en-US");
/// let mut bundle = FluentBundle::new(&[langid_en]);
/// bundle.add_resource(resource)
/// .expect("Failed to add FTL resources to the bundle.");
/// assert_eq!(true, bundle.has_message("hello"));
/// ```
///
/// # Whitespace
///
/// Message ids must have no leading whitespace. Message values that span
/// multiple lines must have leading whitespace on all but the first line. These
/// are standard FTL syntax rules that may prove a bit troublesome in source
/// code formatting. The [`indoc!`] crate can help with stripping extra indentation
/// if you wish to indent your entire message.
///
/// [FTL syntax]: https://projectfluent.org/fluent/guide/
/// [`indoc!`]: https://github.com/dtolnay/indoc
/// [`Result<T>`]: https://doc.rust-lang.org/std/result/enum.Result.html
pub fn add_resource(&mut self, r: R) -> Result<(), Vec<FluentError>>
where
R: Borrow<FluentResource>,
{
let mut errors = vec![];
let res = r.borrow();
let res_pos = self.resources.len();
for (entry_pos, entry) in res.ast().body.iter().enumerate() {
let id = match entry {
ast::ResourceEntry::Entry(ast::Entry::Message(ast::Message { ref id, .. }))
| ast::ResourceEntry::Entry(ast::Entry::Term(ast::Term { ref id, .. })) => id.name,
_ => continue,
};
let (entry, kind) = match entry {
ast::ResourceEntry::Entry(ast::Entry::Message(..)) => {
(Entry::Message([res_pos, entry_pos]), "message")
}
ast::ResourceEntry::Entry(ast::Entry::Term(..)) => {
(Entry::Term([res_pos, entry_pos]), "term")
}
_ => continue,
};
match self.entries.entry(id.to_string()) {
HashEntry::Vacant(empty) => {
empty.insert(entry);
}
HashEntry::Occupied(_) => {
errors.push(FluentError::Overriding {
kind,
id: id.to_string(),
});
}
}
}
self.resources.push(r);
if errors.is_empty() {
Ok(())
} else {
Err(errors)
}
}
/// Adds a resource to the bundle, returning an empty [`Result<T>`] on success.
///
/// If any entry in the resource uses the same identifier as an already
/// existing key in the bundle, the entry will override the previous one.
///
/// The method can take any type that can be borrowed as FluentResource:
/// - FluentResource
/// - &FluentResource
/// - Rc<FluentResource>
/// - Arc<FluentResurce>
///
/// This allows the user to introduce custom resource management and share
/// resources between instances of `FluentBundle`.
///
/// # Examples
///
/// ```
/// use fluent_bundle::{FluentBundle, FluentResource};
/// use unic_langid::langid;
///
/// let ftl_string = String::from("
/// hello = Hi!
/// goodbye = Bye!
/// ");
/// let resource = FluentResource::try_new(ftl_string)
/// .expect("Could not parse an FTL string.");
///
/// let ftl_string = String::from("
/// hello = Another Hi!
/// ");
/// let resource2 = FluentResource::try_new(ftl_string)
/// .expect("Could not parse an FTL string.");
///
/// let langid_en = langid!("en-US");
///
/// let mut bundle = FluentBundle::new(&[langid_en]);
/// bundle.add_resource(resource)
/// .expect("Failed to add FTL resources to the bundle.");
///
/// bundle.add_resource_overriding(resource2);
///
/// let mut errors = vec![];
/// let msg = bundle.get_message("hello")
/// .expect("Failed to retrieve the message");
/// let value = msg.value.expect("Failed to retrieve the value of the message");
/// assert_eq!(bundle.format_pattern(value, None, &mut errors), "Another Hi!");
/// ```
///
/// # Whitespace
///
/// Message ids must have no leading whitespace. Message values that span
/// multiple lines must have leading whitespace on all but the first line. These
/// are standard FTL syntax rules that may prove a bit troublesome in source
/// code formatting. The [`indoc!`] crate can help with stripping extra indentation
/// if you wish to indent your entire message.
///
/// [FTL syntax]: https://projectfluent.org/fluent/guide/
/// [`indoc!`]: https://github.com/dtolnay/indoc
/// [`Result<T>`]: https://doc.rust-lang.org/std/result/enum.Result.html
pub fn add_resource_overriding(&mut self, r: R)
where
R: Borrow<FluentResource>,
{
let res = r.borrow();
let res_pos = self.resources.len();
for (entry_pos, entry) in res.ast().body.iter().enumerate() {
let id = match entry {
ast::ResourceEntry::Entry(ast::Entry::Message(ast::Message { ref id, .. }))
| ast::ResourceEntry::Entry(ast::Entry::Term(ast::Term { ref id, .. })) => id.name,
_ => continue,
};
let entry = match entry {
ast::ResourceEntry::Entry(ast::Entry::Message(..)) => {
Entry::Message([res_pos, entry_pos])
}
ast::ResourceEntry::Entry(ast::Entry::Term(..)) => {
Entry::Term([res_pos, entry_pos])
}
_ => continue,
};
self.entries.insert(id.to_string(), entry);
}
self.resources.push(r);
}
/// When formatting patterns, `FluentBundle` inserts
/// Unicode Directionality Isolation Marks to indicate
/// that the direction of a placeable may differ from
/// the surrounding message.
///
/// This is important for cases such as when a
/// right-to-left user name is presented in the
/// left-to-right message.
///
/// In some cases, such as testing, the user may want
/// to disable the isolating.
pub fn set_use_isolating(&mut self, value: bool) {
self.use_isolating = value;
}
/// This method allows to specify a function that will
/// be called on all textual fragments of the pattern
/// during formatting.
///
/// This is currently primarly used for pseudolocalization,
/// and `fluent-pseudo` crate provides a function
/// that can be passed here.
pub fn set_transform(&mut self, func: Option<fn(&str) -> Cow<str>>) {
if let Some(f) = func {
self.transform = Some(f);
} else {
self.transform = None;
}
}
/// This method allows to specify a function that will
/// be called before any `FluentValue` is formatted
/// allowing overrides.
///
/// It's particularly useful for plugging in an external
/// formatter for `FluentValue::Number`.
pub fn set_formatter(&mut self, func: Option<fn(&FluentValue, &M) -> Option<String>>) {
if let Some(f) = func {
self.formatter = Some(f);
} else {
self.formatter = None;
}
}
/// Returns true if this bundle contains a message with the given id.
///
/// # Examples
///
/// ```
/// use fluent_bundle::{FluentBundle, FluentResource};
/// use unic_langid::langid;
///
/// let ftl_string = String::from("hello = Hi!");
/// let resource = FluentResource::try_new(ftl_string)
/// .expect("Failed to parse an FTL string.");
/// let langid_en = langid!("en-US");
/// let mut bundle = FluentBundle::new(&[langid_en]);
/// bundle.add_resource(&resource)
/// .expect("Failed to add FTL resources to the bundle.");
/// assert_eq!(true, bundle.has_message("hello"));
///
/// ```
pub fn has_message(&self, id: &str) -> bool
where
R: Borrow<FluentResource>,
{
self.get_entry_message(id).is_some()
}
pub fn get_message(&self, id: &str) -> Option<FluentMessage>
where
R: Borrow<FluentResource>,
{
let message = self.get_entry_message(id)?;
let value = message.value.as_ref();
let mut attributes = if message.attributes.is_empty() {
HashMap::new()
} else {
HashMap::with_capacity(message.attributes.len())
};
for attr in message.attributes.iter() {
attributes.insert(attr.id.name, &attr.value);
}
Some(FluentMessage { value, attributes })
}
pub fn format_pattern<'bundle>(
&'bundle self,
pattern: &'bundle ast::Pattern,
args: Option<&'bundle FluentArgs>,
errors: &mut Vec<FluentError>,
) -> Cow<'bundle, str>
where
R: Borrow<FluentResource>,
{
let mut scope = Scope::new(self, args);
let result = pattern.resolve(&mut scope).as_string(&scope);
for err in scope.errors {
errors.push(err.into());
}
result
}
/// Makes the provided rust function available to messages with the name `id`. See
/// the [FTL syntax guide] to learn how these are used in messages.
///
/// FTL functions accept both positional and named args. The rust function you
/// provide therefore has two parameters: a slice of values for the positional
/// args, and a HashMap of values for named args.
///
/// # Examples
///
/// ```
/// use fluent_bundle::{FluentBundle, FluentResource, FluentValue};
/// use unic_langid::langid;
///
/// let ftl_string = String::from("length = { STRLEN(\"12345\") }");
/// let resource = FluentResource::try_new(ftl_string)
/// .expect("Could not parse an FTL string.");
/// let langid_en = langid!("en-US");
/// let mut bundle = FluentBundle::new(&[langid_en]);
/// bundle.add_resource(&resource)
/// .expect("Failed to add FTL resources to the bundle.");
///
/// // Register a fn that maps from string to string length
/// bundle.add_function("STRLEN", |positional, _named| match positional {
/// [FluentValue::String(str)] => str.len().into(),
/// _ => FluentValue::None,
/// }).expect("Failed to add a function to the bundle.");
///
/// let msg = bundle.get_message("length").expect("Message doesn't exist.");
/// let mut errors = vec![];
/// let pattern = msg.value.expect("Message has no value.");
/// let value = bundle.format_pattern(&pattern, None, &mut errors);
/// assert_eq!(&value, "5");
/// ```
///
/// [FTL syntax guide]: https://projectfluent.org/fluent/guide/functions.html
pub fn add_function<F>(&mut self, id: &str, func: F) -> Result<(), FluentError>
where
F: for<'a> Fn(&[FluentValue<'a>], &FluentArgs) -> FluentValue<'a> + Sync + Send + 'static,
{
match self.entries.entry(id.to_owned()) {
HashEntry::Vacant(entry) => {
entry.insert(Entry::Function(Box::new(func)));
Ok(())
}
HashEntry::Occupied(_) => Err(FluentError::Overriding {
kind: "function",
id: id.to_owned(),
}),
}
}
}
impl<R, M: MemoizerKind> Default for FluentBundleBase<R, M> {
fn default() -> Self {
let langid = LanguageIdentifier::default();
FluentBundleBase {
locales: vec![langid.clone()],
resources: vec![],
entries: Default::default(),
use_isolating: true,
intls: M::new(langid),
transform: None,
formatter: None,
}
}
}

31
third_party/rust/fluent-bundle/src/concurrent.rs vendored Normal file
View File

@ -0,0 +1,31 @@
use intl_memoizer::{concurrent::IntlLangMemoizer, Memoizable};
use unic_langid::LanguageIdentifier;
use crate::bundle::FluentBundleBase;
use crate::memoizer::MemoizerKind;
use crate::types::FluentType;
pub type FluentBundle<R> = FluentBundleBase<R, IntlLangMemoizer>;
impl MemoizerKind for IntlLangMemoizer {
fn new(lang: LanguageIdentifier) -> Self
where
Self: Sized,
{
IntlLangMemoizer::new(lang)
}
fn with_try_get_threadsafe<I, R, U>(&self, args: I::Args, cb: U) -> Result<R, I::Error>
where
Self: Sized,
I: Memoizable + Send + Sync + 'static,
I::Args: Send + Sync + 'static,
U: FnOnce(&I) -> R,
{
self.with_try_get(args, cb)
}
fn stringify_value(&self, value: &dyn FluentType) -> std::borrow::Cow<'static, str> {
value.as_string_threadsafe(self)
}
}

65
third_party/rust/fluent-bundle/src/entry.rs vendored Normal file
View File

@ -0,0 +1,65 @@
//! `Entry` is used to store Messages, Terms and Functions in `FluentBundle` instances.
use std::borrow::Borrow;
use fluent_syntax::ast;
use crate::bundle::{FluentArgs, FluentBundleBase};
use crate::resource::FluentResource;
use crate::types::FluentValue;
pub type FluentFunction =
Box<dyn for<'a> Fn(&[FluentValue<'a>], &FluentArgs) -> FluentValue<'a> + Send + Sync>;
pub enum Entry {
Message([usize; 2]),
Term([usize; 2]),
Function(FluentFunction),
}
pub trait GetEntry {
fn get_entry_message(&self, id: &str) -> Option<&ast::Message>;
fn get_entry_term(&self, id: &str) -> Option<&ast::Term>;
fn get_entry_function(&self, id: &str) -> Option<&FluentFunction>;
}
impl<'bundle, R: Borrow<FluentResource>, M> GetEntry for FluentBundleBase<R, M> {
fn get_entry_message(&self, id: &str) -> Option<&ast::Message> {
self.entries.get(id).and_then(|entry| match *entry {
Entry::Message(pos) => {
let res = self.resources.get(pos[0])?.borrow();
if let Some(ast::ResourceEntry::Entry(ast::Entry::Message(ref msg))) =
res.ast().body.get(pos[1])
{
Some(msg)
} else {
None
}
}
_ => None,
})
}
fn get_entry_term(&self, id: &str) -> Option<&ast::Term> {
self.entries.get(id).and_then(|entry| match *entry {
Entry::Term(pos) => {
let res = self.resources.get(pos[0])?.borrow();
if let Some(ast::ResourceEntry::Entry(ast::Entry::Term(ref msg))) =
res.ast().body.get(pos[1])
{
Some(msg)
} else {
None
}
}
_ => None,
})
}
fn get_entry_function(&self, id: &str) -> Option<&FluentFunction> {
self.entries.get(id).and_then(|entry| match entry {
Entry::Function(function) => Some(function),
_ => None,
})
}
}

21
third_party/rust/fluent-bundle/src/errors.rs vendored Normal file
View File

@ -0,0 +1,21 @@
use crate::resolve::ResolverError;
use fluent_syntax::parser::ParserError;
#[derive(Debug, PartialEq)]
pub enum FluentError {
Overriding { kind: &'static str, id: String },
ParserError(ParserError),
ResolverError(ResolverError),
}
impl From<ResolverError> for FluentError {
fn from(error: ResolverError) -> Self {
FluentError::ResolverError(error)
}
}
impl From<ParserError> for FluentError {
fn from(error: ParserError) -> Self {
FluentError::ParserError(error)
}
}

125
third_party/rust/fluent-bundle/src/lib.rs vendored Normal file
View File

@ -0,0 +1,125 @@
//! Fluent is a modern localization system designed to improve how software is translated.
//!
//! The Rust implementation provides the low level components for syntax operations, like parser
//! and AST, and the core localization struct - [`FluentBundle`].
//!
//! [`FluentBundle`] is the low level container for storing and formatting localization messages
//! in a single locale.
//!
//! This crate provides also a number of structures needed for a localization API such as [`FluentResource`],
//! [`FluentMessage`], [`FluentArgs`], and [`FluentValue`].
//!
//! Together, they allow implementations to build higher-level APIs that use [`FluentBundle`]
//! and add user friendly helpers, framework bindings, error fallbacking,
//! language negotiation between user requested languages and available resources,
//! and I/O for loading selected resources.
//!
//! # Example
//!
//! ```
//! use fluent_bundle::{FluentBundle, FluentValue, FluentResource, FluentArgs};
//!
//! // Used to provide a locale for the bundle.
//! use unic_langid::langid;
//!
//! let ftl_string = String::from("
//! hello-world = Hello, world!
//! intro = Welcome, { $name }.
//! ");
//! let res = FluentResource::try_new(ftl_string)
//! .expect("Failed to parse an FTL string.");
//!
//! let langid_en = langid!("en-US");
//! let mut bundle = FluentBundle::new(&[langid_en]);
//!
//! bundle
//! .add_resource(res)
//! .expect("Failed to add FTL resources to the bundle.");
//!
//! let msg = bundle.get_message("hello-world")
//! .expect("Message doesn't exist.");
//! let mut errors = vec![];
//! let pattern = msg.value
//! .expect("Message has no value.");
//! let value = bundle.format_pattern(&pattern, None, &mut errors);
//!
//! assert_eq!(&value, "Hello, world!");
//!
//! let mut args = FluentArgs::new();
//! args.insert("name", FluentValue::from("John"));
//!
//! let msg = bundle.get_message("intro")
//! .expect("Message doesn't exist.");
//! let mut errors = vec![];
//! let pattern = msg.value.expect("Message has no value.");
//! let value = bundle.format_pattern(&pattern, Some(&args), &mut errors);
//!
//! // The FSI/PDI isolation marks ensure that the direction of
//! // the text from the variable is not affected by the translation.
//! assert_eq!(value, "Welcome, \u{2068}John\u{2069}.");
//! ```
//!
//! # Ergonomics & Higher Level APIs
//!
//! Reading the example, you may notice how verbose it feels.
//! Many core methods are fallible, others accumulate errors, and there
//! are intermediate structures used in operations.
//!
//! This is intentional as it serves as building blocks for variety of different
//! scenarios allowing implementations to handle errors, cache and
//! optimize results.
//!
//! At the moment it is expected that users will use
//! the `fluent-bundle` crate directly, while the ecosystem
//! matures and higher level APIs are being developed.
//!
//! [`FluentBundle`]: ./bundle/struct.FluentBundle.html
//! [`FluentResource`]: ./bundle/struct.FluentResource.html
//! [`FluentMessage`]: ./bundle/struct.FluentMessage.html
//! [`FluentArgs`]: ./bundle/type.FluentArgs.html
//! [`FluentValue`]: ./bundle/struct.FluentValue.html
#[macro_use]
extern crate rental;
use intl_memoizer::{IntlLangMemoizer, Memoizable};
use unic_langid::LanguageIdentifier;
mod bundle;
pub mod concurrent;
mod entry;
mod errors;
pub mod memoizer;
pub mod resolve;
mod resource;
pub mod types;
pub use bundle::{FluentArgs, FluentMessage};
pub use errors::FluentError;
pub use resource::FluentResource;
pub use types::FluentValue;
pub type FluentBundle<R> = bundle::FluentBundleBase<R, IntlLangMemoizer>;
impl memoizer::MemoizerKind for IntlLangMemoizer {
fn new(lang: LanguageIdentifier) -> Self
where
Self: Sized,
{
IntlLangMemoizer::new(lang)
}
fn with_try_get_threadsafe<I, R, U>(&self, args: I::Args, cb: U) -> Result<R, I::Error>
where
Self: Sized,
I: Memoizable + Send + Sync + 'static,
I::Args: Send + Sync + 'static,
U: FnOnce(&I) -> R,
{
self.with_try_get(args, cb)
}
fn stringify_value(&self, value: &dyn types::FluentType) -> std::borrow::Cow<'static, str> {
value.as_string(self)
}
}

18
third_party/rust/fluent-bundle/src/memoizer.rs vendored Normal file
View File

@ -0,0 +1,18 @@
use crate::types::FluentType;
use intl_memoizer::Memoizable;
use unic_langid::LanguageIdentifier;
pub trait MemoizerKind: 'static {
fn new(lang: LanguageIdentifier) -> Self
where
Self: Sized;
fn with_try_get_threadsafe<I, R, U>(&self, args: I::Args, cb: U) -> Result<R, I::Error>
where
Self: Sized,
I: Memoizable + Send + Sync + 'static,
I::Args: Send + Sync + 'static,
U: FnOnce(&I) -> R;
fn stringify_value(&self, value: &dyn FluentType) -> std::borrow::Cow<'static, str>;
}

350
third_party/rust/fluent-bundle/src/resolve.rs vendored Normal file
View File

@ -0,0 +1,350 @@
//! The `ResolveValue` trait resolves Fluent AST nodes to [`FluentValues`].
//!
//! This is an internal API used by [`FluentBundle`] to evaluate Messages, Attributes and other
//! AST nodes to [`FluentValues`] which can be then formatted to strings.
//!
//! [`FluentValues`]: ../types/enum.FluentValue.html
//! [`FluentBundle`]: ../bundle/struct.FluentBundle.html
use std::borrow::Borrow;
use std::fmt::Write;
use fluent_syntax::ast;
use fluent_syntax::unicode::unescape_unicode;
use crate::bundle::{FluentArgs, FluentBundleBase};
use crate::entry::GetEntry;
use crate::memoizer::MemoizerKind;
use crate::resource::FluentResource;
use crate::types::DisplayableNode;
use crate::types::FluentValue;
const MAX_PLACEABLES: u8 = 100;
#[derive(Debug, PartialEq, Clone)]
pub enum ResolverError {
Reference(String),
MissingDefault,
Cyclic,
TooManyPlaceables,
}
/// State for a single `ResolveValue::to_value` call.
pub struct Scope<'bundle, R: Borrow<FluentResource>, M> {
/// The current `FluentBundleBase` instance.
pub bundle: &'bundle FluentBundleBase<R, M>,
/// The current arguments passed by the developer.
args: Option<&'bundle FluentArgs<'bundle>>,
/// Local args
local_args: Option<FluentArgs<'bundle>>,
/// The running count of resolved placeables. Used to detect the Billion
/// Laughs and Quadratic Blowup attacks.
placeables: u8,
/// Tracks hashes to prevent infinite recursion.
travelled: smallvec::SmallVec<[&'bundle ast::Pattern<'bundle>; 2]>,
/// Track errors accumulated during resolving.
pub errors: Vec<ResolverError>,
/// Makes the resolver bail.
pub dirty: bool,
}
impl<'bundle, R: Borrow<FluentResource>, M: MemoizerKind> Scope<'bundle, R, M> {
pub fn new(bundle: &'bundle FluentBundleBase<R, M>, args: Option<&'bundle FluentArgs>) -> Self {
Scope {
bundle,
args,
local_args: None,
placeables: 0,
travelled: Default::default(),
errors: vec![],
dirty: false,
}
}
// This method allows us to lazily add Pattern on the stack,
// only if the Pattern::resolve has been called on an empty stack.
//
// This is the case when pattern is called from Bundle and it
// allows us to fast-path simple resolutions, and only use the stack
// for placeables.
pub fn maybe_track(
&mut self,
pattern: &'bundle ast::Pattern,
placeable: &'bundle ast::Expression,
) -> FluentValue<'bundle> {
if self.travelled.is_empty() {
self.travelled.push(pattern);
}
let result = placeable.resolve(self);
if self.dirty {
return FluentValue::Error(placeable.into());
}
result
}
pub fn track(
&mut self,
pattern: &'bundle ast::Pattern,
entry: DisplayableNode<'bundle>,
) -> FluentValue<'bundle> {
if self.travelled.contains(&pattern) {
self.errors.push(ResolverError::Cyclic);
FluentValue::Error(entry)
} else {
self.travelled.push(pattern);
let result = pattern.resolve(self);
self.travelled.pop();
result
}
}
}
fn generate_ref_error<'source, R, M>(
scope: &mut Scope<'source, R, M>,
node: DisplayableNode<'source>,
) -> FluentValue<'source>
where
R: Borrow<FluentResource>,
{
scope
.errors
.push(ResolverError::Reference(node.get_error()));
FluentValue::Error(node)
}
// Converts an AST node to a `FluentValue`.
pub(crate) trait ResolveValue<'source> {
fn resolve<R, M: MemoizerKind>(
&'source self,
scope: &mut Scope<'source, R, M>,
) -> FluentValue<'source>
where
R: Borrow<FluentResource>;
}
impl<'source> ResolveValue<'source> for ast::Pattern<'source> {
fn resolve<R, M: MemoizerKind>(
&'source self,
scope: &mut Scope<'source, R, M>,
) -> FluentValue<'source>
where
R: Borrow<FluentResource>,
{
if scope.dirty {
return FluentValue::None;
}
if self.elements.len() == 1 {
return match self.elements[0] {
ast::PatternElement::TextElement(s) => {
if let Some(ref transform) = scope.bundle.transform {
transform(s).into()
} else {
s.into()
}
}
ast::PatternElement::Placeable(ref p) => scope.maybe_track(self, p),
};
}
let mut string = String::new();
for elem in &self.elements {
if scope.dirty {
return FluentValue::None;
}
match elem {
ast::PatternElement::TextElement(s) => {
if let Some(ref transform) = scope.bundle.transform {
string.push_str(&transform(s))
} else {
string.push_str(&s)
}
}
ast::PatternElement::Placeable(p) => {
scope.placeables += 1;
if scope.placeables > MAX_PLACEABLES {
scope.dirty = true;
scope.errors.push(ResolverError::TooManyPlaceables);
return FluentValue::None;
}
let needs_isolation = scope.bundle.use_isolating
&& match p {
ast::Expression::InlineExpression(
ast::InlineExpression::MessageReference { .. },
)
| ast::Expression::InlineExpression(
ast::InlineExpression::TermReference { .. },
)
| ast::Expression::InlineExpression(
ast::InlineExpression::StringLiteral { .. },
) => false,
_ => true,
};
if needs_isolation {
string.write_char('\u{2068}').expect("Writing failed");
}
let result = scope.maybe_track(self, p);
write!(string, "{}", result.as_string(scope)).expect("Writing failed");
if needs_isolation {
string.write_char('\u{2069}').expect("Writing failed");
}
}
}
}
string.into()
}
}
impl<'source> ResolveValue<'source> for ast::Expression<'source> {
fn resolve<R, M: MemoizerKind>(
&'source self,
scope: &mut Scope<'source, R, M>,
) -> FluentValue<'source>
where
R: Borrow<FluentResource>,
{
match self {
ast::Expression::InlineExpression(exp) => exp.resolve(scope),
ast::Expression::SelectExpression { selector, variants } => {
let selector = selector.resolve(scope);
match selector {
FluentValue::String(_) | FluentValue::Number(_) => {
for variant in variants {
let key = match variant.key {
ast::VariantKey::Identifier { name } => name.into(),
ast::VariantKey::NumberLiteral { value } => {
FluentValue::try_number(value)
}
};
if key.matches(&selector, &scope) {
return variant.value.resolve(scope);
}
}
}
_ => {}
}
for variant in variants {
if variant.default {
return variant.value.resolve(scope);
}
}
scope.errors.push(ResolverError::MissingDefault);
FluentValue::None
}
}
}
}
impl<'source> ResolveValue<'source> for ast::InlineExpression<'source> {
fn resolve<R, M: MemoizerKind>(
&'source self,
mut scope: &mut Scope<'source, R, M>,
) -> FluentValue<'source>
where
R: Borrow<FluentResource>,
{
match self {
ast::InlineExpression::StringLiteral { value } => unescape_unicode(value).into(),
ast::InlineExpression::MessageReference { id, attribute } => scope
.bundle
.get_entry_message(&id.name)
.and_then(|msg| {
if let Some(attr) = attribute {
msg.attributes
.iter()
.find(|a| a.id.name == attr.name)
.map(|attr| scope.track(&attr.value, self.into()))
} else {
msg.value
.as_ref()
.map(|value| scope.track(value, self.into()))
}
})
.unwrap_or_else(|| generate_ref_error(scope, self.into())),
ast::InlineExpression::NumberLiteral { value } => FluentValue::try_number(*value),
ast::InlineExpression::TermReference {
id,
attribute,
arguments,
} => {
let (_, resolved_named_args) = get_arguments(scope, arguments);
scope.local_args = Some(resolved_named_args);
let value = scope
.bundle
.get_entry_term(&id.name)
.and_then(|term| {
if let Some(attr) = attribute {
term.attributes
.iter()
.find(|a| a.id.name == attr.name)
.map(|attr| scope.track(&attr.value, self.into()))
} else {
Some(scope.track(&term.value, self.into()))
}
})
.unwrap_or_else(|| generate_ref_error(scope, self.into()));
scope.local_args = None;
value
}
ast::InlineExpression::FunctionReference { id, arguments } => {
let (resolved_positional_args, resolved_named_args) =
get_arguments(scope, arguments);
let func = scope.bundle.get_entry_function(id.name);
if let Some(func) = func {
func(resolved_positional_args.as_slice(), &resolved_named_args)
} else {
generate_ref_error(scope, self.into())
}
}
ast::InlineExpression::VariableReference { id } => {
let args = scope.local_args.as_ref().or(scope.args);
if let Some(arg) = args.and_then(|args| args.get(id.name)) {
arg.clone()
} else {
let entry: DisplayableNode = self.into();
if scope.local_args.is_none() {
scope
.errors
.push(ResolverError::Reference(entry.get_error()));
}
FluentValue::Error(entry)
}
}
ast::InlineExpression::Placeable { expression } => expression.resolve(scope),
}
}
}
fn get_arguments<'bundle, R, M: MemoizerKind>(
scope: &mut Scope<'bundle, R, M>,
arguments: &'bundle Option<ast::CallArguments<'bundle>>,
) -> (Vec<FluentValue<'bundle>>, FluentArgs<'bundle>)
where
R: Borrow<FluentResource>,
{
let mut resolved_positional_args = Vec::new();
let mut resolved_named_args = FluentArgs::new();
if let Some(ast::CallArguments { named, positional }) = arguments {
for expression in positional {
resolved_positional_args.push(expression.resolve(scope));
}
for arg in named {
resolved_named_args.insert(arg.name.name, arg.value.resolve(scope));
}
}
(resolved_positional_args, resolved_named_args)
}

41
third_party/rust/fluent-bundle/src/resource.rs vendored Normal file
View File

@ -0,0 +1,41 @@
use fluent_syntax::ast;
use fluent_syntax::parser::parse;
use fluent_syntax::parser::ParserError;
rental! {
mod rentals {
use super::*;
#[rental(covariant, debug)]
pub struct FluentResource {
string: String,
ast: ast::Resource<'string>,
}
}
}
/// A resource containing a list of localization messages.
#[derive(Debug)]
pub struct FluentResource(rentals::FluentResource);
impl FluentResource {
pub fn try_new(source: String) -> Result<Self, (Self, Vec<ParserError>)> {
let mut errors = None;
let res = rentals::FluentResource::new(source, |s| match parse(s) {
Ok(ast) => ast,
Err((ast, err)) => {
errors = Some(err);
ast
}
});
if let Some(errors) = errors {
Err((Self(res), errors))
} else {
Ok(Self(res))
}
}
pub fn ast(&self) -> &ast::Resource {
self.0.all().ast
}
}

264
third_party/rust/fluent-bundle/src/types/mod.rs vendored Normal file
View File

@ -0,0 +1,264 @@
mod number;
mod plural;
pub use number::*;
use plural::*;
use std::any::Any;
use std::borrow::{Borrow, Cow};
use std::default::Default;
use std::fmt;
use std::str::FromStr;
use fluent_syntax::ast;
use intl_pluralrules::{PluralCategory, PluralRuleType};
use crate::memoizer::MemoizerKind;
use crate::resolve::Scope;
use crate::resource::FluentResource;
#[derive(Debug, PartialEq, Clone)]
pub enum DisplayableNodeType<'source> {
Message(&'source str),
Term(&'source str),
Variable(&'source str),
Function(&'source str),
Expression,
}
#[derive(Debug, PartialEq, Clone)]
pub struct DisplayableNode<'source> {
node_type: DisplayableNodeType<'source>,
attribute: Option<&'source str>,
}
impl<'source> Default for DisplayableNode<'source> {
fn default() -> Self {
DisplayableNode {
node_type: DisplayableNodeType::Expression,
attribute: None,
}
}
}
impl<'source> DisplayableNode<'source> {
pub fn get_error(&self) -> String {
if self.attribute.is_some() {
format!("Unknown attribute: {}", self)
} else {
match self.node_type {
DisplayableNodeType::Message(..) => format!("Unknown message: {}", self),
DisplayableNodeType::Term(..) => format!("Unknown term: {}", self),
DisplayableNodeType::Variable(..) => format!("Unknown variable: {}", self),
DisplayableNodeType::Function(..) => format!("Unknown function: {}", self),
DisplayableNodeType::Expression => "Failed to resolve an expression.".to_string(),
}
}
}
}
impl<'source> fmt::Display for DisplayableNode<'source> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self.node_type {
DisplayableNodeType::Message(id) => write!(f, "{}", id)?,
DisplayableNodeType::Term(id) => write!(f, "-{}", id)?,
DisplayableNodeType::Variable(id) => write!(f, "${}", id)?,
DisplayableNodeType::Function(id) => write!(f, "{}()", id)?,
DisplayableNodeType::Expression => f.write_str("???")?,
};
if let Some(attr) = self.attribute {
write!(f, ".{}", attr)?;
}
Ok(())
}
}
impl<'source> From<&ast::Expression<'source>> for DisplayableNode<'source> {
fn from(expr: &ast::Expression<'source>) -> Self {
match expr {
ast::Expression::InlineExpression(e) => e.into(),
ast::Expression::SelectExpression { .. } => DisplayableNode::default(),
}
}
}
impl<'source> From<&ast::InlineExpression<'source>> for DisplayableNode<'source> {
fn from(expr: &ast::InlineExpression<'source>) -> Self {
match expr {
ast::InlineExpression::MessageReference { id, attribute } => DisplayableNode {
node_type: DisplayableNodeType::Message(id.name),
attribute: attribute.as_ref().map(|attr| attr.name),
},
ast::InlineExpression::TermReference { id, attribute, .. } => DisplayableNode {
node_type: DisplayableNodeType::Term(id.name),
attribute: attribute.as_ref().map(|attr| attr.name),
},
ast::InlineExpression::VariableReference { id } => DisplayableNode {
node_type: DisplayableNodeType::Variable(id.name),
attribute: None,
},
ast::InlineExpression::FunctionReference { id, .. } => DisplayableNode {
node_type: DisplayableNodeType::Function(id.name),
attribute: None,
},
_ => DisplayableNode::default(),
}
}
}
pub trait FluentType: fmt::Debug + AnyEq + 'static {
fn duplicate(&self) -> Box<dyn FluentType>;
fn as_string(&self, intls: &intl_memoizer::IntlLangMemoizer) -> Cow<'static, str>;
fn as_string_threadsafe(
&self,
intls: &intl_memoizer::concurrent::IntlLangMemoizer,
) -> Cow<'static, str>;
}
impl PartialEq for dyn FluentType {
fn eq(&self, other: &Self) -> bool {
self.equals(other.as_any())
}
}
pub trait AnyEq: Any + 'static {
fn equals(&self, other: &dyn Any) -> bool;
fn as_any(&self) -> &dyn Any;
}
impl<T: Any + PartialEq> AnyEq for T {
fn equals(&self, other: &dyn Any) -> bool {
if let Some(that) = other.downcast_ref::<Self>() {
self == that
} else {
false
}
}
fn as_any(&self) -> &dyn Any {
self
}
}
/// The `FluentValue` enum represents values which can be formatted to a String.
///
/// The [`ResolveValue`][] trait from the [`resolve`][] module evaluates AST nodes into
/// `FluentValues` which can then be formatted to Strings using the i18n formatters stored by the
/// `FluentBundle` instance if required.
///
/// The arguments `HashMap` passed to [`FluentBundle::format`][] should also use `FluentValues`
/// as values of arguments.
///
/// [`ResolveValue`]: ../resolve/trait.ResolveValue.html
/// [`resolve`]: ../resolve
/// [`FluentBundle::format`]: ../bundle/struct.FluentBundle.html#method.format
#[derive(Debug)]
pub enum FluentValue<'source> {
String(Cow<'source, str>),
Number(FluentNumber),
Custom(Box<dyn FluentType>),
Error(DisplayableNode<'source>),
None,
}
impl<'s> PartialEq for FluentValue<'s> {
fn eq(&self, other: &Self) -> bool {
match (self, other) {
(FluentValue::String(s), FluentValue::String(s2)) => s == s2,
(FluentValue::Number(s), FluentValue::Number(s2)) => s == s2,
(FluentValue::Custom(s), FluentValue::Custom(s2)) => s == s2,
_ => false,
}
}
}
impl<'s> Clone for FluentValue<'s> {
fn clone(&self) -> Self {
match self {
FluentValue::String(s) => FluentValue::String(s.clone()),
FluentValue::Number(s) => FluentValue::Number(s.clone()),
FluentValue::Custom(s) => {
let new_value: Box<dyn FluentType> = s.duplicate();
FluentValue::Custom(new_value)
}
FluentValue::Error(e) => FluentValue::Error(e.clone()),
FluentValue::None => FluentValue::None,
}
}
}
impl<'source> FluentValue<'source> {
pub fn try_number<S: ToString>(v: S) -> Self {
let s = v.to_string();
if let Ok(num) = FluentNumber::from_str(&s.to_string()) {
num.into()
} else {
s.into()
}
}
pub fn matches<R: Borrow<FluentResource>, M: MemoizerKind>(
&self,
other: &FluentValue,
scope: &Scope<R, M>,
) -> bool {
match (self, other) {
(&FluentValue::String(ref a), &FluentValue::String(ref b)) => a == b,
(&FluentValue::Number(ref a), &FluentValue::Number(ref b)) => a == b,
(&FluentValue::String(ref a), &FluentValue::Number(ref b)) => {
let cat = match a.as_ref() {
"zero" => PluralCategory::ZERO,
"one" => PluralCategory::ONE,
"two" => PluralCategory::TWO,
"few" => PluralCategory::FEW,
"many" => PluralCategory::MANY,
"other" => PluralCategory::OTHER,
_ => return false,
};
scope
.bundle
.intls
.with_try_get_threadsafe::<PluralRules, _, _>(
(PluralRuleType::CARDINAL,),
|pr| pr.0.select(b) == Ok(cat),
)
.unwrap()
}
_ => false,
}
}
pub fn as_string<R: Borrow<FluentResource>, M: MemoizerKind>(
&self,
scope: &Scope<R, M>,
) -> Cow<'source, str> {
if let Some(formatter) = &scope.bundle.formatter {
if let Some(val) = formatter(self, &scope.bundle.intls) {
return val.into();
}
}
match self {
FluentValue::String(s) => s.clone(),
FluentValue::Number(n) => n.as_string(),
FluentValue::Error(d) => format!("{{{}}}", d.to_string()).into(),
FluentValue::Custom(s) => scope.bundle.intls.stringify_value(&**s),
FluentValue::None => "???".into(),
}
}
}
impl<'source> From<String> for FluentValue<'source> {
fn from(s: String) -> Self {
FluentValue::String(s.into())
}
}
impl<'source> From<&'source str> for FluentValue<'source> {
fn from(s: &'source str) -> Self {
FluentValue::String(s.into())
}
}
impl<'source> From<Cow<'source, str>> for FluentValue<'source> {
fn from(s: Cow<'source, str>) -> Self {
FluentValue::String(s)
}
}

249
third_party/rust/fluent-bundle/src/types/number.rs vendored Normal file
View File

@ -0,0 +1,249 @@
use std::borrow::Cow;
use std::convert::TryInto;
use std::default::Default;
use std::str::FromStr;
use intl_pluralrules::operands::PluralOperands;
use crate::bundle::FluentArgs;
use crate::types::FluentValue;
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub enum FluentNumberStyle {
Decimal,
Currency,
Percent,
}
impl std::default::Default for FluentNumberStyle {
fn default() -> Self {
Self::Decimal
}
}
impl From<&str> for FluentNumberStyle {
fn from(input: &str) -> Self {
match input {
"decimal" => Self::Decimal,
"currency" => Self::Currency,
"percent" => Self::Percent,
_ => Self::default(),
}
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub enum FluentNumberCurrencyDisplayStyle {
Symbol,
Code,
Name,
}
impl std::default::Default for FluentNumberCurrencyDisplayStyle {
fn default() -> Self {
Self::Symbol
}
}
impl From<&str> for FluentNumberCurrencyDisplayStyle {
fn from(input: &str) -> Self {
match input {
"symbol" => Self::Symbol,
"code" => Self::Code,
"name" => Self::Name,
_ => Self::default(),
}
}
}
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct FluentNumberOptions {
pub style: FluentNumberStyle,
pub currency: Option<String>,
pub currency_display: FluentNumberCurrencyDisplayStyle,
pub use_grouping: bool,
pub minimum_integer_digits: Option<usize>,
pub minimum_fraction_digits: Option<usize>,
pub maximum_fraction_digits: Option<usize>,
pub minimum_significant_digits: Option<usize>,
pub maximum_significant_digits: Option<usize>,
}
impl Default for FluentNumberOptions {
fn default() -> Self {
Self {
style: Default::default(),
currency: None,
currency_display: Default::default(),
use_grouping: true,
minimum_integer_digits: None,
minimum_fraction_digits: None,
maximum_fraction_digits: None,
minimum_significant_digits: None,
maximum_significant_digits: None,
}
}
}
impl FluentNumberOptions {
pub fn merge(&mut self, opts: &FluentArgs) {
for (key, value) in opts {
match (*key, value) {
("style", FluentValue::String(n)) => {
self.style = n.as_ref().into();
}
("currency", FluentValue::String(n)) => {
self.currency = Some(n.to_string());
}
("currencyDisplay", FluentValue::String(n)) => {
self.currency_display = n.as_ref().into();
}
("minimumIntegerDigits", FluentValue::Number(n)) => {
self.minimum_integer_digits = Some(n.into());
}
("minimumFractionDigits", FluentValue::Number(n)) => {
self.minimum_fraction_digits = Some(n.into());
}
("maximumFractionDigits", FluentValue::Number(n)) => {
self.maximum_fraction_digits = Some(n.into());
}
("minimumSignificantDigits", FluentValue::Number(n)) => {
self.minimum_significant_digits = Some(n.into());
}
("maximumSignificantDigits", FluentValue::Number(n)) => {
self.maximum_significant_digits = Some(n.into());
}
_ => {}
}
}
}
}
#[derive(Debug, PartialEq, Clone)]
pub struct FluentNumber {
pub value: f64,
pub options: FluentNumberOptions,
}
impl FluentNumber {
pub fn new(value: f64, options: FluentNumberOptions) -> Self {
Self { value, options }
}
pub fn as_string(&self) -> Cow<'static, str> {
let mut val = self.value.to_string();
if let Some(minfd) = self.options.minimum_fraction_digits {
if let Some(pos) = val.find('.') {
let frac_num = val.len() - pos - 1;
let missing = if frac_num > minfd {
0
} else {
minfd - frac_num
};
val = format!("{}{}", val, "0".repeat(missing));
} else {
val = format!("{}.{}", val, "0".repeat(minfd));
}
}
val.into()
}
}
impl FromStr for FluentNumber {
type Err = std::num::ParseFloatError;
fn from_str(input: &str) -> Result<Self, Self::Err> {
f64::from_str(input).map(|n| {
let mfd = input.find('.').map(|pos| input.len() - pos - 1);
let opts = FluentNumberOptions {
minimum_fraction_digits: mfd,
..Default::default()
};
FluentNumber::new(n, opts)
})
}
}
impl<'l> From<FluentNumber> for FluentValue<'l> {
fn from(input: FluentNumber) -> Self {
FluentValue::Number(input)
}
}
macro_rules! from_num {
($num:ty) => {
impl From<$num> for FluentNumber {
fn from(n: $num) -> Self {
FluentNumber {
value: n as f64,
options: FluentNumberOptions::default(),
}
}
}
impl From<&$num> for FluentNumber {
fn from(n: &$num) -> Self {
FluentNumber {
value: *n as f64,
options: FluentNumberOptions::default(),
}
}
}
impl From<FluentNumber> for $num {
fn from(input: FluentNumber) -> Self {
input.value as $num
}
}
impl From<&FluentNumber> for $num {
fn from(input: &FluentNumber) -> Self {
input.value as $num
}
}
impl From<$num> for FluentValue<'_> {
fn from(n: $num) -> Self {
FluentValue::Number(n.into())
}
}
impl From<&$num> for FluentValue<'_> {
fn from(n: &$num) -> Self {
FluentValue::Number(n.into())
}
}
};
($($num:ty)+) => {
$(from_num!($num);)+
};
}
impl From<&FluentNumber> for PluralOperands {
fn from(input: &FluentNumber) -> Self {
let mut operands: PluralOperands = input
.value
.try_into()
.expect("Failed to generate operands out of FluentNumber");
if let Some(mfd) = input.options.minimum_fraction_digits {
if mfd > operands.v {
operands.f *= 10_usize.pow(mfd as u32 - operands.v as u32);
operands.v = mfd;
}
}
// XXX: Add support for other options.
operands
}
}
from_num!(i8 i16 i32 i64 i128 isize);
from_num!(u8 u16 u32 u64 u128 usize);
from_num!(f32 f64);
#[cfg(test)]
mod tests {
use crate::types::FluentValue;
#[test]
fn value_from_copy_ref() {
let x = 1i16;
let y = &x;
let z: FluentValue = y.into();
assert_eq!(z, FluentValue::try_number(1));
}
}

22
third_party/rust/fluent-bundle/src/types/plural.rs vendored Normal file
View File

@ -0,0 +1,22 @@
use fluent_langneg::{negotiate_languages, NegotiationStrategy};
use intl_memoizer::Memoizable;
use intl_pluralrules::{PluralRuleType, PluralRules as IntlPluralRules};
use unic_langid::LanguageIdentifier;
pub struct PluralRules(pub IntlPluralRules);
impl Memoizable for PluralRules {
type Args = (PluralRuleType,);
type Error = &'static str;
fn construct(lang: LanguageIdentifier, args: Self::Args) -> Result<Self, Self::Error> {
let default_lang: LanguageIdentifier = "en".parse().unwrap();
let pr_lang = negotiate_languages(
&[lang],
&IntlPluralRules::get_locales(args.0),
Some(&default_lang),
NegotiationStrategy::Lookup,
)[0]
.clone();
Ok(Self(IntlPluralRules::create(pr_lang, args.0)?))
}
}

37
third_party/rust/fluent-bundle/tests/bundle.rs vendored Normal file
View File

@ -0,0 +1,37 @@
use fluent_bundle::{FluentBundle, FluentResource};
use unic_langid::LanguageIdentifier;
#[test]
fn add_resource_override() {
let res = FluentResource::try_new("key = Value".to_string()).unwrap();
let res2 = FluentResource::try_new("key = Value 2".to_string()).unwrap();
let en_us: LanguageIdentifier = "en-US"
.parse()
.expect("Failed to parse a language identifier");
let mut bundle = FluentBundle::<&FluentResource>::new(&[en_us]);
bundle.add_resource(&res).expect("Failed to add a resource");
assert!(bundle.add_resource(&res2).is_err());
let mut errors = vec![];
let value = bundle
.get_message("key")
.expect("Failed to retireve a message")
.value
.expect("Failed to retireve a value of a message");
assert_eq!(bundle.format_pattern(value, None, &mut errors), "Value");
bundle.add_resource_overriding(&res2);
let value = bundle
.get_message("key")
.expect("Failed to retireve a message")
.value
.expect("Failed to retireve a value of a message");
assert_eq!(bundle.format_pattern(value, None, &mut errors), "Value 2");
assert!(errors.is_empty());
}

230
third_party/rust/fluent-bundle/tests/custom_types.rs vendored Normal file
View File

@ -0,0 +1,230 @@
use fluent_bundle::memoizer::MemoizerKind;
use fluent_bundle::types::FluentType;
use fluent_bundle::FluentArgs;
use fluent_bundle::FluentBundle;
use fluent_bundle::FluentResource;
use fluent_bundle::FluentValue;
use unic_langid::langid;
#[test]
fn fluent_custom_type() {
#[derive(Debug, PartialEq)]
struct DateTime {
epoch: usize,
};
impl DateTime {
pub fn new(epoch: usize) -> Self {
Self { epoch }
}
}
impl FluentType for DateTime {
fn duplicate(&self) -> Box<dyn FluentType> {
Box::new(DateTime { epoch: self.epoch })
}
fn as_string(&self, _: &intl_memoizer::IntlLangMemoizer) -> std::borrow::Cow<'static, str> {
format!("{}", self.epoch).into()
}
fn as_string_threadsafe(
&self,
_: &intl_memoizer::concurrent::IntlLangMemoizer,
) -> std::borrow::Cow<'static, str> {
format!("{}", self.epoch).into()
}
}
let dt = FluentValue::Custom(Box::new(DateTime::new(10)));
let dt2 = FluentValue::Custom(Box::new(DateTime::new(10)));
let dt3 = FluentValue::Custom(Box::new(DateTime::new(15)));
let sv = FluentValue::from("foo");
assert_eq!(dt == dt2, true);
assert_eq!(dt == dt3, false);
assert_eq!(dt == sv, false);
}
#[test]
fn fluent_date_time_builtin() {
#[derive(Debug, PartialEq, Clone)]
enum DateTimeStyleValue {
Full,
Long,
Medium,
Short,
None,
}
impl std::default::Default for DateTimeStyleValue {
fn default() -> Self {
Self::None
}
}
impl<'l> From<&FluentValue<'l>> for DateTimeStyleValue {
fn from(input: &FluentValue) -> Self {
if let FluentValue::String(s) = input {
match s.as_ref() {
"full" => Self::Full,
"long" => Self::Long,
"medium" => Self::Medium,
"short" => Self::Short,
_ => Self::None,
}
} else {
Self::None
}
}
}
#[derive(Debug, PartialEq, Default, Clone)]
struct DateTimeOptions {
pub date_style: DateTimeStyleValue,
pub time_style: DateTimeStyleValue,
}
impl DateTimeOptions {
pub fn merge(&mut self, input: &FluentArgs) {
for (key, value) in input {
match *key {
"dateStyle" => self.date_style = value.into(),
"timeStyle" => self.time_style = value.into(),
_ => {}
}
}
}
}
impl<'l> From<&FluentArgs<'l>> for DateTimeOptions {
fn from(input: &FluentArgs) -> Self {
let mut opts = Self::default();
opts.merge(input);
opts
}
}
#[derive(Debug, PartialEq, Clone)]
struct DateTime {
epoch: usize,
options: DateTimeOptions,
};
impl DateTime {
pub fn new(epoch: usize, options: DateTimeOptions) -> Self {
Self { epoch, options }
}
}
impl FluentType for DateTime {
fn duplicate(&self) -> Box<dyn FluentType> {
Box::new(DateTime::new(self.epoch, DateTimeOptions::default()))
}
fn as_string(&self, _: &intl_memoizer::IntlLangMemoizer) -> std::borrow::Cow<'static, str> {
format!("2020-01-20 {}:00", self.epoch).into()
}
fn as_string_threadsafe(
&self,
_intls: &intl_memoizer::concurrent::IntlLangMemoizer,
) -> std::borrow::Cow<'static, str> {
format!("2020-01-20 {}:00", self.epoch).into()
}
}
let lang = langid!("en");
let mut bundle = FluentBundle::new(&[lang]);
let res = FluentResource::try_new(
r#"
key-explicit = Hello { DATETIME(12, dateStyle: "full") } World
key-ref = Hello { DATETIME($date, dateStyle: "full") } World
"#
.into(),
)
.unwrap();
bundle.add_resource(res).unwrap();
bundle.set_use_isolating(false);
bundle
.add_function("DATETIME", |positional, named| match positional.get(0) {
Some(FluentValue::Custom(custom)) => {
if let Some(that) = custom.as_ref().as_any().downcast_ref::<DateTime>() {
let mut dt = that.clone();
dt.options.merge(named);
FluentValue::Custom(Box::new(dt))
} else {
FluentValue::None
}
}
Some(FluentValue::Number(num)) => {
let num = num.value as usize;
FluentValue::Custom(Box::new(DateTime::new(num, named.into())))
}
_ => FluentValue::None,
})
.unwrap();
let mut errors = vec![];
let mut args = FluentArgs::new();
args.insert(
"date",
FluentValue::Custom(Box::new(DateTime::new(10, DateTimeOptions::default()))),
);
let msg = bundle.get_message("key-explicit").unwrap();
let val = bundle.format_pattern(msg.value.unwrap(), Some(&args), &mut errors);
assert_eq!(val, "Hello 2020-01-20 12:00 World");
let msg = bundle.get_message("key-ref").unwrap();
let val = bundle.format_pattern(msg.value.unwrap(), Some(&args), &mut errors);
assert_eq!(val, "Hello 2020-01-20 10:00 World");
}
#[test]
fn fluent_custom_number_format() {
fn custom_formatter<M: MemoizerKind>(num: &FluentValue, _intls: &M) -> Option<String> {
match num {
FluentValue::Number(_) => Some("CUSTOM".into()),
_ => None,
}
}
let res = FluentResource::try_new(
r#"
key-num-implicit = Hello { 5.000 } World
key-num-explicit = Hello { NUMBER(5, minimumFractionDigits: 2) } World
"#
.into(),
)
.unwrap();
let mut bundle = FluentBundle::default();
bundle.add_resource(res).unwrap();
bundle.set_use_isolating(false);
bundle
.add_function("NUMBER", |positional, named| match positional.get(0) {
Some(FluentValue::Number(n)) => {
let mut num = n.clone();
num.options.merge(named);
FluentValue::Number(num)
}
_ => FluentValue::None,
})
.unwrap();
let mut errors = vec![];
let msg = bundle.get_message("key-num-explicit").unwrap();
let val = bundle.format_pattern(msg.value.unwrap(), None, &mut errors);
assert_eq!(val, "Hello 5.00 World");
let msg = bundle.get_message("key-num-implicit").unwrap();
let val = bundle.format_pattern(msg.value.unwrap(), None, &mut errors);
assert_eq!(val, "Hello 5.000 World");
bundle.set_formatter(Some(custom_formatter));
let msg = bundle.get_message("key-num-implicit").unwrap();
let val = bundle.format_pattern(msg.value.unwrap(), None, &mut errors);
assert_eq!(val, "Hello CUSTOM World");
}

133
third_party/rust/fluent-bundle/tests/fixtures/arguments.yaml vendored Normal file
View File

@ -0,0 +1,133 @@
suites:
-
name: Variables
suites:
-
name: in values
resources:
-
source: |-
foo = Foo { $num }
bar = { foo }
baz =
.attr = Baz Attribute { $num }
qux = { "a" ->
*[a] Baz Variant A { $num }
}
tests:
-
name: can be used in the message value
asserts:
-
id: foo
args:
num: 3
value: Foo 3
-
name: can be used in the message value which is referenced
asserts:
-
id: bar
args:
num: 3
value: Foo 3
-
name: can be used in an attribute
asserts:
-
id: baz
attribute: attr
args:
num: 3
value: Baz Attribute 3
-
name: can be used in a variant
asserts:
-
id: qux
args:
num: 3
value: Baz Variant A 3
-
name: in selectors
resources:
-
source: |-
foo = { $num ->
*[3] Foo
}
tests:
-
name: can be used as a selector
asserts:
-
id: foo
args:
num: 3
value: Foo
-
name: in function calls
resources:
-
source: |-
foo = { NUMBER($num) }
bundles:
-
functions:
- NUMBER
tests:
-
name: can be a positional argument
asserts:
-
id: foo
args:
num: 3
value: 3
-
name: simple errors
resources:
-
source: |-
foo = { $arg }
tests:
-
name: falls back to argument's name if it's missing
asserts:
-
id: foo
value: "{$arg}"
errors:
-
type: Reference
desc: "Unknown variable: $arg"
-
name: and strings
resources:
-
source: |-
foo = { $arg }
tests:
-
name: can be a string
asserts:
-
id: foo
args:
arg: Argument
value: Argument
-
name: and numbers
resources:
-
source: |-
foo = { $arg }
tests:
-
name: can be a number
asserts:
-
id: foo
args:
arg: 1
value: 1

178
third_party/rust/fluent-bundle/tests/fixtures/attributes.yaml vendored Normal file
View File

@ -0,0 +1,178 @@
suites:
-
name: Attributes
suites:
-
name: missing
resources:
-
source: |-
foo = Foo
bar = Bar
.attr = Bar Attribute
baz = { foo } Baz
qux = { foo } Qux
.attr = Qux Attribute
ref-foo = { foo.missing }
ref-bar = { bar.missing }
ref-baz = { baz.missing }
ref-qux = { qux.missing }
tests:
-
name: falls back to id.attr for entities with string values and no attributes
asserts:
-
id: ref-foo
value: "{foo.missing}"
errors:
-
type: Reference
desc: "Unknown attribute: foo.missing"
-
name: falls back to id.attr for entities with string values and other attributes
asserts:
-
id: ref-bar
value: "{bar.missing}"
errors:
-
type: Reference
desc: "Unknown attribute: bar.missing"
-
name: falls back to id.attr for entities with pattern values and no attributes
asserts:
-
id: ref-baz
value: "{baz.missing}"
errors:
-
type: Reference
desc: "Unknown attribute: baz.missing"
-
name: falls back to id.attr for entities with pattern values and other attributes
asserts:
-
id: ref-qux
value: "{qux.missing}"
errors:
-
type: Reference
desc: "Unknown attribute: qux.missing"
-
name: with string values
resources:
-
source: |-
foo = Foo
.attr = Foo Attribute
bar = { foo } Bar
.attr = Bar Attribute
ref-foo = { foo.attr }
ref-bar = { bar.attr }
tests:
-
name: can be referenced for entities with string values
asserts:
-
id: ref-foo
value: Foo Attribute
-
name: can be formatted directly for entities with string values
asserts:
-
id: foo
attribute: attr
value: Foo Attribute
-
name: can be referenced for entities with pattern values
asserts:
-
id: ref-bar
value: Bar Attribute
-
name: can be formatted directly for entities with pattern values
asserts:
-
id: bar
attribute: attr
value: Bar Attribute
-
name: with simple pattern values
resources:
-
source: |-
foo = Foo
bar = Bar
.attr = { foo } Attribute
baz = { foo } Baz
.attr = { foo } Attribute
qux = Qux
.attr = { qux } Attribute
ref-bar = { bar.attr }
ref-baz = { baz.attr }
ref-qux = { qux.attr }
tests:
-
name: can be referenced for entities with string values
asserts:
-
id: ref-bar
value: Foo Attribute
-
name: can be formatted directly for entities with string values
asserts:
-
id: bar
attribute: attr
value: Foo Attribute
-
name: can be referenced for entities with simple pattern values
asserts:
-
id: ref-baz
value: Foo Attribute
-
name: can be formatted directly for entities with simple pattern values
asserts:
-
id: baz
attribute: attr
value: Foo Attribute
-
name: works with self-references
asserts:
-
id: ref-qux
value: Qux Attribute
-
name: can be formatted directly when it uses a self-reference
asserts:
-
id: qux
attribute: attr
value: Qux Attribute
-
name: with values with select expressions
resources:
-
source: |-
foo = Foo
.attr = { "a" ->
[a] A
*[b] B
}
ref-foo = { foo.attr }
tests:
-
name: can be referenced
asserts:
-
id: ref-foo
value: A
-
name: can be formatted directly
asserts:
-
id: foo
attribute: attr
value: A

30
third_party/rust/fluent-bundle/tests/fixtures/bomb.yaml vendored Normal file
View File

@ -0,0 +1,30 @@
suites:
-
name: Reference bombs
suites:
-
name: Billion Laughs
resources:
-
source: |-
lol0 = LOL
lol1 = {lol0} {lol0} {lol0} {lol0} {lol0} {lol0} {lol0} {lol0} {lol0} {lol0}
lol2 = {lol1} {lol1} {lol1} {lol1} {lol1} {lol1} {lol1} {lol1} {lol1} {lol1}
lol3 = {lol2} {lol2} {lol2} {lol2} {lol2} {lol2} {lol2} {lol2} {lol2} {lol2}
lol4 = {lol3} {lol3} {lol3} {lol3} {lol3} {lol3} {lol3} {lol3} {lol3} {lol3}
lol5 = {lol4} {lol4} {lol4} {lol4} {lol4} {lol4} {lol4} {lol4} {lol4} {lol4}
lol6 = {lol5} {lol5} {lol5} {lol5} {lol5} {lol5} {lol5} {lol5} {lol5} {lol5}
lol7 = {lol6} {lol6} {lol6} {lol6} {lol6} {lol6} {lol6} {lol6} {lol6} {lol6}
lol8 = {lol7} {lol7} {lol7} {lol7} {lol7} {lol7} {lol7} {lol7} {lol7} {lol7}
lol9 = {lol8} {lol8} {lol8} {lol8} {lol8} {lol8} {lol8} {lol8} {lol8} {lol8}
lolz = {lol9}
tests:
-
name: does not expand all placeables
asserts:
-
id: lolz
value: "{lol9}"
errors:
-
type: TooManyPlaceables

195
third_party/rust/fluent-bundle/tests/fixtures/context.yaml vendored Normal file
View File

@ -0,0 +1,195 @@
suites:
-
name: Bundle
suites:
-
name: addResource
resources:
-
source: |-
foo = Foo
-bar = Bar
tests:
-
name: adds messages
asserts:
-
id: foo
value: Foo
-
id: bar
missing: true
-
name: allowOverrides
resources:
-
source: key = Foo
tests:
-
name: addResource allowOverrides is false
resources:
-
source: key = Bar
bundles:
-
errors:
-
type: Overriding
asserts:
-
id: key
value: Foo
-
name: addResource allowOverrides is true
skip: true
resources:
-
source: key = Bar
asserts:
-
id: key
value: Bar
-
name: hasMessage
resources:
-
source: |-
foo = Foo
bar =
.attr = Bar Attr
-term = Term
# ERROR No value.
err1 =
# ERROR Broken value.
err2 = {}
# ERROR No attribute value.
err3 =
.attr =
# ERROR Broken attribute value.
err4 =
.attr1 = Attr
.attr2 = {}
errors:
-
type: Parser
-
type: Parser
-
type: Parser
-
type: Parser
tests:
-
name: returns true only for public messages
asserts:
-
id: foo
missing: false
-
name: returns false for terms and missing messages
asserts:
-
id: -term
missing: true
-
id: missing
missing: true
-
id: -missing
missing: true
-
name: returns false for broken messages
asserts:
-
id: err1
missing: true
-
id: err2
missing: true
-
id: err3
missing: true
-
id: err4
# XXX: Difference from JS. We handle partial messages
missing: false
-
name: getMessage
resources:
-
source: |-
foo = Foo
-bar = Bar
tests:
-
name: returns public messages
asserts:
-
id: foo
missing: false
-
name: returns undefined for terms and missing messages
asserts:
-
id: -bar
missing: true
-
id: baz
missing: true
-
id: -baz
missing: true
-
name: (Rust) Entries
resources:
-
source: |-
-foo = Bar
baz = { foo }
qux = { -bar }
fn = { FN() }
tests:
-
name: Entry mismatch doesn't leak
asserts:
-
id: baz
value: "{foo}"
errors:
-
type: Reference
desc: "Unknown message: foo"
-
name: Missing term
asserts:
-
id: qux
value: "{-bar}"
errors:
-
type: Reference
desc: "Unknown term: -bar"
-
name: Missing function
asserts:
-
id: fn
value: "{FN()}"
errors:
-
type: Reference
desc: "Unknown function: FN()"
-
name: (Rust) FluentBundle construction
resources:
-
source: |-
foo = Foo
bundles:
-
functions:
- SUM
- SUM
errors:
-
type: Overriding

6
third_party/rust/fluent-bundle/tests/fixtures/defaults.yaml vendored Normal file
View File

@ -0,0 +1,6 @@
# Those are default settings for all tests
bundle:
useIsolating: false
locales:
- en-US

21
third_party/rust/fluent-bundle/tests/fixtures/errors.yaml vendored Normal file
View File

@ -0,0 +1,21 @@
suites:
-
name: Errors
resources:
-
source: |-
foo = {$one} and {$two}
tests:
-
name: Reporting into an array
asserts:
-
id: foo
value: "{$one} and {$two}"
errors:
-
type: Reference
desc: "Unknown variable: $one"
-
type: Reference
desc: "Unknown variable: $two"

85
third_party/rust/fluent-bundle/tests/fixtures/functions.yaml vendored Normal file
View File

@ -0,0 +1,85 @@
suites:
-
name: Functions
suites:
-
name: missing
resources:
-
source: |-
foo = { MISSING("Foo") }
tests:
-
name: falls back to the name of the function
asserts:
-
id: foo
value: "{MISSING()}"
errors:
-
type: Reference
desc: "Unknown function: MISSING()"
-
name: arguments
resources:
-
source: |-
foo = Foo
.attr = Attribute
pass-nothing = { IDENTITY() }
pass-string = { IDENTITY("a") }
pass-number = { IDENTITY(1) }
pass-message = { IDENTITY(foo) }
pass-attr = { IDENTITY(foo.attr) }
pass-variable = { IDENTITY($var) }
pass-function-call = { IDENTITY(IDENTITY(1)) }
bundles:
-
functions:
- IDENTITY
tests:
-
name: falls back when arguments don't match the arity
asserts:
-
id: pass-nothing
## XXX: Difference from JS
value: "???"
-
name: accepts strings
asserts:
-
id: pass-string
value: a
-
name: accepts numbers
asserts:
-
id: pass-number
value: 1
-
name: accepts entities
asserts:
-
id: pass-message
value: Foo
-
name: accepts attributes
asserts:
-
id: pass-attr
value: Attribute
-
name: accepts variables
asserts:
-
id: pass-variable
args:
var: Variable
value: Variable
-
name: accepts function calls
asserts:
-
id: pass-function-call
value: 1

29
third_party/rust/fluent-bundle/tests/fixtures/functions_runtime.yaml vendored Normal file
View File

@ -0,0 +1,29 @@
suites:
-
name: Runtime-specific functions
suites:
-
name: passing into the constructor
resources:
-
source: |-
foo = { CONCAT("Foo", "Bar") }
bar = { SUM(1, 2) }
bundles:
-
functions:
- CONCAT
- SUM
tests:
-
name: works for strings
asserts:
-
id: foo
value: FooBar
-
name: works for numbers
asserts:
-
id: bar
value: 3

100
third_party/rust/fluent-bundle/tests/fixtures/isolating.yaml vendored Normal file
View File

@ -0,0 +1,100 @@
suites:
-
name: Isolating interpolations
resources:
-
source: |-
foo = Foo
bar = { foo } Bar
baz = { $arg } Baz
qux = { bar } { baz }
bundles:
-
useIsolating: true
tests:
-
name: isolates interpolated message references
skip: true
asserts:
-
id: bar
value: "\u2068Foo\u2069 Bar"
-
name: isolates interpolated string-typed variables
asserts:
-
id: baz
args:
arg: Arg
value: "\u2068Arg\u2069 Baz"
-
name: isolates interpolated number-typed variables
asserts:
-
id: baz
args:
arg: 1
value: "\u20681\u2069 Baz"
-
name: isolates interpolated date-typed variables
skip: true
asserts:
-
id: baz
args:
arg: 1976-07-31
value: "\u20681976-07-31\u2069 Baz"
-
name: isolates complex interpolations
skip: true
asserts:
-
id: qux
args:
arg: Arg
value: "\u2068\u2068Foo\u2069 Bar\u2069 \u2068\u2068Arg\u2069 Baz\u2069"
-
name: Skip isolation cases
resources:
-
source: |-
-brand-short-name = Amaya
foo = { -brand-short-name }
bundles:
-
useIsolating: true
tests:
-
name: skips isolation if the only element is a placeable
asserts:
-
id: foo
value: "Amaya"
-
name: (Rust) Skip isolation of string literals and terms
resources:
-
source: |-
rs-bar = Foo { $foo } { "Bar" } baz
-rs-term = My Term
rs-baz = Foo { $foo } { -rs-term } baz
bundles:
-
useIsolating: true
tests:
-
name: skip isolation of string literals
asserts:
-
id: rs-bar
args:
foo: Test
value: "Foo \u2068Test\u2069 Bar baz"
-
name: skip isolation of term references
asserts:
-
id: rs-baz
args:
foo: Test
value: "Foo \u2068Test\u2069 My Term baz"

69
third_party/rust/fluent-bundle/tests/fixtures/literals.yaml vendored Normal file
View File

@ -0,0 +1,69 @@
suites:
-
name: Literals as selectors
tests:
-
name: a matching string literal selector
resources:
-
source: |-
foo = { "a" ->
[a] A
*[b] B
}
asserts:
-
id: foo
value: A
-
name: a non-matching string literal selector
resources:
-
source: |-
foo = { "c" ->
[a] A
*[b] B
}
asserts:
-
id: foo
value: B
-
name: a matching number literal selector
resources:
-
source: |-
foo = { 0 ->
[0] A
*[1] B
}
asserts:
-
id: foo
value: A
-
name: a non-matching number literal selector
resources:
-
source: |-
foo = { 2 ->
[0] A
*[1] B
}
asserts:
-
id: foo
value: B
-
name: a number literal selector matching a plural category
resources:
-
source: |-
foo = { 1 ->
[one] A
*[other] B
}
asserts:
-
id: foo
value: A

392
third_party/rust/fluent-bundle/tests/fixtures/macros.yaml vendored Normal file
View File

@ -0,0 +1,392 @@
suites:
-
name: Macros
suites:
-
name: References and calls
resources:
-
source: |-
-bar = Bar
term-ref = {-bar}
term-call = {-bar()}
tests:
-
name: terms can be referenced without parens
asserts:
-
id: term-ref
value: Bar
-
name: terms can be parameterized
asserts:
-
id: term-call
value: Bar
-
name: Passing arguments
resources:
-
source: |-
-foo = Foo {$arg}
ref-foo = {-foo}
call-foo-no-args = {-foo()}
call-foo-with-expected-arg = {-foo(arg: 1)}
call-foo-with-other-arg = {-foo(other: 3)}
tests:
-
name: Not parameterized, no externals
asserts:
-
id: ref-foo
value: Foo {$arg}
-
name: Not parameterized but with externals
asserts:
-
id: ref-foo
args:
arg: 1
value: Foo {$arg}
-
name: No arguments, no externals
asserts:
-
id: call-foo-no-args
value: Foo {$arg}
-
name: No arguments, but with externals
asserts:
-
id: call-foo-no-args
args:
arg: 1
value: Foo {$arg}
-
name: With expected args, no externals
asserts:
-
id: call-foo-with-expected-arg
value: Foo 1
-
name: With expected args, and with externals
asserts:
-
id: call-foo-with-expected-arg
args:
arg: 5
value: Foo 1
-
name: With other args, no externals
asserts:
-
id: call-foo-with-other-arg
value: Foo {$arg}
-
name: With other args, and with externals
asserts:
-
id: call-foo-with-other-arg
args:
arg: 5
value: Foo {$arg}
-
name: Nesting message references
resources:
-
source: |-
foo = Foo {$arg}
-bar = {foo}
ref-bar = {-bar}
call-bar = {-bar()}
call-bar-with-arg = {-bar(arg: 1)}
tests:
-
name: No parameterization, no externals
asserts:
-
id: ref-bar
value: Foo {$arg}
-
name: No parameterization, but with externals
asserts:
-
id: ref-bar
args:
arg: 5
value: Foo {$arg}
-
name: No arguments, no externals
asserts:
-
id: call-bar
value: Foo {$arg}
-
name: No arguments, but with externals
asserts:
-
id: call-bar
args:
arg: 5
value: Foo {$arg}
-
name: With arguments, no externals
asserts:
-
id: call-bar-with-arg
value: Foo 1
-
name: With arguments and with externals
asserts:
-
id: call-bar-with-arg
args:
arg: 5
value: Foo 1
-
name: Nesting term references
resources:
-
source: |-
-foo = Foo {$arg}
-bar = {-foo}
-baz = {-foo()}
-qux = {-foo(arg: 1)}
ref-bar = {-bar}
ref-baz = {-baz}
ref-qux = {-qux}
call-bar-no-args = {-bar()}
call-baz-no-args = {-baz()}
call-qux-no-args = {-qux()}
call-bar-with-arg = {-bar(arg: 2)}
call-baz-with-arg = {-baz(arg: 2)}
call-qux-with-arg = {-qux(arg: 2)}
call-qux-with-other = {-qux(other: 3)}
tests:
-
name: No parameterization, no parameterization, no externals
asserts:
-
id: ref-bar
value: Foo {$arg}
-
name: No parameterization, no parameterization, with externals
asserts:
-
id: ref-bar
args:
arg: 5
value: Foo {$arg}
-
name: No parameterization, no arguments, no externals
asserts:
-
id: ref-baz
value: Foo {$arg}
-
name: No parameterization, no arguments, with externals
asserts:
-
id: ref-baz
args:
arg: 5
value: Foo {$arg}
-
name: No parameterization, with arguments, no externals
asserts:
-
id: ref-qux
value: Foo 1
-
name: No parameterization, with arguments, with externals
asserts:
-
id: ref-qux
args:
arg: 5
value: Foo 1
-
name: No arguments, no parametrization, no externals
asserts:
-
id: call-bar-no-args
value: Foo {$arg}
-
name: No arguments, no parametrization, with externals
asserts:
-
id: call-bar-no-args
args:
arg: 5
value: Foo {$arg}
-
name: No arguments, no arguments, no externals
asserts:
-
id: call-baz-no-args
value: Foo {$arg}
-
name: No arguments, no arguments, with externals
asserts:
-
id: call-baz-no-args
args:
arg: 5
value: Foo {$arg}
-
name: No arguments, with arguments, no externals
asserts:
-
id: call-qux-no-args
value: Foo 1
-
name: No arguments, with arguments, with externals
asserts:
-
id: call-qux-no-args
args:
arg: 5
value: Foo 1
-
name: With arguments, no parametrization, no externals
asserts:
-
id: call-bar-with-arg
value: Foo {$arg}
-
name: With arguments, no parametrization, with externals
asserts:
-
id: call-bar-with-arg
args:
arg: 5
value: Foo {$arg}
-
name: With arguments, no arguments, no externals
asserts:
-
id: call-baz-with-arg
value: Foo {$arg}
-
name: With arguments, no arguments, with externals
asserts:
-
id: call-baz-with-arg
args:
arg: 5
value: Foo {$arg}
-
name: With arguments, with arguments, no externals
asserts:
-
id: call-qux-with-arg
value: Foo 1
-
name: With arguments, with arguments, with externals
asserts:
-
id: call-qux-with-arg
args:
arg: 5
value: Foo 1
-
name: With unexpected arguments, with arguments, no externals
asserts:
-
id: call-qux-with-other
value: Foo 1
-
name: With unexpected arguments, with arguments, with externals
asserts:
-
id: call-qux-with-other
args:
arg: 5
value: Foo 1
-
name: Parameterized term attributes
resources:
-
source: |-
-ship = Ship
.gender = {$style ->
*[traditional] neuter
[chicago] feminine
}
ref-attr = {-ship.gender ->
*[masculine] He
[feminine] She
[neuter] It
}
call-attr-no-args = {-ship.gender() ->
*[masculine] He
[feminine] She
[neuter] It
}
call-attr-with-expected-arg = {-ship.gender(style: "chicago") ->
*[masculine] He
[feminine] She
[neuter] It
}
call-attr-with-other-arg = {-ship.gender(other: 3) ->
*[masculine] He
[feminine] She
[neuter] It
}
tests:
-
name: Not parameterized, no externals
asserts:
-
id: ref-attr
value: It
-
name: Not parameterized but with externals
asserts:
-
id: ref-attr
args:
attr: chicago
value: It
-
name: No arguments, no externals
asserts:
-
id: call-attr-no-args
value: It
-
name: No arguments, but with externals
asserts:
-
id: call-attr-no-args
args:
style: chicago
value: It
-
name: With expected args, no externals
asserts:
-
id: call-attr-with-expected-arg
value: She
-
name: With expected args, and with externals
asserts:
-
id: call-attr-with-expected-arg
args:
style: chicago
value: She
-
name: With other args, no externals
asserts:
-
id: call-attr-with-other-arg
value: It
-
name: With other args, and with externals
asserts:
-
id: call-attr-with-other-arg
args:
style: chicago
value: It

218
third_party/rust/fluent-bundle/tests/fixtures/patterns.yaml vendored Normal file
View File

@ -0,0 +1,218 @@
suites:
-
name: Patterns
suites:
-
name: Simple string value
resources:
-
source: foo = Foo
tests:
-
name: returns the value
asserts:
-
id: foo
value: Foo
-
name: Complex string value
resources:
-
source: |-
foo = Foo
-bar = Bar
ref-message = { foo }
ref-term = { -bar }
ref-missing-message = { missing }
ref-missing-term = { -missing }
ref-malformed = { malformed
errors:
-
type: Parser
tests:
-
name: resolves the reference to a message
asserts:
-
id: ref-message
value: Foo
-
name: resolves the reference to a term
asserts:
-
id: ref-term
value: Bar
-
name: returns the id if a message reference is missing
asserts:
-
id: ref-missing-message
value: "{missing}"
errors:
-
type: Reference
desc: "Unknown message: missing"
-
name: returns the id if a term reference is missing
asserts:
-
id: ref-missing-term
value: "{-missing}"
errors:
-
type: Reference
desc: "Unknown term: -missing"
-
name: Complex string referencing a message with null value
resources:
-
source: |-
foo =
.attr = Foo Attr
bar = { foo } Bar
tests:
-
name: returns the null value
skip: true
asserts:
-
id: foo
value: Foo
-
name: formats the attribute
asserts:
-
id: foo
attribute: attr
value: Foo Attr
-
name: formats ??? when the referenced message has no value and no default
asserts:
-
id: bar
# XXX: Difference from JS
value: "{foo} Bar"
errors:
-
type: Reference
desc: "Unknown message: foo"
-
name: Cyclic reference
resources:
-
source: |-
foo = { bar }
bar = { foo }
tests:
-
name: returns ???
asserts:
-
id: foo
value: "{foo}"
errors:
-
type: Cyclic
-
name: Cyclic self-reference
resources:
-
source: foo = { foo }
tests:
-
name: returns the raw string
asserts:
-
id: foo
value: "{foo}"
errors:
-
type: Cyclic
-
name: Cyclic self-reference in a member
resources:
-
source: |-
foo =
{ $sel ->
*[a] { foo }
[b] Bar
}
bar = { foo }
tests:
-
name: returns ???
asserts:
-
id: foo
args:
sel: a
value: "{foo}"
errors:
-
type: Cyclic
-
name: returns the other member if requested
asserts:
-
id: foo
args:
sel: b
value: Bar
-
name: Cyclic reference in a selector
skip: true
resources:
-
source: |-
-foo =
{ -bar.attr ->
*[a] Foo
}
-bar = Bar
.attr = { -foo }
foo = { -foo }
tests:
-
name: returns the default variant
asserts:
-
id: foo
value: Foo
errors:
-
type: Cyclic
-
name: Cyclic self-reference in a selector
skip: true
resources:
-
source: |-
-foo =
{ -bar.attr ->
*[a] Foo
}
.attr = a
-bar =
{ -foo.attr ->
*[a] Bar
}
.attr = { -foo }
foo = { -foo }
bar = { -bar }
tests:
-
name: returns the default variant
asserts:
-
id: foo
value: Foo
errors:
-
type: Cyclic
-
name: can reference an attribute
asserts:
-
id: bar
value: Bar

154
third_party/rust/fluent-bundle/tests/fixtures/primitives.yaml vendored Normal file
View File

@ -0,0 +1,154 @@
suites:
-
name: Primitives
suites:
-
name: Numbers
resources:
-
source: |-
one = { 1 }
select = { 1 ->
*[0] Zero
[1] One
}
tests:
-
name: can be used in a placeable
asserts:
-
id: one
value: 1
-
name: can be used as a selector
asserts:
-
id: select
value: One
-
name: Simple string value
resources:
-
source: |-
foo = Foo
placeable-literal = { "Foo" } Bar
placeable-message = { foo } Bar
selector-literal = { "Foo" ->
*[Foo] Member 1
}
bar =
.attr = Bar Attribute
placeable-attr = { bar.attr }
-baz = Baz
.attr = BazAttribute
selector-attr = { -baz.attr ->
*[BazAttribute] Member 3
}
tests:
-
name: can be used as a value
asserts:
-
id: foo
value: Foo
-
name: can be used in a placeable
asserts:
-
id: placeable-literal
value: Foo Bar
-
name: can be a value of a message referenced in a placeable
asserts:
-
id: placeable-message
value: Foo Bar
-
name: can be a selector
asserts:
-
id: selector-literal
value: Member 1
-
name: can be used as an attribute value
asserts:
-
id: bar
attribute: attr
value: Bar Attribute
-
name: can be a value of an attribute used in a placeable
asserts:
-
id: placeable-attr
value: Bar Attribute
-
name: can be a value of an attribute used as a selector
asserts:
-
id: selector-attr
value: Member 3
-
name: Complex string value
resources:
-
source: |-
foo = Foo
bar = { foo }Bar
placeable-message = { bar }Baz
baz =
.attr = { bar }BazAttribute
-bazTerm = Value
.attr = { bar }BazAttribute
placeable-attr = { baz.attr }
# XXX: This is different from JS fixture which
# illegally uses message attribute as selector.
selector-attr = { -bazTerm.attr ->
[FooBarBazAttribute] FooBarBaz
*[other] Other
}
tests:
-
name: can be used as a value
asserts:
-
id: bar
value: FooBar
-
name: can be a value of a message referenced in a placeable
asserts:
-
id: placeable-message
value: FooBarBaz
-
name: can be used as an attribute value
asserts:
-
id: baz
attribute: attr
value: FooBarBazAttribute
-
name: can be a value of an attribute used in a placeable
asserts:
-
id: placeable-attr
value: FooBarBazAttribute
-
name: can be a value of an attribute used as a selector
asserts:
-
id: selector-attr
value: FooBarBaz
-
name: (Rust) Placeable
resources:
-
source: |-
foo = { { "Foo" } }
tests:
-
name: Placeable in placable work
asserts:
-
id: foo
value: Foo

151
third_party/rust/fluent-bundle/tests/fixtures/select_expression.yaml vendored Normal file
View File

@ -0,0 +1,151 @@
suites:
-
name: Select expressions
tests:
-
name: missing selector
resources:
-
source: |-
select = {$none ->
[a] A
*[b] B
}
asserts:
-
id: select
value: B
errors:
-
type: Reference
desc: "Unknown variable: $none"
suites:
-
name: string selectors
tests:
-
name: matching selector
resources:
-
source: |-
select = {$selector ->
[a] A
*[b] B
}
asserts:
-
id: select
value: A
args:
selector: a
-
name: non-matching selector
resources:
-
source: |-
select = {$selector ->
[a] A
*[b] B
}
asserts:
-
id: select
value: B
args:
selector: c
-
name: number selectors
tests:
-
name: matching selector
resources:
-
source: |-
select = {$selector ->
[0] A
*[1] B
}
asserts:
-
id: select
value: A
args:
selector: 0
-
name: non-matching selector
resources:
-
source: |-
select = {$selector ->
[0] A
*[1] B
}
asserts:
-
id: select
value: B
args:
selector: 2
-
name: plural categories
tests:
-
name: matching number selector
resources:
-
source: |-
select = {$selector ->
[one] A
*[other] B
}
asserts:
-
id: select
value: A
args:
selector: 1
-
name: matching string selector
resources:
-
source: |-
select = {$selector ->
[one] A
*[other] B
}
asserts:
-
id: select
value: A
args:
selector: one
-
name: non-matching number selector
resources:
-
source: |-
select = {$selector ->
[one] A
*[default] D
}
asserts:
-
id: select
value: D
args:
selector: 2
-
name: non-matching string selector
resources:
-
source: |-
select = {$selector ->
[one] A
*[default] D
}
asserts:
-
id: select
value: D
args:
selector: other

48
third_party/rust/fluent-bundle/tests/fixtures/transform.yaml vendored Normal file
View File

@ -0,0 +1,48 @@
suites:
-
name: Transformations
resources:
-
source: |-
foo = Faa
.bar = Bar {foo} Baz
bar = Bar {"Baz"}
qux = {"faa" ->
[faa] Faa
*[bar] Bar
}
arg = Faa {$arg}
bundles:
-
transform: example
tests:
-
name: transforms TextElements
asserts:
-
id: foo
value: FAA
-
id: foo
attribute: bar
value: BAr FAA BAz
-
name: does not transform StringLiterls
asserts:
-
id: bar
value: BAr Baz
-
name: does not transform VariantKeys
asserts:
-
id: qux
value: FAA
-
name: does not transform Variables
asserts:
-
id: arg
args:
arg: aaa
value: FAA aaa

74
third_party/rust/fluent-bundle/tests/fixtures/values_format.yaml vendored Normal file
View File

@ -0,0 +1,74 @@
suites:
-
name: Formatting values
resources:
-
source: |-
key1 = Value 1
key2 = { $sel ->
[a] A2
*[b] B2
}
key3 = Value { 3 }
key4 = { $sel ->
[a] A{ 4 }
*[b] B{ 4 }
}
key5 =
.a = A5
.b = B5
tests:
-
name: returns the value
asserts:
-
id: key1
value: Value 1
-
name: returns the default variant
asserts:
-
id: key2
value: B2
errors:
-
type: Reference
desc: "Unknown variable: $sel"
-
name: returns the value if it is a pattern
asserts:
-
id: key3
value: Value 3
-
name: returns the default variant if it is a pattern
asserts:
-
id: key4
value: B4
errors:
-
type: Reference
desc: "Unknown variable: $sel"
-
name: returns {???} when trying to format a null value
skip: true
asserts:
-
id: key5
value: "{???}"
errors:
-
type: Reference
desc: "Unknown variable: $sel"
-
name: allows to pass traits directly to bundle.formatPattern
asserts:
-
id: key5
attribute: a
value: "A5"
-
id: key5
attribute: b
value: "B5"

140
third_party/rust/fluent-bundle/tests/fixtures/values_ref.yaml vendored Normal file
View File

@ -0,0 +1,140 @@
suites:
-
name: Referencing values
resources:
-
source: |-
key1 = Value 1
-key2 = { $sel ->
[a] A2
*[b] B2
}
key3 = Value { 3 }
-key4 = { $sel ->
[a] A{ 4 }
*[b] B{ 4 }
}
key5 =
.a = A5
.b = B5
ref1 = { key1 }
ref2 = { -key2 }
ref3 = { key3 }
ref4 = { -key4 }
ref5 = { key5 }
ref6 = { -key2(sel: "a") }
ref7 = { -key2(sel: "b") }
ref8 = { -key4(sel: "a") }
ref9 = { -key4(sel: "b") }
ref10 = { key5.a }
ref11 = { key5.b }
ref12 = { key5.c }
ref13 = { key6 }
ref14 = { key6.a }
ref15 = { -key6 }
ref16 = { -key6.a ->
*[a] A
}
tests:
-
name: references the value
asserts:
-
id: ref1
value: Value 1
-
name: references the default variant
asserts:
-
id: ref2
value: B2
-
name: references the value if it is a pattern
asserts:
-
id: ref3
value: Value 3
-
name: references the default variant if it is a pattern
asserts:
-
id: ref4
value: B4
-
name: falls back to id if there is no value
asserts:
-
id: ref5
value: "{key5}"
errors:
-
type: Reference
desc: "Unknown message: key5"
-
name: references the variants
asserts:
-
id: ref6
value: A2
-
id: ref7
value: B2
-
name: references the variants which are patterns
asserts:
-
id: ref8
value: A4
-
id: ref9
value: B4
-
name: references the attributes
asserts:
-
id: ref10
value: A5
-
id: ref11
value: B5
-
id: ref12
value: "{key5.c}"
errors:
-
type: Reference
desc: "Unknown attribute: key5.c"
-
name: missing message reference
asserts:
-
id: ref13
value: "{key6}"
errors:
-
type: Reference
desc: "Unknown message: key6"
-
id: ref14
value: "{key6.a}"
errors:
-
type: Reference
desc: "Unknown attribute: key6.a"
-
name: missing term reference
asserts:
-
id: ref15
value: "{-key6}"
errors:
-
type: Reference
desc: "Unknown term: -key6"
-
id: ref16
value: "A"
errors:
-
type: Reference
desc: "Unknown attribute: -key6.a"

202
third_party/rust/fluent-bundle/tests/helpers/mod.rs vendored Normal file
View File

@ -0,0 +1,202 @@
use std::collections::HashMap;
use std::fs::File;
use std::io;
use std::io::Read;
use serde::{Deserialize, Serialize};
#[macro_export]
macro_rules! test {
($desc:stmt, $closure:block) => {{
$closure
}};
}
#[macro_export]
macro_rules! assert_format {
($bundle:expr, $id:expr, $args:expr, $expected:expr) => {
let msg = $bundle.get_message($id).expect("Message doesn't exist");
let mut errors = vec![];
assert!(msg.value.is_some());
assert_eq!(
$bundle.format_pattern(&msg.value.unwrap(), $args, &mut errors),
$expected
);
assert!(errors.is_empty());
};
($bundle:expr, $id:expr, $args:expr, $expected:expr, $errors:expr) => {
let msg = $bundle.get_message($id).expect("Message doesn't exist.");
let mut errors = vec![];
assert!(msg.value.is_some());
assert_eq!(
$bundle.format_pattern(&msg.value.unwrap(), $args, &mut errors),
$expected
);
assert_eq!(errors, $errors);
};
}
#[macro_export]
macro_rules! assert_format_none {
($bundle:expr, $id:expr) => {
let msg = $bundle.get_message($id).expect("Message doesn't exist");
assert!(msg.value.is_none());
};
}
#[macro_export]
macro_rules! assert_format_attr {
($bundle:expr, $id:expr, $name:expr, $args:expr, $expected:expr) => {
let msg = $bundle.get_message($id).expect("Message doesn't exists");
let mut errors = vec![];
let attr = msg.attributes.get($name).expect("Attribute exists");
assert_eq!($bundle.format_pattern(&attr, $args, &mut errors), $expected);
assert!(errors.is_empty());
};
}
#[macro_export]
macro_rules! assert_get_resource_from_str {
($source:expr) => {
FluentResource::try_new($source.to_owned()).expect("Failed to parse an FTL resource.")
};
}
#[macro_export]
macro_rules! assert_get_bundle {
($res:expr) => {{
let mut bundle: FluentBundle<&FluentResource> = FluentBundle::new(&["x-testing"]);
bundle.set_use_isolating(false);
bundle
.add_resource($res)
.expect("Failed to add FluentResource to FluentBundle.");
bundle
}};
}
pub fn get_fixture(path: &str) -> Result<TestFixture, io::Error> {
let mut f = File::open(path)?;
let mut s = String::new();
f.read_to_string(&mut s)?;
Ok(serde_yaml::from_str(&s).expect("Parsing YAML failed."))
}
pub fn get_defaults(path: &str) -> Result<TestDefaults, io::Error> {
let mut f = File::open(path)?;
let mut s = String::new();
f.read_to_string(&mut s)?;
Ok(serde_yaml::from_str(&s).expect("Parsing YAML failed."))
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct TestBundle {
pub name: Option<String>,
pub locales: Option<Vec<String>>,
pub resources: Option<Vec<String>>,
#[serde(rename = "useIsolating")]
pub use_isolating: Option<bool>,
pub functions: Option<Vec<String>>,
pub transform: Option<String>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub errors: Vec<TestError>,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct TestResource {
pub name: Option<String>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub errors: Vec<TestError>,
pub source: String,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct TestSetup {
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub bundles: Vec<TestBundle>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub resources: Vec<TestResource>,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct TestError {
#[serde(rename = "type")]
pub error_type: String,
pub desc: Option<String>,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
#[serde(untagged)]
pub enum TestArgumentValue {
String(String),
Number(f64),
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct TestAssert {
pub bundle: Option<String>,
pub id: String,
pub attribute: Option<String>,
pub value: Option<String>,
pub args: Option<HashMap<String, TestArgumentValue>>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub errors: Vec<TestError>,
pub missing: Option<bool>,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct Test {
pub name: String,
pub skip: Option<bool>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub bundles: Vec<TestBundle>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub resources: Vec<TestResource>,
pub asserts: Vec<TestAssert>,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct TestSuite {
pub name: String,
pub skip: Option<bool>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub bundles: Vec<TestBundle>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub resources: Vec<TestResource>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub tests: Vec<Test>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub suites: Vec<TestSuite>,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct TestFixture {
pub suites: Vec<TestSuite>,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct BundleDefaults {
#[serde(rename = "useIsolating")]
pub use_isolating: Option<bool>,
pub transform: Option<String>,
pub locales: Option<Vec<String>>,
}
#[derive(Debug, PartialEq, Serialize, Deserialize, Clone)]
#[serde(deny_unknown_fields)]
pub struct TestDefaults {
pub bundle: BundleDefaults,
}

363
third_party/rust/fluent-bundle/tests/resolver_fixtures.rs vendored Normal file
View File

@ -0,0 +1,363 @@
mod helpers;
use std::borrow::Cow;
use std::collections::HashMap;
use std::fs;
use std::iter;
use std::path::Path;
use fluent_bundle::resolve::ResolverError;
use fluent_bundle::FluentArgs;
use fluent_bundle::FluentError;
use fluent_bundle::{FluentBundle as FluentBundleGeneric, FluentResource, FluentValue};
use rand::distributions::Alphanumeric;
use rand::{thread_rng, Rng};
use unic_langid::LanguageIdentifier;
use helpers::*;
type FluentBundle = FluentBundleGeneric<FluentResource>;
fn transform_example(s: &str) -> Cow<str> {
s.replace("a", "A").into()
}
#[derive(Clone)]
struct ScopeLevel {
name: String,
resources: Vec<TestResource>,
bundles: Vec<TestBundle>,
}
#[derive(Clone)]
struct Scope(Vec<ScopeLevel>);
impl Scope {
fn get_path(&self) -> String {
self.0
.iter()
.map(|lvl| lvl.name.as_str())
.collect::<Vec<&str>>()
.join(" > ")
}
fn get_bundles(&self, defaults: &Option<TestDefaults>) -> HashMap<String, FluentBundle> {
let mut bundles = HashMap::new();
let mut available_resources = vec![];
for lvl in self.0.iter() {
for r in lvl.resources.iter() {
available_resources.push(r);
}
for b in lvl.bundles.iter() {
let name = b
.name
.as_ref()
.cloned()
.unwrap_or_else(|| generate_random_hash());
let bundle = create_bundle(Some(b), &defaults, &available_resources);
bundles.insert(name, bundle);
}
}
if bundles.is_empty() {
let bundle = create_bundle(None, defaults, &available_resources);
let name = generate_random_hash();
bundles.insert(name.clone(), bundle);
}
bundles
}
}
fn generate_random_hash() -> String {
let mut rng = thread_rng();
let chars: String = iter::repeat(())
.map(|()| rng.sample(Alphanumeric))
.take(7)
.collect();
chars
}
fn test_fixture(fixture: &TestFixture, defaults: &Option<TestDefaults>) {
for suite in &fixture.suites {
test_suite(&suite, defaults, Scope(vec![]));
}
}
fn create_bundle(
b: Option<&TestBundle>,
defaults: &Option<TestDefaults>,
resources: &Vec<&TestResource>,
) -> FluentBundle {
let mut errors = vec![];
let locales: Vec<LanguageIdentifier> = b
.and_then(|b| b.locales.as_ref())
.or_else(|| {
defaults
.as_ref()
.and_then(|defaults| defaults.bundle.locales.as_ref())
})
.map(|locs| {
locs.into_iter()
.map(|s| s.parse().expect("Parsing failed."))
.collect()
})
.expect("Failed to calculate locales.");
let mut bundle = FluentBundle::new(&locales);
let use_isolating = b.and_then(|b| b.use_isolating).or_else(|| {
defaults
.as_ref()
.and_then(|defaults| defaults.bundle.use_isolating)
});
if let Some(use_isolating) = use_isolating {
bundle.set_use_isolating(use_isolating);
}
let transform = b.and_then(|b| b.transform.as_ref()).or_else(|| {
defaults
.as_ref()
.and_then(|defaults| defaults.bundle.transform.as_ref())
});
if let Some(transform) = transform {
match transform.as_str() {
"example" => bundle.set_transform(Some(transform_example)),
_ => unimplemented!(),
}
}
if let Some(&TestBundle {
functions: Some(ref fns),
..
}) = b
{
for f in fns {
let result = match f.as_str() {
"CONCAT" => bundle.add_function(f.as_str(), |args, _name_args| {
args.iter()
.fold(String::new(), |acc, x| match x {
FluentValue::String(s) => acc + &s.to_string(),
FluentValue::Number(n) => acc + &n.value.to_string(),
_ => acc,
})
.into()
}),
"SUM" => bundle.add_function(f.as_str(), |args, _name_args| {
args.iter()
.fold(0.0, |acc, x| {
if let FluentValue::Number(v) = x {
acc + v.value
} else {
panic!("Type cannot be used in SUM");
}
})
.into()
}),
"IDENTITY" => bundle.add_function(f.as_str(), |args, _name_args| {
args.get(0).cloned().unwrap_or(FluentValue::None)
}),
"NUMBER" => bundle.add_function(f.as_str(), |args, _name_args| {
args.get(0).expect("Argument must be passed").clone()
}),
_ => unimplemented!("No such function."),
};
if let Err(err) = result {
errors.push(err);
}
}
}
let res_subset = b.and_then(|b| b.resources.as_ref());
for res in resources.iter() {
if let Some(res_subset) = res_subset {
if let Some(ref name) = res.name {
if !res_subset.contains(name) {
continue;
}
}
}
let res = get_resource(res);
if let Err(mut err) = bundle.add_resource(res) {
errors.append(&mut err);
}
}
test_errors(&errors, b.map(|b| b.errors.as_ref()));
bundle
}
fn get_resource(resource: &TestResource) -> FluentResource {
let res = FluentResource::try_new(resource.source.clone());
if resource.errors.is_empty() {
res.expect("Failed to parse an FTL resource.")
} else {
let (res, errors) = match res {
Ok(r) => (r, vec![]),
Err((res, err)) => {
let err = err.into_iter().map(|err| err.into()).collect();
(res, err)
}
};
test_errors(&errors, Some(&resource.errors));
res
}
}
fn test_suite(suite: &TestSuite, defaults: &Option<TestDefaults>, mut scope: Scope) {
if suite.skip == Some(true) {
return;
}
scope.0.push(ScopeLevel {
name: suite.name.clone(),
bundles: suite.bundles.clone(),
resources: suite.resources.clone(),
});
for test in &suite.tests {
test_test(test, defaults, scope.clone());
}
for sub_suite in &suite.suites {
test_suite(sub_suite, defaults, scope.clone());
}
}
fn test_test(test: &Test, defaults: &Option<TestDefaults>, mut scope: Scope) {
if test.skip == Some(true) {
return;
}
scope.0.push(ScopeLevel {
name: test.name.clone(),
bundles: test.bundles.clone(),
resources: test.resources.clone(),
});
for assert in &test.asserts {
let bundles = scope.get_bundles(defaults);
let bundle = if let Some(ref bundle_name) = assert.bundle {
bundles
.get(bundle_name)
.expect("Failed to retrieve bundle.")
} else if bundles.len() == 1 {
let name = bundles.keys().into_iter().last().unwrap();
bundles.get(name).expect("Failed to retrieve bundle.")
} else {
panic!();
};
let mut errors = vec![];
if let Some(expected_missing) = assert.missing {
let missing = if let Some(ref attr) = assert.attribute {
if let Some(msg) = bundle.get_message(&assert.id) {
msg.attributes.contains_key(attr.as_str())
} else {
false
}
} else {
!bundle.has_message(&assert.id)
};
assert_eq!(
missing,
expected_missing,
"Expected pattern to be `missing: {}` for {} in {}",
expected_missing,
assert.id,
scope.get_path()
);
} else {
if let Some(ref expected_value) = assert.value {
let msg = bundle.get_message(&assert.id).expect(&format!(
"Failed to retrieve message `{}` in {}.",
&assert.id,
scope.get_path()
));
let val = if let Some(ref attr) = assert.attribute {
msg.attributes.get(attr.as_str()).expect(&format!(
"Failed to retrieve an attribute of a message {}.{}.",
assert.id, attr
))
} else {
msg.value.expect(&format!(
"Failed to retrieve a value of a message {}.",
assert.id
))
};
let args: Option<FluentArgs> = assert.args.as_ref().map(|args| {
args.iter()
.map(|(k, v)| {
let val = match v {
TestArgumentValue::String(s) => s.as_str().into(),
TestArgumentValue::Number(n) => n.into(),
};
(k.as_str(), val)
})
.collect()
});
let value = bundle.format_pattern(&val, args.as_ref(), &mut errors);
assert_eq!(
&value,
expected_value,
"Values don't match in {}",
scope.get_path()
);
test_errors(&errors, Some(&assert.errors));
} else {
panic!("Value field expected.");
}
}
}
}
fn test_errors(errors: &[FluentError], reference: Option<&[TestError]>) {
let reference = reference.unwrap_or(&[]);
assert_eq!(errors.len(), reference.len());
for (error, reference) in errors.into_iter().zip(reference) {
match error {
FluentError::ResolverError(err) => match err {
ResolverError::Reference(desc) => {
assert_eq!(reference.desc.as_ref(), Some(desc));
assert_eq!(reference.error_type, "Reference");
}
ResolverError::Cyclic => {
assert_eq!(reference.error_type, "Cyclic");
}
ResolverError::TooManyPlaceables => {
assert_eq!(reference.error_type, "TooManyPlaceables");
}
_ => unimplemented!(),
},
FluentError::ParserError(_) => {
assert_eq!(reference.error_type, "Parser");
}
FluentError::Overriding { .. } => {
assert_eq!(reference.error_type, "Overriding");
}
}
}
}
#[test]
fn resolve_fixtures() {
let dir = "./tests/fixtures/";
let mut defaults_path = String::from(dir);
defaults_path.push_str("defaults.yaml");
let defaults = if Path::new(&defaults_path).exists() {
Some(get_defaults(&defaults_path).expect("Failed to read defaults."))
} else {
None
};
for entry in fs::read_dir(dir).expect("Failed to read glob pattern.") {
let entry = entry.expect("Entry doesn't exist.");
let path = entry.path();
let path_str = path.to_str().expect("Failed to convert path to string.");
if path_str.contains("defaults.yaml") {
continue;
}
println!("PATH: {:#?}", path_str);
let fixture = get_fixture(path_str).expect("Loading fixture failed.");
test_fixture(&fixture, &defaults);
}
}

156
third_party/rust/fluent-bundle/tests/types_test.rs vendored Normal file
View File

@ -0,0 +1,156 @@
use fluent_bundle::resolve::Scope;
use fluent_bundle::types::{
FluentNumber, FluentNumberCurrencyDisplayStyle, FluentNumberOptions, FluentNumberStyle,
};
use fluent_bundle::FluentArgs;
use fluent_bundle::FluentBundle;
use fluent_bundle::FluentResource;
use fluent_bundle::FluentValue;
use intl_pluralrules::operands::PluralOperands;
use unic_langid::langid;
#[test]
fn fluent_value_try_number() {
let value = FluentValue::try_number("invalid");
assert_eq!(value, "invalid".into());
}
#[test]
fn fluent_value_matches() {
// We'll use `ars` locale since it happens to have all
// plural rules categories.
let langid_ars = langid!("ars");
let bundle: FluentBundle<FluentResource> = FluentBundle::new(&[langid_ars]);
let scope = Scope::new(&bundle, None);
let string_val = FluentValue::from("string1");
let string_val_copy = FluentValue::from("string1");
let string_val2 = FluentValue::from("23.5");
let number_val = FluentValue::from(-23.5);
let number_val_copy = FluentValue::from(-23.5);
let number_val2 = FluentValue::from(23.5);
assert_eq!(string_val.matches(&string_val_copy, &scope), true);
assert_eq!(string_val.matches(&string_val2, &scope), false);
assert_eq!(number_val.matches(&number_val_copy, &scope), true);
assert_eq!(number_val.matches(&number_val2, &scope), false);
assert_eq!(string_val2.matches(&number_val2, &scope), false);
assert_eq!(string_val2.matches(&number_val2, &scope), false);
let string_cat_zero = FluentValue::from("zero");
let string_cat_one = FluentValue::from("one");
let string_cat_two = FluentValue::from("two");
let string_cat_few = FluentValue::from("few");
let string_cat_many = FluentValue::from("many");
let string_cat_other = FluentValue::from("other");
let number_cat_zero = 0.into();
let number_cat_one = 1.into();
let number_cat_two = 2.into();
let number_cat_few = 3.into();
let number_cat_many = 11.into();
let number_cat_other = 101.into();
assert_eq!(string_cat_zero.matches(&number_cat_zero, &scope), true);
assert_eq!(string_cat_one.matches(&number_cat_one, &scope), true);
assert_eq!(string_cat_two.matches(&number_cat_two, &scope), true);
assert_eq!(string_cat_few.matches(&number_cat_few, &scope), true);
assert_eq!(string_cat_many.matches(&number_cat_many, &scope), true);
assert_eq!(string_cat_other.matches(&number_cat_other, &scope), true);
assert_eq!(string_cat_other.matches(&number_cat_one, &scope), false);
assert_eq!(string_val2.matches(&number_cat_one, &scope), false);
}
#[test]
fn fluent_value_from() {
let value_str = FluentValue::from("my str");
let value_string = FluentValue::from(String::from("my string"));
let value_f64 = FluentValue::from(23.5);
let value_isize = FluentValue::from(-23);
assert_eq!(value_str, "my str".into());
assert_eq!(value_string, "my string".into());
assert_eq!(value_f64, FluentValue::from(23.5));
assert_eq!(value_isize, FluentValue::from(-23));
}
#[test]
fn fluent_number_style() {
let fns_decimal: FluentNumberStyle = "decimal".into();
let fns_currency: FluentNumberStyle = "currency".into();
let fns_percent: FluentNumberStyle = "percent".into();
let fns_decimal2: FluentNumberStyle = "other".into();
assert_eq!(fns_decimal, FluentNumberStyle::Decimal);
assert_eq!(fns_currency, FluentNumberStyle::Currency);
assert_eq!(fns_percent, FluentNumberStyle::Percent);
assert_eq!(fns_decimal2, FluentNumberStyle::Decimal);
let fncds_symbol: FluentNumberCurrencyDisplayStyle = "symbol".into();
let fncds_code: FluentNumberCurrencyDisplayStyle = "code".into();
let fncds_name: FluentNumberCurrencyDisplayStyle = "name".into();
let fncds_symbol2: FluentNumberCurrencyDisplayStyle = "other".into();
assert_eq!(fncds_symbol, FluentNumberCurrencyDisplayStyle::Symbol);
assert_eq!(fncds_code, FluentNumberCurrencyDisplayStyle::Code);
assert_eq!(fncds_name, FluentNumberCurrencyDisplayStyle::Name);
assert_eq!(fncds_symbol2, FluentNumberCurrencyDisplayStyle::Symbol);
let mut fno = FluentNumberOptions::default();
let mut args = FluentArgs::new();
args.insert("style", "currency".into());
args.insert("currency", "EUR".into());
args.insert("currencyDisplay", "code".into());
args.insert("useGrouping", "true".into());
args.insert("minimumIntegerDigits", 3.into());
args.insert("minimumFractionDigits", 3.into());
args.insert("maximumFractionDigits", 8.into());
args.insert("minimumSignificantDigits", 1.into());
args.insert("maximumSignificantDigits", 10.into());
args.insert("someRandomOption", 10.into());
fno.merge(&args);
assert_eq!(fno.style, FluentNumberStyle::Currency);
assert_eq!(fno.currency, Some("EUR".to_string()));
assert_eq!(fno.currency_display, FluentNumberCurrencyDisplayStyle::Code);
assert_eq!(fno.use_grouping, true);
let num = FluentNumber::new(0.2, FluentNumberOptions::default());
assert_eq!(num.as_string(), "0.2");
let opts = FluentNumberOptions {
minimum_fraction_digits: Some(3),
..Default::default()
};
let num = FluentNumber::new(0.2, opts.clone());
assert_eq!(num.as_string(), "0.200");
let num = FluentNumber::new(2.0, opts.clone());
assert_eq!(num.as_string(), "2.000");
}
#[test]
fn fluent_number_to_operands() {
let num = FluentNumber::new(2.81, FluentNumberOptions::default());
let operands: PluralOperands = (&num).into();
assert_eq!(
operands,
PluralOperands {
n: 2.81,
i: 2,
v: 2,
w: 2,
f: 81,
t: 81,
}
);
}

1
third_party/rust/fluent-pseudo/.cargo-checksum.json vendored Normal file
View File

@ -0,0 +1 @@
{"files":{"CHANGELOG.md":"643031d0ea21e0539305186fa41c5fb57889f28c49f1fc27c75b4f659522773c","Cargo.toml":"7f0730a670b8ac4edf2e8ead2f5c7e4880b5cc70c87cc466ba7c3b96a36123c1","README.md":"2bde1fc990113ff261d65e107e4c2bda4de9337761a07b9d272959585f1e57cd","src/lib.rs":"ecb33d80c8351fd53dfbeee9f3456737db72d78f0f0fbbfc3603add76d8fdbb3"},"package":"ca3a870aefc42d175d11fb1ec089221ced8a160d66ca1e0c64a57b4ae90d2462"}

16
third_party/rust/fluent-pseudo/CHANGELOG.md vendored Normal file
View File

@ -0,0 +1,16 @@
# Changelog
## Unreleased
- …
## fluent-pseudo 0.2.0 (December 13, 2019)
- Exclude access-keys and other single-char messages.
## fluent-pseudo 0.1.0 (November 26, 2019)
- Update `regex` to 1.3.
## fluent-pseudo 0.0.1 (August 1, 2019)
- This is the first release to be listed in the CHANGELOG.
- Basic support for pseudo-localization matching the fluent.js capabilities.

26
third_party/rust/fluent-pseudo/Cargo.toml vendored Normal file
View File

@ -0,0 +1,26 @@
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g., crates.io) dependencies
#
# If you believe there's an error in this file please file an
# issue against the rust-lang/cargo repository. If you're
# editing this file be aware that the upstream Cargo.toml
# will likely look very different (and much more reasonable)
[package]
edition = "2018"
name = "fluent-pseudo"
version = "0.2.1"
authors = ["Zibi Braniecki <gandalf@mozilla.com>", "Staś Małolepszy <stas@mozilla.com>"]
description = "Pseudolocalization transformation API for use with Project Fluent API.\n"
homepage = "http://www.projectfluent.org"
readme = "README.md"
keywords = ["localization", "l10n", "i18n", "intl", "internationalization"]
categories = ["localization", "internationalization"]
license = "Apache-2.0/MIT"
repository = "https://github.com/projectfluent/fluent-rs"
[dependencies.regex]
version = "1.3"

48
third_party/rust/fluent-pseudo/README.md vendored Normal file
View File

@ -0,0 +1,48 @@
# Fluent
`fluent-pseudo` is a Rust implementation of the pseudolocalization API for [Project Fluent][], a localization
framework designed to unleash the entire expressive power of natural language
translations.
[![crates.io](http://meritbadge.herokuapp.com/fluent-pseudo)](https://crates.io/crates/fluent-pseudo)
[![Build Status](https://travis-ci.org/projectfluent/fluent-rs.svg?branch=master)](https://travis-ci.org/projectfluent/fluent-rs)
[![Coverage Status](https://coveralls.io/repos/github/projectfluent/fluent-rs/badge.svg?branch=master)](https://coveralls.io/github/projectfluent/fluent-rs?branch=master)
Usage
-----
```rust
use fluent_bundle::{FluentBundle, FluentResource};
use unic_langid::langid;
use fluent_pseudo::transform;
fn transform_wrapper(s: &str) -> Cow<str> {
// Not flipped and elongated pseudolocalization.
transform(s, false, true)
}
fn main() {
let ftl_string = "hello-world = Hello, world!".to_owned();
let res = FluentResource::try_new(ftl_string)
.expect("Could not parse an FTL string.");
let langid_en = langid!("en");
let mut bundle = FluentBundle::new(&[langid_en]);
// Set pseudolocalization
bundle.set_transform(Some(transform_wrapper));
bundle.add_resource(&res)
.expect("Failed to add FTL resources to the bundle.");
let msg = bundle.get_message("hello-world")
.expect("Failed to retrieve a message.");
let val = msg.value.expect("Message has no value.");
let mut errors = vec![];
let value = bundle.format_pattern(val, None, &mut errors);
assert_eq!(&value, "Ħḗḗŀŀǿǿ Ẇǿǿřŀḓ!");
}
```

124
third_party/rust/fluent-pseudo/src/lib.rs vendored Normal file
View File

@ -0,0 +1,124 @@
use regex::Captures;
use regex::Regex;
use std::borrow::Cow;
static TRANSFORM_SMALL_MAP: &[char] = &[
'ȧ', 'ƀ', 'ƈ', 'ḓ', 'ḗ', 'ƒ', 'ɠ', 'ħ', 'ī', 'ĵ', 'ķ', 'ŀ', 'ḿ', 'ƞ', 'ǿ', 'ƥ', 'ɋ', 'ř', 'ş',
'ŧ', 'ŭ', 'ṽ', 'ẇ', 'ẋ', 'ẏ', 'ẑ',
];
static TRANSFORM_CAPS_MAP: &[char] = &[
'Ȧ', 'Ɓ', 'Ƈ', 'Ḓ', 'Ḗ', 'Ƒ', 'Ɠ', 'Ħ', 'Ī', 'Ĵ', 'Ķ', 'Ŀ', 'Ḿ', 'Ƞ', 'Ǿ', 'Ƥ', 'Ɋ', 'Ř', 'Ş',
'Ŧ', 'Ŭ', 'Ṽ', 'Ẇ', 'Ẋ', 'Ẏ', 'Ẑ',
];
static FLIPPED_SMALL_MAP: &[char] = &[
'ɐ', 'q', 'ɔ', 'p', 'ǝ', 'ɟ', 'ƃ', 'ɥ', 'ı', 'ɾ', 'ʞ', 'ʅ', 'ɯ', 'u', 'o', 'd', 'b', 'ɹ', 's',
'ʇ', 'n', 'ʌ', 'ʍ', 'x', 'ʎ', 'z',
];
static FLIPPED_CAPS_MAP: &[char] = &[
'∀', 'Ԑ', 'Ↄ', 'ᗡ', 'Ǝ', 'Ⅎ', '⅁', 'H', 'I', 'ſ', 'Ӽ', '⅂', 'W', 'N', 'O', 'Ԁ', 'Ò', 'ᴚ', 'S',
'⊥', '∩', 'Ʌ', 'M', 'X', '⅄', 'Z',
];
pub fn transform_dom(s: &str, flipped: bool, elongate: bool) -> Cow<str> {
// Exclude access-keys and other single-char messages
if s.len() == 1 {
return s.into();
}
// XML entities (&#x202a;) and XML tags.
let re_excluded = Regex::new(r"&[#\w]+;|<\s*.+?\s*>").unwrap();
let mut result = Cow::from(s);
let mut pos = 0;
let mut diff = 0;
for cap in re_excluded.captures_iter(s) {
let capture = cap.get(0).unwrap();
let sub_len = capture.start() - pos;
let range = pos..capture.start();
let result_range = pos + diff..capture.start() + diff;
let sub = &s[range.clone()];
let transform_sub = transform(&sub, false, true);
diff += transform_sub.len() - sub_len;
result
.to_mut()
.replace_range(result_range.clone(), &transform_sub);
pos = capture.end();
}
let range = pos..s.len();
let result_range = pos + diff..result.len();
let transform_sub = transform(&s[range], flipped, elongate);
result
.to_mut()
.replace_range(result_range, &transform_sub);
result
}
pub fn transform(s: &str, flipped: bool, elongate: bool) -> Cow<str> {
// XXX: avoid recreating it on each call.
let re_az = Regex::new(r"[a-zA-Z]").unwrap();
let (small_map, caps_map) = if flipped {
(FLIPPED_SMALL_MAP, FLIPPED_CAPS_MAP)
} else {
(TRANSFORM_SMALL_MAP, TRANSFORM_CAPS_MAP)
};
re_az.replace_all(s, |caps: &Captures| {
let ch = caps[0].chars().nth(0).unwrap();
let cc = ch as u8;
if cc >= 97 && cc <= 122 {
let pos = cc - 97;
let new_char = small_map[pos as usize];
if elongate && (cc == 97 || cc == 101 || cc == 111 || cc == 117) {
let mut s = new_char.to_string();
s.push(new_char);
s
} else {
new_char.to_string()
}
} else if cc >= 65 && cc <= 90 {
let pos = cc - 65;
let new_char = caps_map[pos as usize];
new_char.to_string()
} else {
ch.to_string()
}
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn it_works() {
let x = transform("Hello World", false, true);
assert_eq!(x, "Ħḗḗŀŀǿǿ Ẇǿǿřŀḓ");
let x = transform("Hello World", false, false);
assert_eq!(x, "Ħḗŀŀǿ Ẇǿřŀḓ");
let x = transform("Hello World", true, false);
assert_eq!(x, "Hǝʅʅo Moɹʅp");
let x = transform("f", false, true);
assert_eq!(x, "ƒ");
}
#[test]
fn dom_test() {
let x = transform_dom("Hello <a>World</a>", false, true);
assert_eq!(x, "Ħḗḗŀŀǿǿ <a>Ẇǿǿřŀḓ</a>");
let x = transform_dom("Hello <a>World</a> in <b>my</b> House.", false, true);
assert_eq!(x, "Ħḗḗŀŀǿǿ <a>Ẇǿǿřŀḓ</a> īƞ <b>ḿẏ</b> Ħǿǿŭŭşḗḗ.");
// Don't touch single character values.
let x = transform_dom("f", false, true);
assert_eq!(x, "f");
}
}

1
third_party/rust/fluent-syntax/.cargo-checksum.json vendored Normal file
View File

File diff suppressed because one or more lines are too long

33
third_party/rust/fluent-syntax/CHANGELOG.md vendored Normal file
View File

@ -0,0 +1,33 @@
# Changelog
## Unreleased
- …
## fluent-syntax 0.9.2 (February 13, 2020)
- Import updated tests from the reference parser.
- Minor parser improvements to align with new tests.
## fluent-syntax 0.9.1 (November 26, 2019)
- Dependency updates.
- Better test coverage.
## fluent-syntax 0.9.0 (March 26, 2019)
- Update to Fluent Syntax 0.9
- Unify benchmark testsuite with fluent.js
## fluent-syntax 0.8.0 (January 31, 2019)
- Update to Fluent Syntax 0.8
- Switch to zero-copy parser
- Start using reference FTL fixtures in tests
- Switch to criterion for benchmarks
- Rust 2018 edition
## fluent-syntax 0.1.1 (August 29, 2018)
- enable ParserError to be compared.
## fluent-syntax 0.1.0 (July 29, 2018)
- Initial release of the standalone fluent-syntax.
Based on fluent 0.2.0, and syntax 0.5

43
third_party/rust/fluent-syntax/Cargo.toml vendored Normal file
View File

@ -0,0 +1,43 @@
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g., crates.io) dependencies
#
# If you believe there's an error in this file please file an
# issue against the rust-lang/cargo repository. If you're
# editing this file be aware that the upstream Cargo.toml
# will likely look very different (and much more reasonable)
[package]
edition = "2018"
name = "fluent-syntax"
version = "0.9.2"
authors = ["Zibi Braniecki <gandalf@mozilla.com>", "Staś Małolepszy <stas@mozilla.com>"]
description = "Parser/Serializer tools for Fluent Syntax. \n"
homepage = "http://www.projectfluent.org"
readme = "README.md"
keywords = ["localization", "l10n", "i18n", "intl", "internationalization"]
categories = ["localization", "internationalization"]
license = "Apache-2.0/MIT"
repository = "https://github.com/projectfluent/fluent-rs"
[[bench]]
name = "parser"
harness = false
[dev-dependencies.assert-json-diff]
version = "1.0"
[dev-dependencies.criterion]
version = "0.3"
[dev-dependencies.glob]
version = "0.3"
[dev-dependencies.serde]
version = "1.0"
features = ["derive"]
[dev-dependencies.serde_json]
version = "1.0"

63
third_party/rust/fluent-syntax/README.md vendored Normal file
View File

@ -0,0 +1,63 @@
# Fluent Syntax
`fluent-syntax` is a parser/serializer API for the Fluent Syntax, part of the [Project Fluent](https://projectfluent.org/), a localization
framework designed to unleash the entire expressive power of natural language translations.
[![crates.io](http://meritbadge.herokuapp.com/fluent-syntax)](https://crates.io/crates/fluent-syntax)
[![Build Status](https://travis-ci.org/projectfluent/fluent-rs.svg?branch=master)](https://travis-ci.org/projectfluent/fluent-rs)
[![Coverage Status](https://coveralls.io/repos/github/projectfluent/fluent-rs/badge.svg?branch=master)](https://coveralls.io/github/projectfluent/fluent-rs?branch=master)
Status
------
The crate currently provides just a parser, which is tracking Fluent Syntax on its way to 1.0.
Local Development
-----------------
cargo build
cargo test
cargo bench
When submitting a PR please use [`cargo fmt`][] (nightly).
[`cargo fmt`]: https://github.com/rust-lang-nursery/rustfmt
Learn the FTL syntax
--------------------
FTL is a localization file format used for describing translation resources.
FTL stands for _Fluent Translation List_.
FTL is designed to be simple to read, but at the same time allows to represent
complex concepts from natural languages like gender, plurals, conjugations, and
others.
hello-user = Hello, { $username }!
[Read the Fluent Syntax Guide][] in order to learn more about the syntax. If
you're a tool author you may be interested in the formal [EBNF grammar][].
[Read the Fluent Syntax Guide]: http://projectfluent.org/fluent/guide/
[EBNF grammar]: https://github.com/projectfluent/fluent/tree/master/spec
Get Involved
------------
`fluent-rs` is open-source, licensed under the Apache License, Version 2.0. We
encourage everyone to take a look at our code and we'll listen to your
feedback.
Discuss
-------
We'd love to hear your thoughts on Project Fluent! Whether you're a localizer
looking for a better way to express yourself in your language, or a developer
trying to make your app localizable and multilingual, or a hacker looking for
a project to contribute to, please do get in touch on discourse and the IRC channel.
- Discourse: https://discourse.mozilla.org/c/fluent
- IRC channel: [irc://irc.mozilla.org/l20n](irc://irc.mozilla.org/l20n)

318
third_party/rust/fluent-syntax/benches/menubar.ftl vendored Normal file
View File

@ -0,0 +1,318 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
## browser/locales/en-US/browser/menubar.ftl
## File menu
file-menu =
.label = File
.accesskey = F
tab-cmd =
.label = New Tab
.accesskey = T
new-user-context =
.label = New Container Tab
.accesskey = B
new-navigator-cmd =
.label = New Window
.accesskey = N
new-private-window =
.label = New Private Window
.accesskey = W
# Only displayed on OS X, and only on windows that aren't main browser windows,
# or when there are no windows but Firefox is still running.
open-location-cmd =
.label = Open Location…
open-file-cmd =
.label = Open File…
.accesskey = O
close-cmd =
.label = Close
.accesskey = C
close-window =
.label = Close Window
.accesskey = d
save-page-cmd =
.label = Save Page As…
.accesskey = A
email-page-cmd =
.label = Email Link…
.accesskey = E
print-setup-cmd =
.label = Page Setup…
.accesskey = u
print-preview-cmd =
.label = Print Preview
.accesskey = v
print-cmd =
.label = Print…
.accesskey = P
go-offline-cmd =
.label = Work Offline
.accesskey = k
quit-application-cmd =
.label = Quit
.accesskey = Q
## Edit menu
edit-menu =
.label = Edit
.accesskey = E
undo-cmd =
.label = Undo
.accesskey = U
redo-cmd =
.label = Redo
.accesskey = R
cut-cmd =
.label = Cut
.accesskey = t
copy-cmd =
.label = Copy
.accesskey = C
paste-cmd =
.label = Paste
.accesskey = P
delete-cmd =
.label = Delete
.accesskey = D
select-all-cmd =
.label = Select All
.accesskey = A
find-on-cmd =
.label = Find in This Page…
.accesskey = F
find-again-cmd =
.label = Find Again
.accesskey = g
bidi-switch-text-direction-item =
.label = Switch Text Direction
.accesskey = w
preferences-cmd-unix =
.label = Preferences
.accesskey = n
## View menu
view-menu =
.label = View
.accesskey = V
view-toolbars-menu =
.label = Toolbars
.accesskey = T
view-customize-toolbar =
.label = Customize…
.accesskey = C
view-sidebar-menu =
.label = Sidebar
.accesskey = e
bookmarks-button =
.label = Bookmarks
history-button =
.label = History
synced-tabs =
.label = Synced Tabs
full-zoom =
.label = Zoom
.accesskey = Z
full-zoom-enlarge-cmd =
.label = Zoom In
.accesskey = I
full-zoom-reduce-cmd =
.label = Zoom Out
.accesskey = O
full-zoom-reset-cmd =
.label = Reset
.accesskey = R
full-zoom-toggle-cmd =
.label = Zoom Text Only
.accesskey = T
page-style-menu =
.label = Page Style
.accesskey = y
page-style-no-style =
.label = No Style
.accesskey = n
page-style-persistent-only =
.label = Basic Page Style
.accesskey = b
charset-menu2 =
.label = Text Encoding
.accesskey = c
## Full Screen controls
## Match what Safari and other Apple applications use on OS X Lion.
#
enter-full-screen-cmd =
.accesskey = F
.label = Enter Full Screen
exit-full-screen-cmd =
.accesskey = F
.label = Exit Full Screen
full-screen-cmd =
.accesskey = F
.label = Full Screen
show-all-tabs-cmd =
.accesskey = A
.label = Show All Tabs
bidi-switch-page-direction-item =
.label = Switch Page Direction
.accesskey = D
## History menu
history-menu =
.label = History
.accesskey = s
show-all-history-cmd2 =
.label = Show All History
clear-recent-history =
.label = Clear Recent History…
sync-tabs-menu3 =
.label = Synced Tabs
history-restore-last-session =
.label = Restore Previous Session
hidden-tabs =
.label = Hidden Tabs
history-undo-menu =
.label = Recently Closed Tabs
history-undo-window-menu =
.label = Recently Closed Windows
## Bookmarks menu
bookmarks-menu =
.label = Bookmarks
.accesskey = B
show-all-bookmarks2 =
.label = Show All Bookmarks
add-cur-pages-cmd =
.label = Bookmark All Tabs…
personalbar-cmd =
.label = Bookmarks Toolbar
other-bookmarks-cmd =
.label = Other Bookmarks
mobile-bookmarks-cmd =
.label = Mobile Bookmarks
## Tools menu
tools-menu =
.label = Tools
.accesskey = T
downloads =
.label = Downloads
.accesskey = D
addons =
.label = Add-ons
.accesskey = A
sync-sign-in =
.label = Sign In To { -sync-brand-short-name }…
.accesskey = Y
sync-sync-now-item =
.label = Sync Now
.accesskey = S
sync-re-auth-item =
.label = Reconnect to { -sync-brand-short-name }…
.accesskey = R
web-developer-menu =
.label = Web Developer
.accesskey = W
page-source-cmd =
.label = Page Source
.accesskey = o
page-info-cmd =
.accesskey = I
.label = Page Info
preferences-cmd2 =
.label = Options
.accesskey = O
preferences-ldb-cmd =
.label = Layout Debugger
.accesskey = L
preferences-cmd-mac =
.label = Preferences…
services-menu-mac =
.label = Services
hide-this-app-cmd-mac2 =
.label = Hide { -brand-shorter-name }
hide-other-apps-cmd-mac =
.label = Hide Others
show-all-apps-cmd-mac =
.label = Show All
window-menu =
.label = Window
bring-all-to-front =
.label = Bring All to Front
help-menu =
.label = Help
.accesskey = H
product-help2 =
.label = { -brand-shorter-name } Help
.accesskey = H
help-show-tour2 =
.label = { -brand-shorter-name } Tour
.accesskey = o
help-keyboard-shortcuts =
.label = Keyboard Shortcuts
.accesskey = K
help-troubleshooting-info =
.accesskey = T
.label = Troubleshooting Information
help-feedback-page =
.accesskey = S
.label = Submit Feedback…
help-safe-mode =
.accesskey = R
.label = Restart with Add-ons Disabled…
.stopaccesskey = R
.stoplabel = Restart with Add-ons Enabled
report-deceptive-site-menu =
.label = Report Deceptive Site…
.accesskey = D
safeb =
.label = This isn’t a deceptive site…
.accesskey = d
about-product2 =
.accesskey = A
.label = About { -brand-shorter-name }
# browser/locales/en-US/browser/toolbar.ftl
urlbar-textbox =
.placeholder = Search or enter address
.accesskey = d
## Toolbar items
view-bookmarks-broadcaster =
.label = Bookmarks
view-bookmarks-key =
.key = b
view-bookmarks-key-win =
.key = i
view-history-broadcaster =
.label = History
view-history-key =
.key = h
view-tabs-broadcaster =
.label = Synced Tabs
# browser/branding/official/locales/en-US/brand.ftl
-brand-shorter-name = Firefox
-brand-short-name = Firefox
-brand-full-name = Mozilla Firefox
-vendor-short-name = Mozilla
trademark-info =
Firefox and the Firefox logos are trademarks of the Mozilla Foundation.
-sync-brand-short-name = Sync

70
third_party/rust/fluent-syntax/benches/parser.rs vendored Normal file
View File

@ -0,0 +1,70 @@
use criterion::criterion_group;
use criterion::criterion_main;
use criterion::Criterion;
use std::collections::HashMap;
use std::fs::File;
use std::io;
use std::io::Read;
use fluent_syntax::parser::parse;
use fluent_syntax::unicode::unescape_unicode;
fn read_file(path: &str) -> Result<String, io::Error> {
let mut f = File::open(path)?;
let mut s = String::new();
f.read_to_string(&mut s)?;
Ok(s)
}
fn get_strings(tests: &[&'static str]) -> HashMap<&'static str, String> {
let mut ftl_strings = HashMap::new();
for test in tests {
let path = format!("./benches/{}.ftl", test);
ftl_strings.insert(*test, read_file(&path).expect("Couldn't load file"));
}
return ftl_strings;
}
fn parser_bench(c: &mut Criterion) {
let tests = &["simple", "preferences", "menubar"];
let ftl_strings = get_strings(tests);
c.bench_function_over_inputs(
"parse",
move |b, &&name| {
let source = &ftl_strings[name];
b.iter(|| parse(source).expect("Parsing of the FTL failed."))
},
tests,
);
}
fn unicode_unescape_bench(c: &mut Criterion) {
let strings = &[
"foo",
"This is an example value",
"Hello \\u00e3\\u00e9 World",
"\\u004c\\u006f\\u0072\\u0065\\u006d \\u0069\\u0070\\u0073\\u0075\\u006d \\u0064\\u006f\\u006c\\u006f\\u0072 \\u0073\\u0069\\u0074 \\u0061\\u006d\\u0065\\u0074",
"Let me introduce \\\"The\\\" Fluent",
"And here's an example of \\\\ a character to be escaped",
"But this message is completely unescape free",
"And so is this one",
"Maybe this one is as well completely escape free",
"Welcome to Mozilla Firefox",
"\\u0054\\u0068\\u0065\\u0073\\u0065 \\u0073\\u0065\\u0074\\u0074\\u0069\\u006e\\u0067\\u0073 \\u0061\\u0072\\u0065 \\u0074\\u0061\\u0069\\u006c\\u006f\\u0072\\u0065\\u0064 \\u0074\\u006f \\u0079\\u006f\\u0075\\u0072 \\u0063\\u006f\\u006d\\u0070\\u0075\\u0074\\u0065\\u0072\\u2019\\u0073 \\u0068\\u0061\\u0072\\u0064\\u0077\\u0061\\u0072\\u0065 \\u0061\\u006e\\u0064 \\u006f\\u0070\\u0065\\u0072\\u0061\\u0074\\u0069\\u006e\\u0067 \\u0073\\u0079\\u0073\\u0074\\u0065\\u006d\\u002e",
"These settings are tailored to your computer’s hardware and operating system",
"Use recommended performance settings",
"\\u0041\\u0064\\u0064\\u0069\\u0074\\u0069\\u006f\\u006e\\u0061\\u006c \\u0063\\u006f\\u006e\\u0074\\u0065\\u006e\\u0074 \\u0070\\u0072\\u006f\\u0063\\u0065\\u0073\\u0073\\u0065\\u0073 \\u0063\\u0061\\u006e \\u0069\\u006d\\u0070\\u0072\\u006f\\u0076\\u0065 \\u0070\\u0065\\u0072\\u0066\\u006f\\u0072\\u006d\\u0061\\u006e\\u0063\\u0065 \\u0077\\u0068\\u0065\\u006e \\u0075\\u0073\\u0069\\u006e\\u0067 \\u006d\\u0075\\u006c\\u0074\\u0069\\u0070\\u006c\\u0065 \\u0074\\u0061\\u0062\\u0073\\u002c \\u0062\\u0075\\u0074 \\u0077\\u0069\\u006c\\u006c \\u0061\\u006c\\u0073\\u006f \\u0075\\u0073\\u0065 \\u006d\\u006f\\u0072\\u0065 \\u006d\\u0065\\u006d\\u006f\\u0072\\u0079\\u002e",
"Additional content processes can improve performance when using multiple tabs, but will also use more memory.",
];
c.bench_function("unicode", move |b| {
b.iter(|| {
for s in strings {
unescape_unicode(s);
}
})
});
}
criterion_group!(benches, parser_bench, unicode_unescape_bench);
criterion_main!(benches);

1077
third_party/rust/fluent-syntax/benches/preferences.ftl vendored Normal file
View File

File diff suppressed because it is too large Load Diff

102
third_party/rust/fluent-syntax/benches/simple.ftl vendored Normal file
View File

@ -0,0 +1,102 @@
# Artificial testcase with 100 simple Fluent Messages
key0 = Value 0
key1 = Value 1
key2 = Value 2
key3 = Value 3
key4 = Value 4
key5 = Value 5
key6 = Value 6
key7 = Value 7
key8 = Value 8
key9 = Value 9
key10 = Value 10
key11 = Value 11
key12 = Value 12
key13 = Value 13
key14 = Value 14
key15 = Value 15
key16 = Value 16
key17 = Value 17
key18 = Value 18
key19 = Value 19
key20 = Value 20
key21 = Value 21
key22 = Value 22
key23 = Value 23
key24 = Value 24
key25 = Value 25
key26 = Value 26
key27 = Value 27
key28 = Value 28
key29 = Value 29
key30 = Value 30
key31 = Value 31
key32 = Value 32
key33 = Value 33
key34 = Value 34
key35 = Value 35
key36 = Value 36
key37 = Value 37
key38 = Value 38
key39 = Value 39
key40 = Value 40
key41 = Value 41
key42 = Value 42
key43 = Value 43
key44 = Value 44
key45 = Value 45
key46 = Value 46
key47 = Value 47
key48 = Value 48
key49 = Value 49
key50 = Value 50
key51 = Value 51
key52 = Value 52
key53 = Value 53
key54 = Value 54
key55 = Value 55
key56 = Value 56
key57 = Value 57
key58 = Value 58
key59 = Value 59
key60 = Value 60
key61 = Value 61
key62 = Value 62
key63 = Value 63
key64 = Value 64
key65 = Value 65
key66 = Value 66
key67 = Value 67
key68 = Value 68
key69 = Value 69
key70 = Value 70
key71 = Value 71
key72 = Value 72
key73 = Value 73
key74 = Value 74
key75 = Value 75
key76 = Value 76
key77 = Value 77
key78 = Value 78
key79 = Value 79
key80 = Value 80
key81 = Value 81
key82 = Value 82
key83 = Value 83
key84 = Value 84
key85 = Value 85
key86 = Value 86
key87 = Value 87
key88 = Value 88
key89 = Value 89
key90 = Value 90
key91 = Value 91
key92 = Value 92
key93 = Value 93
key94 = Value 94
key95 = Value 95
key96 = Value 96
key97 = Value 97
key98 = Value 98
key99 = Value 99

125
third_party/rust/fluent-syntax/src/ast.rs vendored Normal file
View File

@ -0,0 +1,125 @@
#[derive(Debug, PartialEq)]
pub struct Resource<'ast> {
pub body: Vec<ResourceEntry<'ast>>,
}
#[derive(Debug, PartialEq)]
pub enum ResourceEntry<'ast> {
Entry(Entry<'ast>),
Junk(&'ast str),
}
#[derive(Debug, PartialEq)]
pub enum Entry<'ast> {
Message(Message<'ast>),
Term(Term<'ast>),
Comment(Comment<'ast>),
}
#[derive(Debug, PartialEq)]
pub struct Message<'ast> {
pub id: Identifier<'ast>,
pub value: Option<Pattern<'ast>>,
pub attributes: Vec<Attribute<'ast>>,
pub comment: Option<Comment<'ast>>,
}
#[derive(Debug, PartialEq)]
pub struct Term<'ast> {
pub id: Identifier<'ast>,
pub value: Pattern<'ast>,
pub attributes: Vec<Attribute<'ast>>,
pub comment: Option<Comment<'ast>>,
}
#[derive(Debug, PartialEq)]
pub struct Pattern<'ast> {
pub elements: Vec<PatternElement<'ast>>,
}
#[derive(Debug, PartialEq)]
pub enum PatternElement<'ast> {
TextElement(&'ast str),
Placeable(Expression<'ast>),
}
#[derive(Debug, PartialEq)]
pub struct Attribute<'ast> {
pub id: Identifier<'ast>,
pub value: Pattern<'ast>,
}
#[derive(Debug, PartialEq)]
pub struct Identifier<'ast> {
pub name: &'ast str,
}
#[derive(Debug, PartialEq)]
pub struct Variant<'ast> {
pub key: VariantKey<'ast>,
pub value: Pattern<'ast>,
pub default: bool,
}
#[derive(Debug, PartialEq)]
pub enum VariantKey<'ast> {
Identifier { name: &'ast str },
NumberLiteral { value: &'ast str },
}
#[derive(Debug, PartialEq)]
pub enum Comment<'ast> {
Comment { content: Vec<&'ast str> },
GroupComment { content: Vec<&'ast str> },
ResourceComment { content: Vec<&'ast str> },
}
#[derive(Debug, PartialEq)]
pub enum InlineExpression<'ast> {
StringLiteral {
value: &'ast str,
},
NumberLiteral {
value: &'ast str,
},
FunctionReference {
id: Identifier<'ast>,
arguments: Option<CallArguments<'ast>>,
},
MessageReference {
id: Identifier<'ast>,
attribute: Option<Identifier<'ast>>,
},
TermReference {
id: Identifier<'ast>,
attribute: Option<Identifier<'ast>>,
arguments: Option<CallArguments<'ast>>,
},
VariableReference {
id: Identifier<'ast>,
},
Placeable {
expression: Box<Expression<'ast>>,
},
}
#[derive(Debug, PartialEq)]
pub struct CallArguments<'ast> {
pub positional: Vec<InlineExpression<'ast>>,
pub named: Vec<NamedArgument<'ast>>,
}
#[derive(Debug, PartialEq)]
pub struct NamedArgument<'ast> {
pub name: Identifier<'ast>,
pub value: InlineExpression<'ast>,
}
#[derive(Debug, PartialEq)]
pub enum Expression<'ast> {
InlineExpression(InlineExpression<'ast>),
SelectExpression {
selector: InlineExpression<'ast>,
variants: Vec<Variant<'ast>>,
},
}

3
third_party/rust/fluent-syntax/src/lib.rs vendored Normal file
View File

@ -0,0 +1,3 @@
pub mod ast;
pub mod parser;
pub mod unicode;

55
third_party/rust/fluent-syntax/src/parser/errors.rs vendored Normal file
View File

@ -0,0 +1,55 @@
#[derive(Debug, PartialEq)]
pub struct ParserError {
pub pos: (usize, usize),
pub slice: Option<(usize, usize)>,
pub kind: ErrorKind,
}
macro_rules! error {
($kind:expr, $start:expr) => {{
Err(ParserError {
pos: ($start, $start + 1),
slice: None,
kind: $kind,
})
}};
($kind:expr, $start:expr, $end:expr) => {{
Err(ParserError {
pos: ($start, $end),
slice: None,
kind: $kind,
})
}};
}
#[derive(Debug, PartialEq)]
pub enum ErrorKind {
Generic,
ExpectedEntry,
ExpectedToken(char),
ExpectedCharRange { range: String },
ExpectedMessageField { entry_id: String },
ExpectedTermField { entry_id: String },
ForbiddenWhitespace,
ForbiddenCallee,
ForbiddenKey,
MissingDefaultVariant,
MissingVariants,
MissingValue,
MissingVariantKey,
MissingLiteral,
MultipleDefaultVariants,
MessageReferenceAsSelector,
TermReferenceAsSelector,
MessageAttributeAsSelector,
TermAttributeAsPlaceable,
UnterminatedStringExpression,
PositionalArgumentFollowsNamed,
DuplicatedNamedArgument(String),
ForbiddenVariantAccessor,
UnknownEscapeSequence(String),
InvalidUnicodeEscapeSequence(String),
UnbalancedClosingBrace,
ExpectedInlineExpression,
ExpectedSimpleExpressionAsSelector,
}

173
third_party/rust/fluent-syntax/src/parser/ftlstream.rs vendored Normal file
View File

@ -0,0 +1,173 @@
use super::errors::{ErrorKind, ParserError};
use super::Result;
use std::str;
pub struct ParserStream<'p> {
pub source: &'p [u8],
pub length: usize,
pub ptr: usize,
}
impl<'p> ParserStream<'p> {
pub fn new(stream: &'p str) -> Self {
ParserStream {
source: stream.as_bytes(),
length: stream.len(),
ptr: 0,
}
}
pub fn is_current_byte(&self, b: u8) -> bool {
self.source.get(self.ptr) == Some(&b)
}
pub fn is_byte_at(&self, b: u8, pos: usize) -> bool {
self.source.get(pos) == Some(&b)
}
pub fn expect_byte(&mut self, b: u8) -> Result<()> {
if !self.is_current_byte(b) {
return error!(ErrorKind::ExpectedToken(b as char), self.ptr);
}
self.ptr += 1;
Ok(())
}
pub fn take_byte_if(&mut self, b: u8) -> bool {
if self.is_current_byte(b) {
self.ptr += 1;
true
} else {
false
}
}
pub fn skip_blank_block(&mut self) -> usize {
let mut count = 0;
loop {
let start = self.ptr;
self.skip_blank_inline();
if !self.skip_eol() {
self.ptr = start;
break;
}
count += 1;
}
count
}
pub fn skip_blank(&mut self) {
loop {
match self.source.get(self.ptr) {
Some(b) if [b' ', b'\n'].contains(b) => self.ptr += 1,
_ => break,
}
}
}
pub fn skip_blank_inline(&mut self) -> usize {
let start = self.ptr;
while let Some(b' ') = self.source.get(self.ptr) {
self.ptr += 1;
}
self.ptr - start
}
pub fn skip_to_next_entry_start(&mut self) {
while let Some(b) = self.source.get(self.ptr) {
let new_line = self.ptr == 0 || self.source.get(self.ptr - 1) == Some(&b'\n');
if new_line && (b.is_ascii_alphabetic() || [b'-', b'#'].contains(b)) {
break;
}
self.ptr += 1;
}
}
pub fn skip_eol(&mut self) -> bool {
match self.source.get(self.ptr) {
Some(b'\n') => {
self.ptr += 1;
true
}
Some(b'\r') if self.is_byte_at(b'\n', self.ptr + 1) => {
self.ptr += 2;
true
}
_ => false,
}
}
pub fn skip_unicode_escape_sequence(&mut self, length: usize) -> Result<()> {
let start = self.ptr;
for _ in 0..length {
match self.source.get(self.ptr) {
Some(b) if b.is_ascii_hexdigit() => self.ptr += 1,
_ => break,
}
}
if self.ptr - start != length {
let end = if self.ptr >= self.length {
self.ptr
} else {
self.ptr + 1
};
return error!(
ErrorKind::InvalidUnicodeEscapeSequence(self.get_slice(start, end).to_owned()),
self.ptr
);
}
Ok(())
}
pub fn is_identifier_start(&self) -> bool {
match self.source.get(self.ptr) {
Some(b) if b.is_ascii_alphabetic() => true,
_ => false,
}
}
pub fn is_byte_pattern_continuation(&self, b: u8) -> bool {
![b'}', b'.', b'[', b'*'].contains(&b)
}
pub fn is_number_start(&self) -> bool {
match self.source.get(self.ptr) {
Some(b) if (b == &b'-') || b.is_ascii_digit() => true,
_ => false,
}
}
pub fn is_eol(&self) -> bool {
match self.source.get(self.ptr) {
Some(b'\n') => true,
Some(b'\r') if self.is_byte_at(b'\n', self.ptr + 1) => true,
_ => false,
}
}
pub fn get_slice(&self, start: usize, end: usize) -> &'p str {
str::from_utf8(&self.source[start..end]).expect("Slicing the source failed")
}
pub fn skip_digits(&mut self) -> Result<()> {
let start = self.ptr;
loop {
match self.source.get(self.ptr) {
Some(b) if b.is_ascii_digit() => self.ptr += 1,
_ => break,
}
}
if start == self.ptr {
error!(
ErrorKind::ExpectedCharRange {
range: "0-9".to_string()
},
self.ptr
)
} else {
Ok(())
}
}
}

776
third_party/rust/fluent-syntax/src/parser/mod.rs vendored Normal file
View File

@ -0,0 +1,776 @@
#[macro_use]
pub mod errors;
mod ftlstream;
use std::cmp;
use std::result;
use std::str;
use self::errors::ErrorKind;
pub use self::errors::ParserError;
use self::ftlstream::ParserStream;
use super::ast;
pub type Result<T> = result::Result<T, ParserError>;
pub fn parse(source: &str) -> result::Result<ast::Resource, (ast::Resource, Vec<ParserError>)> {
let mut errors = vec![];
let mut ps = ParserStream::new(source);
let mut body = vec![];
ps.skip_blank_block();
let mut last_comment = None;
let mut last_blank_count = 0;
while ps.ptr < ps.length {
let entry_start = ps.ptr;
let mut entry = get_entry(&mut ps, entry_start);
if let Some(comment) = last_comment.take() {
match entry {
Ok(ast::Entry::Message(ref mut msg)) if last_blank_count < 2 => {
msg.comment = Some(comment);
}
Ok(ast::Entry::Term(ref mut term)) if last_blank_count < 2 => {
term.comment = Some(comment);
}
_ => {
body.push(ast::ResourceEntry::Entry(ast::Entry::Comment(comment)));
}
}
}
match entry {
Ok(ast::Entry::Comment(comment @ ast::Comment::Comment { .. })) => {
last_comment = Some(comment);
}
Ok(entry) => {
body.push(ast::ResourceEntry::Entry(entry));
}
Err(mut err) => {
ps.skip_to_next_entry_start();
err.slice = Some((entry_start, ps.ptr));
errors.push(err);
let slice = ps.get_slice(entry_start, ps.ptr);
body.push(ast::ResourceEntry::Junk(slice));
}
}
last_blank_count = ps.skip_blank_block();
}
if let Some(last_comment) = last_comment.take() {
body.push(ast::ResourceEntry::Entry(ast::Entry::Comment(last_comment)));
}
if errors.is_empty() {
Ok(ast::Resource { body })
} else {
Err((ast::Resource { body }, errors))
}
}
fn get_entry<'p>(ps: &mut ParserStream<'p>, entry_start: usize) -> Result<ast::Entry<'p>> {
let entry = match ps.source[ps.ptr] {
b'#' => ast::Entry::Comment(get_comment(ps)?),
b'-' => ast::Entry::Term(get_term(ps, entry_start)?),
_ => ast::Entry::Message(get_message(ps, entry_start)?),
};
Ok(entry)
}
fn get_message<'p>(ps: &mut ParserStream<'p>, entry_start: usize) -> Result<ast::Message<'p>> {
let id = get_identifier(ps)?;
ps.skip_blank_inline();
ps.expect_byte(b'=')?;
let pattern = get_pattern(ps)?;
ps.skip_blank_block();
let attributes = get_attributes(ps);
if pattern.is_none() && attributes.is_empty() {
return error!(
ErrorKind::ExpectedMessageField {
entry_id: id.name.to_string()
},
entry_start, ps.ptr
);
}
Ok(ast::Message {
id,
value: pattern,
attributes,
comment: None,
})
}
fn get_term<'p>(ps: &mut ParserStream<'p>, entry_start: usize) -> Result<ast::Term<'p>> {
ps.expect_byte(b'-')?;
let id = get_identifier(ps)?;
ps.skip_blank_inline();
ps.expect_byte(b'=')?;
ps.skip_blank_inline();
let value = get_pattern(ps)?;
ps.skip_blank_block();
let attributes = get_attributes(ps);
if let Some(value) = value {
Ok(ast::Term {
id,
value,
attributes,
comment: None,
})
} else {
error!(
ErrorKind::ExpectedTermField {
entry_id: id.name.to_string()
},
entry_start, ps.ptr
)
}
}
fn get_attributes<'p>(ps: &mut ParserStream<'p>) -> Vec<ast::Attribute<'p>> {
let mut attributes = vec![];
loop {
let line_start = ps.ptr;
ps.skip_blank_inline();
if !ps.is_current_byte(b'.') {
ps.ptr = line_start;
break;
}
match get_attribute(ps) {
Ok(attr) => attributes.push(attr),
Err(_) => {
ps.ptr = line_start;
break;
}
}
}
attributes
}
fn get_attribute<'p>(ps: &mut ParserStream<'p>) -> Result<ast::Attribute<'p>> {
ps.expect_byte(b'.')?;
let id = get_identifier(ps)?;
ps.skip_blank_inline();
ps.expect_byte(b'=')?;
let pattern = get_pattern(ps)?;
match pattern {
Some(pattern) => Ok(ast::Attribute { id, value: pattern }),
None => error!(ErrorKind::MissingValue, ps.ptr),
}
}
fn get_identifier<'p>(ps: &mut ParserStream<'p>) -> Result<ast::Identifier<'p>> {
let mut ptr = ps.ptr;
match ps.source.get(ptr) {
Some(b) if b.is_ascii_alphabetic() => {
ptr += 1;
}
_ => {
return error!(
ErrorKind::ExpectedCharRange {
range: "a-zA-Z".to_string()
},
ptr
);
}
}
while let Some(b) = ps.source.get(ptr) {
if b.is_ascii_alphabetic() || b.is_ascii_digit() || [b'_', b'-'].contains(b) {
ptr += 1;
} else {
break;
}
}
let name = ps.get_slice(ps.ptr, ptr);
ps.ptr = ptr;
Ok(ast::Identifier { name })
}
fn get_attribute_accessor<'p>(ps: &mut ParserStream<'p>) -> Result<Option<ast::Identifier<'p>>> {
if !ps.take_byte_if(b'.') {
Ok(None)
} else {
let ident = get_identifier(ps)?;
Ok(Some(ident))
}
}
fn get_variant_key<'p>(ps: &mut ParserStream<'p>) -> Result<ast::VariantKey<'p>> {
if !ps.take_byte_if(b'[') {
return error!(ErrorKind::ExpectedToken('['), ps.ptr);
}
ps.skip_blank();
let key = if ps.is_number_start() {
ast::VariantKey::NumberLiteral {
value: get_number_literal(ps)?,
}
} else {
ast::VariantKey::Identifier {
name: get_identifier(ps)?.name,
}
};
ps.skip_blank();
ps.expect_byte(b']')?;
Ok(key)
}
fn get_variants<'p>(ps: &mut ParserStream<'p>) -> Result<Vec<ast::Variant<'p>>> {
let mut variants = vec![];
let mut has_default = false;
while ps.is_current_byte(b'*') || ps.is_current_byte(b'[') {
let default = ps.take_byte_if(b'*');
if default {
if has_default {
return error!(ErrorKind::MultipleDefaultVariants, ps.ptr);
} else {
has_default = true;
}
}
let key = get_variant_key(ps)?;
let value = get_pattern(ps)?;
if let Some(value) = value {
variants.push(ast::Variant {
key,
value,
default,
});
ps.skip_blank();
} else {
return error!(ErrorKind::MissingValue, ps.ptr);
}
}
if !has_default {
error!(ErrorKind::MissingDefaultVariant, ps.ptr)
} else {
Ok(variants)
}
}
// This enum tracks the reason for which a text slice ended.
// It is used by the pattern to set the proper state for the next line.
//
// CRLF variant is specific because we want to skip the CR but keep the LF in text elements
// production.
// For example `a\r\n b` will produce (`a`, `\n` and ` b`) TextElements.
#[derive(Debug, PartialEq)]
enum TextElementTermination {
LineFeed,
CRLF,
PlaceableStart,
EOF,
}
// This enum tracks the placement of the text element in the pattern, which is needed for
// dedentation logic.
#[derive(Debug, PartialEq)]
enum TextElementPosition {
InitialLineStart,
LineStart,
Continuation,
}
// This enum allows us to mark pointers in the source which will later become text elements
// but without slicing them out of the source string. This makes the indentation adjustments
// cheaper since they'll happen on the pointers, rather than extracted slices.
#[derive(Debug)]
enum PatternElementPlaceholders<'a> {
Placeable(ast::Expression<'a>),
// (start, end, indent, position)
TextElement(usize, usize, usize, TextElementPosition),
}
// This enum tracks whether the text element is blank or not.
// This is important to identify text elements which should not be taken into account
// when calculating common indent.
#[derive(Debug, PartialEq)]
enum TextElementType {
Blank,
NonBlank,
}
fn get_pattern<'p>(ps: &mut ParserStream<'p>) -> Result<Option<ast::Pattern<'p>>> {
let mut elements = vec![];
let mut last_non_blank = None;
let mut common_indent = None;
ps.skip_blank_inline();
let mut text_element_role = if ps.skip_eol() {
ps.skip_blank_block();
TextElementPosition::LineStart
} else {
TextElementPosition::InitialLineStart
};
while ps.ptr < ps.length {
if ps.is_current_byte(b'{') {
if text_element_role == TextElementPosition::LineStart {
common_indent = Some(0);
}
let exp = get_placeable(ps)?;
last_non_blank = Some(elements.len());
elements.push(PatternElementPlaceholders::Placeable(exp));
text_element_role = TextElementPosition::Continuation;
} else {
let slice_start = ps.ptr;
let mut indent = 0;
if text_element_role == TextElementPosition::LineStart {
indent = ps.skip_blank_inline();
if ps.ptr >= ps.length {
break;
}
let b = ps.source[ps.ptr];
if indent == 0 {
if b != b'\n' {
break;
}
} else if !ps.is_byte_pattern_continuation(b) {
ps.ptr = slice_start;
break;
}
}
let (start, end, text_element_type, termination_reason) = get_text_slice(ps)?;
if start != end {
if text_element_role == TextElementPosition::LineStart
&& text_element_type == TextElementType::NonBlank
{
if let Some(common) = common_indent {
if indent < common {
common_indent = Some(indent);
}
} else {
common_indent = Some(indent);
}
}
if text_element_role != TextElementPosition::LineStart
|| text_element_type == TextElementType::NonBlank
|| termination_reason == TextElementTermination::LineFeed
{
if text_element_type == TextElementType::NonBlank {
last_non_blank = Some(elements.len());
}
elements.push(PatternElementPlaceholders::TextElement(
slice_start,
end,
indent,
text_element_role,
));
}
}
text_element_role = match termination_reason {
TextElementTermination::LineFeed => TextElementPosition::LineStart,
TextElementTermination::CRLF => TextElementPosition::Continuation,
TextElementTermination::PlaceableStart => TextElementPosition::Continuation,
TextElementTermination::EOF => TextElementPosition::Continuation,
};
}
}
if let Some(last_non_blank) = last_non_blank {
let elements = elements
.into_iter()
.take(last_non_blank + 1)
.enumerate()
.map(|(i, elem)| match elem {
PatternElementPlaceholders::Placeable(exp) => ast::PatternElement::Placeable(exp),
PatternElementPlaceholders::TextElement(start, end, indent, role) => {
let start = if role == TextElementPosition::LineStart {
if let Some(common_indent) = common_indent {
start + cmp::min(indent, common_indent)
} else {
start + indent
}
} else {
start
};
let slice = ps.get_slice(start, end);
if last_non_blank == i {
ast::PatternElement::TextElement(slice.trim_end())
} else {
ast::PatternElement::TextElement(slice)
}
}
})
.collect();
return Ok(Some(ast::Pattern { elements }));
}
Ok(None)
}
fn get_text_slice<'p>(
ps: &mut ParserStream<'p>,
) -> Result<(usize, usize, TextElementType, TextElementTermination)> {
let start_pos = ps.ptr;
let mut text_element_type = TextElementType::Blank;
while ps.ptr < ps.length {
match ps.source[ps.ptr] {
b' ' => ps.ptr += 1,
b'\n' => {
ps.ptr += 1;
return Ok((
start_pos,
ps.ptr,
text_element_type,
TextElementTermination::LineFeed,
));
}
b'\r' if ps.is_byte_at(b'\n', ps.ptr + 1) => {
ps.ptr += 1;
return Ok((
start_pos,
ps.ptr - 1,
text_element_type,
TextElementTermination::CRLF,
));
}
b'{' => {
return Ok((
start_pos,
ps.ptr,
text_element_type,
TextElementTermination::PlaceableStart,
));
}
b'}' => {
return error!(ErrorKind::UnbalancedClosingBrace, ps.ptr);
}
_ => {
text_element_type = TextElementType::NonBlank;
ps.ptr += 1
}
}
}
Ok((
start_pos,
ps.ptr,
text_element_type,
TextElementTermination::EOF,
))
}
fn get_comment<'p>(ps: &mut ParserStream<'p>) -> Result<ast::Comment<'p>> {
let mut level = None;
let mut content = vec![];
while ps.ptr < ps.length {
let line_level = get_comment_level(ps);
if line_level == 0 {
ps.ptr -= 1;
break;
}
if level.is_some() && Some(line_level) != level {
ps.ptr -= line_level;
break;
}
level = Some(line_level);
if ps.is_current_byte(b'\n') {
content.push(get_comment_line(ps)?);
} else {
if let Err(e) = ps.expect_byte(b' ') {
if content.is_empty() {
return Err(e);
} else {
ps.ptr -= line_level;
break;
}
}
content.push(get_comment_line(ps)?);
}
ps.skip_eol();
}
let comment = match level {
Some(3) => ast::Comment::ResourceComment { content },
Some(2) => ast::Comment::GroupComment { content },
_ => ast::Comment::Comment { content },
};
Ok(comment)
}
fn get_comment_level<'p>(ps: &mut ParserStream<'p>) -> usize {
let mut chars = 0;
while ps.take_byte_if(b'#') {
chars += 1;
}
chars
}
fn get_comment_line<'p>(ps: &mut ParserStream<'p>) -> Result<&'p str> {
let start_pos = ps.ptr;
while ps.ptr < ps.length && !ps.is_eol() {
ps.ptr += 1;
}
Ok(ps.get_slice(start_pos, ps.ptr))
}
fn get_placeable<'p>(ps: &mut ParserStream<'p>) -> Result<ast::Expression<'p>> {
ps.expect_byte(b'{')?;
ps.skip_blank();
let exp = get_expression(ps)?;
ps.skip_blank_inline();
ps.expect_byte(b'}')?;
let invalid_expression_found = match &exp {
ast::Expression::InlineExpression(ast::InlineExpression::TermReference {
ref attribute,
..
}) => attribute.is_some(),
_ => false,
};
if invalid_expression_found {
return error!(ErrorKind::TermAttributeAsPlaceable, ps.ptr);
}
Ok(exp)
}
fn get_expression<'p>(ps: &mut ParserStream<'p>) -> Result<ast::Expression<'p>> {
let exp = get_inline_expression(ps)?;
ps.skip_blank();
if !ps.is_current_byte(b'-') || !ps.is_byte_at(b'>', ps.ptr + 1) {
if let ast::InlineExpression::TermReference { ref attribute, .. } = exp {
if attribute.is_some() {
return error!(ErrorKind::TermAttributeAsPlaceable, ps.ptr);
}
}
return Ok(ast::Expression::InlineExpression(exp));
}
match exp {
ast::InlineExpression::MessageReference { ref attribute, .. } => {
if attribute.is_none() {
return error!(ErrorKind::MessageReferenceAsSelector, ps.ptr);
} else {
return error!(ErrorKind::MessageAttributeAsSelector, ps.ptr);
}
}
ast::InlineExpression::TermReference { ref attribute, .. } => {
if attribute.is_none() {
return error!(ErrorKind::TermReferenceAsSelector, ps.ptr);
}
}
ast::InlineExpression::StringLiteral { .. }
| ast::InlineExpression::NumberLiteral { .. }
| ast::InlineExpression::VariableReference { .. }
| ast::InlineExpression::FunctionReference { .. } => {}
_ => {
return error!(ErrorKind::ExpectedSimpleExpressionAsSelector, ps.ptr);
}
};
ps.ptr += 2; // ->
ps.skip_blank_inline();
ps.expect_byte(b'\n')?;
ps.skip_blank();
let variants = get_variants(ps)?;
Ok(ast::Expression::SelectExpression {
selector: exp,
variants,
})
}
fn get_inline_expression<'p>(ps: &mut ParserStream<'p>) -> Result<ast::InlineExpression<'p>> {
match ps.source.get(ps.ptr) {
Some(b'"') => {
ps.ptr += 1; // "
let start = ps.ptr;
while ps.ptr < ps.length {
match ps.source[ps.ptr] {
b'\\' => match ps.source.get(ps.ptr + 1) {
Some(b'\\') => ps.ptr += 2,
Some(b'{') => ps.ptr += 2,
Some(b'"') => ps.ptr += 2,
Some(b'u') => {
ps.ptr += 2;
ps.skip_unicode_escape_sequence(4)?;
}
Some(b'U') => {
ps.ptr += 2;
ps.skip_unicode_escape_sequence(6)?;
}
_ => return error!(ErrorKind::Generic, ps.ptr),
},
b'"' => {
break;
}
b'\n' => {
return error!(ErrorKind::Generic, ps.ptr);
}
_ => ps.ptr += 1,
}
}
ps.expect_byte(b'"')?;
let slice = ps.get_slice(start, ps.ptr - 1);
Ok(ast::InlineExpression::StringLiteral { value: slice })
}
Some(b) if b.is_ascii_digit() => {
let num = get_number_literal(ps)?;
Ok(ast::InlineExpression::NumberLiteral { value: num })
}
Some(b'-') => {
ps.ptr += 1; // -
if ps.is_identifier_start() {
let id = get_identifier(ps)?;
let attribute = get_attribute_accessor(ps)?;
let arguments = get_call_arguments(ps)?;
Ok(ast::InlineExpression::TermReference {
id,
attribute,
arguments,
})
} else {
ps.ptr -= 1;
let num = get_number_literal(ps)?;
Ok(ast::InlineExpression::NumberLiteral { value: num })
}
}
Some(b'$') => {
ps.ptr += 1; // -
let id = get_identifier(ps)?;
Ok(ast::InlineExpression::VariableReference { id })
}
Some(b) if b.is_ascii_alphabetic() => {
let id = get_identifier(ps)?;
let arguments = get_call_arguments(ps)?;
if arguments.is_some() {
if !id
.name
.bytes()
.all(|c| c.is_ascii_uppercase() || c.is_ascii_digit() || c == b'_' || c == b'-')
{
return error!(ErrorKind::ForbiddenCallee, ps.ptr);
}
Ok(ast::InlineExpression::FunctionReference { id, arguments })
} else {
let attribute = get_attribute_accessor(ps)?;
Ok(ast::InlineExpression::MessageReference { id, attribute })
}
}
Some(b'{') => {
let exp = get_placeable(ps)?;
Ok(ast::InlineExpression::Placeable {
expression: Box::new(exp),
})
}
_ => error!(ErrorKind::ExpectedInlineExpression, ps.ptr),
}
}
fn get_call_arguments<'p>(ps: &mut ParserStream<'p>) -> Result<Option<ast::CallArguments<'p>>> {
ps.skip_blank();
if !ps.take_byte_if(b'(') {
return Ok(None);
}
let mut positional = vec![];
let mut named = vec![];
let mut argument_names = vec![];
ps.skip_blank();
while ps.ptr < ps.length {
if ps.is_current_byte(b')') {
break;
}
let expr = get_inline_expression(ps)?;
match expr {
ast::InlineExpression::MessageReference {
ref id,
attribute: None,
} => {
ps.skip_blank();
if ps.is_current_byte(b':') {
if argument_names.contains(&id.name.to_owned()) {
return error!(
ErrorKind::DuplicatedNamedArgument(id.name.to_owned()),
ps.ptr
);
}
ps.ptr += 1;
ps.skip_blank();
let val = get_inline_expression(ps)?;
argument_names.push(id.name.to_owned());
named.push(ast::NamedArgument {
name: ast::Identifier { name: id.name },
value: val,
});
} else {
if !argument_names.is_empty() {
return error!(ErrorKind::PositionalArgumentFollowsNamed, ps.ptr);
}
positional.push(expr);
}
}
_ => {
if !argument_names.is_empty() {
return error!(ErrorKind::PositionalArgumentFollowsNamed, ps.ptr);
}
positional.push(expr);
}
}
ps.skip_blank();
ps.take_byte_if(b',');
ps.skip_blank();
}
ps.expect_byte(b')')?;
Ok(Some(ast::CallArguments { positional, named }))
}
fn get_number_literal<'p>(ps: &mut ParserStream<'p>) -> Result<&'p str> {
let start = ps.ptr;
ps.take_byte_if(b'-');
ps.skip_digits()?;
if ps.take_byte_if(b'.') {
ps.skip_digits()?;
}
Ok(ps.get_slice(start, ps.ptr))
}

52
third_party/rust/fluent-syntax/src/unicode.rs vendored Normal file
View File

@ -0,0 +1,52 @@
use std::borrow::Cow;
use std::char;
const UNKNOWN_CHAR: char = '<27>';
fn encode_unicode(s: &str) -> char {
u32::from_str_radix(s, 16)
.ok()
.and_then(char::from_u32)
.unwrap_or(UNKNOWN_CHAR)
}
pub fn unescape_unicode(input: &str) -> Cow<str> {
let bytes = input.as_bytes();
let mut result = Cow::from(input);
let mut ptr = 0;
while let Some(b) = bytes.get(ptr) {
if b != &b'\\' {
if let Cow::Owned(ref mut s) = result {
s.push(*b as char);
}
ptr += 1;
continue;
}
if let Cow::Borrowed(_) = result {
result = Cow::from(&input[0..ptr]);
}
ptr += 1;
let new_char = match bytes.get(ptr) {
Some(b'\\') => '\\',
Some(b'"') => '"',
Some(u @ b'u') | Some(u @ b'U') => {
let start = ptr + 1;
let len = if u == &b'u' { 4 } else { 6 };
ptr += len;
input
.get(start..(start + len))
.map(|slice| encode_unicode(slice))
.unwrap_or(UNKNOWN_CHAR)
}
_ => UNKNOWN_CHAR,
};
result.to_mut().push(new_char);
ptr += 1;
}
result
}

440
third_party/rust/fluent-syntax/tests/ast/mod.rs vendored Normal file
View File

@ -0,0 +1,440 @@
use fluent_syntax::ast;
use serde::ser::SerializeMap;
use serde::ser::SerializeSeq;
use serde::{Serialize, Serializer};
use std::error::Error;
pub fn serialize<'s>(res: &'s ast::Resource) -> Result<String, Box<dyn Error>> {
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "ResourceDef")] &'ast ast::Resource<'ast>);
Ok(serde_json::to_string(&Helper(res)).unwrap())
}
pub fn _serialize_to_pretty_json<'s>(res: &'s ast::Resource) -> Result<String, Box<dyn Error>> {
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "ResourceDef")] &'ast ast::Resource<'ast>);
let buf = Vec::new();
let formatter = serde_json::ser::PrettyFormatter::with_indent(b" ");
let mut ser = serde_json::Serializer::with_formatter(buf, formatter);
Helper(res).serialize(&mut ser).unwrap();
Ok(String::from_utf8(ser.into_inner()).unwrap())
}
#[derive(Serialize)]
#[serde(remote = "ast::Resource")]
#[serde(tag = "type")]
#[serde(rename = "Resource")]
pub struct ResourceDef<'ast> {
#[serde(serialize_with = "serialize_resource_entry_vec")]
pub body: Vec<ast::ResourceEntry<'ast>>,
}
fn serialize_resource_entry_vec<'se, S>(
v: &Vec<ast::ResourceEntry<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
#[serde(tag = "type")]
enum EntryHelper<'ast> {
Junk {
annotations: Vec<&'ast str>,
content: &'ast str,
},
#[serde(with = "MessageDef")]
Message(&'ast ast::Message<'ast>),
#[serde(with = "TermDef")]
Term(&'ast ast::Term<'ast>),
Comment {
content: String,
},
GroupComment {
content: String,
},
ResourceComment {
content: String,
},
}
let mut seq = serializer.serialize_seq(Some(v.len()))?;
for e in v {
let entry = match *e {
ast::ResourceEntry::Entry(ref entry) => match entry {
ast::Entry::Message(ref msg) => EntryHelper::Message(msg),
ast::Entry::Term(ref term) => EntryHelper::Term(term),
ast::Entry::Comment(ast::Comment::Comment { ref content }) => {
EntryHelper::Comment {
content: content.join("\n"),
}
}
ast::Entry::Comment(ast::Comment::GroupComment { ref content }) => {
EntryHelper::GroupComment {
content: content.join("\n"),
}
}
ast::Entry::Comment(ast::Comment::ResourceComment { ref content }) => {
EntryHelper::ResourceComment {
content: content.join("\n"),
}
}
},
ast::ResourceEntry::Junk(ref junk) => EntryHelper::Junk {
content: junk,
annotations: vec![],
},
};
seq.serialize_element(&entry)?;
}
seq.end()
}
#[derive(Serialize)]
#[serde(remote = "ast::Message")]
pub struct MessageDef<'ast> {
#[serde(with = "IdentifierDef")]
pub id: ast::Identifier<'ast>,
#[serde(serialize_with = "serialize_pattern_option")]
pub value: Option<ast::Pattern<'ast>>,
#[serde(serialize_with = "serialize_attribute_vec")]
pub attributes: Vec<ast::Attribute<'ast>>,
#[serde(serialize_with = "serialize_comment_option")]
pub comment: Option<ast::Comment<'ast>>,
}
#[derive(Serialize)]
#[serde(remote = "ast::Identifier")]
#[serde(tag = "type")]
#[serde(rename = "Identifier")]
pub struct IdentifierDef<'ast> {
pub name: &'ast str,
}
#[derive(Serialize)]
#[serde(remote = "ast::Variant")]
#[serde(tag = "type")]
#[serde(rename = "Variant")]
pub struct VariantDef<'ast> {
#[serde(with = "VariantKeyDef")]
pub key: ast::VariantKey<'ast>,
#[serde(with = "PatternDef")]
pub value: ast::Pattern<'ast>,
pub default: bool,
}
#[derive(Serialize)]
#[serde(remote = "ast::Term")]
pub struct TermDef<'ast> {
#[serde(with = "IdentifierDef")]
pub id: ast::Identifier<'ast>,
#[serde(with = "PatternDef")]
pub value: ast::Pattern<'ast>,
#[serde(serialize_with = "serialize_attribute_vec")]
pub attributes: Vec<ast::Attribute<'ast>>,
#[serde(serialize_with = "serialize_comment_option")]
pub comment: Option<ast::Comment<'ast>>,
}
fn serialize_pattern_option<'se, S>(
v: &Option<ast::Pattern<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "PatternDef")] &'ast ast::Pattern<'ast>);
v.as_ref().map(Helper).serialize(serializer)
}
fn serialize_attribute_vec<'se, S>(
v: &Vec<ast::Attribute<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "AttributeDef")] &'ast ast::Attribute<'ast>);
let mut seq = serializer.serialize_seq(Some(v.len()))?;
for e in v {
seq.serialize_element(&Helper(e))?;
}
seq.end()
}
fn serialize_comment_option<'se, S>(
v: &Option<ast::Comment<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "CommentDef")] &'ast ast::Comment<'ast>);
v.as_ref().map(Helper).serialize(serializer)
}
#[derive(Serialize)]
#[serde(remote = "ast::Pattern")]
#[serde(tag = "type")]
#[serde(rename = "Pattern")]
pub struct PatternDef<'ast> {
#[serde(serialize_with = "serialize_pattern_elements")]
pub elements: Vec<ast::PatternElement<'ast>>,
}
fn serialize_pattern_elements<'se, S>(
v: &Vec<ast::PatternElement<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "PatternElementDef")] &'ast ast::PatternElement<'ast>);
let mut seq = serializer.serialize_seq(Some(v.len()))?;
let mut buffer = String::new();
for e in v {
match e {
ast::PatternElement::TextElement(e) => {
buffer.push_str(e);
}
_ => {
if !buffer.is_empty() {
seq.serialize_element(&Helper(&ast::PatternElement::TextElement(&buffer)))?;
buffer = String::new();
}
seq.serialize_element(&Helper(e))?;
}
}
}
if !buffer.is_empty() {
seq.serialize_element(&Helper(&ast::PatternElement::TextElement(&buffer)))?;
}
seq.end()
}
#[derive(Serialize)]
#[serde(remote = "ast::PatternElement")]
#[serde(untagged)]
pub enum PatternElementDef<'ast> {
#[serde(serialize_with = "serialize_text_element")]
TextElement(&'ast str),
#[serde(serialize_with = "serialize_placeable")]
Placeable(ast::Expression<'ast>),
}
fn serialize_text_element<'se, S>(s: &'se str, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut map = serializer.serialize_map(Some(2))?;
map.serialize_entry("type", "TextElement")?;
map.serialize_entry("value", s)?;
map.end()
}
fn serialize_placeable<'se, S>(exp: &ast::Expression<'se>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "ExpressionDef")] &'ast ast::Expression<'ast>);
let mut map = serializer.serialize_map(Some(2))?;
map.serialize_entry("type", "Placeable")?;
map.serialize_entry("expression", &Helper(exp))?;
map.end()
}
#[derive(Serialize, Debug)]
#[serde(remote = "ast::VariantKey")]
#[serde(tag = "type")]
pub enum VariantKeyDef<'ast> {
Identifier { name: &'ast str },
NumberLiteral { value: &'ast str },
}
#[derive(Serialize)]
#[serde(remote = "ast::Comment")]
#[serde(tag = "type")]
pub enum CommentDef<'ast> {
Comment {
#[serde(serialize_with = "serialize_comment_content")]
content: Vec<&'ast str>,
},
GroupComment {
#[serde(serialize_with = "serialize_comment_content")]
content: Vec<&'ast str>,
},
ResourceComment {
#[serde(serialize_with = "serialize_comment_content")]
content: Vec<&'ast str>,
},
}
fn serialize_comment_content<'se, S>(v: &Vec<&'se str>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
serializer.serialize_str(&v.join("\n"))
}
#[derive(Serialize)]
#[serde(remote = "ast::InlineExpression")]
#[serde(tag = "type")]
pub enum InlineExpressionDef<'ast> {
StringLiteral {
value: &'ast str,
},
NumberLiteral {
value: &'ast str,
},
FunctionReference {
#[serde(with = "IdentifierDef")]
id: ast::Identifier<'ast>,
#[serde(serialize_with = "serialize_call_arguments_option")]
arguments: Option<ast::CallArguments<'ast>>,
},
MessageReference {
#[serde(with = "IdentifierDef")]
id: ast::Identifier<'ast>,
#[serde(serialize_with = "serialize_identifier_option")]
attribute: Option<ast::Identifier<'ast>>,
},
TermReference {
#[serde(with = "IdentifierDef")]
id: ast::Identifier<'ast>,
#[serde(serialize_with = "serialize_identifier_option")]
attribute: Option<ast::Identifier<'ast>>,
#[serde(serialize_with = "serialize_call_arguments_option")]
arguments: Option<ast::CallArguments<'ast>>,
},
VariableReference {
#[serde(with = "IdentifierDef")]
id: ast::Identifier<'ast>,
},
Placeable {
#[serde(with = "ExpressionDef")]
expression: ast::Expression<'ast>,
},
}
fn serialize_call_arguments_option<'se, S>(
v: &Option<ast::CallArguments<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "CallArgumentsDef")] &'ast ast::CallArguments<'ast>);
v.as_ref().map(Helper).serialize(serializer)
}
fn serialize_identifier_option<'se, S>(
v: &Option<ast::Identifier<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "IdentifierDef")] &'ast ast::Identifier<'ast>);
v.as_ref().map(Helper).serialize(serializer)
}
#[derive(Serialize)]
#[serde(remote = "ast::Attribute")]
#[serde(tag = "type")]
#[serde(rename = "Attribute")]
pub struct AttributeDef<'ast> {
#[serde(with = "IdentifierDef")]
pub id: ast::Identifier<'ast>,
#[serde(with = "PatternDef")]
pub value: ast::Pattern<'ast>,
}
#[derive(Serialize)]
#[serde(remote = "ast::CallArguments")]
#[serde(tag = "type")]
#[serde(rename = "CallArguments")]
pub struct CallArgumentsDef<'ast> {
#[serde(serialize_with = "serialize_inline_expressions")]
pub positional: Vec<ast::InlineExpression<'ast>>,
#[serde(serialize_with = "serialize_named_arguments")]
pub named: Vec<ast::NamedArgument<'ast>>,
}
#[derive(Serialize)]
#[serde(remote = "ast::NamedArgument")]
#[serde(tag = "type")]
#[serde(rename = "NamedArgument")]
pub struct NamedArgumentDef<'ast> {
#[serde(with = "IdentifierDef")]
pub name: ast::Identifier<'ast>,
#[serde(with = "InlineExpressionDef")]
pub value: ast::InlineExpression<'ast>,
}
#[derive(Serialize)]
#[serde(remote = "ast::Expression")]
#[serde(tag = "type")]
pub enum ExpressionDef<'ast> {
#[serde(with = "InlineExpressionDef")]
InlineExpression(ast::InlineExpression<'ast>),
SelectExpression {
#[serde(with = "InlineExpressionDef")]
selector: ast::InlineExpression<'ast>,
#[serde(serialize_with = "serialize_variants")]
variants: Vec<ast::Variant<'ast>>,
},
}
fn serialize_variants<'se, S>(v: &Vec<ast::Variant<'se>>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "VariantDef")] &'ast ast::Variant<'ast>);
let mut seq = serializer.serialize_seq(Some(v.len()))?;
for e in v {
seq.serialize_element(&Helper(e))?;
}
seq.end()
}
fn serialize_inline_expressions<'se, S>(
v: &Vec<ast::InlineExpression<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "InlineExpressionDef")] &'ast ast::InlineExpression<'ast>);
let mut seq = serializer.serialize_seq(Some(v.len()))?;
for e in v {
seq.serialize_element(&Helper(e))?;
}
seq.end()
}
fn serialize_named_arguments<'se, S>(
v: &Vec<ast::NamedArgument<'se>>,
serializer: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
struct Helper<'ast>(#[serde(with = "NamedArgumentDef")] &'ast ast::NamedArgument<'ast>);
let mut seq = serializer.serialize_seq(Some(v.len()))?;
for e in v {
seq.serialize_element(&Helper(e))?;
}
seq.end()
}

8
third_party/rust/fluent-syntax/tests/fixtures/any_char.ftl vendored Normal file
View File

@ -0,0 +1,8 @@
# ↓ BEL, U+0007
control0 = abcdef
# ↓ DEL, U+007F
delete = abcdef
# ↓ BPM, U+0082
control1 = abc‚def

68
third_party/rust/fluent-syntax/tests/fixtures/any_char.json vendored Normal file
View File

@ -0,0 +1,68 @@
{
"type": "Resource",
"body": [
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "control0"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "abc\u0007def"
}
]
},
"attributes": [],
"comment": {
"type": "Comment",
"content": " ↓ BEL, U+0007"
}
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "delete"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "abcdef"
}
]
},
"attributes": [],
"comment": {
"type": "Comment",
"content": " ↓ DEL, U+007F"
}
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "control1"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "abc‚def"
}
]
},
"attributes": [],
"comment": {
"type": "Comment",
"content": " ↓ BPM, U+0082"
}
}
]
}

20
third_party/rust/fluent-syntax/tests/fixtures/astral.ftl vendored Normal file
View File

@ -0,0 +1,20 @@
face-with-tears-of-joy = 😂
tetragram-for-centre = 𝌆
surrogates-in-text = \uD83D\uDE02
surrogates-in-string = {"\uD83D\uDE02"}
surrogates-in-adjacent-strings = {"\uD83D"}{"\uDE02"}
emoji-in-text = A face 😂 with tears of joy.
emoji-in-string = {"A face 😂 with tears of joy."}
# ERROR Invalid identifier
err-😂 = Value
# ERROR Invalid expression
err-invalid-expression = { 😂 }
# ERROR Invalid variant key
err-invalid-variant-key = { $sel ->
*[😂] Value
}

174
third_party/rust/fluent-syntax/tests/fixtures/astral.json vendored Normal file
View File

@ -0,0 +1,174 @@
{
"type": "Resource",
"body": [
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "face-with-tears-of-joy"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "😂"
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "tetragram-for-centre"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "𝌆"
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "surrogates-in-text"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "\\uD83D\\uDE02"
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "surrogates-in-string"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "Placeable",
"expression": {
"value": "\\uD83D\\uDE02",
"type": "StringLiteral"
}
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "surrogates-in-adjacent-strings"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "Placeable",
"expression": {
"value": "\\uD83D",
"type": "StringLiteral"
}
},
{
"type": "Placeable",
"expression": {
"value": "\\uDE02",
"type": "StringLiteral"
}
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "emoji-in-text"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "A face 😂 with tears of joy."
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "emoji-in-string"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "Placeable",
"expression": {
"value": "A face 😂 with tears of joy.",
"type": "StringLiteral"
}
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Comment",
"content": "ERROR Invalid identifier"
},
{
"type": "Junk",
"annotations": [],
"content": "err-😂 = Value\n\n"
},
{
"type": "Comment",
"content": "ERROR Invalid expression"
},
{
"type": "Junk",
"annotations": [],
"content": "err-invalid-expression = { 😂 }\n\n"
},
{
"type": "Comment",
"content": "ERROR Invalid variant key"
},
{
"type": "Junk",
"annotations": [],
"content": "err-invalid-variant-key = { $sel ->\n *[😂] Value\n}\n"
}
]
}

120
third_party/rust/fluent-syntax/tests/fixtures/call_expressions.ftl vendored Normal file
View File

@ -0,0 +1,120 @@
## Function names
valid-func-name-01 = {FUN1()}
valid-func-name-02 = {FUN_FUN()}
valid-func-name-03 = {FUN-FUN()}
# JUNK 0 is not a valid Identifier start
invalid-func-name-01 = {0FUN()}
# JUNK Function names may not be lowercase
invalid-func-name-02 = {fun()}
# JUNK Function names may not contain lowercase character
invalid-func-name-03 = {Fun()}
# JUNK ? is not a valid Identifier character
invalid-func-name-04 = {FUN?()}
## Arguments
positional-args = {FUN(1, "a", msg)}
named-args = {FUN(x: 1, y: "Y")}
dense-named-args = {FUN(x:1, y:"Y")}
mixed-args = {FUN(1, "a", msg, x: 1, y: "Y")}
# ERROR Positional arg must not follow keyword args
shuffled-args = {FUN(1, x: 1, "a", y: "Y", msg)}
# ERROR Named arguments must be unique
duplicate-named-args = {FUN(x: 1, x: "X")}
## Whitespace around arguments
sparse-inline-call = {FUN ( "a" , msg, x: 1 )}
empty-inline-call = {FUN( )}
multiline-call = {FUN(
"a",
msg,
x: 1
)}
sparse-multiline-call = {FUN
(
"a" ,
msg
, x: 1
)}
empty-multiline-call = {FUN(
)}
unindented-arg-number = {FUN(
1)}
unindented-arg-string = {FUN(
"a")}
unindented-arg-msg-ref = {FUN(
msg)}
unindented-arg-term-ref = {FUN(
-msg)}
unindented-arg-var-ref = {FUN(
$var)}
unindented-arg-call = {FUN(
OTHER())}
unindented-named-arg = {FUN(
x:1)}
unindented-closing-paren = {FUN(
x
)}
## Optional trailing comma
one-argument = {FUN(1,)}
many-arguments = {FUN(1, 2, 3,)}
inline-sparse-args = {FUN( 1, 2, 3, )}
mulitline-args = {FUN(
1,
2,
)}
mulitline-sparse-args = {FUN(
1
,
2
,
)}
## Syntax errors for trailing comma
one-argument = {FUN(1,,)}
missing-arg = {FUN(,)}
missing-sparse-arg = {FUN( , )}
## Whitespace in named arguments
sparse-named-arg = {FUN(
x : 1,
y : 2,
z
:
3
)}
unindented-colon = {FUN(
x
:1)}
unindented-value = {FUN(
x:
1)}

1266
third_party/rust/fluent-syntax/tests/fixtures/call_expressions.json vendored Normal file
View File

File diff suppressed because it is too large Load Diff

46
third_party/rust/fluent-syntax/tests/fixtures/callee_expressions.ftl vendored Normal file
View File

@ -0,0 +1,46 @@
## Callees in placeables.
function-callee-placeable = {FUNCTION()}
term-callee-placeable = {-term()}
# ERROR Messages cannot be parameterized.
message-callee-placeable = {message()}
# ERROR Equivalent to a MessageReference callee.
mixed-case-callee-placeable = {Function()}
# ERROR Message attributes cannot be parameterized.
message-attr-callee-placeable = {message.attr()}
# ERROR Term attributes may not be used in Placeables.
term-attr-callee-placeable = {-term.attr()}
# ERROR Variables cannot be parameterized.
variable-callee-placeable = {$variable()}
## Callees in selectors.
function-callee-selector = {FUNCTION() ->
*[key] Value
}
term-attr-callee-selector = {-term.attr() ->
*[key] Value
}
# ERROR Messages cannot be parameterized.
message-callee-selector = {message() ->
*[key] Value
}
# ERROR Equivalent to a MessageReference callee.
mixed-case-callee-selector = {Function() ->
*[key] Value
}
# ERROR Message attributes cannot be parameterized.
message-attr-callee-selector = {message.attr() ->
*[key] Value
}
# ERROR Term values may not be used as selectors.
term-callee-selector = {-term() ->
*[key] Value
}
# ERROR Variables cannot be parameterized.
variable-callee-selector = {$variable() ->
*[key] Value
}

268
third_party/rust/fluent-syntax/tests/fixtures/callee_expressions.json vendored Normal file
View File

@ -0,0 +1,268 @@
{
"type": "Resource",
"body": [
{
"type": "GroupComment",
"content": "Callees in placeables."
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "function-callee-placeable"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "Placeable",
"expression": {
"type": "FunctionReference",
"id": {
"type": "Identifier",
"name": "FUNCTION"
},
"arguments": {
"type": "CallArguments",
"positional": [],
"named": []
}
}
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "term-callee-placeable"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "Placeable",
"expression": {
"type": "TermReference",
"id": {
"type": "Identifier",
"name": "term"
},
"attribute": null,
"arguments": {
"type": "CallArguments",
"positional": [],
"named": []
}
}
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Comment",
"content": "ERROR Messages cannot be parameterized."
},
{
"type": "Junk",
"annotations": [],
"content": "message-callee-placeable = {message()}\n"
},
{
"type": "Comment",
"content": "ERROR Equivalent to a MessageReference callee."
},
{
"type": "Junk",
"annotations": [],
"content": "mixed-case-callee-placeable = {Function()}\n"
},
{
"type": "Comment",
"content": "ERROR Message attributes cannot be parameterized."
},
{
"type": "Junk",
"annotations": [],
"content": "message-attr-callee-placeable = {message.attr()}\n"
},
{
"type": "Comment",
"content": "ERROR Term attributes may not be used in Placeables."
},
{
"type": "Junk",
"annotations": [],
"content": "term-attr-callee-placeable = {-term.attr()}\n"
},
{
"type": "Comment",
"content": "ERROR Variables cannot be parameterized."
},
{
"type": "Junk",
"annotations": [],
"content": "variable-callee-placeable = {$variable()}\n\n\n"
},
{
"type": "GroupComment",
"content": "Callees in selectors."
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "function-callee-selector"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "Placeable",
"expression": {
"type": "SelectExpression",
"selector": {
"type": "FunctionReference",
"id": {
"type": "Identifier",
"name": "FUNCTION"
},
"arguments": {
"type": "CallArguments",
"positional": [],
"named": []
}
},
"variants": [
{
"type": "Variant",
"key": {
"type": "Identifier",
"name": "key"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "Value"
}
]
},
"default": true
}
]
}
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "term-attr-callee-selector"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "Placeable",
"expression": {
"type": "SelectExpression",
"selector": {
"type": "TermReference",
"id": {
"type": "Identifier",
"name": "term"
},
"attribute": {
"type": "Identifier",
"name": "attr"
},
"arguments": {
"type": "CallArguments",
"positional": [],
"named": []
}
},
"variants": [
{
"type": "Variant",
"key": {
"type": "Identifier",
"name": "key"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "Value"
}
]
},
"default": true
}
]
}
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Comment",
"content": "ERROR Messages cannot be parameterized."
},
{
"type": "Junk",
"annotations": [],
"content": "message-callee-selector = {message() ->\n *[key] Value\n}\n"
},
{
"type": "Comment",
"content": "ERROR Equivalent to a MessageReference callee."
},
{
"type": "Junk",
"annotations": [],
"content": "mixed-case-callee-selector = {Function() ->\n *[key] Value\n}\n"
},
{
"type": "Comment",
"content": "ERROR Message attributes cannot be parameterized."
},
{
"type": "Junk",
"annotations": [],
"content": "message-attr-callee-selector = {message.attr() ->\n *[key] Value\n}\n"
},
{
"type": "Comment",
"content": "ERROR Term values may not be used as selectors."
},
{
"type": "Junk",
"annotations": [],
"content": "term-callee-selector = {-term() ->\n *[key] Value\n}\n"
},
{
"type": "Comment",
"content": "ERROR Variables cannot be parameterized."
},
{
"type": "Junk",
"annotations": [],
"content": "variable-callee-selector = {$variable() ->\n *[key] Value\n}\n"
}
]
}

20
third_party/rust/fluent-syntax/tests/fixtures/comments.ftl vendored Normal file
View File

@ -0,0 +1,20 @@
# Standalone Comment
# Message Comment
foo = Foo
# Term Comment
# with a blank last line.
#
-term = Term
# Another standalone
#
# with indent
## Group Comment
### Resource Comment
# Errors
#error
##error
###error

82
third_party/rust/fluent-syntax/tests/fixtures/comments.json vendored Normal file
View File

@ -0,0 +1,82 @@
{
"type": "Resource",
"body": [
{
"type": "Comment",
"content": "Standalone Comment"
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "foo"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "Foo"
}
]
},
"attributes": [],
"comment": {
"type": "Comment",
"content": "Message Comment"
}
},
{
"type": "Term",
"id": {
"type": "Identifier",
"name": "term"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "Term"
}
]
},
"attributes": [],
"comment": {
"type": "Comment",
"content": "Term Comment\nwith a blank last line.\n"
}
},
{
"type": "Comment",
"content": "Another standalone\n\n with indent"
},
{
"type": "GroupComment",
"content": "Group Comment"
},
{
"type": "ResourceComment",
"content": "Resource Comment"
},
{
"type": "Comment",
"content": "Errors"
},
{
"type": "Junk",
"annotations": [],
"content": "#error\n"
},
{
"type": "Junk",
"annotations": [],
"content": "##error\n"
},
{
"type": "Junk",
"annotations": [],
"content": "###error\n"
}
]
}

1
third_party/rust/fluent-syntax/tests/fixtures/cr.ftl vendored Normal file
View File

@ -0,0 +1 @@
### This entire file uses CR as EOL. + +err01 = Value 01 +err02 = Value 02 + +err03 = + + Value 03 + Continued + + .title = Title + +err04 = { "str + +err05 = { $sel -> }

9
third_party/rust/fluent-syntax/tests/fixtures/cr.json vendored Normal file
View File

@ -0,0 +1,9 @@
{
"type": "Resource",
"body": [
{
"type": "ResourceComment",
"content": "This entire file uses CR as EOL.\r\rerr01 = Value 01\rerr02 = Value 02\r\rerr03 =\r\r Value 03\r Continued\r\r .title = Title\r\rerr04 = { \"str\r\rerr05 = { $sel -> }\r"
}
]
}

14
third_party/rust/fluent-syntax/tests/fixtures/crlf.ftl vendored Normal file
View File

@ -0,0 +1,14 @@
key01 = Value 01
key02 =
Value 02
Continued
.title = Title
# ERROR Unclosed StringLiteral
err03 = { "str
# ERROR Missing newline after ->.
err04 = { $sel -> }

76
third_party/rust/fluent-syntax/tests/fixtures/crlf.json vendored Normal file
View File

@ -0,0 +1,76 @@
{
"type": "Resource",
"body": [
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "key01"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "Value 01"
}
]
},
"attributes": [],
"comment": null
},
{
"type": "Message",
"id": {
"type": "Identifier",
"name": "key02"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "Value 02\nContinued"
}
]
},
"attributes": [
{
"type": "Attribute",
"id": {
"type": "Identifier",
"name": "title"
},
"value": {
"type": "Pattern",
"elements": [
{
"type": "TextElement",
"value": "Title"
}
]
}
}
],
"comment": null
},
{
"type": "Comment",
"content": "ERROR Unclosed StringLiteral"
},
{
"type": "Junk",
"annotations": [],
"content": "err03 = { \"str\r\n\r\n"
},
{
"type": "Comment",
"content": "ERROR Missing newline after ->."
},
{
"type": "Junk",
"annotations": [],
"content": "err04 = { $sel -> }\r\n"
}
]
}

3
third_party/rust/fluent-syntax/tests/fixtures/eof_comment.ftl vendored Normal file
View File

@ -0,0 +1,3 @@
### NOTE: Disable final newline insertion when editing this file.
# No EOL

13
third_party/rust/fluent-syntax/tests/fixtures/eof_comment.json vendored Normal file
View File

@ -0,0 +1,13 @@
{
"type": "Resource",
"body": [
{
"type": "ResourceComment",
"content": "NOTE: Disable final newline insertion when editing this file."
},
{
"type": "Comment",
"content": "No EOL"
}
]
}

0
third_party/rust/fluent-syntax/tests/fixtures/eof_empty.ftl vendored Normal file
View File

4
third_party/rust/fluent-syntax/tests/fixtures/eof_empty.json vendored Normal file
View File

@ -0,0 +1,4 @@
{
"type": "Resource",
"body": []
}

3
third_party/rust/fluent-syntax/tests/fixtures/eof_id.ftl vendored Normal file
View File

@ -0,0 +1,3 @@
### NOTE: Disable final newline insertion when editing this file.
message-id

14
third_party/rust/fluent-syntax/tests/fixtures/eof_id.json vendored Normal file
View File

@ -0,0 +1,14 @@
{
"type": "Resource",
"body": [
{
"type": "ResourceComment",
"content": "NOTE: Disable final newline insertion when editing this file."
},
{
"type": "Junk",
"annotations": [],
"content": "message-id"
}
]
}

3
third_party/rust/fluent-syntax/tests/fixtures/eof_id_equals.ftl vendored Normal file
View File

@ -0,0 +1,3 @@
### NOTE: Disable final newline insertion when editing this file.
message-id =

14
third_party/rust/fluent-syntax/tests/fixtures/eof_id_equals.json vendored Normal file
View File

@ -0,0 +1,14 @@
{
"type": "Resource",
"body": [
{
"type": "ResourceComment",
"content": "NOTE: Disable final newline insertion when editing this file."
},
{
"type": "Junk",
"annotations": [],
"content": "message-id ="
}
]
}

3
third_party/rust/fluent-syntax/tests/fixtures/eof_junk.ftl vendored Normal file
View File

@ -0,0 +1,3 @@
### NOTE: Disable final newline insertion when editing this file.
000

14
third_party/rust/fluent-syntax/tests/fixtures/eof_junk.json vendored Normal file
View File

@ -0,0 +1,14 @@
{
"type": "Resource",
"body": [
{
"type": "ResourceComment",
"content": "NOTE: Disable final newline insertion when editing this file."
},
{
"type": "Junk",
"annotations": [],
"content": "000"
}
]
}

Some files were not shown because too many files have changed in this diff Show More