Magic-Mirror/gecko-dev
1
0
Fork 0
mirror of https://github.com/mozilla/gecko-dev.git synced 2024-11-28 15:23:51 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Bug 1431356 - Update encoding_rs to 0.7.2 and simd to 0.2.1. r=emk.

Browse Source 
MozReview-Commit-ID: Lp3zyF2rLxN

--HG--
extra : rebase_source : 81b515206ca5d28623cbaead16244ef258da2088
This commit is contained in:
Henri Sivonen 2018-01-18 12:26:21 +02:00
parent 0e3d8ae3b5
commit 733ced6d8d
28 changed files with 3601 additions and 231 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
third_party/rust/encoding_rs/.cargo-checksum.json vendored
View File

File diff suppressed because one or more lines are too long

4
third_party/rust/encoding_rs/CONTRIBUTING.md vendored
View File

@ -37,8 +37,8 @@ rustc.
## rustfmt
Please install [`rustfmt`](https://github.com/rust-lang-nursery/rustfmt) and
run `cargo fmt` before creating a pull request.
The `rustfmt` version used for this code is 0.8.4. Please either use that
version or avoid using `rustfmt` (so as not to reformat all the code).
## Unit tests

14
third_party/rust/encoding_rs/Cargo.toml vendored
View File

@ -12,7 +12,7 @@
[package]
name = "encoding_rs"
version = "0.7.1"
version = "0.7.2"
authors = ["Henri Sivonen <hsivonen@hsivonen.fi>"]
description = "A Gecko-oriented implementation of the Encoding Standard"
homepage = "https://docs.rs/encoding_rs/"
@ -24,6 +24,9 @@ license = "MIT/Apache-2.0"
repository = "https://github.com/hsivonen/encoding_rs"
[profile.release]
lto = true
[dependencies.cfg-if]
version = "0.1.0"
[dependencies.serde]
version = "1.0"
optional = true
@ -31,20 +34,17 @@ optional = true
[dependencies.simd]
version = "0.2.0"
optional = true
[dev-dependencies.bincode]
version = "0.8"
[dependencies.cfg-if]
version = "0.1.0"
[dev-dependencies.serde_derive]
version = "1.0"
[dev-dependencies.serde_json]
version = "1.0"
[dev-dependencies.bincode]
version = "0.8"
[features]
simd-accel = ["simd"]
no-static-ideograph-encoder-tables = []
simd-accel = ["simd"]
[badges.travis-ci]
repository = "hsivonen/encoding_rs"

37
third_party/rust/encoding_rs/README.md vendored
View File

@ -9,6 +9,11 @@ encoding_rs an implementation of the (non-JavaScript parts of) the
[Encoding Standard](https://encoding.spec.whatwg.org/) written in Rust and
used in Gecko (starting with Firefox 56).
Additionally, the `mem` module provides various operations for dealing with
in-RAM text (as opposed to data that's coming from or going to an IO boundary).
The `mem` module is a module instead of a separate crate due to internal
implementation detail efficiencies.
## Functionality
Due to the Gecko use case, encoding_rs supports decoding to and encoding from
@ -43,6 +48,26 @@ Specifically, encoding_rs does the following:
workloads than the standard library; hopefully will get upstreamed some
day) and ASCII.
Additionally, `encoding_rs::mem` does the following:
* Checks if a byte buffer contains only ASCII.
* Checks if a potentially-invalid UTF-16 buffer contains only Basic Latin (ASCII).
* Checks if a valid UTF-8, potentially-invalid UTF-8 or potentially-invalid UTF-16
buffer contains only Latin1 code points (below U+0100).
* Checks if a valid UTF-8, potentially-invalid UTF-8 or potentially-invalid UTF-16
buffer or a code point or a UTF-16 code unit can trigger right-to-left behavior
(suitable for checking if the Unicode Bidirectional Algorithm can be optimized
out).
* Combined versions of the above two checks.
* Converts valid UTF-8, potentially-invalid UTF-8 and Latin1 to UTF-16.
* Converts potentially-invalid UTF-16 and Latin1 to UTF-8.
* Converts UTF-8 and UTF-16 to Latin1 (if in range).
* Finds the first invalid code unit in a buffer of potentially-invalid UTF-16.
* Makes a mutable buffer of potential-invalid UTF-16 contain valid UTF-16.
* Copies ASCII from one buffer to another up to the first non-ASCII byte.
* Converts ASCII to UTF-16 up to the first non-ASCII byte.
* Converts UTF-16 to ASCII up to the first non-Basic Latin code unit.
## Licensing
Please see the file named
@ -63,6 +88,8 @@ using the C++ standard library and [GSL](https://github.com/Microsoft/GSL/) type
For the Gecko context, there's a
[C++ wrapper using the MFBT/XPCOM types](https://searchfox.org/mozilla-central/source/intl/Encoding.h#100).
These bindings do not cover the `mem` module.
## Sample programs
* [Rust](https://github.com/hsivonen/recode_rs)
@ -133,9 +160,9 @@ decode-optimized tables. With realistic work loads, this seemed fast enough
not to be user-visibly slow on Raspberry Pi 3 (which stood in for a phone
for testing) in the Web-exposed encoder use cases.
A framework for measuring performance is [available separately][1].
A framework for measuring performance is [available separately][2].
[1]: https://github.com/hsivonen/encoding_bench/
[2]: https://github.com/hsivonen/encoding_bench/
## Rust Version Compatibility
@ -193,6 +220,12 @@ used in Firefox.
## Release Notes
### 0.7.2
* Add the `mem` module.
* Refactor SIMD code which can affect performance outside the `mem`
module.
### 0.7.1
* When encoding from invalid UTF-16, correctly handle U+DC00 followed by

382
third_party/rust/encoding_rs/src/ascii.rs vendored
View File

@ -24,6 +24,14 @@
#[cfg(all(feature = "simd-accel", any(target_feature = "sse2", all(target_endian = "little", target_arch = "aarch64"))))]
use simd_funcs::*;
// `as` truncates, so works on 32-bit, too.
#[allow(dead_code)]
pub const ASCII_MASK: usize = 0x80808080_80808080u64 as usize;
// `as` truncates, so works on 32-bit, too.
#[allow(dead_code)]
pub const BASIC_LATIN_MASK: usize = 0xFF80FF80_FF80FF80u64 as usize;
#[allow(unused_macros)]
macro_rules! ascii_naive {
($name:ident,
@ -212,6 +220,62 @@ macro_rules! basic_latin_alu {
});
}
#[allow(unused_macros)]
macro_rules! latin1_alu {
($name:ident,
$src_unit:ty,
$dst_unit:ty,
$stride_fn:ident) => (
#[cfg_attr(feature = "cargo-clippy", allow(never_loop))]
#[inline(always)]
pub unsafe fn $name(src: *const $src_unit, dst: *mut $dst_unit, len: usize) {
let mut offset = 0usize;
// This loop is only broken out of as a `goto` forward
loop {
let mut until_alignment = {
if ::std::mem::size_of::<$src_unit>() < ::std::mem::size_of::<$dst_unit>() {
// unpack
let src_until_alignment = (ALIGNMENT - ((src as usize) & ALIGNMENT_MASK)) & ALIGNMENT_MASK;
if (dst.offset(src_until_alignment as isize) as usize) & ALIGNMENT_MASK != 0 {
break;
}
src_until_alignment
} else {
// pack
let dst_until_alignment = (ALIGNMENT - ((dst as usize) & ALIGNMENT_MASK)) & ALIGNMENT_MASK;
if (src.offset(dst_until_alignment as isize) as usize) & ALIGNMENT_MASK != 0 {
break;
}
dst_until_alignment
}
};
if until_alignment + STRIDE_SIZE <= len {
while until_alignment != 0 {
let code_unit = *(src.offset(offset as isize));
*(dst.offset(offset as isize)) = code_unit as $dst_unit;
offset += 1;
until_alignment -= 1;
}
let len_minus_stride = len - STRIDE_SIZE;
loop {
$stride_fn(src.offset(offset as isize) as *const usize,
dst.offset(offset as isize) as *mut usize);
offset += STRIDE_SIZE;
if offset > len_minus_stride {
break;
}
}
}
break;
}
while offset < len {
let code_unit = *(src.offset(offset as isize));
*(dst.offset(offset as isize)) = code_unit as $dst_unit;
offset += 1;
}
});
}
#[allow(unused_macros)]
macro_rules! ascii_simd_check_align {
($name:ident,
@ -294,6 +358,89 @@ macro_rules! ascii_simd_check_align {
});
}
#[allow(unused_macros)]
macro_rules! latin1_simd_check_align {
($name:ident,
$src_unit:ty,
$dst_unit:ty,
$stride_both_aligned:ident,
$stride_src_aligned:ident,
$stride_dst_aligned:ident,
$stride_neither_aligned:ident) => (
#[inline(always)]
pub unsafe fn $name(src: *const $src_unit, dst: *mut $dst_unit, len: usize) {
let mut offset = 0usize;
if STRIDE_SIZE <= len {
let len_minus_stride = len - STRIDE_SIZE;
// XXX Should we first process one stride unconditinoally as unaligned to
// avoid the cost of the branchiness below if the first stride fails anyway?
// XXX Should we just use unaligned SSE2 access unconditionally? It seems that
// on Haswell, it would make sense to just use unaligned and not bother
// checking. Need to benchmark older architectures before deciding.
let dst_masked = (dst as usize) & ALIGNMENT_MASK;
if ((src as usize) & ALIGNMENT_MASK) == 0 {
if dst_masked == 0 {
loop {
$stride_both_aligned(src.offset(offset as isize),
dst.offset(offset as isize));
offset += STRIDE_SIZE;
if offset > len_minus_stride {
break;
}
}
} else {
loop {
$stride_src_aligned(src.offset(offset as isize),
dst.offset(offset as isize));
offset += STRIDE_SIZE;
if offset > len_minus_stride {
break;
}
}
}
} else {
if dst_masked == 0 {
loop {
$stride_dst_aligned(src.offset(offset as isize),
dst.offset(offset as isize));
offset += STRIDE_SIZE;
if offset > len_minus_stride {
break;
}
}
} else {
loop {
$stride_neither_aligned(src.offset(offset as isize),
dst.offset(offset as isize));
offset += STRIDE_SIZE;
if offset > len_minus_stride {
break;
}
}
}
}
}
while offset < len {
let code_unit = *(src.offset(offset as isize));
// On x86_64, this loop autovectorizes but in the pack
// case there are instructions whose purpose is to make sure
// each u16 in the vector is truncated before packing. However,
// since we don't care about saturating behavior of SSE2 packing
// when the input isn't Latin1, those instructions are useless.
// Unfortunately, using the `assume` intrinsic to lie to the
// optimizer doesn't make LLVM omit the trunctation that we
// don't need. Possibly this loop could be manually optimized
// to do the sort of thing that LLVM does but without the
// ANDing the read vectors of u16 with a constant that discards
// the high half of each u16. As far as I can tell, the
// optimization assumes that doing a SIMD read past the end of
// the array is OK.
*(dst.offset(offset as isize)) = code_unit as $dst_unit;
offset += 1;
}
});
}
#[allow(unused_macros)]
macro_rules! ascii_simd_unalign {
($name:ident,
@ -328,6 +475,34 @@ macro_rules! ascii_simd_unalign {
});
}
#[allow(unused_macros)]
macro_rules! latin1_simd_unalign {
($name:ident,
$src_unit:ty,
$dst_unit:ty,
$stride_neither_aligned:ident) => (
#[inline(always)]
pub unsafe fn $name(src: *const $src_unit, dst: *mut $dst_unit, len: usize) {
let mut offset = 0usize;
if STRIDE_SIZE <= len {
let len_minus_stride = len - STRIDE_SIZE;
loop {
$stride_neither_aligned(src.offset(offset as isize),
dst.offset(offset as isize));
offset += STRIDE_SIZE;
if offset > len_minus_stride {
break;
}
}
}
while offset < len {
let code_unit = *(src.offset(offset as isize));
*(dst.offset(offset as isize)) = code_unit as $dst_unit;
offset += 1;
}
});
}
#[allow(unused_macros)]
macro_rules! ascii_to_ascii_simd_stride {
($name:ident,
@ -336,7 +511,7 @@ macro_rules! ascii_to_ascii_simd_stride {
#[inline(always)]
pub unsafe fn $name(src: *const u8, dst: *mut u8) -> bool {
let simd = $load(src);
if !is_ascii(simd) {
if !simd_is_ascii(simd) {
return false;
}
$store(dst, simd);
@ -352,7 +527,7 @@ macro_rules! ascii_to_basic_latin_simd_stride {
#[inline(always)]
pub unsafe fn $name(src: *const u8, dst: *mut u16) -> bool {
let simd = $load(src);
if !is_ascii(simd) {
if !simd_is_ascii(simd) {
return false;
}
let (first, second) = simd_unpack(simd);
@ -362,6 +537,20 @@ macro_rules! ascii_to_basic_latin_simd_stride {
});
}
#[allow(unused_macros)]
macro_rules! unpack_simd_stride {
($name:ident,
$load:ident,
$store:ident) => (
#[inline(always)]
pub unsafe fn $name(src: *const u8, dst: *mut u16) {
let simd = $load(src);
let (first, second) = simd_unpack(simd);
$store(dst, first);
$store(dst.offset(8), second);
});
}
#[allow(unused_macros)]
macro_rules! basic_latin_to_ascii_simd_stride {
($name:ident,
@ -371,7 +560,7 @@ macro_rules! basic_latin_to_ascii_simd_stride {
pub unsafe fn $name(src: *const u16, dst: *mut u8) -> bool {
let first = $load(src);
let second = $load(src.offset(8));
if is_basic_latin(first | second) {
if simd_is_basic_latin(first | second) {
$store(dst, simd_pack(first, second));
true
} else {
@ -380,23 +569,40 @@ macro_rules! basic_latin_to_ascii_simd_stride {
});
}
#[allow(unused_macros)]
macro_rules! pack_simd_stride {
($name:ident,
$load:ident,
$store:ident) => (
#[inline(always)]
pub unsafe fn $name(src: *const u16, dst: *mut u8) {
let first = $load(src);
let second = $load(src.offset(8));
$store(dst, simd_pack(first, second));
});
}
cfg_if! {
if #[cfg(all(feature = "simd-accel", target_endian = "little", target_arch = "aarch64"))] {
// SIMD with the same instructions for aligned and unaligned loads and stores
pub const STRIDE_SIZE: usize = 16;
const ALIGNMENT: usize = 8;
// pub const ALIGNMENT: usize = 8;
ascii_to_ascii_simd_stride!(ascii_to_ascii_stride_neither_aligned, load16_unaligned, store16_unaligned);
ascii_to_basic_latin_simd_stride!(ascii_to_basic_latin_stride_neither_aligned, load16_unaligned, store8_unaligned);
unpack_simd_stride!(unpack_stride_neither_aligned, load16_unaligned, store8_unaligned);
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_neither_aligned, load8_unaligned, store16_unaligned);
pack_simd_stride!(pack_stride_neither_aligned, load8_unaligned, store16_unaligned);
ascii_simd_unalign!(ascii_to_ascii, u8, u8, ascii_to_ascii_stride_neither_aligned);
ascii_simd_unalign!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_neither_aligned);
ascii_simd_unalign!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_neither_aligned);
latin1_simd_unalign!(unpack_latin1, u8, u16, unpack_stride_neither_aligned);
latin1_simd_unalign!(pack_latin1, u16, u8, pack_stride_neither_aligned);
} else if #[cfg(all(feature = "simd-accel", target_feature = "sse2"))] {
// SIMD with different instructions for aligned and unaligned loads and stores.
//
@ -406,7 +612,7 @@ cfg_if! {
pub const STRIDE_SIZE: usize = 16;
const ALIGNMENT_MASK: usize = 15;
pub const ALIGNMENT_MASK: usize = 15;
ascii_to_ascii_simd_stride!(ascii_to_ascii_stride_both_aligned, load16_aligned, store16_aligned);
ascii_to_ascii_simd_stride!(ascii_to_ascii_stride_src_aligned, load16_aligned, store16_unaligned);
@ -418,31 +624,37 @@ cfg_if! {
ascii_to_basic_latin_simd_stride!(ascii_to_basic_latin_stride_dst_aligned, load16_unaligned, store8_aligned);
ascii_to_basic_latin_simd_stride!(ascii_to_basic_latin_stride_neither_aligned, load16_unaligned, store8_unaligned);
unpack_simd_stride!(unpack_stride_both_aligned, load16_aligned, store8_aligned);
unpack_simd_stride!(unpack_stride_src_aligned, load16_aligned, store8_unaligned);
unpack_simd_stride!(unpack_stride_dst_aligned, load16_unaligned, store8_aligned);
unpack_simd_stride!(unpack_stride_neither_aligned, load16_unaligned, store8_unaligned);
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_both_aligned, load8_aligned, store16_aligned);
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_src_aligned, load8_aligned, store16_unaligned);
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_dst_aligned, load8_unaligned, store16_aligned);
basic_latin_to_ascii_simd_stride!(basic_latin_to_ascii_stride_neither_aligned, load8_unaligned, store16_unaligned);
pack_simd_stride!(pack_stride_both_aligned, load8_aligned, store16_aligned);
pack_simd_stride!(pack_stride_src_aligned, load8_aligned, store16_unaligned);
pack_simd_stride!(pack_stride_dst_aligned, load8_unaligned, store16_aligned);
pack_simd_stride!(pack_stride_neither_aligned, load8_unaligned, store16_unaligned);
ascii_simd_check_align!(ascii_to_ascii, u8, u8, ascii_to_ascii_stride_both_aligned, ascii_to_ascii_stride_src_aligned, ascii_to_ascii_stride_dst_aligned, ascii_to_ascii_stride_neither_aligned);
ascii_simd_check_align!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_both_aligned, ascii_to_basic_latin_stride_src_aligned, ascii_to_basic_latin_stride_dst_aligned, ascii_to_basic_latin_stride_neither_aligned);
ascii_simd_check_align!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_both_aligned, basic_latin_to_ascii_stride_src_aligned, basic_latin_to_ascii_stride_dst_aligned, basic_latin_to_ascii_stride_neither_aligned);
latin1_simd_check_align!(unpack_latin1, u8, u16, unpack_stride_both_aligned, unpack_stride_src_aligned, unpack_stride_dst_aligned, unpack_stride_neither_aligned);
latin1_simd_check_align!(pack_latin1, u16, u8, pack_stride_both_aligned, pack_stride_src_aligned, pack_stride_dst_aligned, pack_stride_neither_aligned);
} else if #[cfg(all(target_endian = "little", target_pointer_width = "64"))] {
// Aligned ALU word, little-endian, 64-bit
pub const STRIDE_SIZE: usize = 16;
const ALIGNMENT: usize = 8;
pub const ALIGNMENT: usize = 8;
const ALIGNMENT_MASK: usize = 7;
pub const ALIGNMENT_MASK: usize = 7;
#[inline(always)]
unsafe fn ascii_to_basic_latin_stride_little_64(src: *const usize, dst: *mut usize) -> bool {
let word = *src;
let second_word = *(src.offset(1));
// Check if the words contains non-ASCII
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
return false;
}
unsafe fn unpack_alu(word: usize, second_word: usize, dst: *mut usize) {
let first = ((0x00000000_FF000000usize & word) << 24) |
((0x00000000_00FF0000usize & word) << 16) |
((0x00000000_0000FF00usize & word) << 8) |
@ -463,18 +675,10 @@ cfg_if! {
*(dst.offset(1)) = second;
*(dst.offset(2)) = third;
*(dst.offset(3)) = fourth;
true
}
#[inline(always)]
unsafe fn basic_latin_to_ascii_stride_little_64(src: *const usize, dst: *mut usize) -> bool {
let first = *src;
let second = *(src.offset(1));
let third = *(src.offset(2));
let fourth = *(src.offset(3));
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
return false;
}
unsafe fn pack_alu(first: usize, second: usize, third: usize, fourth: usize, dst: *mut usize) {
let word = ((0x00FF0000_00000000usize & second) << 8) |
((0x000000FF_00000000usize & second) << 16) |
((0x00000000_00FF0000usize & second) << 24) |
@ -493,28 +697,18 @@ cfg_if! {
(0x00000000_000000FFusize & third);
*dst = word;
*(dst.offset(1)) = second_word;
true
}
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_little_64);
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_little_64);
} else if #[cfg(all(target_endian = "little", target_pointer_width = "32"))] {
// Aligned ALU word, little-endian, 32-bit
pub const STRIDE_SIZE: usize = 8;
const ALIGNMENT: usize = 4;
pub const ALIGNMENT: usize = 4;
const ALIGNMENT_MASK: usize = 3;
pub const ALIGNMENT_MASK: usize = 3;
#[inline(always)]
unsafe fn ascii_to_basic_latin_stride_little_32(src: *const usize, dst: *mut usize) -> bool {
let word = *src;
let second_word = *(src.offset(1));
// Check if the words contains non-ASCII
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
return false;
}
unsafe fn unpack_alu(word: usize, second_word: usize, dst: *mut usize) {
let first = ((0x0000FF00usize & word) << 8) |
(0x000000FFusize & word);
let second = ((0xFF000000usize & word) >> 8) |
@ -527,18 +721,10 @@ cfg_if! {
*(dst.offset(1)) = second;
*(dst.offset(2)) = third;
*(dst.offset(3)) = fourth;
return true;
}
#[inline(always)]
unsafe fn basic_latin_to_ascii_stride_little_32(src: *const usize, dst: *mut usize) -> bool {
let first = *src;
let second = *(src.offset(1));
let third = *(src.offset(2));
let fourth = *(src.offset(3));
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
return false;
}
unsafe fn pack_alu(first: usize, second: usize, third: usize, fourth: usize, dst: *mut usize) {
let word = ((0x00FF0000usize & second) << 8) |
((0x000000FFusize & second) << 16) |
((0x00FF0000usize & first) >> 8) |
@ -549,28 +735,18 @@ cfg_if! {
(0x000000FFusize & third);
*dst = word;
*(dst.offset(1)) = second_word;
return true;
}
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_little_32);
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_little_32);
} else if #[cfg(all(target_endian = "big", target_pointer_width = "64"))] {
// Aligned ALU word, big-endian, 64-bit
pub const STRIDE_SIZE: usize = 16;
const ALIGNMENT: usize = 8;
pub const ALIGNMENT: usize = 8;
const ALIGNMENT_MASK: usize = 7;
pub const ALIGNMENT_MASK: usize = 7;
#[inline(always)]
unsafe fn ascii_to_basic_latin_stride_big_64(src: *const usize, dst: *mut usize) -> bool {
let word = *src;
let second_word = *(src.offset(1));
// Check if the words contains non-ASCII
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
return false;
}
unsafe fn unpack_alu(word: usize, second_word: usize, dst: *mut usize) {
let first = ((0xFF000000_00000000usize & word) >> 8) |
((0x00FF0000_00000000usize & word) >> 16) |
((0x0000FF00_00000000usize & word) >> 24) |
@ -591,18 +767,10 @@ cfg_if! {
*(dst.offset(1)) = second;
*(dst.offset(2)) = third;
*(dst.offset(3)) = fourth;
return true;
}
#[inline(always)]
unsafe fn basic_latin_to_ascii_stride_big_64(src: *const usize, dst: *mut usize) -> bool {
let first = *src;
let second = *(src.offset(1));
let third = *(src.offset(2));
let fourth = *(src.offset(3));
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
return false;
}
unsafe fn pack_alu(first: usize, second: usize, third: usize, fourth: usize, dst: *mut usize) {
let word = ((0x00FF0000_00000000usize & first) << 8) |
((0x000000FF_00000000usize & first) << 16) |
((0x00000000_00FF0000usize & first) << 24) |
@ -621,28 +789,18 @@ cfg_if! {
(0x00000000_000000FFusize & fourth);
*dst = word;
*(dst.offset(1)) = second_word;
return true;
}
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_big_64);
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_big_64);
} else if #[cfg(all(target_endian = "big", target_pointer_width = "32"))] {
// Aligned ALU word, big-endian, 32-bit
pub const STRIDE_SIZE: usize = 8;
const ALIGNMENT: usize = 4;
pub const ALIGNMENT: usize = 4;
const ALIGNMENT_MASK: usize = 3;
pub const ALIGNMENT_MASK: usize = 3;
#[inline(always)]
unsafe fn ascii_to_basic_latin_stride_big_32(src: *const usize, dst: *mut usize) -> bool {
let word = *src;
let second_word = *(src.offset(1));
// Check if the words contains non-ASCII
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
return false;
}
unsafe fn unpack_alu(word: usize, second_word: usize, dst: *mut usize) {
let first = ((0xFF000000usize & word) >> 8) |
((0x00FF0000usize & word) >> 16);
let second = ((0x0000FF00usize & word) << 8) |
@ -655,18 +813,10 @@ cfg_if! {
*(dst.offset(1)) = second;
*(dst.offset(2)) = third;
*(dst.offset(3)) = fourth;
return true;
}
#[inline(always)]
unsafe fn basic_latin_to_ascii_stride_big_32(src: *const usize, dst: *mut usize) -> bool {
let first = *src;
let second = *(src.offset(1));
let third = *(src.offset(2));
let fourth = *(src.offset(3));
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
return false;
}
unsafe fn pack_alu(first: usize, second: usize, third: usize, fourth: usize, dst: *mut usize) {
let word = ((0x00FF0000usize & first) << 8) |
((0x000000FFusize & first) << 16) |
((0x00FF0000usize & second) >> 8) |
@ -677,11 +827,7 @@ cfg_if! {
(0x000000FFusize & fourth);
*dst = word;
*(dst.offset(1)) = second_word;
return true;
}
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_big_32);
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_big_32);
} else {
ascii_naive!(ascii_to_ascii, u8, u8);
ascii_naive!(ascii_to_basic_latin, u8, u16);
@ -716,7 +862,7 @@ cfg_if! {
let len_minus_stride = len - STRIDE_SIZE;
loop {
let simd = unsafe { load16_unaligned(src.offset(offset as isize)) };
if !is_ascii(simd) {
if !simd_is_ascii(simd) {
break;
}
offset += STRIDE_SIZE;
@ -787,9 +933,51 @@ cfg_if! {
None
}
} else {
// `as` truncates, so works on 32-bit, too.
const ASCII_MASK: usize = 0x80808080_80808080u64 as usize;
const BASIC_LATIN_MASK: usize = 0xFF80FF80_FF80FF80u64 as usize;
#[inline(always)]
unsafe fn unpack_latin1_stride_alu(src: *const usize, dst: *mut usize) {
let word = *src;
let second_word = *(src.offset(1));
unpack_alu(word, second_word, dst);
}
#[inline(always)]
unsafe fn pack_latin1_stride_alu(src: *const usize, dst: *mut usize) {
let first = *src;
let second = *(src.offset(1));
let third = *(src.offset(2));
let fourth = *(src.offset(3));
pack_alu(first, second, third, fourth, dst);
}
#[inline(always)]
unsafe fn ascii_to_basic_latin_stride_alu(src: *const usize, dst: *mut usize) -> bool {
let word = *src;
let second_word = *(src.offset(1));
// Check if the words contains non-ASCII
if (word & ASCII_MASK) | (second_word & ASCII_MASK) != 0 {
return false;
}
unpack_alu(word, second_word, dst);
true
}
#[inline(always)]
unsafe fn basic_latin_to_ascii_stride_alu(src: *const usize, dst: *mut usize) -> bool {
let first = *src;
let second = *(src.offset(1));
let third = *(src.offset(2));
let fourth = *(src.offset(3));
if (first & BASIC_LATIN_MASK) | (second & BASIC_LATIN_MASK) | (third & BASIC_LATIN_MASK) | (fourth & BASIC_LATIN_MASK) != 0 {
return false;
}
pack_alu(first, second, third, fourth, dst);
true
}
basic_latin_alu!(ascii_to_basic_latin, u8, u16, ascii_to_basic_latin_stride_alu);
basic_latin_alu!(basic_latin_to_ascii, u16, u8, basic_latin_to_ascii_stride_alu);
latin1_alu!(unpack_latin1, u8, u16, unpack_latin1_stride_alu);
latin1_alu!(pack_latin1, u16, u8, pack_latin1_stride_alu);
#[inline(always)]
unsafe fn ascii_to_ascii_stride(src: *const usize, dst: *mut usize) -> Option<usize> {

14
third_party/rust/encoding_rs/src/gb18030.rs vendored
View File

@ -394,9 +394,7 @@ fn gbk_encode_non_unified(bmp: u16) -> Option<(usize, usize)> {
// PUA between Hanzi Levels
let bmp_minus_pua_between_hanzi = bmp.wrapping_sub(0xE810);
if bmp_minus_pua_between_hanzi < 5 {
return Some(
(0x81 + 0x56, 0xFF - 5 + bmp_minus_pua_between_hanzi as usize),
);
return Some((0x81 + 0x56, 0xFF - 5 + bmp_minus_pua_between_hanzi as usize));
}
None
}
@ -595,6 +593,14 @@ mod tests {
// 0xFF
decode_gb18030(b"\xFF\x40", "\u{FFFD}\u{0040}");
decode_gb18030(b"\xE3\xFF\x9A\x33", "\u{FFFD}\u{FFFD}"); // not \u{FFFD}\u{FFFD}\u{0033} !
decode_gb18030(b"\xFF\x32\x9A\x33", "\u{FFFD}\u{0032}\u{FFFD}"); // not \u{FFFD}\u{0032}\u{FFFD}\u{0033} !
decode_gb18030(b"\xFF\x40\x00", "\u{FFFD}\u{0040}\u{0000}");
decode_gb18030(b"\xE3\xFF\x9A\x33\x00", "\u{FFFD}\u{FFFD}\u{0033}\u{0000}");
decode_gb18030(
b"\xFF\x32\x9A\x33\x00",
"\u{FFFD}\u{0032}\u{FFFD}\u{0033}\u{0000}",
);
// Four bytes
decode_gb18030(b"\x81\x30\x81\x30", "\u{0080}");
@ -605,7 +611,7 @@ mod tests {
decode_gb18030(b"\xE3\x32\x9A\x36\x81\x30", "\u{FFFD}\u{FFFD}");
decode_gb18030(b"\xE3\x32\x9A\x36\x81\x40", "\u{FFFD}\u{4E02}");
decode_gb18030(b"\xE3\x32\x9A", "\u{FFFD}"); // not \u{FFFD}\u{0032}\u{FFFD} !
decode_gb18030(b"\xE3\x32\x9A\x00", "\u{FFFD}\u{0032}\u{FFFD}\u{0000}");
}
#[test]

58
third_party/rust/encoding_rs/src/lib.rs vendored
View File

@ -8,7 +8,7 @@
// except according to those terms.
#![cfg_attr(feature = "cargo-clippy", allow(doc_markdown, inline_always, new_ret_no_self))]
#![doc(html_root_url = "https://docs.rs/encoding_rs/0.7.1")]
#![doc(html_root_url = "https://docs.rs/encoding_rs/0.7.2")]
//! encoding_rs is a Gecko-oriented Free Software / Open Source implementation
//! of the [Encoding Standard](https://encoding.spec.whatwg.org/) in Rust.
@ -17,6 +17,10 @@
//! streamability goals are browser-oriented, and that FFI-friendliness is a
//! goal.
//!
//! Additionally, the `mem` module provides functions that are useful for
//! applications that need to be able to deal with legacy in-memory
//! representations of Unicode.
//!
//! # Availability
//!
//! The code is available under the
@ -491,7 +495,7 @@
//! </tbody>
//! </table>
#![cfg_attr(feature = "simd-accel", feature(cfg_target_feature, platform_intrinsics))]
#![cfg_attr(feature = "simd-accel", feature(cfg_target_feature, platform_intrinsics, core_intrinsics))]
#[macro_use]
extern crate cfg_if;
@ -539,6 +543,8 @@ mod handles;
mod data;
mod variant;
pub mod mem;
use variant::*;
use utf_8::utf8_valid_up_to;
use ascii::ascii_valid_up_to;
@ -2030,20 +2036,20 @@ static ENCODINGS_IN_LABEL_SORT: [&'static Encoding; 219] = [&WINDOWS_1252_INIT,
/// # Streaming vs. Non-Streaming
///
/// When you have the entire input in a single buffer, you can use the
/// methods [`decode()`][1], [`decode_with_bom_removal()`][2],
/// [`decode_without_bom_handling()`][3],
/// [`decode_without_bom_handling_and_without_replacement()`][4] and
/// [`encode()`][5]. (These methods are available to Rust callers only and are
/// methods [`decode()`][3], [`decode_with_bom_removal()`][3],
/// [`decode_without_bom_handling()`][5],
/// [`decode_without_bom_handling_and_without_replacement()`][6] and
/// [`encode()`][7]. (These methods are available to Rust callers only and are
/// not available in the C API.) Unlike the rest of the API available to Rust,
/// these methods perform heap allocations. You should the `Decoder` and
/// `Encoder` objects when your input is split into multiple buffers or when
/// you want to control the allocation of the output buffers.
///
/// [1]: #method.decode
/// [2]: #method.decode_with_bom_removal
/// [3]: #method.decode_without_bom_handling
/// [4]: #method.decode_without_bom_handling_and_without_replacement
/// [5]: #method.encode
/// [3]: #method.decode
/// [4]: #method.decode_with_bom_removal
/// [5]: #method.decode_without_bom_handling
/// [6]: #method.decode_without_bom_handling_and_without_replacement
/// [7]: #method.encode
///
/// # Instances
///
@ -2222,6 +2228,7 @@ impl Encoding {
/// unsafe fallback for labels that `for_label()` maps to `Some(REPLACEMENT)`.
///
/// Available via the C wrapper.
#[inline]
pub fn for_label_no_replacement(label: &[u8]) -> Option<&'static Encoding> {
match Encoding::for_label(label) {
None => None,
@ -2246,6 +2253,7 @@ impl Encoding {
/// or UTF-16BE BOM or `None` otherwise.
///
/// Available via the C wrapper.
#[inline]
pub fn for_bom(buffer: &[u8]) -> Option<(&'static Encoding, usize)> {
if buffer.starts_with(b"\xEF\xBB\xBF") {
Some((UTF_8, 3))
@ -2264,6 +2272,7 @@ impl Encoding {
/// `document.characterSet` property.
///
/// Available via the C wrapper.
#[inline]
pub fn name(&'static self) -> &'static str {
self.name
}
@ -2272,6 +2281,7 @@ impl Encoding {
/// `char`. (Only true if the output encoding is UTF-8.)
///
/// Available via the C wrapper.
#[inline]
pub fn can_encode_everything(&'static self) -> bool {
self.output_encoding() == UTF_8
}
@ -2280,12 +2290,14 @@ impl Encoding {
/// U+0000...U+007F and vice versa.
///
/// Available via the C wrapper.
#[inline]
pub fn is_ascii_compatible(&'static self) -> bool {
!(self == REPLACEMENT || self == UTF_16BE || self == UTF_16LE || self == ISO_2022_JP)
}
/// Checks whether the bytes 0x00...0x7F map mostly to the characters
/// U+0000...U+007F and vice versa.
#[inline]
fn is_potentially_borrowable(&'static self) -> bool {
!(self == REPLACEMENT || self == UTF_16BE || self == UTF_16LE)
}
@ -2294,6 +2306,7 @@ impl Encoding {
/// UTF-16BE, UTF-16LE and replacement and the encoding itself otherwise.
///
/// Available via the C wrapper.
#[inline]
pub fn output_encoding(&'static self) -> &'static Encoding {
if self == REPLACEMENT || self == UTF_16BE || self == UTF_16LE {
UTF_8
@ -2336,6 +2349,7 @@ impl Encoding {
/// `usize`.
///
/// Available to Rust only.
#[inline]
pub fn decode<'a>(&'static self, bytes: &'a [u8]) -> (Cow<'a, str>, &'static Encoding, bool) {
let (encoding, without_bom) = match Encoding::for_bom(bytes) {
Some((encoding, bom_length)) => (encoding, &bytes[bom_length..]),
@ -2378,6 +2392,7 @@ impl Encoding {
/// `usize`.
///
/// Available to Rust only.
#[inline]
pub fn decode_with_bom_removal<'a>(&'static self, bytes: &'a [u8]) -> (Cow<'a, str>, bool) {
let without_bom = if self == UTF_8 && bytes.starts_with(b"\xEF\xBB\xBF") {
&bytes[3..]
@ -2689,6 +2704,7 @@ impl Encoding {
/// for UTF-8, UTF-16LE or UTF-16BE instead of this encoding.
///
/// Available via the C wrapper.
#[inline]
pub fn new_decoder(&'static self) -> Decoder {
Decoder::new(self, self.new_variant_decoder(), BomHandling::Sniff)
}
@ -2702,6 +2718,7 @@ impl Encoding {
/// encoding.
///
/// Available via the C wrapper.
#[inline]
pub fn new_decoder_with_bom_removal(&'static self) -> Decoder {
Decoder::new(self, self.new_variant_decoder(), BomHandling::Remove)
}
@ -2717,6 +2734,7 @@ impl Encoding {
/// instead of this method to cause the BOM to be removed.
///
/// Available via the C wrapper.
#[inline]
pub fn new_decoder_without_bom_handling(&'static self) -> Decoder {
Decoder::new(self, self.new_variant_decoder(), BomHandling::Off)
}
@ -2724,6 +2742,7 @@ impl Encoding {
/// Instantiates a new encoder for the output encoding of this encoding.
///
/// Available via the C wrapper.
#[inline]
pub fn new_encoder(&'static self) -> Encoder {
let enc = self.output_encoding();
enc.variant.new_encoder(enc)
@ -2767,6 +2786,7 @@ impl Encoding {
}
impl PartialEq for Encoding {
#[inline]
fn eq(&self, other: &Encoding) -> bool {
(self as *const Encoding) == (other as *const Encoding)
}
@ -2775,12 +2795,14 @@ impl PartialEq for Encoding {
impl Eq for Encoding {}
impl Hash for Encoding {
#[inline]
fn hash<H: Hasher>(&self, state: &mut H) {
(self as *const Encoding).hash(state);
}
}
impl std::fmt::Debug for Encoding {
#[inline]
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "Encoding {{ {} }}", self.name)
}
@ -2788,6 +2810,7 @@ impl std::fmt::Debug for Encoding {
#[cfg(feature = "serde")]
impl Serialize for Encoding {
#[inline]
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where S: Serializer
{
@ -3054,6 +3077,7 @@ impl Decoder {
/// of the decoder.
///
/// Available via the C wrapper.
#[inline]
pub fn encoding(&self) -> &'static Encoding {
self.encoding
}
@ -3769,12 +3793,14 @@ impl Encoder {
}
/// The `Encoding` this `Encoder` is for.
#[inline]
pub fn encoding(&self) -> &'static Encoding {
self.encoding
}
/// Returns `true` if this is an ISO-2022-JP encoder that's not in the
/// ASCII state and `false` otherwise.
#[inline]
pub fn has_pending_state(&self) -> bool {
self.variant.has_pending_state()
}
@ -4111,6 +4137,16 @@ fn in_range16(i: u16, start: u16, end: u16) -> bool {
i.wrapping_sub(start) < (end - start)
}
#[inline(always)]
fn in_range32(i: u32, start: u32, end: u32) -> bool {
i.wrapping_sub(start) < (end - start)
}
#[inline(always)]
fn in_inclusive_range8(i: u8, start: u8, end: u8) -> bool {
i.wrapping_sub(start) <= (end - start)
}
#[inline(always)]
fn in_inclusive_range16(i: u16, start: u16, end: u16) -> bool {
i.wrapping_sub(start) <= (end - start)

2873
third_party/rust/encoding_rs/src/mem.rs vendored Normal file
View File

File diff suppressed because it is too large Load Diff

159
third_party/rust/encoding_rs/src/simd_funcs.rs vendored
View File

@ -21,6 +21,7 @@ pub unsafe fn load16_unaligned(ptr: *const u8) -> u8x16 {
simd
}
#[allow(dead_code)]
#[inline(always)]
pub unsafe fn load16_aligned(ptr: *const u8) -> u8x16 {
*(ptr as *const u8x16)
@ -31,6 +32,7 @@ pub unsafe fn store16_unaligned(ptr: *mut u8, s: u8x16) {
::std::ptr::copy_nonoverlapping(&s as *const u8x16 as *const u8, ptr, 16);
}
#[allow(dead_code)]
#[inline(always)]
pub unsafe fn store16_aligned(ptr: *mut u8, s: u8x16) {
*(ptr as *mut u8x16) = s;
@ -43,6 +45,7 @@ pub unsafe fn load8_unaligned(ptr: *const u16) -> u16x8 {
simd
}
#[allow(dead_code)]
#[inline(always)]
pub unsafe fn load8_aligned(ptr: *const u16) -> u16x8 {
*(ptr as *const u16x8)
@ -53,6 +56,7 @@ pub unsafe fn store8_unaligned(ptr: *mut u16, s: u16x8) {
::std::ptr::copy_nonoverlapping(&s as *const u16x8 as *const u8, ptr as *mut u8, 16);
}
#[allow(dead_code)]
#[inline(always)]
pub unsafe fn store8_aligned(ptr: *mut u16, s: u16x8) {
*(ptr as *mut u16x8) = s;
@ -89,7 +93,7 @@ cfg_if! {
cfg_if! {
if #[cfg(target_feature = "sse2")] {
#[inline(always)]
pub fn is_ascii(s: u8x16) -> bool {
pub fn simd_is_ascii(s: u8x16) -> bool {
unsafe {
let signed: i8x16 = ::std::mem::transmute_copy(&s);
x86_mm_movemask_epi8(signed) == 0
@ -101,16 +105,42 @@ cfg_if! {
}
#[inline(always)]
pub fn is_ascii(s: u8x16) -> bool {
pub fn simd_is_ascii(s: u8x16) -> bool {
unsafe {
aarch64_vmaxvq_u8(s) < 0x80
}
}
} else {
#[inline(always)]
pub fn is_ascii(s: u8x16) -> bool {
let highest_ascii = u8x16::splat(0x7F);
!s.gt(highest_ascii).any()
pub fn simd_is_ascii(s: u8x16) -> bool {
let above_ascii = u8x16::splat(0x80);
s.lt(above_ascii).all()
}
}
}
cfg_if! {
if #[cfg(target_feature = "sse2")] {
#[inline(always)]
pub fn simd_is_str_latin1(s: u8x16) -> bool {
if simd_is_ascii(s) {
return true;
}
let above_str_latin1 = u8x16::splat(0xC4);
s.lt(above_str_latin1).all()
}
} else if #[cfg(target_arch = "aarch64")]{
#[inline(always)]
pub fn simd_is_str_latin1(s: u8x16) -> bool {
unsafe {
aarch64_vmaxvq_u8(s) < 0xC4
}
}
} else {
#[inline(always)]
pub fn simd_is_str_latin1(s: u8x16) -> bool {
let above_str_latin1 = u8x16::splat(0xC4);
s.lt(above_str_latin1).all()
}
}
}
@ -122,20 +152,107 @@ cfg_if! {
}
#[inline(always)]
pub fn is_basic_latin(s: u16x8) -> bool {
pub fn simd_is_basic_latin(s: u16x8) -> bool {
unsafe {
aarch64_vmaxvq_u16(s) < 0x80
}
}
#[inline(always)]
pub fn simd_is_latin1(s: u16x8) -> bool {
unsafe {
aarch64_vmaxvq_u16(s) < 0x100
}
}
} else {
#[inline(always)]
pub fn is_basic_latin(s: u16x8) -> bool {
let highest_ascii = u16x8::splat(0x7F);
!s.gt(highest_ascii).any()
pub fn simd_is_basic_latin(s: u16x8) -> bool {
let above_ascii = u16x8::splat(0x80);
s.lt(above_ascii).all()
}
#[inline(always)]
pub fn simd_is_latin1(s: u16x8) -> bool {
// For some reason, on SSE2 this formulation
// seems faster in this case while the above
// function is better the other way round...
let highest_latin1 = u16x8::splat(0xFF);
!s.gt(highest_latin1).any()
}
}
}
#[inline(always)]
pub fn contains_surrogates(s: u16x8) -> bool {
let mask = u16x8::splat(0xF800);
let surrogate_bits = u16x8::splat(0xD800);
(s & mask).eq(surrogate_bits).any()
}
cfg_if! {
if #[cfg(target_arch = "aarch64")]{
macro_rules! aarch64_return_false_if_below_hebrew {
($s:ident) => ({
unsafe {
if aarch64_vmaxvq_u16($s) < 0x0590 {
return false;
}
}
})
}
macro_rules! non_aarch64_return_false_if_all {
($s:ident) => ()
}
} else {
macro_rules! aarch64_return_false_if_below_hebrew {
($s:ident) => ()
}
macro_rules! non_aarch64_return_false_if_all {
($s:ident) => ({
if $s.all() {
return false;
}
})
}
}
}
macro_rules! in_range16x8 {
($s:ident, $start:expr, $end:expr) => ({
// SIMD sub is wrapping
($s - u16x8::splat($start)).lt(u16x8::splat($end - $start))
})
}
#[inline(always)]
pub fn is_u16x8_bidi(s: u16x8) -> bool {
// We try to first quickly refute the RTLness of the vector. If that
// fails, we do the real RTL check, so in that case we end up wasting
// the work for the up-front quick checks. Even the quick-check is
// two-fold in order to return `false` ASAP if everything is below
// Hebrew.
aarch64_return_false_if_below_hebrew!(s);
let below_hebrew = s.lt(u16x8::splat(0x0590));
non_aarch64_return_false_if_all!(below_hebrew);
if (below_hebrew | in_range16x8!(s, 0x0900, 0x200F) | in_range16x8!(s, 0x2068, 0xD802)).all() {
return false;
}
// Quick refutation failed. Let's do the full check.
(in_range16x8!(s, 0x0590, 0x0900) | in_range16x8!(s, 0xFB50, 0xFE00) |
in_range16x8!(s, 0xFE70, 0xFF00) | in_range16x8!(s, 0xD802, 0xD804) |
in_range16x8!(s, 0xD83A, 0xD83C) | s.eq(u16x8::splat(0x200F)) |
s.eq(u16x8::splat(0x202B)) | s.eq(u16x8::splat(0x202E)) | s.eq(u16x8::splat(0x2067)))
.any()
}
#[inline(always)]
pub fn simd_unpack(s: u8x16) -> (u16x8, u16x8) {
unsafe {
@ -206,7 +323,7 @@ mod tests {
}
#[test]
fn test_is_basic_latin_success() {
fn test_simd_is_basic_latin_success() {
let ascii: [u8; 16] = [0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70, 0x71,
0x72, 0x73, 0x74, 0x75, 0x76];
let basic_latin: [u16; 16] = [0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70,
@ -216,7 +333,7 @@ mod tests {
let mut vec = Vec::with_capacity(16);
vec.resize(16, 0u8);
let ptr = vec.as_mut_ptr();
assert!(is_basic_latin(first | second));
assert!(simd_is_basic_latin(first | second));
unsafe {
store16_unaligned(ptr, simd_pack(first, second));
}
@ -224,46 +341,46 @@ mod tests {
}
#[test]
fn test_is_basic_latin_c0() {
fn test_simd_is_basic_latin_c0() {
let input: [u16; 16] = [0x61, 0x62, 0x63, 0x81, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70, 0x71,
0x72, 0x73, 0x74, 0x75, 0x76];
let first = unsafe { load8_unaligned(input.as_ptr()) };
let second = unsafe { load8_unaligned(input.as_ptr().offset(8)) };
assert!(!is_basic_latin(first | second));
assert!(!simd_is_basic_latin(first | second));
}
#[test]
fn test_is_basic_latin_0fff() {
fn test_simd_is_basic_latin_0fff() {
let input: [u16; 16] = [0x61, 0x62, 0x63, 0x0FFF, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70,
0x71, 0x72, 0x73, 0x74, 0x75, 0x76];
let first = unsafe { load8_unaligned(input.as_ptr()) };
let second = unsafe { load8_unaligned(input.as_ptr().offset(8)) };
assert!(!is_basic_latin(first | second));
assert!(!simd_is_basic_latin(first | second));
}
#[test]
fn test_is_basic_latin_ffff() {
fn test_simd_is_basic_latin_ffff() {
let input: [u16; 16] = [0x61, 0x62, 0x63, 0xFFFF, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70,
0x71, 0x72, 0x73, 0x74, 0x75, 0x76];
let first = unsafe { load8_unaligned(input.as_ptr()) };
let second = unsafe { load8_unaligned(input.as_ptr().offset(8)) };
assert!(!is_basic_latin(first | second));
assert!(!simd_is_basic_latin(first | second));
}
#[test]
fn test_is_ascii_success() {
fn test_simd_is_ascii_success() {
let ascii: [u8; 16] = [0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70, 0x71,
0x72, 0x73, 0x74, 0x75, 0x76];
let simd = unsafe { load16_unaligned(ascii.as_ptr()) };
assert!(is_ascii(simd));
assert!(simd_is_ascii(simd));
}
#[test]
fn test_is_ascii_failure() {
fn test_simd_is_ascii_failure() {
let input: [u8; 16] = [0x61, 0x62, 0x63, 0x64, 0x81, 0x66, 0x67, 0x68, 0x69, 0x70, 0x71,
0x72, 0x73, 0x74, 0x75, 0x76];
let simd = unsafe { load16_unaligned(input.as_ptr()) };
assert!(!is_ascii(simd));
assert!(!simd_is_ascii(simd));
}
#[cfg(target_feature = "sse2")]

129
third_party/rust/encoding_rs/src/testing.rs vendored
View File

@ -22,13 +22,20 @@ pub fn decode(encoding: &'static Encoding, bytes: &[u8], expect: &str) {
}
vec.extend_from_slice(bytes);
string.push_str(expect);
decode_without_padding(encoding, &vec[..], &string[..]);
decode_without_padding_impl(encoding, &vec[..], &string[..], i);
}
}
pub fn decode_without_padding(encoding: &'static Encoding, bytes: &[u8], expect: &str) {
decode_to_utf8(encoding, bytes, expect);
decode_to_utf16(encoding, bytes, &utf16_from_utf8(expect)[..]);
decode_without_padding_impl(encoding, bytes, expect, 0);
}
fn decode_without_padding_impl(encoding: &'static Encoding,
bytes: &[u8],
expect: &str,
padding: usize) {
decode_to_utf8_impl(encoding, bytes, expect, padding);
decode_to_utf16_impl(encoding, bytes, &utf16_from_utf8(expect)[..], padding);
decode_to_string(encoding, bytes, expect);
}
@ -56,40 +63,116 @@ pub fn encode_without_padding(encoding: &'static Encoding, string: &str, expect:
}
pub fn decode_to_utf16(encoding: &'static Encoding, bytes: &[u8], expect: &[u16]) {
decode_to_utf16_impl(encoding, bytes, expect, 0);
}
pub fn decode_to_utf16_impl(encoding: &'static Encoding,
bytes: &[u8],
expect: &[u16],
padding: usize) {
for i in padding..bytes.len() {
let (head, tail) = bytes.split_at(i);
decode_to_utf16_with_boundary(encoding, head, tail, expect);
}
}
pub fn decode_to_utf16_with_boundary(encoding: &'static Encoding,
head: &[u8],
tail: &[u8],
expect: &[u16]) {
let mut decoder = encoding.new_decoder();
let mut dest: Vec<u16> =
Vec::with_capacity(decoder.max_utf16_buffer_length(bytes.len()).unwrap());
let mut dest: Vec<u16> = Vec::with_capacity(
decoder
.max_utf16_buffer_length(head.len() + tail.len())
.unwrap()
);
let capacity = dest.capacity();
dest.resize(capacity, 0u16);
let (complete, read, written, _) = decoder.decode_to_utf16(bytes, &mut dest, true);
match complete {
CoderResult::InputEmpty => {}
CoderResult::OutputFull => {
unreachable!();
let mut total_read = 0;
let mut total_written = 0;
{
let (complete, read, written, _) = decoder.decode_to_utf16(head, &mut dest, false);
match complete {
CoderResult::InputEmpty => {}
CoderResult::OutputFull => {
unreachable!();
}
}
total_read += read;
total_written += written;
}
assert_eq!(read, bytes.len());
assert_eq!(written, expect.len());
dest.truncate(written);
{
let (complete, read, written, _) =
decoder.decode_to_utf16(tail, &mut dest[total_written..], true);
match complete {
CoderResult::InputEmpty => {}
CoderResult::OutputFull => {
unreachable!();
}
}
total_read += read;
total_written += written;
}
assert_eq!(total_read, head.len() + tail.len());
assert_eq!(total_written, expect.len());
dest.truncate(total_written);
assert_eq!(&dest[..], expect);
}
pub fn decode_to_utf8(encoding: &'static Encoding, bytes: &[u8], expect: &str) {
decode_to_utf8_impl(encoding, bytes, expect, 0);
}
pub fn decode_to_utf8_impl(encoding: &'static Encoding,
bytes: &[u8],
expect: &str,
padding: usize) {
for i in padding..bytes.len() {
let (head, tail) = bytes.split_at(i);
decode_to_utf8_with_boundary(encoding, head, tail, expect);
}
}
pub fn decode_to_utf8_with_boundary(encoding: &'static Encoding,
head: &[u8],
tail: &[u8],
expect: &str) {
let mut decoder = encoding.new_decoder();
let mut dest: Vec<u8> =
Vec::with_capacity(decoder.max_utf8_buffer_length(bytes.len()).unwrap());
let mut dest: Vec<u8> = Vec::with_capacity(
decoder
.max_utf8_buffer_length(head.len() + tail.len())
.unwrap()
);
let capacity = dest.capacity();
dest.resize(capacity, 0u8);
let (complete, read, written, _) = decoder.decode_to_utf8(bytes, &mut dest, true);
match complete {
CoderResult::InputEmpty => {}
CoderResult::OutputFull => {
unreachable!();
let mut total_read = 0;
let mut total_written = 0;
{
let (complete, read, written, _) = decoder.decode_to_utf8(head, &mut dest, false);
match complete {
CoderResult::InputEmpty => {}
CoderResult::OutputFull => {
unreachable!();
}
}
total_read += read;
total_written += written;
}
assert_eq!(read, bytes.len());
assert_eq!(written, expect.len());
dest.truncate(written);
{
let (complete, read, written, _) =
decoder.decode_to_utf8(tail, &mut dest[total_written..], true);
match complete {
CoderResult::InputEmpty => {}
CoderResult::OutputFull => {
unreachable!();
}
}
total_read += read;
total_written += written;
}
assert_eq!(total_read, head.len() + tail.len());
assert_eq!(total_written, expect.len());
dest.truncate(total_written);
assert_eq!(&dest[..], expect.as_bytes());
}

32
third_party/rust/encoding_rs/src/utf_8.rs vendored
View File

@ -34,21 +34,21 @@ cfg_if! {
}
}
const UTF8_NORMAL_TRAIL: u8 = 1 << 3;
pub const UTF8_NORMAL_TRAIL: u8 = 1 << 3;
const UTF8_THREE_BYTE_SPECIAL_LOWER_BOUND_TRAIL: u8 = 1 << 4;
pub const UTF8_THREE_BYTE_SPECIAL_LOWER_BOUND_TRAIL: u8 = 1 << 4;
const UTF8_THREE_BYTE_SPECIAL_UPPER_BOUND_TRAIL: u8 = 1 << 5;
pub const UTF8_THREE_BYTE_SPECIAL_UPPER_BOUND_TRAIL: u8 = 1 << 5;
const UTF8_FOUR_BYTE_SPECIAL_LOWER_BOUND_TRAIL: u8 = 1 << 6;
pub const UTF8_FOUR_BYTE_SPECIAL_LOWER_BOUND_TRAIL: u8 = 1 << 6;
const UTF8_FOUR_BYTE_SPECIAL_UPPER_BOUND_TRAIL: u8 = 1 << 7;
pub const UTF8_FOUR_BYTE_SPECIAL_UPPER_BOUND_TRAIL: u8 = 1 << 7;
// BEGIN GENERATED CODE. PLEASE DO NOT EDIT.
// Instead, please regenerate using generate-encoding-data.py
/// Bit is 1 if the trail is invalid.
static UTF8_TRAIL_INVALID: [u8; 256] =
pub static UTF8_TRAIL_INVALID: [u8; 256] =
[248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
@ -433,16 +433,18 @@ pub struct Utf8Decoder {
}
impl Utf8Decoder {
pub fn new_inner() -> Utf8Decoder {
Utf8Decoder {
code_point: 0,
bytes_seen: 0,
bytes_needed: 0,
lower_boundary: 0x80u8,
upper_boundary: 0xBFu8,
}
}
pub fn new() -> VariantDecoder {
VariantDecoder::Utf8(
Utf8Decoder {
code_point: 0,
bytes_seen: 0,
bytes_needed: 0,
lower_boundary: 0x80u8,
upper_boundary: 0xBFu8,
}
)
VariantDecoder::Utf8(Utf8Decoder::new_inner())
}
fn extra_from_state(&self) -> usize {

2
third_party/rust/simd/.cargo-checksum.json vendored
View File

@ -1 +1 @@
{"files":{".travis.yml":"e2c720c3633b7671efce49147c62b12bcbf630d7c5d6fc65cd97620bfa4ddcea","Cargo.toml":"608aad04f17a524ee21048fa2ce9f656ae344e0473dd0e331dc954f0f9677c63","LICENSE-APACHE":"a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2","LICENSE-MIT":"6d3a9431e65e69c73a8923e6517b889d17549b23db406b9ec027710d16af701f","README.md":"249294a9a5f63c64c0f7fe4a607060f43f3507dce2378067aa59d25fb3ae681d","benches/mandelbrot.rs":"051b5199e66bca6cf7774e9024915fd4e1349ab39726a10a14e06b60d65d87a4","benches/matrix.rs":"048a21dacdb62365e0105d00d2c8cd6bd2396ac81134f2bff7eb4f7d095fb735","examples/axpy.rs":"4307626045d64ec08361c97c9c72c5dc8d361bdc88f64453b97ac0212041a1b2","examples/convert.rs":"8e658fde050f8a0d8b84ad7570446b10fcf544afbd551b940ca340474f324840","examples/dot-product.rs":"6fe2e007c147af5353804173a593c5b9d57dbccec156e1da37e9e32537363f91","examples/fannkuch-redux-nosimd.rs":"7b2fbde35e8666929d14d67328471cb0483d038a5325232f8db148b30865312b","examples/fannkuch-redux.rs":"ea21fdbd2274488a62cc984acad6e0b65d52f24fb4ff63b7057a3a667e9c8aae","examples/mandelbrot.rs":"8b8fdca1edac50e5a33e0e0592bd41eb75114f31839ccd40d485c61a9a664380","examples/matrix-inverse.rs":"a378d20ef20c2119bb10a86de27c92fec2c2f77f374e6bfd36707c9825a5fe92","examples/nbody-nosimd.rs":"2c8e0a7feacd202fdd65eeceb6420d6e9f43340b81f20a8e532704a587a2796b","examples/nbody.rs":"a864311affab262024479d6348ff51af43d809e9ad332ec30ea4aacceaa2eae1","examples/ops.rs":"1316f915d0afcfa98fdc4077e965ccccf6b4b21c433cbe487ff0cdc60df3cd39","examples/spectral-norm-nosimd.rs":"ffc8512ecde779078ea467f38f423a0ea623c63da7078193f9dd370200773f79","examples/spectral-norm.rs":"edb09c9d477f83939098cfb77a27cc298bc7a0c8a8e29cece0cccae0d70d890e","src/aarch64/mod.rs":"83f52775364c98de0cecb7e1509530c18972e932469f5f1522aa24a735d0fa37","src/aarch64/neon.rs":"1fe769979e07d8e2bc3c78ce116e05d735860744efe097a894cc9421153257fb","src/arm/mod.rs":"dcdd90bc0b39abaf86a0c8946d442b16313563fbae1ff03248628275c74d8617","src/arm/neon.rs":"51cc509856200e80f8e4cc2c982586e6d1cef593ec4537e153dce0cfe31d3428","src/common.rs":"62f4e7e0fefb52ad190d0f2191bc435ac4deab3f2bc70dc427f2a7f9ccb7856e","src/lib.rs":"25f0b39c038fa85af858318135dfd87865be26c33bb4bd1438aec96a1e68d8b5","src/sixty_four.rs":"510a9e00189a61e4f0a5beb7052d5dee37fc8261f94a2af45ef10327e0f3b7df","src/v256.rs":"2e328e49034876d535e0627c7a62191da2b4fb156a657614bf531a5fc75b1385","src/x86/avx.rs":"c66140abefca634b48eae307c3ec8cf5a40f2279b10e246a7e2ac602a2a2bb28","src/x86/avx2.rs":"efe3006b13a13261a3dec3d37dc1d8cb53950f3803c420069231803374949937","src/x86/mod.rs":"0acc5a5e2672e2a0fddc11065663be8b8fa2da87320ea291fa86ff8c2f33edf5","src/x86/sse2.rs":"5ceda75a401958a135fc9d851b22075314cdeed69fd483b6a7be4f11373f40da","src/x86/sse3.rs":"9bd01a4f08069ca4f445952e744d651efe887e3835b18872e757375f0d053bd2","src/x86/sse4_1.rs":"9ceb80dd70a7e7dfeef508cb935e1a2637175bc87a3b090f5dea691ff6aa0516","src/x86/sse4_2.rs":"c59321aed8decdce4d0d8570cff46aed02e1a8265647ef7702e9b180fc581254","src/x86/ssse3.rs":"2290f0269bae316b8e0491495645ee38a9bd73525c8572759c1328341c3bdb4c"},"package":"7a94d14a2ae1f1f110937de5fb69e494372560181c7e1739a097fcc2cee37ba0"}
{"files":{".travis.yml":"e2c720c3633b7671efce49147c62b12bcbf630d7c5d6fc65cd97620bfa4ddcea","Cargo.toml":"27c6a208f0c6253c4580508311d49bb421944abd272a7f9a5a38b51ef657aec2","LICENSE-APACHE":"a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2","LICENSE-MIT":"6d3a9431e65e69c73a8923e6517b889d17549b23db406b9ec027710d16af701f","README.md":"249294a9a5f63c64c0f7fe4a607060f43f3507dce2378067aa59d25fb3ae681d","benches/mandelbrot.rs":"051b5199e66bca6cf7774e9024915fd4e1349ab39726a10a14e06b60d65d87a4","benches/matrix.rs":"048a21dacdb62365e0105d00d2c8cd6bd2396ac81134f2bff7eb4f7d095fb735","examples/axpy.rs":"4307626045d64ec08361c97c9c72c5dc8d361bdc88f64453b97ac0212041a1b2","examples/convert.rs":"8e658fde050f8a0d8b84ad7570446b10fcf544afbd551b940ca340474f324840","examples/dot-product.rs":"6fe2e007c147af5353804173a593c5b9d57dbccec156e1da37e9e32537363f91","examples/fannkuch-redux-nosimd.rs":"7b2fbde35e8666929d14d67328471cb0483d038a5325232f8db148b30865312b","examples/fannkuch-redux.rs":"ea21fdbd2274488a62cc984acad6e0b65d52f24fb4ff63b7057a3a667e9c8aae","examples/mandelbrot.rs":"71be242543c1e487145d7f16341c05d05d86109de4d9e94c5d6bc9a9c6ed9766","examples/matrix-inverse.rs":"93dbc55c66a72e5f7bc730072f35682523fa20dd362755d8443ad6982143cb5d","examples/nbody-nosimd.rs":"9cf46ea02e266c20f811318f1c5856d5afb9575b2d48d552fbd978f5c1856bdb","examples/nbody.rs":"a864311affab262024479d6348ff51af43d809e9ad332ec30ea4aacceaa2eae1","examples/ops.rs":"b08ea83583df71d0052895d677320a9888da5b6729c9b70636d31ede5128bb7f","examples/spectral-norm-nosimd.rs":"ffc8512ecde779078ea467f38f423a0ea623c63da7078193f9dd370200773f79","examples/spectral-norm.rs":"edb09c9d477f83939098cfb77a27cc298bc7a0c8a8e29cece0cccae0d70d890e","src/aarch64/mod.rs":"83f52775364c98de0cecb7e1509530c18972e932469f5f1522aa24a735d0fa37","src/aarch64/neon.rs":"3c05ea43b7261b9af9c0d904b37de01c2ba99caedcb464700f16617b672965a1","src/arm/mod.rs":"dcdd90bc0b39abaf86a0c8946d442b16313563fbae1ff03248628275c74d8617","src/arm/neon.rs":"00aed2c94455b7ff5755b7598fb166a94c7242ad9adf4e5379560ab04af560e7","src/common.rs":"c5a7b937c5cd8c3bccf0fb20d5d77770c0d9b0dd9fa06a661c6f2ddf118e65c0","src/lib.rs":"08c345b6a2ad641daa3c1a40b1dcc6e4f9047939414bd81b05051fc74a563fec","src/sixty_four.rs":"d168776d02acf943bda8044b24e644b7a9584197a223eba1a7c3024b205dc87d","src/v256.rs":"34bfde3676e23f6925db5d0408ae838e3aab7706128fd7c33e855b8579c69318","src/x86/avx.rs":"efcf2120a904a89b0adf2d3d3bdd0ca17df2ec058410af23fb7e81915873f808","src/x86/avx2.rs":"3bcb3f391ad5f16f0a6da0bc1301329beb478ad6265bd3b2c9c124fc2e6198e5","src/x86/mod.rs":"0acc5a5e2672e2a0fddc11065663be8b8fa2da87320ea291fa86ff8c2f33edf5","src/x86/sse2.rs":"8807fb04bbfb404e17fcacf1e21d22616f8b377540a227b1fd03c121879122dd","src/x86/sse3.rs":"9bd01a4f08069ca4f445952e744d651efe887e3835b18872e757375f0d053bd2","src/x86/sse4_1.rs":"9ceb80dd70a7e7dfeef508cb935e1a2637175bc87a3b090f5dea691ff6aa0516","src/x86/sse4_2.rs":"c59321aed8decdce4d0d8570cff46aed02e1a8265647ef7702e9b180fc581254","src/x86/ssse3.rs":"2290f0269bae316b8e0491495645ee38a9bd73525c8572759c1328341c3bdb4c"},"package":"3dd0805c7363ab51a829a1511ad24b6ed0349feaa756c4bc2f977f9f496e6673"}

43
third_party/rust/simd/Cargo.toml vendored
View File

@ -1,25 +1,36 @@
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g. crates.io) dependencies
#
# If you believe there's an error in this file please file an
# issue against the rust-lang/cargo repository. If you're
# editing this file be aware that the upstream Cargo.toml
# will likely look very different (and much more reasonable)
[package]
name = "simd"
version = "0.2.0"
version = "0.2.1"
authors = ["Huon Wilson <dbau.pp+github@gmail.com>"]
repository = "https://github.com/rust-lang-nursery/simd"
description = "`simd` offers limited cross-platform access to SIMD instructions on\nCPUs, as well as raw interfaces to platform-specific instructions.\n"
documentation = "https://rust-lang-nursery.github.io/simd/doc/simd"
license = "MIT/Apache-2.0"
keywords = ["simd", "data-parallel"]
readme = "README.md"
keywords = ["simd", "data-parallel"]
license = "MIT/Apache-2.0"
repository = "https://github.com/rust-lang-nursery/simd"
[package.metadata.docs.rs]
features = ["doc"]
[dependencies.serde]
version = "1.0"
optional = true
description = """
`simd` offers limited cross-platform access to SIMD instructions on
CPUs, as well as raw interfaces to platform-specific instructions.
"""
[dependencies]
serde = { version = "0.8", optional = true }
serde_derive = { version = "0.8", optional = true }
[dev-dependencies]
cfg-if = "0.1"
[dependencies.serde_derive]
version = "1.0"
optional = true
[dev-dependencies.cfg-if]
version = "0.1"
[features]
doc = []

2
third_party/rust/simd/examples/mandelbrot.rs vendored
View File

@ -1,4 +1,4 @@
#![feature(step_by, test)]
#![feature(iterator_step_by, test)]
extern crate test;
extern crate simd;

1
third_party/rust/simd/examples/matrix-inverse.rs vendored
View File

@ -25,6 +25,7 @@ fn mul(x: &[f32x4; 4], y: &[f32x4; 4]) -> [f32x4; 4] {
]
}
#[allow(dead_code)]
fn inverse_naive(x: &[[f32; 4]; 4]) -> [[f32; 4]; 4] {
let mut t = [[0_f32; 4]; 4];
for i in 0..4 {

2
third_party/rust/simd/examples/nbody-nosimd.rs vendored
View File

@ -66,7 +66,7 @@ struct Planet {
}
fn advance(bodies: &mut [Planet;N_BODIES], dt: f64, steps: i32) {
for _ in (0..steps) {
for _ in 0..steps {
let mut b_slice: &mut [_] = bodies;
loop {
let bi = match shift_mut_ref(&mut b_slice) {

1
third_party/rust/simd/examples/ops.rs vendored
View File

@ -2,6 +2,7 @@ extern crate simd;
use simd::*;
#[allow(unused_variables)]
fn main() {
let x = i32x4::splat(1_i32);
let y = -x;

2
third_party/rust/simd/src/aarch64/neon.rs vendored
View File

@ -630,7 +630,7 @@ impl Aarch64I8x16 for i8x16 {
#[doc(hidden)]
pub mod common {
use super::super::super::*;
use std::mem;
use core::mem;
#[inline]
pub fn f32x4_sqrt(x: f32x4) -> f32x4 {

2
third_party/rust/simd/src/arm/neon.rs vendored
View File

@ -473,7 +473,7 @@ impl u8x8 {
pub mod common {
use super::super::super::*;
use super::*;
use std::mem;
use core::mem;
#[inline]
pub fn f32x4_sqrt(x: f32x4) -> f32x4 {

3
third_party/rust/simd/src/common.rs vendored
View File

@ -9,8 +9,7 @@ use super::{
Unalign, bitcast,
};
use std::mem;
use std::ops;
use core::{mem,ops};
#[cfg(any(target_arch = "x86",
target_arch = "x86_64"))]

17
third_party/rust/simd/src/lib.rs vendored
View File

@ -1,4 +1,5 @@
//! `simd` offers a basic interface to the SIMD functionality of CPUs.
#![no_std]
#![feature(cfg_target_feature, repr_simd, platform_intrinsics, const_fn)]
#![allow(non_camel_case_types)]
@ -9,6 +10,8 @@ extern crate serde;
#[macro_use]
extern crate serde_derive;
use core::mem;
/// Boolean type for 8-bit integers.
#[cfg_attr(feature = "with-serde", derive(Serialize, Deserialize))]
#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
@ -172,9 +175,9 @@ simd! {
#[allow(dead_code)]
#[inline]
fn bitcast<T: Simd, U: Simd>(x: T) -> U {
assert_eq!(std::mem::size_of::<T>(),
std::mem::size_of::<U>());
unsafe {std::mem::transmute_copy(&x)}
assert_eq!(mem::size_of::<T>(),
mem::size_of::<U>());
unsafe {mem::transmute_copy(&x)}
}
#[allow(dead_code)]
@ -207,9 +210,15 @@ extern "platform-intrinsic" {
fn simd_xor<T: Simd>(x: T, y: T) -> T;
}
#[repr(packed)]
#[derive(Debug, Copy, Clone)]
#[derive(Copy)]
struct Unalign<T>(T);
impl<T: Clone> Clone for Unalign<T> {
fn clone(&self) -> Unalign<T> {
Unalign(unsafe { self.0.clone() })
}
}
#[macro_use]
mod common;
mod sixty_four;

3
third_party/rust/simd/src/sixty_four.rs vendored
View File

@ -11,8 +11,7 @@ use super::{
Unalign, bitcast,
};
use std::mem;
use std::ops;
use core::{mem,ops};
/// Boolean type for 64-bit integers.
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]

16
third_party/rust/simd/src/v256.rs vendored
View File

@ -1,6 +1,5 @@
#![allow(dead_code)]
use std::ops;
use std::mem;
use core::{mem,ops};
#[allow(unused_imports)]
use super::{
Simd,
@ -329,6 +328,19 @@ impl i32x8 {
}
}
impl f32x8 {
/// Convert each lane to a signed integer.
#[inline]
pub fn to_i32(self) -> i32x8 {
unsafe {simd_cast(self)}
}
/// Convert each lane to an unsigned integer.
#[inline]
pub fn to_u32(self) -> u32x8 {
unsafe {simd_cast(self)}
}
}
impl i16x16 {
/// Convert each lane to an unsigned integer.
#[inline]

2
third_party/rust/simd/src/x86/avx.rs vendored
View File

@ -54,7 +54,7 @@ extern "platform-intrinsic" {
#[doc(hidden)]
pub mod common {
use super::*;
use std::mem;
use core::mem;
macro_rules! bools {
($($ty: ty, $all: ident, $any: ident, $testc: ident, $testz: ident;)*) => {

2
third_party/rust/simd/src/x86/avx2.rs vendored
View File

@ -42,7 +42,7 @@ extern "platform-intrinsic" {
fn x86_mm256_packus_epi32(x: i32x8, y: i32x8) -> u16x16;
fn x86_mm256_permutevar8x32_epi32(x: i32x8, y: i32x8) -> i32x8;
fn x86_mm256_permutevar8x32_ps(x: f32x8, y: i32x8) -> f32x8;
fn x86_mm256_sad_epu8(x: u8x32, y: u8x32) -> u8x32;
fn x86_mm256_sad_epu8(x: u8x32, y: u8x32) -> u64x4;
fn x86_mm256_shuffle_epi8(x: i8x32, y: i8x32) -> i8x32;
fn x86_mm256_sign_epi8(x: i8x32, y: i8x32) -> i8x32;
fn x86_mm256_sign_epi16(x: i16x16, y: i16x16) -> i16x16;

2
third_party/rust/simd/src/x86/sse2.rs vendored
View File

@ -48,7 +48,7 @@ extern "platform-intrinsic" {
#[doc(hidden)]
pub mod common {
use super::super::super::*;
use std::mem;
use core::mem;
#[inline]
pub fn f32x4_sqrt(x: f32x4) -> f32x4 {

14
toolkit/library/gtest/rust/Cargo.lock generated
View File

@ -420,25 +420,25 @@ name = "encoding_c"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
"encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "encoding_glue"
version = "0.1.0"
dependencies = [
"encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
"encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
"nserror 0.1.0",
"nsstring 0.1.0",
]
[[package]]
name = "encoding_rs"
version = "0.7.1"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cfg-if 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"simd 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
"simd 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
@ -1208,7 +1208,7 @@ dependencies = [
[[package]]
name = "simd"
version = "0.2.0"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
@ -1659,7 +1659,7 @@ dependencies = [
"checksum dwrote 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "a207eb7b40e25d1d28dc679f451d321fb6954b73ceaa47986702575865469461"
"checksum either 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "18785c1ba806c258137c937e44ada9ee7e69a37e3c72077542cd2f069d78562a"
"checksum encoding_c 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "93ec52324ca72f423237a413ca0e1c60654c8b3d0934fcd5fd888508dfcc4ba7"
"checksum encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f5215aabf22b83153be3ee44dfe3f940214541b2ce13d419c55e7a115c8c51a9"
"checksum encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "98fd0f24d1fb71a4a6b9330c8ca04cbd4e7cc5d846b54ca74ff376bc7c9f798d"
"checksum env_logger 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3ddf21e73e016298f5cb37d6ef8e8da8e39f91f9ec8b0df44b7deb16a9f8cd5b"
"checksum error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ff511d5dc435d703f4971bc399647c9bc38e20cb41452e3b9feb4765419ed3f3"
"checksum euclid 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)" = "926c639bfdff1f3063f76bb66245f6d2b691aa20fdbaabecc38b2947a13a4eba"
@ -1734,7 +1734,7 @@ dependencies = [
"checksum serde 1.0.27 (registry+https://github.com/rust-lang/crates.io-index)" = "db99f3919e20faa51bb2996057f5031d8685019b5a06139b1ce761da671b8526"
"checksum serde_derive 1.0.27 (git+https://github.com/gankro/serde?branch=deserialize_from_enums4)" = "<none>"
"checksum serde_derive_internals 0.19.0 (git+https://github.com/gankro/serde?branch=deserialize_from_enums4)" = "<none>"
"checksum simd 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a94d14a2ae1f1f110937de5fb69e494372560181c7e1739a097fcc2cee37ba0"
"checksum simd 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3dd0805c7363ab51a829a1511ad24b6ed0349feaa756c4bc2f977f9f496e6673"
"checksum siphasher 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "2ffc669b726f2bc9a3bcff66e5e23b56ba6bf70e22a34c3d7b6d0b3450b65b84"
"checksum slab 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "17b4fcaed89ab08ef143da37bc52adbcc04d4a69014f4c1208d6b51f0c47bc23"
"checksum smallbitvec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "79b776f00dfe01df905fa3b2eaa1659522e99e3fc4a7b1334171622205c4bdcf"

14
toolkit/library/rust/Cargo.lock generated
View File

@ -420,25 +420,25 @@ name = "encoding_c"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
"encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "encoding_glue"
version = "0.1.0"
dependencies = [
"encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
"encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
"nserror 0.1.0",
"nsstring 0.1.0",
]
[[package]]
name = "encoding_rs"
version = "0.7.1"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cfg-if 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"simd 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
"simd 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
@ -1196,7 +1196,7 @@ dependencies = [
[[package]]
name = "simd"
version = "0.2.0"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
@ -1671,7 +1671,7 @@ dependencies = [
"checksum dwrote 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "a207eb7b40e25d1d28dc679f451d321fb6954b73ceaa47986702575865469461"
"checksum either 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "18785c1ba806c258137c937e44ada9ee7e69a37e3c72077542cd2f069d78562a"
"checksum encoding_c 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "93ec52324ca72f423237a413ca0e1c60654c8b3d0934fcd5fd888508dfcc4ba7"
"checksum encoding_rs 0.7.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f5215aabf22b83153be3ee44dfe3f940214541b2ce13d419c55e7a115c8c51a9"
"checksum encoding_rs 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "98fd0f24d1fb71a4a6b9330c8ca04cbd4e7cc5d846b54ca74ff376bc7c9f798d"
"checksum env_logger 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3ddf21e73e016298f5cb37d6ef8e8da8e39f91f9ec8b0df44b7deb16a9f8cd5b"
"checksum error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ff511d5dc435d703f4971bc399647c9bc38e20cb41452e3b9feb4765419ed3f3"
"checksum euclid 0.16.0 (registry+https://github.com/rust-lang/crates.io-index)" = "926c639bfdff1f3063f76bb66245f6d2b691aa20fdbaabecc38b2947a13a4eba"
@ -1746,7 +1746,7 @@ dependencies = [
"checksum serde 1.0.27 (registry+https://github.com/rust-lang/crates.io-index)" = "db99f3919e20faa51bb2996057f5031d8685019b5a06139b1ce761da671b8526"
"checksum serde_derive 1.0.27 (git+https://github.com/gankro/serde?branch=deserialize_from_enums4)" = "<none>"
"checksum serde_derive_internals 0.19.0 (git+https://github.com/gankro/serde?branch=deserialize_from_enums4)" = "<none>"
"checksum simd 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a94d14a2ae1f1f110937de5fb69e494372560181c7e1739a097fcc2cee37ba0"
"checksum simd 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3dd0805c7363ab51a829a1511ad24b6ed0349feaa756c4bc2f977f9f496e6673"
"checksum siphasher 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "2ffc669b726f2bc9a3bcff66e5e23b56ba6bf70e22a34c3d7b6d0b3450b65b84"
"checksum slab 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "17b4fcaed89ab08ef143da37bc52adbcc04d4a69014f4c1208d6b51f0c47bc23"
"checksum smallbitvec 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "79b776f00dfe01df905fa3b2eaa1659522e99e3fc4a7b1334171622205c4bdcf"