Commit Graph

115 Commits

Author SHA1 Message Date
Ralf Jung 9eb726fd6d adjust code generation to make tests bail out early in Miri 2020-08-26 13:01:21 +03:00
Henri Sivonen adcb9b3004 Remove year from copyright notices. 2020-05-13 09:47:28 +03:00
Henri Sivonen c3eb642cdf Make semantics of latin1_byte_compatible_up_to actually useful. 2019-09-16 14:25:25 +03:00
Henri Sivonen 1dfeac8edc Add Decoder::latin1_byte_compatible_up_to. 2019-09-05 10:29:11 +03:00
Henri Sivonen 11b0b46b3c Add code generation for the codepage crate. 2018-12-02 17:04:11 +02:00
Yusuke Sasaki 84465f3769 remove unstable scoped lint 2018-11-17 20:12:19 +02:00
Henri Sivonen 0b15425487 Merge UTF-8 data arrays. 2018-11-02 12:06:02 +02:00
Henri Sivonen 6206d92ba6 Ensure UTF-8 data is aligned to cache lines. 2018-11-02 11:55:02 +02:00
Henri Sivonen 13e6941022 Clean up constants and mem. 2018-11-02 11:46:34 +02:00
Henri Sivonen fe9d54e440 Avoid table lookup for the last trail in three-byte UTF-8 sequences. 2018-11-02 10:49:10 +02:00
Henri Sivonen d2829e270d Use UTF-8 to UTF-16 code structure for UTF-8 validation. 2018-11-01 17:06:32 +02:00
Henri Sivonen 601ed56fbe Read UTF-8 second byte range mask from a table. 2018-11-01 14:43:23 +02:00
Henri Sivonen b87f0e49b7 Address clippy lints. 2018-10-17 14:28:16 +03:00
Henri Sivonen f1d04bde7a Add method is_single_byte() on Encoding.
Closes #12.
2018-10-16 15:57:14 +03:00
Henri Sivonen 29d1cae317 Document the prerequisites for running the code generation script.
Closes #14.
2018-10-16 13:05:13 +03:00
Henri Sivonen 77dab08378 Add option for fast-hanja-encode. 2018-10-08 18:38:56 +03:00
Henri Sivonen 79532d0f3c Avoid storing non-generated parts of data.rs inside the Python script. 2018-10-08 17:47:01 +03:00
Henri Sivonen bc369a0b00 Add fast-big5-hanzi-encode compile-time option. 2018-10-08 17:40:42 +03:00
Henri Sivonen ae7235fa1a Add data table for fast Big5 Unified Ideograph encode. 2018-10-04 14:11:47 +03:00
Henri Sivonen eb3ec27f06 Implement fast-gb-hanzi-encode. 2018-10-04 14:11:47 +03:00
Henri Sivonen 0e851eb628 Make fast-kanji-encode additive over less-slow-kanji-encode without warnings. 2018-10-04 14:11:47 +03:00
Henri Sivonen e9683096e2 Make fast-kanji-encode faster for EUC-JP. 2018-10-04 14:11:47 +03:00
Henri Sivonen bc4cc15253 Non-optimal EUC-JP support for fast-kanji-encode. 2018-10-04 14:11:47 +03:00
Henri Sivonen 8b84c6f1df Add fast-kanji-encode for Shift_JIS. 2018-10-04 14:11:47 +03:00
Henri Sivonen beaff16757 Add cargo feature fast-hangul-encode. 2018-10-04 14:11:47 +03:00
Henri Sivonen 0498cb67dd Organize single-byte data into a 64-byte-aligned struct. 2018-10-04 13:49:28 +03:00
Henri Sivonen 494614253b Generate longest run data. 2018-10-04 13:49:28 +03:00
Henri Sivonen 4377ef10c9 Make single-byte encodings have info about longest run of consecutive Unicode code points. 2018-10-04 13:49:28 +03:00
Henri Sivonen e5256a124d Flip size vs. speed default around for Kanji and Hanzi legacy encode.
This is in preparation of adding actually fast options in a way
that allows cargo features to be additive.
2018-05-29 15:40:41 +03:00
Henri Sivonen 35c6e946fe Add docs for the individual encodings. 2018-05-23 15:44:18 +03:00
Henri Sivonen 06caa06e60 Add a "pub " in generate-encoding-data.py. 2018-05-11 11:41:17 +03:00
Henri Sivonen f309cad1e7 Remove for_name-related code from generate-encoding-data.py. 2017-08-25 10:56:51 +03:00
Henri Sivonen 58a2cbe3c2 Rename not-null macro. 2017-06-21 16:18:31 +03:00
Henri Sivonen bf4a8f8b48 Macroize the type for static encoding pointers in C header. 2017-06-21 09:24:58 +03:00
Henri Sivonen 59bc64306f Use preprocessor define in encoding constant declarations in encoding_rs_statics.h. 2017-05-29 14:34:16 +03:00
Henri Sivonen cee5669748 Allow earlier preprocessor directives to override types in encoding_rs_statics.h. 2017-05-29 14:02:42 +03:00
Henri Sivonen 03376fef97 Map half-width katakana to full-width in ISO-2022-JP encoder per whatwg/encoding#105. 2017-05-08 11:54:57 +03:00
Henri Sivonen bf78f047ae Give InputEmpty correct precedent when encoding with replacement. 2017-05-08 10:33:20 +03:00
Henri Sivonen 7be005fc63 Address Clippy lints. 2017-04-25 10:45:01 +03:00
Henri Sivonen daed296e9e Check for integer overflow in max size computation. 2017-04-24 14:18:35 +03:00
Henri Sivonen 8b1373b53e Make label and name resolution faster and more correct. 2017-02-01 11:22:19 +02:00
Henri Sivonen b47100e357 Remove useless declaration. 2017-02-01 09:54:08 +02:00
Henri Sivonen 38e922ba26 Fix compilation with --features no-static-ideograph-encoder-tables. 2017-01-27 12:04:00 +02:00
Henri Sivonen 0a4726d6c8 Accelerate Big5 Level 1 Hanzi encode. 2017-01-27 11:06:55 +02:00
Henri Sivonen a652977446 Refactor Big5 BMP encode. 2017-01-27 10:50:37 +02:00
Henri Sivonen 683eb5aa53 Optimize Big5 astral encode. 2017-01-27 09:42:56 +02:00
Henri Sivonen e7e6d2f7d8 Clean up Big5 table generation. 2017-01-27 09:11:17 +02:00
Henri Sivonen 4242f23779 Accelerate jis0208 Level 1 Kanji in ISO-2022-JP encode. 2017-01-25 14:41:06 +02:00
Henri Sivonen f3a81d28f1 Accelerate jis0208 Level 1 Kanji in EUC-JP encode. 2017-01-25 14:31:50 +02:00
Henri Sivonen 9abaef662a Accelerate jis0208 Level 1 Kanji in Shift_JIS encode. 2017-01-25 13:55:11 +02:00