9 Commits

Author SHA1 Message Date
Andrew Gallant
6ff285e375 tests: add Unicode tests for 'Vithkuqi'
Vithkuqi support was added to Unicode 14.

Fixes #877
2022-07-05 13:00:10 -04:00
Andrew Gallant
b1489c8445
syntax: make \p{cf} work
It turns out that 'cf' is also an abbreviation for the 'Case_Folding'
property. Even though we don't actually support a 'Case_Folding'
property, a quirk of our code caused 'cf' to fail since it was treated
as a normal boolean property instead of a general category. We fix it be
special casing it.

Note that '\p{gc=cf}' worked and continues to work.

If we ever do add the 'Case_Folding' property, we'll not be able to
support its abbreviation since it is now taken by 'Format'.

Fixes #719
2020-10-13 10:29:03 -04:00
Andrew Gallant
27c0d6d944 style: rust updated rustfmt 2020-01-09 14:26:57 -05:00
Andrew Gallant
0e96af4166
style: start using rustfmt 2019-08-03 14:20:22 -04:00
Andrew Gallant
ecc1a5a70d syntax: add emoji and break properties
This commit adds several emoji properties such as Emoji and
Extended_Pictographic. We also add support for the Grapheme_Cluster_Break,
Word_Break and Sentence_Break enumeration properties.
2018-11-30 20:00:49 -05:00
Andrew Gallant
b906fd55c5 tests: add Unicode general category tests 2018-03-07 19:01:24 -05:00
Andrew Gallant
d44a9f94ab Switch bytes::Regex to using Unicode mode by default. 2016-12-30 01:05:43 -05:00
Andrew Gallant
f9af58c4ca Fixes #234.
It turns out that we weren't compute suffix literals correctly in all
cases. In particular, the bytes from a Unicode character were being
reversed.
2016-05-19 17:47:44 -04:00
Andrew Gallant
d98ec1b1a5 Add regex matching for &[u8].
This commit enables support for compiling regular expressions that can
match on arbitrary byte slices. In particular, we add a new sub-module
called `bytes` that duplicates the API of the top-level module, except
`&str` for subjects is replaced by `&[u8]`. Additionally, Unicode
support in the regular expression is disabled by default but can be
selectively re-enabled with the `u` flag. (Unicode support cannot be
selectively disabled in the standard top-level API.)

Most of the interesting changes occurred in the `regex-syntax` crate,
where the AST now explicitly distinguishes between "ASCII compatible"
expressions and Unicode aware expressions.

This PR makes a few other changes out of convenience:

1. The DFA now knows how to "give up" if it's flushing its cache too
often. When the DFA gives up, either backtracking or the NFA algorithm
take over, which provides better performance.
2. Benchmarks were added for Oniguruma.
3. The benchmarks in general were overhauled to be defined in one place
by using conditional compilation.
4. The tests have been completely reorganized to make it easier to split
up the tests depending on which regex engine we're using. For example,
we occasionally need to be able to write tests specifically for
`regex::Regex` or specifically for `regex::bytes::Regex`.
5. Fixes a bug where NUL bytes weren't represented correctly in the byte
class optimization for the DFA.

Closes #85.
2016-03-09 21:23:29 -05:00