With the regex syntax rewrite, we now support empty subexpressions more
officially. Unfortunately, the compiler has trouble with empty
subexpressions in alternation branches. There's no particular reason to
not support for them, but they are difficult/awkward to express with the
current compiler. So just ban them for now.
If one does need an empty subexpression in an alternate branch, then
amusingly, something like `()?|z` will work. We could rewrite all such
empty sub-expressions into `()?`, which would retain the same match
semantics, but we choose to take the most conservative change possible.
This commit moves the entire regex crate over to the regex-syntax-2
rewrite. Most of this is just rewriting types.
The compiler got the most interesting set of changes. It got simpler
in some respects, but not significantly so.
This commit enables support for compiling regular expressions that can
match on arbitrary byte slices. In particular, we add a new sub-module
called `bytes` that duplicates the API of the top-level module, except
`&str` for subjects is replaced by `&[u8]`. Additionally, Unicode
support in the regular expression is disabled by default but can be
selectively re-enabled with the `u` flag. (Unicode support cannot be
selectively disabled in the standard top-level API.)
Most of the interesting changes occurred in the `regex-syntax` crate,
where the AST now explicitly distinguishes between "ASCII compatible"
expressions and Unicode aware expressions.
This PR makes a few other changes out of convenience:
1. The DFA now knows how to "give up" if it's flushing its cache too
often. When the DFA gives up, either backtracking or the NFA algorithm
take over, which provides better performance.
2. Benchmarks were added for Oniguruma.
3. The benchmarks in general were overhauled to be defined in one place
by using conditional compilation.
4. The tests have been completely reorganized to make it easier to split
up the tests depending on which regex engine we're using. For example,
we occasionally need to be able to write tests specifically for
`regex::Regex` or specifically for `regex::bytes::Regex`.
5. Fixes a bug where NUL bytes weren't represented correctly in the byte
class optimization for the DFA.
Closes#85.
