This slightly expands the set of characters allowed in capture group
names to be `[][_0-9A-Za-z.]` from `[_0-9A-Za-z]`.
This required some delicacy in order to avoid replacement strings like
`$Z[` from referring to invalid capture group names where the intent was
to refer to the capture group named `Z`. That is, in order to use `[`,
`]` or `.` in a capture group name, one must use the explicit brace
syntax: `${Z[}`. We clarify the docs around this issue.
Regretably, we are not much closer to handling #595. In order to
support, say, all Unicode word characters, our replacement parser would
need to become UTF-8 aware on `&[u8]`. But std makes this difficult and
I would prefer not to add another dependency on ad hoc UTF-8 decoding or
a dependency on another crate.
Closes#649
Corrects `/-/.splitn("a", 2)` to return `["a"]` instead of `["a", ""]`.
(`/-/` is shorthand for `Regex::new("-").unwrap()`.)
Fixes#521, Closes#606, Closes#628
Corrects `/-/.split("a-")` to return `["a", ""]` instead of `["a"]`.
(`/-/` is shorthand for `Regex::new("-").unwrap()`.)
This adds tests for both `split()` and `splitn()` covering a variety of
edge cases. One test is commented out because it is failing due to #521.
A future commit will fix it.
Note that the `split2` and `split3` tests were passing incorrectly
before this change. I have fixed them to expect the correct values.
Fixes#627
This commit sets up the infrastructure for supporting various `unicode`
and `perf` features, which permit decreasing binary size, compile times
and the size of the dependency tree.
Most of the work here is in modifying the regex tests to make them
work in concert with the available Unicode features. In cases where
Unicode is irrelevant, we just turn it off. In other cases, we require
the Unicode features to run the tests.
This also introduces a new error in the compiler where by if a Unicode
word boundary is used, but the `unicode-perl` feature is disabled, then
the regex will fail to compile. (Because the necessary data to match
Unicode word boundaries isn't available.)
The escaping of &, - and ~ is only required when the characters are
repeated adjacently, which should be quite rare. Escaping of [ is always
required, unless it appear in the second position of a range.
These rules enable us to add character class sets as described in
UTS#18 RL1.3 in a backward compatible way.
This also removes Captures.{at,pos} and replaces it with Captures.get,
which now returns a Match. Similarly, Captures.name returns a Match as
well.
Fixes#276
The bug shown in #251 has the same underlying cause as the bug in
#255, which has been fixed in a previous commit. This commit just adds
a more specific regression test for #251.
Fixes#251.
The DFA handles word boundaries by tagging each state with an `is_word`
flag that lets us determine whether the next byte in the haystack should
cause a word boundary instruction to match. We were mishandling how this
tagging happened for start states. In particular, the tag was not used as
an index into the start state cache, and therefore could wind up choosing
an incorrect but previously computed start state with the wrong flags set.
This leads to incorrect matches.
We fix this by using the right flags to generate an index.
This commit enables support for compiling regular expressions that can
match on arbitrary byte slices. In particular, we add a new sub-module
called `bytes` that duplicates the API of the top-level module, except
`&str` for subjects is replaced by `&[u8]`. Additionally, Unicode
support in the regular expression is disabled by default but can be
selectively re-enabled with the `u` flag. (Unicode support cannot be
selectively disabled in the standard top-level API.)
Most of the interesting changes occurred in the `regex-syntax` crate,
where the AST now explicitly distinguishes between "ASCII compatible"
expressions and Unicode aware expressions.
This PR makes a few other changes out of convenience:
1. The DFA now knows how to "give up" if it's flushing its cache too
often. When the DFA gives up, either backtracking or the NFA algorithm
take over, which provides better performance.
2. Benchmarks were added for Oniguruma.
3. The benchmarks in general were overhauled to be defined in one place
by using conditional compilation.
4. The tests have been completely reorganized to make it easier to split
up the tests depending on which regex engine we're using. For example,
we occasionally need to be able to write tests specifically for
`regex::Regex` or specifically for `regex::bytes::Regex`.
5. Fixes a bug where NUL bytes weren't represented correctly in the byte
class optimization for the DFA.
Closes#85.
