Commit Graph
15 Commits
Author SHA1 Message Date
Bojan Rajkovic 1bbb0f0880 Revert "Bug 2663: Regular Expression Bug Fixes"
This reverts commit dc6efdab29. This
patch completely regressed performance for regexes in the same class
as the regexes it fixed. Xamarin bug #6198 details the regressions,
and I've confirmed that reverting this commit fixes the 2 order of
magnitude performance drop (with the patch, the test takes almost 7
seconds, without it, it takes .07 seconds). The bug still needs to be
fixed, but this is not the right fix.
2012-10-17 12:35:57 -04:00
Rob Wilkens 60a463c6f3 Bug 2663: Regular Expression Bug Fixes
What this fixes:
IF you were checking a regex which contained something like "(A|AB)C" and then you
checked something like "ABC" against it, it would fail previously, because its
first trip thru the regex it would match "A" first and short circuit out of the
"or" (|)  then next try to match "C" but find "B" and fail.  In this case, it should
be obvious that it needs to, after it fails, go back and try "AB" rather than "A"
and that's where this patch comes in.

Brief summary of each change in order that "git diff --staged" is giving them to
me:

This first one summarizes the background of goal of project:
New Function: EmitJumpTest
For when we are evaluating Alternatives (or's) we now want to be able to go back
and try or paths we haven't already tried if and only if we fail.  This alternate to
EmitJump instead of jumping to target if we have a match immediately, will
on first attempt do the jump, but later if we have already tried this jump
combination, it will jump to ElseTarget, which is the equivalent of this test
failing on subsequent attempts with the same combination of 'or's passing.

RxCompiler:
EmitJumpTest isn't implemented for RxCompiler because i didn't have sample
code to test it on, so it falls back to a regular jump, nothing changes in
RxCompiler in other words other than having this required fucntion front.

arch.cs:
New OpCode added for JumpTest (see EmitJumpTest above)

compiler.cs:
EmitJumpTest added here for regular compiler, it now takes two arguments (two
items on stack) first was the standard where to jump to address, the second
argument is where to jump to in case where we have already tried this
combination)

interpreter.cs:

JumpTestEntry - What we keep track of to know if we've been here before, for
now i presumed just 'pc' (program counter') and 'ptr' (pointer - where we are)
were enough to tell us if we've been to this particular spot before.

Eval has been moved into Eval_Real

Eval if it gets called with parameters indicating we've already been here
(i.e we're being called recursively) just passes the call directly to eval_real
as-was.

Otherwise eval will loop on eval_real while we fail and haven't tried every
possible path through all the or's. (Note: The special case of where we're
running a 'test' (i.e. checking !(a|b|c) rather than just matching (a|b|c)),
and if we're running a test, we turn off JumpTestTracking while it's in its
recursive section.  The reason for this is because while we can rightly check
all the options of (a|ab|abc)  first for a then try again with ab then try again
with abc, we cannot do the same for a not (i.e. !(something or something else)
is really (not something AND not somtehing else), and it is not (not something
or not something else))

JumpTestList tracks our current run's JumpTest calls...
TriedCombo's tracks all the combos (all the JumpTestList results) that we've
already tried.

All calls to anything which calls Eval were modified to call eval passing
the optional new parameters that we now need, and remember if we call
eval with those parameters having a value, it will just forward the call
to Eval_Real.

This included modifying TryMatch because it calls Eval.

OpCode.JumpTest handler was added, what it does is first check if we need
to ByPass to jump to 'else' right away, which again happens when we're in
a test recursive call (Checking not, for example).
Next, it will create and populate a new JumpTestEntry with the current values.
It then calls CheckComboMatch() which sees if we match a combo already on file.
If we don't have a Match we jump to the first label (at pc+1) which is the
regular jump call.  If we have a match, though, it will jump to the else label
which is now at (pc+2).

CheckComboMatch had to be broken up into two functions because it was going
too deep in terms of nesting.  What it does by now should be somewhat understood
it will see if we have a match between our current run and previous runs, and
if we have a match, it returns true.  If we don't have a match, it returns
false.  I have some doubts about whether this function is 100% correct, there
may remain a bug or two, but it does pass all the "System" unit tests which
includes all the regular expression tests, plus it works for all the tests
in the bug report, so i have little reason to mess with it.

syntax.cs:
Here's where i modified so that when evaluating alternatives ("or"s or "|"s)
we instead call EmitJumpTest rather than just a regular EmitJump.

RegExBugs.cs:
This is the unit test (a bunch in one) which fails on mono without this patch
but succeeds with it
commit 48027ac2bdd9b60d18985ec2537eb188607f8164 1 parent 91c4180d55

This patch was originally submitted under another branch which had a bunch of
unrelated commits in it.  I'm hoping if i did this right, these changes stand
alone.
2012-06-18 07:38:57 -04:00
Raja R Harinath a21d173159 Introduce 'gap' concept into all the variants of regex
* arch.cs (IMachineFactory::Gap): New property.
* compiler.cs (InterpreterFactory): Update to changes.
* RxCompiler.cs (RxInterpreterFactory): Likewise.
* JvmReMachineFactory.cs: Likewise.
* parser.cs (GetMapping): Return the 'gap' index.  Move creation
of mapping dictionary ...
* Regex.cs (CreateMachineFactory): ... here.  Update to changes.

svn path=/trunk/mcs/; revision=140285
2009-08-19 20:41:03 +00:00
Zoltan Varga 824710f744 2009-01-14 Zoltan Varga <[email protected]>
* CILCompiler.cs: Avoid throwing exceptions for a few more unhandled opcodes.
	Implement the reverse/ignorecase versions of Reference. Fix the reverse
	version of Bitmap. Error out if a Jump opcode jumps outside the current
	pc range. This works around #466151.

svn path=/trunk/mcs/; revision=123417
2009-01-14 21:33:40 +00:00
Zoltan Varga 6941113b7a 2008-10-25 Zoltan Varga <[email protected]>
* RxOp.cs: Remove unused Generic opcodes.

	* RxCompiler.cs: Emit opcodes with flags using methods which can be 
	overriden by CILCompiler.cs, so we only have to override a few methods.

	* CILCompiler.cs: Simplify based on the RxCompiler changes. Implement
	UnicodeString opcode. Throw a NotImplementedException when an unsupported
	opcode is encountered, since the IL compiler is now believed to be feature 
	complete.

svn path=/trunk/mcs/; revision=117047
2008-10-25 14:00:58 +00:00
Zoltan Varga 4b5c388a7b 2008-09-13 Zoltan Varga <[email protected]>
* RxInterpreter.cs RxCompiler.cs: Implement some missing Category opcodes.

svn path=/trunk/mcs/; revision=112914
2008-09-12 22:28:39 +00:00
Zoltan Varga 01fe2ef4e7 2008-08-31 Zoltan Varga <[email protected]>
* RxInterpreter.cs: Port the Anchor+StartOfString optimization from the old
	interpreter. Add credits.

	* RxCompiler.cs (EmitString): Lowercase the string if ignore is true.

svn path=/trunk/mcs/; revision=111989
2008-08-31 15:21:36 +00:00
Zoltan Varga 01bc43a4e2 2008-08-29 Zoltan Varga <[email protected]>
* RxInterpreter.cs RxCompiler.cs RxOp.cs: Implement/fix this so all the
	regex tests now run. The Repeat code is taken from the old interpreter.

svn path=/trunk/mcs/; revision=111925
2008-08-29 14:44:15 +00:00
Zoltan Varga 2b49ce098a 2008-08-28 Zoltan Varga <[email protected]>
* RxInterpreter.cs RxCompiler.cs RxOp.cs: Add support for 
	Category.AnySingleline.

svn path=/trunk/mcs/; revision=111747
2008-08-27 22:32:26 +00:00
Zoltan Varga 8d926d9b6c 2008-08-27 Zoltan Varga <[email protected]>
* RxCompiler.cs: Use Array.Copy since Buffer.BlockCopy operates on bytes.

svn path=/trunk/mcs/; revision=111679
2008-08-26 23:31:56 +00:00
Paolo Molaro e13ae1763b Thu Feb 14 17:11:10 CET 2008 Paolo Molaro <[email protected]>
* RxCompiler.cs, RxInterpreter.cs, RxOp.cs: replace EcmaDigit with
	Range, fixed matching of "" =~ /$/, implementedt TestCharGroup.
	Fixes and other misc opcodes implemented.


svn path=/trunk/mcs/; revision=95641
2008-02-14 15:31:24 +00:00
Zoltan Varga b44106f869 2008-02-13 Zoltan Varga <[email protected]>
* CILCompiler.cs: New file, extracted from RxCompiler.cs.

svn path=/trunk/mcs/; revision=95582
2008-02-13 14:58:33 +00:00
Zoltan Varga 4f88a475b5 2008-02-13 Zoltan Varga <[email protected]>
* RxInterpreter.cs RxCompiler.cs: First version of regex IL compiler.

svn path=/trunk/mcs/; revision=95580
2008-02-13 14:30:06 +00:00
Paolo Molaro e5481eeed7 Wed Feb 13 13:18:12 CET 2008 Paolo Molaro <[email protected]>
* RxCompiler.cs, RxInterpreter.cs, RxOp.cs: unicode categories and
	some grouping/capture support.


svn path=/trunk/mcs/; revision=95565
2008-02-13 11:36:07 +00:00
Paolo Molaro 17bf580219 Tue Feb 12 19:16:49 CET 2008 Paolo Molaro <[email protected]>
* RxCompiler.cs, RxInterpreter.cs, RxOp.cs: experimental new
	interpreter.


svn path=/trunk/mcs/; revision=95534
2008-02-12 17:35:01 +00:00