A DAG-NOT-DAG is a CHECK-DAG group, X, followed by a CHECK-NOT group,
N, followed by a CHECK-DAG group, Y. Let y be the initial directive
of Y. This patch makes the following changes to the behavior:
1. Directives in N can no longer match within part of Y's match
range just because y happens not to be the earliest match from
Y. Specifically, this patch withdraws N's search range end
from y's match range start to Y's match range start.
2. y can no longer match within X's match range, where a y match
produced a reordering complaint, which is thus no longer
possible. Specifically, this patch withdraws y's search range
start from X's permitted range start to X's match range end,
which was already the search range start for other members of
Y.
Both of these changes can only increase the number of test passes: #1
constrains the ability of CHECK-NOTs to match, and #2 expands the
ability of CHECK-DAGs to match without complaints.
These changes are based on discussions at:
<http://lists.llvm.org/pipermail/llvm-dev/2018-May/123550.html>
<https://reviews.llvm.org/D47106>
which conclude that:
1. These changes simplify the FileCheck conceptual model. First,
it makes search ranges for DAG-NOT-DAG more consistent with
other cases. Second, it was confusing that y was treated
differently from the rest of Y.
2. These changes add theoretical use cases for DAG-NOT-DAG that
had no obvious means to be expressed otherwise. We can justify
the first half of this assertion with the observation that
these changes can only increase the number of test passes.
3. Reordering detection for DAG-NOT-DAG had no obvious real
benefit.
We don't have evidence from real uses cases to help us debate
conclusions #2 and #3, but #1 at least seems intuitive.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D48986
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337605 91177308-0d34-0410-b5e6-96231b3b80d8
-v prints all directive pattern matches.
-vv additionally prints info that might be noise to users but that can
be helpful to FileCheck developers.
To maximize code reuse and to make diagnostics more consistent, this
patch also adjusts and extends some of the existing diagnostics.
CHECK-NOT failures now report variables uses. Many more diagnostics
now report the check prefix and kind of directive.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D47114
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336967 91177308-0d34-0410-b5e6-96231b3b80d8
That is, make CHECK-DAG skip matches that overlap the matches of any
preceding consecutive CHECK-DAG directives. This change makes
CHECK-DAG more consistent with other directives, and there is evidence
it makes CHECK-DAG more intuitive and less error-prone. See the RFC
discussion starting at:
http://lists.llvm.org/pipermail/llvm-dev/2018-May/123010.html
Moreover, this behavior enables CHECK-DAG groups for unordered,
non-unique strings or patterns. For example, it is useful for
verifying output or logs from a parallel program, such as the OpenMP
runtime.
This patch also implements the command-line option
-allow-deprecated-dag-overlap, which reverts CHECK-DAG to the old
overlapping behavior. This option should not be used in new tests.
It is meant only for the existing tests that are broken by this change
and that need time to update.
See the following bugzilla issue for tracking of such tests:
https://bugs.llvm.org/show_bug.cgi?id=37532
Patches to add -allow-deprecated-dag-overlap to those tests will
follow immediately.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D47106
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336847 91177308-0d34-0410-b5e6-96231b3b80d8
That is, make CHECK-DAG skip matches that overlap the matches of any
preceding consecutive CHECK-DAG directives. This change makes
CHECK-DAG more consistent with other directives, and there is evidence
it makes CHECK-DAG more intuitive and less error-prone. See the RFC
discussion starting at:
http://lists.llvm.org/pipermail/llvm-dev/2018-May/123010.html
Moreover, this behavior enables CHECK-DAG groups for unordered,
non-unique strings or patterns. For example, it is useful for
verifying output or logs from a parallel program, such as the OpenMP
runtime.
This patch also implements the command-line option
-allow-deprecated-dag-overlap, which reverts CHECK-DAG to the old
overlapping behavior. This option should not be used in new tests.
It is meant only for the existing tests that are broken by this change
and that need time to update.
See the following bugzilla issue for tracking of such tests:
https://bugs.llvm.org/show_bug.cgi?id=37532
Patches to add -allow-deprecated-dag-overlap to those tests will
follow immediately.
Reviewed By: probinson
Differential Revision: https://reviews.llvm.org/D47106
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336830 91177308-0d34-0410-b5e6-96231b3b80d8
Prior to this change, there was no clean way of getting FileCheck to
check that a line is completely empty. The expected way of using
"CHECK: {{^$}}" does not work because the '^' matches the end of the
previous match (this behaviour may be desirable in certain instances).
For the same reason, "CHECK-NEXT: {{^$}}" will fail when the previous
match was at the end of the line, as the pattern will match there.
Using the recommended [[:space:]] to match an explicit new line could
also match a space, and thus is not always desired. Literal '\n'
matches also do not work. A workaround was suggested in the review, but
it is a little clunky.
This change adds a new directive that behaves the same as CHECK-NEXT,
except that it only matches against empty lines (nothing, not even
whitespace, is allowed). As with CHECK-NEXT, it will fail if more than
one newline occurs before the next blank line. Example usage:
; test.txt
foo
bar
; CHECK: foo
; CHECK-EMPTY:
; CHECK-NEXT: bar
Differential Revision: https://reviews.llvm.org/D28896
Reviewed by: probinson
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335613 91177308-0d34-0410-b5e6-96231b3b80d8
FileCheck tool crashes when trying to parse --check-prefix argument if there is no any
data after it.
For example test like following would crash if there are no symbols and no EOL mark after `boom`:
# REQUIRES: x86
# RUN: <skipped few lines>
# RUN: llvm-readobj -t %t | FileCheck %s --check-prefix=boom
Patch fixes the issue.
Differential revision: https://reviews.llvm.org/D42057
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@322536 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This makes it very easy to test files that only differ in a constant
value somewhere in the test case.
Reviewers: jlebar, hfinkel, chandlerc, probinson
Reviewed By: probinson
Subscribers: probinson, llvm-commits
Differential Revision: https://reviews.llvm.org/D39629
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317572 91177308-0d34-0410-b5e6-96231b3b80d8
If there's enough data in fron of it the skipped region would just
become arbitrarily large, and we scan for the CHECK-NOT everywhere.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@304900 91177308-0d34-0410-b5e6-96231b3b80d8
If `--enable-var-scope` is in effect, variables with names that
start with `$` are considered to be global. All other variables are
local. All local variables get undefined at the beginning of each
CHECK-LABEL block. Global variables are not affected by CHECK-LABEL.
This makes it easier to ensure that individual tests are not affected
by variables set in preceding tests.
Differential Revision: https://reviews.llvm.org/D30749
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@297396 91177308-0d34-0410-b5e6-96231b3b80d8
I unfortunately neglected to add it in r260540, but it has been
sitting in my working dir ever since. D'oh.
Modified to work with r290069, which made the CHECK patterns
themselves whitespace-sensitive as well, and remove the test added
then, as this tests both strict and non-strict modes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@291499 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This new alias takes a comma separated list of prefixes which allows
'--check-prefix=A --check-prefix=B --check-prefix=C' to be written as
'--check-prefixes=A,B,C'.
Reviewers: probinson
Subscribers: probinson, llvm-commits, dsanders
Differential Revision: http://reviews.llvm.org/D21293
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@272670 91177308-0d34-0410-b5e6-96231b3b80d8
Add `CHECK-SAME`, which requires that the pattern matches on the *same*
line as the previous `CHECK`/`CHECK-NEXT` -- in other words, no newline
is allowed in the skipped region. This is similar to `CHECK-NEXT`,
which requires exactly 1 newline in the skipped region.
My motivation is to simplify checking the long lines of LLVM assembly
for the new debug info hierarchy. This allows CHECK sequences like the
following:
CHECK: ![[REF]] = !SomeMDNode(
CHECK-SAME: file: ![[FILE:[0-9]+]]
CHECK-SAME: otherField: 93{{[,)]}}
which is equivalent to:
CHECK: ![[REF]] = !SomeMDNode({{.*}}file: ![[FILE:[0-9]+]]{{.*}}otherField: 93{{[,)]}}
While this example just has two fields, many nodes in debug info have
more than that. `CHECK-SAME` will keep the logic easy to follow.
Morever, it enables interleaving `CHECK-NOT`s without allowing newlines.
Consider the following:
CHECK: ![[REF]] = !SomeMDNode(
CHECK-SAME: file: ![[FILE:[0-9]+]]
CHECK-NOT: unexpectedField:
CHECK-SAME: otherField: 93{{[,)]}}
CHECK-NOT: otherUnexpectedField:
CHECK-SAME: )
which doesn't seem to have an equivalent `CHECK` line.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230612 91177308-0d34-0410-b5e6-96231b3b80d8
Currently FileCheck errors out on empty input. This is usually the
right thing to do, but makes testing things like "this command does
not emit some error message" hard to test. This usually leads to
people using "command 2>&1 | count 0" instead, and then the bots that
use guard malloc fail a few hours later.
By adding a flag to FileCheck that allows empty inputs, we can make
tests that consist entirely of "CHECK-NOT" lines feasible.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215127 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Add FileCheck -implicit-check-not option which allows specifying a
pattern that should only occur in the input when explicitly matched by a
positive check. This feature allows checking tool diagnostics in a way
clang -verify does it for compiler diagnostics.
The option has been tested on a number of clang-tidy checks, I'll post a link to
the clang-tidy patch to this thread.
Once there's an agreement on the general direction, I can add tests and
documentation.
Reviewers: djasper, bkramer
Reviewed By: bkramer
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D4462
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212810 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Directives are being ignored, when they occur between a partial-word false
match and any match on another prefix.
For example, with FOO and BAR prefixes:
_FOO
FOO: foo
BAR: bar
FileCheck incorrectly matches:
fog
bar
This happens because FOO falsely matched as a partial word at '_FOO' and was
ignored while BAR matched at 'BAR:'. The match of BAR is incorrectly returned
as the 'first match' causing the FOO directive to be discarded.
Fixed this the same way as r194565 (D2166) did for a similar test case.
The partial-word false match should be counted as a match for the purposes of
finding the first match of a prefix, but should be returned as a false match
using CheckTy::CheckNone so that it isn't treated as a directive.
Fixes PR17995
Reviewers: samsonov, arsenm
Reviewed By: samsonov
CC: llvm-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D2228
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195248 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Fix a case when "FileCheck --check-prefix=CHECK --check-prefix=CHECKER"
would silently ignore check-lines of the form:
CHECKER: foo
Reviewers: dsanders
Reviewed By: dsanders
CC: llvm-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D2168
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194577 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This fixes a subtle bug in new FileCheck feature added
in r194343. When we search for the first satisfying check-prefix,
we should actually return the first encounter of some check-prefix as a
substring, even if it's not a part of valid check-line. Otherwise
"FileCheck --check-prefix=FOO --check-prefix=BAR" with check file:
FOO not a vaild check-line
FOO: foo
BAR: bar
incorrectly accepted file:
fog
bar
as it skipped the first two encounters of FOO, matching only BAR: line.
Reviewers: arsenm, dsanders
Reviewed By: dsanders
CC: llvm-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D2166
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194565 91177308-0d34-0410-b5e6-96231b3b80d8
This is useful if you want to run multiple variations
of a single test, and the majority of check lines
should be the same.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194343 91177308-0d34-0410-b5e6-96231b3b80d8
- Instead of setting the suffixes in a bunch of places, just set one master
list in the top-level config. We now only modify the suffix list in a few
suites that have one particular unique suffix (.ml, .mc, .yaml, .td, .py).
- Aside from removing the need for a bunch of lit.local.cfg files, this enables
4 tests that were inadvertently being skipped (one in
Transforms/BranchFolding, a .s file each in DebugInfo/AArch64 and
CodeGen/PowerPC, and one in CodeGen/SI which is now failing and has been
XFAILED).
- This commit also fixes a bunch of config files to use config.root instead of
older copy-pasted code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188513 91177308-0d34-0410-b5e6-96231b3b80d8
FileCheck should check to make sure the prefix was found, and not a word
containing it (e.g -check-prefix=BASEREL shouldn't match NOBASEREL).
Patch by Ron Ofir.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188221 91177308-0d34-0410-b5e6-96231b3b80d8
CHECK-LABEL is meant to be used in place on CHECK on lines containing identifiers or other unique labels (they need not actually be labels in the source or output language, though.) This is used to break up the input stream into separate blocks delineated by CHECK-LABEL lines, each of which is checked independently. This greatly improves the accuracy of errors and fix-it hints in many cases, and allows for FileCheck to recover from errors in one block by continuing to subsequent blocks.
Some tests will be converted to use this new directive in forthcoming patches.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186162 91177308-0d34-0410-b5e6-96231b3b80d8
Pattern has source location by itself. After adding a trivial method to
retrieve it, it's unnecessary to pair a source location for CHECK-NOT patterns.
One thing revised after this is the diagnostic info is more accurate by
pointing to the start of the CHECK-NOT pattern instead of the end of the
CHECK-NOT pattern. E.g. diagnostic message previously looks like
<stdin>:1:1: error: CHECK-NOT: string occurred!
test
^
test.txt:1:16: note: CHECK-NOT: pattern specified here
CHECK-NOT: test
^
is changed to
<stdin>:1:1: error: CHECK-NOT: string occurred!
test
^
test.txt:1:12: note: CHECK-NOT: pattern specified here
CHECK-NOT: test
^
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180578 91177308-0d34-0410-b5e6-96231b3b80d8
; CHECK: [[VAR:[a-z]]]
The problem was that to find the end of the regex var definition, it was
simplistically looking for the next ]] and finding the incorrect one. A
better approach is to count nesting of brackets (taking escaping into
account). This way the brackets that are part of the regex can be discovered
and skipped properly, and the ]] ending is detected in the right place.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169109 91177308-0d34-0410-b5e6-96231b3b80d8
