Add a "filecheck" method to the LLDB test base. This allows test authors
to pattern match command output using FileCheck, making it possible to
write stricter tests than what `self.expect` allows.
For context (motivation, examples of stricter checking, etc), see the
lldb-dev thread: "Using FileCheck in lldb inline tests".
Differential Revision: https://reviews.llvm.org/D50751
llvm-svn: 342508
Summary:
With the upstream implementation of dsymutil containing almost all
functionality from the one shipped with Xcode, we want to use the
in-tree version for running the test suite.
This will also allow us to re-enable TestUnicodeSymbols which was
failing because of the discrepancy in how Unicode symbols were hashed in
lldb and older versions of dsymutil.
Reviewers: aprantl, davide, jingham, labath
Subscribers: mgorny, llvm-commits, lldb-commits
Differential Revision: https://reviews.llvm.org/D45518
llvm-svn: 329889
The test "test_fp_special_purpose_register_read" in TestRegisters.py
fails on Darwin machines configured to use an out-of-tree debugserver.
The error message is: 'register read ftag' returns expected result, got
'ftag = 0x80'. This indicates that the debugserver in use is too old.
This commit introduces a decorator which can be used to skip tests which
rely on having a just-built debugserver. This resolves the issue:
$ ./bin/llvm-dotest -p TestRegisters.py -v
1 out of 617 test suites processed - TestRegisters.py
Test Methods: 7
Success: 6
Skip: 1
...
llvm-svn: 327052
This patch is the result of a discussion on lldb-dev, see
http://lists.llvm.org/pipermail/lldb-dev/2018-January/013111.html for
background.
For each test (should be eventually: each test configuration) a
separate build directory is created and we execute
make VPATH=$srcdir/path/to/test -C $builddir/path/to/test -f $srcdir/path/to/test/Makefile -I $srcdir/path/to/test
In order to make this work all LLDB tests need to be updated to find
the executable in the test build directory, since CWD still points at
the test's source directory, which is a requirement for unittest2.
Although we have done extensive testing, I'm expecting that this first
attempt will break a few bots. Please DO NOT HESITATE TO REVERT this
patch in order to get the bots green again. We will likely have to
iterate on this some more.
Differential Revision: https://reviews.llvm.org/D42281
llvm-svn: 323803
Summary:
This has been broken at least since the new test result framework was
added, which was over a year ago. It looks like nobody has missed it
since.
Removing this makes the gmodules handling code saner, as it already did
not know how to handle the multiple-compilers case.
My motivation for this is libc++ data formatters support on android -- I
am trying make a central way of determining whether libc++ tests can be
run, and without this, I would have to resort to similar hacks as the
gmodules code.
Reviewers: jingham, zturner
Subscribers: danalbert, tfiala, lldb-commits
Differential Revision: https://reviews.llvm.org/D30779
llvm-svn: 297811
This patch adds support to the test suite for overriding the path to debugserver, and uses the override to point to the build tree's debugserver on Darwin.
llvm-svn: 297776
Summary: Not everyone names their code sign identity "lldb_codesign", so it is nice to allow this to be overridden.
Reviewers: zturner, tfiala
Subscribers: labath, mgorny, lldb-commits
Differential Revision: https://reviews.llvm.org/D25714
llvm-svn: 284893
Summary: Default installations of OS X do not have system headers installed at /usr/include. This patch allows the LLDB test executables to properly compile when built on a system without headers at /usr/include by specifying a default value for the apple-sdk flag as "macosx".
Reviewers: tfiala, zturner
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D25487
llvm-svn: 284042
Summary:
This patch is necessary because individual test cases are not required
to have unique names. Therefore, test cases must now
be specified explicitly in the form <TestCase>.<TestMethod>.
Because it works by regex matching, passing just <TestCase> will
still disable an entire file.
This also allows for multiple exclusion files to be specified.
Reviewers: zturner, labath, jingham, tfiala
Subscribers: lldb-commits, sas
Differential Revision: https://reviews.llvm.org/D24988
llvm-svn: 283238
Summary:
The current implementation of the test suite allows the user to run
a certain subset of tests using '-p', but does not allow the inverse,
where a user wants to run all but some number of known failing tests.
Implement this functionality.
Reviewers: labath, zturner, tfiala
Subscribers: jingham, sas, lldb-commits
Differential Revision: https://reviews.llvm.org/D24629
llvm-svn: 282298
*** to conform to clang-format’s LLVM style. This kind of mass change has
*** two obvious implications:
Firstly, merging this particular commit into a downstream fork may be a huge
effort. Alternatively, it may be worth merging all changes up to this commit,
performing the same reformatting operation locally, and then discarding the
merge for this particular commit. The commands used to accomplish this
reformatting were as follows (with current working directory as the root of
the repository):
find . \( -iname "*.c" -or -iname "*.cpp" -or -iname "*.h" -or -iname "*.mm" \) -exec clang-format -i {} +
find . -iname "*.py" -exec autopep8 --in-place --aggressive --aggressive {} + ;
The version of clang-format used was 3.9.0, and autopep8 was 1.2.4.
Secondly, “blame” style tools will generally point to this commit instead of
a meaningful prior commit. There are alternatives available that will attempt
to look through this change and find the appropriate prior commit. YMMV.
llvm-svn: 280751
Use of --rerun-all-issues will enable any test method failure, not just
test methods marked with the flakey decorator, to rerun.
Currently this does not change the flakey logic's immediate rerun
attempt. I want to make sure this doesn't cause any significant issues
before changing that part.
The rerun reporting is only known to work properly with the
default (new) BasicResultsFormatter reporting. Once we work out
any issues, I'll go back and make sure the curses output handles
it properly as well.
llvm-svn: 255543
Also adds full path info for exceptional exits and timeouts when
no test method is currently running.
Adds --rerun-all-issues command line arg. If specified, all
test issues are eligible for rerun. If not specified, only tests
marked flakey are eligible for rerun.
The actual rerunning will occur in an upcoming change. This
change just handles tha accounting of what should be rerun.
llvm-svn: 255438
This was an option to display a graphical progress bar. Nobody
is using this, and it doesn't work correctly anyway with the new
result formatter.
llvm-svn: 255153
This removes the failfast command line option as part of an effort
to simplify dotest and remove unused command line options. You can
still Ctrl+C any time you want to exit early.
llvm-svn: 255150
This seems to be a legacy relic from days gone by where the
remote test suite runner operated completely differently than it
does today. git blames and comments traced this functionality
back to about 2012, and nobody seems to know anything about it
now.
llvm-svn: 255060
This removes the option to exclude a single directory. This is
part of an effort to remove unused options and cleanup the interface
to the test suite.
llvm-svn: 255048
Absolute imports were introduced in Python 2.5 as a feature
(e.g. from __future__ import absolute_import), and made default
in Python 3.
When absolute imports are enabled, the import system changes in
a couple of ways:
1) The `import foo` syntax will *only* search sys.path. If `foo`
isn't in sys.path, it won't be found. Period. Without absolute
imports, the import system will also search the same directory
that the importing file resides in, so that you can easily
import from the same folder.
2) From inside a package, you can use a dot syntax to refer to higher
levels of the current package. For example, if you are in the
package lldbsuite.test.utility, then ..foo refers to
lldbsuite.test.foo. You can use this notation with the
`from X import Y` syntax to write intra-package references. For
example, using the previous locationa s a starting point, writing
`from ..support import seven` would import lldbsuite.support.seven
Since this is now the default behavior in Python 3, this means that
importing from the same directory with `import foo` *no longer works*.
As a result, the only way to have portable code is to force absolute
imports for all versions of Python.
See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more
information about absolute and relative imports.
Differential Revision: http://reviews.llvm.org/D14342
Reviewed By: Todd Fiala
llvm-svn: 252191
This is the conclusion of an effort to get LLDB's Python code
structured into a bona-fide Python package. This has a number
of benefits, but most notably the ability to more easily share
Python code between different but related pieces of LLDB's Python
infrastructure (for example, `scripts` can now share code with
`test`).
llvm-svn: 251532
