Commit Graph

143 Commits

Author SHA1 Message Date
David Greene
28140b5288 Fix PR 8199. This patch prepends the build tool dir to LLVM programs
being tested.  This ensures that we test the tools just built and not
some random tools that might happen to be in the user's PATH.  This
makes LLVM testing much more stable and predictable.

llvm-svn: 122341
2010-12-21 16:55:53 +00:00
NAKAMURA Takumi
0413f5e1cb utils/lit/lit/TestFormats.py: [PR8438] unittests: Seek *Tests (not BUILD_MODE/*Tests) under whole unittests/ if BUILD_MODE == '.'
llvm-svn: 121118
2010-12-07 07:41:32 +00:00
NAKAMURA Takumi
913ae8e162 lit.GoogleTest: On case-insensitive filesystem, matching should be case-insensitive when directory name is checked with test_sub_dir.
On MSVS8, ${CMAKE_CFG_INTDIR}, aka $(OutDir), has capitalized name(eg. Debug), although $(OutDir) is made with lower case(eg. debug).

llvm-svn: 119781
2010-11-19 03:19:26 +00:00
Daniel Dunbar
56dbefb6d6 lit: Fix a subtle resource usage bug when executing tests using the internal
shell runner.

We would inadvertently end up holding on to handles to the temporary files
longer than we should have been. On Win32, where open handles lock some file
operations, this caused problems in tests which would try to move temporary
files around (as Clang does by default now).

Many thanks to Francois Pichet for the excellent detective work on this.

llvm-svn: 115040
2010-09-29 15:59:37 +00:00
Michael J. Spencer
35cee49b2f Test unittests built with CMake.
llvm-svn: 114726
2010-09-24 09:01:22 +00:00
Michael J. Spencer
7160458b8e Fix whitespace.
llvm-svn: 114724
2010-09-24 09:00:56 +00:00
Daniel Dunbar
3555832020 lit: Bump version to 0.2.0dev, for no apparent reason.
llvm-svn: 114262
2010-09-18 02:28:15 +00:00
Daniel Dunbar
9bf57694be lit: Tweak setup.py.
llvm-svn: 114261
2010-09-18 02:28:12 +00:00
Daniel Dunbar
98afc51d08 lit: These TODOs are done(ish).
llvm-svn: 114260
2010-09-18 02:28:09 +00:00
Duncan Sands
e06344da06 Some of the tests build an executable with llvm-gcc. For this to work, llvm-gcc
needs to find some libraries, which may require searching the directories given
by LIBRARY_PATH on curiously configured systems.  So pass on LIBRARY_PATH.

llvm-svn: 114186
2010-09-17 17:05:27 +00:00
Daniel Dunbar
14e18ee412 lit: Expose FileBasedTest object and Test module via 'lit' object that gets
exposed to config files.

llvm-svn: 113923
2010-09-15 03:52:38 +00:00
Daniel Dunbar
24c29a251d lit: Rename main lit module to main.py, lit/lit/lit.py was a bit too, err,
alliterate.

llvm-svn: 113922
2010-09-15 03:52:31 +00:00
Daniel Dunbar
7c1d941155 lit/GoogleTest: Add .exe to the suffix when looking for tests.
llvm-svn: 111487
2010-08-19 00:22:45 +00:00
Dan Gohman
c5bc7b091e Expand uses of python 2.6's "A if B else C" syntax into regular
if-else statements, to hopefully support older pythons (PR7850).

llvm-svn: 110638
2010-08-10 01:03:34 +00:00
Dan Gohman
33ef8b480f Print a message when a test failure is due to stderr output
alone, rather than just an exit code.

llvm-svn: 110208
2010-08-04 16:42:38 +00:00
Dan Gohman
ee46b1653d Change the logic which interprets output on stderr as an error so that
it doesn't modify the exit code or the stdout contents, and so that it
doesn't clutter the output with "Command has output on stderr!".

llvm-svn: 110171
2010-08-04 00:12:31 +00:00
Dan Gohman
9e8acd4460 Don't print "Command output (stdout):" when the command has no output,
and same for stderr, to avoid clutter in the output.

llvm-svn: 110169
2010-08-04 00:05:16 +00:00
Dan Gohman
e75f31b55d Use the regular conditional operator syntax instead of a clever hack.
llvm-svn: 110168
2010-08-04 00:00:13 +00:00
Daniel Dunbar
e7c9b43aae lit: Allow clients to define predefined parameters.
llvm-svn: 109999
2010-08-02 00:39:38 +00:00
Daniel Dunbar
e9993379d7 lit: Add some example tests for previous commit.
llvm-svn: 109071
2010-07-22 00:40:19 +00:00
Daniel Dunbar
044ebec2b8 lit: Add support for 'REQUIRES: feature-one, feature-two, ...' in the
integrated-test formats (sh and tcl style). The particular features which get
recognized are up to the test suite itself to define.

llvm-svn: 109062
2010-07-21 23:39:57 +00:00
Daniel Dunbar
6a902f3029 lit: Replace /dev/null in scripts with temporary files on Windows.
llvm-svn: 105888
2010-06-12 16:00:10 +00:00
Daniel Dunbar
d9120853e1 lit: Add a forgotten default argument.
llvm-svn: 105858
2010-06-11 23:47:36 +00:00
Daniel Dunbar
d0e8649780 lit: When running Tcl style tests on Windows, substitute slashes to avoid Tcl
quoting problems. Not particularly ideal, but should work ok. Based on a patch by
Michael Spencer!

llvm-svn: 105855
2010-06-11 23:27:45 +00:00
Daniel Dunbar
1c27a3b79d lit: Add another place to look for bash.
llvm-svn: 104189
2010-05-19 23:56:09 +00:00
Daniel Dunbar
8578537b17 lit: Fix a sh lexing bug which caused annotate-token.m to fail when run with the
internal shell parser; we weren't lexing the quotes in a command like::

  clang -DFOO='hello'

correctly.

llvm-svn: 103652
2010-05-12 21:47:58 +00:00
Daniel Dunbar
8c1208781d lit: Fix OneCommandPerFileTest format when tests are specified directly.
llvm-svn: 103626
2010-05-12 17:56:44 +00:00
Daniel Dunbar
cab93afa90 lit: Add support for 'lit ... @foo', which reads a list of tests to run from
foo.

llvm-svn: 103625
2010-05-12 17:56:42 +00:00
Daniel Dunbar
772b9002e7 lit: Allow test_format to be None.
llvm-svn: 103091
2010-05-05 17:22:35 +00:00
Daniel Dunbar
4ddbf8ad2c lit: Add support to OneCommandPerFileTest format to take input directory from input path.
llvm-svn: 100846
2010-04-09 02:15:10 +00:00
Chris Lattner
a58082f349 add newlines at end of files.
llvm-svn: 100706
2010-04-07 22:54:55 +00:00
Daniel Dunbar
1a27d28404 Add a FIXME.
llvm-svn: 100034
2010-03-31 19:14:05 +00:00
Daniel Dunbar
9c71d03e49 lit: Make sure to close any files we open as part of redirection.
PR6753.

llvm-svn: 100032
2010-03-31 18:48:43 +00:00
Daniel Dunbar
6de90fba7e lit: Add LitTestCase and lit.load_test_suite, for adapting lit based suites for
use with Python's unittest.

llvm-svn: 99498
2010-03-25 07:10:01 +00:00
Jeffrey Yasskin
28c789b8a6 When we don't pass --vg-leak to lit, pass --leak-check=no to valgrind.
llvm-svn: 99112
2010-03-21 01:47:33 +00:00
Jeffrey Yasskin
4927fd0032 Add support for XFAILing valgrind runs with memory leak checking independently
of runs without leak checking.  We add -vg to the triple for non-checked runs,
or -vg_leak for checked runs.  Also use this to XFAIL the TableGen tests, since
tablegen leaks like a sieve.  This includes some valgrindArgs refactoring.

llvm-svn: 99103
2010-03-20 23:08:45 +00:00
Ben Holt
fa824e3279 Test commit (removed extra blank line)
llvm-svn: 98988
2010-03-19 21:12:29 +00:00
Torok Edwin
ec792738a5 Set numThreads to 1 by default when Python is older than 2.5.2.
Python 2.4 always hits this bug: http://bugs.python.org/issue1731717
when running check-lit on multi-core systems.
Setting numThreads to 1 makes it slower, but at least the results reported are
correct.

llvm-svn: 98969
2010-03-19 17:54:21 +00:00
Daniel Dunbar
d655f05fb9 lit: Tweak example tests config.
llvm-svn: 98859
2010-03-18 20:06:05 +00:00
Jeffrey Yasskin
6e1782932d Avoid a problem in libc_freeres() seen on a linux-i686 buildbot when running
under valgrind:

==19577== Invalid free() / delete / delete[]
==19577==    at 0x4C9C866: free (vg_replace_malloc.c:325)
==19577==    by 0x5121104: ??? (in /lib/libc-2.10.2.so)
==19577==    by 0x4C97412: _vgnU_freeres (vg_preloaded.c:62)
==19577==    by 0x5041486: __run_exit_handlers (exit.c:93)
==19577==    by 0x50414FE: exit (exit.c:100)
==19577==    by 0x5028B5C: (below main) (libc-start.c:254)
==19577==  Address 0xffffffff is not stack'd, malloc'd or (recently) free'd
==19577== 

Apparently this happens under certain versions of glibc, so valgrind provides
the --run-libc-freeres=no option to avoid calling freeres().  This may increase
the number of "still reachable" blocks valgrind reports, but we don't care
about those, while this error breaks the buildbots.

There are upstream bugs about this at
http://sourceware.org/bugzilla/show_bug.cgi?id=10610 and
http://bugs.kde.org/show_bug.cgi?id=167483, but they don't look likely to be
fixed.

llvm-svn: 98813
2010-03-18 07:00:12 +00:00
Jeffrey Yasskin
6973958dee Make lit pay attention to --vg for tcl tests too, which makes it work on LLVM's
non-unit tests.

llvm-svn: 98741
2010-03-17 17:04:56 +00:00
Jeffrey Yasskin
48cd56b96a Make lit pay attention to --vg for unittests too.
llvm-svn: 98449
2010-03-13 20:45:41 +00:00
John McCall
643489334d Revert r97726 and r97728 at ddunbar's request; we want to solve this
some other way when it comes to be necessary.

llvm-svn: 97972
2010-03-08 20:02:05 +00:00
Duncan Sands
d18aa07fde If LD_LIBRARY_PATH is set in the system environment, use it.
llvm-svn: 97796
2010-03-05 08:21:02 +00:00
John McCall
45e11bf9a8 Simplify the condition-checking logic and hopefully clear up a build failure
that somehow got through my testing.

llvm-svn: 97728
2010-03-04 11:48:42 +00:00
John McCall
f762d454a4 Teach lit to honor conditional directives. The syntax is:
IF(condition(value)):
If the value satisfies the condition, the line is processed by lit;  otherwise
it is skipped.  A test with no unignored directives is resolved as Unsupported.

The test suite is responsible for defining conditions;  conditions are unary
functions over strings.  I've defined two conditions in the LLVM test suite,
TARGET (with values like those in TARGETS_TO_BUILD) and BINDING (with values
like those in llvm_bindings).  So for example you can write:
  IF(BINDING(ocaml)): RUN: %blah %s -o -
and the RUN line will only execute if LLVM was configured with the ocaml
bindings.

llvm-svn: 97726
2010-03-04 09:36:50 +00:00
Daniel Dunbar
93911572ff Eliminate llvmgcc_version testing variable.
llvm-svn: 96908
2010-02-23 07:56:28 +00:00
Daniel Dunbar
cb977c7241 Kill unused llvmgccmajvers testing variable.
llvm-svn: 96906
2010-02-23 07:56:18 +00:00
Daniel Dunbar
0e4c6eb0d7 lit: Ignore dot files when scanning for tests (e.g., editor temprary files,
etc.)

llvm-svn: 95803
2010-02-10 21:00:55 +00:00
Daniel Dunbar
54d6a41882 Remove stray DOS newline.
llvm-svn: 95765
2010-02-10 04:10:10 +00:00
Jeffrey Yasskin
d0d363d1a9 Make lit's gtest support honor config.environment.
llvm-svn: 95398
2010-02-05 18:09:19 +00:00
Daniel Dunbar
ca283162fa lit: Separate test suite from test name with spaces, to make it easier to cut and paste.
llvm-svn: 94134
2010-01-22 02:04:28 +00:00
Daniel Dunbar
e7b673bbc4 lit: Add setuptools support.
llvm-svn: 92169
2009-12-26 22:58:39 +00:00
Daniel Dunbar
8dae6490fd lit: Sink code into a 'lit' package.
llvm-svn: 92168
2009-12-26 22:58:23 +00:00
Daniel Dunbar
8747fc993f lit: Improve error when gtest discovery fails.
llvm-svn: 91458
2009-12-15 22:00:37 +00:00
Daniel Dunbar
ea0c099b04 lit: Prevent crash-on-invalid (when run on directory which has no test suite).
llvm-svn: 90871
2009-12-08 19:49:30 +00:00
Daniel Dunbar
157d7f6596 lit: Fix exclude dirs functionality.
llvm-svn: 89210
2009-11-18 17:42:17 +00:00
Jeffrey Yasskin
0f846dbb3e Make X86-64 in the Large model always emit 64-bit calls.
The large code model is documented at
http://www.x86-64.org/documentation/abi.pdf and says that calls should
assume their target doesn't live within the 32-bit pc-relative offset
that fits in the call instruction.

To do this, we turn off the global-address->target-global-address
conversion in X86TargetLowering::LowerCall(). The first attempt at
this broke the lazy JIT because it can separate the movabs(imm->reg)
from the actual call instruction. The lazy JIT receives the address of
the movabs as a relocation and needs to record the return address from
the call; and then when that call happens, it needs to patch the
movabs with the newly-compiled target. We could thread the call
instruction into the relocation and record the movabs<->call mapping
explicitly, but that seems to require at least as much new
complication in the code generator as this change.

To fix this, we make lazy functions _always_ go through a call
stub. You'd think we'd only have to force lazy calls through a stub on
difficult platforms, but that turns out to break indirect calls
through a function pointer. The right fix for that is to distinguish
between calls and address-of operations on uncompiled functions, but
that's complex enough to leave for someone else to do.

Another attempt at this defined a new CALL64i pseudo-instruction,
which expanded to a 2-instruction sequence in the assembly output and
was special-cased in the X86CodeEmitter's emitInstruction()
function. That broke indirect calls in the same way as above.

This patch also removes a hack forcing Darwin to the small code model.
Without far-call-stubs, the small code model requires things of the
JITMemoryManager that the DefaultJITMemoryManager can't provide.

Thanks to echristo for lots of testing!

llvm-svn: 88984
2009-11-16 22:41:33 +00:00
Daniel Dunbar
20a9eda5a2 lit: Factor a new OneCommandPerFileTest out of SyntaxCheckTest.
- Used for running a single fixed command on a directory of files, with the
   option of deriving a temporary input file from the test source.

llvm-svn: 88844
2009-11-15 08:10:29 +00:00
Daniel Dunbar
b077010494 Remove duplicate implementation of excludes functionality, and support excluding
dirnames.

Also, add support for the 'unsupported' config property.

llvm-svn: 88838
2009-11-15 07:22:58 +00:00
Daniel Dunbar
a812eb66b3 lit: Add --repeat=N option, for running each test N times.
- Currently just useful for timing, although it could be extended as one (bad) way to deal with flaky tests.

llvm-svn: 88827
2009-11-15 01:02:09 +00:00
Daniel Dunbar
fbe1d52c6a lit: Add ExampleTests, for testing lit and demonstrating test suite features.
llvm-svn: 86654
2009-11-10 02:41:17 +00:00
Daniel Dunbar
da307a7868 lit: Fix bug in --show-suites which accidentally override the list of tests.
llvm-svn: 86653
2009-11-10 02:40:21 +00:00
Daniel Dunbar
d4dcd76f71 lit: Hardcode whence seek value, os.SEEK_END isn't always available.
llvm-svn: 86449
2009-11-08 09:07:33 +00:00
Daniel Dunbar
761032e17e lit: Warn when a test suite contains no tests.
llvm-svn: 86448
2009-11-08 09:07:26 +00:00
Daniel Dunbar
642e76ff50 lit: Drop require_and_and support.
llvm-svn: 86447
2009-11-08 09:07:13 +00:00
Daniel Dunbar
dedae84dd3 lit: Workaround a Win32/subprocess bug when appending.
llvm-svn: 86437
2009-11-08 03:43:06 +00:00
Daniel Dunbar
a8040023d7 lit: Preserve the PATHEXT variable when running subcommands, this is important on Win32
llvm-svn: 86436
2009-11-08 03:35:19 +00:00
Douglas Gregor
1dc69ca319 Teach lit's SyntaxCheckTest two new tricks:
- skip .svn directories
  - add a set of excluded filenames so we can easily skip tests

llvm-svn: 86185
2009-11-05 22:58:04 +00:00
Daniel Dunbar
47718a1f47 lit: Add --param NAME=VALUE option, for test suite specific use (to communicate
arbitrary command line arguments to the test suite).

llvm-svn: 86137
2009-11-05 16:27:33 +00:00
Daniel Dunbar
3d3f9b9c83 lit: Update Clang's test style to use XFAIL: and XTARGET: lines that match
LLVM's tests.

llvm-svn: 85882
2009-11-03 07:26:38 +00:00
Daniel Dunbar
499815e9f9 lit: Add --config-prefix option, to override default config file names.
llvm-svn: 85035
2009-10-25 03:30:55 +00:00
Daniel Dunbar
69b298e1c2 lit: Allow use of /dev/null in redirects on Windows (replace by a temporary
file).

llvm-svn: 85028
2009-10-25 01:37:26 +00:00
Dan Gohman
3c62b33053 Delete a spurious semicolon.
llvm-svn: 85018
2009-10-24 23:24:45 +00:00
Daniel Dunbar
e66bb298d4 lit: Support '>>' redirections when executing scripts internally.
llvm-svn: 85014
2009-10-24 20:32:49 +00:00
Daniel Dunbar
bb58da6d55 lit: When running Tcl scripts via shell, try harder to find 'bash', but fall
back to running them internally if that fails. PR5240.

llvm-svn: 84462
2009-10-19 03:54:21 +00:00
Jeffrey Yasskin
160f3b9191 Support GoogleTest's "typed tests"
(http://code.google.com/p/googletest/wiki/GoogleTestAdvancedGuide#Typed_Tests)
in lit.py.  These tests have names like "ValueMapTest/0.Iteration", which broke
when lit.py os.path.join()ed them onto the path and then assumed it could
os.path.split() them back off.  This patch shifts path components from the
testPath to the testName until the testPath exists.

llvm-svn: 84387
2009-10-18 02:05:42 +00:00
Daniel Dunbar
92fcc49c24 Don't traverse into .svn directories.
llvm-svn: 82978
2009-09-28 07:32:45 +00:00
Daniel Dunbar
b8a623820a lit: When executing shell scripts internally, don't allow piped stderr on any
commands except the last one, instead redirect the stderr to a temporary
file. This sidesteps a potential deadlocking issue.

llvm-svn: 82538
2009-09-22 09:50:38 +00:00
Daniel Dunbar
06695b3ea1 Add a magic LLVM_DISABLE_CRT_DEBUG environment variable which we check in RegisterHandler and use to disable the Win32 crash dialogs. These are a major blocker to any kind of automated testing.
Also, tweak the 'lit' test runner to set this variable unconditionally.

llvm-svn: 82537
2009-09-22 09:50:28 +00:00
Daniel Dunbar
3ea67f7fdc Actually use the arguments with the resolved executable path.
llvm-svn: 82527
2009-09-22 06:09:13 +00:00
Daniel Dunbar
797bf7ca92 lit: Don't use close_fds=True on Windows.
llvm-svn: 82521
2009-09-22 04:44:37 +00:00
Daniel Dunbar
79874e8a8e lit: When executing commands internally, perform PATH resolution ourselves.
llvm-svn: 82520
2009-09-22 04:44:26 +00:00
Daniel Dunbar
dadd71259e lit: Add a custom test format for use in clang.
llvm-svn: 81987
2009-09-16 01:34:52 +00:00
Daniel Dunbar
3d79f78d69 lit: When finding nested test suites, check first in the execpath in case there
is a site configuration.

llvm-svn: 81902
2009-09-15 20:09:17 +00:00
Daniel Dunbar
2808cdfdbe lit: Give test formats control over test discovery.
llvm-svn: 81751
2009-09-14 02:38:46 +00:00
Daniel Dunbar
e19b42b927 tests: Add a %abs_tmp substitution which is guaranteed to be a full path.
llvm-svn: 81662
2009-09-13 01:39:50 +00:00
Daniel Dunbar
a2c2d27cb7 Count test correctly with -q.
llvm-svn: 81313
2009-09-09 02:41:42 +00:00
Daniel Dunbar
338ea74b7e Fix another refactoro.
llvm-svn: 81312
2009-09-09 02:41:32 +00:00
Daniel Dunbar
ca1a6d222f lit needs bash for tcl-as-sh execution, we use set -o pipefail.
llvm-svn: 81197
2009-09-08 06:08:07 +00:00
Daniel Dunbar
57ad7988fd Fix typo that worked on python 2.6.
Also, fix unit tests.

llvm-svn: 81194
2009-09-08 05:46:28 +00:00
Daniel Dunbar
e82e000211 Fix a refactoro.
llvm-svn: 81192
2009-09-08 05:37:51 +00:00
Daniel Dunbar
640436d7af Add 'lit' testing tool.
- make install && man $(llvm-config --prefix)/share/man/man1/lit.1 for more
   information.

llvm-svn: 81190
2009-09-08 05:31:18 +00:00