656 Commits

Author SHA1 Message Date
Serge Guelton
d6bec11b3b Add Support for Creating and Deleting Unicode Files and Directories in Lit
This enables lit to work with unicode file names via mkdir, rm, and redirection.
Lit still uses utf-8 internally, but converts to utf-16 on Windows, or just utf-8
bytes on everything else.

Committed on behalf of Jason Mittertreiner
Differential Revision: https://reviews.llvm.org/D56754



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355122 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-28 19:16:17 +00:00
Dmitri Gribenko
ce3f788c40 Fixed typos in a test: s/CEHCK/CHECK/
Reviewers: ilya-biryukov, serge-sans-paille

Subscribers: delcypher, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58613

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355080 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-28 10:58:13 +00:00
Julian Lettner
0ffe848c65 [lit] Allow setting parallelism groups to None
Check that we do not crash if a parallelism group is explicitly set to
None. Permits usage of the following pattern.

[lit.common.cfg]
  lit_config.parallelism_groups['my_group'] = None
  if <condition>:
    lit_config.parallelism_groups['my_group'] = 3

[project/lit.cfg]
  config.parallelism_group = 'my_group'

Reviewers: rnk

Differential Revision: https://reviews.llvm.org/D58305

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354912 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-26 19:03:26 +00:00
Julian Lettner
2cb3057fcb [lit] Remove LitTestCase
From the docs: `class LitTestCase(unittest.TestCase)`
LitTestCase is an adaptor for providing a 'unittest' compatible
interface to 'lit' tests so that we can run lit tests with standard
python test runners.

It does not seem to be used anywhere.

Differential Revision: https://reviews.llvm.org/D58264

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354188 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-16 00:44:00 +00:00
Julian Lettner
0631be28f6 [lit][NFC] Cleanup lit worker process handling
Move code that is executed on worker process to separate file. This
makes the use of the pickled arguments stored in global variables in the
worker a bit clearer. (Still not pretty though.)

Extract handling of parallelism groups to it's own function.

Use BoundedSemaphore instead of Semaphore. BoundedSemaphore raises for
unmatched release() calls.

Cleanup imports.

Differential Revision: https://reviews.llvm.org/D58196

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354187 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-16 00:40:40 +00:00
Julian Lettner
747940fe8d [lit][NFC] Cleanup copy&paste naming mistake
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354095 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-15 02:44:52 +00:00
Julian Lettner
d8ffa5fe1b [lit] Remove --single-process option (use -j1 instead)
Remove `--single-process` command line option. Use `-j1` instead.

Also see commit: 96adb78b120b6aa9739eb714534dc8e819f7bc52

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354073 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-14 22:46:56 +00:00
Julian Lettner
75b1901bac [lit] Set --single-process for single tests and --threads=1
Summary:
Automatically upgrade debugging experience (single process, no thread
pool) when:
  1) we only run a single test
  2) user specifies `-j1`

Details:
Fix `--max-failures` in single process mode. Option did not have an
effect in single process mode.

Add display feedback for single process mode. Adapted test.

Improve argument checking (require positive integers).

`--single-process` is now essentially an alias for `-j1`. Should we
remove it?

Reviewers: rnk

Differential Revision: https://reviews.llvm.org/D58249

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@354068 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-14 22:30:07 +00:00
Nico Weber
a53d6c2052 lit: Let lit.util.which() return a normcase()ed path
LLVMConfig.with_environment() uses os.path.normcase(os.path.normpath(x)) to
normalize temporary env vars. LLVMConfig.use_clang() uses with_environment() to
temporarily set PATH and then look for clang there. This means that on Windows,
clang will be run with a path like c:\foo\bin\clang.EXE (with a lower-case
"C:").

lit.util.which() used to not do this, which means the executables added in
clang/test/lit.cfg.py (e.g. c-index-test) were run with a path like
C:\foo\bin\c-index-test.EXE (because both CMake and GN happen to write
clang_tools_dir with an upper-case C to lit.site.cfg.py).

clang/test/Index/pch-from-libclang.c requires that both c-index-test and clang
use _exactly_ the same resource dir path (same case and everything), because a
hash of the resource directory is used as module cache path.

This patch is necessary but not sufficient to make pch-from-libclang.c pass on
Windows.

Differential Revision: https://reviews.llvm.org/D57343


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@352704 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-31 00:40:43 +00:00
James Y Knight
b21d80e08f Adjust documentation for git migration.
This fixes most references to the paths:
 llvm.org/svn/
 llvm.org/git/
 llvm.org/viewvc/
 github.com/llvm-mirror/
 github.com/llvm-project/
 reviews.llvm.org/diffusion/

to instead point to https://github.com/llvm/llvm-project.

This is *not* a trivial substitution, because additionally, all the
checkout instructions had to be migrated to instruct users on how to
use the monorepo layout, setting LLVM_ENABLE_PROJECTS instead of
checking out various projects into various subdirectories.

I've attempted to not change any scripts here, only documentation. The
scripts will have to be addressed separately.

Additionally, I've deleted one document which appeared to be outdated
and unneeded:
  lldb/docs/building-with-debug-llvm.txt

Differential Revision: https://reviews.llvm.org/D57330

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@352514 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-29 16:37:27 +00:00
Eric Fiselier
bda64db50e make XFAIL, REQUIRES, and UNSUPPORTED support multi-line expressions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@351668 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-20 00:51:02 +00:00
Chandler Carruth
4cfe1aca69 Update structured references to the license to the new license.
Since these are intended to be short and succinct, I've used the SPDX
full name. It's human readable, but formally agreed upon and will be
part of the SPDX spec for licenses.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@351649 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-19 11:30:51 +00:00
Dan Liew
ba41586e36 [lit] Make it possible for the lit test suite to pass with
`FILECHECK_OPTS=-v` set in the environment.

Follow up to r350850 as requested by Joel E. Denny in
https://reviews.llvm.org/D56541 .

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350854 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-10 17:47:44 +00:00
David Greene
e410b0a76b [lit] Respect PYTHONPATH
If a user has PYTHONPATH set in the environment, append new entries to
it rather than blindly setting PYTHONPATH to a fixed string. This
allows tests to, for example, find psutil if it is in
PYTHONPATH. Without this change, lit will detect psutil but then
various tests will fail because PYTHONPATH has been overwritten and
psutil cannot be found.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350536 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-07 16:24:37 +00:00
Serge Guelton
60ccceba74 Python compat - print statement
Make sure all print statements are compatible with Python 2 and Python3 using
the `from __future__ import print_function` statement.

Differential Revision: https://reviews.llvm.org/D56249

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350307 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-03 14:11:33 +00:00
Nico Weber
576e6bc9fb Revert 349677, it contained a whole bunch of stuff I did not mean to commit
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@349678 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-19 20:19:58 +00:00
Nico Weber
fc16245746 [gn build] Add build file for clang/lib/Basic and dependencies
Adds a build file for clang-tblgen and an action for running it, and uses that
to process all the .td files in include/clang/Basic.

Also adds an action to write include/clang/Config/config.h and
include/clang/Basic/Version.inc.

Differential Revision: https://reviews.llvm.org/D55847


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@349677 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-19 20:18:59 +00:00
Michal Gorny
8dc1a1ab0b [test] Split strip-preserve-time.test, and skip atime test on NetBSD
Split timestamp preservation tests into atime and mtime test, and skip
the former on NetBSD.  When the filesystem is mounted noatime, NetBSD
not only inhibits implicit atime updates but also prevents setting atime
via utime(), causing the test to fail.

Differential Revision: https://reviews.llvm.org/D55271

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@348354 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-05 11:15:46 +00:00
Zachary Turner
c3ceb23729 [lit] Add a generic build script with a lit substitution.
This adds a script called build.py as well as a lit substitution
called %build that we can use to invoke it.  The idea is that
this allows a lit test to build test inferiors without having
to worry about architecture / platform specific differences,
command line syntax, finding / configurationg a proper toolchain,
and other issues.  They can simply write something like:

%build --arch=32 -o %t.exe %p/Inputs/foo.cpp

and it will just work.  This paves the way for being able to
run lit tests with multiple configurations, platforms, and
compilers with a single test.

Differential Revision: https://reviews.llvm.org/D54914

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@348058 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-01 00:22:21 +00:00
Zachary Turner
5b841f76e4 [lit] Pass more environment variables through to child processes.
This arose when I was trying to have a substitution which invoked a
python script P, and that python script tried to invoke clang-cl (or
even cl). Since we invoke P with a custom environment, it doesn't
inherit the environment of the parent, and then when we go to invoke
clang-cl, it's unable to find the MSVC installation directory. There
were many more I could have passed through which are set by vcvarsall,
but I tried to keep it simple and only pass through the important ones.

Differential Revision: https://reviews.llvm.org/D54963

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347691 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-27 19:29:12 +00:00
Reid Kleckner
a4eddf6b39 Fix clang test suite on Windows by reverting part of r347216
Otherwise, the clang analyzer tests fail on Windows when attempting to
unpickle AnalyzerTest objects in the worker processes. The pattern of,
add to path, import, remove from path, serialize, deserialize, doesn't
work. Once something gets added to the path, if we want to move it
across the wire for multiprocessing, we need to keep the module on
sys.path.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347254 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-19 19:36:28 +00:00
Zachary Turner
824fe505a4 Fix some issues with LLDB's lit configuration files.
Recently I tried to port LLDB's lit configuration files over to use a
on the surface, but broke some cases that weren't broken before and also
exposed some additional problems with the old approach that we were just
getting lucky with.

When we set up a lit environment, the goal is to make it as hermetic as
possible. We should not be relying on PATH and enabling the use of
arbitrary shell commands. Instead, only whitelisted commands should be
allowed. These are, generally speaking, the lit builtins such as echo,
cd, etc, as well as anything for which substitutions have been
explicitly set up for. These substitutions should map to the build
output directory, but in some cases it's useful to be able to override
this (for example to point to an installed tools directory).

This is, of course, how it's supposed to work. What was actually
happening is that we were bringing in PATH and LD_LIBRARY_PATH and then
just running the given run line as a shell command. This led to problems
such as finding the wrong version of clang-cl on PATH since it wasn't
even a substitution, and flakiness / non-determinism since the
environment the tests were running in would change per-machine. On the
other hand, it also made other things possible. For example, we had some
tests that were explicitly running cl.exe and link.exe instead of
clang-cl and lld-link and the only reason it worked at all is because it
was finding them on PATH. Unfortunately we can't entirely get rid of
these tests, because they support a few things in debug info that
clang-cl and lld-link don't (notably, the LF_UDT_MOD_SRC_LINE record
which makes some of the tests fail.

The high level changes introduced in this patch are:

1. Removal of functionality - The lit test suite no longer respects
   LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER. This means there is no
   more support for gcc, but nobody was using this anyway (note: The
   functionality is still there for the dotest suite, just not the lit test
   suite). There is no longer a single substitution %cxx and %cc which maps
   to <arbitrary-compiler>, you now explicitly specify the compiler with a
   substitution like %clang or %clangxx or %clang_cl. We can revisit this
   in the future when someone needs gcc.

2. Introduction of the LLDB_LIT_TOOLS_DIR directory. This does in spirit
   what LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER used to do, but now
   more friendly. If this is not specified, all tools are expected to be
   the just-built tools. If it is specified, the tools which are not
   themselves being tested but are being used to construct and run checks
   (e.g. clang, FileCheck, llvm-mc, etc) will be searched for in this
   directory first, then the build output directory.

3. Changes to core llvm lit files. The use_lld() and use_clang()
   functions were introduced long ago in anticipation of using them in
   lldb, but since they were never actually used anywhere but their
   respective problems, there were some issues to be resolved regarding
   generality and ability to use them outside their project.

4. Changes to .test files - These are all just replacing things like
   clang-cl with %clang_cl and %cxx with %clangxx, etc.

5. Changes to lit.cfg.py - Previously we would load up some system
   environment variables and then add some new things to them. Then do a
   bunch of work building out our own substitutions. First, we delete the
   system environment variable code, making the environment hermetic. Then,
   we refactor the substitution logic into two separate helper functions,
   one which sets up substitutions for the tools we want to test (which
   must come from the build output directory), and another which sets up
   substitutions for support tools (like compilers, etc).

6. New substitutions for MSVC -- Previously we relied on location of
   MSVC by bringing in the entire parent's PATH and letting
   subprocess.Popen just run the command line. Now we set up real
   substitutions that should have the same effect. We use PATH to find
   them, and then look for INCLUDE and LIB to construct a substitution
   command line with appropriate /I and /LIBPATH: arguments. The nice thing
   about this is that it opens the door to having separate %msvc-cl32 and
   %msvc-cl64 substitutions, rather than only requiring the user to run
   vcvars first. Because we can deduce the path to 32-bit libraries from
   64-bit library directories, and vice versa. Without these substitutions
   this would have been impossible.

Differential Revision: https://reviews.llvm.org/D54567

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@347216 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-19 15:12:34 +00:00
Joel E. Denny
e861874109 [FileCheck] Parse command-line options from FILECHECK_OPTS
This feature makes it easy to tune FileCheck diagnostic output when
running the test suite via ninja, a bot, or an IDE.  For example:

```
$ FILECHECK_OPTS='-color -v -dump-input-on-failure' \
  LIT_FILTER='OpenMP/for_codegen.cpp' ninja check-clang \
  | less -R
```

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D53517

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346272 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-06 22:07:03 +00:00
Zachary Turner
4860c62539 Refactor the lit configuration files
A year or so ago, I re-wrote most of the lit infrastructure in LLVM so
that it wasn't so boilerplate-y. I added lots of common helper type
stuff, simplifed usage patterns, and made the code more elegant and
maintainable.

We migrated to this in LLVM, clang, and lld's lit files, but not in
LLDBs. This started to bite me recently, as the 4 most recent times I
tried to run the lit test suite in LLDB on a fresh checkout the first
thing that would happen is that python would just start crashing with
unhelpful backtraces and I would have to spend time investigating.

You can reproduce this today by doing a fresh cmake generation, doing
ninja lldb and then python bin/llvm-lit.py -sv ~/lldb/lit/SymbolFile at
which point you'll get a segfault that tells you nothing about what your
problem is.

I started trying to fix the issues with bandaids, but it became clear
that the proper solution was to just bring in the work I did in the rest
of the projects. The side benefit of this is that the lit configuration
files become much cleaner and more understandable as a result.

Differential Revision: https://reviews.llvm.org/D54009

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346008 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-02 17:49:01 +00:00
Greg Bedwell
ee6a84b4fe [lit] Only return a found bash executable on Windows if it can understand Windows paths
Some versions of bash.exe, for example WSL's version expect paths in the form
/mnt/c/path/to/dir rather than c:\\path\\to\\dir so will cause failures
for any tests that require an external shell if used by lit.  If we're on
Windows and looking for an external shell, check that the found version
of bash is able to parse a native path before returning that version.

This patch also partially reverts the behaviour of r228221 by
restoring the warning if bash cannot be found.  This shouldn't pollute
the lit stderr anymore as we're now using internal shell by default on
Windows.  If someone is explicitly specifying to use an external shell, it's
probably worth alerting them to the fact that bash could not be found.

Differential Revision: https://reviews.llvm.org/D52831

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@345019 91177308-0d34-0410-b5e6-96231b3b80d8
2018-10-23 11:34:04 +00:00
Chris Matthews
672a178125 make lit builtins a package
cat.py is not being installed when lit is installed from source. So
tests that use the internal shell fail when using cat.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343347 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-28 17:55:18 +00:00
Tom Stellard
e9139276f3 lit: Make sure the builtin_commands directory is packaged by setup.py
Summary: This directory was missing from the lit package on pypi.org.

Reviewers: ddunbar

Subscribers: delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D51670

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@343115 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-26 14:56:11 +00:00
Alex Langford
7b77a6b5d2 Fix lit/example/many-tests pickling issue
Summary:
The multiprocess module uses pickling to transfer
information between processes and does not know how to pickle
the class created in the lit.cfg file and thus the example
fails.

Implement ManyTests in a separate file and import for the
example test passes

Patch by Nathan Lanza <nathan@lanza.io>

Differential Revision: https://reviews.llvm.org/D51328

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342269 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-14 19:44:09 +00:00
Stella Stamenova
ca3502c26e [lit] Disable shtest-timeout on Windows
Summary: This is the only test that is still failing on Windows - or rather, it is expected to fail on the bots, but passes on the new bot that we're preparing causing a failure, so I'm going to disable it. Since the test has rarely, if ever, passed on the bots, this should have the same effect and it will unblock the creation of the new bot.

Reviewers: asmith, delcypher, zturner

Subscribers: stella.stamenova, llvm-commits

Differential Revision: https://reviews.llvm.org/D51871

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341856 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-10 20:24:05 +00:00
Tom Stellard
c523f7a122 lit: Use sys.executable for executing builtin commands
Summary:
The python executable may not exist on all systems so use sys.executable
instead.

Reviewers: ddunbar, stella.stamenova

Subscribers: delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D51511

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341244 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-31 20:15:31 +00:00
Stella Stamenova
5c077985f3 [lit, shtest-timeout] Always use an internal shell for the shtest-timeout to diagnose buildbot failures
Summary:
Right now this test is failing on the builtbots on Windows but we have a very similar setup where the test passes. The test is meant to test that specifying a timeout works correctly by running an infnite loop and having it timeout - on the buildbot, the infinite loop doesn't actually execute. This change runs all of the tests in the set using an internal shell rather than an external shell. I expect this will make the test pass which means that either the way the external shell is invoked or the external shell setup on the buildbots is not correct. Regardless of whether the test passes with this change, we'll need to undo this change and have a real fix.

@gkistanova was able to get logs from the buildbot to rule out a number of theories as to why this test is failing, but they didn't have enough information to confirm exactly what the issue is. The purpose of this change is to narrow it down, but if someone has a local repro and can aid in debugging, that would make it much speedier (and less prone to making the bots fail).

Reviewers: gkistanova, asmith, zturner, modocache, rnk, delcypher

Reviewed By: rnk

Subscribers: delcypher, llvm-commits, gkistanova

Differential Revision: https://reviews.llvm.org/D51326

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@340840 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-28 16:24:55 +00:00
Petr Hosek
13ced78da1 [CMake] Use normalized Windows target triples
Changes the default Windows target triple returned by
GetHostTriple.cmake from the old environment names (which we wanted to
move away from) to newer, normalized ones. This also requires updating
all tests to use the new systems names in constraints.

Differential Revision: https://reviews.llvm.org/D47381

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339307 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-09 02:16:18 +00:00
Stella Stamenova
ab3088f741 [lit] Disable shtest-timeout on Windows
This test passes on Windows when using Python 3 but fails when using Python 2, so it needs more investigation before it can be enabled as the bots use Python 2.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339184 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-07 21:21:30 +00:00
Stella Stamenova
638925075c [lit, python3] Update lit error logging to work correctly in python3 and other test fixes
Summary:
In Python2 'unicode' is a distinct type from 'str', but in Python3 'unicode' does not exist and instead all 'str' objects are Unicode string. This change updates the logic in the test logging for lit to correctly process each of the types, and more importantly, to not just fail in Python3.

This change also reverses the use of quotes in several of the cfg files. By using '""' we are guaranteeing that the resulting path will work correctly on Windows while "''" only works correctly sometimes. This also fixes one of the failing tests.

Reviewers: asmith, zturner

Subscribers: stella.stamenova, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D50397

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339179 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-07 20:54:38 +00:00
Stella Stamenova
ca626b3197 [lit, tests] Fix failing lit test: shtest-format.py
Summary:
The problem here is that on windows double quotes are used for paths (usually) while single quotes are not. This is not generally a problem for the tests because the lit infrastructure tends to treat both the same. One (and possibly only) exception is when some tests are run in an external shell such as some of the shtest-format tests. In this case on windows the path to python was not created correctly because it had single quotes and the test failed.

This same test is already failing with python 3 which is why our testing missed the new failure. This patch will take care of the immediate failure with python 2 and I'll send a follow up for the python 3 failure.

Reviewers: asmith, zturner

Subscribers: delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D50373

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339091 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-07 04:08:46 +00:00
Stella Stamenova
a27199db06 [lit, python] Always add quotes around the python path in lit
Summary:
The issue with the python path is that the path to python on Windows can contain spaces. To make the tests always work, the path to python needs to be surrounded by quotes.

This change updates several configuration files which specify the path to python as a substitution and also remove quotes from existing tests.

Reviewers: asmith, zturner, alexshap, jakehehrlich

Reviewed By: zturner, alexshap, jakehehrlich

Subscribers: mehdi_amini, nemanjai, eraman, kbarton, jakehehrlich, steven_wu, dexonsmith, stella.stamenova, delcypher, llvm-commits

Differential Revision: https://reviews.llvm.org/D50206

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339073 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-06 22:37:44 +00:00
Hans Wennborg
c6ba19258a lit: bump version number
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338677 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-02 07:48:44 +00:00
Justin Bogner
7550101af9 [lit] Move the shtest-xunit-output check lines into shtest-format
These two tests are operating on the same test suite, which causes
them to be racy about writing temporary files and can cause spurious
failures. Merge them into one test to avoid the issue.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337718 91177308-0d34-0410-b5e6-96231b3b80d8
2018-07-23 18:08:36 +00:00
George Karpenkov
4571da68d1 [FileCheck] Provide an option for FileCheck to dump original input to stderr on failure
The option can be either set using environment variable (e.g. env
FILECHECK_DUMP_INPUT_ON_FAILURE=1 ninja check-fuzzer) or with a
FileCheck flag.

This can be extremely useful for debugging, cf.
https://groups.google.com/forum/#!topic/llvm-dev/kLrzg8OM_h8 for
discussion.

Differential Revision: https://reviews.llvm.org/D49328

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337609 91177308-0d34-0410-b5e6-96231b3b80d8
2018-07-20 20:21:57 +00:00
Hans Wennborg
c4f3d9f40d Relax shtest-run-at-line.py
The test was failing on Windows machines which had bash.exe on PATH (but
not in the so called lit tools dir, containing cmp.exe, grep.exe etc.).

The problem was that the outer lit invocation would load LLVMConfig
from utils/lit/lit/llvm/config.py, which looks up the tools path with
getToolsPath(). That has a surprising side effect of also setting
bashPath, in our case setting it to empty.

The outer lit invocation would thus configure the pdbg0 and pdbg1
substitutions based on not running with bash.

But the inner lit invocation would not load LLVMConfig, so bash
would be found on PATH, that would be used as external shell,
and so the output wouldn't match pdbg0 and pdbg1.

It seems weird to me that getBashPath() will return different results
depending on whether getToolsPath() has been called before, but I
also don't know how to fix it properly.

This commit just relaxes the test case, because there doesn't seem
to be much point in testing for the exact syntax of the run file
as long as it works.

(See https://crbug.com/850023)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334100 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-06 14:53:03 +00:00
Vlad Tsyrklevich
26b7b194d7 [Analyzer] Fix the Z3 lit test config
Summary:
The '%analyze' extra_args config argument seems to have been erroneously
deleted in r315627 disabling Z3 tests for the clang analyzer. Add the
flag back.

Reviewers: george.karpenkov, NoQ, ddcc

Reviewed By: george.karpenkov

Subscribers: xazax.hun, szepet, delcypher, a.sidorin, llvm-commits

Differential Revision: https://reviews.llvm.org/D47722

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334066 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-06 06:25:37 +00:00
Joel E. Denny
deabcc51dc [lit] Fix windows cmd.exe test config for r333620
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333630 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-31 05:48:33 +00:00
Joel E. Denny
1d4f2158ac [lit] Terminate ": RUN at line N" with ";" not "&&"
This fixes projects/compiler-rt/test/fuzzer/sigusr.test, which was
broken by r333614.  The trouble was that "&&" changes the command for
which "$!" gives the pid.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333620 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-31 03:40:37 +00:00
Joel E. Denny
1d95844557 [lit] Report line number for failed RUN command
(Relands r333584, reverted in 333592.)

When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

To support reporting RUN line numbers in the case of windows cmd.exe
as the external shell, this patch extends -vv to set "echo on" instead
of "echo off" in bat files.  (Support for windows cmd.exe as a lit
external shell will likely be dropped later, but I found out too
late.)

Reviewed By: delcypher,	asmith, stella.stamenova, jmorse, lebedev.ri, rnk

Differential Revision: https://reviews.llvm.org/D44598

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333614 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-31 00:55:32 +00:00
Joel E. Denny
22634021a1 Revert r333584: [lit] Report line number for failed RUN command
It breaks test-suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333592 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-30 21:07:27 +00:00
Joel E. Denny
e7c7bc039f [lit] Report line number for failed RUN command
(Relands r330755 (reverted in r330848) with fix for PR37239.)

When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed.  For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.

When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.

To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.

To support reporting RUN line numbers in the case of windows cmd.exe
as the external shell, this patch extends -vv to set "echo on" instead
of "echo off" in bat files.  (Support for windows cmd.exe as a lit
external shell will likely be dropped later, but I found out too
late.)

Reviewed By: delcypher,	asmith, stella.stamenova, jmorse, lebedev.ri, rnk

Differential Revision: https://reviews.llvm.org/D44598

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333584 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-30 19:42:27 +00:00
Chris Matthews
7feb37bec2 Use quoteattr to ensure we make well formed attributes
We were making malformed XML on tests with ' in the name.  Switch to
using saxutils to set all of our attributes, so it can handle quotes
etc correctly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333249 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-25 00:31:36 +00:00
Dan Liew
9f0686b667 [lit] Try to make shtest-timeout.py test more reliable by using a
larger timeout value. This really isn't very good because it will
still be susceptible to machine performance.

While we are here also fix a bug in validation of
`maxIndividualTestTime` where previously it wasn't checked if the
type was an int.

rdar://problem/40221572

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332987 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-22 15:06:29 +00:00
Dan Liew
8cf26099d7 [lit] Don't run slow.py in shtest-timeout.py test.
The program used to be used in `quick_then_slow.py` but that was
removed in r328702. The tests always run `slow.py` on its own but
this doesn't really test additional code so we'll just drop running
`slow.py` so the tests run faster.

rdar://problem/40221572

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332986 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-22 15:06:24 +00:00
Dan Liew
096b48f514 [lit] Don't check output of commands used in shtest-timeout.py test.
If the system is under heavy load 1 second might not be long enough
for it to produce output which could lead to spurious test failures.
What matters is that the right test cases reach a timeout.

rdar://problem/40221572

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332985 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-22 15:06:20 +00:00