547 Commits

Author SHA1 Message Date
Andrew Ng
a7d2ce0679 [Support] MemoryBlock size should reflect the requested size
This patch mirrors the change made to the Unix equivalent in
r351916. This in turn fixes bugs related to the use of FileOutputBuffer
to output to "-", i.e. stdout, on Windows.

Differential Revision: https://reviews.llvm.org/D59663

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@357058 91177308-0d34-0410-b5e6-96231b3b80d8
2019-03-27 10:26:21 +00:00
Alexandre Ganea
79606882d3 Fix SupportTests.exe/AllocationTests/MappedMemoryTest.AllocAndReleaseHuge when the machine doesn't have large pages enabled.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355067 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-28 03:42:07 +00:00
Alexandre Ganea
0a24ac1326 [Memory] Add basic support for large/huge memory pages
This patch introduces Memory::MF_HUGE_HINT which indicates that allocateMappedMemory() shall return a pointer to a large memory page.
However the flag is a hint because we're not guaranteed in any way that we will get back a large memory page. There are several restrictions:

- Large/huge memory pages aren't enabled by default on modern OSes (Windows 10 and Linux at least), and should be manually enabled/reserved.
- Once enabled, it should be kept in mind that large pages are physical only, they can't be swapped.
- Memory fragmentation can affect the availability of large pages, especially after running the OS for a long time and/or running along many other applications.

Memory::allocateMappedMemory() will fallback to 4KB pages if it can't allocate 2MB large pages (if Memory::MF_HUGE_HINT is provided)

Currently, Memory::MF_HUGE_HINT only works on Windows. The hint will be ignored on Linux, 4KB pages will always be returned.

Differential Revision: https://reviews.llvm.org/D58718

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@355065 91177308-0d34-0410-b5e6-96231b3b80d8
2019-02-28 02:47:34 +00:00
Chandler Carruth
6b547686c5 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@351636 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-19 08:50:56 +00:00
Zachary Turner
51d88162e4 Don't write #include "Windows/WindowsSupport.h" from the Windows dir.
This generates -Wnonportable-include-dir warnings, and doesn't need
to be there.  It seems this was just checked in on accident.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@350655 91177308-0d34-0410-b5e6-96231b3b80d8
2019-01-08 21:05:34 +00:00
Shoaib Meenai
c94004ffdb [Support] Fix FileNameLength passed to SetFileInformationByHandle
The rename_internal function used for Windows has a minor bug where the
filename length is passed as a character count instead of a byte count.
Windows internally ignores this field, but other tools that hook NT
api's may use the documented behavior:

MSDN documentation specifying the size should be in bytes:
https://docs.microsoft.com/en-us/windows/desktop/api/winbase/ns-winbase-_file_rename_info

Patch by Ben Hillis.

Differential Revision: https://reviews.llvm.org/D55624

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@348995 91177308-0d34-0410-b5e6-96231b3b80d8
2018-12-13 00:08:25 +00:00
Jonas Devlieghere
9d76438276 [FileSystem] Add expand_tilde function
In D54435 there was some discussion about the expand_tilde flag for
real_path that I wanted to expose through the VFS. The consensus is that
these two things should be separate functions. Since we already have the
code for this I went ahead and added a function expand_tilde that does
just that.

Differential revision: https://reviews.llvm.org/D54448

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346776 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-13 18:23:32 +00:00
Reid Kleckner
db36e6f421 [Windows] Simplify WindowsSupport.h
Sink Windows version detection code from WindowsSupport.h to Path.inc.
These functions don't need to be inlined. I randomly picked Process.inc
for the Windows version helpers, since that's the most related file.

Sink MakeErrMsg to Program.inc since it's the main client.

Move those functions into the llvm namespace, and delete the scoped
handle copy and assignment operators.

Reviewers: zturner, aganea

Differential Revision: https://reviews.llvm.org/D54182

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346280 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-06 23:39:59 +00:00
Reid Kleckner
0bbaef20df Silence deprecation warning for GetVersionEx with clang-cl
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346268 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-06 21:40:32 +00:00
Martin Storsjo
5bc1446f3b [Support] Fix warning: unknown pragma ignored for mingw target
Differential Revision: https://reviews.llvm.org/D54133

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346218 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-06 09:08:20 +00:00
Alexandre Ganea
c0bf8d6c82 Only call FlushFileBuffers() when writing executables on Windows
This is a follow-up for "r325274: Call FlushFileBuffers on output files."

Previously, FlushFileBuffers() was called in all cases when writing a file. The objective was to go around a bug in the Windows kernel (as described here: https://randomascii.wordpress.com/2018/02/25/compiler-bug-linker-bug-windows-kernel-bug/). However that is required only when writing EXEs, any other file type doesn't need flushing.

This patch calls FlushFileBuffers() only for EXEs. In addition, we completly disable FlushFileBuffers() for known Windows 10 versions that do not exhibit the original kernel bug.

Differential Revision: https://reviews.llvm.org/D53727

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@346152 91177308-0d34-0410-b5e6-96231b3b80d8
2018-11-05 19:14:10 +00:00
Nico Weber
cb24b2be15 Remove dead function user_cache_directory()
It's been unused since it was added almost 3 years ago in
https://reviews.llvm.org/D13801

Motivated by https://reviews.llvm.org/rL342002 since it removes one of the
functions keeping a ref to SHGetKnownFolderPath.

Differential Revision: https://reviews.llvm.org/D52184


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342485 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-18 15:06:16 +00:00
Kristina Brooks
90f4ccc2c5 [Support] sys::fs::directory_entry includes the file_type.
This is available on most platforms (Linux/Mac/Win/BSD) with no extra syscalls.
On other platforms (e.g. Solaris) we stat() if this information is requested.

This will allow switching clang's VFS to efficiently expose (path, type) when
traversing a directory. Currently it exposes an entire Status, but does so by
calling fs::status() on all platforms.
Almost all callers only need the path, and all callers only need (path, type).

Patch by sammccall (Sam McCall)

Differential Revision: https://reviews.llvm.org/D51918



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@342089 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-12 22:08:10 +00:00
Reid Kleckner
e207c27bf5 [Support] Quote arguments containing \n on Windows
Fixes at_file.c test failure caused by r341988. We may want to change
how we treat \n in our tokenizer, but this is probably a good fix
regardless, since we can invoke all kinds of programs with different
interpretations of the command line quoting rules.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341992 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-11 21:02:03 +00:00
Reid Kleckner
547c92f475 [Support] Avoid calling CommandLineToArgvW from shell32.dll
Summary:
Shell32.dll depends on gdi32.dll and user32.dll, which are mostly DLLs
for Windows GUI functionality. LLVM's utilities don't typically need GUI
functionality, and loading these DLLs seems to be slowing down startup.
Also, we already have an implementation of Windows command line
tokenization in cl::TokenizeWindowsCommandLine, so we can just use it.

The goal is to get the original argv in UTF-8, so that it can pass
through most LLVM string APIs. A Windows process starts life with a
UTF-16 string for its command line, and it can be retreived with
GetCommandLineW from kernel32.dll.

Previously, we would:
1. Get the wide command line
2. Call CommandLineToArgvW to handle quoting rules and separate it into
   arguments.
3. For each wide argument, expand wildcards (* and ?) using
   FindFirstFileW.
4. Convert each argument to UTF-8

Now we:
1. Get the wide command line, convert the whole thing to UTF-8
2. Tokenize the UTF-8 command line with cl::TokenizeWindowsCommandLine
3. For each argument, expand wildcards if present
   - This requires converting back to UTF-16 to call FindFirstFileW
   - Results of FindFirstFileW must be converted back to UTF-8

Reviewers: zturner

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51941

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341988 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-11 20:22:39 +00:00
David Bolvansky
810335c3b5 Set console mode when -fansi-escape-codes is enabled
Summary:
Windows console now supports supports ANSI escape codes, but we need to enable it using SetConsoleMode with ENABLE_VIRTUAL_TERMINAL_PROCESSING flag.

Fixes https://bugs.llvm.org/show_bug.cgi?id=38817


Tested on Windows 10, screenshot:
https://i.imgur.com/bqYq0Uy.png

Reviewers: zturner, chandlerc

Reviewed By: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D51611

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@341396 91177308-0d34-0410-b5e6-96231b3b80d8
2018-09-04 19:23:05 +00:00
Jordan Rupprecht
f0b29b88a4 [Support] NFC: Allow modifying access/modification times independently in sys::fs::setLastModificationAndAccessTime.
Summary:
Add an overload to sys::fs::setLastModificationAndAccessTime that allows setting last access and modification times separately. This will allow tools to use this API when they want to preserve both the access and modification times from an input file, which may be different.

Also note that both the POSIX (futimens/futimes) and Windows (SetFileTime) APIs take the two timestamps in the order of (1) access (2) modification time, so this renames the method to "setLastAccessAndModificationTime" to make it clear which timestamp is which.

For existing callers, the 1-arg overload just sets both timestamps to the same thing.

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D50521

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339628 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-13 23:03:45 +00:00
Jeremy Morse
8602af619e [Windows FS] Allow moving files in TempFile::keep
In r338216 / D49860 TempFile::keep was extended to allow keeping across
filesystems. The aim on Windows was to have this happen in rename_internal
using the existing system API. However, to fix an issue and preserve the
idea of "renaming" not being a move, put Windows keep-across-filesystem in
TempFile::keep.

Differential Revision: https://reviews.llvm.org/D50048



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338841 91177308-0d34-0410-b5e6-96231b3b80d8
2018-08-03 10:13:35 +00:00
Jonas Devlieghere
58d4a39f63 [dsymutil] Simplify temporary file handling.
Dsymutil's update functionality was broken on Windows because we tried
to rename a file while we're holding open handles to that file. TempFile
provides a solution for this through its keep(Twine) method. This patch
changes dsymutil to make use of that functionality.

Differential revision: https://reviews.llvm.org/D49860

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338216 91177308-0d34-0410-b5e6-96231b3b80d8
2018-07-29 14:56:15 +00:00
Andrew Ng
645f3d0f00 [ThinLTO] Update ThinLTO cache file atimes when on Windows
ThinLTO cache file access times are used for expiration based pruning
and since Vista, file access times are not updated by Windows by
default:

https://blogs.technet.microsoft.com/filecab/2006/11/07/disabling-last-access-time-in-windows-vista-to-improve-ntfs-performance

This means on Windows, cache files are currently being pruned from
creation time. This change manually updates cache files that are
accessed by ThinLTO, when on Windows.

Patch by Owen Reynolds.

Differential Revision: https://reviews.llvm.org/D47266

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336276 91177308-0d34-0410-b5e6-96231b3b80d8
2018-07-04 14:17:10 +00:00
Peter Collingbourne
6965c8b761 LTO: Keep file handles open for memory mapped files.
On Windows we've observed that if you open a file, write to it, map it into
memory and close the file handle, the contents of the memory mapping can
sometimes be incorrect. That was what we did when adding an entry to the
ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing
intermittent build failures on Chromium's ThinLTO bots on Windows. More
details are in the associated Chromium bug (crbug.com/786127).

We can prevent this from happening by keeping a handle to the file open while
the mapping is active. So this patch changes the mapped_file_region class to
duplicate the file handle when mapping the file and close it upon unmapping it.

One gotcha is that the file handle that we keep open must not have been
created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system
will prevent other processes from opening the file. We can achieve this
by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether.  Instead,
we use SetFileInformationByHandle with FileDispositionInfo to manage the
delete-on-close bit. This lets us remove the hack that we used to use to
clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE.

A downside of using SetFileInformationByHandle/FileDispositionInfo as
opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using
CreateFile to open the file while the flag is set, even within the same
process. This doesn't seem to matter for almost every client of TempFile,
except for LockFileManager, which calls sys::fs::create_link to create a
hard link from the lock file, and in the process of doing so tries to open
the file. To prevent this change from breaking LockFileManager I changed it
to stop using TempFile by effectively reverting r318550.

Differential Revision: https://reviews.llvm.org/D48051

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334630 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-13 18:03:14 +00:00
Hans Wennborg
7c4ee81607 Do not enforce absolute path argv0 in windows
Even if we support no-canonical-prefix on
clang-cl(https://reviews.llvm.org/D47480), argv0 becomes absolute path
in clang-cl and that embeds absolute path in /showIncludes.

This patch removes such full path normalization from InitLLVM on
windows, and that removes absolute path from clang-cl output
(obj/stdout/stderr) when debug flag is disabled.

Patch by Takuto Ikuta!

Differential Revision https://reviews.llvm.org/D47578

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334602 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-13 14:29:26 +00:00
Zachary Turner
0dcc1159b4 Refactor ExecuteAndWait to take StringRefs.
This simplifies some code which had StringRefs to begin with, and
makes other code more complicated which had const char* to begin
with.

In the end, I think this makes for a more idiomatic and platform
agnostic API.  Not all platforms launch process with null terminated
c-string arrays for the environment pointer and argv, but the api
was designed that way because it allowed easy pass-through for
posix-based platforms.  There's a little additional overhead now
since on posix based platforms we'll be takign StringRefs which
were constructed from null terminated strings and then copying
them to null terminate them again, but from a readability and
usability standpoint of the API user, I think this API signature
is strictly better.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334518 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-12 17:43:52 +00:00
Zachary Turner
07d33bf7d7 Attempt 3: Resubmit "[Support] Expose flattenWindowsCommandLine."
I took some liberties and quoted fewer characters than before,
based on an article from MSDN which says that only certain characters
cause an arg to require quoting.  This seems to be incorrect, though,
and worse it seems to be a difference in Windows version.  The bot
that fails is Windows 7, and I can't reproduce the failure on Win
10.  But it's definitely related to quoting and special characters,
because both tests that fail have a * in the argument, which is one
of the special characters that would cause an argument to be quoted
before but not any longer after the new patch.

Since I don't have Win 7, all I can do is just guess that I need to
restore the old quoting rules.  So this patch does that in hopes that
it fixes the problem on Windows 7.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334375 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-10 20:57:14 +00:00
Zachary Turner
edd16bdf67 Revert "Resubmit "[Support] Expose flattenWindowsCommandLine.""
This reverts commit 65243b6d19143cb7a03f68df0169dcb63e8b4632.

Seems like it's not a flake.  It might have something to do with
the '*' character being in a command line.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334356 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-10 03:16:25 +00:00
Zachary Turner
432c850773 Resubmit "[Support] Expose flattenWindowsCommandLine."
There were a few linux compilation failures, but other than that
I think this was just a flake that caused the tests to fail.  I'm
going to resubmit and see if the failures go away, if not I'll
revert again.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334355 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-10 02:46:11 +00:00
Zachary Turner
eda86f5c64 Revert "[Support] Expose flattenWindowsCommandLine."
This reverts commit 10d2e88e87150a35dc367ba30716189d2af26774.

This is causing some test failures for some reason, reverting
while I investigate.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334354 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-09 23:07:39 +00:00
Zachary Turner
861985beca [Support] Expose flattenWindowsCommandLine.
This function was internal to Program.inc, but I've needed this
on several occasions when I've had to use CreateProcess without
llvm's sys::Execute functions.  In doing so, I noticed that the
function was written using unsafe C-string access and was pretty
hard to understand / make sense of, so I've also re-written the
functions to use more modern LLVM constructs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334353 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-09 22:44:44 +00:00
Zachary Turner
370ce5a7b0 Clean up some code in Program.
NFC here, this just raises some platform specific ifdef hackery
out of a class and creates proper platform-independent typedefs
for the relevant things.  This allows these typedefs to be
reused in other places without having to reinvent this preprocessor
logic.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334294 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-08 15:16:25 +00:00
Zachary Turner
bc083d568e Add a file open flag that disables O_CLOEXEC.
O_CLOEXEC is the right default, but occasionally you don't
want this.  This is especially true for tools like debuggers
where you might need to spawn the child process with specific
files already open, but it's occasionally useful in other
scenarios as well, like when you want to do some IPC between
parent and child.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334293 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-08 15:15:56 +00:00
Zachary Turner
e6e585b6dc Expose a single global file open function.
This one allows much more flexibility than the standard
openFileForRead / openFileForWrite functions.  Since there is now
just one "real" function that does the work, all other implementations
simply delegate to this one.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334246 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-07 23:25:13 +00:00
Zachary Turner
03bcb2143b [FileSystem] Split up the OpenFlags enumeration.
This breaks the OpenFlags enumeration into two separate
enumerations: OpenFlags and CreationDisposition.  The first
controls the behavior of the API depending on whether or not
the target file already exists, and is not a flags-based
enum.  The second controls more flags-like values.

This yields a more easy to understand API, while also allowing
flags to be passed to the openForRead api, where most of the
values didn't make sense before.  This also makes the apis more
testable as it becomes easy to enumerate all the configurations
which make sense, so I've added many new tests to exercise all
the different values.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334221 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-07 19:58:58 +00:00
Zachary Turner
ea3c8cfdce [Support] Add functions that operate on native file handles on Windows.
Windows' CRT has a limit of 512 open file descriptors, and fds which are
generated by converting a HANDLE via _get_osfhandle count towards this
limit as well.

Regardless, often you find yourself marshalling back and forth between
native HANDLE objects and fds anyway. If we know from the getgo that
we're going to need to work directly with the handle, we can cut out the
marshalling layer while also not contributing to filling up the CRT's
very limited handle table.

On Unix these functions just delegate directly to the existing set of
functions since an fd *is* the native file type. It would be nice, very
long term, if we could convert most uses of fds to file_t.

Differential Revision: https://reviews.llvm.org/D47688

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333945 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-04 19:38:11 +00:00
Zachary Turner
afee80613f Move some function declarations out of WindowsSupport.h
The idea behind WindowsSupport.h is that it's in the source directory so
that windows.h'isms don't leak out into the larger LLVM project. To that
end, any symbol that references a symbol from windows.h must be in this
private header, and not in a public header.

However, we had some useful utility functions in WindowsSupport.h which
have no dependency on the Windows API, but still only make sense on
Windows. Those functions should be usable outside of Support since there
is no risk of causing a windows.h leak. Although this introduces some
preprocessor logic in some header files, It's not too egregious and it's
better than the alternative of duplicating a ton of code.

Differential Revision: https://reviews.llvm.org/D47662

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333798 91177308-0d34-0410-b5e6-96231b3b80d8
2018-06-01 22:23:46 +00:00
Petr Hosek
a50c4ffd89 [Support] Avoid normalization in sys::getDefaultTargetTriple
The return value of sys::getDefaultTargetTriple, which is derived from
-DLLVM_DEFAULT_TRIPLE, is used to construct tool names, default target,
and in the future also to control the search path directly; as such it
should be used textually, without interpretation by LLVM.

Normalization of this value may lead to unexpected results, for example
if we configure LLVM with -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-linux-gnu,
normalization will transform that value to x86_64--linux-gnu. Driver will
use that value to search for tools prefixed with x86_64--linux-gnu- which
may be confusing. This is also inconsistent with the behavior of the
--target flag which is taken as-is without any normalization and overrides
the value of LLVM_DEFAULT_TARGET_TRIPLE.

Users of sys::getDefaultTargetTriple already perform their own
normalization as needed, so this change shouldn't impact existing logic.

Differential Revision: https://reviews.llvm.org/D47153

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@333307 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-25 20:39:37 +00:00
Nico Weber
f161702003 Revert 332750, llvm part (see comment on D46910).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332823 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-20 23:03:17 +00:00
Petr Hosek
9c871324f6 [Support] Avoid normalization in sys::getDefaultTargetTriple
The return value of sys::getDefaultTargetTriple, which is derived from
-DLLVM_DEFAULT_TRIPLE, is used to construct tool names, default target,
and in the future also to control the search path directly; as such it
should be used textually, without interpretation by LLVM.

Normalization of this value may lead to unexpected results, for example
if we configure LLVM with -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-linux-gnu,
normalization will transform that value to x86_64--linux-gnu. Driver will
use that value to search for tools prefixed with x86_64--linux-gnu- which
may be confusing. This is also inconsistent with the behavior of the
--target flag which is taken as-is without any normalization and overrides
the value of LLVM_DEFAULT_TARGET_TRIPLE.

Users of sys::getDefaultTargetTriple already perform their own
normalization as needed, so this change shouldn't impact existing logic.

Differential Revision: https://reviews.llvm.org/D46910

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332750 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-18 18:33:07 +00:00
JF Bastien
96936258fd Signal handling should be signal-safe
Summary:
Before this patch, signal handling wasn't signal safe. This leads to real-world
crashes. It used ManagedStatic inside of signals, this can allocate and can lead
to unexpected state when a signal occurs during llvm_shutdown (because
llvm_shutdown destroys the ManagedStatic). It also used cl::opt without custom
backing storage. Some de-allocation was performed as well. Acquiring a lock in a
signal handler is also a great way to deadlock.

We can't just disable signals on llvm_shutdown because the signals might do
useful work during that shutdown. We also can't just disable llvm_shutdown for
programs (instead of library uses of clang) because we'd have to then mark the
pointers as not leaked and make sure all the ManagedStatic uses are OK to leak
and remain so.

Move all of the code to lock-free datastructures instead, and avoid having any
of them in an inconsistent state. I'm not trying to be fancy, I'm not using any
explicit memory order because this code isn't hot. The only purpose of the
atomics is to guarantee that a signal firing on the same or a different thread
doesn't see an inconsistent state and crash. In some cases we might miss some
state (for example, we might fail to delete a temporary file), but that's fine.

Note that I haven't touched any of the backtrace support despite it not
technically being totally signal-safe. When that code is called we know
something bad is up and we don't expect to continue execution, so calling
something that e.g. sets errno is the least of our problems.

A similar patch should be applied to lib/Support/Windows/Signals.inc, but that
can be done separately.

Fix r332428 which I reverted in r332429. I originally used double-wide CAS
because I was lazy, but some platforms use a runtime function for that which
thankfully failed to link (it would have been bad for signal handlers
otherwise). I use a separate flag to guard the data instead.

<rdar://problem/28010281>

Reviewers: dexonsmith

Subscribers: steven_wu, llvm-commits

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332496 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-16 17:25:35 +00:00
JF Bastien
0302e7b44f Revert "Signal handling should be signal-safe"
Some bots don't have double-pointer width compare-and-exchange. Revert for now.q

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332429 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-16 04:36:37 +00:00
JF Bastien
b60356726c Signal handling should be signal-safe
Summary:
Before this patch, signal handling wasn't signal safe. This leads to real-world
crashes. It used ManagedStatic inside of signals, this can allocate and can lead
to unexpected state when a signal occurs during llvm_shutdown (because
llvm_shutdown destroys the ManagedStatic). It also used cl::opt without custom
backing storage. Some de-allocation was performed as well. Acquiring a lock in a
signal handler is also a great way to deadlock.

We can't just disable signals on llvm_shutdown because the signals might do
useful work during that shutdown. We also can't just disable llvm_shutdown for
programs (instead of library uses of clang) because we'd have to then mark the
pointers as not leaked and make sure all the ManagedStatic uses are OK to leak
and remain so.

Move all of the code to lock-free datastructures instead, and avoid having any
of them in an inconsistent state. I'm not trying to be fancy, I'm not using any
explicit memory order because this code isn't hot. The only purpose of the
atomics is to guarantee that a signal firing on the same or a different thread
doesn't see an inconsistent state and crash. In some cases we might miss some
state (for example, we might fail to delete a temporary file), but that's fine.

Note that I haven't touched any of the backtrace support despite it not
technically being totally signal-safe. When that code is called we know
something bad is up and we don't expect to continue execution, so calling
something that e.g. sets errno is the least of our problems.

A similar patch should be applied to lib/Support/Windows/Signals.inc, but that
can be done separately.

<rdar://problem/28010281>

Reviewers: dexonsmith

Subscribers: aheejin, llvm-commits

Differential Revision: https://reviews.llvm.org/D46858

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332428 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-16 04:30:00 +00:00
JF Bastien
54e2a7e2a5 [NFC] Update comments
Don't prepend function or data name before each comment. Split into its own NFC patch as requested in D46858.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332323 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-15 04:06:28 +00:00
Brian Gesiak
fae67c2efe [Support] Add docs for 'openFileFor{Write,Read}'
Summary:
Add documentation for the LLVM Support functions `openFileForWrite` and
`openFileForRead`. The `openFileForRead` parameter `RealPath`, in
particular, I think warranted some explanation.

In addition, make the behavior of the functions more consistent across
platforms. Prior to this patch, Windows would set or not set the result
file descriptor based on the nature of the error, whereas Unix would
consistently set it to `-1` if the open failed. Make Windows
consistently set it to `-1` as well.

Test Plan:
1. `ninja check-llvm`
2. `ninja docs-llvm-html`

Reviewers: zturner, rnk, danielmartin, scanon

Reviewed By: danielmartin, scanon

Subscribers: scanon, danielmartin, llvm-commits

Differential Revision: https://reviews.llvm.org/D46499

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@332075 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-11 01:47:27 +00:00
Adrian Prantl
26b584c691 Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

  for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done

Differential Revision: https://reviews.llvm.org/D46290

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@331272 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-01 15:54:18 +00:00
Aaron Smith
a1ce0364e1 [support] Revert the changes made to Path.inc for the default Windows code page
Path.inc/widenPath tries to decode the path using both UTF-8 and the default Windows code page.
This is no longer necessary with the new InitLLVM method which ensures that the command line
arguemnts are already UTF-8 on Windows.
 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330266 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-18 15:26:26 +00:00
Rui Ueyama
3e79068e73 Rename sys::Process::GetArgumentVector -> sys::windows::GetCommandLineArguments
GetArgumentVector (or GetCommandLineArguments) is very Windows-specific.
I think it doesn't make much sense to provide that function from sys::Process.

I also made a change so that the function takes a BumpPtrAllocator
instead of a SpecificBumpPtrAllocator. The latter is the class to call
dtors, but since char * is trivially destructible, we should use the
former class.

Differential Revision: https://reviews.llvm.org/D45641

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@330216 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-17 21:09:16 +00:00
Martin Storsjo
4f9163a74d [Support] Fix building for Windows on ARM
The commit in SVN r310001 that added support for this actually didn't
use the right struct field for the frame pointer - for ARM, there is
no register named Fp in the CONTEXT struct. On Windows, the R11
register is used as frame pointer.

Differential Revision: https://reviews.llvm.org/D45590

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329991 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-13 06:38:02 +00:00
Aaron Smith
d1d162f3a7 Windows needs the current codepage instead of utf8 sometimes
Llvm-mc (and tools that use Path.inc on Windows) assume that strings are utf-8 
encoded, however, this is not always the case. On Windows the default codepage 
is not utf-8, so most of the time the strings are not utf-8 encoded.

The lld test 'format-binary-non-ascii' uses llvm-mc with a file with non-ascii 
characters in the name which is how this bug was found. The test fails when run 
using Python 3 because it uses properly encoded unicode strings (Python 2 actually 
ends up using a byte string which is not utf-8 encoded, so the test passes, but 
that's separate issue). 

Patch by Stella Stamenova!


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@329468 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-07 00:32:59 +00:00
Nico Weber
bdcaabb328 Remove HAVE_LIBPSAPI, HAVE_SHELL32.
These used to be set in the old autoconf build, but the cmake build has had a
"TODO: actually check for these" comment since it was checked in, and they
were set to 1 on mingw unconditionally.  It seems safe to say that they always
exist under mingw, so just remove them and assume they're set exactly when on
mingw (with msvc, we use `pragma comment` instead of linking these via flags).


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@328992 91177308-0d34-0410-b5e6-96231b3b80d8
2018-04-02 17:32:48 +00:00
Zachary Turner
c29d246f51 [Support] Add WriteThroughMemoryBuffer.
This is like MemoryBuffer (read-only) and WritableMemoryBuffer
(writable private), but where the underlying file can be modified
after writing.  This is useful when you want to open a file, make
some targeted edits, and then write it back out.

Differential Revision: https://reviews.llvm.org/D44230

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@327057 91177308-0d34-0410-b5e6-96231b3b80d8
2018-03-08 20:34:47 +00:00
Serge Pavlov
06c71d8a6f Report fatal error in the case of out of memory
This is the second part of recommit of r325224. The previous part was
committed in r325426, which deals with C++ memory allocation. Solution
for C memory allocation involved functions `llvm::malloc` and similar.
This was a fragile solution because it caused ambiguity errors in some
cases. In this commit the new functions have names like `llvm::safe_malloc`.

The relevant part of original comment is below, updated for new function
names.

Analysis of fails in the case of out of memory errors can be tricky on
Windows. Such error emerges at the point where memory allocation function
fails, but manifests itself when null pointer is used. These two points
may be distant from each other. Besides, next runs may not exhibit
allocation error.

In some cases memory is allocated by a call to some of C allocation
functions, malloc, calloc and realloc. They are used for interoperability
with C code, when allocated object has variable size and when it is
necessary to avoid call of constructors. In many calls the result is not
checked for null pointer. To simplify checks, new functions are defined
in the namespace 'llvm': `safe_malloc`, `safe_calloc` and `safe_realloc`.
They behave as corresponding standard functions but produce fatal error if
allocation fails. This change replaces the standard functions like 'malloc'
in the cases when the result of the allocation function is not checked
for null pointer.

Finally, there are plain C code, that uses malloc and similar functions. If
the result is not checked, assert statement is added.

Differential Revision: https://reviews.llvm.org/D43010


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@325551 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-20 05:41:26 +00:00