On Windows we've observed that if you open a file, write to it, map it into
memory and close the file handle, the contents of the memory mapping can
sometimes be incorrect. That was what we did when adding an entry to the
ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing
intermittent build failures on Chromium's ThinLTO bots on Windows. More
details are in the associated Chromium bug (crbug.com/786127).
We can prevent this from happening by keeping a handle to the file open while
the mapping is active. So this patch changes the mapped_file_region class to
duplicate the file handle when mapping the file and close it upon unmapping it.
One gotcha is that the file handle that we keep open must not have been
created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system
will prevent other processes from opening the file. We can achieve this
by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether. Instead,
we use SetFileInformationByHandle with FileDispositionInfo to manage the
delete-on-close bit. This lets us remove the hack that we used to use to
clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE.
A downside of using SetFileInformationByHandle/FileDispositionInfo as
opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using
CreateFile to open the file while the flag is set, even within the same
process. This doesn't seem to matter for almost every client of TempFile,
except for LockFileManager, which calls sys::fs::create_link to create a
hard link from the lock file, and in the process of doing so tries to open
the file. To prevent this change from breaking LockFileManager I changed it
to stop using TempFile by effectively reverting r318550.
Differential Revision: https://reviews.llvm.org/D48051
llvm-svn: 334630
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46290
llvm-svn: 331272
See r331124 for how I made a list of files missing the include.
I then ran this Python script:
for f in open('filelist.txt'):
f = f.strip()
fl = open(f).readlines()
found = False
for i in xrange(len(fl)):
p = '#include "llvm/'
if not fl[i].startswith(p):
continue
if fl[i][len(p):] > 'Config':
fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
found = True
break
if not found:
print 'not found', f
else:
open(f, 'w').write(''.join(fl))
and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.
No intended behavior change.
llvm-svn: 331184
LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too. Just use the
default macro instead of a reinvented one.
See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.
No intended behavior change.
This moves over all uses of the macro, but doesn't remove the definition
of it in (llvm-)config.h yet.
llvm-svn: 331127
This move some of the complexity over to the lower level TempFile.
It also makes it a bit more explicit where errors are ignored since we
now have a call to consumeError.
llvm-svn: 318550
std::error_code can represent success, so we don't need a
Optional<std::error_code>.
Rename the variable to avoid confusion with the type Error.
llvm-svn: 318111
Summary:
Previously, we would emit error messages like "IO failure on output
stream". This change causes use to include information about what
actually went wrong, e.g. "No space left on device".
Reviewers: sunfish, rnk
Reviewed By: rnk
Subscribers: mehdi_amini, llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D39203
llvm-svn: 316404
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.
I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.
This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.
Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).
llvm-svn: 304787
Go back to behavior pre-r231309 and reduce the timeout from 8 to ~1.5
min now that we have (a) PCMCache mechanism (r298165) and (b) timeout
that doesn't cause a failure, but actually build the module (r298175).
rdar://problem/30297862
llvm-svn: 298176
This is currently used by clang to lock access to modules; improve the
error message so that clang can use better output messages from locking
error issues.
rdar://problem/26529101
Differential Review: http://reviews.llvm.org/D20942
llvm-svn: 271755
Reapplies r241005 after fixing the build on non-Mac platforms. Original
commit message below.
The hostname can be very unstable when there are many machines on the
network competing for the same name. Using the hardware UUID makes it
less likely to have collisions or to consider files written by the
current host to be owned by a different one at a later time.
rdar://problem/21512307
llvm-svn: 241012
The hostname can be very unstable when there are many machines on the
network competing for the same name. Using the hardware UUID makes it
less likely to have collisions or to consider files written by the
current host to be owned by a different one at a later time.
rdar://problem/21512307
llvm-svn: 241005
Make sure to remove the unique lock file, which is what the .lock
symlink points to, if there is a signal while the lock is held. This
will release the lock, since the symlink will point to nothing (already
tested in unit tests). For good measure, also clean up the unique lock
file if there is an error or signal before the lock is acquired.
I will add a clang test.
rdar://problem/21512307
llvm-svn: 240967
Now that SmallString is a first-class citizen, most SmallString::str()
calls are not required. This patch removes a whole bunch of them, yet
there are lots more.
There are two use cases where str() is really needed:
1) To use one of StringRef member functions which is not available in
SmallString.
2) To convert to std::string, as StringRef implicitly converts while
SmallString do not. We may wish to change this, but it may introduce
ambiguity.
llvm-svn: 232622
This is true in clang, and let's us remove the problematic code that
waits around for the original file and then times out if it doesn't get
created in short order. This caused any 'dead' lock file or legitimate
time out to cause a cascade of timeouts in any processes waiting on the
same lock (even if they only just showed up).
llvm-svn: 229881
5 minutes is an eternity, so try to strike a better balance between
waiting long enough for any reasonable module build and not so long that
users kill the process because they think it's hanging.
Also give the client a way to delete the lock file after a timeout.
llvm-svn: 228603
The main difference is the removal of
std::error_code exists(const Twine &path, bool &result);
It was an horribly redundant interface since a file not existing is also a valid
error_code. Now we have an access function that returns just an error_code. This
is the only function that has to be implemented for Unix and Windows. The
functions can_write, exists and can_execute an now just wrappers.
One still has to be very careful using these function to avoid introducing
race conditions (Time of check to time of use).
llvm-svn: 217625
While std::error_code itself seems to work OK in all platforms, there
are few annoying differences with regards to the std::errc enumeration.
This patch adds a simple llvm enumeration, which will hopefully avoid build
breakages in other platforms and surprises as we get more uses of
std::error_code.
llvm-svn: 210920
The idea of this patch is to turn llvm/Support/system_error.h into a
transitional header that just brings in the erorr_code api to the llvm
namespace. I will remove it shortly afterwards.
The cases where the general idea needed some tweaking:
* std::errc is a namespace in msvc, so we cannot use "using std::errc". I could
add an #ifdef, but there were not that many uses, so I just added std:: to
them in this patch.
* Template specialization had to be moved to the std namespace in this
patch set already.
* The msvc implementation of default_error_condition doesn't seem to
provide the same transformations as we need. Not too surprising since
the standard doesn't actually say what "equivalent" means. I fixed the
problem by keeping our old mapping and using it at error_code
construction time.
Despite these shortcomings I think this is still a good thing. Some reasons:
* The different implementations of system_error might improve over time.
* It removes 925 lines of code from llvm already.
* It removes 6313 bytes from the text segment of the clang binary when
it is built with gcc and 2816 bytes when building with clang and
libstdc++.
llvm-svn: 210687
Before this patch the unix code for creating hardlinks was unused. The code
for creating symbolic links was implemented in lib/Support/LockFileManager.cpp
and the code for creating hard links in lib/Support/*/Path.inc.
The only use we have for these is in LockFileManager.cpp and it can use both
soft and hard links. Just have a create_link function that creates one or the
other depending on the platform.
llvm-svn: 203596
This reverts commits r203136, r203137, and r203138.
This code doesn't build on Windows. Even on Vista+, Windows requires
elevated privileges to create a symlink. Therefore we can't use
symlinks in the compiler. We'll have to find another approach.
llvm-svn: 203143
There was a race where:
- The LockFileManager tries to own the lock file and fails.
- The other owner then releases and removes the lock file.
- The LockFileManager tries to read the owner info from the lock file but fails now.
In such a case have LockFileManager try to get ownership again, instead of error'ing out.
llvm-svn: 203138
Hard links do not work on SMB network directories, and it causes us to fail
to build clang module files if the module cache is in such a directory.
rdar://15944959
llvm-svn: 203137
This compiles with no changes to clang/lld/lldb with MSVC and includes
overloads to various functions which are used by those projects and llvm
which have OwningPtr's as parameters. This should allow out of tree
projects some time to move. There are also no changes to libs/Target,
which should help out of tree targets have time to move, if necessary.
llvm-svn: 203083
LLVM's coding standards recommend raw_ostream and MemoryBuffer for
reading and writing text.
This has the side effect of allowing clang to compile more of Support
and TableGen in the Microsoft C++ ABI.
llvm-svn: 187826
This function is complementary to createTemporaryFile. It handles the case were
the unique file is *not* temporary: we will rename it in the end. Since we
will rename it, the file has to be in the same filesystem as the final
destination and we don't prepend the system temporary directory.
This has a small semantic difference from unique_file: the default mode is 0666.
This matches the behavior of most unix tools. For example, with this change
lld now produces files with the same permissions as ld. I will add a test
of this change when I port clang over to createUniqueFile (next commit).
llvm-svn: 185726
It's possible for the lock file to disappear and the owning process to
return before we're able to see the generated file. Spin for a little
while to see if it shows up before failing.
llvm-svn: 178909
