Summary:
Add an overload to sys::fs::setLastModificationAndAccessTime that allows setting last access and modification times separately. This will allow tools to use this API when they want to preserve both the access and modification times from an input file, which may be different.
Also note that both the POSIX (futimens/futimes) and Windows (SetFileTime) APIs take the two timestamps in the order of (1) access (2) modification time, so this renames the method to "setLastAccessAndModificationTime" to make it clear which timestamp is which.
For existing callers, the 1-arg overload just sets both timestamps to the same thing.
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D50521
llvm-svn: 339628
In r338216 / D49860 TempFile::keep was extended to allow keeping across
filesystems. The aim on Windows was to have this happen in rename_internal
using the existing system API. However, to fix an issue and preserve the
idea of "renaming" not being a move, put Windows keep-across-filesystem in
TempFile::keep.
Differential Revision: https://reviews.llvm.org/D50048
llvm-svn: 338841
Dsymutil's update functionality was broken on Windows because we tried
to rename a file while we're holding open handles to that file. TempFile
provides a solution for this through its keep(Twine) method. This patch
changes dsymutil to make use of that functionality.
Differential revision: https://reviews.llvm.org/D49860
llvm-svn: 338216
On Windows we've observed that if you open a file, write to it, map it into
memory and close the file handle, the contents of the memory mapping can
sometimes be incorrect. That was what we did when adding an entry to the
ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing
intermittent build failures on Chromium's ThinLTO bots on Windows. More
details are in the associated Chromium bug (crbug.com/786127).
We can prevent this from happening by keeping a handle to the file open while
the mapping is active. So this patch changes the mapped_file_region class to
duplicate the file handle when mapping the file and close it upon unmapping it.
One gotcha is that the file handle that we keep open must not have been
created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system
will prevent other processes from opening the file. We can achieve this
by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether. Instead,
we use SetFileInformationByHandle with FileDispositionInfo to manage the
delete-on-close bit. This lets us remove the hack that we used to use to
clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE.
A downside of using SetFileInformationByHandle/FileDispositionInfo as
opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using
CreateFile to open the file while the flag is set, even within the same
process. This doesn't seem to matter for almost every client of TempFile,
except for LockFileManager, which calls sys::fs::create_link to create a
hard link from the lock file, and in the process of doing so tries to open
the file. To prevent this change from breaking LockFileManager I changed it
to stop using TempFile by effectively reverting r318550.
Differential Revision: https://reviews.llvm.org/D48051
llvm-svn: 334630
I took some liberties and quoted fewer characters than before,
based on an article from MSDN which says that only certain characters
cause an arg to require quoting. This seems to be incorrect, though,
and worse it seems to be a difference in Windows version. The bot
that fails is Windows 7, and I can't reproduce the failure on Win
10. But it's definitely related to quoting and special characters,
because both tests that fail have a * in the argument, which is one
of the special characters that would cause an argument to be quoted
before but not any longer after the new patch.
Since I don't have Win 7, all I can do is just guess that I need to
restore the old quoting rules. So this patch does that in hopes that
it fixes the problem on Windows 7.
llvm-svn: 334375
This reverts commit 65243b6d19143cb7a03f68df0169dcb63e8b4632.
Seems like it's not a flake. It might have something to do with
the '*' character being in a command line.
llvm-svn: 334356
There were a few linux compilation failures, but other than that
I think this was just a flake that caused the tests to fail. I'm
going to resubmit and see if the failures go away, if not I'll
revert again.
llvm-svn: 334355
O_CLOEXEC is the right default, but occasionally you don't
want this. This is especially true for tools like debuggers
where you might need to spawn the child process with specific
files already open, but it's occasionally useful in other
scenarios as well, like when you want to do some IPC between
parent and child.
llvm-svn: 334293
This one allows much more flexibility than the standard
openFileForRead / openFileForWrite functions. Since there is now
just one "real" function that does the work, all other implementations
simply delegate to this one.
llvm-svn: 334246
This breaks the OpenFlags enumeration into two separate
enumerations: OpenFlags and CreationDisposition. The first
controls the behavior of the API depending on whether or not
the target file already exists, and is not a flags-based
enum. The second controls more flags-like values.
This yields a more easy to understand API, while also allowing
flags to be passed to the openForRead api, where most of the
values didn't make sense before. This also makes the apis more
testable as it becomes easy to enumerate all the configurations
which make sense, so I've added many new tests to exercise all
the different values.
llvm-svn: 334221
Windows' CRT has a limit of 512 open file descriptors, and fds which are
generated by converting a HANDLE via _get_osfhandle count towards this
limit as well.
Regardless, often you find yourself marshalling back and forth between
native HANDLE objects and fds anyway. If we know from the getgo that
we're going to need to work directly with the handle, we can cut out the
marshalling layer while also not contributing to filling up the CRT's
very limited handle table.
On Unix these functions just delegate directly to the existing set of
functions since an fd *is* the native file type. It would be nice, very
long term, if we could convert most uses of fds to file_t.
Differential Revision: https://reviews.llvm.org/D47688
llvm-svn: 333945
The idea behind WindowsSupport.h is that it's in the source directory so
that windows.h'isms don't leak out into the larger LLVM project. To that
end, any symbol that references a symbol from windows.h must be in this
private header, and not in a public header.
However, we had some useful utility functions in WindowsSupport.h which
have no dependency on the Windows API, but still only make sense on
Windows. Those functions should be usable outside of Support since there
is no risk of causing a windows.h leak. Although this introduces some
preprocessor logic in some header files, It's not too egregious and it's
better than the alternative of duplicating a ton of code.
Differential Revision: https://reviews.llvm.org/D47662
llvm-svn: 333798
Summary:
Add documentation for the LLVM Support functions `openFileForWrite` and
`openFileForRead`. The `openFileForRead` parameter `RealPath`, in
particular, I think warranted some explanation.
In addition, make the behavior of the functions more consistent across
platforms. Prior to this patch, Windows would set or not set the result
file descriptor based on the nature of the error, whereas Unix would
consistently set it to `-1` if the open failed. Make Windows
consistently set it to `-1` as well.
Test Plan:
1. `ninja check-llvm`
2. `ninja docs-llvm-html`
Reviewers: zturner, rnk, danielmartin, scanon
Reviewed By: danielmartin, scanon
Subscribers: scanon, danielmartin, llvm-commits
Differential Revision: https://reviews.llvm.org/D46499
llvm-svn: 332075
Path.inc/widenPath tries to decode the path using both UTF-8 and the default Windows code page.
This is no longer necessary with the new InitLLVM method which ensures that the command line
arguemnts are already UTF-8 on Windows.
llvm-svn: 330266
Llvm-mc (and tools that use Path.inc on Windows) assume that strings are utf-8
encoded, however, this is not always the case. On Windows the default codepage
is not utf-8, so most of the time the strings are not utf-8 encoded.
The lld test 'format-binary-non-ascii' uses llvm-mc with a file with non-ascii
characters in the name which is how this bug was found. The test fails when run
using Python 3 because it uses properly encoded unicode strings (Python 2 actually
ends up using a byte string which is not utf-8 encoded, so the test passes, but
that's separate issue).
Patch by Stella Stamenova!
llvm-svn: 329468
This is like MemoryBuffer (read-only) and WritableMemoryBuffer
(writable private), but where the underlying file can be modified
after writing. This is useful when you want to open a file, make
some targeted edits, and then write it back out.
Differential Revision: https://reviews.llvm.org/D44230
llvm-svn: 327057
There is a latent Windows kernel bug, the exact trigger
conditions are not well understood, which can cause a file
to be correctly written, but unable to be correctly read.
The workaround appears to be simply calling FlushFileBuffers.
Differential Revision: https://reviews.llvm.org/D42925
llvm-svn: 325274
Without this when lld failed to replace the output file it would leave
the temporary behind. The problem is that the existing logic is
- cancel the delete flag
- rename
We have to cancel first to avoid renaming and then crashing and
deleting the old version. What is missing then is deleting the
temporary file if the rename fails.
This can be an issue on both unix and windows, but I am not sure how
to cause the rename to fail reliably on unix. I think it can be done
on ZFS since it has an ACL system similar to what windows uses, but
adding support for checking that in llvm-lit is probably not worth it.
llvm-svn: 319786
Summary:
zturner suggested that mapped_file_region::init() on Windows seems to
create mappings that are larger than they need to be: Offset+Size
instead of Size. Indeed, that appears to be the case. I confirmed that
tests pass with mappings of just Size bytes, and fail with Size-1
bytes, suggesting that Size is indeed the correct value.
Reviewers: amccarth, zturner
Reviewed By: zturner
Subscribers: hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D39876
llvm-svn: 317850
Summary:
The removed code checks that we are able to handle a 64-bit number, but
the code we're calling takes two dwords (for a total of 64 bits), so this
is always true.
Reviewers: zturner, rnk, majnemer, compnerd
Reviewed By: zturner
Subscribers: amccarth, hiraditya, lebedev.ri, llvm-commits
Differential Revision: https://reviews.llvm.org/D39263
llvm-svn: 316814
In r315079 I added a check for the ERROR_CALL_NOT_IMPLEMENTED error
code, but it turns out earlier versions of Wine just returned false
without setting any error code.
This patch handles the unset error code case.
llvm-svn: 315597
In r315079, fs::rename was reimplemented in terms of CreateFile and
SetFileInformationByHandle. Unfortunately, the latter isn't supported by
Wine. This adds a fallback to MoveFileEx for that case.
Differential Revision: https://reviews.llvm.org/D38817
llvm-svn: 315520
This allows clients to avoid an unnecessary fs::status() call on each
directory entry. Because the information returned by FindFirstFileEx
is a subset of the information returned by a regular status() call,
I needed to extract a base class from file_status that contains only
that information.
On my machine, this reduces the time required to enumerate a ThinLTO
cache directory containing 520k files from almost 4 minutes to less
than 2 seconds.
Differential Revision: https://reviews.llvm.org/D38716
llvm-svn: 315378
Microsoft's debug implementation of std::copy checks if the destination is an
array and then does some bounds checking. This was causing an assertion
failure in fs::rename_internal which copies to a buffer of the appropriate
size but that's type-punned to an array of length 1 for API compatibility
reasons.
Fix is to make make the destination a pointer rather than an array.
llvm-svn: 315222
The current implementation of rename uses ReplaceFile if the
destination file already exists. According to the documentation for
ReplaceFile, the source file is opened without a sharing mode. This
means that there is a short interval of time between when ReplaceFile
renames the file and when it closes the file during which the
destination file cannot be opened.
This behaviour is not POSIX compliant because rename is supposed
to be atomic. It was also causing intermittent link failures when
linking with a ThinLTO cache; the ThinLTO cache implementation expects
all cache files to be openable.
This patch addresses that problem by re-implementing rename
using CreateFile and SetFileInformationByHandle. It is roughly a
reimplementation of ReplaceFile with a better sharing policy as well
as support for renaming in the case where the destination file does
not exist.
This implementation is still not fully POSIX. Specifically in the case
where the destination file is open at the point when rename is called,
there will be a short interval of time during which the destination
file will not exist. It isn't clear whether it is possible to avoid
this using the Windows API.
Differential Revision: https://reviews.llvm.org/D38570
llvm-svn: 315079
Summary:
The function widenPath() for Windows also normalizes long path names by
iterating over the path's components and calling append(). The
assumption during the iteration that separators are not returned by the
iterator doesn't hold because the iterators do return a separator when
the path has a drive name. Handle this case by ignoring separators
during iteration.
Reviewers: rnk
Subscribers: danalbert, srhines
Differential Revision: https://reviews.llvm.org/D36752
llvm-svn: 311382
Summary:
Tools like clang that use RemoveFileOnSignal on their output files
weren't actually able to clean up their outputs before this change. Now
the call to llvm::sys::fs::remove succeeds and the temporary file is
deleted. This is a stop-gap to fix clang before implementing the
solution outlined in PR34070.
Reviewers: davide
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D36337
llvm-svn: 310137
Summary:
It was added to support clang warnings about includes with case
mismatches, but it ended up not being necessary.
Reviewers: twoh, rafael
Subscribers: hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D36328
llvm-svn: 310078
Previously which path syntax we supported dependend on what
platform we were compiling LLVM on. While this is normally
desirable, there are situations where we need to be able to
handle a path that we know was generated on a remote host.
Remote debugging, for example, or parsing debug info.
99% of the code in LLVM for handling paths was platform
agnostic and literally just a few branches were gated behind
pre-processor checks, so this changes those sites to use
runtime checks instead, and adds a flag to every path
API that allows one to override the host native syntax.
Differential Revision: https://reviews.llvm.org/D30858
llvm-svn: 298004
This change adds support for functions to set and get file permissions, in a similar manner to the C++17 permissions() function in <filesystem>. The setter uses chmod on Unix systems and SetFileAttributes on Windows, setting the permissions as passed in. The getter simply uses the existing status() function.
Prior to this change, status() would always return an unknown value for the permissions on a Windows file, making it impossible to test the new function on Windows. I have therefore added support for this as well. On Linux, prior to this change, the permissions included the file type, which should actually be accessed via a different member of the file_status class.
Note that on Windows, only the *_write permission bits have any affect - if any are set, the file is writable, and if not, the file is read-only. This is in common with what MSDN describes for their behaviour of std::filesystem::permissions(), and also what boost::filesystem does.
The motivation behind this change is so that we can easily test behaviour on read-only files in LLVM unit tests, but I am sure that others may find it useful in some situations.
Reviewers: zturner, amccarth, aaron.ballman
Differential Revision: https://reviews.llvm.org/D30736
llvm-svn: 297945
In openFileForRead, we would not previously return an error
if real_path resolution failed. After a recent patch, we
started propagating this error up. This caused a failure
in clang when trying to call openFileForRead("nul"). This
patch restores the previous behavior of not propagating this
error up.
llvm-svn: 297488
LLVM already has real_path like functionality, but it is
cumbersome to use and involves clean up after (e.g. you have
to call openFileForRead, then close the resulting FD).
Furthermore, on Windows it doesn't work for directories since
opening a directory and opening a file require slightly
different flags.
So I add a simple function `real_path` which works for all
paths on all platforms and has a simple to use interface.
In doing so, I add the ability to opt in to resolving tilde
expressions (e.g. ~/foo), which are normally handled by
the shell.
Differential Revision: https://reviews.llvm.org/D30668
llvm-svn: 297483
We already have a function create_directories() which can create
an entire tree, and remove() which can remove an empty directory,
but we do not have remove_directories() which can remove an entire
tree. This patch adds such a function.
Because removing a directory tree can have dangerous consequences
when the tree contains a directory symlink, the patch here updates
the existing directory_iterator construct to optionally not follow
symlinks (previously it would always follow symlinks). The delete
algorithm uses this flag so that for symlinks, only the links are
removed, and not the targets.
On Windows this is implemented with SHFileOperation, which also
does not recurse into symbolic links or junctions.
Differential Revision: https://reviews.llvm.org/D30676
llvm-svn: 297314
