On windows when raw_fd_ostream::write_impl calls write, a 32 bit input is required for character count. As a variable with size_t is used for this argument, on x64 integral demotion occurs. In the case of large files an infinite loop follows.
See: https://bugs.llvm.org/show_bug.cgi?id=37926
This fix allows the output of files larger than the previous int32 limit.
Differential Revision: https://reviews.llvm.org/D48948
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339027 91177308-0d34-0410-b5e6-96231b3b80d8
Currently we use #pragma push_macro(LLVM_DEBUG) to fiddle with the LLVM_DEBUG
macro so that we can silence debugging the Knuth division algorithm unless it's
actually desired. Unfortunately this is incompatible with enabling modules
while building LLVM (via LLVM_ENABLE_MODULES=ON), probably due to a bug being
fixed by D33004.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@339009 91177308-0d34-0410-b5e6-96231b3b80d8
resize() (zeroing) makes every allocated page resident. The actual size of the compressed buffer is usually much
smaller. Making every page resident is wasteful.
When linking a test binary with ~1.9GiB uncompressed debug info with LLD, this optimization decreases max RSS by ~1.5GiB.
Differential Revision: https://reviews.llvm.org/50223
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338913 91177308-0d34-0410-b5e6-96231b3b80d8
In r338216 / D49860 TempFile::keep was extended to allow keeping across
filesystems. The aim on Windows was to have this happen in rename_internal
using the existing system API. However, to fix an issue and preserve the
idea of "renaming" not being a move, put Windows keep-across-filesystem in
TempFile::keep.
Differential Revision: https://reviews.llvm.org/D50048
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338841 91177308-0d34-0410-b5e6-96231b3b80d8
r337748 made us start incrementing DebugCounters all of the time. This
makes tsan unhappy in multithreaded environments.
Since it doesn't make much sense to use DebugCounters with multiple
threads, this patch makes us only count anything if the user passed a
-debug-counter option or if some other piece of code explicitly asks
for it (e.g. the pass in D50031).
The amount of global state here makes writing a unittest for this
behavior somewhat awkward. So, no test is provided.
Differential Revision: https://reviews.llvm.org/D50150
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338762 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
On Windows, TempFile::create() was prone to failing with permission
denied errors when a process created many tempfiles without providing
a model large enough to accommodate them. There was also a problem
with createUniqueEntity getting into an infinite loop when all names
permitted by the model are in use. This change fixes both of these
problems and adds a unit test for them.
Reviewers: pcc, rnk, zturner
Reviewed By: zturner
Subscribers: inglorion, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D50126
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338745 91177308-0d34-0410-b5e6-96231b3b80d8
Dsymutil's update functionality was broken on Windows because we tried
to rename a file while we're holding open handles to that file. TempFile
provides a solution for this through its keep(Twine) method. This patch
changes dsymutil to make use of that functionality.
Differential revision: https://reviews.llvm.org/D49860
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338216 91177308-0d34-0410-b5e6-96231b3b80d8
The standard library functions ::isprint/std::isprint have platform-
and locale-dependent behavior which makes LLVM's output less
predictable. In particular, regression tests my fail depending on the
implementation of these functions.
Implement llvm::isPrint in StringExtras.h with a standard behavior and
replace all uses of ::isprint/std::isprint by a call it llvm::isPrint.
The function is inlined and does not look up language settings so it
should perform better than the standard library's version.
Such a replacement has already been done for isdigit, isalpha, isxdigit
in r314883. gtest does the same in gtest-printers.cc using the following
justification:
// Returns true if c is a printable ASCII character. We test the
// value of c directly instead of calling isprint(), which is buggy on
// Windows Mobile.
inline bool IsPrintableAscii(wchar_t c) {
return 0x20 <= c && c <= 0x7E;
}
Similar issues have also been encountered by Julia:
https://github.com/JuliaLang/julia/issues/7416
I noticed the problem myself when on Windows isprint('\t') started to
evaluate to true (see https://stackoverflow.com/questions/51435249) and
thus caused several unit tests to fail. The result of isprint doesn't
seem to be well-defined even for ASCII characters. Therefore I suggest
to replace isprint by a platform-independent version.
Differential Revision: https://reviews.llvm.org/D49680
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338034 91177308-0d34-0410-b5e6-96231b3b80d8
On Windows when raw_fd_ostream::write_impl calls write, a 32 bit input
is required for character count. As a variable with size_t is used for
this argument on x64 integral demotion occurs. In the case of large
files an infinite loop follows.
See PR37926.
This fix allows the output of files larger than previous int32 limit.
Differential Revision: https://reviews.llvm.org/D48948
Patch by Owen Reynolds
Reviewed by: zturner
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@338027 91177308-0d34-0410-b5e6-96231b3b80d8
The function in question is copy-pasted lots of times in DWARF-related classes.
Thus it will make sense to place its implementation into the Support library.
Reviewed by: lhames
Differential Revision: https://reviews.llvm.org/D49824
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337995 91177308-0d34-0410-b5e6-96231b3b80d8
SmallVectorTemplateCommon wants to know the address of the first element
so it can detect whether it's in "small size" mode.
The old implementation split the small array, creating the storage for
the first element in SmallVectorTemplateCommon, and pulling the rest
into SmallVectorStorage where we know the size of the array. This
bloats SmallVector size 0 by the larger of sizeof(void*) and sizeof(T),
and we're not even using the storage.
The new implementation leaves the full small storage to
SmallVectorStorage. To calculate the offset of the first element in
SmallVectorTemplateCommon, we just need to know how far to jump, which
we can calculate out-of-band. One subtlety is that we need
SmallVectorStorage to be properly aligned even when the size is 0, to be
sure that (for large alignments) we actually have the padding and it's
well defined to do the pointer math.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337820 91177308-0d34-0410-b5e6-96231b3b80d8
This patch makes debug counters keep track of the total number of times
we've called `shouldExecute` for each counter, so it's easier to build
automated tooling on top of these.
A patch to print these counts is coming soon.
Patch by Zhizhou Yang!
Differential Revision: https://reviews.llvm.org/D49560
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337748 91177308-0d34-0410-b5e6-96231b3b80d8
I'm optimistically reverting commit r337511, effectively reapplying
r337504 *without* changes.
The failing bots that had `SmallVector` in the backtrace recovered after
the unrelated commit r337508. The backtraces looked bogus anyway, with
`SmallVector::size()` calling (e.g.) `ConstantArray::get()`.
Here's the original commit message:
ADT: Shrink size of SmallVector by 8B on 64-bit platforms
Represent size and capacity directly as unsigned and calculate
`end()` using `begin() + size()`.
This limits the maximum size/capacity of a vector to UINT32_MAX.
https://reviews.llvm.org/D48518
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337514 91177308-0d34-0410-b5e6-96231b3b80d8
Some trivial cases in udivrem were handled by directly assigning 0 or 1
to APInt objects. This would set the bit width to 1, instead of the bit
width of the inputs. A potentially undesirable side effect of that is
that with the bit width of 1, 1 equals -1.
Differential Revision: https://reviews.llvm.org/D49554
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337478 91177308-0d34-0410-b5e6-96231b3b80d8
Haiku does not expose information about local versus remote mounts, so just
return false, like Cygwin.
Patch by Niels Sascha Reedijk.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@337389 91177308-0d34-0410-b5e6-96231b3b80d8
Parsing invalid UTF-8 input is now a parse error.
Creating JSON values from invalid UTF-8 now triggers an assertion, and
(in no-assert builds) substitutes the unicode replacement character.
Strings retrieved from json::Value are always valid UTF-8.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336657 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch adds a new "integer" ValueType, and renames Number -> Double.
This allows us to preserve the full precision of int64_t when parsing integers
from the wire, or constructing from an integer.
The API is unchanged, other than giving asInteger() a clearer contract.
In addition, always output doubles with enough precision that parsing will
reconstruct the same double.
Reviewers: simon_tatham
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D46209
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336541 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This consists of four main parts:
- an type json::Expr representing JSON values of dynamic kind, which can be
composed, inspected, and modified
- a JSON parser from string -> json::Expr
- a JSON printer from json::Expr -> string, with optional pretty-printing
- a convention for mapping json::Expr <=> native types (fromJSON/toJSON)
Mapping functions are provided for primitives (e.g. int, vector) and the
ObjectMapper helper helps implement fromJSON for struct/object types.
Based on clangd's usage, a couple of places I'd appreciate review attention:
- fromJSON returns only bool. A richer error-signaling mechanism may be useful
to provide useful messages, or let recursive fromJSONs (containers/structs)
do careful error recovery.
- should json::obj be always explicitly written (like json::ary)
- there's no streaming parse API. I suspect there are some simple wins like
a callback API where the document is a long array, and each element is small.
But this can probably be bolted on easily when we see the need.
Reviewers: bkramer, labath
Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits
Differential Revision: https://reviews.llvm.org/D45753
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336534 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: The lib paths are not correctly picked up for OpenEmbedded sysroots
(like arm-oe-linux-gnueabi). I fix this in a follow-up clang patch. But in
order to add the correct libs I need to detect if the vendor is oe. For this
reason, it is first necessary to teach llvm to detect oe vendor, which is what
this patch does.
Reviewers: chandlerc, compnerd, rengolin, javed.absar
Reviewed By: compnerd
Subscribers: kristof.beyls, dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D48861
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@336401 91177308-0d34-0410-b5e6-96231b3b80d8
Initial patch adding assembly support for Armv8.4-A.
Besides adding v8.4 as a supported architecture to the usual places, this also
adds target features for the different crypto algorithms. Armv8.4-A introduced
new crypto algorithms, made them optional, and allows different combinations:
- none of the v8.4 crypto functions are supported, which is independent of the
implementation of the Armv8.0 SHA1 and SHA2 instructions.
- the v8.4 SHA512 and SHA3 support is implemented, in this case the Armv8.0
SHA1 and SHA2 instructions must also be implemented.
- the v8.4 SM3 and SM4 support is implemented, which is independent of the
implementation of the Armv8.0 SHA1 and SHA2 instructions.
- all of the v8.4 crypto functions are supported, in this case the Armv8.0 SHA1
and SHA2 instructions must also be implemented.
The v8.4 crypto instructions are added to AArch64 only, and not AArch32,
and are made optional extensions to Armv8.2-A.
The user-facing Clang options will map on these new target features, their
naming will be compatible with GCC and added in follow-up patches.
The Armv8.4-A instruction sets can be downloaded here:
https://developer.arm.com/products/architecture/a-profile/exploration-tools
Differential Revision: https://reviews.llvm.org/D48625
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335953 91177308-0d34-0410-b5e6-96231b3b80d8
FileOutputBuffer creates a temp file and on commit atomically
renames the temp file to the destination file. Sometimes we
want to modify an existing file in place, but still have the
atomicity guarantee. To do this we can initialize the contents
of the temp file from the destination file (if it exists), that
way the resulting FileOutputBuffer can have only selective
bytes modified. Committing will then atomically replace the
destination file as desired.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335902 91177308-0d34-0410-b5e6-96231b3b80d8
SmallVectorStorage is empty when its size is 1; use inheritance so that
the empty base class optimization kicks in.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@335421 91177308-0d34-0410-b5e6-96231b3b80d8
We were unnecessarily going from SmallString to std::string just to
get a null-terminated C string. So just...don't do that. Crash
slightly faster!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334841 91177308-0d34-0410-b5e6-96231b3b80d8
This is failing to compile when LLVM_ENABLE_THREADS is false,
and the fix is not immediately obvious, so reverting while I look
into it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334658 91177308-0d34-0410-b5e6-96231b3b80d8
Previously ThreadPool could only queue async "jobs", i.e. work
that was done for its side effects and not for its result. It's
useful occasionally to queue async work that returns a value.
From an API perspective, this is very intuitive. The previous
API just returned a shared_future<void>, so all we need to do is
make it return a shared_future<T>, where T is the type of value
that the operation returns.
Making this work required a little magic, but ultimately it's not
too bad. Instead of keeping a shared queue<packaged_task<void()>>
we just keep a shared queue<unique_ptr<TaskBase>>, where TaskBase
is a class with a pure virtual execute() method, then have a
templated derived class that stores a packaged_task<T()>. Everything
else works out pretty cleanly.
Differential Revision: https://reviews.llvm.org/D48115
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334643 91177308-0d34-0410-b5e6-96231b3b80d8
On Windows we've observed that if you open a file, write to it, map it into
memory and close the file handle, the contents of the memory mapping can
sometimes be incorrect. That was what we did when adding an entry to the
ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing
intermittent build failures on Chromium's ThinLTO bots on Windows. More
details are in the associated Chromium bug (crbug.com/786127).
We can prevent this from happening by keeping a handle to the file open while
the mapping is active. So this patch changes the mapped_file_region class to
duplicate the file handle when mapping the file and close it upon unmapping it.
One gotcha is that the file handle that we keep open must not have been
created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system
will prevent other processes from opening the file. We can achieve this
by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether. Instead,
we use SetFileInformationByHandle with FileDispositionInfo to manage the
delete-on-close bit. This lets us remove the hack that we used to use to
clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE.
A downside of using SetFileInformationByHandle/FileDispositionInfo as
opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using
CreateFile to open the file while the flag is set, even within the same
process. This doesn't seem to matter for almost every client of TempFile,
except for LockFileManager, which calls sys::fs::create_link to create a
hard link from the lock file, and in the process of doing so tries to open
the file. To prevent this change from breaking LockFileManager I changed it
to stop using TempFile by effectively reverting r318550.
Differential Revision: https://reviews.llvm.org/D48051
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334630 91177308-0d34-0410-b5e6-96231b3b80d8
Even if we support no-canonical-prefix on
clang-cl(https://reviews.llvm.org/D47480), argv0 becomes absolute path
in clang-cl and that embeds absolute path in /showIncludes.
This patch removes such full path normalization from InitLLVM on
windows, and that removes absolute path from clang-cl output
(obj/stdout/stderr) when debug flag is disabled.
Patch by Takuto Ikuta!
Differential Revision https://reviews.llvm.org/D47578
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334602 91177308-0d34-0410-b5e6-96231b3b80d8
This simplifies some code which had StringRefs to begin with, and
makes other code more complicated which had const char* to begin
with.
In the end, I think this makes for a more idiomatic and platform
agnostic API. Not all platforms launch process with null terminated
c-string arrays for the environment pointer and argv, but the api
was designed that way because it allowed easy pass-through for
posix-based platforms. There's a little additional overhead now
since on posix based platforms we'll be takign StringRefs which
were constructed from null terminated strings and then copying
them to null terminate them again, but from a readability and
usability standpoint of the API user, I think this API signature
is strictly better.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@334518 91177308-0d34-0410-b5e6-96231b3b80d8
