Bug 1518726 - Apply https://reviews.llvm.org/D56475 to clang. r=froydnj

Firefox uses multiple processes. It has intentional leaks, and when
running with ASAN, we have suppressions to eliminate those. When running
ASAN builds through CI tests, when Firefox exits, each of the processes
(parent and child) exits and goes through its leaks and when there are
(which is a given), the ASAN runtime runs llvm-symbolizer to symbolicate
and match against suppressions. So each process runs llvm-symbolizer. At
the same time.

Some of the addresses to symbolicate are in libxul. Which contains all
DWARF info, making it a ~1GB monster. Oh, and because you're lucky,
things align perfectly such that libxul size is a multiple of the page
size. That makes llvm-symbolizer pread() the file instead of mmap()ing
it. Did I say there are multiple processes? So suddenly you have n
processes simultaneously allocating and filling 1GB of memory each, on
CI machines that have enough memory for the job they usually run, but
not enough for a sudden rush of n GB.

And things go awry. When you're lucky and the OOM killer didn't take
care of killing the CI entirely, symbolication couldn't happen and the
suppressions are not matched, and leaks are reported.

This all turns out it originates in how llvm-symbolicate chooses between
pread() and mmap(), which turns out is just defaults not being made for
binary files.

Differential Revision: https://phabricator.services.mozilla.com/D16010

--HG--
extra : moz-landing-system : lando
This commit is contained in:
Mike Hommey 2019-01-10 00:18:28 +00:00
parent 005832324b
commit a0b6688faa
6 changed files with 26 additions and 5 deletions

View File

@ -28,6 +28,7 @@
"patches": [
"static-llvm-symbolizer.patch",
"find_symbolizer_linux.patch",
"rename_gcov_flush_.patch"
"rename_gcov_flush_.patch",
"r350774.patch"
]
}

View File

@ -18,6 +18,7 @@
"patches": [
"static-llvm-symbolizer.patch",
"find_symbolizer_linux.patch",
"rename_gcov_flush_.patch"
"rename_gcov_flush_.patch",
"r350774.patch"
]
}

View File

@ -23,6 +23,7 @@
"patches": [
"static-llvm-symbolizer.patch",
"compiler-rt-cross-compile.patch",
"compiler-rt-no-codesign.patch"
"compiler-rt-no-codesign.patch",
"r350774.patch"
]
}

View File

@ -14,5 +14,8 @@
"gcc_dir": "/builds/worker/workspace/build/src/gcc",
"cc": "/builds/worker/workspace/build/src/gcc/bin/gcc",
"cxx": "/builds/worker/workspace/build/src/gcc/bin/g++",
"as": "/builds/worker/workspace/build/src/gcc/bin/gcc"
"as": "/builds/worker/workspace/build/src/gcc/bin/gcc",
"patches": [
"r350774.patch"
]
}

View File

@ -21,6 +21,7 @@
"aarch64-vastart-checking.patch",
"downgrade-mangling-error.patch",
"r346300-compiler-rt-windows-mmap.patch",
"loosen-msvc-detection.patch"
"loosen-msvc-detection.patch",
"r350774.patch"
]
}

View File

@ -0,0 +1,14 @@
diff --git a/llvm/lib/Object/Binary.cpp b/llvm/lib/Object/Binary.cpp
index d7c25921ec3..fe41987f5c2 100644
--- a/llvm/lib/Object/Binary.cpp
+++ b/llvm/lib/Object/Binary.cpp
@@ -88,7 +88,8 @@ Expected<std::unique_ptr<Binary>> object::createBinary(MemoryBufferRef Buffer,
Expected<OwningBinary<Binary>> object::createBinary(StringRef Path) {
ErrorOr<std::unique_ptr<MemoryBuffer>> FileOrErr =
- MemoryBuffer::getFileOrSTDIN(Path);
+ MemoryBuffer::getFileOrSTDIN(Path, /*FileSize=*/-1,
+ /*RequiresNullTerminator=*/false);
if (std::error_code EC = FileOrErr.getError())
return errorCodeToError(EC);
std::unique_ptr<MemoryBuffer> &Buffer = FileOrErr.get();