mirror of
https://github.com/mozilla/gecko-dev.git
synced 2024-11-23 21:01:08 +00:00
380 lines
14 KiB
ReStructuredText
380 lines
14 KiB
ReStructuredText
Address Sanitizer
|
|
=================
|
|
|
|
What is Address Sanitizer?
|
|
--------------------------
|
|
|
|
Address Sanitizer (ASan) is a fast memory error detector that detects
|
|
use-after-free and out-of-bound bugs in C/C++ programs. It uses a
|
|
compile-time instrumentation to check all reads and writes during the
|
|
execution. In addition, the runtime part replaces the ``malloc`` and
|
|
``free`` functions to check dynamically allocated memory. More
|
|
information on how ASan works can be found on `the Address Sanitizer
|
|
wiki <https://github.com/google/sanitizers/wiki/AddressSanitizer>`__.
|
|
|
|
A `meta bug called asan-maintenance <https://bugzilla.mozilla.org/show_bug.cgi?id=asan-maintenance>`__
|
|
is maintained to keep track of all the bugs found with ASan.
|
|
|
|
Downloading artifact builds
|
|
---------------------------
|
|
|
|
For Linux and Windows users, the easiest way to get Firefox builds with
|
|
Address Sanitizer is to download a continuous integration asan build of
|
|
mozilla-central (updated at least daily):
|
|
|
|
- mozilla-central optimized builds:
|
|
`linux <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.linux64-asan-opt/artifacts/public/build/target.tar.bz2>`__
|
|
\|
|
|
`windows <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.win64-asan-opt/artifacts/public/build/target.zip>`__
|
|
(recommended for testing)
|
|
- mozilla-central debug builds:
|
|
`linux <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.linux64-asan-debug/artifacts/public/build/target.tar.bz2>`__
|
|
\|
|
|
`windows <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.win64-asan-debug/artifacts/public/build/target.zip>`__
|
|
(recommended for debugging if the optimized builds don't do the job)
|
|
|
|
The fuzzing team also offers a tool called ``fuzzfetch`` to download these and many
|
|
other CI builds. It makes downloading and unpacking these builds much easier and
|
|
can be used not just for fuzzing but for all purposes that require a CI build download.
|
|
|
|
You can install ``fuzzfetch`` from
|
|
`Github <https://github.com/MozillaSecurity/fuzzfetch>`__ or
|
|
`via pip <https://pypi.org/project/fuzzfetch/>`__.
|
|
|
|
Afterwards, you can run e.g.
|
|
|
|
::
|
|
|
|
$ python -m fuzzfetch --asan -n firefox-asan
|
|
|
|
to get the optimized Linux ASan build mentioned above unpacked into a directory called ``firefox-asan``.
|
|
The ``--debug`` and ``--os`` switches can be used to get the other variants listed above.
|
|
|
|
Creating Try builds
|
|
-------------------
|
|
|
|
If for some reason you can't use the pre-built binaries mentioned in the
|
|
previous section (e.g. you want a non-Linux build or you need to test a
|
|
patch), you can either build Firefox yourself (see the following
|
|
section) or use the :ref:`try server <Pushing to Try>` to
|
|
create the customized build for you. Pushing to try requires L1 commit
|
|
access. If you don't have this access yet you can request access (see
|
|
`Becoming A Mozilla
|
|
Committer <https://www.mozilla.org/about/governance/policies/commit/>`__
|
|
and `Mozilla Commit Access
|
|
Policy <https://www.mozilla.org/about/governance/policies/commit/access-policy/>`__
|
|
for the requirements).
|
|
|
|
The tree contains `several mozconfig files for creating asan
|
|
builds <https://searchfox.org/mozilla-central/search?q=&case=true&path=browser%2Fconfig%2Fmozconfigs%2F*%2F*asan*>`__
|
|
(the "nightly-asan" files create release builds, whereas the
|
|
"debug-asan" files create debug+opt builds). For Linux builds, the
|
|
appropriate configuration file is used by the ``linux64-asan`` target.
|
|
If you want to create a macOS or Windows build, you'll need to copy the
|
|
appropriate configuration file over the regular debug configuration
|
|
before pushing to try. For example:
|
|
|
|
::
|
|
|
|
cp browser/config/mozconfigs/macosx64/debug-asan browser/config/mozconfigs/macosx64/debug
|
|
|
|
You can then `push to Try in the usual
|
|
way </tools/try/index.html#using-try>`__
|
|
and, once the build is complete, download the appropriate build
|
|
artifact.
|
|
|
|
Creating local builds on Windows
|
|
--------------------------------
|
|
|
|
On Windows, ASan is supported only in 64-bit builds.
|
|
|
|
Run ``mach bootstrap`` to get an updated clang-cl in your
|
|
``~/.mozbuild`` directory, then use the following
|
|
:ref:`mozconfig <Configuring Build Options>`:
|
|
|
|
::
|
|
|
|
ac_add_options --enable-address-sanitizer
|
|
ac_add_options --disable-jemalloc
|
|
|
|
export LDFLAGS="clang_rt.asan_dynamic-x86_64.lib clang_rt.asan_dynamic_runtime_thunk-x86_64.lib"
|
|
CLANG_LIB_DIR="$(cd ~/.mozbuild/clang/lib/clang/*/lib/windows && pwd)"
|
|
export MOZ_CLANG_RT_ASAN_LIB_PATH="${CLANG_LIB_DIR}/clang_rt.asan_dynamic-x86_64.dll"
|
|
export LIB=$LIB:$CLANG_LIB_DIR
|
|
|
|
If you launch an ASan build under WinDbg, you may see spurious
|
|
first-chance Access Violation exceptions. These come from ASan creating
|
|
shadow memory pages on demand, and can be ignored. Run ``sxi av`` to
|
|
ignore these exceptions. (You will still catch second-chance Access
|
|
Violation exceptions if you actually crash.)
|
|
|
|
LeakSanitizer (LSan) is not supported on Windows.
|
|
|
|
Creating local builds on Linux or Mac
|
|
-------------------------------------
|
|
|
|
Build prerequisites
|
|
~~~~~~~~~~~~~~~~~~~
|
|
|
|
LLVM/Clang
|
|
^^^^^^^^^^
|
|
|
|
The ASan instrumentation is implemented as an LLVM pass and integrated
|
|
into Clang. Any clang version that is capable of compiling Firefox has
|
|
everything needed to do an ASAN build.
|
|
|
|
Building Firefox
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
Getting the source
|
|
^^^^^^^^^^^^^^^^^^
|
|
|
|
Using that or any later revision, all you need to do is to :ref:`get yourself
|
|
a clone of mozilla-central <Mercurial overview>`.
|
|
|
|
Adjusting the build configuration
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Create the build configuration file ``mozconfig`` with the following
|
|
content in your mozilla-central directory:
|
|
|
|
::
|
|
|
|
# Combined .mozconfig file for ASan on Linux+Mac
|
|
|
|
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-ff-asan
|
|
|
|
# Enable ASan specific code and build workarounds
|
|
ac_add_options --enable-address-sanitizer
|
|
|
|
# These three are required by ASan
|
|
ac_add_options --disable-jemalloc
|
|
ac_add_options --disable-crashreporter
|
|
ac_add_options --disable-elf-hack
|
|
|
|
# Keep symbols to symbolize ASan traces later
|
|
export MOZ_DEBUG_SYMBOLS=1
|
|
ac_add_options --enable-debug-symbols
|
|
ac_add_options --disable-install-strip
|
|
|
|
# Settings for an opt build (preferred)
|
|
# The -gline-tables-only ensures that all the necessary debug information for ASan
|
|
# is present, but the rest is stripped so the resulting binaries are smaller.
|
|
ac_add_options --enable-optimize="-O2 -gline-tables-only"
|
|
ac_add_options --disable-debug
|
|
|
|
# Settings for a debug+opt build
|
|
#ac_add_options --enable-optimize
|
|
#ac_add_options --enable-debug
|
|
|
|
# MacOSX only: Uncomment and adjust this path to match your SDK
|
|
# ac_add_options --with-macos-sdk=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk
|
|
|
|
You may also need this, as seen in
|
|
``browser/config/mozconfigs/linux64/nightly-asan`` (the config file used
|
|
for Address Sanitizer builds used for automated testing):
|
|
|
|
::
|
|
|
|
# ASan specific options on Linux
|
|
ac_add_options --enable-valgrind
|
|
|
|
Starting the build process
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Now you start the build process using the regular ``./mach build``
|
|
command.
|
|
|
|
Starting Firefox
|
|
^^^^^^^^^^^^^^^^
|
|
|
|
After the build has completed, ``./mach run`` with the usual options for
|
|
running in a debugger (``gdb``, ``lldb``, ``rr``, etc.) work fine, as do
|
|
the ``--disable-e10s`` and other options.
|
|
|
|
Building only the JavaScript shell
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
If you want to build only the JavaScript shell instead of doing a full
|
|
Firefox build, the build script below will probably help you to do so.
|
|
Execute this script in the ``js/src/`` subdirectory and pass a directory
|
|
name as the first parameter. The build will then be created in a new
|
|
subdirectory with that name.
|
|
|
|
::
|
|
|
|
#! /bin/sh
|
|
|
|
if [ -z $1 ] ; then
|
|
echo "usage: $0 <dirname>"
|
|
elif [ -d $1 ] ; then
|
|
echo "directory $1 already exists"
|
|
else
|
|
autoconf2.13
|
|
mkdir $1
|
|
cd $1
|
|
CC="clang" \
|
|
CXX="clang++" \
|
|
CFLAGS="-fsanitize=address" \
|
|
CXXFLAGS="-fsanitize=address" \
|
|
LDFLAGS="-fsanitize=address" \
|
|
../configure --enable-debug --enable-optimize --enable-address-sanitizer --disable-jemalloc
|
|
fi
|
|
|
|
Getting Symbols in Address Sanitizer Traces
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
By default, ASan traces are unsymbolized and only print the
|
|
binary/library and a memory offset instead. In order to get more useful
|
|
traces, containing symbols, there are two approaches.
|
|
|
|
Using the LLVM Symbolizer (recommended)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
LLVM ships with a symbolizer binary that ASan will readily use to
|
|
immediately output symbolized traces. To use it, just set the
|
|
environment variable ``ASAN_SYMBOLIZER_PATH`` to reflect the location of
|
|
your ``llvm-symbolizer`` binary, before running the process. This
|
|
program is usually included in an LLVM distribution. Stacks without
|
|
symbols can also be post-processed, see below.
|
|
|
|
.. warning::
|
|
|
|
.. note::
|
|
|
|
**Warning:** On OS X, the content sandbox prevents the symbolizer
|
|
from running. To use llvm-symbolizer on ASan output from a
|
|
content process, the content sandbox must be disabled. This can be
|
|
done by setting ``MOZ_DISABLE_CONTENT_SANDBOX=1`` in your run
|
|
environment. Setting this in .mozconfig has no effect.
|
|
|
|
|
|
Post-Processing Traces with asan_symbolize.py
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Instead of using the llvm-symbolizer binary, you can also pipe the
|
|
output through the ``asan_symbolize.py`` script, shipped with LLVM
|
|
(``$LLVM_HOME/projects/compiler-rt/lib/asan/scripts/asan_symbolize.py``),
|
|
often included in LLVM distributions. The disadvantage is that the
|
|
script will need to use ``addr2line`` to get the symbols, which means
|
|
that every library will have to be loaded into memory
|
|
(including``libxul``, which takes a bit).
|
|
|
|
However, in certain situations it makes sense to use this script. For
|
|
example, if you have/received an unsymbolized trace, then you can still
|
|
use the script to turn it into a symbolized trace, given that you can
|
|
get the original binaries that produced the unsymbolized trace. In order
|
|
for the script to work in such cases, you need to ensure that the paths
|
|
in the trace point to the actual binaries, or change the paths
|
|
accordingly.
|
|
|
|
Since the output of the ``asan_symbolize.py`` script is still mangled,
|
|
you might want to pipe the output also through ``c++filt`` afterwards.
|
|
|
|
Troubleshooting / Known problems
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Cannot specify -o when generating multiple output files
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
If you get the error
|
|
"``cannot specify -o when generating multiple output files"`` from
|
|
clang, disable ``elf-hack`` in your ``mozconfig`` to work around the
|
|
issue:
|
|
|
|
::
|
|
|
|
ac_add_options --disable-elf-hack
|
|
|
|
Optimized build
|
|
^^^^^^^^^^^^^^^
|
|
|
|
Since `an issue with -O2/-Os and
|
|
ASan <https://github.com/google/sanitizers/issues/20>`__
|
|
has been resolved, the regular optimizations used by Firefox should work
|
|
without any problems. The optimized build has only a barely noticeable
|
|
speed penalty and seems to be even faster than regular debug builds.
|
|
|
|
No "AddressSanitizer: **libc** interceptors initialized" shows after running ./mach run
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
::
|
|
|
|
$ ASAN_OPTIONS=verbosity=2 ./mach run
|
|
|
|
Use the above command instead
|
|
|
|
"An admin user name and password" is required to enter Developer Mode
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Please enable **Developer** **mode** by:
|
|
|
|
::
|
|
|
|
$ /usr/sbin/DevToolsSecurity -enable
|
|
Developer mode is now enabled.
|
|
|
|
Debugging issues that ASan finds
|
|
--------------------------------
|
|
|
|
When ASan discovers an issue it will simply print an error message and
|
|
exit the app. To stop the app in a debugger before ASan exits it, set a
|
|
breakpoint on ``__asan::ReportGenericError``. For more info on using
|
|
ASan and debugging issues that it uncovers, see the page `Address
|
|
sanitizer and a
|
|
debugger <https://github.com/google/sanitizers/wiki/AddressSanitizerAndDebugger>`__
|
|
page on the upstream wiki.
|
|
|
|
``__asan_describe_address(pointer)`` issued at the debugger prompt or
|
|
even directly in the code allows outputting lots of information about
|
|
this memory address (thread and stack of allocation, of deallocation,
|
|
whether or not it is a bit outside a known buffer, thread and stack of
|
|
allocation of this buffer, etc.). This can be useful to understand where
|
|
some buffer that is not aligned was allocated, when doing SIMD work, for
|
|
example.
|
|
|
|
`rr <https://rr-project.org/>`__ (Linux x86 only) works great with ASan
|
|
and combined, this combo allows doing some very powerful debugging
|
|
strategies.
|
|
|
|
LeakSanitizer
|
|
-------------
|
|
|
|
LeakSanitizer (LSan) is a special execution mode for regular ASan. It
|
|
takes advantage of how ASan tracks the set of live blocks at any given
|
|
point to print out the allocation stack of any block that is still alive
|
|
at shutdown, but is not reachable from the stack, according to a
|
|
conservative scan. This is very useful for detecting leaks of things
|
|
such as ``char*`` that do not participate in the usual Gecko shutdown
|
|
leak detection. LSan is supported on x86_64 Linux and OS X.
|
|
|
|
LSan is enabled by default in ASan builds, as of more recent versions of
|
|
Clang. To make an ASan build not run LSan, set the environment variable
|
|
``ASAN_OPTIONS`` to ``detect_leaks=0`` (or add it as an entry to a
|
|
``:``-separated list if it is already set to something). If you want to
|
|
enable it when it is not for some reason, set it to 1 instead of 0. If
|
|
LSan is enabled and you are using a non-debug build, you will also want
|
|
to set the environment variable ``MOZ_CC_RUN_DURING_SHUTDOWN=1``, to
|
|
ensure that we run shutdown GCs and CCs to avoid spurious leaks.
|
|
|
|
If an object that is reported by LSan is intentionally never freed, a
|
|
symbol can be added to ``build/sanitizers/lsan_suppressions.txt`` to get
|
|
LSan to ignore it.
|
|
|
|
For some more information on LSan, see the `Leak Sanitizer wiki
|
|
page <https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer>`__.
|
|
|
|
|
|
A `meta bug called lsan <https://bugzilla.mozilla.org/show_bug.cgi?id=lsan>`__
|
|
is maintained to keep track of all the bugs found with LSan.
|
|
|
|
|
|
|
|
Frequently Asked Questions about ASan
|
|
-------------------------------------
|
|
|
|
How does ASan work exactly?
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
More information on how ASan works can be found on `the Address Sanitizer wiki <https://github.com/google/sanitizers/wiki/AddressSanitizer>`__.
|