It turns out that not recycling some kinds of chunk can lead to the
recycle queue being starved in some scenarios. When that happens, we end
up mmap()ing new memory, but that turns out to be significantly slower.
So instead of not recycling huge chunks, we force-clean them, before
madvising so that the pages can still be reclaimed in case of memory
pressure.
--HG--
extra : rebase_source : 2dbd028daca92c9cd7c8079eb3dc5a0cfa06495b
Going through the system zone allocator for every call to realloc/free
on OSX is costly, because the zone allocator needs to first verify that
the allocations do belong to the allocator it invokes (which ends up
calling jemalloc's malloc_usable_size), which is unnecessary when we
expect the allocations to belong to jemalloc.
So, we export the malloc/realloc/free/etc. symbols from
libmozglue.dylib, such that libraries and programs linked against it
call directly into jemalloc instead of going through the system zone
allocator, effectively shortcutting the allocator verification.
The risk is that some things in Gecko try to realloc/free pointers it
got from system libraries, if those were allocated with a system zone
that is not jemalloc.
--HG--
extra : rebase_source : ee0b29e1275176f52e64f4648dfa7ce25d61292e
MOZ_REPLACE_MALLOC_LINKAGE was added back when there were problems with
getting weak references working properly for replace-malloc.
Versions of OSX < 10.6 needed flat namespace, but aren't supported
anymore.
Versions of Xcode < 4.5 required flat namespace + a dummy library in
order to produce proper weak references. There is virtually nobody still
building with such an ancient toolchain.
Keeping those around doesn't /really/ hurt, except recent versions of
Xcode don't expose dyldinfo in /usr/bin, used for the configure test.
Consequently, MOZ_REPLACE_MALLOC_LINKAGE ended up being set to use the
dummy library setup, which, by using flat namespace, now causes harm in
bug 1356701, causing bug 1378332.
--HG--
extra : rebase_source : e3edc1f2cf905943c33fafeb631f2f88fc87167e
At the same time:
- replace "(nullptr)" with "nullptr",
- replace "x == nullptr" with "!x",
- replace "x != nullptr" with "x".
--HG--
extra : rebase_source : 5bc5577d0de53244afae9ebadb84962f38306368
malloc_print_stats is designed as a end-of-process dump of malloc stats,
but it has limited value in Firefox itself. It has some usefulness as
something to invoke from a debugger while Firefox is running (after
manually setting opt_print_stats to true), but it also lacks information
about live allocations, which would make it more useful.
All in all, if we want some stats about jemalloc allocations in a more
useful way, we'd be better off augmenting the jemalloc_stats() API and
make the data available to e.g. about:memory, than keeping
malloc_print_stats.
--HG--
extra : rebase_source : f6fbc97dfddd52c8620e37e2e189ae0087a845b0
Going through the system zone allocator for every call to realloc/free
on OSX is costly, because the zone allocator needs to first verify that
the allocations do belong to the allocator it invokes (which ends up
calling jemalloc's malloc_usable_size), which is unnecessary when we
expect the allocations to belong to jemalloc.
So, we export the malloc/realloc/free/etc. symbols from
libmozglue.dylib, such that libraries and programs linked against it
calls directly into jemalloc instead of going through the system zone
allocator, effectively shortcutting the allocator verification.
The risk is that some things in Gecko try to realloc/free pointers it
got from system libraries, if those were allocated with a system zone
that is not jemalloc.
--HG--
extra : rebase_source : 45b9b98499760a7f946878d41d2fdaadb6dff4d6
Setting MALLOC_XMALLOC enables a runtime toggle that allows to make
allocations abort() on OOM instead of returning NULL. In other words,
it enables a toggle that allows to turn all allocations into infallible
allocations.
The toggle however still defaults to being disabled, which means even
when MALLOC_XMALLOC is defined when building mozjemalloc, there is no
change in behavior, unless a MALLOC_OPTIONS is set.
Even if this were useful to anyone (MALLOC_XMALLOC is only defined on
debug builds, limiting the usefulness), this is something
replace-malloc, in Firefox, is meant to be used for.
So let's remove this feature, and possibly add an equivalent
replace-malloc later if deemed necessary.
--HG--
extra : rebase_source : 1ccc893e9a8e842c3fa90e91f076a528df2f7dfe
Replace-malloc libraries, such as DMD, don't really need to care about
the details of implementing all the variants of aligned memory
allocation functions. Currently, by defining MOZ_REPLACE_ONLY_MEMALIGN
before including replace_malloc.h, they get predefined functions.
Instead of making that an opt-in at build time, we make the
replace-malloc initialization just fill the replace-malloc
malloc_table_t with implementations that rely on the replace_memalign
the library provides.
--HG--
extra : rebase_source : 0842a67d9bc27a9a86c33d14d98b9c25f39982fb
Until now, the malloc implementation functions would call the
replace-malloc functions if they exist, and fallback to the real
allocator in no such function exists. Instead of doing this, we now
fill the empty slots in the malloc_table_t with the real allocator
functions.
--HG--
extra : rebase_source : b54634f23188906939e4dc01fc5a3007de0f3f2c
We make replace_malloc_init_funcs called on all platforms and fill out a
malloc_table_t for the replace-malloc functions with what comes from
dlsym/GetProcAddress on Android/Windows, and from the dynamically linked
weak symbols replace_* on other platforms.
replace_malloc.h contains definitions of *_impl_t types for each of the
functions in the malloc_table_t, which is redundant with the
replace_*_impl_t types we were creating, so we remove those typedefs,
except for the two functions (init and get_bridge) that don't have such
a typedef. Those functions don't appear in malloc_table_t.
--HG--
extra : rebase_source : 3705a99ee07f63dbaa66973eef19ddab224e0911
We want, in a subsequent patch, to have replace_malloc_init_funcs be
called on all platforms (including those relying on the replace-malloc
library being loaded already) and perform more initialization.
To prepare for that, we move the non-platform-specific pieces out.
--HG--
extra : rebase_source : 239ed363ee168bf4f8a96e0a1ca52981cb941b71
All the _impl functions in replace-malloc.c are largely identical. This
replaces all of them with macro expansions.
--HG--
extra : rebase_source : 67a1809b0b0fc4645ea5041154fa3a6dcb6cce6b
This makes no significant difference in practice in the macro
expansions, but will help down the line.
--HG--
extra : rebase_source : 6d61c1f28c558321478d7e5f26390d27ae8ae3ac
MOZ_REPLACE_JEMALLOC was only defined when building jemalloc4 as a
replace-malloc library.
--HG--
extra : rebase_source : fa5c402da07fa96448c170b6db99629469691efe
It's a Solaris-only optimization that was used as a workaround for some
infinite loop in pages_map (bug 457189). In the meanwhile, the way
pages_map works has changed in such a way the infinite loop can't happen
anymore.
Specifically, the original problem was that pages_map would try to
allocate something larger than a chunk, then deallocate it, and
reallocate at a hinted aligned address near the address of the now
deallocated mapping, hopefully getting an aligned chunk. But Solaris
would ignore the hint and the chunk would never be aligned, causing the
infinite loop.
What the code does now it over-allocate, find an aligned chunk in there,
and deallocate what ends up not being needed. Which leaves no room for
the original infinite loop.
We thus remove the workaround and put Solaris in par with other Unix
platforms.
--HG--
extra : rebase_source : 9ce4509d9c5ac6123cedabf87c5738672af35d1b
In Gecko code, MOZ_RELEASE_ASSERT means assertions that happen on all
types of builds.
In mozjemalloc, RELEASE_ASSERT means assertions that happen when
MOZ_JEMALLOC_HARD_ASSERTS is set, otherwise normal assertions. Which is
confusing. On the other hand, it's very similar to
MOZ_DIAGNOSTIC_ASSERT, and we may just want to use that instead.
Moreover, with release promotion, the check setting
MOZ_JEMALLOC_HARD_ASSERTS means releases (promoted from beta) would end
up with those asserts while they're not meant to, so
MOZ_DIAGNOSTIC_ASSERT is actually closer to the intent. It however means
we'd lose the beta population running those assertions.
--HG--
extra : rebase_source : 606ab47af8d9ee793b13b806acadb9781c99a078
Support for them was only enabled on debug builds, and required an
opt-in through e.g. MALLOC_OPTIONS to actually enable at runtime.
--HG--
extra : rebase_source : 60c27585c2901a73eb790cec124b880c93da6ef7
While it makes sense for a global system allocator, it's not really
interesting for Firefox to respect that. Plus, newer versions of
jemalloc, which are more likely to be used as a global system allocator
use a different format for the options passed through that file.
--HG--
extra : rebase_source : 0f2283cf5660bc437bd097bf48c2b5989fa08011
Those options, complementing the MALLOC_OPTIONS environment variable,
were always empty since the removal of b2g.
--HG--
extra : rebase_source : 0e34cfce0b537ebb8ed3902bd46180dc205cb0e4
Android NDK defines SYS_mmap2 but it expands to a nonexistent symbol.
mmap2 may not be supported in any case in some AArch64 kernels, so we
should just use mmap.
MozReview-Commit-ID: 5Vjuja5fLIL
