Commit Graph

17 Commits

Author SHA1 Message Date
Mike Hommey
e483ecaab0 Bug 1424709 - Force disable the OSX system "nano allocator". r=spohl
We're not actually using it, and it messes up with the zone allocator in
mozjemalloc after fork(). See the lengthy analysis in
https://bugzilla.mozilla.org/show_bug.cgi?id=1424709#c34 and following.

--HG--
extra : rebase_source : c58e13b897dde7b32d83c43fbb2a04a0db3a5dc9
2018-01-31 17:18:01 +09:00
Mike Hommey
eab43e4a6c Bug 1413475 - Run clang-format on all files in memory/build/. r=njn
--HG--
extra : rebase_source : a0a7ebff22c2387389d2f1dc75f8a5084f76ebb7
2017-11-01 17:20:54 +09:00
Mike Hommey
af14262e54 Bug 1413475 - Change comments to use C++ style // instead of /* */ in memory/build/. r=njn
--HG--
extra : rebase_source : 8d8b85e8123f414cb1e0e1eb067e0d198b3ebb8f
2017-11-01 17:15:12 +09:00
Mike Hommey
1de1ed32d0 Bug 1413475 - Normalize license boilerplates in memory/build/. r=njn
--HG--
extra : rebase_source : 9689f766211fbe1476c5e6d4774f1e95bb8e0208
2017-11-01 16:56:27 +09:00
Mike Hommey
078c8d1896 Bug 1399921 - Register zone allocator independently, and delay jemalloc initialization on mac. r=njn
In bug 1361258, we unified the initialization sequence on mac, and
chose to make the zone registration happen after jemalloc
initialization.

The order between jemalloc init and zone registration shouldn't actually
matter, because jemalloc initializes the first time the allocator is
actually used.

On the other hand, in some build setups (e.g. with light optimization),
the initialization of the thread_arena thread local variable can happen
after the forced jemalloc initialization because of the order the
corresponding static initializers run. In some levels of optimization,
the thread_arena initializer resets the value the jemalloc
initialization has set, which subsequently makes choose_arena() return
a bogus value (or hit an assertion in ThreadLocal.h on debug builds).

So instead of initializing jemalloc from a static initializer, which
then registers the zone, we instead register the zone and let jemalloc
initialize itself when used, which increases the chances of the
thread_arena initializer running first.

--HG--
extra : rebase_source : 4d9a5340d097ac8528dc4aaaf0c05bbef40b59bb
2017-09-15 07:34:48 +09:00
Phil Ringnalda
eab70d0a07 Backed out changeset 0e349b74bfc6 (bug 1399921) for failure to thrive on Mac
MozReview-Commit-ID: HoBhxzIzn2d
2017-09-14 20:36:41 -07:00
Mike Hommey
c53e695a6e Bug 1399921 - Register zone allocator independently, and delay jemalloc initialization on mac. r=njn
In bug 1361258, we unified the initialization sequence on mac, and
chose to make the zone registration happen after jemalloc
initialization.

The order between jemalloc init and zone registration shouldn't actually
matter, because jemalloc initializes the first time the allocator is
actually used.

On the other hand, in some build setups (e.g. with light optimization),
the initialization of the thread_arena thread local variable can happen
after the forced jemalloc initialization because of the order the
corresponding static initializers run. In some levels of optimization,
the thread_arena initializer resets the value the jemalloc
initialization has set, which subsequently makes choose_arena() return
a bogus value (or hit an assertion in ThreadLocal.h on debug builds).

So instead of initializing jemalloc from a static initializer, which
then registers the zone, we instead register the zone and let jemalloc
initialize itself when used, which increases the chances of the
thread_arena initializer running first.

--HG--
extra : rebase_source : 4d9a5340d097ac8528dc4aaaf0c05bbef40b59bb
2017-09-15 07:34:48 +09:00
Mike Hommey
acb85e2b41 Bug 1396361 - Avoid crashing when some system library calls malloc_zone_free(zone, NULL). r=njn
Some system libraries call malloc_zone_free directly instead of free,
and sometimes they do that with the wrong zone. When that happens, we
circle back, trying to find the right zone, and call malloc_zone_free
with the right one, but when we can't find one, we crash, which matches
what the system free() would do. Except in one case where the pointer
we're being passed is NULL, in which case we can't trace it back to any
zone, but shouldn't crash (system free() explicitly doesn't crash in
that case).

--HG--
extra : rebase_source : 17efdcd80f1a53be7ab6b7293bfb6060a9aa4a48
2017-09-04 07:32:42 +09:00
Mike Hommey
19448fe709 Bug 1356701 - Export unprefixed malloc and duplication functions on OSX. r=njn
Going through the system zone allocator for every call to realloc/free
on OSX is costly, because the zone allocator needs to first verify that
the allocations do belong to the allocator it invokes (which ends up
calling jemalloc's malloc_usable_size), which is unnecessary when we
expect the allocations to belong to jemalloc.

So, we export the malloc/realloc/free/etc. symbols from
libmozglue.dylib, such that libraries and programs linked against it
call directly into jemalloc instead of going through the system zone
allocator, effectively shortcutting the allocator verification.

The risk is that some things in Gecko try to realloc/free pointers it
got from system libraries, if those were allocated with a system zone
that is not jemalloc.

--HG--
extra : rebase_source : ee0b29e1275176f52e64f4648dfa7ce25d61292e
2017-07-04 15:01:50 +09:00
Mike Hommey
02a06f7958 Bug 1363992 - Remove support for making jemalloc4 the default. r=njn
--HG--
extra : rebase_source : 77e1c13aa329637d0ec875439d572ee06e6919fa
2017-05-11 13:16:00 +09:00
Mike Hommey
fcd9a5550a Bug 1361258 - Use Thread Local Storage in mozjemalloc on mac. r=erahm
NO_TLS used to be hardcoded on mac because up to 10.6, __thread was not
supported. Until recently, we still supported for 10.6, and it's not the
case anymore, so we could make mac builds use __thread.

Unfortunately, on OSX, __thread circles back calling malloc to allocate
storage on first access, so we have an infinite loop problem here.
Fortunately, pthread_keys don't have this property, so we can use that
instead. It doesn't appear to have significantly more overhead (and TLS
overhead is small anyways compared to the amount of work involved in
allocating memory with mozjemalloc).

At the same time, we uniformize the initialization sequence between
mozjemalloc and mozjemalloc+replace-malloc, such that we have less
occasions for surprises when riding the trains (replace-malloc being
nightly only), ensuring the zone registration happens at the end of
mozjemalloc's initialization.
2017-05-12 18:12:20 +09:00
Alex Gaynor
d304ba72b3 Bug 1353752 - Fixed building with jemalloc4 r=glandium
It was broken by 516551993d16.

r=glandium

MozReview-Commit-ID: BvejKjAPB84

--HG--
extra : rebase_source : efaf5c0afb7454ecc75a733e184c82cbd16c5e5f
2017-04-05 13:05:43 -04:00
Mike Hommey
c9370d609b Bug 1332508 - Reinitialize allocator mutexes in fork() child processes. r=njn
Adapted from
4e2e3dd9cf
and
d9f7b2a430

As per the latter commit, it would seem unlocking, in fork() child
processes, mutexes that were locked in the parent process is not really
well supported on OSX 10.12. The addition of the zone_reinit_lock
function in 10.12 supports this idea.

--HG--
extra : rebase_source : b3b58558cc195d63200078085c7e9b6c9b8d83ff
2017-01-20 10:06:41 +09:00
Mike Hommey
4e8f02fe22 Bug 1286613 - Add dummy implementations for most remaining OSX zone allocator functions. r=njn
Some system libraries are using malloc_default_zone() and then using
some of the malloc_zone_* API. Under normal conditions, those functions
check the malloc_zone_t/malloc_introspection_t struct for the values
that are allowed to be NULL, so that a NULL deref doesn't happen.

As of OSX 10.12, malloc_default_zone() doesn't return the actual default
zone anymore, but returns a fake, wrapper zone. The wrapper zone defines
all the possible functions in the malloc_zone_t/malloc_introspection_t
struct (almost), and calls the function from the registered default zone
(jemalloc in our case) on its own. Without checking whether the pointers
are NULL.

This means that a system library that calls e.g.
malloc_zone_batch_malloc(malloc_default_zone(), ...) ends up trying to
call jemalloc_zone.batch_malloc, which is NULL, and crash follows.

So as of OSX 10.12, the default zone is required to have all the
functions available (really, the same as the wrapper zone), even if they
do nothing.

This is arguably a bug in libsystem_malloc in OSX 10.12, but jemalloc
still needs to work in that case.

[Adapted from
c6943acb3c]

--HG--
extra : rebase_source : 7d7a5b47fa18f56183e99c3655aee003c9be161e
2017-01-18 14:35:11 +09:00
Mike Hommey
dae510ade1 Bug 1286613 - Don't rely on OSX SDK malloc/malloc.h for malloc_zone struct definitions. r=njn
The SDK jemalloc is built against might be not be the latest for various
reasons, but the resulting binary ought to work on newer versions of
OSX.

In order to ensure this, we need the fullest definitions possible, so
copy what we need from the latest version of malloc/malloc.h available
on opensource.apple.com.

[Adapted from
c68bb41793]

--HG--
extra : rebase_source : ab19c478b568ea24095a3be62c39fb81efc1920a
2017-01-18 13:50:35 +09:00
Mike Hommey
e52769e834 Bug 1286613 - Use the same zone allocator implementation as replace-malloc for mozjemalloc. r=njn
We have been using a different zone allocator between mozjemalloc and
replace-malloc for a long time. Jemalloc 4 uses the same as
replace-malloc, albeit as part of the jemalloc upstream code base.

We've been bitten many times in the past with Apple changes breaking the
zone allocator, and each time we've had to make changes to the three
instances, although two of them are similar and the changes there are
straightforward.

It also turns out that the way the mozjemalloc zone allocator is set up,
when a new version of OSX appears with a new version of the system zone
allocator, Firefox ends up using the system allocator, because the zone
allocator version is not supported.

So, we use the same zone allocator for both replace-malloc and
mozjemalloc, making everything on par with jemalloc 4.

--HG--
extra : rebase_source : 9c0e245b5f82bb71294370d607e690c05cc89fbc
2017-01-18 11:45:45 +09:00
Mike Hommey
d293cc01a1 Bug 1286613 - Move replace-malloc zone allocator to a separate file. r=njn
The intent here is to reuse the zone allocator for mozjemalloc, to avoid
all the shortcomings of mozjemalloc using a different one. This change
only moves the replace-malloc zone allocator out of replace-malloc.c, to
make changes for mozjemalloc integration clearer.

--HG--
rename : memory/build/replace_malloc.c => memory/build/zone.c
extra : rebase_source : 8b98efaa4a88862f2967c855b511e92beb9c4031
2017-01-18 11:39:29 +09:00