Magic-Mirror/gecko-dev
1
0
Fork 0
mirror of https://github.com/mozilla/gecko-dev.git synced 2024-11-24 21:31:04 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Bug 1665029 - LUL: reduce space consumption by not storing duplicate RuleSets. r=fitzgen.

Browse Source 
LUL (a Lightweight Unwind Library) performs unwinding on targets that use
Dwarf CFI metadata.  As each Linux/Android shared object is mapped into
memory, it reads unwind data from the objects .eh_frame and/or .debug_frame
sections, and from that info produces a set of canned how-to-unwind recipes,
called RuleSets, which are stored in a SecMap object.  There is one SecMap
object for each mapped object in the process.

Each RuleSet describes how to do a step of unwinding for some code address
range.  Most code address ranges are very short (a few bytes) and so there are
many RuleSets.  libxul.so as of Sept 2020 has approaching 4 million RuleSets,
for example.  Currently, each is 48 bytes long, and so storing them all
requires considerable space, over 200MB.

This patch reduces the storage requirement almost by a factor of 6.  The key
observation is that although there are many RuleSets, almost all of them are
duplicates.  libxul.so typically has less than 300 different RuleSets.  This
patch exploits that observation using two different compression schemes.

Firstly, it makes sense to store each different RuleSet only once, in a vector
("the dictionary").  Then, instead of storing (for libxul.so) a vector of 4
million 48-byte-sized RuleSets, we store a vector of 4 million triples, of the
form

 (code_address, len, dictionary_index)

If `code_address` is 64-bit, and we (entirely reasonably) constrain `len` and
`dictionary_index` to be 32 bits, then a triple takes 16 bytes.  This would
give a factor of 3 memory saving, assuming (again reasonably) that the
dictionary's size is insignificant.

Secondly, we observe that (a) all `code_address`es for any specific shared
object (hence, for the associated RuleSet) span at maximum about 120MB, (b)
that the highest observed `dictionary_index` is less than 400, and (c) that
almost all `len` values are less than 2^16.  Hence we can represent that
triple as

  (32-bit offset_from_RuleSet_base_address, 16-bit len, 16-bit dictionary_index)

For the few `len` values that won't fit into 16 bits, we can chop the range up
into a sequence of 2^16-1 long chunks.  This is exceedingly rare in practice.

With this arrangement, each triple is 8 bytes, hence giving the final
compression figure of 6 == 48 / 8.

In the patch, the triple is represented by a new struct, `Extent`.

This scheme is described (more or less) in
https://blog.mozilla.org/jseward/2013/09/03/how-compactly-can-cfiexidx-stack-unwinding-info-be-represented/

and there is background information on the representations at
https://blog.mozilla.org/jseward/2013/08/29/how-fast-can-cfiexidx-based-stack-unwinding-be/

---

Specific changes are:

class RuleSet: fields `mAddr` and `mLen`, which used to specify the address
range to which the RuleSet applied, have been removed.  They can no longer be
part of RuleSet because each RuleSet is now stored only once, and referenced
by each address range fragment which uses it.  The address information has
instead been moved to ..

struct Extent: this is a new, 8 byte structure, which specifies address
ranges, and indices into the dictionary of RuleSets, as described above.

class SecMap: this holds all the unwind information harvested from a single
Linux/Android shared object.

* Per the description above, the may-contain-duplicates vector of RuleSets,
  `mRuleSet`, has been removed.  Instead it is replaced by a vector of
  `Extent`s, `mExtents`, and the duplicate-free vector of RuleSets,
  `mDictionary`, entries in which are referred to from `mExtents`.

* `mDictionary` cannot be efficiently created until we know all the RuleSets
  that it will need to contain.  Hence, while reading unwind data, a hash
  table, `mUniqifier`, is used as an intermediate.  This maps RuleSets to
  unique integer IDs.  Once reading is complete, `mUniqifier` is enumerated in
  order of the unique IDs, and the RuleSets are copied into their correct
  locations in `mDictionary`.  `mUniqifier` is then deleted, and plays no
  further role.

In terms of actions, the main changes are:

* SecMap::AddRuleSet: the new RuleSet is looked up in `mUniqifier`, or added
  if missing.  This generates a dictionary-index for it.  This is the core of
  the de-duplication process.  Also, a new `mExtent` entry is added for the
  range.

* SecMap::PrepareRuleSets: this is called once all info has been read, but
  before we commence unwinding.  The `mExtent`s implied-address-ranges are
  sorted, trimmed and generally tidied up.  `mDictionary` is created from
  `mUniqifier` and the latter is deleted.

Secondary changes:

* SecMap::mSummaryMinAddr and SecMap::mSummaryMaxAddr have been removed and
  replaced by `mMapMinAVMA` and `mMapMaxAVMA`.

  `mSummaryMinAddr` and `mSummaryMaxAddr` previously held the minimum and
  maximum code addresses of any RuleSets in this SecMap.  However, computing
  them incrementally is no longer possible, and in any case we need to have a
  fixed address for the SecMap against which the Extent::offset fields are
  based.

  Hence we store instead the lowest and highest code addresses for the mapped
  text segment that this SecMap covers -- hence `mMapMinAVMA` and
  `mMapMaxAVMA`.  These are known before we start reading unwind info for this
  SecMap, and are guaranteed to be a superset of the range previously
  specified by `mSummaryMinAddr` and `mSummaryMaxAddr`.  These ranges are
  guaranteed not to overlap the ranges of any other SecMap in the system, and
  hence can still be used for their intended purpose of binary-searching to
  top level collection of SecMaps (which is owned by the one-and-only PriMap).

* Some comments have been cleaned up.  Some imprecise uses of the term
  "address" have been replaced with the more precise terminology "AVMA"
  (Actual Virtual Memory Address).  See existing comment at the top of
  LulMain.h.

Differential Revision: https://phabricator.services.mozilla.com/D90289
This commit is contained in:
Julian Seward 2020-09-16 10:18:36 +00:00
parent 706ceffb4b
commit d952612d22
3 changed files with 464 additions and 130 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
tools/profiler/lul/LulDwarfSummariser.cpp
View File

@ -96,12 +96,10 @@ void Summariser::Rule(uintptr_t aAddress, int aNewReg, LExprHow how,
if (mCurrAddr < aAddress) {
// Flush the existing summary first.
mCurrRules.mAddr = mCurrAddr;
mCurrRules.mLen = aAddress - mCurrAddr;
mSecMap->AddRuleSet(&mCurrRules);
mSecMap->AddRuleSet(&mCurrRules, mCurrAddr, aAddress - mCurrAddr);
if (DEBUG_SUMMARISER) {
mLog("LUL ");
mCurrRules.Print(mLog);
mCurrRules.Print(mCurrAddr, aAddress - mCurrAddr, mLog);
mLog("\n");
}
mCurrAddr = aAddress;
@ -539,12 +537,10 @@ void Summariser::End() {
mLog("LUL End\n");
}
if (mCurrAddr < mMax1Addr) {
mCurrRules.mAddr = mCurrAddr;
mCurrRules.mLen = mMax1Addr - mCurrAddr;
mSecMap->AddRuleSet(&mCurrRules);
mSecMap->AddRuleSet(&mCurrRules, mCurrAddr, mMax1Addr - mCurrAddr);
if (DEBUG_SUMMARISER) {
mLog("LUL ");
mCurrRules.Print(mLog);
mCurrRules.Print(mCurrAddr, mMax1Addr - mCurrAddr, mLog);
mLog("\n");
}
}

302
tools/profiler/lul/LulMain.cpp
View File

@ -127,10 +127,11 @@ string LExpr::ShowRule(const char* aNewReg) const {
return res;
}
void RuleSet::Print(void (*aLog)(const char*)) const {
void RuleSet::Print(uintptr_t avma, uintptr_t len,
void (*aLog)(const char*)) const {
char buf[96];
SprintfLiteral(buf, "[%llx .. %llx]: let ", (unsigned long long int)mAddr,
(unsigned long long int)(mAddr + mLen - 1));
SprintfLiteral(buf, "[%llx .. %llx]: let ", (unsigned long long int)avma,
(unsigned long long int)(avma + len - 1));
string res = string(buf);
res += mCfaExpr.ShowRule("cfa");
res += " in";
@ -207,10 +208,7 @@ LExpr* RuleSet::ExprForRegno(DW_REG_NUMBER aRegno) {
}
RuleSet::RuleSet() {
mAddr = 0;
mLen = 0;
// The only other fields are of type LExpr and those are initialised
// by LExpr::LExpr().
// All fields are of type LExpr and so are initialised by LExpr::LExpr().
}
////////////////////////////////////////////////////////////////
@ -219,23 +217,43 @@ RuleSet::RuleSet() {
// See header file LulMainInt.h for comments about invariants.
SecMap::SecMap(void (*aLog)(const char*))
: mSummaryMinAddr(1), mSummaryMaxAddr(0), mUsable(true), mLog(aLog) {}
SecMap::SecMap(uintptr_t mapStartAVMA, uint32_t mapLen,
void (*aLog)(const char*))
: mUsable(false),
mUniqifier(new mozilla::HashMap<RuleSet, uint32_t, RuleSet,
InfallibleAllocPolicy>),
mLog(aLog) {
if (mapLen == 0) {
// Degenerate case.
mMapMinAVMA = 1;
mMapMaxAVMA = 0;
} else {
mMapMinAVMA = mapStartAVMA;
mMapMaxAVMA = mapStartAVMA + (uintptr_t)mapLen - 1;
}
}
SecMap::~SecMap() { mRuleSets.clear(); }
SecMap::~SecMap() {
mExtents.clear();
mDictionary.clear();
if (mUniqifier) {
mUniqifier->clear();
mUniqifier = nullptr;
}
}
// RUNS IN NO-MALLOC CONTEXT
RuleSet* SecMap::FindRuleSet(uintptr_t ia) {
// Binary search mRuleSets to find one that brackets |ia|.
// Binary search mExtents to find one that brackets |ia|.
// lo and hi need to be signed, else the loop termination tests
// don't work properly. Note that this works correctly even when
// mRuleSets.size() == 0.
// mExtents.size() == 0.
// Can't do this until the array has been sorted and preened.
MOZ_ASSERT(mUsable);
long int lo = 0;
long int hi = (long int)mRuleSets.size() - 1;
long int hi = (long int)mExtents.size() - 1;
while (true) {
// current unsearched space is from lo to hi, inclusive.
if (lo > hi) {
@ -243,9 +261,11 @@ RuleSet* SecMap::FindRuleSet(uintptr_t ia) {
return nullptr;
}
long int mid = lo + ((hi - lo) / 2);
RuleSet* mid_ruleSet = &mRuleSets[mid];
uintptr_t mid_minAddr = mid_ruleSet->mAddr;
uintptr_t mid_maxAddr = mid_minAddr + mid_ruleSet->mLen - 1;
Extent* mid_extent = &mExtents[mid];
uintptr_t mid_offset = mid_extent->offset();
uintptr_t mid_len = mid_extent->len();
uintptr_t mid_minAddr = mMapMinAVMA + mid_offset;
uintptr_t mid_maxAddr = mid_minAddr + mid_len - 1;
if (ia < mid_minAddr) {
hi = mid - 1;
continue;
@ -255,16 +275,68 @@ RuleSet* SecMap::FindRuleSet(uintptr_t ia) {
continue;
}
MOZ_ASSERT(mid_minAddr <= ia && ia <= mid_maxAddr);
return mid_ruleSet;
uint32_t mid_extent_dictIx = mid_extent->dictIx();
MOZ_RELEASE_ASSERT(mid_extent_dictIx < mExtents.size());
return &mDictionary[mid_extent_dictIx];
}
// NOTREACHED
}
// Add a RuleSet to the collection. The rule is copied in. Calling
// this makes the map non-searchable.
void SecMap::AddRuleSet(const RuleSet* rs) {
void SecMap::AddRuleSet(const RuleSet* rs, uintptr_t avma, uintptr_t len) {
mUsable = false;
mRuleSets.push_back(*rs);
// Zero length RuleSet? Meaningless, but ignore it anyway.
if (len == 0) {
return;
}
// Ignore attempts to add RuleSets whose address range doesn't fall within
// the declared address range for the SecMap. Maybe we should print some
// kind of error message rather than silently ignoring them.
if (!(avma >= mMapMinAVMA && avma + len - 1 <= mMapMaxAVMA)) {
return;
}
// Because `mMapStartAVMA` .. `mMapEndAVMA` can specify at most a 2^32-1 byte
// chunk of address space, the following must now hold.
MOZ_RELEASE_ASSERT(len <= (uintptr_t)0xFFFFFFFF);
// See if `mUniqifier` already has `rs`. If so set `dictIx` to the assigned
// dictionary index; if not, add `rs` to `mUniqifier` and assign a new
// dictionary index. This is the core of the RuleSet-de-duplication process.
uint32_t dictIx = 0;
mozilla::HashMap<RuleSet, uint32_t, RuleSet, InfallibleAllocPolicy>::AddPtr
p = mUniqifier->lookupForAdd(*rs);
if (!p) {
dictIx = mUniqifier->count();
// If this ever fails, Extents::dictIx will need to be changed to be a
// type wider than the current uint16_t.
MOZ_RELEASE_ASSERT(dictIx < (1 << 16));
// This returns `false` on OOM. We ignore the return value since we asked
// for it to use the InfallibleAllocPolicy.
DebugOnly<bool> addedOK = mUniqifier->add(p, *rs, dictIx);
MOZ_ASSERT(addedOK);
} else {
dictIx = p->value();
}
uint32_t offset = (uint32_t)(avma - mMapMinAVMA);
while (len > 0) {
// Because Extents::len is a uint16_t, we have to add multiple `mExtents`
// entries to cover the case where `len` is equal to or greater than 2^16.
// This happens only exceedingly rarely. In order to get more test
// coverage on what would otherwise be a very low probability (less than
// 0.0002%) corner case, we do this in steps of 4095. On libxul.so as of
// Sept 2020, this increases the number of `mExtents` entries by about
// 0.05%, hence has no meaningful effect on space use, but increases the
// use of this corner case, and hence its test coverage, by a factor of 250.
uint32_t this_step_len = (len > 4095) ? 4095 : len;
mExtents.emplace_back(offset, this_step_len, dictIx);
offset += this_step_len;
len -= this_step_len;
}
}
// Add a PfxInstr to the vector of such instrs, and return the index
@ -275,36 +347,60 @@ uint32_t SecMap::AddPfxInstr(PfxInstr pfxi) {
return mPfxInstrs.size() - 1;
}
static bool CmpRuleSetsByAddrLE(const RuleSet& rs1, const RuleSet& rs2) {
return rs1.mAddr < rs2.mAddr;
static bool CmpExtentsByOffsetLE(const Extent& ext1, const Extent& ext2) {
return ext1.offset() < ext2.offset();
}
// Prepare the map for searching. Completely remove any which don't
// fall inside the specified range [start, +len).
void SecMap::PrepareRuleSets(uintptr_t aStart, size_t aLen) {
if (mRuleSets.empty()) {
// Prepare the map for searching, by sorting it, de-overlapping entries and
// removing any resulting zero-length entries. At the start of this routine,
// all Extents should fall within [mMapMinAVMA, mMapMaxAVMA] and not have zero
// length, as a result of the checks in AddRuleSet().
void SecMap::PrepareRuleSets() {
// At this point, the de-duped RuleSets are in `mUniqifier`, and
// `mDictionary` is empty. This method will, amongst other things, copy
// them into `mDictionary` in order of their assigned dictionary-index
// values, as established by `SecMap::AddRuleSet`, and free `mUniqifier`;
// after this method, it has no further use.
MOZ_RELEASE_ASSERT(mUniqifier);
MOZ_RELEASE_ASSERT(mDictionary.empty());
if (mExtents.empty()) {
mUniqifier->clear();
mUniqifier = nullptr;
return;
}
MOZ_ASSERT(aLen > 0);
if (aLen == 0) {
// This should never happen.
mRuleSets.clear();
if (mMapMinAVMA == 1 && mMapMaxAVMA == 0) {
// The map is empty. This should never happen.
mExtents.clear();
mUniqifier->clear();
mUniqifier = nullptr;
return;
}
MOZ_RELEASE_ASSERT(mMapMinAVMA <= mMapMaxAVMA);
// We must have at least one Extent, and as a consequence there must be at
// least one entry in the uniqifier.
MOZ_RELEASE_ASSERT(!mExtents.empty() && !mUniqifier->empty());
#ifdef DEBUG
// Check invariants on incoming Extents.
for (size_t i = 0; i < mExtents.size(); ++i) {
Extent* ext = &mExtents[i];
uint32_t len = ext->len();
MOZ_ASSERT(len > 0);
MOZ_ASSERT(len <= 4095 /* per '4095' in AddRuleSet() */);
uint32_t offset = ext->offset();
uintptr_t avma = mMapMinAVMA + (uintptr_t)offset;
// Upper bounds test. There's no lower bounds test because `offset` is a
// positive displacement from `mMapMinAVMA`, so a small underrun will
// manifest as `len` being close to 2^32.
MOZ_ASSERT(avma + (uintptr_t)len - 1 <= mMapMaxAVMA);
}
#endif
// Sort by start addresses.
std::sort(mRuleSets.begin(), mRuleSets.end(), CmpRuleSetsByAddrLE);
// Detect any entry not completely contained within [start, +len).
// Set its length to zero, so that the next pass will remove it.
for (size_t i = 0; i < mRuleSets.size(); ++i) {
RuleSet* rs = &mRuleSets[i];
if (rs->mLen > 0 &&
(rs->mAddr < aStart || rs->mAddr + rs->mLen > aStart + aLen)) {
rs->mLen = 0;
}
}
std::sort(mExtents.begin(), mExtents.end(), CmpExtentsByOffsetLE);
// Iteratively truncate any overlaps and remove any zero length
// entries that might result, or that may have been present
@ -312,7 +408,7 @@ void SecMap::PrepareRuleSets(uintptr_t aStart, size_t aLen) {
// expected to iterate only once.
while (true) {
size_t i;
size_t n = mRuleSets.size();
size_t n = mExtents.size();
size_t nZeroLen = 0;
if (n == 0) {
@ -320,16 +416,18 @@ void SecMap::PrepareRuleSets(uintptr_t aStart, size_t aLen) {
}
for (i = 1; i < n; ++i) {
RuleSet* prev = &mRuleSets[i - 1];
RuleSet* here = &mRuleSets[i];
MOZ_ASSERT(prev->mAddr <= here->mAddr);
if (prev->mAddr + prev->mLen > here->mAddr) {
prev->mLen = here->mAddr - prev->mAddr;
Extent* prev = &mExtents[i - 1];
Extent* here = &mExtents[i];
MOZ_ASSERT(prev->offset() <= here->offset());
if (prev->offset() + prev->len() > here->offset()) {
prev->setLen(here->offset() - prev->offset());
}
if (prev->len() == 0) {
nZeroLen++;
}
if (prev->mLen == 0) nZeroLen++;
}
if (mRuleSets[n - 1].mLen == 0) {
if (mExtents[n - 1].len() == 0) {
nZeroLen++;
}
@ -342,53 +440,67 @@ void SecMap::PrepareRuleSets(uintptr_t aStart, size_t aLen) {
// Slide back the entries to remove the zero length ones.
size_t j = 0; // The write-point.
for (i = 0; i < n; ++i) {
if (mRuleSets[i].mLen == 0) {
if (mExtents[i].len() == 0) {
continue;
}
if (j != i) mRuleSets[j] = mRuleSets[i];
if (j != i) {
mExtents[j] = mExtents[i];
}
++j;
}
MOZ_ASSERT(i == n);
MOZ_ASSERT(nZeroLen <= n);
MOZ_ASSERT(j == n - nZeroLen);
while (nZeroLen > 0) {
mRuleSets.pop_back();
mExtents.pop_back();
nZeroLen--;
}
MOZ_ASSERT(mRuleSets.size() == j);
MOZ_ASSERT(mExtents.size() == j);
}
size_t n = mRuleSets.size();
size_t nExtents = mExtents.size();
#ifdef DEBUG
// Do a final check on the rules: their address ranges must be
// Do a final check on the extents: their address ranges must be
// ascending, non overlapping, non zero sized.
if (n > 0) {
MOZ_ASSERT(mRuleSets[0].mLen > 0);
for (size_t i = 1; i < n; ++i) {
RuleSet* prev = &mRuleSets[i - 1];
RuleSet* here = &mRuleSets[i];
MOZ_ASSERT(prev->mAddr < here->mAddr);
MOZ_ASSERT(here->mLen > 0);
MOZ_ASSERT(prev->mAddr + prev->mLen <= here->mAddr);
if (nExtents > 0) {
MOZ_ASSERT(mExtents[0].len() > 0);
for (size_t i = 1; i < nExtents; ++i) {
const Extent* prev = &mExtents[i - 1];
const Extent* here = &mExtents[i];
MOZ_ASSERT(prev->offset() < here->offset());
MOZ_ASSERT(here->len() > 0);
MOZ_ASSERT(prev->offset() + prev->len() <= here->offset());
}
}
#endif
// Set the summary min and max address values.
if (n == 0) {
// Use the values defined in comments in the class declaration.
mSummaryMinAddr = 1;
mSummaryMaxAddr = 0;
} else {
mSummaryMinAddr = mRuleSets[0].mAddr;
mSummaryMaxAddr = mRuleSets[n - 1].mAddr + mRuleSets[n - 1].mLen - 1;
// Create the final dictionary by enumerating the uniqifier.
size_t nUniques = mUniqifier->count();
RuleSet dummy;
mozilla::PodZero(&dummy);
mDictionary.reserve(nUniques);
for (size_t i = 0; i < nUniques; i++) {
mDictionary.push_back(dummy);
}
for (auto iter = mUniqifier->iter(); !iter.done(); iter.next()) {
MOZ_RELEASE_ASSERT(iter.get().value() < nUniques);
mDictionary[iter.get().value()] = iter.get().key();
}
mUniqifier = nullptr;
char buf[150];
SprintfLiteral(buf, "PrepareRuleSets: %d entries, smin/smax 0x%llx, 0x%llx\n",
(int)n, (unsigned long long int)mSummaryMinAddr,
(unsigned long long int)mSummaryMaxAddr);
SprintfLiteral(
buf,
"PrepareRuleSets: %lu extents, %lu rulesets, "
"avma min/max 0x%llx, 0x%llx\n",
(unsigned long int)nExtents, (unsigned long int)mDictionary.size(),
(unsigned long long int)mMapMinAVMA, (unsigned long long int)mMapMaxAVMA);
buf[sizeof(buf) - 1] = 0;
mLog(buf);
@ -397,24 +509,31 @@ void SecMap::PrepareRuleSets(uintptr_t aStart, size_t aLen) {
#if 0
mLog("\nRulesets after preening\n");
for (size_t i = 0; i < mRuleSets.size(); ++i) {
mRuleSets[i].Print(mLog);
for (size_t i = 0; i < nExtents; ++i) {
const Extent* extent = &mExtents[i];
uintptr_t avma = mMapMinAVMA + (uintptr_t)extent->offset();
mDictionary[extent->dictIx()].Print(avma, extent->len(), mLog);
mLog("\n");
}
mLog("\n");
#endif
}
bool SecMap::IsEmpty() { return mRuleSets.empty(); }
bool SecMap::IsEmpty() { return mExtents.empty(); }
size_t SecMap::SizeOfIncludingThis(MallocSizeOf aMallocSizeOf) const {
size_t n = aMallocSizeOf(this);
// It's conceivable that these calls would be unsafe with some
// implementations of std::vector, but it seems to be working for now...
n += aMallocSizeOf(mRuleSets.data());
n += aMallocSizeOf(mPfxInstrs.data());
if (mUniqifier) {
n += mUniqifier->shallowSizeOfIncludingThis(aMallocSizeOf);
}
n += aMallocSizeOf(mDictionary.data());
n += aMallocSizeOf(mExtents.data());
return n;
}
@ -567,14 +686,14 @@ class PriMap {
// non-overlapping invariant is preserved (and, generally, holds).
// FIXME: this gives a cost that is O(N^2) in the total number of
// shared objects in the system. ToDo: better.
MOZ_ASSERT(aSecMap->mSummaryMinAddr <= aSecMap->mSummaryMaxAddr);
MOZ_ASSERT(aSecMap->mMapMinAVMA <= aSecMap->mMapMaxAVMA);
size_t num_secMaps = mSecMaps.size();
uintptr_t i;
for (i = 0; i < num_secMaps; ++i) {
mozilla::UniquePtr<SecMap>& sm_i = mSecMaps[i];
MOZ_ASSERT(sm_i->mSummaryMinAddr <= sm_i->mSummaryMaxAddr);
if (aSecMap->mSummaryMinAddr < sm_i->mSummaryMaxAddr) {
MOZ_ASSERT(sm_i->mMapMinAVMA <= sm_i->mMapMaxAVMA);
if (aSecMap->mMapMinAVMA < sm_i->mMapMaxAVMA) {
// |aSecMap| needs to be inserted immediately before mSecMaps[i].
break;
}
@ -608,8 +727,7 @@ class PriMap {
// to the number of elements in the map.
for (i = (intptr_t)num_secMaps - 1; i >= 0; i--) {
mozilla::UniquePtr<SecMap>& sm_i = mSecMaps[i];
if (sm_i->mSummaryMaxAddr < avma_min ||
avma_max < sm_i->mSummaryMinAddr) {
if (sm_i->mMapMaxAVMA < avma_min || avma_max < sm_i->mMapMinAVMA) {
// There's no overlap. Move on.
continue;
}
@ -653,8 +771,8 @@ class PriMap {
}
long int mid = lo + ((hi - lo) / 2);
mozilla::UniquePtr<SecMap>& mid_secMap = mSecMaps[mid];
uintptr_t mid_minAddr = mid_secMap->mSummaryMinAddr;
uintptr_t mid_maxAddr = mid_secMap->mSummaryMaxAddr;
uintptr_t mid_minAddr = mid_secMap->mMapMinAVMA;
uintptr_t mid_maxAddr = mid_secMap->mMapMaxAVMA;
if (ia < mid_minAddr) {
hi = mid - 1;
continue;
@ -765,10 +883,20 @@ void LUL::NotifyAfterMap(uintptr_t aRXavma, size_t aSize, const char* aFileName,
buf[sizeof(buf) - 1] = 0;
mLog(buf);
// We can't have a SecMap covering more than 2^32-1 bytes of address space.
// See the definition of SecMap for why. Rather than crash the system, just
// limit the SecMap's size accordingly. This case is never actually
// expected to happen.
if (((unsigned long long int)aSize) > 0xFFFFFFFFULL) {
aSize = (uintptr_t)0xFFFFFFFF;
}
MOZ_RELEASE_ASSERT(aSize <= 0xFFFFFFFF);
// Ignore obviously-stupid notifications.
if (aSize > 0) {
// Here's a new mapping, for this object.
mozilla::UniquePtr<SecMap> smap = mozilla::MakeUnique<SecMap>(mLog);
mozilla::UniquePtr<SecMap> smap =
mozilla::MakeUnique<SecMap>(aRXavma, (uint32_t)aSize, mLog);
// Read CFI or EXIDX unwind data into |smap|.
if (!aMappedImage) {
@ -782,7 +910,7 @@ void LUL::NotifyAfterMap(uintptr_t aRXavma, size_t aSize, const char* aFileName,
mLog("NotifyMap .. preparing entries\n");
smap->PrepareRuleSets(aRXavma, aSize);
smap->PrepareRuleSets();
SprintfLiteral(buf, "NotifyMap got %lld entries\n",
(long long int)smap->Size());
@ -1396,7 +1524,7 @@ void LUL::Unwind(/*OUT*/ uintptr_t* aFramePCs,
// So, do we have a ruleset for this address? If so, use it now.
if (ruleset) {
if (DEBUG_MAIN) {
ruleset->Print(mLog);
ruleset->Print(ia.Value(), 1 /*bogus, but doesn't matter*/, mLog);
mLog("\n");
}
// Use the RuleSet to compute the registers for the previous

280
tools/profiler/lul/LulMainInt.h
View File

@ -14,8 +14,10 @@
#include <vector>
#include "mozilla/Assertions.h"
#include "mozilla/HashFunctions.h"
#include "mozilla/HashTable.h"
// This file is provides internal interface inside LUL. If you are an
// This file provides an internal interface inside LUL. If you are an
// end-user of LUL, do not include it in your code. The end-user
// interface is in LulMain.h.
@ -161,7 +163,45 @@ struct LExpr {
MOZ_ASSERT(reg == 0 && offset >= 0);
break;
default:
MOZ_ASSERT(0, "LExpr::LExpr: invalid how");
MOZ_RELEASE_ASSERT(0, "LExpr::LExpr: invalid how");
}
}
// Hash it, carefully looking only at defined parts.
mozilla::HashNumber hash() const {
mozilla::HashNumber h = mHow;
switch (mHow) {
case UNKNOWN:
break;
case NODEREF:
case DEREF:
h = mozilla::AddToHash(h, mReg);
h = mozilla::AddToHash(h, mOffset);
break;
case PFXEXPR:
h = mozilla::AddToHash(h, mOffset);
break;
default:
MOZ_RELEASE_ASSERT(0, "LExpr::hash: invalid how");
}
return h;
}
// And structural equality.
bool equals(const LExpr& other) const {
if (mHow != other.mHow) {
return false;
}
switch (mHow) {
case UNKNOWN:
return true;
case NODEREF:
case DEREF:
return mReg == other.mReg && mOffset == other.mOffset;
case PFXEXPR:
return mOffset == other.mOffset;
default:
MOZ_RELEASE_ASSERT(0, "LExpr::equals: invalid how");
}
}
@ -212,9 +252,12 @@ static_assert(sizeof(LExpr) <= 8, "LExpr size changed unexpectedly");
// RuleSet //
////////////////////////////////////////////////////////////////
// This is platform-dependent. For some address range, describes how
// to recover the CFA and then how to recover the registers for the
// previous frame.
// This is platform-dependent. It describes how to recover the CFA and then
// how to recover the registers for the previous frame. Such "recipes" are
// specific to particular ranges of machine code, but the associated range
// is not stored in RuleSet, because in general each RuleSet may be used
// for many such range fragments ("extents"). See the comments below for
// Extent and SecMap.
//
// The set of LExprs contained in a given RuleSet describe a DAG which
// says how to compute the caller's registers ("new registers") from
@ -255,13 +298,11 @@ static_assert(sizeof(LExpr) <= 8, "LExpr size changed unexpectedly");
class RuleSet {
public:
RuleSet();
void Print(void (*aLog)(const char*)) const;
void Print(uintptr_t avma, uintptr_t len, void (*aLog)(const char*)) const;
// Find the LExpr* for a given DW_REG_ value in this class.
LExpr* ExprForRegno(DW_REG_NUMBER aRegno);
uintptr_t mAddr;
uintptr_t mLen;
// How to compute the CFA.
LExpr mCfaExpr;
// How to compute caller register values. These may reference the
@ -288,6 +329,59 @@ class RuleSet {
#else
# error "Unknown arch"
#endif
// Machinery in support of hashing.
typedef RuleSet Lookup;
static mozilla::HashNumber hash(RuleSet rs) {
mozilla::HashNumber h = rs.mCfaExpr.hash();
#if defined(GP_ARCH_amd64) || defined(GP_ARCH_x86)
h = mozilla::AddToHash(h, rs.mXipExpr.hash());
h = mozilla::AddToHash(h, rs.mXspExpr.hash());
h = mozilla::AddToHash(h, rs.mXbpExpr.hash());
#elif defined(GP_ARCH_arm)
h = mozilla::AddToHash(h, rs.mR15expr.hash());
h = mozilla::AddToHash(h, rs.mR14expr.hash());
h = mozilla::AddToHash(h, rs.mR13expr.hash());
h = mozilla::AddToHash(h, rs.mR12expr.hash());
h = mozilla::AddToHash(h, rs.mR11expr.hash());
h = mozilla::AddToHash(h, rs.mR7expr.hash());
#elif defined(GP_ARCH_arm64)
h = mozilla::AddToHash(h, rs.mX29expr.hash());
h = mozilla::AddToHash(h, rs.mX30expr.hash());
h = mozilla::AddToHash(h, rs.mSPexpr.hash());
#elif defined(GP_ARCH_mips64)
h = mozilla::AddToHash(h, rs.mPCexpr.hash());
h = mozilla::AddToHash(h, rs.mFPexpr.hash());
h = mozilla::AddToHash(h, rs.mSPexpr.hash());
#else
# error "Unknown arch"
#endif
return h;
}
static bool match(const RuleSet& rs1, const RuleSet& rs2) {
return rs1.mCfaExpr.equals(rs2.mCfaExpr) &&
#if defined(GP_ARCH_amd64) || defined(GP_ARCH_x86)
rs1.mXipExpr.equals(rs2.mXipExpr) &&
rs1.mXspExpr.equals(rs2.mXspExpr) &&
rs1.mXbpExpr.equals(rs2.mXbpExpr);
#elif defined(GP_ARCH_arm)
rs1.mR15expr.equals(rs2.mR15expr) &&
rs1.mR14expr.equals(rs2.mR14expr) &&
rs1.mR13expr.equals(rs2.mR13expr) &&
rs1.mR12expr.equals(rs2.mR12expr) &&
rs1.mR11expr.equals(rs2.mR11expr) && rs1.mR7expr.equals(rs2.mR7expr);
#elif defined(GP_ARCH_arm64)
rs1.mX29expr.equals(rs2.mX29expr) &&
rs1.mX30expr.equals(rs2.mX30expr) && rs1.mSPexpr.equals(rs2.mSPexpr);
#elif defined(GP_ARCH_mips64)
rs1.mPCexpr.equals(rs2.mPCexpr) && rs1.mFPexpr.equals(rs2.mFPexpr) &&
rs1.mSPexpr.equals(rs2.mSPexpr);
#else
# error "Unknown arch"
#endif
}
};
// Returns |true| for Dwarf register numbers which are members
@ -325,6 +419,88 @@ static inline bool registerIsTracked(DW_REG_NUMBER reg) {
}
}
////////////////////////////////////////////////////////////////
// Extent //
////////////////////////////////////////////////////////////////
struct Extent {
// Three fields, which together take 8 bytes.
uint32_t mOffset;
uint16_t mLen;
uint16_t mDictIx;
// What this means is: suppose we are looking for the unwind rules for some
// code address (AVMA) `avma`. If we can find some SecMap `secmap` such
// that `avma` falls in the range
//
// `[secmap.mMapMinAVMA, secmap.mMapMaxAVMA]`
//
// then the RuleSet to use is `secmap.mDictionary[dictIx]` iff we can find
// an `extent` in `secmap.mExtents` such that `avma` falls into the range
//
// `[secmap.mMapMinAVMA + extent.offset(),
// secmap.mMapMinAVMA + extent.offset() + extent.len())`.
//
// Packing Extent into the minimum space is important, since there will be
// huge numbers of Extents -- around 3 million for libxul.so as of Sept
// 2020. Here, we aim for an 8-byte size, with the field sizes chosen
// carefully, as follows:
//
// `offset` denotes a byte offset inside the text section for some shared
// object. libxul.so is by far the largest. As of Sept 2020 it has a text
// size of up to around 120MB, that is, close to 2^27 bytes. Hence a 32-bit
// `offset` field gives a safety margin of around a factor of 32
// (== 2 ^(32 - 27)).
//
// `dictIx` indicates a unique `RuleSet` for some code address range.
// Experimentation on x86_64-linux indicates that only around 300 different
// `RuleSet`s exist, for libxul.so. A 16-bit bit field allows up to 65536
// to be recorded, hence leaving us a generous safety margin.
//
// `len` indicates the length of the associated address range.
//
// Note the representation becomes unusable if either `offset` overflows 32
// bits or `dictIx` overflows 16 bits. On the other hand, it does not
// matter (although is undesirable) if `len` overflows 16 bits, because in
// that case we can add multiple size-65535 entries to `secmap.mExtents` to
// cover the entire range. Hence the field sizes are biased so as to give a
// good safety margin for `offset` and `dictIx` at the cost of stealing bits
// from `len`. Almost all `len` values we will ever see in practice are
// 65535 or less, so stealing those bits does not matter much.
//
// If further compression is required, it would be feasible to implement
// Extent using 29 bits for the offset, 8 bits for the length and 11 bits
// for the dictionary index, giving a total of 6 bytes, provided that the
// data is packed into 3 uint16_t's. That would be a bit slower, though,
// due to the bit packing, and it would be more fragile, in the sense that
// it would fail for any object with more than 512MB of text segment, or
// with more than 2048 different `RuleSet`s. For the current (Sept 2020)
// libxul.so situation, though, it would work fine.
Extent(uint32_t offset, uint32_t len, uint32_t dictIx) {
MOZ_RELEASE_ASSERT(len < (1 << 16));
MOZ_RELEASE_ASSERT(dictIx < (1 << 16));
mOffset = offset;
mLen = len;
mDictIx = dictIx;
}
uint32_t offset() const { return mOffset; }
uint32_t len() const { return mLen; }
uint32_t dictIx() const { return mDictIx; }
void setLen(uint32_t len) {
MOZ_RELEASE_ASSERT(len < (1 << 16));
mLen = len;
}
void Print(void (*aLog)(const char*)) const {
char buf[64];
SprintfLiteral(buf, "Extent(offs=0x%x, len=%u, dictIx=%u)", this->offset(),
this->len(), this->dictIx());
aLog(buf);
}
};
static_assert(sizeof(Extent) == 8);
////////////////////////////////////////////////////////////////
// SecMap //
////////////////////////////////////////////////////////////////
@ -336,26 +512,45 @@ static inline bool registerIsTracked(DW_REG_NUMBER reg) {
class SecMap {
public:
// These summarise the contained mRuleSets, in that they give
// exactly the lowest and highest addresses that any of the entries
// in this SecMap cover. Hence invariants:
// In the constructor, `mapStartAVMA` and `mapLen` define the actual
// (in-process) virtual addresses covered by the SecMap. All RuleSets
// subsequently added to it by calling `AddRuleSet` must fall into this
// address range, and attempts to add ones outside the range will be
// ignored. This restriction exists because the type Extent (see below)
// indicates an address range for a RuleSet, but for reasons of compactness,
// it does not contain the start address of the range. Instead, it contains
// a 32-bit offset from the base address of the SecMap. This is also the
// reason why the map's size is a `uint32_t` and not a `uintptr_t`.
//
// mRuleSets is nonempty
// <=> mSummaryMinAddr <= mSummaryMaxAddr
// && mSummaryMinAddr == mRuleSets[0].mAddr
// && mSummaryMaxAddr == mRuleSets[#rulesets-1].mAddr
// + mRuleSets[#rulesets-1].mLen - 1;
// The effect is to limit this mechanism to shared objects / executables
// whose text section size does not exceed 4GB (2^32 bytes). Given that, as
// of Sept 2020, libxul.so's text section size is around 120MB, this does
// not seem like much of a limitation.
//
// This requires that no RuleSet has zero length.
// From the supplied `mapStartAVMA` and `mapLen`, fields `mMapMinAVMA` and
// `mMapMaxAVMA` are calculated. It is intended that no two SecMaps owned
// by the same PriMap contain overlapping address ranges, and the PriMap
// logic enforces that.
//
// mRuleSets is empty
// <=> mSummaryMinAddr > mSummaryMaxAddr
// Some invariants:
//
// This doesn't constrain mSummaryMinAddr and mSummaryMaxAddr uniquely,
// so let's use mSummaryMinAddr == 1 and mSummaryMaxAddr == 0 to denote
// this case.
// mExtents is nonempty
// <=> mMapMinAVMA <= mMapMaxAVMA
// && mMapMinAVMA <= apply_delta(mExtents[0].offset())
// && apply_delta(mExtents[#rulesets-1].offset()
// + mExtents[#rulesets-1].len() - 1) <= mMapMaxAVMA
// where
// apply_delta(off) = off + mMapMinAVMA
//
// This requires that no RuleSet has zero length.
//
// mExtents is empty
// <=> mMapMinAVMA > mMapMaxAVMA
//
// This doesn't constrain mMapMinAVMA and mMapMaxAVMA uniquely, so let's use
// mMapMinAVMA == 1 and mMapMaxAVMA == 0 to denote this case.
explicit SecMap(void (*aLog)(const char*));
SecMap(uintptr_t mapStartAVMA, uint32_t mapLen, void (*aLog)(const char*));
~SecMap();
// Binary search mRuleSets to find one that brackets |ia|, or nullptr
@ -365,7 +560,7 @@ class SecMap {
// Add a RuleSet to the collection. The rule is copied in. Calling
// this makes the map non-searchable.
void AddRuleSet(const RuleSet* rs);
void AddRuleSet(const RuleSet* rs, uintptr_t avma, uintptr_t len);
// Add a PfxInstr to the vector of such instrs, and return the index
// in the vector. Calling this makes the map non-searchable.
@ -374,29 +569,44 @@ class SecMap {
// Returns the entire vector of PfxInstrs.
const vector<PfxInstr>* GetPfxInstrs() { return &mPfxInstrs; }
// Prepare the map for searching. Also, remove any rules for code
// address ranges which don't fall inside [start, +len). |len| may
// not be zero.
void PrepareRuleSets(uintptr_t start, size_t len);
// Prepare the map for searching, by sorting it, de-overlapping entries and
// removing any resulting zero-length entries. At the start of this
// routine, all Extents should fall within [mMapMinAVMA, mMapMaxAVMA] and
// not have zero length, as a result of the checks in AddRuleSet().
void PrepareRuleSets();
bool IsEmpty();
size_t Size() { return mRuleSets.size(); }
size_t Size() { return mExtents.size() + mDictionary.size(); }
size_t SizeOfIncludingThis(mozilla::MallocSizeOf aMallocSizeOf) const;
// The min and max addresses of the addresses in the contained
// RuleSets. See comment above for invariants.
uintptr_t mSummaryMinAddr;
uintptr_t mSummaryMaxAddr;
// The extent of this SecMap as a whole. The extents of all contained
// RuleSets must fall inside this. See comment above for details.
uintptr_t mMapMinAVMA;
uintptr_t mMapMaxAVMA;
private:
// False whilst adding entries; true once it is safe to call FindRuleSet.
// Transition (false->true) is caused by calling PrepareRuleSets().
bool mUsable;
// A vector of RuleSets, sorted, nonoverlapping (post Prepare()).
vector<RuleSet> mRuleSets;
// This is used to find and remove duplicate RuleSets while we are adding
// them to the SecMap. Almost all RuleSets are duplicates, so de-duping
// them is a huge space win. This is non-null while `mUsable` is false, and
// becomes null (is discarded) after the call to PrepareRuleSets, which
// copies all the entries into `mDictionary`.
mozilla::UniquePtr<
mozilla::HashMap<RuleSet, uint32_t, RuleSet, InfallibleAllocPolicy>>
mUniqifier;
// This will contain final contents of `mUniqifier`, but ordered
// (implicitly) by the `uint32_t` value fields, for fast access.
vector<RuleSet> mDictionary;
// A vector of Extents, sorted by offset value, nonoverlapping (post
// PrepareRuleSets()).
vector<Extent> mExtents;
// A vector of PfxInstrs, which are referred to by the RuleSets.
// These are provided as a representation of Dwarf expressions