ValueMapper: Eliminate cross-file co-recursion, NFC

Eliminate co-recursion of Mapper::mapValue through
ValueMaterializer::materializeInitFor, through a major redesign of the
ValueMapper.cpp interface.

  - Expose a ValueMapper class that controls the entry points to the
    mapping algorithms.
  - Change IRLinker to use ValueMapper directly, rather than
    llvm::RemapInstruction, llvm::MapValue, etc.
  - Use (e.g.) ValueMapper::scheduleMapGlobalInit to add mapping work to
    a worklist in ValueMapper instead of recursing.

There were two fairly major complications.

Firstly, IRLinker::linkAppendingVarProto incorporates an on-the-fly IR
ugprade that I had to split apart.  Long-term, this upgrade should be
done in the bitcode reader (and we should only accept the "new" form),
but for now I've just made it work and added a FIXME.  The hold-op is
that we need to deprecate C API that relies on this.

Secondly, IRLinker has special logic to correctly implement aliases with
comdats, and uses two ValueToValueMapTy instances and two
ValueMaterializers.  I supported this by allowing clients to register an
alternate mapping context, whose MCID can be passed in when scheduling
new work.

While out of scope for this commit, it should now be straightforward to
remove recursion from Mapper::mapValue.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266503 91177308-0d34-0410-b5e6-96231b3b80d8

include/llvm/Transforms/Utils/ValueMapper.h

@@ -98,6 +98,92 @@ static inline RemapFlags operator|(RemapFlags LHS, RemapFlags RHS) {
   return RemapFlags(unsigned(LHS) | unsigned(RHS));
 }
 
+class ValueMapperImpl;
+
+/// Context for (re-)mapping values (and metadata).
+///
+/// A shared context used for mapping and remapping of Value and Metadata
+/// instances using \a ValueToValueMapTy, \a RemapFlags, \a
+/// ValueMapTypeRemapper, and \a ValueMaterializer.
+///
+/// There are a number of top-level entry points:
+/// - \a mapValue() (and \a mapConstant());
+/// - \a mapMetadata() (and \a mapMDNode());
+/// - \a remapInstruction(); and
+/// - \a remapFunction().
+///
+/// The \a ValueMaterializer can be used as a callback, but cannot invoke any
+/// of these top-level functions recursively.  Instead, callbacks should use
+/// one of the following to schedule work lazily in the \a ValueMapper
+/// instance:
+/// - \a scheduleMapGlobalInitializer()
+/// - \a scheduleMapAppendingVariable()
+/// - \a scheduleMapGlobalAliasee()
+/// - \a scheduleRemapFunction()
+///
+/// Sometimes a callback needs a diferent mapping context.  Such a context can
+/// be registered using \a registerAlternateMappingContext(), which takes an
+/// alternate \a ValueToValueMapTy and \a ValueMaterializer and returns a ID to
+/// pass into the schedule*() functions.
+///
+/// TODO: lib/Linker really doesn't need the \a ValueHandle in the \a
+/// ValueToValueMapTy.  We should template \a ValueMapper (and its
+/// implementation classes), and explicitly instantiate on two concrete
+/// instances of \a ValueMap (one as \a ValueToValueMap, and one with raw \a
+/// Value pointers).  It may be viable to do away with \a TrackingMDRef in the
+/// \a Metadata side map for the lib/Linker case as well, in which case we'll
+/// need a new template parameter on \a ValueMap.
+///
+/// TODO: Update callers of \a RemapInstruction() and \a MapValue() (etc.) to
+/// use \a ValueMapper directly.
+class ValueMapper {
+  void *pImpl;
+
+  ValueMapper(ValueMapper &&) = delete;
+  ValueMapper(const ValueMapper &) = delete;
+  ValueMapper &operator=(ValueMapper &&) = delete;
+  ValueMapper &operator=(const ValueMapper &) = delete;
+
+public:
+  ValueMapper(ValueToValueMapTy &VM, RemapFlags Flags = RF_None,
+              ValueMapTypeRemapper *TypeMapper = nullptr,
+              ValueMaterializer *Materializer = nullptr);
+  ~ValueMapper();
+
+  /// Register an alternate mapping context.
+  ///
+  /// Returns a MappingContextID that can be used with the various schedule*()
+  /// API to switch in a different value map on-the-fly.
+  unsigned
+  registerAlternateMappingContext(ValueToValueMapTy &VM,
+                                  ValueMaterializer *Materializer = nullptr);
+
+  /// Add to the current \a RemapFlags.
+  ///
+  /// \note Like the top-level mapping functions, \a addFlags() must be called
+  /// at the top level, not during a callback in a \a ValueMaterializer.
+  void addFlags(RemapFlags Flags);
+
+  Metadata *mapMetadata(const Metadata &MD);
+  MDNode *mapMDNode(const MDNode &N);
+
+  Value *mapValue(const Value &V);
+  Constant *mapConstant(const Constant &C);
+
+  void remapInstruction(Instruction &I);
+  void remapFunction(Function &F);
+
+  void scheduleMapGlobalInitializer(GlobalVariable &GV, Constant &Init,
+                                    unsigned MappingContextID = 0);
+  void scheduleMapAppendingVariable(GlobalVariable &GV, Constant *InitPrefix,
+                                    bool IsOldCtorDtor,
+                                    ArrayRef<Constant *> NewMembers,
+                                    unsigned MappingContextID = 0);
+  void scheduleMapGlobalAliasee(GlobalAlias &GA, Constant &Aliasee,
+                                unsigned MappingContextID = 0);
+  void scheduleRemapFunction(Function &F, unsigned MappingContextID = 0);
+};
+
 /// Look up or compute a value in the value map.
 ///
 /// Return a mapped value for a function-local value (Argument, Instruction,
@@ -115,10 +201,12 @@ static inline RemapFlags operator|(RemapFlags LHS, RemapFlags RHS) {
 ///  6. Else if \c V is a \a MetadataAsValue, rewrap the return of \a
 ///     MapMetadata().
 ///  7. Else, compute the equivalent constant, and return it.
-Value *MapValue(const Value *V, ValueToValueMapTy &VM,
+inline Value *MapValue(const Value *V, ValueToValueMapTy &VM,
-                RemapFlags Flags = RF_None,
+                       RemapFlags Flags = RF_None,
-                ValueMapTypeRemapper *TypeMapper = nullptr,
+                       ValueMapTypeRemapper *TypeMapper = nullptr,
-                ValueMaterializer *Materializer = nullptr);
+                       ValueMaterializer *Materializer = nullptr) {
+  return ValueMapper(VM, Flags, TypeMapper, Materializer).mapValue(*V);
+}
 
 /// Lookup or compute a mapping for a piece of metadata.
 ///
@@ -135,16 +223,20 @@ Value *MapValue(const Value *V, ValueToValueMapTy &VM,
 ///
 /// \note \a LocalAsMetadata is completely unsupported by \a MapMetadata.
 /// Instead, use \a MapValue() with its wrapping \a MetadataAsValue instance.
-Metadata *MapMetadata(const Metadata *MD, ValueToValueMapTy &VM,
+inline Metadata *MapMetadata(const Metadata *MD, ValueToValueMapTy &VM,
-                      RemapFlags Flags = RF_None,
+                             RemapFlags Flags = RF_None,
-                      ValueMapTypeRemapper *TypeMapper = nullptr,
+                             ValueMapTypeRemapper *TypeMapper = nullptr,
-                      ValueMaterializer *Materializer = nullptr);
+                             ValueMaterializer *Materializer = nullptr) {
+  return ValueMapper(VM, Flags, TypeMapper, Materializer).mapMetadata(*MD);
+}
 
 /// Version of MapMetadata with type safety for MDNode.
-MDNode *MapMetadata(const MDNode *MD, ValueToValueMapTy &VM,
+inline MDNode *MapMetadata(const MDNode *MD, ValueToValueMapTy &VM,
-                    RemapFlags Flags = RF_None,
+                           RemapFlags Flags = RF_None,
-                    ValueMapTypeRemapper *TypeMapper = nullptr,
+                           ValueMapTypeRemapper *TypeMapper = nullptr,
-                    ValueMaterializer *Materializer = nullptr);
+                           ValueMaterializer *Materializer = nullptr) {
+  return ValueMapper(VM, Flags, TypeMapper, Materializer).mapMDNode(*MD);
+}
 
 /// Convert the instruction operands from referencing the current values into
 /// those specified by VM.
@@ -154,10 +246,12 @@ MDNode *MapMetadata(const MDNode *MD, ValueToValueMapTy &VM,
 ///
 /// Note that \a MapValue() only returns \c nullptr for SSA values missing from
 /// \c VM.
-void RemapInstruction(Instruction *I, ValueToValueMapTy &VM,
+inline void RemapInstruction(Instruction *I, ValueToValueMapTy &VM,
-                      RemapFlags Flags = RF_None,
+                             RemapFlags Flags = RF_None,
-                      ValueMapTypeRemapper *TypeMapper = nullptr,
+                             ValueMapTypeRemapper *TypeMapper = nullptr,
-                      ValueMaterializer *Materializer = nullptr);
+                             ValueMaterializer *Materializer = nullptr) {
+  ValueMapper(VM, Flags, TypeMapper, Materializer).remapInstruction(*I);
+}
 
 /// Remap the operands, metadata, arguments, and instructions of a function.
 ///
@@ -165,19 +259,19 @@ void RemapInstruction(Instruction *I, ValueToValueMapTy &VM,
 /// function; calls \a MapMetadata() on each attached MDNode; remaps the
 /// argument types using the provided \c TypeMapper; and calls \a
 /// RemapInstruction() on every instruction.
-void RemapFunction(Function &F, ValueToValueMapTy &VM,
+inline void RemapFunction(Function &F, ValueToValueMapTy &VM,
-                   RemapFlags Flags = RF_None,
+                          RemapFlags Flags = RF_None,
-                   ValueMapTypeRemapper *TypeMapper = nullptr,
+                          ValueMapTypeRemapper *TypeMapper = nullptr,
-                   ValueMaterializer *Materializer = nullptr);
+                          ValueMaterializer *Materializer = nullptr) {
+  ValueMapper(VM, Flags, TypeMapper, Materializer).remapFunction(F);
+}
 
 /// Version of MapValue with type safety for Constant.
 inline Constant *MapValue(const Constant *V, ValueToValueMapTy &VM,
                           RemapFlags Flags = RF_None,
                           ValueMapTypeRemapper *TypeMapper = nullptr,
                           ValueMaterializer *Materializer = nullptr) {
-  // This can be null for RF_NullMapMissingGlobalValues.
+  return ValueMapper(VM, Flags, TypeMapper, Materializer).mapConstant(*V);
-  return cast_or_null<Constant>(
-      MapValue((const Value *)V, VM, Flags, TypeMapper, Materializer));
 }
 
 } // End llvm namespace

lib/Linker/IRMover.cpp

@@ -397,8 +397,9 @@ class IRLinker {
 
   bool HasError = false;
 
-  /// Flags to pass to value mapper invocations.
+  /// Entry point for mapping values and alternate context for mapping aliases.
-  RemapFlags ValueMapperFlags = RF_MoveDistinctMDs | RF_IgnoreMissingLocals;
+  ValueMapper Mapper;
+  unsigned AliasMCID;
 
   /// Handles cloning of a global values from the source module into
   /// the destination module, including setting the attributes and visibility.
@@ -470,7 +471,11 @@ public:
            std::unique_ptr<Module> SrcM, ArrayRef<GlobalValue *> ValuesToLink,
            std::function<void(GlobalValue &, IRMover::ValueAdder)> AddLazyFor)
       : DstM(DstM), SrcM(std::move(SrcM)), AddLazyFor(AddLazyFor), TypeMap(Set),
-        GValMaterializer(*this), LValMaterializer(*this) {
+        GValMaterializer(*this), LValMaterializer(*this),
+        Mapper(ValueMap, RF_MoveDistinctMDs | RF_IgnoreMissingLocals, &TypeMap,
+               &GValMaterializer),
+        AliasMCID(Mapper.registerAlternateMappingContext(AliasValueMap,
+                                                         &LValMaterializer)) {
     for (GlobalValue *GV : ValuesToLink)
       maybeAdd(GV);
   }
@@ -712,6 +717,10 @@ Constant *IRLinker::linkAppendingVarProto(GlobalVariable *DstGV,
   Type *EltTy = cast<ArrayType>(TypeMap.get(SrcGV->getValueType()))
                     ->getElementType();
 
+  // FIXME: This upgrade is done during linking to support the C API.  Once the
+  // old form is deprecated, we should move this upgrade to
+  // llvm::UpgradeGlobalVariable() and simplify the logic here and in
+  // Mapper::mapAppendingVariable() in ValueMapper.cpp.
   StringRef Name = SrcGV->getName();
   bool IsNewStructor = false;
   bool IsOldStructor = false;
@@ -729,8 +738,10 @@ Constant *IRLinker::linkAppendingVarProto(GlobalVariable *DstGV,
     EltTy = StructType::get(SrcGV->getContext(), Tys, false);
   }
 
+  uint64_t DstNumElements = 0;
   if (DstGV) {
     ArrayType *DstTy = cast<ArrayType>(DstGV->getValueType());
+    DstNumElements = DstTy->getNumElements();
 
     if (!SrcGV->hasAppendingLinkage() || !DstGV->hasAppendingLinkage()) {
       emitError(
@@ -774,10 +785,6 @@ Constant *IRLinker::linkAppendingVarProto(GlobalVariable *DstGV,
     }
   }
 
-  SmallVector<Constant *, 16> DstElements;
-  if (DstGV)
-    getArrayElements(DstGV->getInitializer(), DstElements);
-
   SmallVector<Constant *, 16> SrcElements;
   getArrayElements(SrcGV->getInitializer(), SrcElements);
 
@@ -793,7 +800,7 @@ Constant *IRLinker::linkAppendingVarProto(GlobalVariable *DstGV,
                          return !shouldLink(DGV, *Key);
                        }),
         SrcElements.end());
-  uint64_t NewSize = DstElements.size() + SrcElements.size();
+  uint64_t NewSize = DstNumElements + SrcElements.size();
   ArrayType *NewType = ArrayType::get(EltTy, NewSize);
 
   // Create the new global variable.
@@ -810,25 +817,9 @@ Constant *IRLinker::linkAppendingVarProto(GlobalVariable *DstGV,
   // Stop recursion.
   ValueMap[SrcGV] = Ret;
 
-  for (auto *V : SrcElements) {
+  Mapper.scheduleMapAppendingVariable(*NG,
-    Constant *NewV;
+                                      DstGV ? DstGV->getInitializer() : nullptr,
-    if (IsOldStructor) {
+                                      IsOldStructor, SrcElements);
-      auto *S = cast<ConstantStruct>(V);
-      auto *E1 = MapValue(S->getOperand(0), ValueMap, ValueMapperFlags,
-                          &TypeMap, &GValMaterializer);
-      auto *E2 = MapValue(S->getOperand(1), ValueMap, ValueMapperFlags,
-                          &TypeMap, &GValMaterializer);
-      Value *Null = Constant::getNullValue(VoidPtrTy);
-      NewV =
-          ConstantStruct::get(cast<StructType>(EltTy), E1, E2, Null, nullptr);
-    } else {
-      NewV =
-          MapValue(V, ValueMap, ValueMapperFlags, &TypeMap, &GValMaterializer);
-    }
-    DstElements.push_back(NewV);
-  }
-
-  NG->setInitializer(ConstantArray::get(NewType, DstElements));
 
   // Replace any uses of the two global variables with uses of the new
   // global.
@@ -935,8 +926,7 @@ Constant *IRLinker::linkGlobalValueProto(GlobalValue *SGV, bool ForAlias) {
 /// referenced are in Dest.
 void IRLinker::linkGlobalInit(GlobalVariable &Dst, GlobalVariable &Src) {
   // Figure out what the initializer looks like in the dest module.
-  Dst.setInitializer(MapValue(Src.getInitializer(), ValueMap, ValueMapperFlags,
+  Mapper.scheduleMapGlobalInitializer(Dst, *Src.getInitializer());
-                              &TypeMap, &GValMaterializer));
 }
 
 /// Copy the source function over into the dest function and fix up references
@@ -968,15 +958,12 @@ bool IRLinker::linkFunctionBody(Function &Dst, Function &Src) {
   Dst.getBasicBlockList().splice(Dst.end(), Src.getBasicBlockList());
 
   // Everything has been moved over.  Remap it.
-  RemapFunction(Dst, ValueMap, ValueMapperFlags, &TypeMap, &GValMaterializer);
+  Mapper.scheduleRemapFunction(Dst);
   return false;
 }
 
 void IRLinker::linkAliasBody(GlobalAlias &Dst, GlobalAlias &Src) {
-  Constant *Aliasee = Src.getAliasee();
+  Mapper.scheduleMapGlobalAliasee(Dst, *Src.getAliasee(), AliasMCID);
-  Constant *Val = MapValue(Aliasee, AliasValueMap, ValueMapperFlags, &TypeMap,
-                           &LValMaterializer);
-  Dst.setAliasee(Val);
 }
 
 bool IRLinker::linkGlobalValueBody(GlobalValue &Dst, GlobalValue &Src) {
@@ -1000,9 +987,7 @@ void IRLinker::linkNamedMDNodes() {
     NamedMDNode *DestNMD = DstM.getOrInsertNamedMetadata(NMD.getName());
     // Add Src elements into Dest node.
     for (const MDNode *Op : NMD.operands())
-      DestNMD->addOperand(MapMetadata(
+      DestNMD->addOperand(Mapper.mapMDNode(*Op));
-          Op, ValueMap, ValueMapperFlags | RF_NullMapMissingGlobalValues,
-          &TypeMap, &GValMaterializer));
   }
 }
 
@@ -1242,7 +1227,7 @@ bool IRLinker::run() {
       continue;
 
     assert(!GV->isDeclaration());
-    MapValue(GV, ValueMap, ValueMapperFlags, &TypeMap, &GValMaterializer);
+    Mapper.mapValue(*GV);
     if (HasError)
       return true;
   }
@@ -1250,6 +1235,7 @@ bool IRLinker::run() {
   // Note that we are done linking global value bodies. This prevents
   // metadata linking from creating new references.
   DoneLinkingBodies = true;
+  Mapper.addFlags(RF_NullMapMissingGlobalValues);
 
   // Remap all of the named MDNodes in Src into the DstM module. We do this
   // after linking GlobalValues so that MDNodes that reference GlobalValues

lib/Transforms/Utils/ValueMapper.cpp

@@ -16,6 +16,8 @@
 #include "llvm/IR/CallSite.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/Function.h"
+#include "llvm/IR/GlobalAlias.h"
+#include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/InlineAsm.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/Metadata.h"
@@ -30,14 +32,6 @@ void ValueMaterializer::materializeInitFor(GlobalValue *New, GlobalValue *Old) {
 
 namespace {
 
-/// A GlobalValue whose initializer needs to be materialized.
-struct DelayedGlobalValueInit {
-  GlobalValue *Old;
-  GlobalValue *New;
-  DelayedGlobalValueInit(const GlobalValue *Old, GlobalValue *New)
-      : Old(const_cast<GlobalValue *>(Old)), New(New) {}
-};
-
 /// A basic block used in a BlockAddress whose function body is not yet
 /// materialized.
 struct DelayedBasicBlock {
@@ -58,30 +52,88 @@ struct DelayedBasicBlock {
         TempBB(BasicBlock::Create(Old.getContext())) {}
 };
 
+struct WorklistEntry {
+  enum EntryKind {
+    MapGlobalInit,
+    MapAppendingVar,
+    MapGlobalAliasee,
+    RemapFunction
+  };
+  struct GVInitTy {
+    GlobalVariable *GV;
+    Constant *Init;
+  };
+  struct AppendingGVTy {
+    GlobalVariable *GV;
+    Constant *InitPrefix;
+  };
+  struct GlobalAliaseeTy {
+    GlobalAlias *GA;
+    Constant *Aliasee;
+  };
+
+  unsigned Kind : 2;
+  unsigned MCID : 29;
+  unsigned AppendingGVIsOldCtorDtor : 1;
+  unsigned AppendingGVNumNewMembers;
+  union {
+    GVInitTy GVInit;
+    AppendingGVTy AppendingGV;
+    GlobalAliaseeTy GlobalAliasee;
+    Function *RemapF;
+  } Data;
+};
+
+struct MappingContext {
+  ValueToValueMapTy *VM;
+  ValueMaterializer *Materializer = nullptr;
+
+  /// Construct a MappingContext with a value map and materializer.
+  explicit MappingContext(ValueToValueMapTy &VM,
+                          ValueMaterializer *Materializer = nullptr)
+      : VM(&VM), Materializer(Materializer) {}
+};
+
 class MDNodeMapper;
 class Mapper {
   friend class MDNodeMapper;
 
-  ValueToValueMapTy *VM;
   RemapFlags Flags;
   ValueMapTypeRemapper *TypeMapper;
-  ValueMaterializer *Materializer;
+  unsigned CurrentMCID = 0;
-
+  SmallVector<MappingContext, 2> MCs;
-  SmallVector<DelayedGlobalValueInit, 8> DelayedInits;
+  SmallVector<WorklistEntry, 4> Worklist;
   SmallVector<DelayedBasicBlock, 1> DelayedBBs;
+  SmallVector<Constant *, 16> AppendingInits;
 
 public:
   Mapper(ValueToValueMapTy &VM, RemapFlags Flags,
          ValueMapTypeRemapper *TypeMapper, ValueMaterializer *Materializer)
-      : VM(&VM), Flags(Flags), TypeMapper(TypeMapper),
+      : Flags(Flags), TypeMapper(TypeMapper),
-        Materializer(Materializer) {}
+        MCs(1, MappingContext(VM, Materializer)) {}
 
-  ~Mapper();
+  /// ValueMapper should explicitly call \a flush() before destruction.
+  ~Mapper() { assert(!hasWorkToDo() && "Expected to be flushed"); }
+
+  bool hasWorkToDo() const { return !Worklist.empty(); }
+
+  unsigned
+  registerAlternateMappingContext(ValueToValueMapTy &VM,
+                                  ValueMaterializer *Materializer = nullptr) {
+    MCs.push_back(MappingContext(VM, Materializer));
+    return MCs.size() - 1;
+  }
+
+  void addFlags(RemapFlags Flags);
 
   Value *mapValue(const Value *V);
   void remapInstruction(Instruction *I);
   void remapFunction(Function &F);
 
+  Constant *mapConstant(const Constant *C) {
+    return cast_or_null<Constant>(mapValue(C));
+  }
+
   /// Map metadata.
   ///
   /// Find the mapping for MD.  Guarantees that the return will be resolved
@@ -102,8 +154,28 @@ public:
   // through metadata operands, always return nullptr on unmapped locals.
   Metadata *mapLocalAsMetadata(const LocalAsMetadata &LAM);
 
+  void scheduleMapGlobalInitializer(GlobalVariable &GV, Constant &Init,
+                                    unsigned MCID);
+  void scheduleMapAppendingVariable(GlobalVariable &GV, Constant *InitPrefix,
+                                    bool IsOldCtorDtor,
+                                    ArrayRef<Constant *> NewMembers,
+                                    unsigned MCID);
+  void scheduleMapGlobalAliasee(GlobalAlias &GA, Constant &Aliasee,
+                                unsigned MCID);
+  void scheduleRemapFunction(Function &F, unsigned MCID);
+
+  void flush();
+
 private:
-  ValueToValueMapTy &getVM() { return *VM; }
+  void mapGlobalInitializer(GlobalVariable &GV, Constant &Init);
+  void mapAppendingVariable(GlobalVariable &GV, Constant *InitPrefix,
+                            bool IsOldCtorDtor,
+                            ArrayRef<Constant *> NewMembers);
+  void mapGlobalAliasee(GlobalAlias &GA, Constant &Aliasee);
+  void remapFunction(Function &F, ValueToValueMapTy &VM);
+
+  ValueToValueMapTy &getVM() { return *MCs[CurrentMCID].VM; }
+  ValueMaterializer *getMaterializer() { return MCs[CurrentMCID].Materializer; }
 
   Value *mapBlockAddress(const BlockAddress &BA);
 
@@ -264,12 +336,6 @@ private:
 
 } // end namespace
 
-Value *llvm::MapValue(const Value *V, ValueToValueMapTy &VM, RemapFlags Flags,
-                      ValueMapTypeRemapper *TypeMapper,
-                      ValueMaterializer *Materializer) {
-  return Mapper(VM, Flags, TypeMapper, Materializer).mapValue(V);
-}
-
 Value *Mapper::mapValue(const Value *V) {
   ValueToValueMapTy::iterator I = getVM().find(V);
 
@@ -278,13 +344,13 @@ Value *Mapper::mapValue(const Value *V) {
     return I->second;
 
   // If we have a materializer and it can materialize a value, use that.
-  if (Materializer) {
+  if (auto *Materializer = getMaterializer()) {
     if (Value *NewV =
             Materializer->materializeDeclFor(const_cast<Value *>(V))) {
       getVM()[V] = NewV;
       if (auto *NewGV = dyn_cast<GlobalValue>(NewV))
-        DelayedInits.push_back(
+        Materializer->materializeInitFor(
-            DelayedGlobalValueInit(cast<GlobalValue>(V), NewGV));
+            NewGV, cast<GlobalValue>(const_cast<Value *>(V)));
       return NewV;
     }
   }
@@ -684,12 +750,6 @@ Optional<Metadata *> Mapper::mapSimpleMetadata(const Metadata *MD) {
   return None;
 }
 
-Metadata *llvm::MapMetadata(const Metadata *MD, ValueToValueMapTy &VM,
-                            RemapFlags Flags, ValueMapTypeRemapper *TypeMapper,
-                            ValueMaterializer *Materializer) {
-  return Mapper(VM, Flags, TypeMapper, Materializer).mapMetadata(MD);
-}
-
 Metadata *Mapper::mapLocalAsMetadata(const LocalAsMetadata &LAM) {
   // Lookup the mapping for the value itself, and return the appropriate
   // metadata.
@@ -716,36 +776,42 @@ Metadata *Mapper::mapMetadata(const Metadata *MD) {
   return MDNodeMapper(*this).map(*cast<MDNode>(MD));
 }
 
-Mapper::~Mapper() {
+void Mapper::flush() {
-  // Materialize global initializers.
+  // Flush out the worklist of global values.
-  while (!DelayedInits.empty()) {
+  while (!Worklist.empty()) {
-    auto Init = DelayedInits.pop_back_val();
+    WorklistEntry E = Worklist.pop_back_val();
-    Materializer->materializeInitFor(Init.New, Init.Old);
+    CurrentMCID = E.MCID;
+    switch (E.Kind) {
+    case WorklistEntry::MapGlobalInit:
+      E.Data.GVInit.GV->setInitializer(mapConstant(E.Data.GVInit.Init));
+      break;
+    case WorklistEntry::MapAppendingVar: {
+      unsigned PrefixSize = AppendingInits.size() - E.AppendingGVNumNewMembers;
+      mapAppendingVariable(*E.Data.AppendingGV.GV,
+                           E.Data.AppendingGV.InitPrefix,
+                           E.AppendingGVIsOldCtorDtor,
+                           makeArrayRef(AppendingInits).slice(PrefixSize));
+      AppendingInits.resize(PrefixSize);
+      break;
+    }
+    case WorklistEntry::MapGlobalAliasee:
+      E.Data.GlobalAliasee.GA->setAliasee(
+          mapConstant(E.Data.GlobalAliasee.Aliasee));
+      break;
+    case WorklistEntry::RemapFunction:
+      remapFunction(*E.Data.RemapF);
+      break;
+    }
   }
+  CurrentMCID = 0;
 
-  // Process block addresses delayed until global inits.
+  // Finish logic for block addresses now that all global values have been
+  // handled.
   while (!DelayedBBs.empty()) {
     DelayedBasicBlock DBB = DelayedBBs.pop_back_val();
     BasicBlock *BB = cast_or_null<BasicBlock>(mapValue(DBB.OldBB));
     DBB.TempBB->replaceAllUsesWith(BB ? BB : DBB.OldBB);
   }
-
-  // We don't expect these to grow after clearing.
-  assert(DelayedInits.empty());
-  assert(DelayedBBs.empty());
-}
-
-MDNode *llvm::MapMetadata(const MDNode *MD, ValueToValueMapTy &VM,
-                          RemapFlags Flags, ValueMapTypeRemapper *TypeMapper,
-                          ValueMaterializer *Materializer) {
-  return cast_or_null<MDNode>(MapMetadata(static_cast<const Metadata *>(MD), VM,
-                                          Flags, TypeMapper, Materializer));
-}
-
-void llvm::RemapInstruction(Instruction *I, ValueToValueMapTy &VM,
-                            RemapFlags Flags, ValueMapTypeRemapper *TypeMapper,
-                            ValueMaterializer *Materializer) {
-  Mapper(VM, Flags, TypeMapper, Materializer).remapInstruction(I);
 }
 
 void Mapper::remapInstruction(Instruction *I) {
@@ -782,7 +848,7 @@ void Mapper::remapInstruction(Instruction *I) {
     if (New != Old)
       I->setMetadata(MI.first, New);
   }
-  
+
   if (!TypeMapper)
     return;
 
@@ -808,12 +874,6 @@ void Mapper::remapInstruction(Instruction *I) {
   I->mutateType(TypeMapper->remapType(I->getType()));
 }
 
-void llvm::RemapFunction(Function &F, ValueToValueMapTy &VM, RemapFlags Flags,
-                         ValueMapTypeRemapper *TypeMapper,
-                         ValueMaterializer *Materializer) {
-  Mapper(VM, Flags, TypeMapper, Materializer).remapFunction(F);
-}
-
 void Mapper::remapFunction(Function &F) {
   // Remap the operands.
   for (Use &Op : F.operands())
@@ -836,3 +896,185 @@ void Mapper::remapFunction(Function &F) {
     for (Instruction &I : BB)
       remapInstruction(&I);
 }
+
+void Mapper::mapAppendingVariable(GlobalVariable &GV, Constant *InitPrefix,
+                                  bool IsOldCtorDtor,
+                                  ArrayRef<Constant *> NewMembers) {
+  SmallVector<Constant *, 16> Elements;
+  if (InitPrefix) {
+    unsigned NumElements =
+        cast<ArrayType>(InitPrefix->getType())->getNumElements();
+    for (unsigned I = 0; I != NumElements; ++I)
+      Elements.push_back(InitPrefix->getAggregateElement(I));
+  }
+
+  PointerType *VoidPtrTy;
+  Type *EltTy;
+  if (IsOldCtorDtor) {
+    // FIXME: This upgrade is done during linking to support the C API.  See
+    // also IRLinker::linkAppendingVarProto() in IRMover.cpp.
+    VoidPtrTy = Type::getInt8Ty(GV.getContext())->getPointerTo();
+    auto &ST = *cast<StructType>(NewMembers.front()->getType());
+    Type *Tys[3] = {ST.getElementType(0), ST.getElementType(1), VoidPtrTy};
+    EltTy = StructType::get(GV.getContext(), Tys, false);
+  }
+
+  for (auto *V : NewMembers) {
+    Constant *NewV;
+    if (IsOldCtorDtor) {
+      auto *S = cast<ConstantStruct>(V);
+      auto *E1 = mapValue(S->getOperand(0));
+      auto *E2 = mapValue(S->getOperand(1));
+      Value *Null = Constant::getNullValue(VoidPtrTy);
+      NewV =
+          ConstantStruct::get(cast<StructType>(EltTy), E1, E2, Null, nullptr);
+    } else {
+      NewV = cast_or_null<Constant>(mapValue(V));
+    }
+    Elements.push_back(NewV);
+  }
+
+  GV.setInitializer(ConstantArray::get(
+      cast<ArrayType>(GV.getType()->getElementType()), Elements));
+}
+
+void Mapper::scheduleMapGlobalInitializer(GlobalVariable &GV, Constant &Init,
+                                          unsigned MCID) {
+  assert(MCID < MCs.size() && "Invalid mapping context");
+
+  WorklistEntry WE;
+  WE.Kind = WorklistEntry::MapGlobalInit;
+  WE.MCID = MCID;
+  WE.Data.GVInit.GV = &GV;
+  WE.Data.GVInit.Init = &Init;
+  Worklist.push_back(WE);
+}
+
+void Mapper::scheduleMapAppendingVariable(GlobalVariable &GV,
+                                          Constant *InitPrefix,
+                                          bool IsOldCtorDtor,
+                                          ArrayRef<Constant *> NewMembers,
+                                          unsigned MCID) {
+  assert(MCID < MCs.size() && "Invalid mapping context");
+
+  WorklistEntry WE;
+  WE.Kind = WorklistEntry::MapAppendingVar;
+  WE.MCID = MCID;
+  WE.Data.AppendingGV.GV = &GV;
+  WE.Data.AppendingGV.InitPrefix = InitPrefix;
+  WE.AppendingGVIsOldCtorDtor = IsOldCtorDtor;
+  WE.AppendingGVNumNewMembers = NewMembers.size();
+  Worklist.push_back(WE);
+  AppendingInits.append(NewMembers.begin(), NewMembers.end());
+}
+
+void Mapper::scheduleMapGlobalAliasee(GlobalAlias &GA, Constant &Aliasee,
+                                      unsigned MCID) {
+  assert(MCID < MCs.size() && "Invalid mapping context");
+
+  WorklistEntry WE;
+  WE.Kind = WorklistEntry::MapGlobalAliasee;
+  WE.MCID = MCID;
+  WE.Data.GlobalAliasee.GA = &GA;
+  WE.Data.GlobalAliasee.Aliasee = &Aliasee;
+  Worklist.push_back(WE);
+}
+
+void Mapper::scheduleRemapFunction(Function &F, unsigned MCID) {
+  assert(MCID < MCs.size() && "Invalid mapping context");
+
+  WorklistEntry WE;
+  WE.Kind = WorklistEntry::RemapFunction;
+  WE.MCID = MCID;
+  WE.Data.RemapF = &F;
+  Worklist.push_back(WE);
+}
+
+void Mapper::addFlags(RemapFlags Flags) {
+  assert(!hasWorkToDo() && "Expected to have flushed the worklist");
+  this->Flags = this->Flags | Flags;
+}
+
+static Mapper *getAsMapper(void *pImpl) {
+  return reinterpret_cast<Mapper *>(pImpl);
+}
+
+namespace {
+
+class FlushingMapper {
+  Mapper &M;
+
+public:
+  explicit FlushingMapper(void *pImpl) : M(*getAsMapper(pImpl)) {
+    assert(!M.hasWorkToDo() && "Expected to be flushed");
+  }
+  ~FlushingMapper() { M.flush(); }
+  Mapper *operator->() const { return &M; }
+};
+
+} // end namespace
+
+ValueMapper::ValueMapper(ValueToValueMapTy &VM, RemapFlags Flags,
+                         ValueMapTypeRemapper *TypeMapper,
+                         ValueMaterializer *Materializer)
+    : pImpl(new Mapper(VM, Flags, TypeMapper, Materializer)) {}
+
+ValueMapper::~ValueMapper() { delete getAsMapper(pImpl); }
+
+unsigned
+ValueMapper::registerAlternateMappingContext(ValueToValueMapTy &VM,
+                                             ValueMaterializer *Materializer) {
+  return getAsMapper(pImpl)->registerAlternateMappingContext(VM, Materializer);
+}
+
+void ValueMapper::addFlags(RemapFlags Flags) {
+  FlushingMapper(pImpl)->addFlags(Flags);
+}
+
+Value *ValueMapper::mapValue(const Value &V) {
+  return FlushingMapper(pImpl)->mapValue(&V);
+}
+
+Constant *ValueMapper::mapConstant(const Constant &C) {
+  return cast_or_null<Constant>(mapValue(C));
+}
+
+Metadata *ValueMapper::mapMetadata(const Metadata &MD) {
+  return FlushingMapper(pImpl)->mapMetadata(&MD);
+}
+
+MDNode *ValueMapper::mapMDNode(const MDNode &N) {
+  return cast_or_null<MDNode>(mapMetadata(N));
+}
+
+void ValueMapper::remapInstruction(Instruction &I) {
+  FlushingMapper(pImpl)->remapInstruction(&I);
+}
+
+void ValueMapper::remapFunction(Function &F) {
+  FlushingMapper(pImpl)->remapFunction(F);
+}
+
+void ValueMapper::scheduleMapGlobalInitializer(GlobalVariable &GV,
+                                               Constant &Init,
+                                               unsigned MCID) {
+  getAsMapper(pImpl)->scheduleMapGlobalInitializer(GV, Init, MCID);
+}
+
+void ValueMapper::scheduleMapAppendingVariable(GlobalVariable &GV,
+                                               Constant *InitPrefix,
+                                               bool IsOldCtorDtor,
+                                               ArrayRef<Constant *> NewMembers,
+                                               unsigned MCID) {
+  getAsMapper(pImpl)->scheduleMapAppendingVariable(
+      GV, InitPrefix, IsOldCtorDtor, NewMembers, MCID);
+}
+
+void ValueMapper::scheduleMapGlobalAliasee(GlobalAlias &GA, Constant &Aliasee,
+                                           unsigned MCID) {
+  getAsMapper(pImpl)->scheduleMapGlobalAliasee(GA, Aliasee, MCID);
+}
+
+void ValueMapper::scheduleRemapFunction(Function &F, unsigned MCID) {
+  getAsMapper(pImpl)->scheduleRemapFunction(F, MCID);
+}