llvm/lib/CodeGen
Philip Reames 3239eb1016 [CGP] Duplicate addressing computation in cold paths if required to sink addressing mode
This patch teaches CGP to duplicate addressing mode computations into cold paths (detected via explicit cold attribute on calls) if required to let addressing mode be safely sunk into the basic block containing each load and store.

In general, duplicating code into cold blocks may result in code growth, but should not effect performance. In this case, it's better to duplicate some code than to put extra pressure on the register allocator by making it keep the address through the entirely of the fast path.

This patch only handles addressing computations, but in principal, we could implement a more general cold cold scheduling heuristic which tries to reduce register pressure in the fast path by duplicating code into the cold path. Getting the profitability of the general case right seemed likely to be challenging, so I stuck to the existing case (addressing computation) we already had.

Differential Revision: http://reviews.llvm.org/D17652



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@263074 91177308-0d34-0410-b5e6-96231b3b80d8
2016-03-09 23:13:12 +00:00
..
AsmPrinter Make headers self-contained again. 2016-03-04 10:49:30 +00:00
GlobalISel [GlobalISel] Introduce initializer method to support start/stop-after features. 2016-03-08 01:38:55 +00:00
MIRParser [MIR] Change the token name for '<' and '>' to be consitent with the LLVM IR parser. 2016-03-08 02:00:43 +00:00
SelectionDAG SelectionDAG: Fix a crash on inline asm when output register supports multiple types 2016-03-09 16:02:52 +00:00
AggressiveAntiDepBreaker.cpp CodeGen: Use MachineInstr& in AntiDepBreaker API, NFC 2016-02-27 19:33:37 +00:00
AggressiveAntiDepBreaker.h CodeGen: Use MachineInstr& in AntiDepBreaker API, NFC 2016-02-27 19:33:37 +00:00
AllocationOrder.cpp
AllocationOrder.h
Analysis.cpp One more batch of self-containing headers. 2016-01-27 19:29:56 +00:00
AntiDepBreaker.h CodeGen: Use MachineInstr& in AntiDepBreaker API, NFC 2016-02-27 19:33:37 +00:00
AtomicExpandPass.cpp ARM: sink atomic release barrier as far as possible into cmpxchg. 2016-02-22 20:55:50 +00:00
BasicTargetTransformInfo.cpp
BranchFolding.cpp [BranchFolding] Change function name related with merging MMOs. NFC 2016-03-03 03:57:20 +00:00
BranchFolding.h
BuiltinGCs.cpp [GC] Consolidate all built in GCs into a single file [NFC] 2016-01-19 03:57:18 +00:00
CalcSpillWeights.cpp CodeGen: Update LiveIntervalAnalysis API to use MachineInstr&, NFC 2016-02-27 20:14:29 +00:00
CallingConvLower.cpp Avoid unnecessary stack realignment in musttail thunks with SSE2 enabled 2016-01-21 22:23:22 +00:00
CMakeLists.txt Refactor duplicated code for linking with pthread. 2016-03-01 15:54:40 +00:00
CodeGen.cpp Move SafeStack to CodeGen. 2016-01-27 16:53:42 +00:00
CodeGenPrepare.cpp [CGP] Duplicate addressing computation in cold paths if required to sink addressing mode 2016-03-09 23:13:12 +00:00
CriticalAntiDepBreaker.cpp CodeGen: Use MachineInstr& in AntiDepBreaker API, NFC 2016-02-27 19:33:37 +00:00
CriticalAntiDepBreaker.h CodeGen: Use MachineInstr& in AntiDepBreaker API, NFC 2016-02-27 19:33:37 +00:00
DeadMachineInstructionElim.cpp
DFAPacketizer.cpp Add DAG mutation interface to the DFA packetizer 2016-03-08 15:33:51 +00:00
DwarfEHPrepare.cpp Move EH-specific helper functions to a more appropriate place 2015-12-02 23:06:39 +00:00
EarlyIfConversion.cpp Reapply "CodeGen: Use references in MachineTraceMetrics::Trace, NFC" 2016-02-22 03:33:28 +00:00
EdgeBundles.cpp
ExecutionDepsFix.cpp use range-based for-loops; NFCI 2015-12-29 17:15:22 +00:00
ExpandISelPseudos.cpp
ExpandPostRAPseudos.cpp
FaultMaps.cpp
FuncletLayout.cpp
GCMetadata.cpp
GCMetadataPrinter.cpp Revert 258157 2016-01-19 18:41:10 +00:00
GCRootLowering.cpp [opaque pointer types] Alloca: use getAllocatedType() instead of getType()->getPointerElementType(). 2016-01-18 00:10:01 +00:00
GCStrategy.cpp Revert 258157 2016-01-19 18:41:10 +00:00
GlobalMerge.cpp Make some headers self-contained, remove unused includes that violate layering. 2016-01-27 16:05:37 +00:00
IfConversion.cpp CodeGen: Change MachineInstr to use MachineInstr&, NFC 2016-02-27 20:01:33 +00:00
ImplicitNullChecks.cpp [TII] Allow getMemOpBaseRegImmOfs() to accept negative offsets. NFC. 2016-03-09 16:00:35 +00:00
InlineSpiller.cpp CodeGen: Use MachineInstr& in InlineSpiller::rematerializeFor() 2016-02-27 20:23:14 +00:00
InterferenceCache.cpp
InterferenceCache.h
InterleavedAccessPass.cpp
IntrinsicLowering.cpp getParent() ^ 3 == getModule() ; NFCI 2015-12-14 17:24:23 +00:00
LatencyPriorityQueue.cpp
LexicalScopes.cpp
LiveDebugValues.cpp Speed up LiveDebugValues 2016-01-10 18:08:32 +00:00
LiveDebugVariables.cpp CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC 2016-02-27 06:40:41 +00:00
LiveDebugVariables.h
LiveInterval.cpp CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC 2016-02-27 06:40:41 +00:00
LiveIntervalAnalysis.cpp CodeGen: Update LiveIntervalAnalysis API to use MachineInstr&, NFC 2016-02-27 20:14:29 +00:00
LiveIntervalUnion.cpp
LivePhysRegs.cpp WIP: CodeGen: Use MachineInstr& in MachineInstrBundle.h, NFC 2016-02-27 17:05:33 +00:00
LiveRangeCalc.cpp CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC 2016-02-27 06:40:41 +00:00
LiveRangeCalc.h
LiveRangeEdit.cpp CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC 2016-02-27 06:40:41 +00:00
LiveRegMatrix.cpp
LiveStackAnalysis.cpp
LiveVariables.cpp Annotate dump() methods with LLVM_DUMP_METHOD, addressing Richard Smith r259192 post commit comment. 2016-01-29 20:50:44 +00:00
LLVMBuild.txt Include ProfileData as CodeGen's required library. 2016-02-22 22:54:14 +00:00
LLVMTargetMachine.cpp When printing MIR, output to errs() rather than outs(). 2016-02-19 00:18:46 +00:00
LocalStackSlotAllocation.cpp
LowerEmuTLS.cpp Move passes that live in lib/CodeGen out of Scalar.h 2016-01-27 16:05:42 +00:00
MachineBasicBlock.cpp WIP: CodeGen: Use MachineInstr& in MachineInstrBundle.h, NFC 2016-02-27 17:05:33 +00:00
MachineBlockFrequencyInfo.cpp
MachineBlockPlacement.cpp [MBP] Renaming a confusing variable and add clarifying comments 2016-03-03 00:58:43 +00:00
MachineBranchProbabilityInfo.cpp Use getEdgeProbability() instead of getEdgeWeight() in BFI and remove getEdgeWeight() interfaces from MBPI. 2015-12-18 21:53:24 +00:00
MachineCombiner.cpp Minor code cleanup. NFC. 2016-02-27 01:10:43 +00:00
MachineCopyPropagation.cpp MachineCopyPropagation: Catch copies of the form A<-B;A<-B 2016-02-26 03:18:55 +00:00
MachineCSE.cpp rangify; NFCI 2016-01-06 00:45:42 +00:00
MachineDominanceFrontier.cpp
MachineDominators.cpp
MachineFunction.cpp [X86] Create mergeable constant pool entries for AVX 2016-02-22 22:23:11 +00:00
MachineFunctionAnalysis.cpp
MachineFunctionPass.cpp Introduce DominanceFrontierAnalysis to the new PassManager to compute DominanceFrontier. NFC 2016-02-25 17:54:15 +00:00
MachineFunctionPrinterPass.cpp Recommit r256952 "Filtering IR printing for print-after-all/print-before-all" 2016-01-06 22:55:03 +00:00
MachineInstr.cpp [MachineInstr] Get rid of some GlobalISel ifdefs. 2016-03-07 22:47:23 +00:00
MachineInstrBundle.cpp MachineInstrBundle: Fix reversed isSuperRegisterEq() call 2016-01-05 00:45:35 +00:00
MachineLICM.cpp rangify; NFCI 2016-01-06 23:45:05 +00:00
MachineLoopInfo.cpp ADT: Remove == and != comparisons between ilist iterators and pointers 2016-02-21 20:39:50 +00:00
MachineModuleInfo.cpp Remove uses of builtin comma operator. 2016-02-18 22:09:30 +00:00
MachineModuleInfoImpls.cpp
MachinePassRegistry.cpp
MachinePostDominators.cpp
MachineRegionInfo.cpp Introduce DominanceFrontierAnalysis to the new PassManager to compute DominanceFrontier. NFC 2016-02-25 17:54:15 +00:00
MachineRegisterInfo.cpp [MachineRegisterInfo] Add a method to set the size of a virtual register a posteriori. 2016-03-07 21:41:39 +00:00
MachineScheduler.cpp [TII] Allow getMemOpBaseRegImmOfs() to accept negative offsets. NFC. 2016-03-09 16:00:35 +00:00
MachineSink.cpp [TII] Allow getMemOpBaseRegImmOfs() to accept negative offsets. NFC. 2016-03-09 16:00:35 +00:00
MachineSSAUpdater.cpp
MachineTraceMetrics.cpp Reapply "CodeGen: Use references in MachineTraceMetrics::Trace, NFC" 2016-02-22 03:33:28 +00:00
MachineVerifier.cpp WIP: CodeGen: Use MachineInstr& in MachineInstrBundle.h, NFC 2016-02-27 17:05:33 +00:00
MIRPrinter.cpp [MIR] Teach the parser/printer that generic virtual registers do not need a register class. 2016-03-08 01:17:03 +00:00
MIRPrinter.h
MIRPrintingPass.cpp
module.modulemap
OptimizePHIs.cpp
ParallelCG.cpp Change split code gen to use ThreadPool 2016-03-04 15:39:13 +00:00
Passes.cpp Remove extra whitespace. NFC. 2016-01-18 06:42:51 +00:00
PeepholeOptimizer.cpp fix formatting; NFC 2015-12-29 19:34:53 +00:00
PHIElimination.cpp CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC 2016-02-27 06:40:41 +00:00
PHIEliminationUtils.cpp
PHIEliminationUtils.h
PostRASchedulerList.cpp Invoke DAG postprocessing in the post-RA scheduler 2016-03-08 16:54:20 +00:00
ProcessImplicitDefs.cpp Revert "CodeGen: MachineInstr::getIterator() => getInstrIterator(), NFC" 2016-02-22 20:49:58 +00:00
PrologEpilogInserter.cpp [WinEH] Allocate the registration node before the catch objects 2016-03-01 04:30:16 +00:00
PseudoSourceValue.cpp
README.txt
RegAllocBase.cpp
RegAllocBase.h
RegAllocBasic.cpp
RegAllocFast.cpp [RegAllocFast] Properly track the physical register definitions on calls. 2016-02-20 00:32:29 +00:00
RegAllocGreedy.cpp Remove uses of builtin comma operator. 2016-02-18 22:09:30 +00:00
RegAllocPBQP.cpp Annotate dump() methods with LLVM_DUMP_METHOD, addressing Richard Smith r259192 post commit comment. 2016-01-29 20:50:44 +00:00
RegisterClassInfo.cpp
RegisterCoalescer.cpp RegisterCoalescer: Remap subregister lanemasks before exchanging operands 2016-03-05 04:36:13 +00:00
RegisterCoalescer.h
RegisterPressure.cpp RegisterPressure: Small cleanup 2016-03-05 04:36:08 +00:00
RegisterScavenging.cpp
SafeStack.cpp [safestack] Make sure the unsafe stack pointer is popped in all cases 2016-02-02 01:03:11 +00:00
ScheduleDAG.cpp
ScheduleDAGInstrs.cpp WIP: CodeGen: Use MachineInstr& in MachineInstrBundle.h, NFC 2016-02-27 17:05:33 +00:00
ScheduleDAGPrinter.cpp
ScoreboardHazardRecognizer.cpp Annotate dump() methods with LLVM_DUMP_METHOD, addressing Richard Smith r259192 post commit comment. 2016-01-29 20:50:44 +00:00
ShadowStackGCLowering.cpp [GC] Consolidate all built in GCs into a single file [NFC] 2016-01-19 03:57:18 +00:00
ShrinkWrap.cpp [ShrinkWrapping] Give up on irreducible CFGs. 2016-01-07 01:23:49 +00:00
SjLjEHPrepare.cpp ADT: Remove == and != comparisons between ilist iterators and pointers 2016-02-21 20:39:50 +00:00
SlotIndexes.cpp CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC 2016-02-27 06:40:41 +00:00
Spiller.h
SpillPlacement.cpp
SpillPlacement.h
SplitKit.cpp CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC 2016-02-27 06:40:41 +00:00
SplitKit.h
StackColoring.cpp [X86] Don't give catch objects a displacement of zero 2016-03-03 00:01:25 +00:00
StackMapLivenessAnalysis.cpp
StackMaps.cpp
StackProtector.cpp [BPI] Replace weights by probabilities in BPI. 2015-12-22 18:56:14 +00:00
StackSlotColoring.cpp CodeGen: Update LiveIntervalAnalysis API to use MachineInstr&, NFC 2016-02-27 20:14:29 +00:00
TailDuplication.cpp Don't tail-duplicate blocks that contain convergent instructions. 2016-02-22 17:50:52 +00:00
TargetFrameLoweringImpl.cpp
TargetInstrInfo.cpp CodeGen: Change MachineInstr to use MachineInstr&, NFC 2016-02-27 20:01:33 +00:00
TargetLoweringBase.cpp [Power PC] softening long double type 2016-02-04 14:43:50 +00:00
TargetLoweringObjectFileImpl.cpp Add prefix based function layout when profile is available. 2016-02-23 03:39:24 +00:00
TargetOptionsImpl.cpp
TargetRegisterInfo.cpp raw_ostream: << operator for callables with raw_ostream argument 2015-12-04 01:31:59 +00:00
TargetSchedule.cpp CodeGen: TII: Take MachineInstr& in predicate API, NFC 2016-02-23 02:46:52 +00:00
TwoAddressInstructionPass.cpp CodeGen: Update LiveIntervalAnalysis API to use MachineInstr&, NFC 2016-02-27 20:14:29 +00:00
UnreachableBlockElim.cpp
VirtRegMap.cpp CodeGen: Take MachineInstr& in SlotIndexes and LiveIntervals, NFC 2016-02-27 06:40:41 +00:00
WinEHPrepare.cpp [WinEH] Don't remove unannotated inline-asm calls 2016-02-26 00:04:25 +00:00

//===---------------------------------------------------------------------===//

Common register allocation / spilling problem:

        mul lr, r4, lr
        str lr, [sp, #+52]
        ldr lr, [r1, #+32]
        sxth r3, r3
        ldr r4, [sp, #+52]
        mla r4, r3, lr, r4

can be:

        mul lr, r4, lr
        mov r4, lr
        str lr, [sp, #+52]
        ldr lr, [r1, #+32]
        sxth r3, r3
        mla r4, r3, lr, r4

and then "merge" mul and mov:

        mul r4, r4, lr
        str r4, [sp, #+52]
        ldr lr, [r1, #+32]
        sxth r3, r3
        mla r4, r3, lr, r4

It also increase the likelihood the store may become dead.

//===---------------------------------------------------------------------===//

bb27 ...
        ...
        %reg1037 = ADDri %reg1039, 1
        %reg1038 = ADDrs %reg1032, %reg1039, %NOREG, 10
    Successors according to CFG: 0x8b03bf0 (#5)

bb76 (0x8b03bf0, LLVM BB @0x8b032d0, ID#5):
    Predecessors according to CFG: 0x8b0c5f0 (#3) 0x8b0a7c0 (#4)
        %reg1039 = PHI %reg1070, mbb<bb76.outer,0x8b0c5f0>, %reg1037, mbb<bb27,0x8b0a7c0>

Note ADDri is not a two-address instruction. However, its result %reg1037 is an
operand of the PHI node in bb76 and its operand %reg1039 is the result of the
PHI node. We should treat it as a two-address code and make sure the ADDri is
scheduled after any node that reads %reg1039.

//===---------------------------------------------------------------------===//

Use local info (i.e. register scavenger) to assign it a free register to allow
reuse:
        ldr r3, [sp, #+4]
        add r3, r3, #3
        ldr r2, [sp, #+8]
        add r2, r2, #2
        ldr r1, [sp, #+4]  <==
        add r1, r1, #1
        ldr r0, [sp, #+4]
        add r0, r0, #2

//===---------------------------------------------------------------------===//

LLVM aggressively lift CSE out of loop. Sometimes this can be negative side-
effects:

R1 = X + 4
R2 = X + 7
R3 = X + 15

loop:
load [i + R1]
...
load [i + R2]
...
load [i + R3]

Suppose there is high register pressure, R1, R2, R3, can be spilled. We need
to implement proper re-materialization to handle this:

R1 = X + 4
R2 = X + 7
R3 = X + 15

loop:
R1 = X + 4  @ re-materialized
load [i + R1]
...
R2 = X + 7 @ re-materialized
load [i + R2]
...
R3 = X + 15 @ re-materialized
load [i + R3]

Furthermore, with re-association, we can enable sharing:

R1 = X + 4
R2 = X + 7
R3 = X + 15

loop:
T = i + X
load [T + 4]
...
load [T + 7]
...
load [T + 15]
//===---------------------------------------------------------------------===//

It's not always a good idea to choose rematerialization over spilling. If all
the load / store instructions would be folded then spilling is cheaper because
it won't require new live intervals / registers. See 2003-05-31-LongShifts for
an example.

//===---------------------------------------------------------------------===//

With a copying garbage collector, derived pointers must not be retained across
collector safe points; the collector could move the objects and invalidate the
derived pointer. This is bad enough in the first place, but safe points can
crop up unpredictably. Consider:

        %array = load { i32, [0 x %obj] }** %array_addr
        %nth_el = getelementptr { i32, [0 x %obj] }* %array, i32 0, i32 %n
        %old = load %obj** %nth_el
        %z = div i64 %x, %y
        store %obj* %new, %obj** %nth_el

If the i64 division is lowered to a libcall, then a safe point will (must)
appear for the call site. If a collection occurs, %array and %nth_el no longer
point into the correct object.

The fix for this is to copy address calculations so that dependent pointers
are never live across safe point boundaries. But the loads cannot be copied
like this if there was an intervening store, so may be hard to get right.

Only a concurrent mutator can trigger a collection at the libcall safe point.
So single-threaded programs do not have this requirement, even with a copying
collector. Still, LLVM optimizations would probably undo a front-end's careful
work.

//===---------------------------------------------------------------------===//

The ocaml frametable structure supports liveness information. It would be good
to support it.

//===---------------------------------------------------------------------===//

The FIXME in ComputeCommonTailLength in BranchFolding.cpp needs to be
revisited. The check is there to work around a misuse of directives in inline
assembly.

//===---------------------------------------------------------------------===//

It would be good to detect collector/target compatibility instead of silently
doing the wrong thing.

//===---------------------------------------------------------------------===//

It would be really nice to be able to write patterns in .td files for copies,
which would eliminate a bunch of explicit predicates on them (e.g. no side 
effects).  Once this is in place, it would be even better to have tblgen 
synthesize the various copy insertion/inspection methods in TargetInstrInfo.

//===---------------------------------------------------------------------===//

Stack coloring improvements:

1. Do proper LiveStackAnalysis on all stack objects including those which are
   not spill slots.
2. Reorder objects to fill in gaps between objects.
   e.g. 4, 1, <gap>, 4, 1, 1, 1, <gap>, 4 => 4, 1, 1, 1, 1, 4, 4

//===---------------------------------------------------------------------===//

The scheduler should be able to sort nearby instructions by their address. For
example, in an expanded memset sequence it's not uncommon to see code like this:

  movl $0, 4(%rdi)
  movl $0, 8(%rdi)
  movl $0, 12(%rdi)
  movl $0, 0(%rdi)

Each of the stores is independent, and the scheduler is currently making an
arbitrary decision about the order.

//===---------------------------------------------------------------------===//

Another opportunitiy in this code is that the $0 could be moved to a register:

  movl $0, 4(%rdi)
  movl $0, 8(%rdi)
  movl $0, 12(%rdi)
  movl $0, 0(%rdi)

This would save substantial code size, especially for longer sequences like
this. It would be easy to have a rule telling isel to avoid matching MOV32mi
if the immediate has more than some fixed number of uses. It's more involved
to teach the register allocator how to do late folding to recover from
excessive register pressure.