Easwaran Raman 0e9b0bc63d [Inliner] Do not apply any bonus for cold callsites. ...

Summary:
Inlining threshold is increased by application of bonuses when the
callee has a single reachable basic block or is rich in vector
instructions. Similarly, inlining cost is reduced by applying a large
bonus when the last call to a static function is considered for
inlining. This patch disables the application of these bonuses when the
callsite or the callee is cold. The intention here is to prevent a large
cold callsite from being inlined to a non-cold caller that could prevent
the caller from being inlined. This is especially important when the
cold callsite is a last call to a static since the associated bonus is
very high.

Reviewers: chandlerc, davidxl

Subscribers: danielcdh, llvm-commits

Differential Revision: https://reviews.llvm.org/D35823

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@309441 91177308-0d34-0410-b5e6-96231b3b80d8

2017-07-28 21:47:36 +00:00

..

AArch64

[TTI] Refine the cost of EXT in getUserCost()

2017-07-15 02:12:16 +00:00

ARM

[ARM] Fix typo in test added in r307889

2017-07-13 08:53:43 +00:00

PowerPC

[TTI] Refine the cost of EXT in getUserCost()

2017-07-15 02:12:16 +00:00

X86

[TTI] Refine the cost of EXT in getUserCost()

2017-07-15 02:12:16 +00:00

2003-09-14-InlineValue.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

2003-09-22-PHINodeInlineFail.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

2003-09-22-PHINodesInExceptionDest.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

2003-09-22-PHINodesInNormalInvokeDest.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

2003-10-13-AllocaDominanceProblem.ll

Change tests from "opt %s" to "opt < %s" so that opt doesn't see the

2009-09-11 18:01:28 +00:00

2004-04-15-InlineDeletesCall.ll

Replace more uses of 'unwind' in the tests with calls to landingpad and

2011-09-19 22:11:35 +00:00

2004-04-20-InlineLinkOnce.ll

Change tests from "opt %s" to "opt < %s" so that opt doesn't see the

2009-09-11 18:01:28 +00:00

2004-10-17-InlineFunctionWithoutReturn.ll

Replace more uses of 'unwind' in the tests with calls to landingpad and

2011-09-19 22:11:35 +00:00

2006-01-14-CallGraphUpdate.ll

Change tests from "opt %s" to "opt < %s" so that opt doesn't see the

2009-09-11 18:01:28 +00:00

2006-07-12-InlinePruneCGUpdate.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

2006-11-09-InlineCGUpdate-2.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

2006-11-09-InlineCGUpdate.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

2007-04-15-InlineEH.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

2007-06-25-WeakInline.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

2007-12-19-InlineNoUnwind.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

2008-09-02-NoInline.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

2009-01-08-NoInlineDynamicAlloca.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

2009-01-13-RecursiveInlineCrash.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

2009-05-07-CallUsingSelfCrash.ll

Change tests from "opt %s" to "opt < %s" so that opt doesn't see the

2009-09-11 18:01:28 +00:00

2010-05-12-ValueMap.ll

Tests: rewrite 'opt ... %s' to 'opt ... < %s' so that opt does not emit a ModuleID

2013-01-01 13:57:25 +00:00

align.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

alloca_test.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

alloca-bonus.ll

Add address space mangling to lifetime intrinsics

2017-04-10 20:18:21 +00:00

alloca-dbgdeclare-merge.ll

[PR27284] Reverse the ownership between DICompileUnit and DISubprogram.

2016-04-15 15:57:41 +00:00

alloca-dbgdeclare.ll

Remove the obsolete offset parameter from @llvm.dbg.value

2017-07-28 20:21:02 +00:00

alloca-in-scc.ll

[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction

2015-02-27 19:29:02 +00:00

alloca-merge-align.ll

Make DataLayout Non-Optional in the Module

2015-03-04 18:43:29 +00:00

always-inline.ll

Inliner: Don't remove calls to readnone+nounwind (but not always_inline) functions in the AlwaysInliner

2017-06-12 23:01:17 +00:00

arg-attr-propagation.ll

Revert r296366 "[InlineFunction] add nonnull assumptions based on argument attributes"

2017-02-27 22:33:02 +00:00

array_merge.ll

Commit test change, forgotten as part of r131838.

2011-05-22 05:31:47 +00:00

array-alloca.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

attributes.ll

[PM] Teach the inliner in the new PM to merge attributes after inlining.

2016-12-27 03:39:54 +00:00

basictest.ll

Inliner: Don't touch indirect calls

2017-06-09 03:29:20 +00:00

bfi-update.ll

Fix a bug in caller's BFI update code after inlining.

2017-02-14 22:49:28 +00:00

blockaddress.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

byval_lifetime.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

byval-tail-call.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

byval.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

callgraph-update.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

casts.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

cfg_preserve_test.ll

[Inliner] Modernize all of the inliner tests that were using grep.

2016-12-27 02:47:37 +00:00

cgscc-incremental-invalidate.ll

[PM] Fix a nasty bug in the new PM where we failed to properly

2017-07-09 13:45:11 +00:00

cgscc-invalidate.ll

[PM/Inliner] Make the new PM's inliner process call edges across an

2017-03-09 11:35:40 +00:00

cgscc-update.ll

[PM] Teach the inliner's call graph update to handle inserting new edges

2016-12-28 03:13:12 +00:00

clear-analyses.ll

[PH] Replace uses of AssertingVH from members of analysis results with

2017-01-24 12:55:57 +00:00

comdat-ipo.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

crash2.ll

Remove the ScalarReplAggregates pass

2016-06-15 00:19:09 +00:00

crash-lifetime-marker.ll

Add address space mangling to lifetime intrinsics

2017-04-10 20:18:21 +00:00

crash.ll

Move the personality function from LandingPadInst to Function

2015-06-17 20:52:32 +00:00

debug-info-duplicate-calls.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

debug-invoke.ll

[PR27284] Reverse the ownership between DICompileUnit and DISubprogram.

2016-04-15 15:57:41 +00:00

delete-call.ll

Fix FileCheck --check-prefix lines.

2013-08-12 12:43:26 +00:00

deopt-bundles.ll

Nonnull elements in OperandBundleCallSites are not all Instructions

2015-12-19 22:40:28 +00:00

deoptimize-intrinsic-cconv.ll

All llvm.deoptimize declarations must use the same calling convention

2016-05-12 01:17:38 +00:00

deoptimize-intrinsic.ll

All llvm.deoptimize declarations must use the same calling convention

2016-05-12 01:17:38 +00:00

devirtualize-2.ll

[PM] Introduce a devirtualization iteration layer for the new PM.

2016-12-28 11:07:33 +00:00

devirtualize-3.ll

Remove the ScalarReplAggregates pass

2016-06-15 00:19:09 +00:00

devirtualize.ll

[opaque pointer type] Add textual IR support for explicit type parameter to gep operator

2015-03-13 18:20:45 +00:00

dynamic_alloca_test.ll

Update Transforms tests to use CHECK-LABEL for easier debugging. No functionality change.

2013-07-14 01:42:54 +00:00

ephemeral.ll

[InlineCost] Reduce inline thresholds to compensate for cost changes

2016-11-28 11:07:37 +00:00

externally_available.ll

[Inliner] Modernize all of the inliner tests that were using grep.

2016-12-27 02:47:37 +00:00

frameescape.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

function-count-update-2.ll

Improve PGO support for the new inliner

2017-01-20 22:44:04 +00:00

function-count-update-3.ll

Improve PGO support for the new inliner

2017-01-20 22:44:04 +00:00

function-count-update.ll

Improve PGO support for the new inliner

2017-01-20 22:44:04 +00:00

guard-intrinsic.ll

Introduce a @llvm.experimental.guard intrinsic

2016-03-31 00:18:46 +00:00

gvn-inline-iteration.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

ignore-debug-info.ll

Remove the obsolete offset parameter from @llvm.dbg.value

2017-07-28 20:21:02 +00:00

inalloca-not-static.ll

[PM] Wire up another test to the new pass manager.

2016-12-27 06:46:16 +00:00

inline_cleanup.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline_constprop.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline_dbg_declare.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline_dce.ll

[Inliner] Modernize all of the inliner tests that were using grep.

2016-12-27 02:47:37 +00:00

inline_invoke.ll

[PM] Move the collection of call sites to a more appropriate place

2016-12-27 01:24:50 +00:00

inline_minisize.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

inline_prune.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline_returns_twice.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline_ssp.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline_stats.ll

[ThinLTO] Correct counting of functions in inliner stats

2017-03-24 17:59:06 +00:00

inline_unreachable-2.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline_unreachable.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-assume.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-byval-bonus.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-cold-callee.ll

Improve PGO support for the new inliner

2017-01-20 22:44:04 +00:00

inline-cold-callsite-pgo.ll

[NewPM/Inliner] Reduce threshold for cold callsites in the non-PGO case

2017-06-27 23:11:18 +00:00

inline-cold-callsite.ll

[NewPM/Inliner] Reduce threshold for cold callsites in the non-PGO case

2017-06-27 23:11:18 +00:00

inline-cold.ll

Decrease inlinecold-threshold to 45

2017-05-11 21:36:28 +00:00

inline-constexpr-addrspacecast-argument.ll

ConstantFold: Handle gep nonnull, undef as well

2017-05-08 17:37:33 +00:00

inline-fast-math-flags.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-fp.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-funclets.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-hot-callee.ll

[Inliner] Do not mix callsite and callee hotness based updates.

2017-05-16 21:18:09 +00:00

inline-hot-callsite-2.ll

Improve PGO support for the new inliner

2017-01-20 22:44:04 +00:00

inline-hot-callsite.ll

[PM] Add ProfileSummaryAnalysis as a required pass in the new pipeline.

2017-05-04 16:58:45 +00:00

inline-indirect.ll

Fix a crash in the assumption cache when inlining indirect function calls

2015-02-12 21:04:22 +00:00

inline-invoke-tail.ll

[Inliner] Fix a test where I typo'ed 'CHECK' as 'CHCEK' when converting

2017-01-04 11:15:01 +00:00

inline-invoke-with-asm-call.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-musttail-varargs.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-optnone.ll

The 'optnone' attribute means don't inline anything into this function

2013-11-18 21:44:03 +00:00

inline-optsize.ll

[InlineCost] Remove skew when calculating call costs

2016-11-14 11:14:41 +00:00

inline-probe-stack.ll

Add a "probe-stack" attribute

2017-06-21 18:46:50 +00:00

inline-stack-probe-size.ll

Define behavior of "stack-probe-size" attribute when inlining.

2017-06-22 23:22:36 +00:00

inline-tail.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

inline-threshold.ll

Allow -inline-threshold to override default threshold.

2016-05-19 23:02:09 +00:00

inline-vla.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

internal-scc-members.ll

[PM/Inliner] Fix a bug in r297374 where we would leave stale calls in

2017-03-16 10:45:42 +00:00

invoke_test-1.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

invoke_test-2.ll

[Inliner] Modernize all of the inliner tests that were using grep.

2016-12-27 02:47:37 +00:00

invoke_test-3.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

invoke-cleanup.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

invoke-combine-clauses.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

invoke-cost.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

last-call-bonus.ll

[Inliner] Do not apply any bonus for cold callsites.

2017-07-28 21:47:36 +00:00

last-call-no-bonus.ll

[Inliner] Do not apply any bonus for cold callsites.

2017-07-28 21:47:36 +00:00

last-callsite.ll

[PM] Provide an initial, minimal port of the inliner to the new pass manager.

2016-12-20 03:15:32 +00:00

lifetime-no-datalayout.ll

Add address space mangling to lifetime intrinsics

2017-04-10 20:18:21 +00:00

lifetime.ll

Add address space mangling to lifetime intrinsics

2017-04-10 20:18:21 +00:00

local-as-metadata-undominated-use.ll

Remove the obsolete offset parameter from @llvm.dbg.value

2017-07-28 20:21:02 +00:00

monster_scc.ll

[PM/Inliner] Add a test case that encapsulates the core issue addressed

2017-03-16 10:13:55 +00:00

nested-inline.ll

[PM] Provide an initial, minimal port of the inliner to the new pass manager.

2016-12-20 03:15:32 +00:00

noalias2.ll

Have a single way for creating unique value names.

2015-11-22 00:16:24 +00:00

noalias-calls.ll

Revert "Change memcpy/memset/memmove to have dest and source alignments."

2015-11-19 05:56:52 +00:00

noalias-cs.ll

Have a single way for creating unique value names.

2015-11-22 00:16:24 +00:00

noalias.ll

[opaque pointer type] Add textual IR support for explicit type parameter to load instruction

2015-02-27 21:17:42 +00:00

noinline-recursive-fn.ll

[PM] Add one of the features left out of the initial inliner patch:

2016-12-27 06:46:20 +00:00

noinline.ll

Tests: rewrite 'opt ... %s' to 'opt ... < %s' so that opt does not emit a ModuleID

2013-01-01 13:57:25 +00:00

nonnull.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

null-function.ll

Add new lines to debugging information.

2014-07-03 02:11:49 +00:00

optimization-remarks-passed-yaml.ll

[InlineCost] Remove skew when calculating call costs

2016-11-14 11:14:41 +00:00

optimization-remarks-with-hotness.ll

[Inliner] Fold analysis remarks into missed remarks

2017-01-30 16:22:45 +00:00

optimization-remarks-yaml.ll

[ORE] Add diagnostics hotness threshold

2017-06-30 23:14:53 +00:00

optimization-remarks.ll

[Inliner] Fold analysis remarks into missed remarks

2017-01-30 16:22:45 +00:00

parallel-loop-md.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

partial-inline-act.ll

Renable test that was disabled due to cost analysis

2017-05-14 02:58:39 +00:00

pr21206.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

pr22285.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

pr26698.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

pr28298.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

pr33637.ll

[Cloner] Re-map simplfied cloned instructions.

2017-07-01 03:29:33 +00:00

PR4909.ll

[PM] Port PartialInlining to the new PM

2016-06-27 16:50:18 +00:00

prof-update-instr.ll

Fix update VP metadata after inlining for instrumentation PGO

2017-05-22 20:28:18 +00:00

prof-update-sample.ll

Fix update VP metadata after inlining for instrumentation PGO

2017-05-22 20:28:18 +00:00

profile-meta.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

ptr-diff.ll

[InlineCost] Remove skew when calculating call costs

2016-11-14 11:14:41 +00:00

recursive.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

store-sroa.ll

Correctly estimate SROA savings for store operands in inline cost analysis.

2015-03-20 18:33:12 +00:00

switch.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

vector-bonus.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00

vector-no-bonus.ll

[Inliner] Do not apply any bonus for cold callsites.

2017-07-28 21:47:36 +00:00

zero-cost.ll

[PM] Turn on the new PM's inliner in addition to the current one for

2016-12-27 07:18:43 +00:00