linux/arch/x86/kernel/cpu
Peter Zijlstra 9536c8d2da perf/x86: Optimize intel_pmu_pebs_fixup_ip()
There's been reports of high NMI handler overhead, highlighted by
such kernel messages:

  [ 3697.380195] perf samples too long (10009 > 10000), lowering kernel.perf_event_max_sample_rate to 13000
  [ 3697.389509] INFO: NMI handler (perf_event_nmi_handler) took too long to run: 9.331 msecs

Don Zickus analyzed the source of the overhead and reported:

 > While there are a few places that are causing latencies, for now I focused on
 > the longest one first.  It seems to be 'copy_user_from_nmi'
 >
 > intel_pmu_handle_irq ->
 >	intel_pmu_drain_pebs_nhm ->
 >		__intel_pmu_drain_pebs_nhm ->
 >			__intel_pmu_pebs_event ->
 >				intel_pmu_pebs_fixup_ip ->
 >					copy_from_user_nmi
 >
 > In intel_pmu_pebs_fixup_ip(), if the while-loop goes over 50, the sum of
 > all the copy_from_user_nmi latencies seems to go over 1,000,000 cycles
 > (there are some cases where only 10 iterations are needed to go that high
 > too, but in generall over 50 or so).  At this point copy_user_from_nmi
 > seems to account for over 90% of the nmi latency.

The solution to that is to avoid having to call copy_from_user_nmi() for
every instruction.

Since we already limit the max basic block size, we can easily
pre-allocate a piece of memory to copy the entire thing into in one
go.

Don reported this test result:

 > Your patch made a huge difference in improvement.  The
 > copy_from_user_nmi() no longer hits the million of cycles.  I still
 > have a batch of 100,000-300,000 cycles.  My longest NMI paths used
 > to be dominated by copy_from_user_nmi, now it is not (I have to dig
 > up the new hot path).

Reported-and-tested-by: Don Zickus <dzickus@redhat.com>
Cc: jmario@redhat.com
Cc: acme@infradead.org
Cc: dave.hansen@linux.intel.com
Cc: eranian@google.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20131016105755.GX10651@twins.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-10-16 15:44:00 +02:00
..
mcheck Bit 12 may or may not be set in MCi_STATUS.MCACOD when 2013-08-12 19:51:43 +02:00
mtrr mm: vmstats: track TLB flush stats on UP too 2013-09-11 15:57:09 -07:00
.gitignore
amd.c Merge branch 'x86-asmlinkage-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2013-09-04 08:42:44 -07:00
bugs_64.c
bugs.c x86: Get rid of ->hard_math and all the FPU asm fu 2013-06-06 14:32:04 -07:00
centaur.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
common.c x86, asmlinkage: Make several variables used from assembler/linker script visible 2013-08-06 14:20:13 -07:00
cpu.h
cyrix.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
hypervisor.c x86: Correctly detect hypervisor 2013-08-05 06:35:33 -07:00
intel_cacheinfo.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
intel.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
Makefile perf/x86/amd: AMD IOMMU Performance Counter PERF uncore PMU implementation 2013-06-19 13:04:53 +02:00
match.c
mkcapflags.sh mkcapflags.pl: convert to mkcapflags.sh 2013-04-29 15:54:27 -07:00
mshyperv.c x86: Correctly detect hypervisor 2013-08-05 06:35:33 -07:00
perf_event_amd_ibs.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
perf_event_amd_iommu.c perf/x86/amd: Do not print an error when the device is not present 2013-07-05 08:27:15 +02:00
perf_event_amd_iommu.h perf/x86/amd: AMD IOMMU Performance Counter PERF uncore PMU implementation 2013-06-19 13:04:53 +02:00
perf_event_amd_uncore.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
perf_event_amd.c perf: Convert kmalloc_node(...GFP_ZERO...) to kzalloc_node() 2013-09-02 08:42:49 +02:00
perf_event_intel_ds.c perf/x86: Optimize intel_pmu_pebs_fixup_ip() 2013-10-16 15:44:00 +02:00
perf_event_intel_lbr.c perf/x86: Suppress duplicated abort LBR records 2013-10-04 10:06:16 +02:00
perf_event_intel_uncore.c perf/x86/intel/uncore: Don't use smp_processor_id() in validate_group() 2013-09-20 06:54:36 +02:00
perf_event_intel_uncore.h perf/x86/intel/uncore: Enable EV_SEL_EXT bit for PCU 2013-08-16 17:55:50 +02:00
perf_event_intel.c perf/x86: Suppress duplicated abort LBR records 2013-10-04 10:06:16 +02:00
perf_event_knc.c x86: Constify a few items 2013-03-11 15:11:03 +01:00
perf_event_p4.c perf/x86/intel/P4: Robistify P4 PMU types 2013-04-26 09:31:41 +02:00
perf_event_p6.c x86: Constify a few items 2013-03-11 15:11:03 +01:00
perf_event.c perf/x86: Clean up cap_user_time* setting 2013-10-04 09:58:55 +02:00
perf_event.h perf/x86: Suppress duplicated abort LBR records 2013-10-04 10:06:16 +02:00
perfctr-watchdog.c perf/x86: Add support for Intel Xeon-Phi Knights Corner PMU 2012-10-04 13:32:37 +02:00
powerflags.c update AMD powerflags comments 2013-05-28 12:02:10 +02:00
proc.c x86: Get rid of ->hard_math and all the FPU asm fu 2013-06-06 14:32:04 -07:00
rdrand.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
scattered.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
topology.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
transmeta.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
umc.c x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
vmware.c x86: Correctly detect hypervisor 2013-08-05 06:35:33 -07:00