linux/include/asm-generic
Peter Zijlstra 609ca06638 sched/core: Create preempt_count invariant
Assuming units of PREEMPT_DISABLE_OFFSET for preempt_count() numbers.

Now that TASK_DEAD no longer results in preempt_count() == 3 during
scheduling, we will always call context_switch() with preempt_count()
== 2.

However, we don't always end up with preempt_count() == 2 in
finish_task_switch() because new tasks get created with
preempt_count() == 1.

Create FORK_PREEMPT_COUNT and set it to 2 and use that in the right
places. Note that we cannot use INIT_PREEMPT_COUNT as that serves
another purpose (boot).

After this, preempt_count() is invariant across the context switch,
with exception of PREEMPT_ACTIVE.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-10-06 17:08:14 +02:00
..
bitops arch: Mass conversion of smp_mb__*() 2014-04-18 14:20:48 +02:00
4level-fixup.h mm, asm-generic: define PUD_SHIFT in <asm-generic/4level-fixup.h> 2015-02-11 17:06:03 -08:00
asm-offsets.h asm-generic: Add common asm-offsets.h 2015-06-23 13:35:49 +09:00
atomic64.h atomic: Provide atomic_{or,xor,and} 2015-07-27 14:06:24 +02:00
atomic-long.h locking, asm-generic: Add _{relaxed|acquire|release}() variants for 'atomic_long_t' 2015-08-12 11:59:01 +02:00
atomic.h atomic: Collapse all atomic_{set,clear}_mask definitions 2015-07-27 14:06:24 +02:00
audit_change_attr.h audit: Modify a set of system calls in audit class definitions 2014-01-17 17:01:46 -05:00
audit_dir_write.h
audit_read.h
audit_signal.h
audit_write.h audit: Modify a set of system calls in audit class definitions 2014-01-17 17:01:46 -05:00
barrier.h locking, arch: use WRITE_ONCE()/READ_ONCE() in smp_store_release()/smp_load_acquire() 2015-08-03 10:59:30 +02:00
bitops.h arch: Prepare for smp_mb__{before,after}_atomic() 2014-04-18 11:40:30 +02:00
bitsperlong.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
bug.h bug: Make BUG() always stop the machine 2014-04-07 16:36:10 -07:00
bugs.h
cache.h
cacheflush.h asm-generic/cacheflush.h: flush icache when copying to user pages 2011-05-25 08:39:37 -07:00
checksum.h asm-generic headers: Allow yet more arch overrides in checksum.h 2013-02-11 20:00:33 +05:30
clkdev.h asm-generic: COMMON_CLK defines __clk_{get,put} 2014-09-25 18:00:45 -07:00
cmpxchg-local.h LLVMLinux: Remove warning about returning an uninitialized variable 2014-04-09 13:44:35 -07:00
cmpxchg.h locking/rtmutex: Drop usage of __HAVE_ARCH_CMPXCHG 2015-05-13 10:51:28 +02:00
cputime_jiffies.h sched, time: Fix build error with 64 bit cputime_t on 32 bit systems 2014-10-03 05:46:55 +02:00
cputime_nsecs.h sched, time: Fix build error with 64 bit cputime_t on 32 bit systems 2014-10-03 05:46:55 +02:00
cputime.h cputime: Generic on-demand virtual cputime accounting 2013-01-27 19:23:27 +01:00
current.h
delay.h asm-generic: delay.h fix udelay and ndelay for 8 bit args 2011-07-22 18:45:33 +02:00
device.h
div64.h
dma-coherent.h DMA-API: Change dma_declare_coherent_memory() CPU address to phys_addr_t 2014-05-20 16:55:23 -06:00
dma-contiguous.h asm-generic: Add dma-contiguous.h 2014-09-22 13:35:51 +02:00
dma-mapping-broken.h asm-generic/dma-mapping-broken.h: Provide dma_alloc_attrs()/dma_free_attrs() 2012-12-25 20:14:54 +01:00
dma-mapping-common.h dma-mapping: consolidate dma_set_mask 2015-09-10 13:29:01 -07:00
dma.h
early_ioremap.h Merge branch 'akpm' (patches from Andrew) 2015-09-08 17:52:23 -07:00
emergency-restart.h
exec.h Split arch_align_stack() out from asm-generic/system.h 2012-03-28 18:30:03 +01:00
fb.h
fixmap.h mm: provide early_memremap_ro to establish read-only mapping 2015-08-20 12:24:22 +01:00
ftrace.h
futex.h sched/preempt, futex: Disable preemption in UP futex_atomic_cmpxchg_inatomic() explicitly 2015-05-19 08:39:16 +02:00
getorder.h bitops: Add missing parentheses to new get_order macro 2012-02-24 10:39:27 -08:00
gpio.h gpio: remove gpiod_sysfs_set_active_low 2015-05-12 10:46:53 +02:00
hardirq.h
hugetlb.h mm: Fix generic hugetlb pte check return type. 2013-10-02 20:02:35 -04:00
hw_irq.h
ide_iops.h
int-ll64.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
io-64-nonatomic-hi-lo.h readq/writeq: Add explicit lo_hi_[read|write]_q and hi_lo_[read|write]_q 2014-07-04 13:27:30 +02:00
io-64-nonatomic-lo-hi.h readq/writeq: Add explicit lo_hi_[read|write]_q and hi_lo_[read|write]_q 2014-07-04 13:27:30 +02:00
io.h x86/mm, asm-generic: Add IOMMU ioremap_uc() variant default 2015-07-21 10:47:03 +02:00
ioctl.h include/asm-generic/ioctl.h: fix _IOC_TYPECHECK sparse error 2014-06-06 16:08:13 -07:00
iomap.h x86/mm, asm-generic: Add ioremap_wt() for creating Write-Through mappings 2015-06-07 15:28:56 +02:00
irq_regs.h
irq_work.h irq_work: Introduce arch_irq_work_has_interrupt() 2014-09-13 18:38:07 +02:00
irq.h
irqflags.h
Kbuild.asm UAPI: Set up uapi/asm/Kbuild.asm 2012-10-02 18:01:56 +01:00
kdebug.h
kmap_types.h asm-generic: remove km_type definitions 2012-07-24 15:27:30 +08:00
kvm_para.h KVM: add kvm_para_available to asm-generic/kvm_para.h 2013-06-05 13:21:29 +03:00
libata-portmap.h
linkage.h
local64.h atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
local.h atomic: use <linux/atomic.h> 2011-07-26 16:49:47 -07:00
mcs_spinlock.h locking/mcs: Allow architecture specific asm files to be used for contended case 2014-02-09 21:18:52 +01:00
memory_model.h mm: fix type cast in __pfn_to_phys() 2015-09-19 03:58:10 -04:00
mm_hooks.h mm: Make arch_unmap()/bprm_mm_init() available to all architectures 2014-11-19 11:54:13 +01:00
mm-arch-hooks.h mm: clean up per architecture MM hook header files 2015-07-17 16:39:53 -07:00
mmu_context.h asm-generic: Remove asm-generic arch_bprm_mm_init() 2014-11-22 21:52:08 +01:00
mmu.h asm-generic/mmu.h: Add support for FDPIC 2012-12-09 23:14:14 +01:00
module.h Make most arch asm/module.h files use asm-generic/module.h 2012-09-28 14:31:03 +09:30
msi.h asm-generic: Add msi.h 2014-11-23 13:01:47 +01:00
mutex-dec.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex-null.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex-xchg.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex.h
page.h The following changes since commit 3ee72ca992 2012-01-10 17:39:40 -08:00
param.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
parport.h include: remove __dev* attributes. 2013-01-03 15:57:16 -08:00
pci_iomap.h PCI: Add pci_iomap_wc() variants 2015-08-25 09:59:45 +02:00
pci-bridge.h PCI: work around Stratus ftServer broken PCIe hierarchy 2012-04-30 15:21:02 -06:00
pci-dma-compat.h pci-dma-compat: add pci_zalloc_consistent helper 2014-08-08 15:57:28 -07:00
pci.h PCI: Remove unused pcibios_select_root() (again) 2015-06-08 07:56:21 -05:00
percpu.h percpu: preffity percpu header files 2014-06-17 19:12:40 -04:00
pgalloc.h
pgtable-nopmd.h
pgtable-nopud.h
pgtable.h mm: clarify that the function operates on hugepage pte 2015-06-24 17:49:44 -07:00
preempt.h sched/core: Create preempt_count invariant 2015-10-06 17:08:14 +02:00
ptrace.h asm-generic/ptrace.h: start a common low level ptrace helper 2011-05-26 17:12:36 -07:00
qrwlock_types.h locking/rwlocks: Introduce 'qrwlocks' - fair, queued rwlocks 2014-06-06 07:58:28 +02:00
qrwlock.h locking/qrwlock: Make use of _{acquire|release|relaxed}() atomics 2015-08-12 11:59:06 +02:00
qspinlock_types.h locking/qspinlock: Optimize for smaller NR_CPUS 2015-05-08 12:36:48 +02:00
qspinlock.h locking/qspinlock/x86: Fix performance regression under unaccelerated VMs 2015-09-11 07:49:42 +02:00
resource.h asm-generic: remove _STK_LIM_MAX 2014-05-15 00:32:09 +01:00
rtc.h rtc: cmos: century support 2015-09-05 13:19:09 +02:00
rwsem.h asm-generic: rwsem: de-PPCify rwsem.h 2014-03-14 18:02:08 +00:00
seccomp.h seccomp: allow COMPAT sigreturn overrides 2015-04-17 09:04:09 -04:00
sections.h nosave: consolidate __nosave_{begin,end} in <asm/sections.h> 2014-10-09 22:26:04 -04:00
segment.h
serial.h
siginfo.h constify copy_siginfo_to_user{,32}() 2013-11-09 00:16:29 -05:00
signal.h unify default ptrace_signal_deliver 2012-11-29 00:01:23 -05:00
simd.h crypto: create generic version of ablk_helper 2013-09-24 06:02:24 +10:00
sizes.h ARM: 7430/1: sizes.h: move from asm-generic to <linux/sizes.h> 2012-06-28 17:14:34 +01:00
spinlock.h
statfs.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
string.h
switch_to.h Split the switch_to() wrapper out of asm-generic/system.h 2012-03-28 18:30:03 +01:00
syscall.h syscall.h: fix doc text for syscall_get_arch() 2014-09-23 16:20:00 -04:00
syscalls.h burying unused conditionals 2013-02-14 09:21:15 -05:00
termios-base.h
termios.h UAPI: (Scripted) Disintegrate include/asm-generic 2012-10-04 18:20:15 +01:00
timex.h
tlb.h mm: mmu_gather: use tlb->end != 0 only for TLB invalidation 2015-01-13 15:20:40 +13:00
tlbflush.h BUG: headers with BUG/BUG_ON etc. need linux/bug.h 2012-03-04 17:54:34 -05:00
topology.h
trace_clock.h tracing,x86: Add a TSC trace_clock 2012-11-13 15:48:27 -05:00
uaccess-unaligned.h
uaccess.h asm-generic: uaccess: Spelling s/a ny/any/ 2014-01-02 10:45:23 +01:00
unaligned.h asm-generic: allow generic unaligned access if the arch supports it 2014-05-08 10:22:23 +02:00
unistd.h We get rid of the general module prefix confusion with a binary config option, 2013-05-05 10:58:06 -07:00
user.h
vga.h
vmlinux.lds.h kbuild: Fix .text.unlikely placement 2015-08-20 14:55:54 +02:00
vtime.h include/asm-generic/vtime.h: avoid zero-length file 2013-09-30 14:31:02 -07:00
word-at-a-time.h Make asm/word-at-a-time.h available on all architectures 2015-07-08 16:41:55 -04:00
xor.h asm-generic: xor: mark static functions as __maybe_unused 2012-10-03 21:21:06 +02:00