linux/arch
Nick Piggin 314cdbefd1 x86: FIFO ticket spinlocks
Introduce ticket lock spinlocks for x86 which are FIFO. The implementation
is described in the comments. The straight-line lock/unlock instruction
sequence is slightly slower than the dec based locks on modern x86 CPUs,
however the difference is quite small on Core2 and Opteron when working out of
cache, and becomes almost insignificant even on P4 when the lock misses cache.
trylock is more significantly slower, but they are relatively rare.

On an 8 core (2 socket) Opteron, spinlock unfairness is extremely noticable,
with a userspace test having a difference of up to 2x runtime per thread, and
some threads are starved or "unfairly" granted the lock up to 1 000 000 (!)
times. After this patch, all threads appear to finish at exactly the same
time.

The memory ordering of the lock does conform to x86 standards, and the
implementation has been reviewed by Intel and AMD engineers.

The algorithm also tells us how many CPUs are contending the lock, so
lockbreak becomes trivial and we no longer have to waste 4 bytes per
spinlock for it.

After this, we can no longer spin on any locks with preempt enabled
and cannot reenable interrupts when spinning on an irq safe lock, because
at that point we have already taken a ticket and the would deadlock if
the same CPU tries to take the lock again.  These are questionable anyway:
if the lock happens to be called under a preempt or interrupt disabled section,
then it will just have the same latency problems. The real fix is to keep
critical sections short, and ensure locks are reasonably fair (which this
patch does).

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-01-30 13:31:21 +01:00
..
alpha remove __attribute_used__ 2008-01-28 23:21:18 +01:00
arm spinlock: lockbreak cleanup 2008-01-30 13:31:20 +01:00
avr32 all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
blackfin all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
cris all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
frv Remove references to "make dep" 2008-01-28 23:22:13 +01:00
h8300 all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
ia64 spinlock: lockbreak cleanup 2008-01-30 13:31:20 +01:00
m32r spinlock: lockbreak cleanup 2008-01-30 13:31:20 +01:00
m68k all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
m68knommu all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
mips spinlock: lockbreak cleanup 2008-01-30 13:31:20 +01:00
parisc spinlock: lockbreak cleanup 2008-01-30 13:31:20 +01:00
powerpc spinlock: lockbreak cleanup 2008-01-30 13:31:20 +01:00
ppc Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-2.6.25 2008-01-29 22:54:01 +11:00
s390 all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
sh all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
sparc all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
sparc64 spinlock: lockbreak cleanup 2008-01-30 13:31:20 +01:00
um UML: remove remaining FASTCALL uses 2008-01-30 13:31:14 +01:00
v850 all archs: consolidate init and exit sections in vmlinux.lds.h 2008-01-28 23:21:17 +01:00
x86 x86: FIFO ticket spinlocks 2008-01-30 13:31:21 +01:00
xtensa Remove references to "make dep" 2008-01-28 23:22:13 +01:00
.gitignore arch: Ignore arch/i386 and arch/x86_64 2008-01-19 21:29:39 -08:00