linux/net/core
Eric Dumazet 29b4433d99 net: percpu net_device refcount
We tried very hard to remove all possible dev_hold()/dev_put() pairs in
network stack, using RCU conversions.

There is still an unavoidable device refcount change for every dst we
create/destroy, and this can slow down some workloads (routers or some
app servers, mmap af_packet)

We can switch to a percpu refcount implementation, now dynamic per_cpu
infrastructure is mature. On a 64 cpus machine, this consumes 256 bytes
per device.

On x86, dev_hold(dev) code :

before
        lock    incl 0x280(%ebx)
after:
        movl    0x260(%ebx),%eax
        incl    fs:(%eax)

Stress bench :

(Sending 160.000.000 UDP frames,
IP route cache disabled, dual E5540 @2.53GHz,
32bit kernel, FIB_TRIE)

Before:

real    1m1.662s
user    0m14.373s
sys     12m55.960s

After:

real    0m51.179s
user    0m15.329s
sys     10m15.942s

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2010-10-12 12:35:25 -07:00
..
datagram.c net: poll() optimizations 2010-09-06 18:48:45 -07:00
dev_addr_lists.c
dev.c net: percpu net_device refcount 2010-10-12 12:35:25 -07:00
drop_monitor.c
dst.c net dst: use a percpu_counter to track entries 2010-10-11 13:06:53 -07:00
ethtool.c Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2010-10-11 12:30:34 -07:00
fib_rules.c fib: RCU conversion of fib_lookup() 2010-10-05 20:39:38 -07:00
filter.c net: sk_{detach|attach}_filter() rcu fixes 2010-09-27 21:30:44 -07:00
flow.c net: return operator cleanup 2010-09-23 14:33:39 -07:00
gen_estimator.c pkt_sched: remov unnecessary bh_disable 2010-09-10 12:47:59 -07:00
gen_stats.c
iovec.c Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2010-10-04 11:56:38 -07:00
kmap_skb.h
link_watch.c
Makefile
neighbour.c neigh: Protect neigh->ha[] with a seqlock 2010-10-11 12:54:04 -07:00
net_namespace.c
net-sysfs.c net: Fix rxq ref counting 2010-10-08 14:34:32 -07:00
net-sysfs.h net: Allow changing number of RX queues after device allocation 2010-09-27 22:09:49 -07:00
net-traces.c
netevent.c
netpoll.c
pktgen.c net: core: use kernel's converter from hex to bin 2010-09-21 18:04:45 -07:00
request_sock.c
rtnetlink.c net: copy_rtnl_link_stats64() simplification 2010-08-23 20:43:16 -07:00
scm.c
skbuff.c Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2010-09-09 22:27:33 -07:00
sock.c Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2010-09-27 01:03:03 -07:00
stream.c net: Fix the condition passed to sk_wait_event() 2010-10-03 20:41:32 -07:00
sysctl_net_core.c
timestamping.c
user_dma.c
utils.c net: return operator cleanup 2010-09-23 14:33:39 -07:00