linux/net/sched
Eric Dumazet ca6fb06518 tcp: attach SYNACK messages to request sockets instead of listener
If a listen backlog is very big (to avoid syncookies), then
the listener sk->sk_wmem_alloc is the main source of false
sharing, as we need to touch it twice per SYNACK re-transmit
and TX completion.

(One SYN packet takes listener lock once, but up to 6 SYNACK
are generated)

By attaching the skb to the request socket, we remove this
source of contention.

Tested:

 listen(fd, 10485760); // single listener (no SO_REUSEPORT)
 16 RX/TX queue NIC
 Sustain a SYNFLOOD attack of ~320,000 SYN per second,
 Sending ~1,400,000 SYNACK per second.
 Perf profiles now show listener spinlock being next bottleneck.

    20.29%  [kernel]  [k] queued_spin_lock_slowpath
    10.06%  [kernel]  [k] __inet_lookup_established
     5.12%  [kernel]  [k] reqsk_timer_handler
     3.22%  [kernel]  [k] get_next_timer_interrupt
     3.00%  [kernel]  [k] tcp_make_synack
     2.77%  [kernel]  [k] ipt_do_table
     2.70%  [kernel]  [k] run_timer_softirq
     2.50%  [kernel]  [k] ip_finish_output
     2.04%  [kernel]  [k] cascade

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-10-03 04:32:43 -07:00
..
act_api.c net_sched: make tcf_hash_destroy() static 2015-08-26 11:01:44 -07:00
act_bpf.c bpf: add bpf_redirect() helper 2015-09-17 21:09:07 -07:00
act_connmark.c netfilter: nf_conntrack: Add a struct net parameter to l4_pkt_to_tuple 2015-09-18 22:00:04 +02:00
act_csum.c net: sched: add percpu stats to actions 2015-07-08 13:50:41 -07:00
act_gact.c net_sched: act_gact: remove spinlock in fast path 2015-07-08 13:50:42 -07:00
act_ipt.c netfilter: x_tables: Pass struct net in xt_action_param 2015-09-18 21:58:14 +02:00
act_mirred.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-08-13 16:23:11 -07:00
act_nat.c net: Change pseudohdr argument of inet_proto_csum_replace* to be a bool 2015-08-17 21:33:06 -07:00
act_pedit.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-07-31 23:52:20 -07:00
act_police.c
act_simple.c net: sched: add percpu stats to actions 2015-07-08 13:50:41 -07:00
act_skbedit.c net: sched: add percpu stats to actions 2015-07-08 13:50:41 -07:00
act_vlan.c net: sched: add percpu stats to actions 2015-07-08 13:50:41 -07:00
cls_api.c net: sched: fix call_rcu() race on classifier module unloads 2015-05-21 18:48:18 -04:00
cls_basic.c
cls_bpf.c cls_bpf: further limit exec opcodes subset 2015-09-23 14:29:02 -07:00
cls_cgroup.c cls_cgroup: factor out classid retrieval 2015-07-20 12:41:30 -07:00
cls_flow.c flow_dissector: Add flags argument to skb_flow_dissector functions 2015-09-01 15:06:22 -07:00
cls_flower.c flow_dissector: Add flags argument to skb_flow_dissector functions 2015-09-01 15:06:22 -07:00
cls_fw.c net: revert "net_sched: move tp->root allocation into fw_init()" 2015-09-24 14:33:30 -07:00
cls_route.c
cls_rsvp6.c
cls_rsvp.c
cls_rsvp.h net_sched: convert rsvp to call tcf_exts_destroy from rcu callback 2015-08-26 11:01:45 -07:00
cls_tcindex.c net_sched: convert tcindex to call tcf_exts_destroy from rcu callback 2015-08-26 11:01:44 -07:00
cls_u32.c cls_u32: complete the check for non-forced case in u32_destroy() 2015-08-25 17:02:48 -07:00
em_canid.c
em_cmp.c
em_ipset.c netfilter: x_tables: Pass struct net in xt_action_param 2015-09-18 21:58:14 +02:00
em_meta.c
em_nbyte.c
em_text.c
em_u32.c
ematch.c
Kconfig
Makefile
sch_api.c net: sched: don't break line in tc_classify loop notification 2015-08-28 14:01:15 -07:00
sch_atm.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_blackhole.c
sch_cbq.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_choke.c flow_dissector: Add flags argument to skb_flow_dissector functions 2015-09-01 15:06:22 -07:00
sch_codel.c
sch_drr.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_dsmark.c sch_dsmark: improve memory locality 2015-09-17 22:37:19 -07:00
sch_fifo.c net: sched: drop all special handling of tx_queue_len == 0 2015-08-18 11:55:08 -07:00
sch_fq_codel.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_fq.c tcp: attach SYNACK messages to request sockets instead of listener 2015-10-03 04:32:43 -07:00
sch_generic.c net: sched: simplify attach_one_default_qdisc() 2015-08-27 17:14:30 -07:00
sch_gred.c net: sched: drop all special handling of tx_queue_len == 0 2015-08-18 11:55:08 -07:00
sch_hfsc.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_hhf.c
sch_htb.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_ingress.c
sch_mq.c
sch_mqprio.c
sch_multiq.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_netem.c
sch_pie.c
sch_plug.c net: sched: drop all special handling of tx_queue_len == 0 2015-08-18 11:55:08 -07:00
sch_prio.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_qfq.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_red.c
sch_sfb.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_sfq.c net: sched: consolidate tc_classify{,_compat} 2015-08-27 14:18:48 -07:00
sch_tbf.c
sch_teql.c