Jiri Kosina 4a6cc4bd32 sched: move rq_weight data array out of .percpu ...

Commit 34d76c41 introduced percpu array update_shares_data, size of which
being proportional to NR_CPUS. Unfortunately this blows up ia64 for large
NR_CPUS configuration, as ia64 allows only 64k for .percpu section.

Fix this by allocating this array dynamically and keep only pointer to it
percpu.

The per-cpu handling doesn't impose significant performance penalty on
potentially contented path in tg_shares_up().

...
ffffffff8104337c:       65 48 8b 14 25 20 cd    mov    %gs:0xcd20,%rdx
ffffffff81043383:       00 00
ffffffff81043385:       48 c7 c0 00 e1 00 00    mov    $0xe100,%rax
ffffffff8104338c:       48 c7 45 a0 00 00 00    movq   $0x0,-0x60(%rbp)
ffffffff81043393:       00
ffffffff81043394:       48 c7 45 a8 00 00 00    movq   $0x0,-0x58(%rbp)
ffffffff8104339b:       00
ffffffff8104339c:       48 01 d0                add    %rdx,%rax
ffffffff8104339f:       49 8d 94 24 08 01 00    lea    0x108(%r12),%rdx
ffffffff810433a6:       00
ffffffff810433a7:       b9 ff ff ff ff          mov    $0xffffffff,%ecx
ffffffff810433ac:       48 89 45 b0             mov    %rax,-0x50(%rbp)
ffffffff810433b0:       bb 00 04 00 00          mov    $0x400,%ebx
ffffffff810433b5:       48 89 55 c0             mov    %rdx,-0x40(%rbp)
...

After:

...
ffffffff8104337c:       65 8b 04 25 28 cd 00    mov    %gs:0xcd28,%eax
ffffffff81043383:       00
ffffffff81043384:       48 98                   cltq
ffffffff81043386:       49 8d bc 24 08 01 00    lea    0x108(%r12),%rdi
ffffffff8104338d:       00
ffffffff8104338e:       48 8b 15 d3 7f 76 00    mov    0x767fd3(%rip),%rdx        # ffffffff817ab368 <update_shares_data>
ffffffff81043395:       48 8b 34 c5 00 ee 6d    mov    -0x7e921200(,%rax,8),%rsi
ffffffff8104339c:       81
ffffffff8104339d:       48 c7 45 a0 00 00 00    movq   $0x0,-0x60(%rbp)
ffffffff810433a4:       00
ffffffff810433a5:       b9 ff ff ff ff          mov    $0xffffffff,%ecx
ffffffff810433aa:       48 89 7d c0             mov    %rdi,-0x40(%rbp)
ffffffff810433ae:       48 c7 45 a8 00 00 00    movq   $0x0,-0x58(%rbp)
ffffffff810433b5:       00
ffffffff810433b6:       bb 00 04 00 00          mov    $0x400,%ebx
ffffffff810433bb:       48 01 f2                add    %rsi,%rdx
ffffffff810433be:       48 89 55 b0             mov    %rdx,-0x50(%rbp)
...

Signed-off-by: Jiri Kosina <jkosina@suse.cz>
Acked-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Tejun Heo <tj@kernel.org>

2009-10-29 00:26:00 +09:00

..

gcov

microblaze: Enable GCOV_PROFILE_ALL

2009-09-21 14:29:21 +02:00

irq

Merge branch 'irq-threaded-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-09-11 13:21:31 -07:00

power

headers: utsname.h redux

2009-09-23 18:13:10 -07:00

time

Merge branch 'timers-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-09-26 10:14:41 -07:00

trace

Merge branch 'tracing-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-09-26 10:13:54 -07:00

.gitignore

…

acct.c

bsdacct: switch credentials for writing to the accounting file

2009-08-24 11:33:40 +10:00

async.c

async: Fix lack of boot-time console due to insufficient synchronization

2009-06-08 12:31:53 -07:00

audit_tree.c

Fix rule eviction order for AUDIT_DIR

2009-06-24 00:02:38 -04:00

audit_watch.c

Audit: reorganize struct audit_watch to save 8 bytes

2009-09-24 03:50:25 -04:00

audit.c

Audit: send signal info if selinux is disabled

2009-09-24 03:50:26 -04:00

audit.h

Fix rule eviction order for AUDIT_DIR

2009-06-24 00:02:38 -04:00

auditfilter.c

Audit: clean up all op= output to include string quoting

2009-06-24 00:00:52 -04:00

auditsc.c

Audit: rearrange audit_context to save 16 bytes per struct

2009-09-24 03:50:26 -04:00

backtracetest.c

…

bounds.c

…

capability.c

…

cgroup_freezer.c

cgroups: let ss->can_attach and ss->attach do whole threadgroups at a time

2009-09-24 07:20:58 -07:00

cgroup.c

cgroups: let ss->can_attach and ss->attach do whole threadgroups at a time

2009-09-24 07:20:58 -07:00

compat.c

signals: implement sys_rt_tgsigqueueinfo

2009-04-30 19:24:24 +02:00

configs.c

…

cpu.c

Merge branch 'x86-pat-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-09-15 09:19:38 -07:00

cpuset.c

cgroups: let ss->can_attach and ss->attach do whole threadgroups at a time

2009-09-24 07:20:58 -07:00

cred-internals.h

…

cred.c

creds_are_invalid() needs to be exported for use by modules:

2009-09-23 11:02:26 -07:00

delayacct.c

headers: taskstats_kern.h trim

2009-09-18 09:48:52 -07:00

dma.c

…

exec_domain.c

Get rid of indirect include of fs_struct.h

2009-03-31 23:00:27 -04:00

exit.c

task_struct cleanup: move binfmt field to mm_struct

2009-09-24 07:21:05 -07:00

extable.c

Merge branch 'tracing-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-04-05 11:04:19 -07:00

fork.c

task_struct cleanup: move binfmt field to mm_struct

2009-09-24 07:21:05 -07:00

freezer.c

sched: fix nr_uninterruptible accounting of frozen tasks really

2009-07-18 14:19:53 +02:00

futex_compat.c

futex: Fix compat_futex to be same as futex for REQUEUE_PI

2009-08-10 15:41:12 +02:00

futex.c

futex: Add memory barrier commentary to futex_wait_queue_me()

2009-09-24 22:30:10 +02:00

groups.c

groups: move code to kernel/groups.c

2009-06-16 19:47:48 -07:00

hrtimer.c

Merge branch 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-09-27 10:39:04 -07:00

hung_task.c

sysctl: remove "struct file *" argument of ->proc_handler

2009-09-24 07:21:04 -07:00

itimer.c

itimers: Add tracepoints for itimer

2009-08-29 14:10:07 +02:00

kallsyms.c

kallsyms: use new arch_is_kernel_text()

2009-09-23 07:39:30 -07:00

Kconfig.freezer

…

Kconfig.hz

…

Kconfig.preempt

…

kexec.c

kexec: fix omitting offset in extended crashkernel syntax

2009-07-29 19:10:34 -07:00

kfifo.c

kfifo: Use "const" definitions

2009-09-19 13:13:17 -07:00

kgdb.c

sysrq, intel_fb: fix sysrq g collision

2009-05-15 07:56:24 -05:00

kmod.c

Revert "kmod: fix race in usermodehelper code"

2009-09-23 18:12:10 -07:00

kprobes.c

seq_file: constify seq_operations

2009-09-23 07:39:29 -07:00

ksysfs.c

…

kthread.c

sched: Keep kthreads at default priority

2009-09-09 17:30:06 +02:00

latencytop.c

…

lockdep_internals.h

lockdep: BFS cleanup

2009-07-24 10:53:29 +02:00

lockdep_proc.c

seq_file: constify seq_operations

2009-09-23 07:39:29 -07:00

lockdep_states.h

…

lockdep.c

lockdep: use new arch_is_kernel_data()

2009-09-23 07:39:30 -07:00

Makefile

cgroups: move the cgroup debug subsys into cgroup.c to access internal state

2009-09-24 07:20:57 -07:00

module.c

Merge branch 'tracing-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-09-26 10:13:54 -07:00

mutex-debug.c

…

mutex-debug.h

…

mutex.c

Merge branch 'linus' into perfcounters/core

2009-06-11 17:55:42 +02:00

mutex.h

…

notifier.c

…

ns_cgroup.c

cgroups: let ss->can_attach and ss->attach do whole threadgroups at a time

2009-09-24 07:20:58 -07:00

nsproxy.c

nsproxy: extract create_nsproxy()

2009-06-18 13:03:56 -07:00

panic.c

trivial: Correct print_tainted routine name in comment

2009-09-21 15:14:53 +02:00

params.c

param: allow whitespace as kernel parameter separator

2009-09-25 00:32:58 +09:30

perf_event.c

const: mark struct vm_struct_operations

2009-09-27 11:39:25 -07:00

pid_namespace.c

pidns: deny CLONE_PARENT|CLONE_NEWPID combination

2009-09-24 07:21:04 -07:00

pid.c

mm: also use alloc_large_system_hash() for the PID hash table

2009-09-22 07:17:38 -07:00

pm_qos_params.c

…

posix-cpu-timers.c

itimers: Add tracepoints for itimer

2009-08-29 14:10:07 +02:00

posix-timers.c

time: Introduce CLOCK_REALTIME_COARSE

2009-08-21 21:43:46 +02:00

printk.c

printk: add printk_delay to make messages readable for some scenarios

2009-09-23 07:39:28 -07:00

profile.c

kernel/profile.c: Switch /proc/irq/prof_cpu_mask to seq_file

2009-09-20 20:15:40 +02:00

ptrace.c

ptrace: __ptrace_detach: do __wake_up_parent() if we reap the tracee

2009-09-24 07:20:59 -07:00

rcupdate.c

rcu: Fix whitespace inconsistencies

2009-09-19 08:53:22 +02:00

rcutorture.c

rcu: Fix whitespace inconsistencies

2009-09-19 08:53:22 +02:00

rcutree_plugin.h

rcu: Fix whitespace inconsistencies

2009-09-19 08:53:22 +02:00

rcutree_trace.c

rcu: Fix whitespace inconsistencies

2009-09-19 08:53:22 +02:00

rcutree.c

rcu: Fix whitespace inconsistencies

2009-09-19 08:53:22 +02:00

rcutree.h

rcu: Fix whitespace inconsistencies

2009-09-19 08:53:22 +02:00

relay.c

const: mark struct vm_struct_operations

2009-09-27 11:39:25 -07:00

res_counter.c

memory controller: soft limit organize cgroups

2009-09-24 07:20:59 -07:00

resource.c

walk system ram range

2009-09-23 07:39:41 -07:00

rtmutex_common.h

rt_mutex: add proxy lock routines

2009-04-06 11:14:02 +02:00

rtmutex-debug.c

…

rtmutex-debug.h

…

rtmutex-tester.c

…

rtmutex.c

rtmutex: Avoid deadlock in rt_mutex_start_proxy_lock()

2009-08-06 05:50:21 +02:00

rtmutex.h

…

rwsem.c

…

sched_clock.c

sched_clock: Make it NMI safe

2009-09-18 20:47:30 +02:00

sched_cpupri.c

sched: Add new prio to cpupri before removing old prio

2009-08-02 14:26:09 +02:00

sched_cpupri.h

cpumask: remove cpumask_t from core

2009-03-30 22:05:17 +10:30

sched_debug.c

sched: Add new wakeup preemption mode: WAKEUP_RUNNING

2009-09-17 10:17:25 +02:00

sched_fair.c

sysctl: remove "struct file *" argument of ->proc_handler

2009-09-24 07:21:04 -07:00

sched_features.h

sched: Add new wakeup preemption mode: WAKEUP_RUNNING

2009-09-17 10:17:25 +02:00

sched_idletask.c

sched: Simplify sys_sched_rr_get_interval() system call

2009-09-21 09:53:55 +02:00

sched_rt.c

sched: Simplify sys_sched_rr_get_interval() system call

2009-09-21 09:53:55 +02:00

sched_stats.h

sched: remove unused fields from struct rq

2009-03-24 23:16:51 +01:00

sched.c

sched: move rq_weight data array out of .percpu

2009-10-29 00:26:00 +09:00

seccomp.c

x86-64: seccomp: fix 32/64 syscall hole

2009-03-02 15:41:30 -08:00

semaphore.c

…

signal.c

signals: inline __fatal_signal_pending

2009-09-24 07:21:01 -07:00

slow-work.c

sysctl: remove "struct file *" argument of ->proc_handler

2009-09-24 07:21:04 -07:00

smp.c

cpumask: remove arch_send_call_function_ipi

2009-09-24 09:34:47 +09:30

softirq.c

softirq: add BLOCK_IOPOLL to softirq_to_name

2009-09-17 15:53:44 -04:00

softlockup.c

sysctl: remove "struct file *" argument of ->proc_handler

2009-09-24 07:21:04 -07:00

spinlock.c

locking: Allow arch-inlined spinlocks

2009-08-31 18:08:50 +02:00

srcu.c

…

stacktrace.c

…

stop_machine.c

cpumask: remove cpumask_t from core

2009-03-30 22:05:17 +10:30

sys_ni.c

Merge branch 'master' of /home/davem/src/GIT/linux-2.6/

2009-09-24 15:13:11 -07:00

sys.c

Merge branch 'hwpoison' of git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6

2009-09-24 07:53:22 -07:00

sysctl_check.c

…

sysctl.c

Merge branch 'hwpoison' of git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6

2009-09-24 07:53:22 -07:00

taskstats.c

genetlink: make netns aware

2009-07-12 14:03:27 -07:00

test_kprobes.c

…

time.c

time: Prevent 32 bit overflow with set_normalized_timespec()

2009-09-15 10:17:30 +02:00

timeconst.pl

…

timer.c

Merge branch 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-09-23 09:46:15 -07:00

tracepoint.c

trivial: fix typo "to to" in multiple files

2009-09-21 15:14:55 +02:00

tsacct.c

Fix fixpoint divide exception in acct_update_integrals

2009-03-09 08:13:35 -07:00

uid16.c

headers: utsname.h redux

2009-09-23 18:13:10 -07:00

up.c

…

user_namespace.c

Fix recursive lock in free_uid()/free_user_ns()

2009-02-27 16:26:21 -08:00

user.c

sched: delayed cleanup of user_struct

2009-06-15 21:30:23 -07:00

utsname_sysctl.c

sysctl: remove "struct file *" argument of ->proc_handler

2009-09-24 07:21:04 -07:00

utsname.c

utsns: extract creeate_uts_ns()

2009-06-18 13:03:55 -07:00

wait.c

locking, sched: Give waitqueue spinlocks their own lockdep classes

2009-08-10 14:43:09 +02:00

workqueue.c

Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

2009-09-11 13:23:18 -07:00