linux/mm
Christoph Lameter 62bc62a873 page allocator: use a pre-calculated value instead of num_online_nodes() in fast paths
num_online_nodes() is called in a number of places but most often by the
page allocator when deciding whether the zonelist needs to be filtered
based on cpusets or the zonelist cache.  This is actually a heavy function
and touches a number of cache lines.

This patch stores the number of online nodes at boot time and updates the
value when nodes get onlined and offlined.  The value is then used in a
number of important paths in place of num_online_nodes().

[rientjes@google.com: do not override definition of node_set_online() with macro]
Signed-off-by: Christoph Lameter <cl@linux-foundation.org>
Signed-off-by: Mel Gorman <mel@csn.ul.ie>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-06-16 19:47:35 -07:00
..
allocpercpu.c percpu: __percpu_depopulate_mask can take a const mask 2009-04-06 13:44:15 -07:00
backing-dev.c block: change the request allocation/congestion logic to be sync/async based 2009-04-06 08:04:53 -07:00
bootmem.c bootmem: fix slab fallback on numa 2009-06-11 19:15:54 +03:00
bounce.c Merge branch 'for-2.6.31' of git://git.kernel.dk/linux-2.6-block 2009-06-11 11:10:35 -07:00
debug-pagealloc.c generic debug pagealloc 2009-04-01 08:59:13 -07:00
dmapool.c
fadvise.c readahead: move max_sane_readahead() calls into force_page_cache_readahead() 2009-06-16 19:47:28 -07:00
failslab.c kmemtrace, mm: fix slab.h dependency problem in mm/failslab.c 2009-04-03 12:23:01 +02:00
filemap_xip.c mm: do_xip_mapping_read: fix length calculation 2009-04-02 19:04:49 -07:00
filemap.c page allocator: do not check NUMA node ID when the caller knows the node is valid 2009-06-16 19:47:32 -07:00
fremap.c Do not account for the address space used by hugetlbfs using VM_ACCOUNT 2009-02-10 10:48:42 -08:00
highmem.c mm: introduce debug_kmap_atomic 2009-04-01 08:59:14 -07:00
hugetlb.c page allocator: use a pre-calculated value instead of num_online_nodes() in fast paths 2009-06-16 19:47:35 -07:00
init-mm.c mm: consolidate init_mm definition 2009-06-16 19:47:28 -07:00
internal.h page allocator: do not disable interrupts in free_page_mlock() 2009-06-16 19:47:34 -07:00
Kconfig security: use mmap_min_addr indepedently of security models 2009-06-04 12:07:48 +10:00
Kconfig.debug generic debug pagealloc: build fix 2009-04-02 19:04:48 -07:00
kmemleak-test.c kmemleak: Simple testing module for kmemleak 2009-06-11 17:04:19 +01:00
kmemleak.c kmemleak: Add the base support 2009-06-11 17:03:28 +01:00
maccess.c [S390] maccess: add weak attribute to probe_kernel_write 2009-06-12 10:27:37 +02:00
madvise.c readahead: move max_sane_readahead() calls into force_page_cache_readahead() 2009-06-16 19:47:28 -07:00
Makefile mm: consolidate init_mm definition 2009-06-16 19:47:28 -07:00
memcontrol.c memcg: fix build warning and avoid checking for mem != null again and again 2009-05-29 08:40:03 -07:00
memory_hotplug.c mm: remove GFP_HIGHUSER_PAGECACHE 2009-01-06 15:59:01 -08:00
memory.c mm: clean up get_user_pages_fast() documentation 2009-06-16 19:47:30 -07:00
mempolicy.c page allocator: do not check NUMA node ID when the caller knows the node is valid 2009-06-16 19:47:32 -07:00
mempool.c
migrate.c page allocator: do not check NUMA node ID when the caller knows the node is valid 2009-06-16 19:47:32 -07:00
mincore.c [CVE-2009-0029] System call wrappers part 14 2009-01-14 14:15:24 +01:00
mlock.c x86, bts, mm: clean up buffer allocation 2009-04-24 10:18:52 +02:00
mm_init.c
mmap.c Merge branch 'perfcounters-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-11 14:01:07 -07:00
mmu_notifier.c
mmzone.c [ARM] Double check memmap is actually valid with a memmap has unexpected holes V2 2009-05-18 11:22:24 +01:00
mprotect.c perf_counter: Add mmap event hooks to mprotect() 2009-06-08 23:10:43 +02:00
mremap.c [CVE-2009-0029] System call wrappers part 13 2009-01-14 14:15:23 +01:00
msync.c [CVE-2009-0029] System call wrappers part 13 2009-01-14 14:15:23 +01:00
nommu.c nommu: Provide mmap_min_addr definition. 2009-06-10 09:24:09 +10:00
oom_kill.c oom: fix possible oom_dump_tasks NULL pointer 2009-05-29 08:40:01 -07:00
page_alloc.c page allocator: use a pre-calculated value instead of num_online_nodes() in fast paths 2009-06-16 19:47:35 -07:00
page_cgroup.c memcg: fix page_cgroup fatal error in FLATMEM 2009-06-12 11:00:54 +03:00
page_io.c block: fix bad definition of BIO_RW_SYNC 2009-02-18 10:32:00 +01:00
page_isolation.c
page-writeback.c mm/page-writeback.c: dirty limit type should be unsigned long 2009-06-16 19:47:31 -07:00
pagewalk.c
pdflush.c Revert "mm: add /proc controls for pdflush threads" 2009-05-15 11:32:24 +02:00
percpu.c percpu: remove rbtree and use page->index instead 2009-04-08 18:31:31 +02:00
prio_tree.c
quicklist.c cpumask: replace node_to_cpumask with cpumask_of_node. 2009-03-13 14:49:46 +10:30
readahead.c readahead: introduce context readahead algorithm 2009-06-16 19:47:30 -07:00
rmap.c hugh: update email address 2009-05-21 13:14:32 -07:00
shmem_acl.c
shmem.c integrity: move ima_counts_get 2009-05-22 09:45:33 +10:00
slab.c page allocator: do not check NUMA node ID when the caller knows the node is valid 2009-06-16 19:47:32 -07:00
slob.c page allocator: do not check NUMA node ID when the caller knows the node is valid 2009-06-16 19:47:32 -07:00
slub.c page allocator: use a pre-calculated value instead of num_online_nodes() in fast paths 2009-06-16 19:47:35 -07:00
sparse-vmemmap.c
sparse.c mm: mminit_validate_memmodel_limits(): remove redundant test 2009-04-01 08:59:11 -07:00
swap_state.c memcg: fix deadlock between lock_page_cgroup and mapping tree_lock 2009-05-29 08:40:02 -07:00
swap.c mm: fix Committed_AS underflow on large NR_CPUS environment 2009-05-02 15:36:10 -07:00
swapfile.c PM/hibernate: fix "swap breaks after hibernation failures" 2009-02-21 14:17:17 -08:00
thrash.c
truncate.c memcg: fix deadlock between lock_page_cgroup and mapping tree_lock 2009-05-29 08:40:02 -07:00
util.c mm: clean up get_user_pages_fast() documentation 2009-06-16 19:47:30 -07:00
vmalloc.c Merge branch 'for-linus' of git://linux-arm.org/linux-2.6 2009-06-11 14:15:57 -07:00
vmscan.c page allocator: use allocation flags as an index to the zone watermark 2009-06-16 19:47:35 -07:00
vmstat.c page allocator: use allocation flags as an index to the zone watermark 2009-06-16 19:47:35 -07:00