linux/mm
Tejun Heo ca23e405e0 vmalloc: implement pcpu_get_vm_areas()
To directly use spread NUMA memories for percpu units, percpu
allocator will be updated to allow sparsely mapping units in a chunk.
As the distances between units can be very large, this makes
allocating single vmap area for each chunk undesirable.  This patch
implements pcpu_get_vm_areas() and pcpu_free_vm_areas() which
allocates and frees sparse congruent vmap areas.

pcpu_get_vm_areas() take @offsets and @sizes array which define
distances and sizes of vmap areas.  It scans down from the top of
vmalloc area looking for the top-most address which can accomodate all
the areas.  The top-down scan is to avoid interacting with regular
vmallocs which can push up these congruent areas up little by little
ending up wasting address space and page table.

To speed up top-down scan, the highest possible address hint is
maintained.  Although the scan is linear from the hint, given the
usual large holes between memory addresses between NUMA nodes, the
scanning is highly likely to finish after finding the first hole for
the last unit which is scanned first.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Nick Piggin <npiggin@suse.de>
2009-08-14 15:00:52 +09:00
..
allocpercpu.c percpu: use dynamic percpu allocator as the default percpu allocator 2009-06-24 15:13:35 +09:00
backing-dev.c Fix congestion_wait() sync/async vs read/write confusion 2009-07-10 20:31:53 +02:00
bootmem.c kmemleak: Add callbacks to the bootmem allocator 2009-07-08 14:25:14 +01:00
bounce.c
debug-pagealloc.c
dmapool.c dmapools: protect page_list walk in show_pools() 2009-06-30 18:56:00 -07:00
fadvise.c
failslab.c
filemap_xip.c
filemap.c mm: mark page accessed before we write_end() 2009-07-06 13:57:03 -07:00
fremap.c
highmem.c
hugetlb.c hugetlbfs: fix i_blocks accounting 2009-07-29 19:10:35 -07:00
init-mm.c
internal.h vmscan: do not unconditionally treat zones that fail zone_reclaim() as full 2009-06-16 19:47:45 -07:00
Kconfig Merge branch 'akpm' 2009-06-16 19:50:13 -07:00
Kconfig.debug
kmemcheck.c
kmemleak-test.c percpu: clean up percpu variable definitions 2009-06-24 15:13:48 +09:00
kmemleak.c kmemleak: Protect the seq start/next/stop sequence by rcu_read_lock() 2009-07-29 12:34:58 -07:00
maccess.c
madvise.c mm: madvise(): correct return code 2009-06-16 19:47:40 -07:00
Makefile percpu: use dynamic percpu allocator as the default percpu allocator 2009-06-24 15:13:35 +09:00
memcontrol.c cgroup avoid permanent sleep at rmdir 2009-07-29 19:10:35 -07:00
memory_hotplug.c page-allocator: reset wmark_min and inactive ratio of zone when hotplug happens 2009-06-16 19:47:42 -07:00
memory.c mm: Pass virtual address to [__]p{te,ud,md}_free_tlb() 2009-07-27 12:10:38 -07:00
mempolicy.c mm: make set_mempolicy(MPOL_INTERLEAV) N_HIGH_MEMORY aware 2009-08-07 10:39:55 -07:00
mempool.c mempool.c: clean up type-casting 2009-08-10 08:31:16 -07:00
migrate.c migration: only migrate_prep() once per move_pages() 2009-06-16 19:47:41 -07:00
mincore.c
mlock.c mm: remove CONFIG_UNEVICTABLE_LRU config option 2009-06-16 19:47:42 -07:00
mm_init.c
mmap.c
mmu_notifier.c
mmzone.c
mprotect.c
mremap.c
msync.c
nommu.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/lethal/sh-2.6 2009-07-01 11:46:30 -07:00
oom_kill.c oom: only oom kill exiting tasks with attached memory 2009-06-16 19:47:45 -07:00
page_alloc.c page-allocator: allow too high-order warning messages to be suppressed with __GFP_NOWARN 2009-07-29 19:10:35 -07:00
page_cgroup.c memcg: remove some redundant checks 2009-06-18 13:03:47 -07:00
page_io.c mm: remove file argument from swap_readpage() 2009-06-16 19:47:44 -07:00
page_isolation.c
page-writeback.c Merge branch 'percpu-for-linus' into percpu-for-next 2009-08-14 14:45:31 +09:00
pagewalk.c
pdflush.c
percpu.c percpu: add chunk->base_addr 2009-08-14 15:00:51 +09:00
prio_tree.c
quicklist.c percpu: cleanup percpu array definitions 2009-06-24 15:13:45 +09:00
readahead.c
rmap.c memcg: add file-based RSS accounting 2009-06-18 13:03:47 -07:00
shmem_acl.c switch shmem to inode->i_acl 2009-06-24 08:17:06 -04:00
shmem.c Get "no acls for this inode" right, fix shmem breakage 2009-06-24 16:58:48 -04:00
slab.c SLAB: Fix lockdep annotations 2009-06-29 09:57:10 +03:00
slob.c fix RCU-callback-after-kmem_cache_destroy problem in sl[aou]b 2009-06-26 12:10:47 +03:00
slub.c Merge branch 'percpu-for-linus' into percpu-for-next 2009-08-14 14:45:31 +09:00
sparse-vmemmap.c
sparse.c
swap_state.c mm: remove file argument from swap_readpage() 2009-06-16 19:47:44 -07:00
swap.c
swapfile.c PM / Hibernate: Replace bdget call with simple atomic_inc of i_count 2009-07-29 21:07:55 +02:00
thrash.c mm: pass mm to grab_swap_token 2009-06-23 12:50:05 -07:00
truncate.c mm: remove __invalidate_mapping_pages variant 2009-06-16 19:47:43 -07:00
util.c Merge branches 'slab/documentation', 'slab/fixes', 'slob/cleanups' and 'slub/fixes' into for-linus 2009-06-17 08:30:15 +03:00
vmalloc.c vmalloc: implement pcpu_get_vm_areas() 2009-08-14 15:00:52 +09:00
vmscan.c Fix congestion_wait() sync/async vs read/write confusion 2009-07-10 20:31:53 +02:00
vmstat.c vmscan: count the number of times zone_reclaim() scans and fails 2009-06-16 19:47:46 -07:00