linux/mm
Vlastimil Babka a9f2a846f0 mm, slab: reschedule cache_reap() on the same CPU
cache_reap() is initially scheduled in start_cpu_timer() via
schedule_delayed_work_on(). But then the next iterations are scheduled
via schedule_delayed_work(), i.e. using WORK_CPU_UNBOUND.

Thus since commit ef55718044 ("workqueue: schedule WORK_CPU_UNBOUND
work on wq_unbound_cpumask CPUs") there is no guarantee the future
iterations will run on the originally intended cpu, although it's still
preferred.  I was able to demonstrate this with
/sys/module/workqueue/parameters/debug_force_rr_cpu.  IIUC, it may also
happen due to migrating timers in nohz context.  As a result, some cpu's
would be calling cache_reap() more frequently and others never.

This patch uses schedule_delayed_work_on() with the current cpu when
scheduling the next iteration.

Link: http://lkml.kernel.org/r/20180411070007.32225-1-vbabka@suse.cz
Fixes: ef55718044 ("workqueue: schedule WORK_CPU_UNBOUND work on wq_unbound_cpumask CPUs")
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Pekka Enberg <penberg@kernel.org>
Acked-by: Christoph Lameter <cl@linux.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Stephen Boyd <sboyd@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-04-13 17:10:27 -07:00
..
kasan slab, slub: skip unnecessary kasan_cache_shutdown() 2018-04-05 21:36:24 -07:00
backing-dev.c mm/vmscan: don't mess with pgdat->flags in memcg reclaim 2018-04-11 10:28:30 -07:00
balloon_compaction.c
bootmem.c
cleancache.c
cma_debug.c
cma.c mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE 2018-04-11 10:28:32 -07:00
cma.h
compaction.c mm/cma: remove ALLOC_CMA 2018-04-11 10:28:32 -07:00
debug_page_ref.c
debug.c
dmapool.c
early_ioremap.c
fadvise.c
failslab.c mm: make should_failslab always available for fault injection 2018-04-05 21:36:26 -07:00
filemap.c mm/filemap.c: provide dummy filemap_page_mkwrite() for NOMMU 2018-04-13 17:10:27 -07:00
frame_vector.c
frontswap.c
gup_benchmark.c mm/gup_benchmark: handle gup failures 2018-04-13 17:10:27 -07:00
gup.c mm/gup.c: document return value 2018-04-13 17:10:27 -07:00
highmem.c
hmm.c mm/hmm.c: remove superfluous RCU protection around radix tree lookup 2018-04-11 10:28:31 -07:00
huge_memory.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
hugetlb_cgroup.c
hugetlb.c mm, hugetlbfs: introduce ->pagesize() to vm_operations_struct 2018-04-05 21:36:26 -07:00
hwpoison-inject.c
init-mm.c
internal.h mm/cma: remove ALLOC_CMA 2018-04-11 10:28:32 -07:00
interval_tree.c
Kconfig
Kconfig.debug
khugepaged.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
kmemleak-test.c
kmemleak.c mm: kernel-doc: add missing parameter descriptions 2018-04-05 21:36:27 -07:00
ksm.c mm/ksm.c: fix inconsistent accounting of zero pages 2018-04-11 10:28:31 -07:00
list_lru.c mm: make counting of list_lru_one::nr_items lockless 2018-04-05 21:36:27 -07:00
maccess.c
madvise.c
Makefile mm/swap_slots.c: use conditional compilation 2018-04-05 21:36:24 -07:00
memblock.c powerpc updates for 4.17 2018-04-07 12:08:19 -07:00
memcontrol.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
memory_hotplug.c mm: unclutter THP migration 2018-04-11 10:28:32 -07:00
memory-failure.c mm, migrate: remove reason argument from new_page_t 2018-04-11 10:28:32 -07:00
memory.c mm: swap: unify cluster-based and vma-based swap readahead 2018-04-05 21:36:25 -07:00
mempolicy.c mm: unclutter THP migration 2018-04-11 10:28:32 -07:00
mempool.c
memtest.c
migrate.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
mincore.c
mlock.c
mm_init.c
mmap.c mm: introduce MAP_FIXED_NOREPLACE 2018-04-11 10:28:38 -07:00
mmu_context.c
mmu_notifier.c
mmzone.c
mprotect.c sched/numa: avoid trapping faults and attempting migration of file-backed dirty pages 2018-04-11 10:28:31 -07:00
mremap.c
msync.c
nobootmem.c
nommu.c mm/nommu: remove description of alloc_vm_area 2018-04-05 21:36:26 -07:00
oom_kill.c mm,oom_reaper: check for MMF_OOM_SKIP before complaining 2018-04-05 21:36:27 -07:00
page_alloc.c xen, mm: allow deferred page initialization for xen pv domains 2018-04-11 10:28:38 -07:00
page_counter.c
page_ext.c
page_idle.c mm: thp: fix potential clearing to referenced flag in page_idle_clear_pte_refs_one() 2018-04-05 21:36:25 -07:00
page_io.c
page_isolation.c mm, migrate: remove reason argument from new_page_t 2018-04-11 10:28:32 -07:00
page_owner.c mm/page_owner.c: make early_page_owner_param() __init 2018-04-05 21:36:26 -07:00
page_poison.c mm/page_poison.c: make early_page_poison_param() __init 2018-04-05 21:36:26 -07:00
page_vma_mapped.c
page-writeback.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
pagewalk.c mm: kernel-doc: add missing parameter descriptions 2018-04-05 21:36:27 -07:00
percpu-internal.h
percpu-km.c
percpu-stats.c mm: reuse DEFINE_SHOW_ATTRIBUTE() macro 2018-04-05 21:36:25 -07:00
percpu-vm.c
percpu.c
pgtable-generic.c
process_vm_access.c
quicklist.c
readahead.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
rmap.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
rodata_test.c
shmem.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
slab_common.c mm: make should_failslab always available for fault injection 2018-04-05 21:36:26 -07:00
slab.c mm, slab: reschedule cache_reap() on the same CPU 2018-04-13 17:10:27 -07:00
slab.h slab, slub: skip unnecessary kasan_cache_shutdown() 2018-04-05 21:36:24 -07:00
slob.c
slub.c kasan, slub: fix handling of kasan_slab_free hook 2018-04-11 10:28:32 -07:00
sparse-vmemmap.c
sparse.c mm/memory_hotplug: optimize memory hotplug 2018-04-05 21:36:25 -07:00
swap_cgroup.c
swap_slots.c mm/swap_slots.c: use conditional compilation 2018-04-05 21:36:24 -07:00
swap_state.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
swap.c mm/swap.c: remove @cold parameter description for release_pages() 2018-04-05 21:36:26 -07:00
swapfile.c mm/swapfile.c: make pointer swap_avail_heads static 2018-04-11 10:28:32 -07:00
truncate.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
usercopy.c
userfaultfd.c
util.c mm/gup.c: document return value 2018-04-13 17:10:27 -07:00
vmacache.c
vmalloc.c
vmpressure.c
vmscan.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
vmstat.c mm: introduce NR_INDIRECTLY_RECLAIMABLE_BYTES 2018-04-11 10:28:29 -07:00
workingset.c page cache: use xa_lock 2018-04-11 10:28:39 -07:00
z3fold.c mm/z3fold.c: use gfpflags_allow_blocking 2018-04-11 10:28:31 -07:00
zbud.c
zpool.c
zsmalloc.c mm: kernel-doc: add missing parameter descriptions 2018-04-05 21:36:27 -07:00
zswap.c