linux/lib
Mel Gorman d0164adc89 mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd
__GFP_WAIT has been used to identify atomic context in callers that hold
spinlocks or are in interrupts.  They are expected to be high priority and
have access one of two watermarks lower than "min" which can be referred
to as the "atomic reserve".  __GFP_HIGH users get access to the first
lower watermark and can be called the "high priority reserve".

Over time, callers had a requirement to not block when fallback options
were available.  Some have abused __GFP_WAIT leading to a situation where
an optimisitic allocation with a fallback option can access atomic
reserves.

This patch uses __GFP_ATOMIC to identify callers that are truely atomic,
cannot sleep and have no alternative.  High priority users continue to use
__GFP_HIGH.  __GFP_DIRECT_RECLAIM identifies callers that can sleep and
are willing to enter direct reclaim.  __GFP_KSWAPD_RECLAIM to identify
callers that want to wake kswapd for background reclaim.  __GFP_WAIT is
redefined as a caller that is willing to enter direct reclaim and wake
kswapd for background reclaim.

This patch then converts a number of sites

o __GFP_ATOMIC is used by callers that are high priority and have memory
  pools for those requests. GFP_ATOMIC uses this flag.

o Callers that have a limited mempool to guarantee forward progress clear
  __GFP_DIRECT_RECLAIM but keep __GFP_KSWAPD_RECLAIM. bio allocations fall
  into this category where kswapd will still be woken but atomic reserves
  are not used as there is a one-entry mempool to guarantee progress.

o Callers that are checking if they are non-blocking should use the
  helper gfpflags_allow_blocking() where possible. This is because
  checking for __GFP_WAIT as was done historically now can trigger false
  positives. Some exceptions like dm-crypt.c exist where the code intent
  is clearer if __GFP_DIRECT_RECLAIM is used instead of the helper due to
  flag manipulations.

o Callers that built their own GFP flags instead of starting with GFP_KERNEL
  and friends now also need to specify __GFP_KSWAPD_RECLAIM.

The first key hazard to watch out for is callers that removed __GFP_WAIT
and was depending on access to atomic reserves for inconspicuous reasons.
In some cases it may be appropriate for them to use __GFP_HIGH.

The second key hazard is callers that assembled their own combination of
GFP flags instead of starting with something like GFP_KERNEL.  They may
now wish to specify __GFP_KSWAPD_RECLAIM.  It's almost certainly harmless
if it's missed in most cases as other activity will wake kswapd.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Vitaly Wool <vitalywool@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-11-06 17:50:42 -08:00
..
842 crypto: 842 - Add CRC and validation support 2015-10-14 22:23:17 +08:00
fonts fonts: Add 6x10 font 2014-10-09 11:35:48 +03:00
lz4 lz4: fix system halt at boot kernel on x86_64 2015-05-24 11:56:29 -07:00
lzo lzo: check for length overrun in variable length encoding. 2014-09-28 11:08:01 +02:00
mpi lib/mpi: fix off by one in mpi_read_raw_from_sgl 2015-10-20 22:10:47 +08:00
raid6 md/raid6: delta syndrome for ARM NEON 2015-08-31 19:29:05 +02:00
reed_solomon
xz lib/xz: enable all filters by default in Kconfig 2014-06-04 16:54:18 -07:00
zlib_deflate zlib_deflate/deftree: remove bi_reverse() 2015-09-10 13:29:01 -07:00
zlib_inflate zlib: clean up some dead code 2014-08-06 18:01:24 -07:00
.gitignore X.509: Implement simple static OID registry 2012-10-08 13:50:18 +10:30
argv_split.c argv_split(): teach it to handle mutable strings 2013-04-29 18:28:19 -07:00
asn1_decoder.c ASN.1: Handle 'ANY OPTIONAL' in grammar 2015-08-05 13:38:07 +01:00
assoc_array.c assoc_array: Include rcupdate.h for call_rcu() definition 2015-01-07 16:08:41 +00:00
atomic64_test.c atomic: Add simple atomic_t tests 2015-07-27 14:06:24 +02:00
atomic64.c atomic: Provide atomic_{or,xor,and} 2015-07-27 14:06:24 +02:00
audit.c syscalls: implement execveat() system call 2014-12-13 12:42:51 -08:00
bcd.c usb/core: use bin2bcd() for bcdDevice in RH 2012-09-10 11:13:16 -07:00
bch.c
bitmap.c lib/bitmap.c: bitmap_parselist can accept string with whitespaces on head or tail 2015-09-10 13:29:01 -07:00
bitrev.c ARM: 8187/1: add CONFIG_HAVE_ARCH_BITREVERSE to support rbit instruction 2014-12-22 16:43:06 +00:00
bsearch.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
btree.c lib/btree.c: fix leak of whole btree nodes 2014-06-04 16:54:18 -07:00
bug.c module: Sanitize RCU usage and locking 2015-05-28 11:31:52 +09:30
build_OID_registry X.509: do not emit any informational output 2013-06-19 17:54:06 +02:00
bust_spinlocks.c printk: Provide a wake_up_klogd() off-case 2013-03-22 16:41:20 -07:00
check_signature.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
checksum.c lib/checksum.c: fix build for generic csum_tcpudp_nofold 2015-01-29 11:57:38 -08:00
clz_ctz.c lib/clz_ctz.c: add prototype declarations in lib/clz_ctz.c 2014-04-03 16:21:12 -07:00
clz_tab.c lib: Fix multiple definitions of clz_tab 2012-02-02 10:34:23 +11:00
cmdline.c lib: Add a generic cmdline parse function parse_option_str 2014-10-03 18:40:58 +01:00
compat_audit.c audit: Add generic compat syscall support 2014-03-20 10:11:35 -04:00
cordic.c
cpu_rmap.c sched/topology: Rename topology_thread_cpumask() to topology_sibling_cpumask() 2015-05-27 15:22:15 +02:00
cpu-notifier-error-inject.c cpu: rewrite cpu-notifier-error-inject module 2012-07-30 17:25:22 -07:00
cpumask.c revert "cpumask: don't perform while loop in cpumask_next_and()" 2015-06-18 17:00:23 -10:00
crc7.c lib/crc7: Shift crc7() output left 1 bit 2014-05-16 14:26:52 -04:00
crc8.c
crc16.c
crc32.c lib: crc32: Add some additional __pure annotations 2014-06-25 16:04:00 -07:00
crc32defs.h crc32: select an algorithm via Kconfig 2012-03-23 16:58:38 -07:00
crc-ccitt.c
crc-itu-t.c lib: crc-itu-t.[ch] fix 0x0x prefix in integer constants 2015-05-26 15:26:43 +02:00
crc-t10dif.c lib: introduce crc_t10dif_update() 2015-05-30 22:42:24 -07:00
ctype.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
debug_info.c kbuild: include core debug info when DEBUG_INFO_REDUCED 2015-06-11 15:08:32 +02:00
debug_locks.c mutex: Add support for wound/wait style locks 2013-06-26 12:10:56 +02:00
debugobjects.c lib/debugobjects.c: convert printk(KERN_DEBUG to pr_debug 2014-06-04 16:53:53 -07:00
dec_and_lock.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
decompress_bunzip2.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress_inflate.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress_unlz4.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress_unlzma.c lib/decompress_unlzma: Do a NULL check for pointer 2015-09-10 13:29:01 -07:00
decompress_unlzo.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress_unxz.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress.c lib/decompress: set the compressor name to NULL on error 2015-07-17 16:39:54 -07:00
devres.c devres: fix a for loop bounds check 2015-10-05 04:49:54 +01:00
digsig.c KEYS: Merge the type-specific data with the payload data 2015-10-21 15:18:36 +01:00
div64.c lib: correct link to the original source for div64_u64 2015-03-06 23:19:27 +01:00
dma-debug.c debugfs: Pass bool pointer to debugfs_create_bool() 2015-10-04 11:36:07 +01:00
dump_stack.c asmlinkage: Add explicit __visible to drivers/*, lib/*, kernel/* 2014-05-05 16:07:46 -07:00
dynamic_debug.c module: add extra argument for parse_params() callback 2015-05-20 00:25:24 -07:00
dynamic_queue_limits.c lib/dynamic_queue_limits.c: simplify includes 2015-02-12 18:54:15 -08:00
earlycpio.c earlycpio.c: Fix the confusing comment of find_cpio_data(). 2013-08-14 23:24:01 +02:00
extable.c
fault-inject.c fault-inject: fix inverted interval/probability values in printk 2015-10-23 17:55:10 +09:00
fdt_empty_tree.c lib: add fdt_empty_tree.c 2014-04-30 19:49:37 +01:00
fdt_ro.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_rw.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_strerror.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_sw.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt_wip.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
fdt.c of/lib: Allow scripts/dtc/libfdt to be used from kernel code 2012-07-23 13:54:52 +01:00
find_bit.c lib: rename lib/find_next_bit.c to lib/find_bit.c 2015-04-17 09:03:54 -04:00
flex_array.c reciprocal_divide: update/correction of the algorithm 2014-01-21 23:17:20 -08:00
flex_proportions.c proportions: add @gfp to init functions 2014-09-08 09:51:30 +09:00
gcd.c lib/gcd.c: prevent possible div by 0 2012-10-06 03:04:57 +09:00
gen_crc32table.c lib: crc32: constify crc32 lookup table 2015-02-13 21:21:35 -08:00
genalloc.c genalloc: add support of multiple gen_pools per device 2015-09-04 16:54:41 -07:00
glob.c lib/glob.c: add CONFIG_GLOB_SELFTEST 2014-08-06 18:01:25 -07:00
halfmd4.c lib/halfmd4.c: simplify includes 2015-02-12 18:54:15 -08:00
hexdump.c hexdump: fix for non-aligned buffers 2015-07-17 16:39:53 -07:00
hweight.c Make ARCH_HAS_FAST_MULTIPLIER a real config variable 2014-09-13 11:14:53 -07:00
idr.c mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd 2015-11-06 17:50:42 -08:00
inflate.c
int_sqrt.c lib/int_sqrt.c: optimize square root algorithm 2013-04-29 18:28:19 -07:00
interval_tree_test.c lib: Export interval_tree 2014-05-05 09:09:14 +02:00
interval_tree.c lib/interval_tree.c: simplify includes 2015-02-12 18:54:15 -08:00
iomap_copy.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
iomap.c Kconfig: rename HAS_IOPORT to HAS_IOPORT_MAP 2014-04-07 16:36:11 -07:00
iommu-common.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc 2015-11-05 16:34:48 -08:00
iommu-helper.c The following text was taken from the original review request: 2012-03-24 10:24:31 -07:00
ioremap.c x86, mm: support huge KVA mappings on x86 2015-04-14 16:49:04 -07:00
iov_iter.c Merge branch 'iov_iter' into for-next 2015-04-11 22:26:51 -04:00
irq_regs.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
is_single_threaded.c
jedec_ddr_data.c ddr: add LPDDR2 data from JESD209-2 2012-05-02 00:04:06 -07:00
kasprintf.c lib/kasprintf.c: use kmalloc_track_caller() to get accurate traces for kvasprintf 2012-10-11 08:50:15 +09:00
Kconfig lib/Kconfig: ZLIB_DEFLATE must select BITREVERSE 2015-10-16 11:42:28 -07:00
Kconfig.debug lib/Kconfig.debug: disable -Wframe-larger-than warnings with KASAN=y 2015-10-23 17:55:10 +09:00
Kconfig.kasan mm, slub, kasan: enable user tracking by default with KASAN=y 2015-11-05 19:34:48 -08:00
Kconfig.kgdb kdb: Allow access to sensitive commands to be restricted by default 2014-11-11 09:31:52 -06:00
Kconfig.kmemcheck
kfifo.c kfifo: use BUG_ON 2014-08-08 15:57:25 -07:00
klist.c klist: implement klist_prev() 2015-07-28 08:50:42 +01:00
kobject_uevent.c lib/kobject_uevent.c: remove redundant include 2015-02-12 18:54:15 -08:00
kobject.c kobject: move EXPORT_SYMBOL() macros next to corresponding definitions 2015-10-04 11:46:16 +01:00
kstrtox.c kstrto*: accept "-0" for signed conversion 2015-09-10 13:29:01 -07:00
kstrtox.h
lcm.c block: fix blk_stack_limits() regression due to lcm() change 2015-03-31 09:45:50 -06:00
libcrc32c.c crypto: LLVMLinux: Remove VLAIS usage from libcrc32c.c 2014-10-14 10:51:23 +02:00
list_debug.c rcu: Fix broken strings in RCU's source code. 2012-07-06 06:01:49 -07:00
list_sort.c lib/list_sort: use late_initcall to hook in self tests 2015-06-16 14:12:35 -04:00
llist.c lib/llist.c: remove redundant include 2015-02-12 18:54:15 -08:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c locking/lockdep: Revert qrwlock recusive stuff 2014-10-03 06:09:30 +02:00
lockref.c locking/lockref: Remove homebrew cmpxchg64_relaxed() macro definition 2015-08-12 11:59:04 +02:00
lru_cache.c lru_cache: remove use of seq_printf return value 2015-04-15 16:35:25 -07:00
Makefile net: move net_get_random_once to lib 2015-10-08 05:26:35 -07:00
md5.c lib/md5.c: simplify include 2015-02-12 18:54:15 -08:00
memory-notifier-error-inject.c memory: memory notifier error injection module 2012-07-30 17:25:22 -07:00
memweight.c string: introduce memweight() 2012-07-30 17:25:16 -07:00
net_utils.c mac_pton: Use bool not int return 2014-06-25 17:45:43 -07:00
nlattr.c netlink: pad nla_memcpy dest buffer with zeroes 2015-03-31 14:07:24 -04:00
nmi_backtrace.c ARM: 8439/1: Fix backtrace generation when IPI is masked 2015-10-03 16:40:51 +01:00
notifier-error-inject.c mode_t, whack-a-mole at 11... 2013-04-09 14:13:05 -04:00
notifier-error-inject.h fault-injection: notifier error injection 2012-07-30 17:25:22 -07:00
of-reconfig-notifier-error-inject.c powerpc+of: Rename and fix OF reconfig notifier error inject module 2012-12-14 10:32:52 +11:00
oid_registry.c Give the OID registry file module info to avoid kernel tainting 2013-05-05 14:38:00 -07:00
once.c once: make helper generic for calling functions once 2015-10-08 05:26:36 -07:00
parser.c lib/parser.c: put EXPORT_SYMBOLs in the conventional place 2014-01-23 16:36:55 -08:00
pci_iomap.c libnvdimm for 4.3: 2015-09-08 14:35:59 -07:00
percpu_counter.c percpu_counter: batch size aware __percpu_counter_compare() 2015-05-29 07:39:34 +10:00
percpu_ida.c lib/percpu_ida.c: remove redundant includes 2015-02-12 18:54:16 -08:00
percpu_test.c percpu: add test module for various percpu operations 2013-11-13 12:09:11 +09:00
percpu-refcount.c percpu_ref: make INIT_ATOMIC and switch_to_atomic() sticky 2014-09-24 13:31:50 -04:00
plist.c lib/plist.c: remove redundant include 2015-02-12 18:54:16 -08:00
pm-notifier-error-inject.c PM: PM notifier error injection module 2012-07-30 17:25:22 -07:00
proportions.c proportions: add @gfp to init functions 2014-09-08 09:51:30 +09:00
radix-tree.c mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd 2015-11-06 17:50:42 -08:00
random32.c random32: add prandom_init_once helper for own rngs 2015-10-08 05:26:38 -07:00
ratelimit.c lib: reduce the use of module.h wherever possible 2012-03-07 15:04:04 -05:00
rational.c lib: Change mail address of Oskar Schirmer 2012-05-17 15:18:37 +02:00
rbtree_test.c rbtree/test: test rbtree_postorder_for_each_entry_safe() 2014-01-23 16:37:03 -08:00
rbtree.c rbtree: Make lockless searches non-fatal 2015-05-28 11:32:04 +09:30
reciprocal_div.c reciprocal_divide: update/correction of the algorithm 2014-01-21 23:17:20 -08:00
rhashtable.c lib: fix data race in rhashtable_rehash_one 2015-09-22 17:36:07 -07:00
scatterlist.c scatterlist: allow limited chaining without ARCH_HAS_SG_CHAIN 2015-08-17 08:12:51 -06:00
seq_buf.c seq_buf: Fix seq_buf_bprintf() truncation 2015-03-04 23:40:19 -05:00
sg_split.c lib: scatterlist: add sg splitting function 2015-08-24 14:28:01 -06:00
sha1.c lib: EXPORT_SYMBOL sha_init 2015-03-23 22:12:08 -04:00
show_mem.c lib/show_mem.c: correct reserved memory calculation 2015-09-08 15:35:28 -07:00
smp_processor_id.c percpu: add preemption checks to __this_cpu ops 2014-04-07 16:36:14 -07:00
sort.c lib/sort: Add 64 bit swap function 2015-06-25 17:00:40 -07:00
stmp_device.c lib/stmp_device.c: replace module.h include 2015-02-12 18:54:16 -08:00
string_helpers.c lib/string_helpers.c: fix infinite loop in string_get_size() 2015-09-17 21:16:07 -07:00
string.c strscpy: zero any trailing garbage bytes in the destination 2015-10-06 14:53:18 -04:00
strncpy_from_user.c lib: move strncpy_from_unsafe() into mm/maccess.c 2015-08-31 12:36:10 -07:00
strnlen_user.c Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-06-22 15:52:04 -07:00
swiotlb.c Merge branch 'for-4.2/sg' of git://git.kernel.dk/linux-block 2015-06-25 15:22:36 -07:00
syscall.c lib/syscall.c: unexport task_current_syscall() 2014-04-03 16:21:06 -07:00
test_bpf.c test_bpf: add tests checking that JIT/interpreter sets A and X to 0. 2015-08-06 22:02:32 -07:00
test_firmware.c test: add firmware_class loader test 2014-07-17 18:44:19 -07:00
test_kasan.c lib: test_kasan: add some testcases 2015-11-05 19:34:48 -08:00
test_module.c test: add minimal module for verification testing 2014-01-23 16:36:57 -08:00
test_rhashtable.c rhashtable-test: extend to test concurrency 2015-08-17 14:33:47 -07:00
test_static_key_base.c locking/static_keys: Provide a selftest 2015-08-03 11:51:12 +02:00
test_static_keys.c locking/static_keys: Make verify_keys() static 2015-08-05 09:53:40 +02:00
test_user_copy.c test: check copy_to/from_user boundary validation 2014-01-23 16:36:57 -08:00
test-hexdump.c hexdump: Make test data really const 2015-06-25 17:00:40 -07:00
test-kstrtox.c kstrto*: accept "-0" for signed conversion 2015-09-10 13:29:01 -07:00
test-string_helpers.c lib/string_helpers.c: change semantics of string_escape_mem 2015-04-15 16:35:24 -07:00
textsearch.c lib/textsearch.c: remove textsearch_put reference from comments 2014-10-14 02:18:14 +02:00
timerqueue.c timerqueue: Let timerqueue_add/del return information 2015-04-22 17:06:49 +02:00
ts_bm.c
ts_fsm.c
ts_kmp.c
ucs2_string.c Move utf16 functions to kernel core and rename 2013-04-15 21:23:03 +01:00
usercopy.c Kconfig: consolidate CONFIG_DEBUG_STRICT_USER_COPY_CHECKS 2013-04-30 17:04:09 -07:00
uuid.c uuid: use prandom_bytes() 2013-04-29 18:28:42 -07:00
vsprintf.c lib/vsprintf.c: Include clk.h 2015-07-20 10:52:48 -07:00