Linux kernel source tree
Go to file
Roman Gushchin 2e9bd48315 mm: memcg/slab: pre-allocate obj_cgroups for slab caches with SLAB_ACCOUNT
In general it's unknown in advance if a slab page will contain accounted
objects or not.  In order to avoid memory waste, an obj_cgroup vector is
allocated dynamically when a need to account of a new object arises.  Such
approach is memory efficient, but requires an expensive cmpxchg() to set
up the memcg/objcgs pointer, because an allocation can race with a
different allocation on another cpu.

But in some common cases it's known for sure that a slab page will contain
accounted objects: if the page belongs to a slab cache with a SLAB_ACCOUNT
flag set.  It includes such popular objects like vm_area_struct, anon_vma,
task_struct, etc.

In such cases we can pre-allocate the objcgs vector and simple assign it
to the page without any atomic operations, because at this early stage the
page is not visible to anyone else.

A very simplistic benchmark (allocating 10000000 64-bytes objects in a
row) shows ~15% win.  In the real life it seems that most workloads are
not very sensitive to the speed of (accounted) slab allocations.

[guro@fb.com: open-code set_page_objcgs() and add some comments, by Johannes]
  Link: https://lkml.kernel.org/r/20201113001926.GA2934489@carbon.dhcp.thefacebook.com
[akpm@linux-foundation.org: fix it for mm-slub-call-account_slab_page-after-slab-page-initialization-fix.patch]

Link: https://lkml.kernel.org/r/20201110195753.530157-2-guro@fb.com
Signed-off-by: Roman Gushchin <guro@fb.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-02-24 13:38:29 -08:00
arch hexagon: remove CONFIG_EXPERIMENTAL from defconfigs 2021-02-24 13:38:26 -08:00
block for-5.12/block-ipi-2021-02-21 2021-02-22 10:53:05 -08:00
certs certs: Replace K{U,G}IDT_INIT() with GLOBAL_ROOT_{U,G}ID 2021-01-21 16:16:10 +00:00
crypto Keyrings miscellany 2021-02-23 16:09:23 -08:00
Documentation mm/debug_vm_pgtable/basic: add validation for dirtiness after write protect 2021-02-24 13:38:27 -08:00
drivers Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc 2021-02-23 15:09:53 -08:00
fs fs/buffer.c: add checking buffer head stat before clear 2021-02-24 13:38:28 -08:00
include mm: memcg/slab: pre-allocate obj_cgroups for slab caches with SLAB_ACCOUNT 2021-02-24 13:38:29 -08:00
init mm, slub: remove slub_memcg_sysfs boot param and CONFIG_SLUB_MEMCG_SYSFS_ON 2021-02-24 13:38:27 -08:00
ipc fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
kernel Keyrings miscellany 2021-02-23 16:09:23 -08:00
lib Modules updates for v5.12 2021-02-23 10:15:33 -08:00
LICENSES LICENSES: Add the CC-BY-4.0 license 2020-12-08 10:33:27 -07:00
mm mm: memcg/slab: pre-allocate obj_cgroups for slab caches with SLAB_ACCOUNT 2021-02-24 13:38:29 -08:00
net idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
samples Keyrings miscellany 2021-02-23 16:09:23 -08:00
scripts scripts/spelling.txt: add more spellings to spelling.txt 2021-02-24 13:38:26 -08:00
security Keyrings miscellany 2021-02-23 16:09:23 -08:00
sound ARM updates for 5.12-rc1: 2021-02-22 14:27:07 -08:00
tools clang-lto for v5.12-rc1 (part2) 2021-02-23 15:13:45 -08:00
usr arch: ia64: Remove rest of perfmon support 2021-01-22 12:12:20 +05:30
virt KVM/arm64 fixes for 5.11, take #2 2021-02-12 14:07:39 +00:00
.clang-format clang-format: Update with the latest for_each macro list 2021-01-29 15:00:23 +01:00
.cocciconfig
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore clang-lto series for v5.12-rc1 2021-02-23 09:28:51 -08:00
.mailmap Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2021-02-21 17:23:56 -08:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: dccp: move Gerrit Renker to CREDITS 2021-01-14 10:53:49 -08:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS dmaengine updates for v5.12-rc1 2021-02-23 15:05:10 -08:00
Makefile clang-lto for v5.12-rc1 (part2) 2021-02-23 15:13:45 -08:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.