linux/fs
Zach Brown 9470dd5d35 fs: check bh blocknr earlier when searching lru
It's very common for the buffer heads in the lru to have different block
numbers.  By comparing the blocknr before the bdev and size we can
reduce the cost of searching in the very common case where all the
entries have the same bdev and size.

In quick hot cache cycle counting tests on a single fs workstation this
cut the cost of a miss by about 20%.

A diff of the disassembly shows the reordering of the bdev and blocknr
comparisons.  This is in such a tiny loop that skipping one comparison
is a meaningful portion of the total work being done:

     1628:      83 c1 01                add    $0x1,%ecx
     162b:      83 f9 08                cmp    $0x8,%ecx
     162e:      74 60                   je     1690 <__find_get_block+0xa0>
     1630:      89 c8                   mov    %ecx,%eax
     1632:      65 4c 8b 04 c5 00 00    mov    %gs:0x0(,%rax,8),%r8
     1639:      00 00
     163b:      4d 85 c0                test   %r8,%r8
     163e:      4c 89 c3                mov    %r8,%rbx
     1641:      74 e5                   je     1628 <__find_get_block+0x38>
-    1643:      4d 3b 68 30             cmp    0x30(%r8),%r13
+    1643:      4d 3b 68 18             cmp    0x18(%r8),%r13
     1647:      75 df                   jne    1628 <__find_get_block+0x38>
-    1649:      4d 3b 60 18             cmp    0x18(%r8),%r12
+    1649:      4d 3b 60 30             cmp    0x30(%r8),%r12
     164d:      75 d9                   jne    1628 <__find_get_block+0x38>
     164f:      49 39 50 20             cmp    %rdx,0x20(%r8)
     1653:      75 d3                   jne    1628 <__find_get_block+0x38>

Signed-off-by: Zach Brown <zab@zabbo.net>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-10-14 02:18:26 +02:00
..
9p 9p: switch to %p[dD] 2014-10-09 02:39:04 -04:00
adfs
affs fs/affs: remove redundant sys_tz declarations 2014-10-14 02:18:22 +02:00
afs Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2014-10-13 16:23:15 +02:00
autofs4 autofs4: d_manage() should return -EISDIR when appropriate in rcu-walk mode. 2014-10-14 02:18:16 +02:00
befs fs/befs/btree.c: remove typedef befs_btree_node 2014-10-14 02:18:20 +02:00
bfs
btrfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-10-13 11:28:42 +02:00
cachefiles cachefiles_write_page(): switch to __kernel_write() 2014-10-09 02:39:05 -04:00
ceph vfs: Remove d_drop calls from d_revalidate implementations 2014-10-09 02:38:58 -04:00
cifs cifs: replace strnicmp with strncasecmp 2014-10-14 02:18:24 +02:00
coda
configfs
cramfs
debugfs
devpts
dlm locks: Remove unused conf argument from lm_grant 2014-09-09 16:01:06 -04:00
ecryptfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-10-13 11:28:42 +02:00
efivarfs
efs
exofs
exportfs
ext2 percpu_counter: add @gfp to percpu_counter_init() 2014-09-08 09:51:29 +09:00
ext3 Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs 2014-10-11 08:02:31 -04:00
ext4 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block into for-3.18 2014-09-24 13:00:21 -04:00
f2fs f2fs: support volatile operations for transient data 2014-10-07 11:54:41 -07:00
fat fat: remove redundant sys_tz declaration 2014-10-14 02:18:20 +02:00
freevxfs
fscache FS-Cache: refcount becomes corrupt under vma pressure. 2014-09-17 22:41:40 +01:00
fuse vfs: Make d_invalidate return void 2014-10-09 02:38:57 -04:00
gfs2 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-10-13 11:28:42 +02:00
hfs fs/hfs/hfs_fs.h: remove redundant sys_tz declaration 2014-10-14 02:18:20 +02:00
hfsplus
hostfs
hpfs
hppfs
hugetlbfs
isofs isofs: replace strnicmp with strncasecmp 2014-10-14 02:18:24 +02:00
jbd
jbd2 jbd2: fix descriptor block size handling errors with journal_csum 2014-08-28 22:22:29 -04:00
jffs2 [jffs2] kill wbuf_queued/wbuf_dwork_lock 2014-10-09 02:39:01 -04:00
jfs Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2014-10-13 16:23:15 +02:00
kernfs vfs: Remove unnecessary calls of check_submounts_and_drop 2014-10-09 02:38:56 -04:00
lockd File locking related changes for v3.18 (pile #1) 2014-10-11 13:21:34 -04:00
logfs
minix
ncpfs fs/ncpfs/dir.c: remove redundant sys_tz declaration 2014-10-14 02:18:16 +02:00
nfs Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2014-10-13 16:23:15 +02:00
nfs_common lockd: move lockd's grace period handling into its own module 2014-09-17 16:33:11 -04:00
nfsd Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2014-10-13 16:23:15 +02:00
nilfs2 nilfs2: improve the performance of fdatasync() 2014-10-14 02:18:20 +02:00
nls
notify File locking related changes for v3.18 (pile #1) 2014-10-11 13:21:34 -04:00
ntfs ntfs: remove bogus space 2014-10-09 22:25:46 -04:00
ocfs2 ocfs2: replace strnicmp with strncasecmp 2014-10-14 02:18:24 +02:00
omfs FS/OMFS: block number sanity check during fill_super operation 2014-10-14 02:18:22 +02:00
openpromfs
proc Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-10-13 11:28:42 +02:00
pstore
qnx4
qnx6
quota percpu_counter: add @gfp to percpu_counter_init() 2014-09-08 09:51:29 +09:00
ramfs
reiserfs fs/reiserfs/journal.c: fix sparse context imbalance warning 2014-10-14 02:18:20 +02:00
romfs
squashfs
sysfs
sysv
ubifs
udf udf: Fix loading of special inodes 2014-10-09 13:06:14 +02:00
ufs fs/ufs/balloc.c: remove unused variable 2014-10-14 02:18:20 +02:00
xfs Merge branch 'xfs-misc-fixes-for-3.18-3' into for-next 2014-10-13 10:22:45 +11:00
aio.c percpu_ref: add PERCPU_REF_INIT_* flags 2014-09-24 13:31:50 -04:00
anon_inodes.c
attr.c
bad_inode.c
binfmt_aout.c handle suicide on late failure exits in execve() in search_binary_handler() 2014-10-09 02:39:00 -04:00
binfmt_elf_fdpic.c handle suicide on late failure exits in execve() in search_binary_handler() 2014-10-09 02:39:00 -04:00
binfmt_elf.c handle suicide on late failure exits in execve() in search_binary_handler() 2014-10-09 02:39:00 -04:00
binfmt_em86.c
binfmt_flat.c
binfmt_misc.c binfmt_misc: work around gcc-4.9 warning 2014-10-14 02:18:16 +02:00
binfmt_script.c
binfmt_som.c
block_dev.c block_dev: implement readpages() to optimize sequential read 2014-10-09 22:25:53 -04:00
buffer.c fs: check bh blocknr earlier when searching lru 2014-10-14 02:18:26 +02:00
char_dev.c
compat_binfmt_elf.c
compat_ioctl.c
compat.c vfs: move getname() from callers to do_mount() 2014-10-09 02:39:16 -04:00
coredump.c coredump: add %i/%I in core_pattern to report the tid of the crashed thread 2014-10-14 02:18:21 +02:00
dcache.c take dname_external() into fs/dcache.c 2014-10-12 17:09:05 -04:00
dcookies.c
direct-io.c fuse: honour max_read and max_write in direct_io mode 2014-09-26 21:16:51 -04:00
drop_caches.c
eventfd.c
eventpoll.c eventpoll: fix uninitialized variable in epoll_ctl 2014-09-10 15:42:12 -07:00
exec.c handle suicide on late failure exits in execve() in search_binary_handler() 2014-10-09 02:39:00 -04:00
fcntl.c security: make security_file_set_fowner, f_setown and __f_setown void return 2014-09-09 16:01:36 -04:00
fhandle.c
file_table.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-10-13 11:28:42 +02:00
file.c Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2014-10-13 15:44:12 +02:00
filesystems.c
fs_pin.c
fs_struct.c
fs-writeback.c
inode.c
internal.h Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-10-13 11:28:42 +02:00
ioctl.c
Kconfig lockd: move lockd's grace period handling into its own module 2014-09-17 16:33:11 -04:00
Kconfig.binfmt
libfs.c locks: plumb a "priv" pointer into the setlease routines 2014-10-07 14:06:12 -04:00
locks.c locks: flock_make_lock should return a struct file_lock (or PTR_ERR) 2014-10-07 14:06:13 -04:00
Makefile
mbcache.c
mount.h vfs: Add a function to lazily unmount all mounts from any dentry. 2014-10-09 02:38:55 -04:00
mpage.c vfs: guard end of device for mpage interface 2014-10-09 22:25:53 -04:00
namei.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-10-13 11:28:42 +02:00
namespace.c vfs: move getname() from callers to do_mount() 2014-10-09 02:39:16 -04:00
no-block.c
open.c
pipe.c
pnode.c get rid of propagate_umount() mistakenly treating slaves as busy. 2014-08-30 18:31:41 -04:00
pnode.h
posix_acl.c
proc_namespace.c
read_write.c cachefiles_write_page(): switch to __kernel_write() 2014-10-09 02:39:05 -04:00
readdir.c
select.c
seq_file.c
signalfd.c
splice.c
stack.c fs: fix comment for 'CONFIG_LBADF' 2014-08-26 09:35:56 +02:00
stat.c
statfs.c
super.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-10-13 11:28:42 +02:00
sync.c Export sync_filesystem() for modular ->remount_fs() use 2014-09-05 08:16:21 -07:00
timerfd.c timerfd: Remove an always true check 2014-08-27 11:17:48 +02:00
utimes.c
xattr.c vfs: Deduplicate code shared by xattr system calls operating on paths 2014-10-12 17:09:10 -04:00