linux/fs/xfs/libxfs
Brian Foster 0ab32086d0 xfs: account only rmapbt-used blocks against rmapbt perag res
The rmapbt perag metadata reservation reserves blocks for the
reverse mapping btree (rmapbt). Since the rmapbt uses blocks from
the agfl and perag accounting is updated as blocks are allocated
from the allocation btrees, the reservation actually accounts blocks
as they are allocated to (or freed from) the agfl rather than the
rmapbt itself.

While this works for blocks that are eventually used for the rmapbt,
not all agfl blocks are destined for the rmapbt. Blocks that are
allocated to the agfl (and thus "reserved" for the rmapbt) but then
used by another structure leads to a growing inconsistency over time
between the runtime tracking of rmapbt usage vs. actual rmapbt
usage. Since the runtime tracking thinks all agfl blocks are rmapbt
blocks, it essentially believes that less future reservation is
required to satisfy the rmapbt than what is actually necessary.

The inconsistency is rectified across mount cycles because the perag
reservation is initialized based on the actual rmapbt usage at mount
time. The problem, however, is that the excessive drain of the
reservation at runtime opens a window to allocate blocks for other
purposes that might be required for the rmapbt on a subsequent
mount. This problem can be demonstrated by a simple test that runs
an allocation workload to consume agfl blocks over time and then
observe the difference in the agfl reservation requirement across an
unmount/mount cycle:

  mount ...: xfs_ag_resv_init: ... resv 3193 ask 3194 len 3194
  ...
  ...      : xfs_ag_resv_alloc_extent: ... resv 2957 ask 3194 len 1
  umount...: xfs_ag_resv_free: ... resv 2956 ask 3194 len 0
  mount ...: xfs_ag_resv_init: ... resv 3052 ask 3194 len 3194

As the above tracepoints show, the reservation requirement reduces
from 3194 blocks to 2956 blocks as the workload runs.  Without any
other changes in the filesystem, the same reservation requirement
jumps from 2956 to 3052 blocks over a umount/mount cycle.

To address this divergence, update the RMAPBT reservation to account
blocks used for the rmapbt only rather than all blocks filled into
the agfl. This patch makes several high-level changes toward that
end:

1.) Reintroduce an AGFL reservation type to serve as an accounting
    no-op for blocks allocated to (or freed from) the AGFL.
2.) Invoke RMAPBT usage accounting from the actual rmapbt block
    allocation path rather than the AGFL allocation path.

The first change is required because agfl blocks are considered free
blocks throughout their lifetime. The perag reservation subsystem is
invoked unconditionally by the allocation subsystem, so we need a
way to tell the perag subsystem (via the allocation subsystem) to
not make any accounting changes for blocks filled into the AGFL.

The second change causes the in-core RMAPBT reservation usage
accounting to remain consistent with the on-disk state at all times
and eliminates the risk of leaving the rmapbt reservation
underfilled.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
2018-03-11 20:27:57 -07:00
..
xfs_ag_resv.c xfs: account only rmapbt-used blocks against rmapbt perag res 2018-03-11 20:27:57 -07:00
xfs_ag_resv.h xfs: account only rmapbt-used blocks against rmapbt perag res 2018-03-11 20:27:57 -07:00
xfs_alloc_btree.c Cleanup old XFS_BTREE_* traces 2018-03-11 20:27:55 -07:00
xfs_alloc_btree.h
xfs_alloc.c xfs: account only rmapbt-used blocks against rmapbt perag res 2018-03-11 20:27:57 -07:00
xfs_alloc.h xfs: convert XFS_AGFL_SIZE to a helper function 2018-03-11 20:27:56 -07:00
xfs_attr_leaf.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_attr_leaf.h xfs: create structure verifier function for shortform xattrs 2018-01-08 10:54:46 -08:00
xfs_attr_remote.c xfs: create a new buf_ops pointer to verify structure metadata 2018-01-08 10:54:47 -08:00
xfs_attr_remote.h
xfs_attr_sf.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_attr.c xfs: cancel tx on xfs_defer_finish() error during xattr set/remove 2018-01-16 14:53:28 -08:00
xfs_bit.c libxfs: Optimize the loop for xfs_bitmap_empty 2016-01-04 16:10:19 +11:00
xfs_bit.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_bmap_btree.c Cleanup old XFS_BTREE_* traces 2018-03-11 20:27:55 -07:00
xfs_bmap_btree.h xfs: pass struct xfs_bmbt_irec to xfs_bmbt_validate_extent 2017-11-06 11:53:41 -08:00
xfs_bmap.c xfs: don't screw up direct writes when freesp is fragmented 2018-01-29 07:27:24 -08:00
xfs_bmap.h xfs: simplify xfs_reflink_convert_cow 2017-11-06 11:53:40 -08:00
xfs_btree.c Cleanup old XFS_BTREE_* traces 2018-03-11 20:27:55 -07:00
xfs_btree.h Cleanup old XFS_BTREE_* traces 2018-03-11 20:27:55 -07:00
xfs_cksum.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xfs_da_btree.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_da_btree.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_da_format.c xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_da_format.h xfs: refactor the geometry structure filling function 2018-01-08 10:54:48 -08:00
xfs_defer.c xfs: add the ability to join a held buffer to a defer_ops 2017-12-14 09:17:35 -08:00
xfs_defer.h xfs: add the ability to join a held buffer to a defer_ops 2017-12-14 09:17:35 -08:00
xfs_dir2_block.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_dir2_data.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_dir2_leaf.c xfs: convert a few more directory asserts to corruption 2018-03-11 20:27:56 -07:00
xfs_dir2_node.c xfs: convert a few more directory asserts to corruption 2018-03-11 20:27:56 -07:00
xfs_dir2_priv.h xfs: refactor short form directory structure verifier function 2018-01-08 10:54:46 -08:00
xfs_dir2_sf.c xfs: directory scrubber must walk through data block to offset 2018-01-17 21:00:46 -08:00
xfs_dir2.c xfs: refactor the geometry structure filling function 2018-01-08 10:54:48 -08:00
xfs_dir2.h xfs: directory scrubber must walk through data block to offset 2018-01-17 21:00:46 -08:00
xfs_dquot_buf.c xfs: standardize quota verification function outputs 2018-01-08 10:54:47 -08:00
xfs_errortag.h xfs: move error injection tags into their own file 2017-11-01 15:03:16 -07:00
xfs_format.h xfs: convert XFS_AGFL_SIZE to a helper function 2018-03-11 20:27:56 -07:00
xfs_fs.h xfs: check sb_agblocks and sb_agblklog when validating superblock 2018-01-17 21:00:47 -08:00
xfs_ialloc_btree.c Cleanup old XFS_BTREE_* traces 2018-03-11 20:27:55 -07:00
xfs_ialloc_btree.h xfs: use per-AG reservations for the finobt 2017-01-25 07:49:35 -08:00
xfs_ialloc.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_ialloc.h xfs: add scrub cross-referencing helpers for the inode btrees 2018-01-17 21:00:44 -08:00
xfs_iext_tree.c xfs: move xfs_iext_insert tracepoint to report useful information 2017-12-14 09:20:11 -08:00
xfs_inode_buf.c New in this version: 2018-01-31 10:18:00 -08:00
xfs_inode_buf.h xfs: have buffer verifier functions report failing address 2018-01-08 10:54:46 -08:00
xfs_inode_fork.c xfs: btree format ifork loader should check for zero numrecs 2018-01-17 21:00:46 -08:00
xfs_inode_fork.h xfs: provide a centralized method for verifying inline fork data 2018-01-08 10:54:47 -08:00
xfs_log_format.h xfs: fix type usage 2017-11-16 12:06:45 -08:00
xfs_log_recover.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_log_rlimit.c xfs: trace log reservations at mount time 2018-01-08 10:54:47 -08:00
xfs_quota_defs.h xfs: standardize quota verification function outputs 2018-01-08 10:54:47 -08:00
xfs_refcount_btree.c Cleanup old XFS_BTREE_* traces 2018-03-11 20:27:55 -07:00
xfs_refcount_btree.h xfs: use the actual AG length when reserving blocks 2017-01-03 18:39:33 -08:00
xfs_refcount.c xfs: add scrub cross-referencing helpers for the refcount btrees 2018-01-17 21:00:44 -08:00
xfs_refcount.h xfs: add scrub cross-referencing helpers for the refcount btrees 2018-01-17 21:00:44 -08:00
xfs_rmap_btree.c xfs: account only rmapbt-used blocks against rmapbt perag res 2018-03-11 20:27:57 -07:00
xfs_rmap_btree.h xfs: use the actual AG length when reserving blocks 2017-01-03 18:39:33 -08:00
xfs_rmap.c xfs: add scrub cross-referencing helpers for the rmap btrees 2018-01-17 21:00:44 -08:00
xfs_rmap.h xfs: add scrub cross-referencing helpers for the rmap btrees 2018-01-17 21:00:44 -08:00
xfs_rtbitmap.c xfs: cross-reference the realtime bitmap 2018-01-17 21:00:46 -08:00
xfs_sb.c xfs: fix u32 type usage in sb validation function 2018-01-31 20:39:20 -08:00
xfs_sb.h xfs: refactor the geometry structure filling function 2018-01-08 10:54:48 -08:00
xfs_shared.h xfs: trace log reservations at mount time 2018-01-08 10:54:47 -08:00
xfs_symlink_remote.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_trans_resv.c xfs: eliminate duplicate icreate tx reservation functions 2018-01-08 10:41:38 -08:00
xfs_trans_resv.h xfs: increase log reservations for reflink 2016-10-05 16:26:29 -07:00
xfs_trans_space.h xfs: reserve enough blocks to handle btree splits when remapping 2017-05-03 13:21:40 -07:00
xfs_types.h xfs: move xfs_bmbt_irec and xfs_exntst_t to xfs_types.h 2017-11-06 11:53:41 -08:00