Commit Graph

973 Commits

Author SHA1 Message Date
Theotime Combes
5c723d1713 mkfs.f2fs: add -T flag
T flag sets timestamps to a given value

Signed-off-by: Theotime Combes <tcombes@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-20 09:22:48 -07:00
Daeho Jeong
f7029e2cf2 sload.f2fs: handle root mount point properly when setting file attribute
Need to remove "/" of mount point name from the file path name
when mount point is "/". Otherwise, we will transfer file path
name whose first two characters are like "//" to fs_config function.

Signed-off-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-20 09:08:34 -07:00
Aravind Ramesh
f8410857b7 f2fs-tools: zns zone-capacity support
NVM Express Zoned Namespace (ZNS) devices can have zone-capacity(zc) less
than the zone-size. ZNS defines a per zone capacity which can be equal
or less than the zone-size. Zone-capacity is the number of usable blocks
in the zone. If zone-capacity is less than zone-size, then the segments
which start at/after zone-capacity are considered unusable. Only those
segments which start before the zone-capacity are considered as usable
and added to the free_segment_count and free_segment_bitmap of the kernel.
In such cases, the filesystem should not write/read beyond the
zone-capacity.

Update the super block with the usable number of blocks and free segment
count in the ZNS device zones, if zone-capacity is less than zone-size.
Set reserved segment count and overprovision ratio based on the usable
segments in the zone.

Allow fsck to find the free_segment_count based on the zone-capacity and
compare with checkpoint values.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
[Jaegeuk Kim: add UNUSED to is_usable_seg()]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-20 09:08:34 -07:00
Jaegeuk Kim
1e3dd7ae5d f2fs_io: measure performance of write()
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-20 09:08:31 -07:00
Jaegeuk Kim
334047bdca f2fs_io: add OSYNC option
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-20 09:08:07 -07:00
Chao Yu
2579fe8099 fsck.f2fs: correct return value
As Norbert Lange reported:

"
$ fsck.f2fs -a /dev/mmcblk0p5; echo $?
Info: Fix the reported corruption.
Info: Mounted device!
Info: Check FS only on RO mounted device
Error: Failed to open the device!
255
"

Michael Laß reminds:

"
I think the return value is exactly the problem here. See fsck(8) (
https://linux.die.net/man/8/fsck) which specifies the return values.
Systemd looks at these and decides how to proceed:

a859abf062/src/fsck/fsck.c (L407)

That means, if fsck.f2fs returns 255, then
the FSCK_SYSTEM_SHOULD_REBOOT bit is set and systemd will reboot.
"

So the problem here is fsck.f2fs didn't return correct value to userspace
apps, result in later unexpected behavior of rebooting, let's fix this.

Reported-by: Norbert Lange <nolange79@gmail.com>
Reported-by: Michael Laß <bevan@bi-co.net>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: fix FSCK_ERROR_CORRECTED]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-17 23:52:38 -07:00
Daeho Jeong
5c92399f8f f2fs_io: open file in read only mode for pinning file
To use "pinfile" command even with read only file, changed open
mode for it.

Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-04 17:52:56 -07:00
Jaegeuk Kim
3aef9bc820 mkfs.f2fs: add casefolding and project quota config
This can be used for Android build support.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-20 17:39:02 -07:00
Jaegeuk Kim
ec0f3c910e mkfs.f2fs: should initialize sparse file in Windows
Otherwise it fails to format in Windows.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-20 17:39:02 -07:00
Jaegeuk Kim
17463f9c42 f2fs_io: add mmap read operation
This patch adds an option, mmap, when reading data.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-20 17:39:02 -07:00
Theotime Combes
391e490a12 fsck.f2fs: avoid randomness in sum_blk
This patch removes random bytes in sum_blk when loading a new directory.

Signed-off-by: Theotime Combes <tcombes@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-15 18:12:28 -07:00
Jaegeuk Kim
aac7f4cb42 dump.f2fs: check block address first before reading the block
This should avoid to read out wrong block address.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-15 10:30:03 -07:00
Jaegeuk Kim
e767223422 sload.f2fs: fix missing fsck_init
This patch allocates some data structures that will be used for loading files
on existing image.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-15 10:29:57 -07:00
Zoltán Böszörményi
31acef0417 mkfs.f2fs: allow setting volume UUID manually
This patch adds an option to assign UUID manually.

Usage:
 # mkfs.f2fs -U uuid_string

Signed-off-by: Zoltán Böszörményi <zboszor@pr.hu>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-09 19:58:05 -07:00
Robin Hsu
d03cab6c93 fsck.f2fs: Split build_segment_manager() to speed up auto-fix
Speed up fsck in auto-fix mode by splitting
build_segment_manager() into two disjoint parts:
	early_build_segment_manager(), and
	late_build_segment_manager(),
where in some cases (when !need_fsync_data_record(), or cannot
find_fsync_inode()), late_build_segment_manager() won't be needed in
auto-fix mode.  This speeds up a little bit in those cases.

Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-09 18:16:58 -07:00
Jaegeuk Kim
e3c4e1f89a dump.f2fs: dump symlink contents in lost_found
Let's allow to show symlink path.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-06 20:13:11 -07:00
Robin Hsu
0d0158cf23 fsck.f2fs: Fix slow fsck in auto-fix mode
Split f2fs_init_nid_bitmap() into two disjoint parts:
	f2fs_early_init_nid_bitmap(), and
	f2fs_late_init_nid_bitmap(),
where f2fs_late_init_nid_bitmap() won't be called in auto-fix mode, when
no errors were found.

f2fs_late_init_nid_bitmap() contains the loop to create NID bitmap from
NAT. which is the main reason for slow fsck.

Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-29 08:48:21 -07:00
Robin Hsu
e66e4c1f3b fsck: prevent buffer overrun in quota code
A maliciously corrupted file systems can trigger buffer overruns in
the quota code used by fsck.

To fix it, quota file sizes are checked against real allocated
block index tables (inode, direct nodes, indirect nodes, double
indirect nodes).  If the size mismatches, the quota file is considered
corrupted and will be regenerated.

Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-23 16:58:44 -07:00
Wuyun Zhao
712aca364f f2fs-tools: set cold flag for non-dir node
Signed-off-by: Wuyun Zhao <zhaowuyun@wingtech.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-18 12:38:18 -07:00
Rolf Eike Beer
895b26e118 lib: pass all needed include paths to libf2fs
Otherwise the paths to uuid and blkid headers extracted from pkg-config are not
respected.

Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-18 12:34:29 -07:00
Rolf Eike Beer
4b55c43828 lib: use ${libuuid_LIBS} instead of -luuid everywhere
Otherwise linking will fail if the library is in an uncommon location or has a
non-standard name.

Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-18 12:34:23 -07:00
Rolf Eike Beer
1ceb06bfcc lib: fix include path for blkid.h
$ pkg-config --cflags blkid
-I/usr/include/blkid

The "blkid/" directory is actually part of the include path. This usually still
works because most people have the path one level up in their default include
path.

Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-18 12:34:13 -07:00
Rolf Eike Beer
59a37413f8 lib: fix include path for uuid.h
$ pkg-config --cflags uuid
-I/usr/include/uuid

The "uuid/" directory is actually part of the include path. This usually still
works because most people have the path one level up in their default include
path.

Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-18 12:34:03 -07:00
Jaegeuk Kim
a60cd1a3f9 libf2fs: fix build error on Windows
Windows doesn't support S_ISREG, so let's avoid depedency.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-08 11:37:28 -07:00
Eric Biggers
e65d9c7adf fsck.f2fs: fix dirent position check for encrypted+casefolded dentries
fsck.f2fs reports corruption if the filesystem contains any encrypted +
casefolded directories with any substantial number of dentries:

    [ASSERT] (f2fs_check_dirent_position:1374)  --> Wrong position of dirent pino:8, name:۟�[I�^*�(�5~�}�D��#]7�8�ˎ�, level:1, dir_level:0, pgofs:4, correct range:[2, 3]

The problem is that f2fs_check_dirent_position() computes the wrong hash
for encrypted+casefolded dentries.  It's not actually possible for it to
compute the correct hash, because it would need the encryption key.

However, the on-disk dentry already contains the hash code, and its
correctness was already verified by f2fs_check_hash_code() if possible.

So, make f2fs_check_dirent_position() use the hash code from disk rather
than recompute it.

Also fix it to print the filename in human-readable form.

This bug was causing 'kvm-xfstests -c f2fs/encrypt -g casefold'
to fail with the test_dummy_encryption_v2 and encryption+casefolding
kernel patches applied.

Fixes: 7f3767ee8d ("f2fs-tools: Casefolded Encryption support")
Cc: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-30 08:18:51 -07:00
Daeho Jeong
b23b2b97a6 f2fs_io: add randread
I've added a new command to evaluate random read.

Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-27 13:59:25 -07:00
Chao Yu
e8d9f6d7b3 fsck.f2fs: fix to avoid overflow during print_inode_info()
While dumping files during fsck, print_inode_info() didn't check
sanity of inode, so insane i_extra_isize could cause overflow
when printing i_addr, to avoid that, let's add a check condition.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:41:06 -07:00
Chao Yu
7a22451bc2 fsck.f2fs: fix to check validation of i_xattr_nid
Otherwise, fsck.f2fs will access invalid memory address as below:

- fsck_verify
 - dump_node
  - dump_file
   - dump_inode_blk
    - dump_xattr
     - read_all_xattrs
       - get_node_info
        access &(F2FS_FSCK(sbi)->entries[nid])

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:41:06 -07:00
Chao Yu
f73f222723 fsck.f2fs: fix to check validation of block address
Otherwise, if block address is invalid, we may access invalid
memory address in is_sit_bitmap_set().

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:41:06 -07:00
Érico Nogueira
f8f14b9c0f mkfs.f2fs: update manual and help message
Add options that were missing from the mkfs.f2fs man page, and explain
them more thoroughly when possible. Special care was given to document
all the features that can be enabled during the creation of the
filesystem.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:41:06 -07:00
Chao Yu
88d1bd9615 fsck.f2fs: fix wrong addrs_per_{inode,block}
generic/339 reports below assertion on image w/ compression feature
enabled.

[ASSERT] (f2fs_check_dirent_position:1366)  -->
Wrong position of dirent pino:4521, name:"....", level:9, dir_level:0,
pgofs:1880, correct range:[1882, 1883]

The root cause is we calculate blkaddr number in direct node
incorrectly for directory inode, since during calculation, we only
need align blkaddr number to cluster size for regular inode rather
than directory inode, let's fix it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:41:06 -07:00
Chao Yu
a64fbe53c3 dump.f2fs: fix to print all blkaddr in .i_addr
There are totally ADDRS_PER_INODE() blkaddrs in .i_addr, fix to
print all of them.

In addition, use get_extra_isize() rather than __get_extra_isize()

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:41:06 -07:00
Theodore Ts'o
1ba0a74ef2 man: add missing man page for f2fs_io
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
[Jaegeuk Kim: add missing entries and fix Makefile.am]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:41:06 -07:00
Jaegeuk Kim
42f739c4c7 f2fs_io: show more flags
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:37:55 -07:00
Jaegeuk Kim
d21dd9aaca f2fs_io: don't give garbage data in upper 32bits
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:37:50 -07:00
Jaegeuk Kim
850861988f f2fs_io: add fsync
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-01 17:29:07 -07:00
Jaegeuk Kim
46df2dff65 fsck.f2fs: quick fix of CLOCK_BOOTTIME in mac
This fixes build error on mac.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-04-15 13:56:29 -07:00
Sahitya Tummala
9a31cefa29 fsck.f2fs: allow fsck to fix issues with online resize due to SPO
Add support for new CP flag CP_RESIZEFS_FLAG set during online
resize FS. If SPO happens after SB is updated but CP isn't, then
allow fsck to fix it.

The fsck errors without this fix -
    Info: CKPT version = 6ed7bccb
            Wrong user_block_count(2233856)
    [f2fs_do_mount:3365] Checkpoint is polluted

The subsequent mount failure without this fix -
[   11.294650] F2FS-fs (sda8): Wrong user_block_count: 2233856
[   11.300272] F2FS-fs (sda8): Failed to get valid F2FS checkpoint

Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-03-24 08:47:52 -07:00
Chao Yu
11e4322127 fsck.f2fs: account COMPRESS_ADDR as reserved blocks correctly
We should not account COMPRESS_ADDR as reserved block once we
released compress block on compress inode.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-03-24 08:47:42 -07:00
Chao Yu
596408fd82 f2fs_io: support compress blocks ioctls
Adds to support below three commands to operate reserved blocks
in compressed inode.
- get_cblocks
- release_cblocks
- reserve_cblocks

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-03-18 18:07:18 -07:00
Wei Wang
36053752e8 fsck.f2fs: report real wall time
clock_t time is per-process time, which is not wall time. For example,
if the CPU is shared by other processes, clock_t time may advance slower
than wall clock. On the other hand, if the current process is
multithreaded and more than one execution core is available, clock_t
time may advance faster than wall clock.

this CL changes it to use CLOCK_BOOTTIME (Linux-specific)

Signed-off-by: Wei Wang <wvw@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-03-18 18:07:13 -07:00
Tom Cherry
2b1dd73248 Include private/fs_config.h directly when needed
This is no longer a transitive include of android_filesystem_config.h

Signed-off-by: Tom Cherry <tomcherry@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-03-18 18:05:35 -07:00
Daniel Rosenberg
7f3767ee8d f2fs-tools: Casefolded Encryption support
This adds support for casefolded and encrypted directories.
Fsck cannot check the hashes of such directories because it would
require access to the encryption key to generate the siphash

Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-03-09 14:17:41 -07:00
Chao Yu
64750e90d6 fsck.f2fs: allow --dry-run to check readonly mounted fs
As Ondřej Jirman <megi@xff.cz> reported:

I was trying to run: fsck.f2fs --dry-run /dev/mmcblk0p2 on a RO mounted device,
and fsck refuses to run. Strace shows that it tries to open the block device
with O_EXCL even in RO mode, which will always fail if the block device
is mounted.

fsck.f2fs --dry-run /dev/mmcblk0p2
Info: Dry run
Info: Mounted device!
Info: Check FS only on RO mounted device
	Error: Failed to open the device!

I suggest not using O_EXCL for --dry-run check.

Let's change to allow --dry-run to check readonly mounted fs.

Reported-by: Ondřej Jirman <megi@xff.cz>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-02-24 14:01:53 -08:00
xiongping1
daa0f8b9e9 resize.f2fs: add option for large_nat_bitmap feature
resize.f2fs has already supported large_nat_bitmap feature, but has no
option to turn on it.

This change add a new '-i' option to control turning on it.

Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-14 07:43:05 -08:00
Chao Yu
d77bca8e16 dump.f2fs: print more info of inode layout
This patch adds to print all valid blkaddrs in inode's i_addr field,
besides, it also adds to print meaning of specific blkaddr(flag).

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-14 07:43:00 -08:00
Chao Yu
fdd47b2ea7 f2fs-tools: support data compression
This patch adds to support compression, introducing '-O compression'
option to enable this feature in image.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-14 07:42:54 -08:00
Robin Hsu
cf7329e5b6 fsck.f2fs: Enable user-space cache
Added command line options -c <num_cache_entry> and -m <max_hash_collision>
to activate cache for fsck.  It may significantly speed up fsck.

Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-14 07:42:02 -08:00
Robin Hsu
70c5bab971 libf2fs_io: Add user-space cache
Implemented cache options in F2FS configuration 'c' (i.e. user
options):
    * use c.cache_config.num_cache_entry to set the number of
      cache entries (in block), minimum 1024, or 0 to disable cache.
    * use c.cache_config.max_hash_collision to set maximum hash
      collision (max 16).
    * use c.cavhe_config.dbg_en to enable printing of debug messages.

Cache properties:
    * Per block save based (block size = f2fs block size = 4K Bytes)
    * Device block indices are used to hash (simple modular hash
      function) to cache indices.
    * A hash collision (when a block is hashed to an already occupied
      cache block) will trigger offset-based relocation to max 16
      other places, at first empty slot found policy.
    * A hash collision with all 17 possible relocating places occupied
      will evict the oldest used one (replaced by the new read block).
      A global 'use' tick is kept ticking-up per block read for this
      purpose.
    * On write cache hit, it will update the corresponding cached
      block.
    * On write cache miss, the data won't be cached.
    * Write-through policy only: never delayed write.
    * On read cache hit, it returns the cached block.
    * On read cache miss, it issues an I/O read to the kernel, and
      then caches and returns the block.
    * Currently read ahead is not implemented.  Yet, the read ahead
      logic is kept as before, i.e. using kernel's read ahead
      mechanism.

Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: Fix one bug in dcache_update_rw()]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-14 07:42:02 -08:00
Jaegeuk Kim
75cf4ec7e7 f2fs-tools: handle /sys is not mounted
Format a partition, even if /sys is not mounted.

Reported-by: David Heidelberg <david@ixit.cz>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-14 07:42:02 -08:00