As zhaowuyun reported:
we met one problem of f2fs, and found one issue of make_f2fs, so I write
this email to search for your help to confirm this issue.
The issue was found on one of Android projects. We use f2fs as the filesystem
of userdata, and make sparse userdata.img using following command, which
invoked in script mkf2fsuserimg.sh
make_f2fs -S $SIZE -f -O encrypt -O quota -O verity $MKFS_OPTS $OUTPUT_FILE
use fastboot to flash this userdata.img to device, and it encountered f2fs
problem and leading to the mount fail of data partition.
we can make this issue 100% persent reproduced by making the data partition
dirty before flashing userdata.img.
suspect that issue is caused by the dirty data in the data partition.
so we checked that source code of make_f2fs in f2fs-tool, found that when
making f2fs, it use dev_fill to do some process:
...
we change code to the following, and the issue is gone.
if (c.sparse_mode)
return dev_write(buf, offset, len);
Chao Yu:
>
> After checking the codes, IIUC, I guess the problem here is, unlike
> img2simg, mkfs.f2fs won't record zeroed block in sparse image, so
> during transforming to normal image, some critical region like
> NAT/SIT/CP.payload area weren't be zeroed correctly, later kernel may
> load obsoleting data from those region.
>
> Also, The way you provide will obviously increase the size of sparse
> file, since with it we need to write all zeroed blocks of
> NAT/SIT/CP.payload to sparse file, it's not needed.
>
> Not sure, maybe we should use sparse_file_add_fill() to record zeroed
> blocks, so that this will make formatted image more like img2simged one.
Jaegeuk:
> We have to call sparse_file_add_fill() for dev_fill().
This patch fixes to support writing fill chunk sparse file for those
zeroed blocks in mkfs.f2fs.
Reported-and-tested-by: zhaowuyun <zhaowuyun@wingtech.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As Jaegeuk reminded:
Once user updates f2fs-tools which support new fields in inode layout,
but do keep the kernel which can not support those fields, it will cause
old f2fs fail to mount new image due to root_inode's i_extra_isize value
sanity check.
So if f2fs-tools doesn't enable feature which will use new fields of
inode, we don't need to expand i_extra_isize to include them, let's just
let i_extra_isize point to the end of last valid extra field's position.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
- Print the values of xattrs that have an unknown ->e_name_index, rather
than ignoring them.
- Replace char with u8. Otherwise xattr values containing bytes >= 0x80
are printed incorrectly on platforms where char is signed.
- Only parse the encryption xattr if it has a known format number and
size. Otherwise print it as hex.
- Remove incorrect le16_to_cpu() on ->e_name_len which has type u8.
- Consolidate the code that prints the xattr value as hex.
- Constify pointers to the xattr entry.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
- Make buffers for pretty-printed filenames 341 bytes long, long enough
for 255 (NAME_MAX) base64-encoded bytes. Then print encrypted
filenames in full, base64-encoded. This will be useful for tests I'm
writing which verify the correct ciphertext is stored on-disk.
- Rename convert_encrypted_name() to pretty_print_filename(), to make it
clear that it handles unencrypted names too. Also make the output
'char' rather than 'unsigned char', as it's for printing; and remove
the unnecessary return value.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch fixes the "open failure" issue on ro disk, reported by Hagbard.
"
If I boot with kernel option "ro rootfstype=f2fs
I get the following halfway trough boot:
* Checking local filesystems ...
Info: Use default preen mode
Info: Mounted device!
Info: Check FS only due to RO
Error: Failed to open the device!
* Filesystems couldn't be fixed
"
Reported-by: Hagbard Celine <hagbardcelin@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As Hagbard Celine reported:
"
Referring to the output from the fsck running against a "ro"
filesystem, especially this line:
Info: Check FS only due to RO
As far as i can tell this says that opposed to other filesystems
running fsck against a "ro" mounted f2fs partition will never fix any
errors.
So I tried running fsck against the same partition mounted "rw":
- mount -o remount,rw /mnt/f2fstest/
- fsck.f2fs -f /dev/nvme0n1p7
Info: Force to fix corruption
Info: Mounted device!
Error: Not available on mounted device!
I might be misunderstanding something, but all this tells me that
unless one make a custom initramfs that runs fsck before root is
mounted (something no distributions has, as far as I know), fsck will
never fix an f2fs formatted root partition during boot.
If this is by design and not a bug/unintended behavior, it should be
documented somewhere least more people will experience system crashes
like mine.
All tests above done with kernel 5.0.5 and f2fs-tools 1.12.0 with
"fsck.f2fs: allow to fsck readonly image w/ -f option"-patch by Chao
Yu.
"
We try to make our fsck behavior keeping line with e2fsprogs, but w/
-f option, we can just check a RO mounted device rather repair it, so
let's fix this.
Reported-and-Tested-by: Hagbard Celine <hagbardcelin@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
this patch free ddquot in qtree_write_dquot to avoid memory leak
Signed-off-by: Xiaojun Wang <wangxiaojun11@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds to do sanity check with inode.i_inline_xattr_size,
and once it is corrupted, recover it to default value.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Added the necessary statements so that the mount procedure gracefully terminates in case the program runs out of memory.
Signed-off-by: Sotirios-Efstathios Maneas <smaneas@cs.toronto.edu>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Added a missing free statement related to each device’s path.
Signed-off-by: Sotirios-Efstathios Maneas <smaneas@cs.toronto.edu>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As reported by Aravind:
I built f2fs tools from source (at tag v1.12.0) and was able to get this backtrace in gdb:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7f8eb54 in f2fs_set_bit (nr=1041170432,
addr=0x7fffff621010 <error: Cannot access memory at address 0x7fffff621010>) at libf2fs.c:312
312 mask = 1 << (7 - (nr & 0x07));
(gdb) where
addr=0x7fffff621010 <error: Cannot access memory at address 0x7fffff621010>) at libf2fs.c:312
> [ 5338.040024] nats:8781, sits:6
> [ 5338.040027] F2FS-fs (sda2): Failed to initialize F2FS segment manager
> [ 5338.128893] nats:8781, sits:6
> [ 5338.128895] F2FS-fs (sda2): Failed to initialize F2FS segment manager
nat_count/nid/blkaddr recorded in journal may be corrupted, let's do
sanity check on them, skip loading invalid ones during build_node_manager().
Reported-by: Aravind R S <aravindet@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Avoid various compilation warnings due to strncpy:
libf2fs.c:590:33: warning: ‘snprintf’ output may be truncated before
the last format character [-Wformat-truncation=]
snprintf(rootdev, ret, "/dev/%s", buf);
../include/f2fs_fs.h:1384:2: warning: ‘strncpy’ specified bound
depends on the length of the source argument [-Wstringop-overflow=]
strncpy(buf, features, strlen(features) + 1);
f2fstat.c:243:3: warning: ‘strncpy’ output truncated before
terminating nul copying as many bytes from a string as its length
[-Wstringop-truncation]
strncpy(ptr_buf, name[i], strlen(name[i]));
This patch does not change any functionality.
Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
e.g.,
f2fs_io gc_urgent dm-4 [start/end/run] [time in sec]
This controls sysfs/gc_urgent to run f2fs_gc urgently.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
f2fs_io read [chunk_size in 4kb] [offset in chunk_size] [count] [IO] [print_nbytes] [file_path]
Read data in file_path and print nbytes
IO can be
buffered : buffered IO
dio : direct IO
Change-Id: I912adc4f443c3656ad067d29a1e2f581b79d28e6
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
f2fs_io write [chunk_size in 4kb] [offset in chunk_size] [count] [pattern] [IO] [file_path]
Write given patten data in file_path
pattern can be
zero : zeros
inc_num : incrementing numbers
rand : random numbers
IO can be
buffered : buffered IO
dio : direct IO
Change-Id: Icc866d5b9933423639d1d0e4d5e556ddf4f15feb
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
It doesn't need to ask recovering corruption on dry-run mode.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
f2fs_inode.i_mode is variable of __le16 type, during endian converison,
it should to use le16_to_cpu instead of le32_to_cpu, fix it.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Since "fsck.f2fs: integrate sanity_check_inode to __check_inode_mode",
big endian platforms have been broken due to an endianness bug.
More info: https://github.com/openwrt/openwrt/pull/1575
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
It is incorrectly taking blkaddr as len because it missed to parse
op_flags in the trace print of f2fs_trace_ios.
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As Michael reported:
after updating to f2fs-tools 1.12.0, a routine fsck of my file systems
took quite a while and output ten-thousands instances of the following
line:
> [FIX] (fsck_chk_inode_blk: 954) --> Regular: 0xXYZ reset i_gc_failures from 0x1 to 0x00
The reason is that:
We start to support reseting .i_gc_failures's value to zero in fsck since
91bb7b21f7 ("f2fs-tools: fix to reset i_gc_failures offline"), this is
because if .i_gc_failures continues increasing and exceed threshold, it
can make f2fs break atomic_write semantics during GC, so I added that
patch to avoid such condition.
But the problem here is even .i_gc_failures's value is one which was
initialized duing inode creation by old kernel, and it never be increased
by GC flow, we will still trigger such fix in fsck. I think it's not
necessary.
As Jaegeuk suggested, we introduce a new preen mode 2 to handle such
condition:
- 2: same as 0, but skip some checks for old kernel
Suggested-by: Jaegeuk Kim <jaegeuk@kernel.org>
Reported-by: Michael Laß <bevan@bi-co.net>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
To keep line with e2fsprogs, let's allow to fsck mounted image as
readonly w/ -f option.
Reported-by: Perfect Gentleman <perfect007gentleman@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This release includes:
- add android default configuration
- fix resgid/resuid
- add more sanity checks
- fix corrupted quota in clean umount
- add superblock checksum
And, there are several bug fixes in fsck.f2fs/sload.f2fs/mkfs.f2fs.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We must create a sparse file first before calling stat().
Fixes: eb9d8037ed ("f2fs-tools: avoid mounting f2fs if tools already open the device")
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Dynamic memory stored in 'xattr_node' allocated through function 'calloc'
at line 98 can be lost at line 111
Dynamic memory stored in 'inode' allocated through function 'calloc'
at line 144 can be lost at line 227
Signed-off-by: taebum81.kim <taebum81.kim@lge.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Malloc Failure occurs in 32bit Windows, when using fastboot.exe flash the
f2fs sparse image filling with up to 2G chunk size.
Signed-off-by: Gao Ming <gaoming20@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The following patch adds a few missing statements related to error checking.
Signed-off-by: Sotirios-Efstathios Maneas <smaneas@cs.toronto.edu>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
In 32-bits platform, {f,}stat on a large size file during mkfs, it will
cause EOVERFLOW error, this patch fixes to add macro definition
_FILE_OFFSET_BITS to avoid that error.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If the len is less than 24, heap buffer overflow is reported.
Signed-off-by: Randall Huang <huangrandall@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Currently, when format disk used with mkfs.f2fs, even if the volume label
is not specified, it still shows the info of volume label with no content
due to the check condition is always true, this patch fixes it.
[root@localhost home]# mkfs.f2fs -f /dev/sdb1
F2FS-tools: mkfs.f2fs Ver: 1.12.0 (2018-10-08)
Info: Disable heap-based policy
Info: Debug level = 0
Info: Label =
Info: Trim is enabled
Signed-off-by: Tiezhu Yang <kernelpatch@126.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We have to keep data for roll-forward recovery. Without this patch, we're
able to lose there-in data by quota overwrites.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Once quota file is corrupted, kernel will set CP_QUOTA_NEED_FSCK_FLAG
into checkpoint pack, this patch makes fsck supporting to detect the flag
and try to rebuild corrupted quota file.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
If sb checksum is not enabled, and cp pack is valid due to no
crc inconsistence, let's try to recover cp_payload based on
cp_pack_start_sum in cp pack.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
