If a file is encrypted, its content is cipher text on the storage. So
there is no need to dump an encrypted file.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Fixing corrupted data depends on c.fix_on. If it's not set, we should not
force fixing corrupted data. So if nat entries are found invalid when
building nat_area_bitmap, we should just set c.bug_on, and fix it later.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Do not set fix_on if it is allowed by user.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch let fsck to check and fix quota file contents.
Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
This patch adds f2fs_read() and f2fs_filesize_update(). It also refactors
f2fs_write_block() and renamed as f2fs_write().
Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Found and fixed following three bugs in f2fs_write_block() function.
- Write (4096 - offset) bytes for the first block even for small count.
- For overwriting, found blkaddr is not used for writing.
- dn.idirty status can be lost by set_new_dnode().
- missing inode_checksum
Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
This patch makes mkfs to support flexible inline xattr feature, also
this enables fsck to recognize new disk-layout of inode which be of
flexible inline xattr size.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We use f2fs_finalize_device to fsync previous data in
checkpoint area before write last CP pack, in order to avoid
cp corruption in sudden-power-off case. But this function will
close the device, so this patch introduce a function to call
fsync() only.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Current design will lose recovery process when check_curseg_offset is OK.
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If flag bit to check is in 0xFFFFFF00, then the return value will be false,
this is not correct, so fix it.
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Write back last cp block in the end in case of sudden power off
during fsck process
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch introduce a new option 'inode_checksum' for enabling inode
checksum functionality in mkfs/fsck/sload.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch allows properly printing all addresses in an inode.
It is useful in commands like dump.f2fs.
Signed-off-by: Piotr Sokołowski <piosoko1@student.pg.edu.pl>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch introduce a new option 'project_quota' for enabling project
quota functionality during mkfs.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds an option 'extra_attr' in mkfs for enabling v2 f2fs
inode format in kernel codes.
Also this patch makes fsck to support recognize v2 inode format,
below is v2 format description:
Original one:
struct f2fs_inode {
...
struct f2fs_extent i_ext;
__le32 i_addr[DEF_ADDRS_PER_INODE];
__le32 i_nid[DEF_NIDS_PER_INODE];
}
Extended one:
struct f2fs_inode {
...
struct f2fs_extent i_ext;
union {
struct {
__le16 i_extra_isize;
__le16 i_padding;
__le32 i_extra_end[0];
};
__le32 i_addr[DEF_ADDRS_PER_INODE];
};
__le32 i_nid[DEF_NIDS_PER_INODE];
}
Once F2FS_EXTRA_ATTR is set, we will steal four bytes in the head of
i_addr field for storing i_extra_isize and i_padding. with i_extra_isize,
we can calculate actual size of reserved space in i_addr, available
attribute fields included in total extra attribute fields for current
inode can be described as below:
+--------------------+
| .i_mode |
| ... |
| .i_ext |
+--------------------+
| .i_extra_isize |-----+
| .i_padding | |
| .i_prjid | |
| .i_atime_extra | |
| .i_ctime_extra | |
| .i_mtime_extra |<----+
| .i_inode_cs |<----- store blkaddr/inline from here
| .i_xattr_cs |
| ... |
+--------------------+
| |
| block address |
| |
+--------------------+
| .i_nid |
+--------------------+
| node_footer |
| (nid, ino, offset) |
+--------------------+
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Use f2fs_dentry_ptr structure to indicate inline dentry structure as
much as possible, so we can wrap inline dentry with size-fixed fields
to the one with size-changeable fields. With this change, we can
handle size-changeable inline dentry more easily.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If f2fs is unmounted with the -l option, the process can still use the
file system. In this case, the command mkfs.f2fs, fsck.f2fs, dump.f2fs,
defrag.f2fs, resize.f2fs and sload.f2fs failed due to the device is in
use by the system. At the moment f2fs is already not mounted, so it is
wrong to print error message "Error: Not available on mounted device!",
this patch fixes it.
[root@localhost home]# mkfs.f2fs -f /dev/sdb1
[root@localhost home]# mkdir -p /mnt/f2fs_mnt_point
[root@localhost home]# mount -t f2fs /dev/sdb1 /mnt/f2fs_mnt_point/
[root@localhost home]# cd /mnt/f2fs_mnt_point/
[root@localhost f2fs_mnt_point]# umount /dev/sdb1
umount: /mnt/f2fs_mnt_point: target is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))
[root@localhost f2fs_mnt_point]# umount -l /dev/sdb1
[root@localhost f2fs_mnt_point]# umount /dev/sdb1
umount: /dev/sdb1: not mounted
[root@localhost f2fs_mnt_point]# mkfs.f2fs -f /dev/sdb1
F2FS-tools: mkfs.f2fs Ver: 1.8.0 (2017-04-28)
Info: Disable heap-based policy
Info: Debug level = 0
Info: Label =
Info: Trim is enabled
Error: In use by the system!
Error: Not available on mounted device!
[root@localhost f2fs_mnt_point]# fsck.f2fs /dev/sdb1
Error: In use by the system!
Error: Not available on mounted device!
[root@localhost f2fs_mnt_point]# dump.f2fs /dev/sdb1
Error: In use by the system!
Error: Not available on mounted device!
[root@localhost f2fs_mnt_point]# defrag.f2fs /dev/sdb1
Error: In use by the system!
Error: Not available on mounted device!
[root@localhost f2fs_mnt_point]# resize.f2fs /dev/sdb1
Error: In use by the system!
Error: Not available on mounted device!
[root@localhost f2fs_mnt_point]# sload.f2fs /dev/sdb1
Error: In use by the system!
Error: Not available on mounted device!
Signed-off-by: Tiezhu Yang <kernelpatch@126.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
During mkfs, once we have issued discard for all device, we can set
CP_TRIMMED_FLAG in cp pack, this can make kernel module to record
undiscard blocks correctly, it will help to avoid unneeded discard.
Also this patch makes f2fs-tools be with the ability of recognizing
that new flag.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
segno and blk_off were read from input image without sanity check. This
could lead to buffer overflow when accessing internal arrays like SIT
sentries and seg_entry cur_valid_map.
Signed-off-by: Jin Qian <jinqian@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
cp_payload is not sanity checked from input image. A invalid size
can cause buffer overflow when reading checkpoint blks into memory.
Signed-off-by: Jin Qian <jinqian@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
crafted malicious f2fs partition can fill an out of bound blk_off,
which cause overflow when accessing summary block entries.
Signed-off-by: Jin Qian <jinqian@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
make sure segment count in super block doesn't exceed F2FS_MAX_SEGMENT.
Signed-off-by: Jin Qian <jinqian@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
blk_off is read from image. Attacker can construct an image with big
blk_off that trigger overflow on se->cur_valid_map.
Signed-off-by: Jin Qian <jinqian@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
It seems the new created directory always includes inline dentries,
let sload.f2fs supporting it.
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The dir_level in the filesystem maybe larger than zero,
supports it.
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
An encrypted directory under an unencrypted directory, the filename isn't encrypted.
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The following messages is better than before.
Before,
# fsck.f2fs -d 1
Error: Need argument for -d
Now,
# fsck.f2fs -d 1
Info: Debug level = 1
Error: Device not specified
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Showing directory tree reuses dbg_lv that if setting "-t",
fsck.f2fs does not show any others.
Users may want much information include the debug info and directory tree.
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The crc_offset towards or beyond the end of block is wrong,
sanity check it.
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch fix a resize faill for sector size mismatch.
The initial fs comes from an image file, with sector size
512KB, and the block device sector size is equal to block
size.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
resize.f2fs rebuilds a new checkpoint at the new position. Its
checkpoint_ver should be increased.
Signed-off-by: Shuoran Liu <liushuoran@huawei.com>
Signed-off-by: Weichao Guo <guoweichao@huawei.com>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch removes stale dentries remained by hash conversion issue which
happened long time ago.
Fixes: aafe2ca3dc ("fsck.f2fs: fix wrong hash_code made by previous buggy code")
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If the segment type and sum footer type are node, but its
sit type is corrupted as cold data, this will always trigger
type error crash in do_garbage_collect but not get fixed by fsck,
so fix seg entry type when it is not DATA.
Reported-by: KARBOWSKI Piotr <piotr.karbowski@gmail.com>
Signed-off-by: Liu Xue <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch allows fsck run -p without argument. So we could use -p as
-p, -p 0, -p 1. '-p' and '-p 0' have the same meaning as '-a'. '-p 1'
check more meta data than '-a'.
Reported-by: KARBOWSKI Piotr <piotr.karbowski@gmail.com>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
It is reported that fsck.f2fs behaves abnormally when running on MIPS32
rel 2 big endian cpu, since incorrect endianness. So let's correct all
endianess issues of f2fs-tools.
Reported-by: <k@vodka.home.kg>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We should not check the consistency between i_name and its dentry for directory.
The i_name is used only for normal file recovery when conducting roll-forward
recovery.
Moreover, this condition was totally missing encryption case which is breaking by
1. encrypt ./test
2. mkdir ./test/a
3. mkdir ./test/b
4. rename ./test/a ./test/b
-> fsck.f2fs reports wrong inconsistency in terms of i_name.
Reported-by: liushuoran <liushuoran@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch remove unused parameters in these three functions:
get_new_sb, migrate_main and rebuild_superblock
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If the offset passed in migrate_ssa is not zero, it means that there're
offset segments of old main will disappear after migrating, then there're
offset blocks of old ssa should be invalidated and removed accordingly.
So, the number of moved ssa blocks should be: TOTAL_SEGS(sbi) - offset,
and the expanded summary, which is filled with zero_blocks, should start
from: new_sum_blkaddr + TOTAL_SEGS(sbi) - offset.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Flush curseg sit entries, since type of sit entries may
be changed after move_curseg_info.
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch add a simple check orphan inodes in -p 1 mode
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds an option to specify multiple devices for an f2fs instance.
Up to 7 devices in addition to the default device can be added.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Refactor build_nat_area_bitmap, move out lookup_nat_in_journal from the
dual loops. Instead of looking up all nids in journals, we traverse nat
journals, whose entries are used to replace the NAT entries with the
corresponding nid to reduce the time build_nat_area_bitmap costs.
For "fsck -p 1", it reduces over half of the time.
Empty Fragmented Entirely
32G(ms):
original 625.8 727.3
improved 226.3 298.2
64G(ms):
original 1159 1637.7
improved 515.8 676.7
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch add a judgment after calloc avoid memory alloc
failure
Signed-off-by: Yunlei He <heyunlei@huawei.com>
[Jaegeuk Kim: remove unneeded memset to avoid build warning]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
When we move whole main area into new expanded area, we do not change its
segment numbers from sit and ssa.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Add SSR-based allocation when free segment reach the limit of
reserved segments.
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The entries info in segment manager may be not correct, since
corrupted blocks may already cleaned. So rewrite se info with
main area bitmap.
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
[Jaegeuk Kim: remove obsolte variable in f2fs_fsck]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The BLKZONED feature is mandatory for host-managed zoned block
devices. So check that it is set.
Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
SMR stands for "Shingled Magnetic Recording" which makes sense
only for hard disk drives (spinning rust). The ZBC/ZAC standards
enable management of SMR disks, but solid state drives may also
support those standards. So replace "SMR" with "zoned block device"
to avoid a HDD centric terminology.
In particular, rename the HMSMR feature to BLKZONED.
Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The first segment is already checked, so there is no need to check it
again if we want to make sure the whole section is freed.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If fscrypt is enabled, we need to convert the encrypted file name before
printing it. So let's export convert_encrypted_name for other functions,
and make it returns the length of converted string.
This patch also changes the parameter of file_is_encrypt to f2fs_inode.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Some systems do not have libselinux. Allow to build sload.f2fs without
support for selinux.
Signed-off-by: Christian Hesse <mail@eworm.de>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds to support restoring lost files into ./lost_found/.
For example,
# fsck.f2fs /dev/sdXX
...
NID[0x87d7] is unreachable
NID[0x87d8] is unreachable
NID[0x87d9] is unreachable
[FSCK] Unreachable nat entries [Fail] [0x4988]
[FSCK] SIT valid block bitmap checking [Fail]
[FSCK] Hard link checking for regular file [Ok..] [0x0]
[FSCK] valid_block_count matching with CP [Fail] [0x2]
[FSCK] valid_node_count matcing with CP (de lookup) [Fail] [0x1]
[FSCK] valid_node_count matcing with CP (nat lookup) [Ok..] [0x4989]
[FSCK] valid_inode_count matched with CP [Fail] [0x1]
[FSCK] free segment_count matched with CP [Ok..] [0x1f7]
[FSCK] next block offset is free [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs [Fail]
Do you want to restore lost files into ./lost_found/? [Y/N] y
Then, you can restore unreachable files from ./lost_found/ in its best effort.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch should fix the below bug.
Info: checkpoint state = 4 : compacted_summary sudden-power-off
[FIX] (is_valid_ssa_data_blk: 301) --> Set data summary 0x1039 -> [0x3] [0x0] [0x0]
[ASSERT] (fsck_chk_data_blk:1459) --> SIT bitmap is 0x0. blk_addr[0x20b6d8]
[ASSERT] (__chk_dentries:1255) --> Bad dentry 0x2 with invalid NID/ino 0x1644b164
[FIX] (__chk_dentries:1258) --> Clear bad dentry 0x2 with bad ino 0x1644b164
[ASSERT] (__chk_dentries:1269) --> Bad dentry 0x0 with unexpected ftype 0x2c
[FIX] (__chk_dentries:1272) --> Clear bad dentry 0x4 with bad ftype 0x2c
[ASSERT] (__chk_dentries:1255) --> Bad dentry 0x5 with invalid NID/ino 0x49000000
[FIX] (__chk_dentries:1258) --> Clear bad dentry 0x5 with bad ino 0x49000000
[ASSERT] (__chk_dentries:1255) --> Bad dentry 0x7 with invalid NID/ino 0x67706a2e
[FIX] (__chk_dentries:1258) --> Clear bad dentry 0x7 with bad ino 0x67706a2e
[ASSERT] (__chk_dentries:1255) --> Bad dentry 0x8 with invalid NID/ino 0x4a437044
[FIX] (__chk_dentries:1258) --> Clear bad dentry 0x8 with bad ino 0x4a437044
[ASSERT] (__chk_dentries:1269) --> Bad dentry 0x0 with unexpected ftype 0x0
[FIX] (__chk_dentries:1272) --> Clear bad dentry 0xf with bad ftype 0x0
[ASSERT] (__chk_dentries:1255) --> Bad dentry 0x20 with invalid NID/ino 0x4a150009
[FIX] (__chk_dentries:1258) --> Clear bad dentry 0x20 with bad ino 0x4a150009
[ASSERT] (__chk_dentries:1255) --> Bad dentry 0x21 with invalid NID/ino 0x1800094a
[FIX] (__chk_dentries:1258) --> Clear bad dentry 0x21 with bad ino 0x1800094a
*** buffer overflow detected ***: fsck.f2fs terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7338f)[0x7fa56b17b38f]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7fa56b212c9c]
/lib/x86_64-linux-gnu/libc.so.6(+0x109b60)[0x7fa56b211b60]
fsck.f2fs[0x4032f6]
fsck.f2fs[0x4063f7]
fsck.f2fs[0x406d09]
fsck.f2fs[0x4046b8]
fsck.f2fs[0x405378]
fsck.f2fs[0x405d86]
fsck.f2fs[0x402071]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fa56b129ec5]
Reported-by: Mirh <mirh@protonmail.ch>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch porting avoid unneeded loop in build_sit_entries to fsck
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Acked-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Call current_nat_addr() to calculate nat block address.
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch check sector size if formatted from a regular file,
write the sector info to superblock from device info.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch replace the overprovision calculation method from
mkfs tool.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch implements loading files into the existing partition.
For example,
# sload.f2fs -f ./ /dev/sdb1
Then, all the directories and files will be loaded into /dev/sdb1.
By default, newly files should have inline_data and inline_xattr, if possible.
Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
Signed-off-by: Liu Shuoran <liushuoran@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Now user can expand existing partition with resize.f2fs.
Currently, it doesn't support shrink an image.
For example,
# resize.f2fs -t [# of sectors] [image]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch detects any corrupted superblock and fix misalignment when it finds,
which is synced with the f2fs kernel module.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Otherwise, we get a double free error by:
In fsck/main.c,
545 out_err:
546 if (sbi->ckpt)
547 free(sbi->ckpt);
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
All nat entries are cached during building nat_area_bitmap, so that, in
fsck_chk_meta, we can get and check blk_addr and ino directly, to see if
they are in the valid range. Also, blk_addr is checked to see if the block
is valid in sit's valid maps.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The nid and ino of an inode are the same. So during building
nat_area_bitmap, we can know which nat entry represents an
inode, thus, we can count the number of inodes.
Then in fsck_chk_meta, the amount of inodes can be compared
with that recorded in CP.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch introduces a new option '-p' to do more checks on NAT/SIT areas.
'-p' has 2 levels: level 1 has the same sematics as '-a'; level 2 checks
NAT/SIT counters to see if they matches the status in SB and CP.
A new function, fsck_chk_meta, is called by '-p 1' to implement these
comparsion. If errors are detected, fix_on is set, which means fsck will
do a 'fsck -f' immediately.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Commit 4ea4f1db ("fsck.f2fs: large volume support") added the support
for large volume over about 3TB. The "cp_payload" is added to start_blk,
but not removed from orphan_blkaddr.
Fixes: 4ea4f1db ("fsck.f2fs: large volume support")
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
There are some issues in original approach:
- memory allocated in i_extent.map will leak
- do not convert fields in i_ext from on-disk format to cpu format
- do not support checking file offset with extent info
This patch refactors the flow for fixing above issues and supporting
file offset check.
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch enables fsck.f2fs to detect incorrect position where dirent
locates in an hierarchical hash structure directory.
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Commit 843f5b9388 ("fsck.f2fs: check ino of an inode") checks
nat_entry->ino for inode. However, the checking is irrelevant to node
type, and if ino of other kinds of nodes gets corrupted, fsck.f2fs
cannot detect it. This patch fixes the commit.
Fixes: 843f5b9388 ("fsck.f2fs: check ino of an inode")
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Struct f2fs_summary_block has been changed in upstream commits:
- commit 8f1dbbbbdfe9 ("f2fs: introduce lifetime write IO statistics")
- commit dfc08a12e49a ("f2fs: introduce f2fs_journal struct to wrap journal info")
This patch does the sync work for f2fs_summary_block structure between
f2fs-utils and f2fs kernel module.
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Fix incorrect block_addr of node or meta inode into 0x1 during
build_nat_area_bitmap().
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Liu Xue <liuxueliu.liu@huawei.com>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Correctly calculate the total FS size in init_sb_info(), for sector
size may not be 512 bytes.
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Check valid values for '.' and '..', such as ino, hash_code, and the
number of dot or dotdot in the directory.
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Yong Sheng <shengyong1@huawei.com>
Reviewed-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The first nat entry must be null. If it is corrupted, we set its bit in
nat_area_bitmap, without increasing valid_nat_entry_cnt, fsck_verify will
nullify it at last.
Reported-by: Cheng Yinchao <chengyinchao@huawei.com>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
For indirect/double-indirect node, if the blk_addr in its nat entry is
corrupted, fsck could figure this out and nullify the nat entry. However,
the indirect/double-indirect node still keeps the corrupted nid. As a
result, fsck reports valid blkaddr but fixes nothing each time during
scanning all nodes, like:
=============================
[ASSERT] (sanity_check_nid: 356) --> blkaddres is not valid. [0x0]
delete in.nid[i] = 0;
[ASSERT] (sanity_check_nid: 356) --> blkaddres is not valid. [0x0]
delete in.nid[i] = 0;
[FSCK] Unreachable nat entries [Ok..] [0x0]
[FSCK] SIT valid block bitmap checking [Ok..]
[FSCK] Hard link checking for regular file [Ok..] [0x0]
[FSCK] valid_block_count matching with CP [Ok..] [0xa51b]
[FSCK] valid_node_count matcing with CP (de lookup) [Ok..] [0x98]
[FSCK] valid_node_count matcing with CP (nat lookup) [Ok..] [0x98]
[FSCK] valid_inode_count matched with CP [Ok..] [0x7b]
[FSCK] free segment_count matched with CP [Ok..] [0x1a]
[FSCK] next block offset is free [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs [Fail]
=============================
So let's clean it.
Reported-by: Chen Yinchao <chengyinchao@huawei.com>
Signed-off-by: Liu Xue <liuxueliu.liu@huawei.com>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If the node is an inode, its ino in the nat entry should be equal to the
ino in its node footer. Otherwise, we drop the file.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If a nid is valid, the ino in its nat entry should record its parent nid.
However if the ino is zero, we should drop the node.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
In some case, we need confirm node or data blkaddr is ok at first,
and then fix the conflict in ssa block. If the node blkaddr happen to
locate in data type segment caused by something error, may be we 'd
better ignore it, and it's same with data blkaddr.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The invalid checkpoin is freed in validate_checkpoint().
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If fsck.f2fs reports a bug, move_curseg_info tries to change the current
segment info.
When it changes the new summary block, it overwrites the existing journal
entries.
This patch fixes it not to overwrite journal entry space.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Previously SSA is updated if it is not included in current segment info.
But, defrag.f2fs doesn't handle current segment info during the process, and
instead lastly update the whole current segment info at a time.
So, we need to update summary entries all the time.
Otherwise, we can lose the SSA entry.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
When moving data or node blocks, it changes current segment information
dynamtically. Meanwhile, its journal entry space is recovered by old
stale data.
This patch makes sure that its journal space is zeroed out.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch fixes to store volume label as utf16 correctly.
Many conversion codes are copied from exfat-tools.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Deal with realloc failure to avoid memory leak and memory free,
and assert realloc size to avoid double free for tree_mark.
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This tool tries to move the valid blocks ranging from blkaddr to blkaddr + len
to targeted blkaddr with a direction like expand or shrink.
The option includes:
-d debug level [default:0]
-s start block address [default: main_blkaddr]
-l length [default:512 (2MB)]
-t target block address [default: main_blkaddr + 2MB]
-i set direction as shrink [default: expand]
For example,
# defrag.f2fs -s 0x100 -l 0x10 -t 0x4000 /dev/sdb1
This will move data blocks between 0x100 and 0x110 to the right side of
0x4000 space.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch checks alloc_type of current segment type.
If it is LFS, the last of segment should have no valid block.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Deal with realloc failure to avoid memory leak and memory free.
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This avoids the following warning.
mount.c:783:27: warning: SM_I is static but used in inline function
check_seg_range which is not static.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Previously, check_sit_types didn't handle different numbers of active logs and
SSR cases. But, it didn't cause any problem since fixing sit types is harmless.
Nevertheless, we still don't need to change them all.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If a directory has no dot and dotdot dentries, fsck.f2fs sets inline_dots for
the inode so that f2fs module can handle that properly.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If a child is a directory, we should increase parent's i_links.
Previously, it counts direct dentry blocks excluding indirect blocks.
This patch fixes to count child_cnt correctly in order to get precise i_links.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If some of children including directories are corrupted, we should not include
them into i_links of their parent.
This patch fixes that.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Previously, if data and node summary was corrupted, the block was deallocated.
But, this patch fixes their summary first, so that we can keep their blocks.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If fsck.f2fs found some orphan blocks, previously fsck.f2fs drops them and
fixes checkpoint, resulting in false alarm on corruption.
This patch preserves orphan blocks to avoid such the alarms.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If i_links is wrong, we should not check block count, since it doesn't count
correctly which results in changing the block count to 1.
This patch fixes that.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Each inode has its filename inside inode block.
For directory, the name should be matched all the time.
In other cases, it shouldn't, since there is able to be linked to other file
names.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The current build system fails to `make distcheck` correctly
for two reasons: 1) Some header files are not listed in the
source files for fsck.f2fs and mkfs.f2fs, and so don't make it
into the dist tarball. 2) By setting a default prefix in
configure.ac, the mock install doesn't add the correct prefix
and instead tries to install directly to the file system.
This patch corrects those problems and also adds an autogen.sh
script for convenience.
Signed-off-by: Anthony G. Basile <blueness@gentoo.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Previously, only data blocks locating in its inode block could be fixed.
This patch investigates direct node blocks to fix indices too.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Since f2fs support large sector size in commit 55cf9cb63f0e "f2fs: support large
sector size", block device with sector size of 512/1024/2048/4096 bytes can be
supported.
But fsck.f2fs still use default F2FS_LOG_SECTOR_SIZE/F2FS_LOG_SECTORS_PER_BLOCK to
verify related data in f2fs image, it's wrong, let's fix this issue in this patch.
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch is to show the checkpoint flag.
With this flag, we can guess the last status of the partition.
And, this can be used by xfstests.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
One of the cases would update the dentries but not set
fixed=1 causing the update to not be written out.
Change-Id: I111d599f93f7d5306dacd7d62da946fdb5c262ee
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch remains user specified triggering information in superblock.
Then, if the information was changed, fsck.f2fs is triggered.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This is a followup from 056e4b04fc
Now the blk is marked as fixed so that the callers can write it back.
Change-Id: I99c94bc39e772e0d0baab81bd741ce55416ce0e6
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If a dentry has a bad ino (fsck_chk_nod_blk() fails)
and has a name len of 0, then __chk_dentries() will end up stuck:
....
[__chk_dentries: 663] [ 4]-[0x0] name[] len[0x0] ino[0x5f1710dc] type[0xd1]
[ASSERT] (sanity_check_nid: 184) --> nid is not valid. [0x5f1710dc]
[__chk_dentries: 663] [ 4]-[0x0] name[] len[0x0] ino[0x5f1710dc] type[0xd1]
[ASSERT] (sanity_check_nid: 184) --> nid is not valid. [0x5f1710dc]
[__chk_dentries: 663] [ 4]-[0x0] name[] len[0x0] ino[0x5f1710dc] type[0xd1]
[ASSERT] (sanity_check_nid: 184) --> nid is not valid. [0x5f1710dc]
....
This change tries to identify bad dentries:
- is the ino a valid NID?
- is the FILE_TYPE a happy type?
Change-Id: I5fb2f3869c96f2c928baaace148de1af102e558e
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The previous f2fs remained wrong hash_code for international characters.
So, fsck.f2fs should fix that hash_code with current valid names.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch summarizes the usage of options.
-a : auto_fix, fix corruption, only if f2fs reported some potential errors
-f : force, fix corruption entire partition
None : prompt, if fsck.f2fs detets any corruption
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds an auto_fix feature which fixes inconsistency of f2fs images.
E.g.,
With this option, -a, fsck.f2fs tries to fix inconsistency only if its valid
checkpoint has CP_FSCK_FLAG, written by previous bug_on cases.
So, normally it does fix nothing, so that there is no performance regression.
But, if a sort of corruption was reported by the f2fs module, this tries to
fix potential corrupted partition.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch skips initial verfication on SIT entries, which checks the number of
valid blocks from its bitmap.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds supporting dump_file, which can extract a file from image.
You can simply select [yes|no] when doing dump.f2fs -i [inode number] [img].
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch support large volume over about 3TB.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
[Jaegeuk Kim: add missing cp_payload in f2fs_super_block]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
* removed unused includes.
* removed unused parameters.
* Fixed a bunch of warnings around:
int i;
if (i < some_uint) ...
and
u32 x;
...
if (x < 0) return error;
* Protect BLKDISCARD usage if it is not available.
Change-Id: Iede035b1beb2df01c961589a69aff47a5258ecd2
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Let's move miscellaneous utilities into tools directory.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Add inline data check
Signed-off-by: Weihong Xu <weihong.xu@intel.com>
Signed-off-by: Huajun Li <huajun.li@intel.com>
Signed-off-by: Haicheng Li <haicheng.li@linux.intel.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Previously, fsck handles only the orphan inode block excpet its indirect node
blocks.
This causes false-alarm, unreachable nat entries [xxx].
This patch fixes to cover indirect node blocks in an orphan inode.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
The device files use i_addr or i_nid fields for their rdev numbers.
So, fsck should skip to check block addresses inside of them.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
The fibmap tool is for debugging f2fs especially.
CC: Changman Lee <cm224.lee@samsung.com>
CC: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch shows stat information about a file with fragmented state.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch adds debug message to confirm xattr block is checked.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
In the case of lazy umount, "umount -l", some processes are able to use the
file system even if its mountpoint was disconnected.
At this moment, we should not allow mkfs.f2fs.
This patch adds this condition check.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch supports both hex and decimal when using nid as option.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
The debug message is too verbose therefore decrease message.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This is to get physical location of file data.
We can check the percentage of fragmentation on a file.
Note that it uses fsync to get exact blkaddr.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch is to dump inode which blkaddr belongs in.
Usage:
dump.f2fs /dev/sdx -b blk_addr (in 4KB)
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
fsck.f2fs checks file system consistency, but does not repair a broken
file system yet.
dump.f2fs shows the information of a specific inode and makes dump file
of SSA and SIT.
f2fs checks file system consistency as follows:
o When data about used area and its metadata are identical,
f2fs is considered consistent. To verify such consistency, we use
three bitmaps: nat_area_bitmap, sit_area_bitmap, and main_area_bitmap.
First, each bit in nat_area_bitmap corresponds to a nid in NAT.
Second, each bit in sit_area_bitmap corresponds to a valid block in a
segment. This bitmap is same to the total valid_map of f2fs_sit_entries
in SIT.
Last, each bit in main_area_bitmap corresponds to a block in main area
except meta area.
After a consistency check of each block, we set or clear the
corresponding bit of each bitmap.
From the root node, we start consistency check. The verified
information varies according to block type.
1. NODE
- Read information of node block from NAT
- Check if block address is allocated using node info.
- Check if the type of f2fs_summary related to nid in SSA is NODE.
- Update the corresponding bit in nat_area_bitmap.
- Update the corresponding bit in sit_area_bitmap.
- Set the corresponding bit in main_area_bitmap to 1.
- Then, read node block. According to its attribute, explore
inode/direct node/indirect node/double indirect node
recursively.
- If it is an inode block, we also check its xattr and hard link.
2. DATA
- Check if the type of f2fs_summary related to nid in SSA is DATA.
- Set the corresponding bits of sit_area_bitmap and
main_area_bitmap to visited
- If it is a dentry block, traverse each dentries that may be
regular
file or directory. At this time, it will check inode block again.
Finally, we verify whether
- every nat_area_bitmap is visited
- any unreachable hard link exists
- values of sit_area_bitmap and main_area_bitmap are identical
- total_valid_block_count/node_count/inode_count are correct
Usage:
o fsck.f2fs
# fsck.f2fs /dev/sdx
options:
-d debug level [default:0]
o dump.f2fs
# dump.f2fs -i [ino] /dev/sdx
# dump.f2fs -s 0~-1 /dev/sdx (SIT dump)
# dump.f2fs -a 0~-1 /dev/sdx (SSA dump)
options:
-d debug level [default:0]
-i inode no (hex)
-s [SIT dump segno from #1~#2 (decimal), for all 0~-1]
-a [SSA dump segno from #1~#2 (decimal), for all 0~-1]
Note: To use dump.f2fs, please run make install or ln -s fsck.f2fs
dump.f2fs
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Byoung Geun Kim <bgbg.kim@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
