Commit Graph

84 Commits

Author SHA1 Message Date
Jaegeuk Kim
b7bb88a0ba fsck|dump.f2fs: add -M to get file map
commit acd2518e26
category: bugfix
issue: #I6VAS0
CVE: NA

Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------

This option shows all the file names in the disk.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
2023-04-13 14:13:45 +08:00
Jordan Webb
7adec5bc39 f2fs-tools: Make sload.f2fs reproduce hard links
commit 747b74cb9c
category: bugfix
issue: #I6VAS0
CVE: NA

Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------

If sload.f2fs encounters a file with nr_links > 1, it will mark it
as a possible hard link by remembering the original device and
inode. When sload.f2fs creates the file, it will check if it has
already created a file for the same original device and inode. If
so, it will add the original inode to the directory and increment
the number of links to it, instead of writing a new inode.

This allows sload.f2fs to accurately reproduce a directory tree that
contains hard links, such as those created by ostree. Without this
patch, directory trees containing hard links result in the content of
the files being duplicated.

This is version 2 of the patch; it has been rebased against the dev
branch and includes a fix from Jaegeuk Kim to avoid building data
contents twice on hard linked files.

Co-authored-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: Jordan Webb <jordan@getseam.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
2023-04-13 14:13:44 +08:00
Robin Hsu
ecdcd6189d f2fs-tools:sload.f2fs compression support
commit b585244e72
category: bugfix
issue: #I6VAS0
CVE: NA

Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------

Add F2FS compression support for sload
* Support file extension filter, either default-accept or default-deny
  policy
* Support choice of compression algorithm, LZO (version 2) or LZ4
  (default)
* Support custom log of cluster size
* Support minimum number of compressed blocks per cluster (default 1).
  A cluster will not be compressed if the number can not be met.
* suuport -r (read-only) option
  This releases compressed blocks to secure free space in advance. Note that,
  all compressed files will have the immutable bit.
* Added manpage update
* Remove unecessary qbuf allocation (Jaegeuk, suggested by Satya)

Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: fix some bugs and refactor names]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>

 create mode 100644 fsck/compress.c
 create mode 100644 fsck/compress.h

Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
2023-04-13 14:13:44 +08:00
Aravind Ramesh
f8410857b7 f2fs-tools: zns zone-capacity support
NVM Express Zoned Namespace (ZNS) devices can have zone-capacity(zc) less
than the zone-size. ZNS defines a per zone capacity which can be equal
or less than the zone-size. Zone-capacity is the number of usable blocks
in the zone. If zone-capacity is less than zone-size, then the segments
which start at/after zone-capacity are considered unusable. Only those
segments which start before the zone-capacity are considered as usable
and added to the free_segment_count and free_segment_bitmap of the kernel.
In such cases, the filesystem should not write/read beyond the
zone-capacity.

Update the super block with the usable number of blocks and free segment
count in the ZNS device zones, if zone-capacity is less than zone-size.
Set reserved segment count and overprovision ratio based on the usable
segments in the zone.

Allow fsck to find the free_segment_count based on the zone-capacity and
compare with checkpoint values.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
[Jaegeuk Kim: add UNUSED to is_usable_seg()]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-20 09:08:34 -07:00
Chao Yu
2579fe8099 fsck.f2fs: correct return value
As Norbert Lange reported:

"
$ fsck.f2fs -a /dev/mmcblk0p5; echo $?
Info: Fix the reported corruption.
Info: Mounted device!
Info: Check FS only on RO mounted device
Error: Failed to open the device!
255
"

Michael Laß reminds:

"
I think the return value is exactly the problem here. See fsck(8) (
https://linux.die.net/man/8/fsck) which specifies the return values.
Systemd looks at these and decides how to proceed:

a859abf062/src/fsck/fsck.c (L407)

That means, if fsck.f2fs returns 255, then
the FSCK_SYSTEM_SHOULD_REBOOT bit is set and systemd will reboot.
"

So the problem here is fsck.f2fs didn't return correct value to userspace
apps, result in later unexpected behavior of rebooting, let's fix this.

Reported-by: Norbert Lange <nolange79@gmail.com>
Reported-by: Michael Laß <bevan@bi-co.net>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: fix FSCK_ERROR_CORRECTED]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-08-17 23:52:38 -07:00
Robin Hsu
e66e4c1f3b fsck: prevent buffer overrun in quota code
A maliciously corrupted file systems can trigger buffer overruns in
the quota code used by fsck.

To fix it, quota file sizes are checked against real allocated
block index tables (inode, direct nodes, indirect nodes, double
indirect nodes).  If the size mismatches, the quota file is considered
corrupted and will be regenerated.

Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-23 16:58:44 -07:00
Chao Yu
7a22451bc2 fsck.f2fs: fix to check validation of i_xattr_nid
Otherwise, fsck.f2fs will access invalid memory address as below:

- fsck_verify
 - dump_node
  - dump_file
   - dump_inode_blk
    - dump_xattr
     - read_all_xattrs
       - get_node_info
        access &(F2FS_FSCK(sbi)->entries[nid])

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-06 07:41:06 -07:00
Shin'ichiro Kawasaki
42326f61e9 fsck: Check write pointer consistency of open zones
On sudden f2fs shutdown, write pointers of zoned block devices can go
further but f2fs meta data keeps current segments at positions before the
write operations. After remounting the f2fs, this inconsistency causes
write operations not at write pointers and "Unaligned write command"
error is reported.

To avoid the error, have f2fs.fsck check consistency of write pointers
of open zones that current segments point to. Compare each current
segment's position and the write pointer position of the open zone. If
inconsistency is found and 'fix_on' flag is set, assign a new zone to the
current segment and check the newly assigned zone has write pointer at
the zone start. Leave the original zone as is to keep data recorded in
it.

To care about fsync data, refer each seg_entry's ckpt_valid_map to get
the last valid block in the zone. If the last valid block is beyond the
current segments position, fsync data exits in the zone. In case fsync
data exists, do not assign a new zone to the current segment not to lose
the fsync data. It is expected that the kernel replay the fsync data and
fix the write pointer inconsistency at mount time.

Also check consistency between write pointer of the zone the current
segment points to with valid block maps of the zone. If the last valid
block is beyond the write pointer position, report to indicate a bug. If
'fix_on' flag is set, assign a new zone to the current segment.

When inconsistencies are found, turn on 'bug_on' flag in fsck_verify() to
ask users to fix them or not. When inconsistencies get fixed, turn on
'force' flag in fsck_verify() to enforce fixes in following checks.

This check and fix is done twice. The first is done at the beginning of
do_fsck() function so that other fixes can reflect the current segment
modification. The second is done in fsck_verify() to reflect updated meta
data by other fixes.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-12-09 17:23:19 -08:00
Shin'ichiro Kawasaki
75bdccdf0b fsck: Check fsync data always for zoned block devices
Fsck checks fsync data when UMOUNT flag is not set. When the f2fs was not
cleanly unmouted, UMOUNT flag is not recorded in meta data and fsync data
can be left in the f2fs. The first fsck run checks fsync data to reflect
it on quota status recovery. After that, fsck writes UMOUNT flag in the
f2fs meta data so that second fsck run can skip fsync data check.

However, fsck for zoned block devices need to care fsync data for all
fsck runs. The first fsck run checks fsync data, then fsck can check
write pointer consistency with fsync data. However, since second fsck run
does not check fsync data, fsck detects write pointer at fsync data end
is not consistent with f2fs meta data. This results in meta data update
by fsck and fsync data gets lost.

To have fsck check fsync data always for zoned block devices, introduce
need_fsync_data_record() helper function which returns boolean to tell
if fsck needs fsync data check or not. For zoned block devices, always
return true. Otherwise, return true if UMOUNT flag is not set in CP.
Replace UMOUNT flag check codes for fsync data with the function call.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-12-09 17:23:19 -08:00
Shin'ichiro Kawasaki
c0ce8e646c fsck: Find free zones instead of blocks to assign to current segments
When fsck needs to assign a new area to a curreng segment, it calls
find_next_free_block() function to find a new block to assign. For zoned
block devices, fsck checks write pointer consistency with current
segments' positions. In case a curseg is inconsistent with the
write pointer of the zone it points to, fsck should assign not a new free
block but a new free zone/section with write pointer at the zone start,
so that next write to the current segment succeeds without error.

To extend find_next_free_block() function's capability to find not only
a block but also a zone/section, add new_sec flag to
find_next_free_block() function. When new_sec flag is true, skip check
for each block's availability so that the check is done with unit of
section. Note that it is ensured that one zone has one section for f2fs
on zoned block devices. Then the logic to find a new free section is good
to find a new free zone.

When fsck target devices have ZONED_HM model, set new_sec flag true to
call find_next_free_block() from move_curseg_info(). Set curseg's
alloc_type not SSR but LFS for the devices with ZONED_HM model, because
SSR block allocation is not allowed for zoned block devices. Also skip
relocate_curseg_offset() for the devices with ZONED_HM model for the
same reason.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-12-09 17:23:19 -08:00
Chao Yu
de7bcc843a f2fs-tools: fix to skip block allocation for fsynced data
Previously, we don't allow block allocation on unclean umounted image,
result in failing to repair quota system file.

In this patch, we port most recovery codes from kernel to userspace
tools, so that on unclean image, during fsck initialization, we will
record all data/node block address we may recover in kernel, and
then during allocation of quota file repair, we can skip those blocks
to avoid block use conflict.

Eventually, if free space is enough, we can repair the quota system
file on an unclean umounted image.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: remove unnecessary parameter]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-27 14:51:05 -07:00
Chao Yu
156ac96e19 f2fs-tools: fix stat update in sload/fsck
Change logic as below:
- fix to account block/node/inode stats correctly in reserve_new_block()
- check overflow in reserve_new_block()
- move stat update from f2fs_alloc_nid() to reserve_new_block()
- adjust write_checkpoint() to update stat for sload/fsck

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-27 14:51:05 -07:00
Chao Yu
9e64efe276 f2fs-tools: introduce f2fs_ra_meta_pages()
Introduce f2fs_ra_meta_pages() to readahead meta pages like we did
in kernel.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-27 14:51:05 -07:00
Daniel Rosenberg
ce64ea0815 f2fs-tools: Add support for Casefolding
This adds support for f2fs casefolding. Similarly to ext4 casefolding,
this is controlled per-folder via the +F attribute. It can be toggled on
empty directories only. It is not currently compatible with encryption,
but that will likely change.

When enabling the casefold feature, use the -C flag. The format is:
-C encoding[:flag1,flag2,etc]

Signed-off-by: Daniel Rosenberg <drosen@google.com>
[Jaegeuk Kim: print "casefold" in sb->feature]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-27 14:51:05 -07:00
Chao Yu
81bad9d11e fsck.f2fs: write checkpoint with OPU mode
This original patch was from Weichao Guo.

We may encounter both checkpoints invalid in such a case:
1. kernel writes CP A;
2. power-cut when kernel writes CP B, then CP B is corrupted;
3. fsck: load CP A, fix meta/data;
4. power-cut when fsck writes CP A in-place, then CP A is corrupted too;

To avoid both checkpoints being invalid, this patch changes to duplicate
valid checkpoint to mirror position first, and then, write fixed checkpoint
to CP #0 position.

This can make sure that, while fsck repairing, even there is sudden
power-cut, last valid checkpoint can be kept in CP #1 position.

Signed-off-by: Weichao Guo <guoweichao@huawei.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-20 11:23:58 -07:00
Chao Yu
7d4480baa9 f2fs-tools: fix to check total valid block count before block allocation
Now, kernel can catch incorrect total valid block count which is exceed
max user block count of image.

Then, generic/051,476 of fstest reports below message:

Apr 15 11:08:03 szvp000201624 kernel: [ 2533.515813] F2FS-fs (zram1): Wrong valid_user_blocks: 469505, user_block_count: 469504
Apr 15 11:08:03 szvp000201624 kernel: [ 2533.519166] F2FS-fs (zram1): Failed to get valid F2FS checkpoint

The reason is that when fsck repairs corrupted quota sysfile, it didn't
check max user block count when allocating new block for quota sysfile,
so ckpt.valid_block_count can exceed max user block count, result in
mount failure later.

Adding upper boundary check of block count in reserve_new_block() to
fix this issue.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-20 11:23:51 -07:00
Chao Yu
54244dcafd fsck.f2fs: update curseg .next_blkoff/.alloc_type preferential
If .next_blkoff is inconsistent, we can update curseg .next_blkoff
to first unused block address, and change .alloc_type to SSR
preferential, instead of move curseg to other position.

This can help to repair fuzzed image which has no more free segment.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:42:52 -07:00
Chao Yu
a423b5f91e f2fs-tools: relocate chksum_offset for large_nat_bitmap feature
For large_nat_bitmap feature, there is a design flaw:

Previous:

struct f2fs_checkpoint layout:
+--------------------------+  0x0000
| checkpoint_ver           |
| ......                   |
| checksum_offset          |------+
| ......                   |      |
| sit_nat_version_bitmap[] |<-----|-------+
| ......                   |      |       |
| checksum_value           |<-----+       |
+--------------------------+  0x1000      |
|                          |      nat_bitmap + sit_bitmap
| payload blocks           |              |
|                          |              |
+--------------------------|<-------------+

Obviously, if nat_bitmap size + sit_bitmap size is larger than
MAX_BITMAP_SIZE_IN_CKPT, nat_bitmap or sit_bitmap may overlap
checkpoint checksum's position, once checkpoint() is triggered
from kernel, nat or sit bitmap will be damaged by checksum field.

In order to fix this, let's relocate checksum_value's position
to the head of sit_nat_version_bitmap as below, then nat/sit
bitmap and chksum value update will become safe.

After:

struct f2fs_checkpoint layout:
+--------------------------+  0x0000
| checkpoint_ver           |
| ......                   |
| checksum_offset          |------+
| ......                   |      |
| sit_nat_version_bitmap[] |<-----+
| ......                   |<-------------+
|                          |              |
+--------------------------+  0x1000      |
|                          |      nat_bitmap + sit_bitmap
| payload blocks           |              |
|                          |              |
+--------------------------|<-------------+

Related report and discussion:

https://sourceforge.net/p/linux-f2fs/mailman/message/36642346/

In addition, during writing checkpoint, if large_nat_bitmap feature is
enabled, we need to set CP_LARGE_NAT_BITMAP_FLAG flag in checkpoint.

Reported-by: Park Ju Hyung <qkrwngud825@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:30:16 -07:00
Chao Yu
bdc7da10a9 sload.f2fs: introduce f2fs_sparse_initialize_meta()
This patch fixes to initialize NAT/SIT/CP.payload region in sparse
file mode for sload.

Reported-and-tested-by: zhaowuyun <zhaowuyun@wingtech.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:29:50 -07:00
Eric Biggers
04bbe69311 f2fs-tools: improve filename printing
- Make buffers for pretty-printed filenames 341 bytes long, long enough
  for 255 (NAME_MAX) base64-encoded bytes.  Then print encrypted
  filenames in full, base64-encoded.  This will be useful for tests I'm
  writing which verify the correct ciphertext is stored on-disk.

- Rename convert_encrypted_name() to pretty_print_filename(), to make it
  clear that it handles unencrypted names too.  Also make the output
  'char' rather than 'unsigned char', as it's for printing; and remove
  the unnecessary return value.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:29:50 -07:00
Chao Yu
4d120f0c13 fsck.f2fs: introduce preen mode 2
As Michael reported:

after updating to f2fs-tools 1.12.0, a routine fsck of my file systems
took quite a while and output ten-thousands instances of the following
line:

> [FIX] (fsck_chk_inode_blk: 954)  --> Regular: 0xXYZ reset i_gc_failures from 0x1 to 0x00

The reason is that:

We start to support reseting .i_gc_failures's value to zero in fsck since
91bb7b21f7 ("f2fs-tools: fix to reset i_gc_failures offline"), this is
because if .i_gc_failures continues increasing and exceed threshold, it
can make f2fs break atomic_write semantics during GC, so I added that
patch to avoid such condition.

But the problem here is even .i_gc_failures's value is one which was
initialized duing inode creation by old kernel, and it never be increased
by GC flow, we will still trigger such fix in fsck. I think it's not
necessary.

As Jaegeuk suggested, we introduce a new preen mode 2 to handle such
condition:
  - 2: same as 0, but skip some checks for old kernel

Suggested-by: Jaegeuk Kim <jaegeuk@kernel.org>
Reported-by: Michael Laß <bevan@bi-co.net>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-28 00:14:30 -08:00
Jaegeuk Kim
83c64019e8 fsck.f2fs: don't allocate new blocks on unclean shutdown
We have to keep data for roll-forward recovery. Without this patch, we're
able to lose there-in data by quota overwrites.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Junling Zheng
238fa8ccf1 fsck.f2fs: unify the updating of superblocks
Rename write_superblock() to update_superblock() and make it support updating
specified one superblock or both two superblocks, then unify all places where
sb needs to be updated.

Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Daniel Rosenberg
1ccd8313ba WIP: Add partial shrinking support to f2fs
allow 'safe' resizing leaving the metadata nearly unchanged.

Will likely need to adjust growing for the case where a fs
had previously shrunk without altering data, and the newly grown
fs would be smaller than the original. Probably makes sense to
case it out into metadata size changes, instead of fs size changes

Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2018-11-21 11:38:23 -08:00
Sheng Yong
d31e1a0d52 f2fs-tools: set namelen parameter of convert_encrypted_name as unsigned
To avoid overflow, set namelen parameter of convert_encrypted_name as
unsigned int. convert_encrypted_name() will check if namelen exceeds the
limitation.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:49:26 -07:00
Sheng Yong
356f8627c4 fsck.f2fs: check extent of inline data/dentry inode
Check extent for inline data/dentry inode. If an inode contains inline
data/dentry, it should have no extent.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=200175
Reported-by: Wen Xu <wen.xu@gatech.edu>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:49:26 -07:00
Junling Zheng
ddb87f568f dump.f2fs: only dump nat inside the specified nid range
Only dump nat info of nids inside the specified range.

Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:49:26 -07:00
Sheng Yong
df8065e529 fsck.f2fs: introduce fsck_chk_curseg_info
If curseg is an empty segment, it will not be checked. This patch
introduces fsck_chk_curseg_info() to check SIT/SSA type of cursegs
to avoid curseg corruption.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:49:26 -07:00
Jaegeuk Kim
689fd9b885 fsck.f2fs: add -O features to tune the bits
This patch add -O features for fsck.f2fs in order to tune the feature bits.
Currently, it supports -O encrypt only.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-07-13 22:46:00 -07:00
Sheng Yong
eb61637ad1 fsck.f2fs: reconnect unreachable files to lost+found
This patch introduces lost_found feature to fsck. If a file is found
unreachable by fsck. Fsck tries to reconnect the file to lost+found
directory:
  1. Scan all unreachable file inodes, ignore non-inodes ones and
     directories.
  2. Check them and fix incorrupted data to make sure filesystem
     metadata (mainly counters and main/nat bitmap) are all consistent.
  3. Reconnect these files to lost+found. If lost+found does not exist,
     create it first. During reconnecting, expand lost+found's dentry
     block automatically. Reconnected files are renamed after its ino
     number.
  4. If reconnect fails drop the node and restore filesystem metadata.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-07 14:49:38 -07:00
Yunlei He
06addd3c1e fsck.f2fs: reduce mem alloc during read sit block
This patch reduce mem alloc during read sit block

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-07 14:49:37 -07:00
Yunlong Song
ddc2d9b4ec fsck.f2fs: check and fix i_namelen to avoid double free
This checks i_namelen between inode block and its dentry.

Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-12-19 15:51:16 -08:00
Jaegeuk Kim
bb5aa5cd6c sload.f2fs: update build for android
This patch updates sload.f2fs for android build.

- do fsck() after sload() to update quota information.
- return success, if there's no source directory

  -C fs_config

  -f source directory [path of the source directory]

  -p product out directory
    : path on fs_config file for uid/gid/mode

  -s file_contexts
    : selabel file

  -t mount point [prefix of target fs path, default:/]
     : path stored in selabel/fs_config files

  -T timestamp

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-30 22:36:35 -08:00
Sheng Yong
ace53e2ff4 dump/fsck: introduce print_xattr_entry
This patch exports read_all_xattrs to allow dump/fsck to get all xattrs,
and introduces print_xattr_entry which tries to parse an xattr entry
accroding to its xattr index.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
[Jaegeuk Kim: fix configure.ac]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-11 18:01:07 -08:00
Sheng Yong
7935c4e1f0 dump.f2fs: introduce dump_xattr
This patch introduces dump_xattr to create xattrs for dumped files.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: detect header files]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-11 18:01:04 -08:00
Jaegeuk Kim
e53ad5b071 f2fs-tools: fix warnings when building in android
This patch tries to avoid all the warnings.

Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-11-06 20:07:42 -08:00
Hyojun Kim
6c6bb35c89 fsck.f2fs: support quota
This patch let fsck to check and fix quota file contents.

Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-11-06 20:07:40 -08:00
Hyojun Kim
68593f4981 f2fs-tools: f2fs_read() and f2fs_filesize_update() are added
This patch adds f2fs_read() and f2fs_filesize_update(). It also refactors
f2fs_write_block() and renamed as f2fs_write().

Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-10-30 20:41:49 -07:00
Chao Yu
dad33a1ee7 f2fs-tools: enhance on-disk inode structure scalability
This patch adds an option 'extra_attr' in mkfs for enabling v2 f2fs
inode format in kernel codes.

Also this patch makes fsck to support recognize v2 inode format,
below is v2 format description:

Original one:

struct f2fs_inode {
	...
	struct f2fs_extent i_ext;
	__le32 i_addr[DEF_ADDRS_PER_INODE];
	__le32 i_nid[DEF_NIDS_PER_INODE];
}

Extended one:

struct f2fs_inode {
        ...
        struct f2fs_extent i_ext;
	union {
		struct {
			__le16 i_extra_isize;
			__le16 i_padding;
			__le32 i_extra_end[0];
		};
		__le32 i_addr[DEF_ADDRS_PER_INODE];
	};
        __le32 i_nid[DEF_NIDS_PER_INODE];
}

Once F2FS_EXTRA_ATTR is set, we will steal four bytes in the head of
i_addr field for storing i_extra_isize and i_padding. with i_extra_isize,
we can calculate actual size of reserved space in i_addr, available
attribute fields included in total extra attribute fields for current
inode can be described as below:

  +--------------------+
  | .i_mode            |
  | ...                |
  | .i_ext             |
  +--------------------+
  | .i_extra_isize     |-----+
  | .i_padding         |     |
  | .i_prjid           |     |
  | .i_atime_extra     |     |
  | .i_ctime_extra     |     |
  | .i_mtime_extra     |<----+
  | .i_inode_cs        |<----- store blkaddr/inline from here
  | .i_xattr_cs        |
  | ...                |
  +--------------------+
  |                    |
  |    block address   |
  |                    |
  +--------------------+
  | .i_nid             |
  +--------------------+
  |   node_footer      |
  | (nid, ino, offset) |
  +--------------------+

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-07-26 11:42:49 -07:00
Chao Yu
8b4406a5bd f2fs-tools: spread struct f2fs_dentry_ptr for inline path
Use f2fs_dentry_ptr structure to indicate inline dentry structure as
much as possible, so we can wrap inline dentry with size-fixed fields
to the one with size-changeable fields. With this change, we can
handle size-changeable inline dentry more easily.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-07-17 14:35:33 -07:00
Jaegeuk Kim
8e29c2be47 f2fs-tools: avoid build warnings
This patch resolves warnings while building it in android.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-06-05 13:58:50 -07:00
Jaegeuk Kim
191573e71b mkfs.f2fs: support nat_bits feature
This patch adds nat_bits in the last one ore two 4KB-sized blocks of first
checkpoint segment. So, now a set of checkpoint *segment* consists of:

< checkpoint segment >
----------------------
| cp page #1         | 4KB
----------------------
| compacted data sum | 4KB
----------------------
| node sum  #1       | 4KB
----------------------
| node sum  #2       | 4KB
----------------------
| node sum  #3       | 4KB
----------------------
| cp page #2         | 4KB
----------------------
...
----------------------
| crc|cpver          | 8 bytes
----------------------
| full NAT bits      | # of NAT segments / 2 * 512 / 8 bytes
----------------------
| empty NAT bits     | # of NAT segments / 2 * 512 / 8 bytes
----------------------

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-02-23 10:11:41 -08:00
Jaegeuk Kim
88753dbcf3 fsck.f2fs: show parse errors neatly
This patch is just to clean up.

Cc: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-01-21 14:00:15 +08:00
Liu Xue
b88767e606 fsck.f2fs: flush curseg sit entries
Flush curseg sit entries, since type of sit entries may
be changed after move_curseg_info.

Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-23 10:12:56 -08:00
Yunlei He
4f1ded4ad2 fsck.f2fs add a simple check orphan inodes in -p 1 mode
This patch add a simple check orphan inodes in -p 1 mode

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-21 17:02:26 -08:00
Liu Xue
112bf05cb8 fsck.f2fs: rewrite entries info in segment manager
The entries info in segment manager may be not correct, since
corrupted blocks may already cleaned. So rewrite se info with
main area bitmap.

Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
[Jaegeuk Kim: remove obsolte variable in f2fs_fsck]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-02 10:58:53 -07:00
Sheng Yong
c4c65265da dump/fsck: convert encrypted file name
If fscrypt is enabled, we need to convert the encrypted file name before
printing it. So let's export convert_encrypted_name for other functions,
and make it returns the length of converted string.
This patch also changes the parameter of file_is_encrypt to f2fs_inode.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-10-31 10:59:24 -07:00
Jaegeuk Kim
390fe587b1 fsck.f2fs: support restore lost files into ./lost_found/
This patch adds to support restoring lost files into ./lost_found/.

For example,

 # fsck.f2fs /dev/sdXX

...
NID[0x87d7] is unreachable
NID[0x87d8] is unreachable
NID[0x87d9] is unreachable
[FSCK] Unreachable nat entries                        [Fail] [0x4988]
[FSCK] SIT valid block bitmap checking                [Fail]
[FSCK] Hard link checking for regular file            [Ok..] [0x0]
[FSCK] valid_block_count matching with CP             [Fail] [0x2]
[FSCK] valid_node_count matcing with CP (de lookup)   [Fail] [0x1]
[FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x4989]
[FSCK] valid_inode_count matched with CP              [Fail] [0x1]
[FSCK] free segment_count matched with CP             [Ok..] [0x1f7]
[FSCK] next block offset is free                      [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs                           [Fail]

Do you want to restore lost files into ./lost_found/? [Y/N] y

Then, you can restore unreachable files from ./lost_found/ in its best effort.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-10-13 16:57:32 -07:00
Jaegeuk Kim
bdc1a0ac8b f2fs-tools: clean up and avoid build warning
This patch is to clean up and avoid warnings when compiling aosp.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-10-12 14:57:58 -07:00
Yunlei He
4f105bbaba fsck.f2fs: modify sit dump && add nat dump
modify sit dump && add nat dump like this:

dump_sit:

segment_type(0:HD, 1:WD, 2:CD, 3:HN, 4:WN, 5:CN)

segno:      13  vblocks:512     seg_type:2      sit_pack:2

  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff
  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff
  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff
  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff

dump_nat:

nid:    3       ino:    3       offset:    0    blkaddr:   2076417      pack:2
nid: 1042       ino: 1042       offset:    0    blkaddr:   2087156      pack:2
nid:57076       ino:57076       offset:    0    blkaddr:   2071232      pack:2
nid:57118       ino:57118       offset:    0    blkaddr:   2071233      pack:2
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-06-08 10:10:38 -07:00