Commit Graph

46 Commits

Author SHA1 Message Date
Chao Yu
dad33a1ee7 f2fs-tools: enhance on-disk inode structure scalability
This patch adds an option 'extra_attr' in mkfs for enabling v2 f2fs
inode format in kernel codes.

Also this patch makes fsck to support recognize v2 inode format,
below is v2 format description:

Original one:

struct f2fs_inode {
	...
	struct f2fs_extent i_ext;
	__le32 i_addr[DEF_ADDRS_PER_INODE];
	__le32 i_nid[DEF_NIDS_PER_INODE];
}

Extended one:

struct f2fs_inode {
        ...
        struct f2fs_extent i_ext;
	union {
		struct {
			__le16 i_extra_isize;
			__le16 i_padding;
			__le32 i_extra_end[0];
		};
		__le32 i_addr[DEF_ADDRS_PER_INODE];
	};
        __le32 i_nid[DEF_NIDS_PER_INODE];
}

Once F2FS_EXTRA_ATTR is set, we will steal four bytes in the head of
i_addr field for storing i_extra_isize and i_padding. with i_extra_isize,
we can calculate actual size of reserved space in i_addr, available
attribute fields included in total extra attribute fields for current
inode can be described as below:

  +--------------------+
  | .i_mode            |
  | ...                |
  | .i_ext             |
  +--------------------+
  | .i_extra_isize     |-----+
  | .i_padding         |     |
  | .i_prjid           |     |
  | .i_atime_extra     |     |
  | .i_ctime_extra     |     |
  | .i_mtime_extra     |<----+
  | .i_inode_cs        |<----- store blkaddr/inline from here
  | .i_xattr_cs        |
  | ...                |
  +--------------------+
  |                    |
  |    block address   |
  |                    |
  +--------------------+
  | .i_nid             |
  +--------------------+
  |   node_footer      |
  | (nid, ino, offset) |
  +--------------------+

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-07-26 11:42:49 -07:00
Chao Yu
8b4406a5bd f2fs-tools: spread struct f2fs_dentry_ptr for inline path
Use f2fs_dentry_ptr structure to indicate inline dentry structure as
much as possible, so we can wrap inline dentry with size-fixed fields
to the one with size-changeable fields. With this change, we can
handle size-changeable inline dentry more easily.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-07-17 14:35:33 -07:00
Jaegeuk Kim
8e29c2be47 f2fs-tools: avoid build warnings
This patch resolves warnings while building it in android.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-06-05 13:58:50 -07:00
Jaegeuk Kim
191573e71b mkfs.f2fs: support nat_bits feature
This patch adds nat_bits in the last one ore two 4KB-sized blocks of first
checkpoint segment. So, now a set of checkpoint *segment* consists of:

< checkpoint segment >
----------------------
| cp page #1         | 4KB
----------------------
| compacted data sum | 4KB
----------------------
| node sum  #1       | 4KB
----------------------
| node sum  #2       | 4KB
----------------------
| node sum  #3       | 4KB
----------------------
| cp page #2         | 4KB
----------------------
...
----------------------
| crc|cpver          | 8 bytes
----------------------
| full NAT bits      | # of NAT segments / 2 * 512 / 8 bytes
----------------------
| empty NAT bits     | # of NAT segments / 2 * 512 / 8 bytes
----------------------

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-02-23 10:11:41 -08:00
Jaegeuk Kim
88753dbcf3 fsck.f2fs: show parse errors neatly
This patch is just to clean up.

Cc: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-01-21 14:00:15 +08:00
Liu Xue
b88767e606 fsck.f2fs: flush curseg sit entries
Flush curseg sit entries, since type of sit entries may
be changed after move_curseg_info.

Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-23 10:12:56 -08:00
Yunlei He
4f1ded4ad2 fsck.f2fs add a simple check orphan inodes in -p 1 mode
This patch add a simple check orphan inodes in -p 1 mode

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-21 17:02:26 -08:00
Liu Xue
112bf05cb8 fsck.f2fs: rewrite entries info in segment manager
The entries info in segment manager may be not correct, since
corrupted blocks may already cleaned. So rewrite se info with
main area bitmap.

Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
[Jaegeuk Kim: remove obsolte variable in f2fs_fsck]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-02 10:58:53 -07:00
Sheng Yong
c4c65265da dump/fsck: convert encrypted file name
If fscrypt is enabled, we need to convert the encrypted file name before
printing it. So let's export convert_encrypted_name for other functions,
and make it returns the length of converted string.
This patch also changes the parameter of file_is_encrypt to f2fs_inode.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-10-31 10:59:24 -07:00
Jaegeuk Kim
390fe587b1 fsck.f2fs: support restore lost files into ./lost_found/
This patch adds to support restoring lost files into ./lost_found/.

For example,

 # fsck.f2fs /dev/sdXX

...
NID[0x87d7] is unreachable
NID[0x87d8] is unreachable
NID[0x87d9] is unreachable
[FSCK] Unreachable nat entries                        [Fail] [0x4988]
[FSCK] SIT valid block bitmap checking                [Fail]
[FSCK] Hard link checking for regular file            [Ok..] [0x0]
[FSCK] valid_block_count matching with CP             [Fail] [0x2]
[FSCK] valid_node_count matcing with CP (de lookup)   [Fail] [0x1]
[FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x4989]
[FSCK] valid_inode_count matched with CP              [Fail] [0x1]
[FSCK] free segment_count matched with CP             [Ok..] [0x1f7]
[FSCK] next block offset is free                      [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs                           [Fail]

Do you want to restore lost files into ./lost_found/? [Y/N] y

Then, you can restore unreachable files from ./lost_found/ in its best effort.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-10-13 16:57:32 -07:00
Jaegeuk Kim
bdc1a0ac8b f2fs-tools: clean up and avoid build warning
This patch is to clean up and avoid warnings when compiling aosp.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-10-12 14:57:58 -07:00
Yunlei He
4f105bbaba fsck.f2fs: modify sit dump && add nat dump
modify sit dump && add nat dump like this:

dump_sit:

segment_type(0:HD, 1:WD, 2:CD, 3:HN, 4:WN, 5:CN)

segno:      13  vblocks:512     seg_type:2      sit_pack:2

  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff
  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff
  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff
  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff  ff

dump_nat:

nid:    3       ino:    3       offset:    0    blkaddr:   2076417      pack:2
nid: 1042       ino: 1042       offset:    0    blkaddr:   2087156      pack:2
nid:57076       ino:57076       offset:    0    blkaddr:   2071232      pack:2
nid:57118       ino:57118       offset:    0    blkaddr:   2071233      pack:2
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-06-08 10:10:38 -07:00
Jaegeuk Kim
603f8f9d36 sload.f2fs: support loading files into partition directly
This patch implements loading files into the existing partition.
For example,
 # sload.f2fs -f ./ /dev/sdb1

Then, all the directories and files will be loaded into /dev/sdb1.
By default, newly files should have inline_data and inline_xattr, if possible.

Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
Signed-off-by: Liu Shuoran <liushuoran@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-04-18 21:11:26 -04:00
Jaegeuk Kim
d3be08825e resize.f2fs: support to expand partition size
Now user can expand existing partition with resize.f2fs.
Currently, it doesn't support shrink an image.

For example,
 # resize.f2fs -t [# of sectors] [image]

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-04-18 21:11:26 -04:00
Sheng Yong
31f461cd89 fsck.f2fs: cache all nat entries and check each of them
All nat entries are cached during building nat_area_bitmap, so that, in
fsck_chk_meta, we can get and check blk_addr and ino directly, to see if
they are in the valid range. Also, blk_addr is checked to see if the block
is valid in sit's valid maps.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-03-22 14:10:00 -07:00
Sheng Yong
bc28669d6b fsck.f2fs: count the number of inodes during building nat_area_bitmap
The nid and ino of an inode are the same. So during building
nat_area_bitmap, we can know which nat entry represents an
inode, thus, we can count the number of inodes.

Then in fsck_chk_meta, the amount of inodes can be compared
with that recorded in CP.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-03-22 14:10:00 -07:00
Sheng Yong
6e225697bd fsck.f2fs: introduce -p option to check meta
This patch introduces a new option '-p' to do more checks on NAT/SIT areas.
'-p' has 2 levels: level 1 has the same sematics as '-a'; level 2 checks
NAT/SIT counters to see if they matches the status in SB and CP.

A new function, fsck_chk_meta, is called by '-p 1' to implement these
comparsion. If errors are detected, fix_on is set, which means fsck will
do a 'fsck -f' immediately.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-03-22 14:10:00 -07:00
Chao Yu
2a802319c3 fsck.f2fs: refactor extent info verification flow
There are some issues in original approach:
- memory allocated in i_extent.map will leak
- do not convert fields in i_ext from on-disk format to cpu format
- do not support checking file offset with extent info

This patch refactors the flow for fixing above issues and supporting
file offset check.

Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-03-16 12:49:34 -07:00
Chao Yu
c8327c909f fsck.f2fs: check dirent position
This patch enables fsck.f2fs to detect incorrect position where dirent
locates in an hierarchical hash structure directory.

Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-03-16 12:46:58 -07:00
Liu Xue
22778016fb fsck.f2fs: check valid values for dot dentries
Check valid values for '.' and '..', such as ino, hash_code, and the
number of dot or dotdot in the directory.

Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
Signed-off-by: Yong Sheng <shengyong1@huawei.com>
Reviewed-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-02-29 09:18:41 -08:00
Jaegeuk Kim
17acba6d3d defrag.f2fs: eliminate unexpected journal entries
When moving data or node blocks, it changes current segment information
dynamtically. Meanwhile, its journal entry space is recovered by old
stale data.
This patch makes sure that its journal space is zeroed out.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-16 17:46:32 -08:00
Jaegeuk Kim
2c1ebe237a defrag.f2fs: introduce defragmentation tool
This tool tries to move the valid blocks ranging from blkaddr to blkaddr + len
to targeted blkaddr with a direction like expand or shrink.

The option includes:
 -d debug level [default:0]
 -s start block address [default: main_blkaddr]
 -l length [default:512 (2MB)]
 -t target block address [default: main_blkaddr + 2MB]
 -i set direction as shrink [default: expand]

For example,
 # defrag.f2fs -s 0x100 -l 0x10 -t 0x4000 /dev/sdb1

This will move data blocks between 0x100 and 0x110 to the right side of
0x4000 space.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-11 15:58:50 -08:00
Jaegeuk Kim
003b102d8a fsck.f2fs: LFS alloc_type must have free segment after blkoff
This patch checks alloc_type of current segment type.
If it is LFS, the last of segment should have no valid block.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-11 15:57:51 -08:00
Jaegeuk Kim
1a35b5e29d f2fs-tools: export print_raw_sb_info
This patch exports print_raw_sb_info().

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-10 19:46:28 -08:00
Jaegeuk Kim
c605c5eb4c fsck.f2fs: sanity_check for extent_cache entry
This patch adds to check the stored extent_cache entry is consistent or not.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-10 19:46:28 -08:00
Jaegeuk Kim
d24f1bb16d fsck.f2fs: add hash conversion for encrypted dentries
If dentry is encrypted, we should convert its hash value.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-05-07 15:50:49 -07:00
Jaegeuk Kim
f1e4f9c0e9 fsck.f2fs: fix missing dentries
If a directory has no dot and dotdot dentries, fsck.f2fs sets inline_dots for
the inode so that f2fs module can handle that properly.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-03-30 13:37:24 -07:00
Jaegeuk Kim
56b8930368 fsck.f2fs: clean up child information
This patch adds a child_info data structure to clean up dirty parameters.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-03-30 13:00:54 -07:00
Jaegeuk Kim
e358413d4c fsck.f2fs: fix missing i_links
If a child is a directory, we should increase parent's i_links.
Previously, it counts direct dentry blocks excluding indirect blocks.
This patch fixes to count child_cnt correctly in order to get precise i_links.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-03-30 12:27:40 -07:00
Jaegeuk Kim
fce2ff183e fsck.f2fs: return summary block pointer and types
This patch adds to return summry_block pointer to callers, which will be used
when fixing them.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-03-27 16:34:48 -07:00
Jaegeuk Kim
c0feedadd9 fsck.f2fs: fix inodes having wrong i_links
This patch fixes inodes which have wrong i_links.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-03-25 18:51:53 -07:00
Jaegeuk Kim
7d253ac582 fsck.f2fs: remove inconsistent named directories
Each inode has its filename inside inode block.
For directory, the name should be matched all the time.
In other cases, it shouldn't, since there is able to be linked to other file
names.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-03-25 17:50:06 -07:00
Jaegeuk Kim
7ba9eec293 fsck.f2fs: show cp_state for fsck.f2fs
This patch adds to show cp_state during fsck.f2fs.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-03-20 17:01:41 -07:00
Jaegeuk Kim
9c6617d06d dump.f2fs: dump owner of data given block address
This patch introduces a feature to dump owner information of given block
address.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2014-12-17 18:39:34 -08:00
Jaegeuk Kim
6d8864014f fsck.f2fs: support inline_dentry
This patch implements inline_dentry feature.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2014-10-14 15:31:19 -07:00
Jaegeuk Kim
b700e313b4 fsck.f2fs: fix checkpoint
- fix nat entries
- fix sit entries
- fix checkpoint

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2014-08-29 13:43:29 -07:00
Jaegeuk Kim
1bc74e1eed fsck.f2fs: remove corrupted xattr block
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2014-08-29 13:43:29 -07:00
Jaegeuk Kim
48863d422e fsck.f2fs: give a chance to recover sit entries
This patch skips initial verfication on SIT entries, which checks the number of
valid blocks from its bitmap.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2014-08-29 13:43:29 -07:00
Jaegeuk Kim
5cd7001f5d fsck.f2fs: handle error cases
Do sanity check first and then update metadata.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2014-08-29 13:43:29 -07:00
Jaegeuk Kim
2701c94ab0 fsck.f2fs: remove return value of get_node_info
We don't need to get the return value.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2014-08-29 13:43:29 -07:00
Jaegeuk Kim
f6d1d582e9 fsck.f2fs: clean up codes
Fix wrong coding style.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2014-08-29 13:43:29 -07:00
JP Abgrall
a68ee58a5d Cleanup various build warnings (signed vs unsigned, unused vars,...)
* removed unused includes.
* removed unused parameters.
* Fixed a bunch of warnings around:
    int i;
    if (i < some_uint) ...
  and
    u32 x;
    ...
    if (x < 0) return error;
* Protect BLKDISCARD usage if it is not available.

Change-Id: Iede035b1beb2df01c961589a69aff47a5258ecd2
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
2014-05-15 08:40:55 +09:00
Changman Lee
6e8f2d5d22 f2fs-tools: add debug message for checking xattr
This patch adds debug message to confirm xattr block is checked.

Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
2013-08-07 18:21:24 +09:00
Changman Lee
c81ded6cf0 f2fs-tools: fix missing function declaration
Explicitly declare function name.

Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
2013-07-30 17:06:44 +09:00
Changman Lee
8843554c8c dump: dump inode information from blkaddr
This patch is to dump inode which blkaddr belongs in.
Usage:
  dump.f2fs /dev/sdx -b blk_addr (in 4KB)

Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
2013-07-18 15:45:30 +09:00
Changman Lee
7f35b548d4 f2fs-tools: add fsck.f2fs and dump.f2fs
fsck.f2fs checks file system consistency, but does not repair a broken
file system yet.
dump.f2fs shows the information of a specific inode and makes dump file
of SSA and SIT.
f2fs checks file system consistency as follows:
 o When data about used area and its metadata are identical,
   f2fs is considered consistent. To verify such consistency, we use
   three bitmaps: nat_area_bitmap, sit_area_bitmap, and main_area_bitmap.
   First, each bit in nat_area_bitmap corresponds to a nid in NAT.
   Second, each bit in sit_area_bitmap corresponds to a valid block in a
   segment. This bitmap is same to the total valid_map of f2fs_sit_entries
   in SIT.
   Last, each bit in main_area_bitmap corresponds to a block in main area
   except meta area.
   After a consistency check of each block, we set or clear the
   corresponding bit of each bitmap.
   From the root node, we start consistency check. The verified
   information varies according to block type.
   1. NODE
     - Read information of node block from NAT
     - Check if block address is allocated using node info.
     - Check if the type of f2fs_summary related to nid in SSA is NODE.
     - Update the corresponding bit in nat_area_bitmap.
     - Update the corresponding bit in sit_area_bitmap.
     - Set the corresponding bit in main_area_bitmap to 1.
     - Then, read node block. According to its attribute, explore
       inode/direct node/indirect node/double indirect node
       recursively.
     - If it is an inode block, we also check its xattr and hard link.
   2. DATA
     - Check if the type of f2fs_summary related to nid in SSA is DATA.
     - Set the corresponding bits of sit_area_bitmap and
       main_area_bitmap to visited
     - If it is a dentry block, traverse each dentries that may be
       regular
       file or directory. At this time, it will check inode block again.
   Finally, we verify whether
     - every nat_area_bitmap is visited
     - any unreachable hard link exists
     - values of sit_area_bitmap and main_area_bitmap are identical
     - total_valid_block_count/node_count/inode_count are correct

Usage:
 o fsck.f2fs
   # fsck.f2fs /dev/sdx
   options:
     -d debug level [default:0]
 o dump.f2fs
   # dump.f2fs -i [ino] /dev/sdx
   # dump.f2fs -s 0~-1 /dev/sdx (SIT dump)
   # dump.f2fs -a 0~-1 /dev/sdx (SSA dump)
   options:
     -d debug level [default:0]
     -i inode no (hex)
     -s [SIT dump segno from #1~#2 (decimal), for all 0~-1]
     -a [SSA dump segno from #1~#2 (decimal), for all 0~-1]

Note: To use dump.f2fs, please run make install or ln -s fsck.f2fs
dump.f2fs

Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Byoung Geun Kim <bgbg.kim@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
2013-07-04 23:15:25 +09:00