Commit Graph

192 Commits

Author SHA1 Message Date
Chao Yu
b4dbf57747 fsck.f2fs: add more sanity check for superblock
Add more sanity check logic for superblock like we did in kernel side.

This fixes bug reported by Seulbae Kim from bugzilla.

https://bugzilla.kernel.org/show_bug.cgi?id=203861

Reproted-by: Seulbae Kim <seulbae@gatech.edu>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-20 11:23:51 -07:00
Xue Liu
ee0025a13f f2fs-tools: calculate inode checksum and writeback
Calculate inode checksum and writeback when processing
sload or resize.

Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
[Chao Yu: change commit message and code]
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-20 11:23:51 -07:00
Damien Le Moal
658a0c8f20 f2fs-tools: Improve zoned model check
Return an error if an unknown zoned model is reported for a device or
if parsing of the device zoned model fails. Also add comments to
briefly explain the zone models and what to do in the absence of a
kernel reported zoned model for a device.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
[Jaegeuk Kim: Fix one missing function def change]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-20 11:23:51 -07:00
Chao Yu
a423b5f91e f2fs-tools: relocate chksum_offset for large_nat_bitmap feature
For large_nat_bitmap feature, there is a design flaw:

Previous:

struct f2fs_checkpoint layout:
+--------------------------+  0x0000
| checkpoint_ver           |
| ......                   |
| checksum_offset          |------+
| ......                   |      |
| sit_nat_version_bitmap[] |<-----|-------+
| ......                   |      |       |
| checksum_value           |<-----+       |
+--------------------------+  0x1000      |
|                          |      nat_bitmap + sit_bitmap
| payload blocks           |              |
|                          |              |
+--------------------------|<-------------+

Obviously, if nat_bitmap size + sit_bitmap size is larger than
MAX_BITMAP_SIZE_IN_CKPT, nat_bitmap or sit_bitmap may overlap
checkpoint checksum's position, once checkpoint() is triggered
from kernel, nat or sit bitmap will be damaged by checksum field.

In order to fix this, let's relocate checksum_value's position
to the head of sit_nat_version_bitmap as below, then nat/sit
bitmap and chksum value update will become safe.

After:

struct f2fs_checkpoint layout:
+--------------------------+  0x0000
| checkpoint_ver           |
| ......                   |
| checksum_offset          |------+
| ......                   |      |
| sit_nat_version_bitmap[] |<-----+
| ......                   |<-------------+
|                          |              |
+--------------------------+  0x1000      |
|                          |      nat_bitmap + sit_bitmap
| payload blocks           |              |
|                          |              |
+--------------------------|<-------------+

Related report and discussion:

https://sourceforge.net/p/linux-f2fs/mailman/message/36642346/

In addition, during writing checkpoint, if large_nat_bitmap feature is
enabled, we need to set CP_LARGE_NAT_BITMAP_FLAG flag in checkpoint.

Reported-by: Park Ju Hyung <qkrwngud825@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:30:16 -07:00
Chao Yu
24dec562ae f2fs-tools: allow unfixed f2fs_checkpoint.checksum_offset
Previously, f2fs_checkpoint.checksum_offset points fixed position of
f2fs_checkpoint structure:

"#define CP_CHKSUM_OFFSET	4092"

It is unnecessary, and it breaks the consecutiveness of nat and sit
bitmap stored across checkpoint park block and payload blocks.

This patch allows f2fs-tools to handle unfixed .checksum_offset.

In addition, for the case checksum value is stored in the middle of
checkpoint park, calculating checksum value with superposition method
like we did for inode_checksum.

In addition, add below change:
- using MAX_BITMAP_SIZE_IN_CKPT to clean up codes.
- introduce verify_checksum_chksum() to verify chksum_{offset,value}

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:29:55 -07:00
Chao Yu
362f3171dc f2fs-tools: get rid of unneeded fields in on-disk inode
As Jaegeuk reminded:

Once user updates f2fs-tools which support new fields in inode layout,
but do keep the kernel which can not support those fields, it will cause
old f2fs fail to mount new image due to root_inode's i_extra_isize value
sanity check.

So if f2fs-tools doesn't enable feature which will use new fields of
inode, we don't need to expand i_extra_isize to include them, let's just
let i_extra_isize point to the end of last valid extra field's position.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:29:50 -07:00
Eric Biggers
04bbe69311 f2fs-tools: improve filename printing
- Make buffers for pretty-printed filenames 341 bytes long, long enough
  for 255 (NAME_MAX) base64-encoded bytes.  Then print encrypted
  filenames in full, base64-encoded.  This will be useful for tests I'm
  writing which verify the correct ciphertext is stored on-disk.

- Rename convert_encrypted_name() to pretty_print_filename(), to make it
  clear that it handles unencrypted names too.  Also make the output
  'char' rather than 'unsigned char', as it's for printing; and remove
  the unnecessary return value.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:29:50 -07:00
Chao Yu
264d7da4d6 fsck.f2fs: fix to repair ro mounted device w/ -f
As Hagbard Celine reported:

"
Referring to the output from the fsck running against a "ro"
filesystem, especially this line:
Info: Check FS only due to RO

As far as i can tell this says that opposed to other filesystems
running fsck against a "ro" mounted f2fs partition will never fix any
errors.
So I tried running fsck against the same partition mounted "rw":
- mount -o remount,rw /mnt/f2fstest/
- fsck.f2fs  -f /dev/nvme0n1p7
Info: Force to fix corruption
Info: Mounted device!
        Error: Not available on mounted device!

I might be misunderstanding something, but all this tells me that
unless one make a custom initramfs that runs fsck before root is
mounted (something no distributions has, as far as I know), fsck will
never fix an f2fs formatted root partition during boot.
If this is by design and not a bug/unintended behavior, it should be
documented somewhere least more people will experience system crashes
like mine.

All tests above done with kernel 5.0.5 and f2fs-tools 1.12.0 with
"fsck.f2fs: allow to fsck readonly image w/ -f option"-patch by Chao
Yu.
"

We try to make our fsck behavior keeping line with e2fsprogs, but w/
-f option, we can just check a RO mounted device rather repair it, so
let's fix this.

Reported-and-Tested-by: Hagbard Celine <hagbardcelin@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-04-28 06:25:58 -07:00
Chao Yu
c6968d44b9 fsck.f2fs: fix to do sanity check with inode.i_inline_xattr_size
This patch adds to do sanity check with inode.i_inline_xattr_size,
and once it is corrupted, recover it to default value.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-03-27 07:02:56 -07:00
Damien Le Moal
8f80581f0d f2fs-tools: Fix various compilation warnings
Avoid various compilation warnings due to strncpy:

libf2fs.c:590:33: warning: ‘snprintf’ output may be truncated before
the last format character [-Wformat-truncation=]
  snprintf(rootdev, ret, "/dev/%s", buf);

../include/f2fs_fs.h:1384:2: warning: ‘strncpy’ specified bound
depends on the length of the source argument [-Wstringop-overflow=]
  strncpy(buf, features, strlen(features) + 1);

f2fstat.c:243:3: warning: ‘strncpy’ output truncated before
terminating nul copying as many bytes from a string as its length
[-Wstringop-truncation]
   strncpy(ptr_buf, name[i], strlen(name[i]));

This patch does not change any functionality.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-03-23 18:59:24 +09:00
Jaegeuk Kim
50e22bd947 f2fs-tools: add c++ for UNUSED
This can avoid build failure for f2fscrypt.

Change-Id: I33eddc62eea486dc219c25d089112e011864dd4f
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2019-03-01 14:18:16 -08:00
Jaegeuk Kim
af915e4571 f2fscrypt: avoid to fail android build
This fixes some static build errors.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-03-01 14:18:13 -08:00
Jaegeuk Kim
870817c410 fsck.f2fs: check nat_bits separately from bug_on
It checks nat_bits separately, and enable it seamlessly.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-01-08 20:21:32 -08:00
Chao Yu
c4a74535e9 fsck.f2fs: allow to fsck readonly image w/ -f option
To keep line with e2fsprogs, let's allow to fsck mounted image as
readonly w/ -f option.

Reported-by: Perfect Gentleman <perfect007gentleman@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-26 16:40:39 -08:00
Jaegeuk Kim
2367b0e9d2 f2fs-tools: show versions if exist
If it's not defined, we need to skip to show the definition.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Jaegeuk Kim
83c64019e8 fsck.f2fs: don't allocate new blocks on unclean shutdown
We have to keep data for roll-forward recovery. Without this patch, we're
able to lose there-in data by quota overwrites.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Chao Yu
dfede78aa9 fsck.f2fs: detect and recover corrupted quota file
Once quota file is corrupted, kernel will set CP_QUOTA_NEED_FSCK_FLAG
into checkpoint pack, this patch makes fsck supporting to detect the flag
and try to rebuild corrupted quota file.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
2018-11-21 11:38:23 -08:00
Junling Zheng
886a924bfa f2fs-tools: introduce sb checksum
This patch introduced crc for superblock.

Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Daniel Rosenberg
1ccd8313ba WIP: Add partial shrinking support to f2fs
allow 'safe' resizing leaving the metadata nearly unchanged.

Will likely need to adjust growing for the case where a fs
had previously shrunk without altering data, and the newly grown
fs would be smaller than the original. Probably makes sense to
case it out into metadata size changes, instead of fs size changes

Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2018-11-21 11:38:23 -08:00
Jaegeuk Kim
a48bda057a fsck.f2fs: support checkpoint=disable
This patch shows checkpoint is disabled and keeps the flag for next mount.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Junling Zheng
a6cbb57192 f2fs-tools: rename CHECKSUM_OFFSET to CP_CHKSUM_OFFSET
This patch renamed CHECKSUM_OFFSET to CP_CHKSUM_OFFSET.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Junling Zheng
767ac306c8 fsck.f2fs: supply more check entries for checkpoint
Supply some more check entries for checkpoint in sanity_check_ckpt()
and validate_checkpoint() to sync them with kernel.

Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:52:39 -07:00
Chao Yu
91bb7b21f7 f2fs-tools: fix to reset i_gc_failures offline
This patch synchronize f2fs_inode structure from kernel side, in
addition, it adds to check .i_gc_failures and do resetting in fsck.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:52:39 -07:00
Jaegeuk Kim
bdd39ce73b mkfs.f2fs: add root_owner to give uid/gid
This patch adds an option to mkfs.f2fs in order for user to assign uid/gid
to the target partition.
This requires when vold in android formats a sdcard partition.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:49:26 -07:00
Junling Zheng
8af586d3a2 f2fs-tools: fix compile errors on AOSP
Include needed header files directly to fix compile errors on AOSP.

Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:49:26 -07:00
Jaegeuk Kim
f77e7ce327 mkfs.f2fs: factor out feature table from mkfs.f2fs
This patch makes feature bit work be global in f2fs-tools.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-07-13 22:46:00 -07:00
Jaegeuk Kim
1772e31423 f2fs-tools: add -g to give default options
This patch adds -g option to set default options for specific environment.
I added it for android as a example.

 # mkfs.f2fs -g android $dev
  : gives "-d1 -f -O encrypt -O quota -w 4096"

 # fsck.f2fs -g android $dev
  : gives "-a"

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-07-13 22:46:00 -07:00
Sheng Yong
b112b51604 f2fs-tools: remove duplicated declaration of f2fs_configuration c
The variable `c' is declared twice in f2fs_fs.h. This patch removes
the second declaration.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-07-13 22:46:00 -07:00
Sheng Yong
8feed81aa9 f2fs-tools: introduce new option V to show version
This patch introduces a new option -V to show the version of f2fs tools
and exit after that.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-07-13 22:46:00 -07:00
Sheng Yong
ff3782931f mkfs.f2fs: create lost+found directory
This patch introduces a new feature F2FS_FEATURE_LOST_FOUND. It can be
switched on by indicating `-O lost_found'. If LOST_FOUND feature is
enabled, an empty directory lost+found is created by mkfs.

This is a preparation for fsck. During fsck, the directory is used to
save unreachable files, which have no parent directory or their parent
directory is removed by fsck. Encrypted files are also allowed to be
saved here.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: use uint64_t to avoid build warning]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-07 14:49:38 -07:00
Chao Yu
6fdc37ff26 mkfs.f2fs: support hot file extension
This patch adds new option '-E' to accept user configured hot file
extension, in order to let kernel module handle hot/cold file's datas
separately better.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-07 14:49:38 -07:00
Sheng Yong
5e91b3640c mkfs.f2fs: introduce mkfs parameters in f2fs_configuration
Introduce new parameters in f2fs_configuration for mkfs:
  * next_free_nid: save the next free nid
  * quota_inum: save how many blocks are used for quota inodes
  * quota_dnum: save how many blocks are used for quota data

Use these parameters to avoid duplicated count of these values. And
discard obsolete dnodes after all inodes are created.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-07 14:49:38 -07:00
Chao Yu
baaa076b4d mkfs.f2fs: expand scalability of nat bitmap
Previously, our total node number (nat_bitmap) and total nat segment count
will not monotonously increase along with image size, and max nat_bitmap size
is limited by "CHECKSUM_OFFSET - sizeof(struct f2fs_checkpoint) + 1", it is
with bad scalability when user wants to create more inode/node in larger image.

So this patch tries to relieve the limitation, by default, limitting total nat
entry number with 20% of total block number.

Before:
image_size(GB)	nat_bitmap	sit_bitmap	nat_segment	sit_segment
16		3836		64		36		2
32		3836		64		72		2
64		3772		128		116		4
128		3708		192		114		6
256		3580		320		110		10
512		3260		640		100		20
1024		2684		1216		82		38
2048		1468		2432		44		76
4096		3900		4800		120		150

After:
image_size(GB)	nat_bitmap	sit_bitmap	nat_segment	sit_segment
16		256		64		8		2
32		512		64		16		2
64		960		128		30		4
128		1856		192		58		6
256		3712		320		116		10
512		7424		640		232		20
1024		14787		1216		462		38
2048		29504		2432		922		76
4096		59008		4800		1844		150

Add a new option '-i' to control turning on/off this feature.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-07 14:49:37 -07:00
Jaegeuk Kim
595fd57a4f f2fs-tools: get kernel version via uname(2)
This patch introduces uname(2) to get kernel version.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-04 12:19:40 -07:00
Jaegeuk Kim
76cd37753a mkfs.f2fs: support fsverity feature
This is aligned to f2fs which reserves fsverity feature bit.

Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2018-04-04 22:22:33 -07:00
Hyojun Kim
f7f93c754d f2fs-tools: change to use #pragma pack(push, 1)
It was reported that #pragma pack(1) could create unwanted
influences. pack(push, 1) and pack(pop) are used instead.

Signed-off-by: Hyojun Kim <hyojun@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-04-03 00:10:08 -07:00
katao
74bd6a8469 libf2fs,mkfs.f2fs: add wanted_sector_size for wanted_total_sectors
The wanted_total_sectors was determined by device sector size, but sometimes
we don't know precise sector_size by default. So, let's give wanted_sector_size
in such the ambiguous situation.

Signed-off-by: katao <katao@xiaomi.com>
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2018-04-02 22:57:31 -07:00
Chao Yu
48620808e5 f2fs-tools: support inode creation time
This patch supports inode_crtime feature to enable recording inode
creation time in inode layout.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-01-25 14:11:32 -08:00
Jaegeuk Kim
f789444b9c fsck.f2fs: allow roll-forward for small checkpoint fix
This allows roll-forward when fsck.f2fs changes checkpoint crc.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-01-22 11:29:03 -08:00
Gaoxiang (OS)
2e65d03730 resize.f2fs: fix max_nat_bitmap_size miscalculatation
For the same reason as "mkfs.f2fs: fix to calculate
left space of checkpoint page correctly",
this patch also fixes miscalculatation in resize.f2fs.

Signed-off-by: Gao Xiang <gaoxiang25@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-01-02 19:25:21 -08:00
Hyojun Kim
fde3554947 mkfs.f2fs: Limit pragma pack(1) only for Windows Host
A side-effect was found for #pragma pack(1), which were added for
Windows SDK support. Limiting it with ANDROID_WINDOWS_HOST.

Signed-off-by: Hyojun Kim <hyojun@google.com>
2017-12-14 10:25:06 -08:00
Hyojun Kim
b44f7262a1 mkfs.f2fs: Windows Android SDK support
This patch enables cross compilation for Windows Android SDK
by using ANDROID_WINDOWS_HOST.

Signed-off-by: Hyojun Kim <hyojun@google.com>
2017-12-04 18:48:12 -08:00
Jaegeuk Kim
789ac2df1f f2fs-tools: enable sparse_file opeartions
This adds reading data from sparse_file.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-12-04 18:48:10 -08:00
Jaegeuk Kim
bb5aa5cd6c sload.f2fs: update build for android
This patch updates sload.f2fs for android build.

- do fsck() after sload() to update quota information.
- return success, if there's no source directory

  -C fs_config

  -f source directory [path of the source directory]

  -p product out directory
    : path on fs_config file for uid/gid/mode

  -s file_contexts
    : selabel file

  -t mount point [prefix of target fs path, default:/]
     : path stored in selabel/fs_config files

  -T timestamp

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-30 22:36:35 -08:00
Jaegeuk Kim
3a55b82af8 f2fs-tools: build tools selectively
This patch enables to build tools except mkfs.f2fs selectively.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-29 17:22:11 -08:00
Chao Yu
e813df3194 libf2fs: propagate error from f2fs_{finalize,fsync}_device
tests/generic/405 of fstest suit expects that mkfs will return error
when it hits EIO, so let's propagate error from
f2fs_{finalize,fsync}_device, then mkfs can be aware of fsync error.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: initialize ret to zero]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-27 19:26:50 -08:00
Jaegeuk Kim
c8de435a18 f2fs-tools: define WITH_ANDROID if __ANDROID__ is defined
This matches with __ANDROID__ with WITH_ANDROID.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-16 15:47:29 -08:00
Jaegeuk Kim
43bb7b6c99 f2fs-tools: build binaries in Mac
This patch modifies f2fs-tools to be built in mac.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-14 19:43:48 -08:00
Jaegeuk Kim
bbdbca4e22 fsck.f2fs: add an option to preserve quota limits
If it detects quota file errors, we can see insane quota limits. In order
to recover that, this patch adds an option to reset them as zeros.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-11 18:01:07 -08:00
Sheng Yong
ffabb34ea1 f2fs-tools: remove unused list.h
Commit 7d96d138a3 ("fsck.f2fs: remove list.h") has already the usage
of list.h. So let's remove list.h itself.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-11 18:01:07 -08:00
Sheng Yong
f2f1341ffe fsck.f2fs: introduce new option --dry-run
With --dry-run enabled, fsck.f2fs will do all checks and "fixes" except
that all fixes will not be written to storage at last.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-11 18:01:07 -08:00
Hyojun Kim
6c6bb35c89 fsck.f2fs: support quota
This patch let fsck to check and fix quota file contents.

Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-11-06 20:07:40 -08:00
Hyojun Kim
23a872f9ba mkfs.f2fs: support quota option in mkfs
This patch let mkfs to handle quota option and create quota files.

Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-11-03 10:37:34 -07:00
Chao Yu
8d46072f07 f2fs-tools: support flexible inline xattr size
This patch makes mkfs to support flexible inline xattr feature, also
this enables fsck to recognize new disk-layout of inode which be of
flexible inline xattr size.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-10-30 02:16:35 -07:00
Yunlei He
1797a5f092 fsck: introduce generic function f2fs_fsync_device
We use f2fs_finalize_device to fsync previous data in
checkpoint area before write last CP pack, in order to avoid
cp corruption in sudden-power-off case. But this function will
close the device, so this patch introduce a function to call
fsync() only.

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-10-26 12:25:27 +02:00
Chao Yu
7457726261 f2fs-tools: support inode checksum
This patch introduce a new option 'inode_checksum' for enabling inode
checksum functionality in mkfs/fsck/sload.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-07-26 14:38:43 -07:00
Chao Yu
a3d9db1f88 f2fs-tools: support project quota
This patch introduce a new option 'project_quota' for enabling project
quota functionality during mkfs.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-07-26 11:42:56 -07:00
Chao Yu
dad33a1ee7 f2fs-tools: enhance on-disk inode structure scalability
This patch adds an option 'extra_attr' in mkfs for enabling v2 f2fs
inode format in kernel codes.

Also this patch makes fsck to support recognize v2 inode format,
below is v2 format description:

Original one:

struct f2fs_inode {
	...
	struct f2fs_extent i_ext;
	__le32 i_addr[DEF_ADDRS_PER_INODE];
	__le32 i_nid[DEF_NIDS_PER_INODE];
}

Extended one:

struct f2fs_inode {
        ...
        struct f2fs_extent i_ext;
	union {
		struct {
			__le16 i_extra_isize;
			__le16 i_padding;
			__le32 i_extra_end[0];
		};
		__le32 i_addr[DEF_ADDRS_PER_INODE];
	};
        __le32 i_nid[DEF_NIDS_PER_INODE];
}

Once F2FS_EXTRA_ATTR is set, we will steal four bytes in the head of
i_addr field for storing i_extra_isize and i_padding. with i_extra_isize,
we can calculate actual size of reserved space in i_addr, available
attribute fields included in total extra attribute fields for current
inode can be described as below:

  +--------------------+
  | .i_mode            |
  | ...                |
  | .i_ext             |
  +--------------------+
  | .i_extra_isize     |-----+
  | .i_padding         |     |
  | .i_prjid           |     |
  | .i_atime_extra     |     |
  | .i_ctime_extra     |     |
  | .i_mtime_extra     |<----+
  | .i_inode_cs        |<----- store blkaddr/inline from here
  | .i_xattr_cs        |
  | ...                |
  +--------------------+
  |                    |
  |    block address   |
  |                    |
  +--------------------+
  | .i_nid             |
  +--------------------+
  |   node_footer      |
  | (nid, ino, offset) |
  +--------------------+

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-07-26 11:42:49 -07:00
Jaegeuk Kim
8721519c8d f2fs-tools: add atomic_write feature flag
This patch adds atomic_write feature flag.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-07-21 13:49:28 -07:00
Yang Jin
4abc53c38f add sparse support for f2fs
Description:
modify I/O operations to support writing in sparse format
build host executable for making sparse image

Signed-off-by: Yang Jin <yajin@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-06-16 14:06:21 -07:00
Jaegeuk Kim
8e29c2be47 f2fs-tools: avoid build warnings
This patch resolves warnings while building it in android.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-06-05 13:58:50 -07:00
Chao Yu
f642b24822 f2fs-tools: support to set and recognize CP_TRIMMED_FLAG
During mkfs, once we have issued discard for all device, we can set
CP_TRIMMED_FLAG in cp pack, this can make kernel module to record
undiscard blocks correctly, it will help to avoid unneeded discard.

Also this patch makes f2fs-tools be with the ability of recognizing
that new flag.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-04-28 12:04:21 -07:00
Kinglong Mee
7b03ccbb9b fsck.f2fs: check encrypted filename depends FADVISE_ENC_NAME_BIT
An encrypted directory under an unencrypted directory, the filename isn't encrypted.

Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-03-19 18:45:37 -04:00
Kinglong Mee
6fb673abc0 fsck.f2fs: a separate option for showing directory tree
Showing directory tree reuses dbg_lv that if setting "-t",
fsck.f2fs does not show any others.
Users may want much information include the debug info and directory tree.

Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-03-16 08:16:39 +08:00
Jaegeuk Kim
191573e71b mkfs.f2fs: support nat_bits feature
This patch adds nat_bits in the last one ore two 4KB-sized blocks of first
checkpoint segment. So, now a set of checkpoint *segment* consists of:

< checkpoint segment >
----------------------
| cp page #1         | 4KB
----------------------
| compacted data sum | 4KB
----------------------
| node sum  #1       | 4KB
----------------------
| node sum  #2       | 4KB
----------------------
| node sum  #3       | 4KB
----------------------
| cp page #2         | 4KB
----------------------
...
----------------------
| crc|cpver          | 8 bytes
----------------------
| full NAT bits      | # of NAT segments / 2 * 512 / 8 bytes
----------------------
| empty NAT bits     | # of NAT segments / 2 * 512 / 8 bytes
----------------------

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-02-23 10:11:41 -08:00
Jaegeuk Kim
a0bd5b9822 f2fs-tools: catch up up-to-date checkpoint flag
It needs to use up-to-date ckpt flag.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-02-23 10:11:41 -08:00
Sheng Yong
9deb6fd9f2 f2fs-tools: correct endianness
It is reported that fsck.f2fs behaves abnormally when running on MIPS32
rel 2 big endian cpu, since incorrect endianness. So let's correct all
endianess issues of f2fs-tools.

Reported-by: <k@vodka.home.kg>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-01-19 15:29:08 +08:00
Jaegeuk Kim
0a48220641 mkfs.f2fs: detect small partition by overprovision ratio and # of segments
We should avoid zero overprovision ratio or # of segments caused by too small
device.

Reported-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-28 21:26:15 -08:00
Jaegeuk Kim
de7e07e011 f2fs-tools: support multiple devices
This patch adds an option to specify multiple devices for an f2fs instance.

Up to 7 devices in addition to the default device can be added.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-09 18:03:47 -08:00
Damien Le Moal
9691420a99 f2fs-tools: introduce support for zoned block devices
With the availability of the BLKREPORTZONE and BLKRESETZONE
ioctls, there is no need for using SG_IO to discover zoned
block devices characteristics. This simplifies the code.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-01 17:27:34 -07:00
Damien Le Moal
81c62e2537 f2fs-tools: use zoned block device terminology
SMR stands for "Shingled Magnetic Recording" which makes sense
only for hard disk drives (spinning rust). The ZBC/ZAC standards
enable management of SMR disks, but solid state drives may also
support those standards. So replace "SMR" with "zoned block device"
to avoid a HDD centric terminology.

In particular, rename the HMSMR feature to BLKZONED.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-10-31 14:52:27 -07:00
Sheng Yong
c4c65265da dump/fsck: convert encrypted file name
If fscrypt is enabled, we need to convert the encrypted file name before
printing it. So let's export convert_encrypted_name for other functions,
and make it returns the length of converted string.
This patch also changes the parameter of file_is_encrypt to f2fs_inode.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-10-31 10:59:24 -07:00
Tobias Jakobi
e5b3ec89ee libf2fs: increase robustness when called on root device
On some systems '/proc/mounts' contains the entry '/dev/root'
as alias for the root device, while that alias doesn't actually
exist as symlink in /dev.

/proc/mounts:
/dev/root / f2fs rw,noatime,background_gc=on,user_xattr,acl,inline_data,extent_cache,active_logs=6 0 0

mount:
/dev/mmcblk1p1 on / type f2fs (rw,noatime,background_gc=on,user_xattr,acl,inline_data,extent_cache,active_logs=6)

If the root device is mounted RO, and we try to scan it, via
'fsck.f2fs /dev/mmcblk1p1', then fsck.f2fs exits because it can't
find the entry in /proc/mounts.

Try harder to identify the root device of the system first, and
in case we are operating on it, check also for '/dev/root' in mounts.

Signed-off-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-09-28 15:26:54 -07:00
Jaegeuk Kim
3ffc5a8f3e f2fs-tools: use shorter config variable name
This patch has no functional change.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-09-22 19:32:14 -07:00
Jaegeuk Kim
79b1858e79 libf2fs: use dev_read_block and dev_write_block
This patche tries to use dev_read_block and dev_write_block as much as possible.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-09-22 19:32:12 -07:00
Jaegeuk Kim
70a8fc3df7 mkfs/fsck: add host-managed smr feature
This feature flag will be used by f2fs to detect HMSMR device.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-06-15 09:43:28 -07:00
Jaegeuk Kim
74673eb7e6 mkfs.f2fs: ZBC device support
This patch adds "-m" option to configure ZBC device.

This is to support host-managed SMR device and configure some major features
and on-disk layout in f2fs.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-06-08 10:10:38 -07:00
Jaegeuk Kim
603f8f9d36 sload.f2fs: support loading files into partition directly
This patch implements loading files into the existing partition.
For example,
 # sload.f2fs -f ./ /dev/sdb1

Then, all the directories and files will be loaded into /dev/sdb1.
By default, newly files should have inline_data and inline_xattr, if possible.

Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
Signed-off-by: Liu Shuoran <liushuoran@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-04-18 21:11:26 -04:00
Jaegeuk Kim
d3be08825e resize.f2fs: support to expand partition size
Now user can expand existing partition with resize.f2fs.
Currently, it doesn't support shrink an image.

For example,
 # resize.f2fs -t [# of sectors] [image]

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-04-18 21:11:26 -04:00
Sheng Yong
6e225697bd fsck.f2fs: introduce -p option to check meta
This patch introduces a new option '-p' to do more checks on NAT/SIT areas.
'-p' has 2 levels: level 1 has the same sematics as '-a'; level 2 checks
NAT/SIT counters to see if they matches the status in SB and CP.

A new function, fsck_chk_meta, is called by '-p 1' to implement these
comparsion. If errors are detected, fix_on is set, which means fsck will
do a 'fsck -f' immediately.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-03-22 14:10:00 -07:00
Chao Yu
c8327c909f fsck.f2fs: check dirent position
This patch enables fsck.f2fs to detect incorrect position where dirent
locates in an hierarchical hash structure directory.

Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-03-16 12:46:58 -07:00
Chao Yu
f102ca4c3f f2fs-tools: rebase struct f2fs_summary_block with upstream
Struct f2fs_summary_block has been changed in upstream commits:
 - commit 8f1dbbbbdfe9 ("f2fs: introduce lifetime write IO statistics")
 - commit dfc08a12e49a ("f2fs: introduce f2fs_journal struct to wrap journal info")

This patch does the sync work for f2fs_summary_block structure between
f2fs-utils and f2fs kernel module.

Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-03-14 09:26:53 -07:00
Junling Zheng
ecc43cc53f f2fs-tools: redefine MAX_SIT_BITMAP_SIZE
During mkfs, segment_count_sit is calculated using ALIGN and
SEG_ALIGN, which are both upward. Then, in some extreme scenarios,
such as 16TB, sit_ver_bitmap_bytesize got from segment_count_sit
will exceed MAX_SIT_BITMAP_SIZE, which is calculated in a downward
method.

Unify the two calculating methods and redefine MAX_SIT_BITMAP_SIZE
to prevent sit_ver_bitmap_bytesize overflowing.

Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-02-24 09:57:18 -08:00
Sheng Yong
1952ab467c f2fs-tools: fix endianness of on-disk check_sum in summary_footer
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-02-01 16:27:46 -08:00
Chao Yu
dbe78ae529 libf2fs: fix incorrect type define of declaration
When compiling f2fs toolset in 32-bits machine, following error was
reported:

libf2fs.c:342:5: error: conflicting types for 'find_next_bit_le'
../include/f2fs_fs.h:864:22: note: previous declaration of 'find_next_bit_le' was here
libf2fs.c:348:5: error: conflicting types for 'find_next_zero_bit_le'
../include/f2fs_fs.h:865:22: note: previous declaration of 'find_next_zero_bit_le' was here

This is because our type of return value(u64 in define) and (unsigned long
in declaration) of find_next_{,zero_}bit_le were not same in non-64-bits
machine:

extern unsigned long find_next_bit_le(const u8 *, u64, u64);
u64 find_next_bit_le(const u8 *addr, u64 size, u64 offset)

Fix it.

Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-18 14:13:05 -08:00
Jaegeuk Kim
b008605e99 lib: use u8/u32/u64 for bit operations
This cleans up the bit operations.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-15 11:15:00 -08:00
Jaegeuk Kim
ee65f6ebf2 lib: fix test_bit_le functions
This patch fixes test_bit_le functions for dentry bit operations.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-15 10:40:37 -08:00
Jaegeuk Kim
9e5c021b67 mkfs.f2fs: fix storing volume label correctly in utf16
This patch fixes to store volume label as utf16 correctly.

Many conversion codes are copied from exfat-tools.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-14 16:35:39 -08:00
Jaegeuk Kim
2c1ebe237a defrag.f2fs: introduce defragmentation tool
This tool tries to move the valid blocks ranging from blkaddr to blkaddr + len
to targeted blkaddr with a direction like expand or shrink.

The option includes:
 -d debug level [default:0]
 -s start block address [default: main_blkaddr]
 -l length [default:512 (2MB)]
 -t target block address [default: main_blkaddr + 2MB]
 -i set direction as shrink [default: expand]

For example,
 # defrag.f2fs -s 0x100 -l 0x10 -t 0x4000 /dev/sdb1

This will move data blocks between 0x100 and 0x110 to the right side of
0x4000 space.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-11 15:58:50 -08:00
Jaegeuk Kim
c8ce2bf7b1 mkfs.f2fs: export get_best_overprovision
This patch exports get_best_overprovision() function.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-10 19:46:28 -08:00
Jaegeuk Kim
1ea31c47f4 libf2fs: enhance the bit operations
This patch modifies the existing bit operations.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-10 19:46:22 -08:00
Jaegeuk Kim
06a19451ee mkfs.f2fs: export get_{sb|cp} and set_{sb|cp}
This patch exports get_sb, set_sb, get_cp, and set_cp for other tools.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-12-02 21:36:03 -08:00
Jaegeuk Kim
67de42f76a fsck.f2fs: support a readonly filesystem
If f2fs is mounted as ro, we can do fsck.f2fs.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-11-22 11:26:52 +08:00
Jaegeuk Kim
2cdb04b52f mkfs.f2fs: support <1% overprovision ratio
Big partition size needs uner 1% overprovision space to acquire more space.

    section  ovp ratio  ovp size
For 8TB,
    -s1    : 0.07%     -> 10GB
    -s32   : 0.39%     -> 65GB
    -s64   : 0.55%     -> 92GB
    -s128  : 0.78%     -> 132GB

For 128GB,
    -s1    : 0.55%     -> 1.4GB
    -s32   : 3.14%     -> 8GB
    -s64   : 4.45%     -> 12GB
    -s128  : 6.32%     -> 17GB

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-09-25 10:32:12 -07:00
Jaegeuk Kim
6f9c275125 mkfs.f2fs: don't need to limit MIN_VOLUME SIZE
The minimum volume size is determined while preparing superblock.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-08-10 17:58:40 -07:00
Jaegeuk Kim
d24f1bb16d fsck.f2fs: add hash conversion for encrypted dentries
If dentry is encrypted, we should convert its hash value.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-05-07 15:50:49 -07:00
Jaegeuk Kim
6e6c713240 mkfs.f2fs: set encryption feature
This patch add to support encryption feature.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-04-21 20:06:16 -07:00
Jaegeuk Kim
f1e4f9c0e9 fsck.f2fs: fix missing dentries
If a directory has no dot and dotdot dentries, fsck.f2fs sets inline_dots for
the inode so that f2fs module can handle that properly.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-03-30 13:37:24 -07:00
Chao Yu
37580f1c5b fsck.f2fs: support large sector size
Since f2fs support large sector size in commit 55cf9cb63f0e "f2fs: support large
sector size", block device with sector size of 512/1024/2048/4096 bytes can be
supported.

But fsck.f2fs still use default F2FS_LOG_SECTOR_SIZE/F2FS_LOG_SECTORS_PER_BLOCK to
verify related data in f2fs image, it's wrong, let's fix this issue in this patch.

Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-02-05 18:42:46 -08:00
Jaegeuk Kim
631f8166bb fsck.f2fs: readahead node blocks to speed up
This patch adds readahead system call to speed up node block reads.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2015-01-29 13:36:54 -08:00