Commit Graph

36 Commits

Author SHA1 Message Date
liuchao12
9401f8f0eb resize.f2fs: add option to manually specify new overprovision
Make.f2fs supports manually specifying overprovision, and we expect
resize.f2fs to support it as well.

This change add a new '-o' option to manually specify overprovision,
and fix to check free space before grow. Otherwise, after grow,
kernel may report below error message when we mount the image if -o
parameter is specified during resize:

F2FS-fs (loop0): invalid crc_offset: 0
F2FS-fs (loop0): Wrong valid_user_blocks: 16404, user_block_count: 13312
F2FS-fs (loop0): Failed to get valid F2FS checkpoint
mount(2) system call failed: Structure needs cleaning.

Signed-off-by: liuchao12 <liuchao12@xiaomi.com>
Signed-off-by: qixiaoyu1 <qixiaoyu1@xiaomi.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Change-Id: I2ad0ecc5546a1712b7aa36230e91c3e0b1beb7e0
2024-04-15 11:31:38 +08:00
Bart Van Assche
a8c6a6d93f Switch from the u_int to the uint types
commit 87d7a95e3134b668e9f1e8519a19f390509fac60
category: bugfix
issue: #I6VAS0
CVE: NA

Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------

Many format strings use one of the PRI* macros. These macros are compatible
with the uint types but not with the u_int types. Hence this patch
that switches from the u_int to the uint types.

Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
2023-04-13 14:13:48 +08:00
Seung-Woo Kim
63f4592769 resize.f2fs: fix memory leak caused by migrate_nat()
commit 5cc365c1e7
category: bugfix
issue: #I6VAS0
CVE: NA

Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------

Alloced nat_block doesn't freed from migrate_nat(). Fix to free
nat_block.

Signed-off-by: Seung-Woo Kim <sw0312.kim@samsung.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
2023-04-13 14:13:45 +08:00
Chao Yu
228d850dd2 resize.f2fs: fix to check free space before shrink
commit 5263ae25fb
category: bugfix
issue: #I6VAS0
CVE: NA

Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------

Otherwise, after shrink, kernel will report below error message
when we mount the image:

F2FS-fs (loop0): invalid crc_offset: 0
F2FS-fs (loop0): Wrong valid_user_blocks: 16404, user_block_count: 13312
F2FS-fs (loop0): Failed to get valid F2FS checkpoint
mount(2) system call failed: Structure needs cleaning.

Reported-and-tested-by: beroal <me@beroal.in.ua>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
2023-04-13 14:13:44 +08:00
Chao Yu
e61960e5e2 resize.f2fs: fix wrong sit/nat bitmap during rebuild_checkpoint()
commit 159752dd3c
category: bugfix
issue: #I6VAS0
CVE: NA

Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------

As beroal <me@beroal.in.ua> reported: f2fs image can be corrupted
after below testcase:

1. truncate -s $((256*1024*1024)) img
2. mkfs.f2fs -f img
3. mount -t f2fs -o loop img /mnt
4. xfs_io -f /mnt/file -c "pwrite 0 2M" -c "fsync"
5. umount /mnt
6. resize.f2fs -s -t 262144 img
7. fsck.f2fs img

The root cause is we forgot to copy original sit/nat bitmap to
new checkpoint during rebuild_checkpoint(), fix it.

Reported-and-testd-by: beroal <me@beroal.in.ua>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
2023-04-13 14:13:44 +08:00
Jaegeuk Kim
143204c59f resize.f2fs: add force option to rewrite broken calculation
commit 98e64635bd
category: bugfix
issue: #I6VAS0
CVE: NA

Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------

This patch adds "-f" for resize.f2fs to fix broken resized f2fs.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
2023-04-13 14:13:44 +08:00
xiongping1
daa0f8b9e9 resize.f2fs: add option for large_nat_bitmap feature
resize.f2fs has already supported large_nat_bitmap feature, but has no
option to turn on it.

This change add a new '-i' option to control turning on it.

Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-14 07:43:05 -08:00
Chao Yu
a423b5f91e f2fs-tools: relocate chksum_offset for large_nat_bitmap feature
For large_nat_bitmap feature, there is a design flaw:

Previous:

struct f2fs_checkpoint layout:
+--------------------------+  0x0000
| checkpoint_ver           |
| ......                   |
| checksum_offset          |------+
| ......                   |      |
| sit_nat_version_bitmap[] |<-----|-------+
| ......                   |      |       |
| checksum_value           |<-----+       |
+--------------------------+  0x1000      |
|                          |      nat_bitmap + sit_bitmap
| payload blocks           |              |
|                          |              |
+--------------------------|<-------------+

Obviously, if nat_bitmap size + sit_bitmap size is larger than
MAX_BITMAP_SIZE_IN_CKPT, nat_bitmap or sit_bitmap may overlap
checkpoint checksum's position, once checkpoint() is triggered
from kernel, nat or sit bitmap will be damaged by checksum field.

In order to fix this, let's relocate checksum_value's position
to the head of sit_nat_version_bitmap as below, then nat/sit
bitmap and chksum value update will become safe.

After:

struct f2fs_checkpoint layout:
+--------------------------+  0x0000
| checkpoint_ver           |
| ......                   |
| checksum_offset          |------+
| ......                   |      |
| sit_nat_version_bitmap[] |<-----+
| ......                   |<-------------+
|                          |              |
+--------------------------+  0x1000      |
|                          |      nat_bitmap + sit_bitmap
| payload blocks           |              |
|                          |              |
+--------------------------|<-------------+

Related report and discussion:

https://sourceforge.net/p/linux-f2fs/mailman/message/36642346/

In addition, during writing checkpoint, if large_nat_bitmap feature is
enabled, we need to set CP_LARGE_NAT_BITMAP_FLAG flag in checkpoint.

Reported-by: Park Ju Hyung <qkrwngud825@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:30:16 -07:00
Chao Yu
24dec562ae f2fs-tools: allow unfixed f2fs_checkpoint.checksum_offset
Previously, f2fs_checkpoint.checksum_offset points fixed position of
f2fs_checkpoint structure:

"#define CP_CHKSUM_OFFSET	4092"

It is unnecessary, and it breaks the consecutiveness of nat and sit
bitmap stored across checkpoint park block and payload blocks.

This patch allows f2fs-tools to handle unfixed .checksum_offset.

In addition, for the case checksum value is stored in the middle of
checkpoint park, calculating checksum value with superposition method
like we did for inode_checksum.

In addition, add below change:
- using MAX_BITMAP_SIZE_IN_CKPT to clean up codes.
- introduce verify_checksum_chksum() to verify chksum_{offset,value}

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-20 17:29:55 -07:00
Junling Zheng
886a924bfa f2fs-tools: introduce sb checksum
This patch introduced crc for superblock.

Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Junling Zheng
238fa8ccf1 fsck.f2fs: unify the updating of superblocks
Rename write_superblock() to update_superblock() and make it support updating
specified one superblock or both two superblocks, then unify all places where
sb needs to be updated.

Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Daniel Rosenberg
1ccd8313ba WIP: Add partial shrinking support to f2fs
allow 'safe' resizing leaving the metadata nearly unchanged.

Will likely need to adjust growing for the case where a fs
had previously shrunk without altering data, and the newly grown
fs would be smaller than the original. Probably makes sense to
case it out into metadata size changes, instead of fs size changes

Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2018-11-21 11:38:23 -08:00
Junling Zheng
a6cbb57192 f2fs-tools: rename CHECKSUM_OFFSET to CP_CHKSUM_OFFSET
This patch renamed CHECKSUM_OFFSET to CP_CHKSUM_OFFSET.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-21 11:38:23 -08:00
Yunlong Song
9c6cec2c10 f2fs-tools: fix overflow bug of start_sector when computing zone_align_start_offset
zone_align_start_offset should be u64, but config.start_sector is u32,
so it may be overflow when computing zone_align_start_offset.

Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:49:26 -07:00
Yunlong Song
baf9acd2c4 f2fs-tools: fix to match with the start_sector
f2fs-tools uses ioctl BLKSSZGET to get sector_size, however, this ioctl
will return a value which may be larger than 512 (according to the value
of q->limits.logical_block_size), then this will be inconsistent with
the start_sector, since start_sector is got from ioctl HDIO_GETGEO and
is always in 512 size unit for a sector. To fix this problem, just
change the sector_size to the default value when computing with
start_sector. And fix sectors_per_blk as well.

Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-27 23:49:26 -07:00
Jaegeuk Kim
689fd9b885 fsck.f2fs: add -O features to tune the bits
This patch add -O features for fsck.f2fs in order to tune the feature bits.
Currently, it supports -O encrypt only.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-07-13 22:46:00 -07:00
Sheng Yong
cc31eb7aba resize.f2fs: clear CP_COMPACT_SUM_FLAG when rebuilding checkpoint
Resize rebuilds checkpoint with 6 summary blocks, so if
CP_COMPACT_SUM_FLAG is set in the old checkpoint, clear it.

Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-07-13 22:46:00 -07:00
Jaegeuk Kim
1c15c8d551 resize.f2fs: fix wrong nat_bits migration
When wring the last nat_bits in new checkpoint, we call write_nat_bits()
to recover the nat_bits. But, we didn't update nm_i->nat_bitmap which we can
get the up-to-date nat blocks. After sit migration, old checkpoint has no
longer valid nat blocks, resulting in wrong free nid allocation at the following
mount time.

Let's indicate new nat_blocks by adjusting nat_bitmap in nat migration.

Reported-by: Lianjun Huang <huanglianjun@vivo.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-06-28 19:21:49 -07:00
Chao Yu
baaa076b4d mkfs.f2fs: expand scalability of nat bitmap
Previously, our total node number (nat_bitmap) and total nat segment count
will not monotonously increase along with image size, and max nat_bitmap size
is limited by "CHECKSUM_OFFSET - sizeof(struct f2fs_checkpoint) + 1", it is
with bad scalability when user wants to create more inode/node in larger image.

So this patch tries to relieve the limitation, by default, limitting total nat
entry number with 20% of total block number.

Before:
image_size(GB)	nat_bitmap	sit_bitmap	nat_segment	sit_segment
16		3836		64		36		2
32		3836		64		72		2
64		3772		128		116		4
128		3708		192		114		6
256		3580		320		110		10
512		3260		640		100		20
1024		2684		1216		82		38
2048		1468		2432		44		76
4096		3900		4800		120		150

After:
image_size(GB)	nat_bitmap	sit_bitmap	nat_segment	sit_segment
16		256		64		8		2
32		512		64		16		2
64		960		128		30		4
128		1856		192		58		6
256		3712		320		116		10
512		7424		640		232		20
1024		14787		1216		462		38
2048		29504		2432		922		76
4096		59008		4800		1844		150

Add a new option '-i' to control turning on/off this feature.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-07 14:49:37 -07:00
Gaoxiang (OS)
2e65d03730 resize.f2fs: fix max_nat_bitmap_size miscalculatation
For the same reason as "mkfs.f2fs: fix to calculate
left space of checkpoint page correctly",
this patch also fixes miscalculatation in resize.f2fs.

Signed-off-by: Gao Xiang <gaoxiang25@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-01-02 19:25:21 -08:00
Jaegeuk Kim
43bb7b6c99 f2fs-tools: build binaries in Mac
This patch modifies f2fs-tools to be built in mac.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-14 19:43:48 -08:00
Jaegeuk Kim
191573e71b mkfs.f2fs: support nat_bits feature
This patch adds nat_bits in the last one ore two 4KB-sized blocks of first
checkpoint segment. So, now a set of checkpoint *segment* consists of:

< checkpoint segment >
----------------------
| cp page #1         | 4KB
----------------------
| compacted data sum | 4KB
----------------------
| node sum  #1       | 4KB
----------------------
| node sum  #2       | 4KB
----------------------
| node sum  #3       | 4KB
----------------------
| cp page #2         | 4KB
----------------------
...
----------------------
| crc|cpver          | 8 bytes
----------------------
| full NAT bits      | # of NAT segments / 2 * 512 / 8 bytes
----------------------
| empty NAT bits     | # of NAT segments / 2 * 512 / 8 bytes
----------------------

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-02-23 10:11:41 -08:00
Sheng Yong
2b4b86066b resize.f2fs: correct checkpoint_ver of new checkpoint
resize.f2fs rebuilds a new checkpoint at the new position. Its
checkpoint_ver should be increased.

Signed-off-by: Shuoran Liu <liushuoran@huawei.com>
Signed-off-by: Weichao Guo <guoweichao@huawei.com>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-02-06 14:49:39 -08:00
Yunlei He
d466dea52f resize.f2fs: remove unused parameters
This patch remove unused parameters in these three functions:
get_new_sb, migrate_main and rebuild_superblock

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-12-12 11:09:30 -08:00
Junling Zheng
e8768a998d resize.f2fs: fix the number of moved ssa blocks in migrate_ssa
If the offset passed in migrate_ssa is not zero, it means that there're
offset segments of old main will disappear after migrating, then there're
offset blocks of old ssa should be invalidated and removed accordingly.
So, the number of moved ssa blocks should be: TOTAL_SEGS(sbi) - offset,
and the expanded summary, which is filled with zero_blocks, should start
from: new_sum_blkaddr + TOTAL_SEGS(sbi) - offset.

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-12-01 15:12:46 -08:00
Junling Zheng
1d81a373bd resize.f2fs: correct the max segno in migrate_main
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-28 21:26:15 -08:00
Yunlei He
5e056bef17 resize.f2fs: modify condition for resize
it better to decide resize or not by sector num.

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-07 09:57:52 -08:00
Jaegeuk Kim
7401b7ce2d resize.f2fs: fill zeros for expanded ssa area
This will avoid wrong behavior after expanding volume.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-02 22:25:29 -07:00
Jaegeuk Kim
8f346ef3a5 resize.f2fs: get latest free segments
The # of free segments can be changed by defragmentation.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-02 21:19:37 -07:00
Jaegeuk Kim
b467030078 resize.f2fs: fix wrong end_blkaddr
The end_blkaddr should add main area segments.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-02 18:01:55 -07:00
Jaegeuk Kim
6ab16edfcf resize.f2fs: fix wrong offset calculation
When we move whole main area into new expanded area, we do not change its
segment numbers from sit and ssa.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-02 17:02:02 -07:00
Jaegeuk Kim
0033920853 resize.f2fs: add more information in debug messages
Print more information.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-11-02 17:02:02 -07:00
Jaegeuk Kim
3ffc5a8f3e f2fs-tools: use shorter config variable name
This patch has no functional change.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-09-22 19:32:14 -07:00
Sheng Yong
49503225c0 f2fs-tools: update the format of output message
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-07-15 10:17:20 -07:00
Yunlei He
91bd80c441 resize.f2fs: replace the overprovision calculation method
This patch replace the overprovision calculation method from
mkfs tool.

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-04-27 14:02:26 -07:00
Jaegeuk Kim
d3be08825e resize.f2fs: support to expand partition size
Now user can expand existing partition with resize.f2fs.
Currently, it doesn't support shrink an image.

For example,
 # resize.f2fs -t [# of sectors] [image]

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2016-04-18 21:11:26 -04:00