Update allocation policy for ro feature:
- hot_data: allocating blocks by LBA ascending order
- hot_node: allocating blocks by LBA descending order
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
commit fdff1ab3157c2787d908e013b9d42db84184e196
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
Since 'remained_blkentries' is unsigned, the assert statement that
verifies whether that variable is positive will never fail. Hence
remove that assert statement.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit e61203c2aae8656a178234bc20df2e4a5994407b
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
Suppress the following compiler warning:
segment.c:698:12: warning: ‘n’ may be used uninitialized in this function [-Wmaybe-uninitialized]
698 | if (n < 0)
| ^
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit 1d2683f551
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
This patch adds a readonly feature on small partitions by eliminating
on-disk layout for data writes such as provisioning/reserved space and
SSA.
This requires f2fs updates to understand RO feature in superblock.
Tested 4 cases:
sload.f2fs -c -a lz0 -f kernel/fs $DEV
sload.f2fs -c -a lz4 -f kernel/fs $DEV
sload.f2fs -c -r -a lz4 -f kernel/fs $DEV
sload.f2fs -c -L 3 -r -a lz4 -f kernel/fs $DEV
after:
mkfs.f2fs -O ro,compression,extra_attr -f $DEV
fsck.f2fs $DEV
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit 747b74cb9c
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
If sload.f2fs encounters a file with nr_links > 1, it will mark it
as a possible hard link by remembering the original device and
inode. When sload.f2fs creates the file, it will check if it has
already created a file for the same original device and inode. If
so, it will add the original inode to the directory and increment
the number of links to it, instead of writing a new inode.
This allows sload.f2fs to accurately reproduce a directory tree that
contains hard links, such as those created by ostree. Without this
patch, directory trees containing hard links result in the content of
the files being duplicated.
This is version 2 of the patch; it has been rebased against the dev
branch and includes a fix from Jaegeuk Kim to avoid building data
contents twice on hard linked files.
Co-authored-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: Jordan Webb <jordan@getseam.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit b585244e72
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
Add F2FS compression support for sload
* Support file extension filter, either default-accept or default-deny
policy
* Support choice of compression algorithm, LZO (version 2) or LZ4
(default)
* Support custom log of cluster size
* Support minimum number of compressed blocks per cluster (default 1).
A cluster will not be compressed if the number can not be met.
* suuport -r (read-only) option
This releases compressed blocks to secure free space in advance. Note that,
all compressed files will have the immutable bit.
* Added manpage update
* Remove unecessary qbuf allocation (Jaegeuk, suggested by Satya)
Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: fix some bugs and refactor names]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
create mode 100644 fsck/compress.c
create mode 100644 fsck/compress.h
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
A maliciously corrupted file systems can trigger buffer overruns in
the quota code used by fsck.
To fix it, quota file sizes are checked against real allocated
block index tables (inode, direct nodes, indirect nodes, double
indirect nodes). If the size mismatches, the quota file is considered
corrupted and will be regenerated.
Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds to support compression, introducing '-O compression'
option to enable this feature in image.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Fsck checks fsync data when UMOUNT flag is not set. When the f2fs was not
cleanly unmouted, UMOUNT flag is not recorded in meta data and fsync data
can be left in the f2fs. The first fsck run checks fsync data to reflect
it on quota status recovery. After that, fsck writes UMOUNT flag in the
f2fs meta data so that second fsck run can skip fsync data check.
However, fsck for zoned block devices need to care fsync data for all
fsck runs. The first fsck run checks fsync data, then fsck can check
write pointer consistency with fsync data. However, since second fsck run
does not check fsync data, fsck detects write pointer at fsync data end
is not consistent with f2fs meta data. This results in meta data update
by fsck and fsync data gets lost.
To have fsck check fsync data always for zoned block devices, introduce
need_fsync_data_record() helper function which returns boolean to tell
if fsck needs fsync data check or not. For zoned block devices, always
return true. Otherwise, return true if UMOUNT flag is not set in CP.
Replace UMOUNT flag check codes for fsync data with the function call.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
When fsck needs to assign a new area to a curreng segment, it calls
find_next_free_block() function to find a new block to assign. For zoned
block devices, fsck checks write pointer consistency with current
segments' positions. In case a curseg is inconsistent with the
write pointer of the zone it points to, fsck should assign not a new free
block but a new free zone/section with write pointer at the zone start,
so that next write to the current segment succeeds without error.
To extend find_next_free_block() function's capability to find not only
a block but also a zone/section, add new_sec flag to
find_next_free_block() function. When new_sec flag is true, skip check
for each block's availability so that the check is done with unit of
section. Note that it is ensured that one zone has one section for f2fs
on zoned block devices. Then the logic to find a new free section is good
to find a new free zone.
When fsck target devices have ZONED_HM model, set new_sec flag true to
call find_next_free_block() from move_curseg_info(). Set curseg's
alloc_type not SSR but LFS for the devices with ZONED_HM model, because
SSR block allocation is not allowed for zoned block devices. Also skip
relocate_curseg_offset() for the devices with ZONED_HM model for the
same reason.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Previously, we don't allow block allocation on unclean umounted image,
result in failing to repair quota system file.
In this patch, we port most recovery codes from kernel to userspace
tools, so that on unclean image, during fsck initialization, we will
record all data/node block address we may recover in kernel, and
then during allocation of quota file repair, we can skip those blocks
to avoid block use conflict.
Eventually, if free space is enough, we can repair the quota system
file on an unclean umounted image.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: remove unnecessary parameter]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Change logic as below:
- fix to account block/node/inode stats correctly in reserve_new_block()
- check overflow in reserve_new_block()
- move stat update from f2fs_alloc_nid() to reserve_new_block()
- adjust write_checkpoint() to update stat for sload/fsck
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
if we new node block in fsck flow, we need to update
the valid_node_cnt at the same time.
Signed-off-by: Lihong Kou <koulihong@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Calculate inode checksum and writeback when processing
sload or resize.
Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
[Chao Yu: change commit message and code]
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Now, kernel can catch incorrect total valid block count which is exceed
max user block count of image.
Then, generic/051,476 of fstest reports below message:
Apr 15 11:08:03 szvp000201624 kernel: [ 2533.515813] F2FS-fs (zram1): Wrong valid_user_blocks: 469505, user_block_count: 469504
Apr 15 11:08:03 szvp000201624 kernel: [ 2533.519166] F2FS-fs (zram1): Failed to get valid F2FS checkpoint
The reason is that when fsck repairs corrupted quota sysfile, it didn't
check max user block count when allocating new block for quota sysfile,
so ckpt.valid_block_count can exceed max user block count, result in
mount failure later.
Adding upper boundary check of block count in reserve_new_block() to
fix this issue.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As Jaegeuk reminded:
Once user updates f2fs-tools which support new fields in inode layout,
but do keep the kernel which can not support those fields, it will cause
old f2fs fail to mount new image due to root_inode's i_extra_isize value
sanity check.
So if f2fs-tools doesn't enable feature which will use new fields of
inode, we don't need to expand i_extra_isize to include them, let's just
let i_extra_isize point to the end of last valid extra field's position.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We have to keep data for roll-forward recovery. Without this patch, we're
able to lose there-in data by quota overwrites.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch updates sload.f2fs for android build.
- do fsck() after sload() to update quota information.
- return success, if there's no source directory
-C fs_config
-f source directory [path of the source directory]
-p product out directory
: path on fs_config file for uid/gid/mode
-s file_contexts
: selabel file
-t mount point [prefix of target fs path, default:/]
: path stored in selabel/fs_config files
-T timestamp
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch let fsck to check and fix quota file contents.
Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
This patch adds f2fs_read() and f2fs_filesize_update(). It also refactors
f2fs_write_block() and renamed as f2fs_write().
Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Found and fixed following three bugs in f2fs_write_block() function.
- Write (4096 - offset) bytes for the first block even for small count.
- For overwriting, found blkaddr is not used for writing.
- dn.idirty status can be lost by set_new_dnode().
- missing inode_checksum
Signed-off-by: Hyojun Kim <hyojun@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
This patch introduce a new option 'inode_checksum' for enabling inode
checksum functionality in mkfs/fsck/sload.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds an option 'extra_attr' in mkfs for enabling v2 f2fs
inode format in kernel codes.
Also this patch makes fsck to support recognize v2 inode format,
below is v2 format description:
Original one:
struct f2fs_inode {
...
struct f2fs_extent i_ext;
__le32 i_addr[DEF_ADDRS_PER_INODE];
__le32 i_nid[DEF_NIDS_PER_INODE];
}
Extended one:
struct f2fs_inode {
...
struct f2fs_extent i_ext;
union {
struct {
__le16 i_extra_isize;
__le16 i_padding;
__le32 i_extra_end[0];
};
__le32 i_addr[DEF_ADDRS_PER_INODE];
};
__le32 i_nid[DEF_NIDS_PER_INODE];
}
Once F2FS_EXTRA_ATTR is set, we will steal four bytes in the head of
i_addr field for storing i_extra_isize and i_padding. with i_extra_isize,
we can calculate actual size of reserved space in i_addr, available
attribute fields included in total extra attribute fields for current
inode can be described as below:
+--------------------+
| .i_mode |
| ... |
| .i_ext |
+--------------------+
| .i_extra_isize |-----+
| .i_padding | |
| .i_prjid | |
| .i_atime_extra | |
| .i_ctime_extra | |
| .i_mtime_extra |<----+
| .i_inode_cs |<----- store blkaddr/inline from here
| .i_xattr_cs |
| ... |
+--------------------+
| |
| block address |
| |
+--------------------+
| .i_nid |
+--------------------+
| node_footer |
| (nid, ino, offset) |
+--------------------+
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
It is reported that fsck.f2fs behaves abnormally when running on MIPS32
rel 2 big endian cpu, since incorrect endianness. So let's correct all
endianess issues of f2fs-tools.
Reported-by: <k@vodka.home.kg>
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch implements loading files into the existing partition.
For example,
# sload.f2fs -f ./ /dev/sdb1
Then, all the directories and files will be loaded into /dev/sdb1.
By default, newly files should have inline_data and inline_xattr, if possible.
Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
Signed-off-by: Liu Shuoran <liushuoran@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
