Change to calculate and export max image size with valid blocks
used of image rather than last used block of image.
[FSCK] Max image size: xx MB, Free space: xx MB
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
commit b964b79d495d058f645f5a63357699a6c295025e
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
It is recommended to test a HAVE_* macro instead of directly testing the
host type in source code. Hence this patch.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit 2e59ab8fa6519e1169631189bd50810cb6b49426
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
The <sparse/sparse.h> header is available in Android but not in the
Android NDK. Hence this patch that only includes the sparse header file
if it is available.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit f3033fbc8ab80a026740171cdbad44d8bd5fd882
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
This makes it easier to build f2fs-tools for Windows.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit 87d7a95e3134b668e9f1e8519a19f390509fac60
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
Many format strings use one of the PRI* macros. These macros are compatible
with the uint types but not with the u_int types. Hence this patch
that switches from the u_int to the uint types.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit 1d2683f551c6d78c325dac7526dea3571879b845
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
This patch adds a readonly feature on small partitions by eliminating
on-disk layout for data writes such as provisioning/reserved space and
SSA.
This requires f2fs updates to understand RO feature in superblock.
Tested 4 cases:
sload.f2fs -c -a lz0 -f kernel/fs $DEV
sload.f2fs -c -a lz4 -f kernel/fs $DEV
sload.f2fs -c -r -a lz4 -f kernel/fs $DEV
sload.f2fs -c -L 3 -r -a lz4 -f kernel/fs $DEV
after:
mkfs.f2fs -O ro,compression,extra_attr -f $DEV
fsck.f2fs $DEV
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
commit b585244e726c1024f7b842033689992a383808cc
category: bugfix
issue: #I6VAS0
CVE: NA
Signed-off-by: DongSenhao <dongsenhao2@huawei.com>
---------------------------------------
Add F2FS compression support for sload
* Support file extension filter, either default-accept or default-deny
policy
* Support choice of compression algorithm, LZO (version 2) or LZ4
(default)
* Support custom log of cluster size
* Support minimum number of compressed blocks per cluster (default 1).
A cluster will not be compressed if the number can not be met.
* suuport -r (read-only) option
This releases compressed blocks to secure free space in advance. Note that,
all compressed files will have the immutable bit.
* Added manpage update
* Remove unecessary qbuf allocation (Jaegeuk, suggested by Satya)
Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: fix some bugs and refactor names]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
create mode 100644 fsck/compress.c
create mode 100644 fsck/compress.h
Signed-off-by: dongsenhao <dongsenhao2@huawei.com>
NVM Express Zoned Namespace (ZNS) devices can have zone-capacity(zc) less
than the zone-size. ZNS defines a per zone capacity which can be equal
or less than the zone-size. Zone-capacity is the number of usable blocks
in the zone. If zone-capacity is less than zone-size, then the segments
which start at/after zone-capacity are considered unusable. Only those
segments which start before the zone-capacity are considered as usable
and added to the free_segment_count and free_segment_bitmap of the kernel.
In such cases, the filesystem should not write/read beyond the
zone-capacity.
Update the super block with the usable number of blocks and free segment
count in the ZNS device zones, if zone-capacity is less than zone-size.
Set reserved segment count and overprovision ratio based on the usable
segments in the zone.
Allow fsck to find the free_segment_count based on the zone-capacity and
compare with checkpoint values.
Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
[Jaegeuk Kim: add UNUSED to is_usable_seg()]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Implemented cache options in F2FS configuration 'c' (i.e. user
options):
* use c.cache_config.num_cache_entry to set the number of
cache entries (in block), minimum 1024, or 0 to disable cache.
* use c.cache_config.max_hash_collision to set maximum hash
collision (max 16).
* use c.cavhe_config.dbg_en to enable printing of debug messages.
Cache properties:
* Per block save based (block size = f2fs block size = 4K Bytes)
* Device block indices are used to hash (simple modular hash
function) to cache indices.
* A hash collision (when a block is hashed to an already occupied
cache block) will trigger offset-based relocation to max 16
other places, at first empty slot found policy.
* A hash collision with all 17 possible relocating places occupied
will evict the oldest used one (replaced by the new read block).
A global 'use' tick is kept ticking-up per block read for this
purpose.
* On write cache hit, it will update the corresponding cached
block.
* On write cache miss, the data won't be cached.
* Write-through policy only: never delayed write.
* On read cache hit, it returns the cached block.
* On read cache miss, it issues an I/O read to the kernel, and
then caches and returns the block.
* Currently read ahead is not implemented. Yet, the read ahead
logic is kept as before, i.e. using kernel's read ahead
mechanism.
Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: Fix one bug in dcache_update_rw()]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch tries to fix memory leak problem reported in Android.
Fixed the following problems in fsck.f2fs, make_f2fs and sload_f2fs:
* reuse of same pointer without clean-up
* exit on error without clean-up
Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: add missing definition to avoid build error]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As zhaowuyun reported:
we met one problem of f2fs, and found one issue of make_f2fs, so I write
this email to search for your help to confirm this issue.
The issue was found on one of Android projects. We use f2fs as the filesystem
of userdata, and make sparse userdata.img using following command, which
invoked in script mkf2fsuserimg.sh
make_f2fs -S $SIZE -f -O encrypt -O quota -O verity $MKFS_OPTS $OUTPUT_FILE
use fastboot to flash this userdata.img to device, and it encountered f2fs
problem and leading to the mount fail of data partition.
we can make this issue 100% persent reproduced by making the data partition
dirty before flashing userdata.img.
suspect that issue is caused by the dirty data in the data partition.
so we checked that source code of make_f2fs in f2fs-tool, found that when
making f2fs, it use dev_fill to do some process:
...
we change code to the following, and the issue is gone.
if (c.sparse_mode)
return dev_write(buf, offset, len);
Chao Yu:
>
> After checking the codes, IIUC, I guess the problem here is, unlike
> img2simg, mkfs.f2fs won't record zeroed block in sparse image, so
> during transforming to normal image, some critical region like
> NAT/SIT/CP.payload area weren't be zeroed correctly, later kernel may
> load obsoleting data from those region.
>
> Also, The way you provide will obviously increase the size of sparse
> file, since with it we need to write all zeroed blocks of
> NAT/SIT/CP.payload to sparse file, it's not needed.
>
> Not sure, maybe we should use sparse_file_add_fill() to record zeroed
> blocks, so that this will make formatted image more like img2simged one.
Jaegeuk:
> We have to call sparse_file_add_fill() for dev_fill().
This patch fixes to support writing fill chunk sparse file for those
zeroed blocks in mkfs.f2fs.
Reported-and-tested-by: zhaowuyun <zhaowuyun@wingtech.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Added a missing free statement related to each device’s path.
Signed-off-by: Sotirios-Efstathios Maneas <smaneas@cs.toronto.edu>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Malloc Failure occurs in 32bit Windows, when using fastboot.exe flash the
f2fs sparse image filling with up to 2G chunk size.
Signed-off-by: Gao Ming <gaoming20@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The libsparse in AOSP is updating the type for 'len', from 'int' to
'size_t', in the callback parameter of sparse_file_foreach_chunk(). The
value represents a chunk size, which could be legitimately larger than
INT_MAX. This patch tracks the libsparse API change.
The change is guarded with SPARSE_CALLBACK_USES_SIZE_T that's exposed as
part of the libsparse change. This allows f2fs-tools to keep working
against older libsparse versions.
Signed-off-by: Tao Bao <tbao@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
tests/generic/405 of fstest suit expects that mkfs will return error
when it hits EIO, so let's propagate error from
f2fs_{finalize,fsync}_device, then mkfs can be aware of fsync error.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: initialize ret to zero]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
With --dry-run enabled, fsck.f2fs will do all checks and "fixes" except
that all fixes will not be written to storage at last.
Signed-off-by: Sheng Yong <shengyong1@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We use f2fs_finalize_device to fsync previous data in
checkpoint area before write last CP pack, in order to avoid
cp corruption in sudden-power-off case. But this function will
close the device, so this patch introduce a function to call
fsync() only.
Signed-off-by: Yunlei He <heyunlei@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Description:
modify I/O operations to support writing in sparse format
build host executable for making sparse image
Signed-off-by: Yang Jin <yajin@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds an option to specify multiple devices for an f2fs instance.
Up to 7 devices in addition to the default device can be added.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch remains user specified triggering information in superblock.
Then, if the information was changed, fsck.f2fs is triggered.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds supporting dump_file, which can extract a file from image.
You can simply select [yes|no] when doing dump.f2fs -i [inode number] [img].
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This is required so that libf2fs.c can be used against any kind
of device (E.g. an in-memory sparse file) just by linking against
something that provides the libf2fs_io.c functions.
libf2fs is currently libf2fs.c + libf2fs_io.c
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
* removed unused includes.
* removed unused parameters.
* Fixed a bunch of warnings around:
int i;
if (i < some_uint) ...
and
u32 x;
...
if (x < 0) return error;
* Protect BLKDISCARD usage if it is not available.
Change-Id: Iede035b1beb2df01c961589a69aff47a5258ecd2
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Support an fs_io dev_fill().
This helps with devices that actually already return 0 on uninitialized
data. It also helps with in memory devices using sparse libs: don't
allocate a block of 0s.
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This will allow turning mkfs into a libarary more easily.
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
