Looking at bsdiff's comment, original author want to generate a diff
patch only if diff is no smaller than 8 bytes. Actual code only only
permits the diff to be generated if diff > 8 bytes.
Partners reported a pair of problematic files where BSDIFF takes more
than 10 hours(input file is only ~80MB). After applying this patch,
bsdiff takes ~1 minute.
To make sure this patch does not harm patch size for non-pathological
cases, I tested multiple pixel OTAs. Total OTA size increased by just
0.002% . Basically negligible.
Test: Multiple pixle OTAs
Bug: 285271992
Change-Id: I2a9b52939f8d17ea48dc2908942b2cc21cbe4e09
We modified the patch writer to have the capability to produce the smallest
patch if multiple compressors are given. This type is supported for bsdf2
format only.
Example usage:
bsdiff --format=bsdf2 --type=bz2:brotli <src_file> <tgt_file> <patch_file>
Test: diff & patch an apk file
Change-Id: Icd4322f21975b82c5ee09f8feb9321f52c9813a4
Switch the internal member of the patch writer from a single compressor to a
list of compressors for ctrl|diff|extra streams. The change doesn't affect the
functionality of the existing bz2|brotli compressors; but later allow us to try
all the compressors and select the one with the best compression ratio.
With this new writer, callers will be able to produce the smallest patch without
calling bsdiff explictly multiple times.
Test: unit tests pass
Change-Id: I441ae6636e2299b2cfc9d14f50952c85585dbb4f
Right now the quality is only used in the brotli compressor. And it
would be ambiguous if we add another compressor in the future; since the
compressors may have different interpretion on the quality value.
Test: unit tests pass; run bsdiff
Change-Id: I23af06a135b2b47df5209171f7db0773ac02a326
Add a parser of format, type and quality in the bsdiff main function.
This allow the bsdiff binary to choose between the BSDFF40|BSDF2 patch
format and compress the patch with desired algorithms.
Bug: 34220646
Test: unittest pass; generate brotli compressed patches with bsdiff
binary
Change-Id: Ia4f7941e4eca7e6047e2749315127d1cf4a988ee
The logger supports LOG & PLOG. It also adds a std::endl at the end of
the stream so that we don't need to add it manually at the callsite.
Test: run bsdiff and check the logs.
Change-Id: Ice94902a501532ceaeb480c3576e752e435cab70
Add a new BSDF2 format that allows compression of ctrl/diff/extra streams
with different algorithms. The compression algorithm supported for now is
bz2 and brotli. This new format is similar to the legacy "BSDIFF40" format
except for the magic headers.
File format:
0 8 magic header
8 8 X
16 8 Y
24 8 new_file_size
32 X compressed control block
32+X Y compressed diff block
32+X+Y ??? compressed extra block
The magic header for BSDF2 format:
0 5 BSDF2
5 1 compressed type for control stream
6 1 compressed type for diff stream
7 1 compressed type for extra stream
Bug: 34220646
Test: unittest pass
Change-Id: I6eb0867e88476dbc4a4b7be609783f8c8a0a41cd
This CL implements the brotli decompressor used in PatchReader. Also
extract the CompressorType and patch's MagicHeader to a common header file.
Bug: 34220646
Test: Run bsdiff/bspatch with brotli compressor/decompressor over a list
of files from angler's system image.
Change-Id: I76e11168075c6481490ffbc025ec4ca81e828732
Add a new compressor interface; and the brotli compressor which is
similar to the existing bz2 one.
Also add a new CompressorType argument to PatchWriter's init function;
this allows us to choose the right compressor based on the input option
in the future.
Bug: 34220646
Test: Run bsdiff/bspatch with brotli compressor/decompressor over a
list of files from angler's system image. Make succeeds.
Change-Id: Id9a3db2d7d051dcb751a1fc362f4c3b9226e878b
Most patch formats include the size of the new file in the header.
To help streaming the patch to disk while generating it, this CL
passes the size of the new file to the patch writer on initialization.
To do this, we also move the Init() call to the patch writer to the
DiffEncoder, which makes more sense since the Close() call is also made
from the DiffEnconder.
Bug: None
Test: Updated tests to check for this value.
Change-Id: Idfaedbd492d68ab6e6cb2c1cb3883947f068c3aa
Add a wrapper class to separate the patch read from data stream
decompression. Therefore, bspatch will be able to process the patch
that is compressed with various tools.
Test: unittest pass
Change-Id: I5214e0451bde80366e8a70b960703afb2b2a7d97
The recently introduced PatchWriterInterface had both the patch file
format logic and the diff/extra streams selection from the control
entries. While this simplifies the bsdiff main algorithm and makes
invalid usages of the BsdiffPatchWriter evident, writing alternative
PatchWriterInterface classes required to replicate the diff/extra
stream selection logic.
This patch splits out the diff/extra stream generation and all the
checks around those to a new helper class DiffEncoder. The public
interface PatchWriterInterface now has two methods to accept the diff
and extra stream data, and does not compute them.
Bug: 34220646
Test: Added unittests. Ran bsdiff on some pairs of files obtaining the same result as before.
Change-Id: I5f303c06f1e10910eb00dcfda38c6811977a91cf
Currently, all bsdiff() functions take a filename for the patch, which
is called with a temporary file in all cases. To help expose an
interface that allows to write bsdiff patches in different formats (for
example, changing the compressor and header format) we expose the
PatchWriterInterface class in the public interface so callers can use
a different one or define their own to help experimenting with bsdiff
enconding improvements.
Bug: 34220646
Test: make bsdiff and update_engine; ran bsdiff_unittest
Change-Id: Ie450b2790137665bc033cb36d037171090b18a4b
This prevents accidentally including a header file from another project
that happens to have the same name.
Test: `make checkbuild`; emerge-${BOARD} bsdiff
Bug: None
Change-Id: I8af132ed388738c30a8e3d7434de70b1e9d2f924
bsdiff.cc had code to handle both the sub-streams compression and the
patch format. This CL moves the patch file format logic to a new class
in a separated file, and the bzip2 compression to another class. This
allows to replace the core bsdiff logic separatedly from the patch
format.
There are no changes to the functionality of bsdiff and the generated
patches. The code does a bit more of memcpy() since it doesn't write
the control stream to disk as it goes.
Bug: 34220646
Test: Ran bsdiff and compared results with previous version.
Change-Id: Ibdc1482e98cbd370cbed677c326008e19a217ef8