17 Commits
Author SHA1 Message Date
Kelvin Zhang 023e6aef67 Exit loop earlier for bsdiff's pathological case
Looking at bsdiff's comment, original author want to generate a diff
patch only if diff is no smaller than 8 bytes. Actual code only only
permits the diff to be generated if diff > 8 bytes.

Partners reported a pair of problematic files where BSDIFF takes more
than 10 hours(input file is only ~80MB). After applying this patch,
bsdiff takes ~1 minute.

To make sure this patch does not harm patch size for non-pathological
cases, I tested multiple pixel OTAs. Total OTA size increased by just
0.002% . Basically negligible.

Test: Multiple pixle OTAs
Bug: 285271992

Change-Id: I2a9b52939f8d17ea48dc2908942b2cc21cbe4e09
2023-06-15 10:58:41 -07:00
Kelvin Zhang 7a375134b7 Expose PatchWriter for consumption
Test: th
Change-Id: I1d0657bc886967e73b926fae13300bd401696d6a
2021-07-15 16:42:15 -04:00
Sen Jiang eb95b7ed86 Return false for no compression.
Test: None
Change-Id: I21991c6ae9e55eac30d411e827d579e762c1fe86
2018-11-26 13:32:54 -08:00
Tianjie Xu 77833b6477 Allow bsdiff to accept multiple compressors
We modified the patch writer to have the capability to produce the smallest
patch if multiple compressors are given. This type is supported for bsdf2
format only.

Example usage:
bsdiff --format=bsdf2 --type=bz2:brotli <src_file> <tgt_file> <patch_file>

Test: diff & patch an apk file

Change-Id: Icd4322f21975b82c5ee09f8feb9321f52c9813a4
2018-03-13 19:10:13 -07:00
Tianjie Xu 32b1f21338 Modify the patch writer to use a list of compressors for each stream
Switch the internal member of the patch writer from a single compressor to a
list of compressors for ctrl|diff|extra streams. The change doesn't affect the
functionality of the existing bz2|brotli compressors; but later allow us to try
all the compressors and select the one with the best compression ratio.

With this new writer, callers will be able to produce the smallest patch without
calling bsdiff explictly multiple times.

Test: unit tests pass
Change-Id: I441ae6636e2299b2cfc9d14f50952c85585dbb4f
2018-03-07 11:15:38 -08:00
Tianjie Xu 2e70b558e7 Rename the variable 'quality' to 'brotli_quality'
Right now the quality is only used in the brotli compressor. And it
would be ambiguous if we add another compressor in the future; since the
compressors may have different interpretion on the quality value.

Test: unit tests pass; run bsdiff
Change-Id: I23af06a135b2b47df5209171f7db0773ac02a326
2018-03-02 16:47:09 -08:00
Tianjie Xu 1f1cdb2b6b Add an argument parser in bsdiff
Add a parser of format, type and quality in the bsdiff main function.
This allow the bsdiff binary to choose between the BSDFF40|BSDF2 patch
format and compress the patch with desired algorithms.

Bug: 34220646
Test: unittest pass; generate brotli compressed patches with bsdiff
binary
Change-Id: Ia4f7941e4eca7e6047e2749315127d1cf4a988ee
2018-01-18 12:32:09 -08:00
Tianjie Xu 18480eb706 Implement a simple logger for bsdiff
The logger supports LOG & PLOG. It also adds a std::endl at the end of
the stream so that we don't need to add it manually at the callsite.

Test: run bsdiff and check the logs.
Change-Id: Ice94902a501532ceaeb480c3576e752e435cab70
2017-11-30 12:28:06 -08:00
Tianjie Xu b4cba64184 Implement a new bsdiff format
Add a new BSDF2 format that allows compression of ctrl/diff/extra streams
with different algorithms. The compression algorithm supported for now is
bz2 and brotli. This new format is similar to the legacy "BSDIFF40" format
except for the magic headers.

File format:
0       8    magic header
8       8    X
16      8    Y
24      8    new_file_size
32      X    compressed control block
32+X    Y    compressed diff block
32+X+Y  ???  compressed extra block

The magic header for BSDF2 format:
0 5 BSDF2
5 1 compressed type for control stream
6 1 compressed type for diff stream
7 1 compressed type for extra stream

Bug: 34220646
Test: unittest pass
Change-Id: I6eb0867e88476dbc4a4b7be609783f8c8a0a41cd
2017-11-16 14:46:37 -08:00
Tianjie Xu 4d10c3efd7 Add a brotli decompressor in bsdiff
This CL implements the brotli decompressor used in PatchReader. Also
extract the CompressorType and patch's MagicHeader to a common header file.

Bug: 34220646
Test: Run bsdiff/bspatch with brotli compressor/decompressor over a list
of files from angler's system image.

Change-Id: I76e11168075c6481490ffbc025ec4ca81e828732
2017-11-14 14:36:55 -08:00
Tianjie Xu 1c26e2ed9b Add the brotli compressor for bsdiff
Add a new compressor interface; and the brotli compressor which is
similar to the existing bz2 one.

Also add a new CompressorType argument to PatchWriter's init function;
this allows us to choose the right compressor based on the input option
in the future.

Bug: 34220646
Test:  Run bsdiff/bspatch with brotli compressor/decompressor over a
list of files from angler's system image. Make succeeds.
Change-Id: Id9a3db2d7d051dcb751a1fc362f4c3b9226e878b
2017-10-31 11:37:45 -07:00
Alex Deymo 4dadd8b443 Pass the size of the new file to the PatchWriterInterface::Init()
Most patch formats include the size of the new file in the header.
To help streaming the patch to disk while generating it, this CL
passes the size of the new file to the patch writer on initialization.

To do this, we also move the Init() call to the patch writer to the
DiffEncoder, which makes more sense since the Close() call is also made
from the DiffEnconder.

Bug: None
Test: Updated tests to check for this value.
Change-Id: Idfaedbd492d68ab6e6cb2c1cb3883947f068c3aa
2017-10-27 13:52:46 +02:00
Tianjie Xu 6528812247 Add an interface for bspatch reader
Add a wrapper class to separate the patch read from data stream
decompression. Therefore, bspatch will be able to process the patch
that is compressed with various tools.

Test: unittest pass
Change-Id: I5214e0451bde80366e8a70b960703afb2b2a7d97
2017-10-26 14:50:34 -07:00
Alex Deymo 68c0e7f206 Reduce PatchWriterInterface functionality.
The recently introduced PatchWriterInterface had both the patch file
format logic and the diff/extra streams selection from the control
entries. While this simplifies the bsdiff main algorithm and makes
invalid usages of the BsdiffPatchWriter evident, writing alternative
PatchWriterInterface classes required to replicate the diff/extra
stream selection logic.

This patch splits out the diff/extra stream generation and all the
checks around those to a new helper class DiffEncoder. The public
interface PatchWriterInterface now has two methods to accept the diff
and extra stream data, and does not compute them.

Bug: 34220646
Test: Added unittests. Ran bsdiff on some pairs of files obtaining the same result as before.
Change-Id: I5f303c06f1e10910eb00dcfda38c6811977a91cf
2017-10-09 12:50:33 +02:00
Alex Deymo 538a75d13b Make BsdiffPatchWriter into an Interface.
Currently, all bsdiff() functions take a filename for the patch, which
is called with a temporary file in all cases. To help expose an
interface that allows to write bsdiff patches in different formats (for
example, changing the compressor and header format) we expose the
PatchWriterInterface class in the public interface so callers can use
a different one or define their own to help experimenting with bsdiff
enconding improvements.

Bug: 34220646
Test: make bsdiff and update_engine; ran bsdiff_unittest

Change-Id: Ie450b2790137665bc033cb36d037171090b18a4b
2017-09-28 19:36:53 +02:00
Alex Deymo dcd423bf89 Add the "bsdiff/" prefix to all include paths.
This prevents accidentally including a header file from another project
that happens to have the same name.

Test: `make checkbuild`; emerge-${BOARD} bsdiff
Bug: None

Change-Id: I8af132ed388738c30a8e3d7434de70b1e9d2f924
2017-09-25 10:56:24 +02:00
Alex Deymo a28e01937d Refactor bsdiff patch writing code to another file.
bsdiff.cc had code to handle both the sub-streams compression and the
patch format. This CL moves the patch file format logic to a new class
in a separated file, and the bzip2 compression to another class. This
allows to replace the core bsdiff logic separatedly from the patch
format.

There are no changes to the functionality of bsdiff and the generated
patches. The code does a bit more of memcpy() since it doesn't write
the control stream to disk as it goes.

Bug: 34220646
Test: Ran bsdiff and compared results with previous version.

Change-Id: Ibdc1482e98cbd370cbed677c326008e19a217ef8
2017-09-15 15:33:11 +02:00