mirror of
https://github.com/borgbackup/borg.git
synced 2026-10-12 02:11:36 +00:00
Thomas WaldmannandClaude Opus 5
da0c417312
The byte counts of a "modified" file were derived from the difference of the two chunk id SETS, which loses everything about order and multiplicity: a file whose chunks were only reordered was reported as modified with 0 B added and 0 B removed, and duplicating a chunk added no bytes at all. Align the two chunk lists as sequences instead (difflib.SequenceMatcher, the way a text diff aligns lines) and count the chunks that are not part of the alignment: the ones of archive1 as removed, the ones of archive2 as added. Insertions, removals, moves and duplicated chunks are now all reflected by the byte counts. The common prefix and suffix of the two lists are stripped first. That is what makes the usual cases cheap (e.g. a file that was appended to) and it also keeps the matcher away from the long runs of identical chunks it is slow on. SequenceMatcher still degrades to quadratic runtime on chunk lists that repeat the same chunk id very often (sparse files, VM images with big all-zero ranges), so the alignment is skipped above MAX_ALIGN_CHUNKS / MAX_ALIGN_WORK and the chunk ids are only counted then, which is what borg did before. Co-Authored-By: Claude Opus 5 <[email protected]>