borgbackup/borg
1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2026-10-12 02:11:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

rabin-aes: new chunker with provable fingerprinting resistance

Browse Source 
Add a "UHF-then-PRF" content-defined chunker following the Chk-PHTE
construction of "Breaking and Fixing Content-Defined Chunking"
(Truong et al., eprint 2025/558): a rolling Rabin fingerprint over
GF(2)[x]/P(x) - with P a secret, random, irreducible polynomial of
degree 63 - hashes the last 64 bytes (a universal hash family), and
AES-128 with a secret key is applied to the digest; the cut decision
only looks at the AES output. Observed chunk boundaries are thus
pseudo-random and yield no usable equations about the chunking
secrets, unlike chunkers cutting directly on (keyed) rolling-hash
bits. Both secrets are derived from the id key (domain "rabin-aes").

The C scan kernel has three bit-identical paths: batched OpenSSL EVP
AES-128-ECB (portable), arm64 crypto-extension intrinsics and x86-64
AES-NI (runtime detected), interleaving the AES work with the serial
rolling-hash chain. Batching ahead of the cut decision is possible
because the digest stream does not depend on where cuts happen.

Throughput on an Apple M-series (1 GiB random, 19,23,21,2): 489 MB/s
(arm64 hw path), 334 MB/s (EVP path); chunk-size distribution, shift
resilience and dedup behavior are identical to fastcdc/buzhash64.
Normalized chunking, param shape rabin-aes,min,max,mask,nc_level.

Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
Thomas Waldmann
2026-08-04 15:43:33 +02:00
co-authored by Claude Fable 5
parent f8dee0c3ff
commit b7dcf30bd4
15 changed files with 1188 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/chunker_bench.py
+4 -4
View File
@@ -131,8 +131,8 @@ def chunk_stats(algo, data, min_exp, max_exp, mask_bits, win, nc_level=0, normal
if algo == "buzhash64":
params.append(nc_level) # nc_level is a positional param
kw["normal_size"] = normal_size
elif algo == "fastcdc":
params = [min_exp, max_exp, mask_bits, nc_level] # fastcdc is window-less
elif algo in ("fastcdc", "rabin-aes"):
params = [min_exp, max_exp, mask_bits, nc_level] # window-less param shape
kw["normal_size"] = normal_size
chunker = get_chunker(algo, *params, **kw)
sizes = []
@@ -288,11 +288,11 @@ def main():
print(f"shift test: {args.shift_edits} edits repeats: {args.repeat}")
print("-" * 118)
# build (algo, nc_level) variants; for buzhash64/fastcdc also run the requested NC level
# build (algo, nc_level) variants; for buzhash64/fastcdc/rabin-aes also run the requested NC level
variants = []
for algo in args.algo:
variants.append((algo, 0))
if algo in ("buzhash64", "fastcdc") and args.nc_level > 0:
if algo in ("buzhash64", "fastcdc", "rabin-aes") and args.nc_level > 0:
variants.append((algo, args.nc_level))
for algo, nc in variants: