Since the PyPy nightly of 2026-10-11 (PyPy 8.0.2-alpha0, rev 544bd2f768f9), 12 tests
in archive_test.py (test_missing_chunk, test_corrupt_chunk, test_resync_*) fail with:
BufferError: Existing exports of data: object cannot be re-sized
at `del self._buffered_data[:drop_len]` in RobustUnpacker._search().
_search() scanned the buffer inside `with memoryview(self._buffered_data) as data:`
and called valid_msgpacked_dict(data[offset:]) for every offset, which also sliced the
key out of that view. CPython frees these unreferenced child views immediately, so the
bytearray is exportless again as soon as the with block releases the parent view.
PyPy has no refcounting: the child views stay alive until the GC runs. PyPy 8.0.0 did
not count child views as exports of the bytearray, the new nightly does (like CPython
always did), so the resize after the with block fails.
Fix: valid_msgpacked_dict(d, offset=0) only indexes d, it never slices it, and _search()
passes (data, offset) instead of a slice. Together with the explicitly released slice in
_try_item_start(), no unreleased view of _buffered_data exists anymore when it is
resized. The scan loop also gets a bit faster on CPython (no slice object per offset).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
The "repository was previously located at" check compared path strings only,
so reaching the same local repository directory via a symlink, a bind mount or
a different letter case on a case-insensitive filesystem looked like a
relocation. Answering yes rewrote the stored location, so alternating between
the paths kept triggering the prompt (breaking e.g. unattended prune).
If the strings differ, but both are local paths referring to the same existing
directory (same st_dev/st_ino, st_ino != 0), do not warn. The stored location
and all displayed paths stay as they were. Remote locations are not affected,
and a real move/rename (old path gone) still triggers the warning.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
That cost an extra open/fstat/close per directory (noticeable when e.g.
find feeds millions of entries, and on network filesystems).
It is not needed: if such a directory is an automount point that is not
mounted yet, the race condition check in process_dir triggers a retry,
which then sees the mounted filesystem. Usually, the program producing
the paths has already triggered the mount anyway.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The files cache (file sizes, timestamps, inode numbers, chunk lists and
--digests content digests) was stored as plaintext msgpack in the local
cache directory, protected only by a checksum without a key, so anyone
who could read the cache directory could read it, and anyone who could
write it could make borg skip changed files.
It is now written as a sealed stream: encrypted and authenticated with
the repository key (authenticated only in the authenticated-* modes),
bound to the repository id and the file name of the files cache. A files
cache that fails the verification is ignored with a warning (the exit
code is not changed) and rebuilt from the previous archive.
The checksums in the [integrity] section of the cache config are not
needed any more, so the cache config is no longer rewritten on every
cache close().
The FAQ about cache security now describes what is protected and what is not.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A sealed stream cuts a byte stream into 1 MiB frames and puts every
frame into the key's envelope (encrypted and authenticated in the
encrypting modes, authenticated only in the authenticated-* modes),
so neither the writer nor the reader needs memory for more than a frame.
The AAD of a frame is a caller-given context plus the frame number and
a "last frame" flag, so the reader detects changed, reordered, dropped
or duplicated frames, truncation and trailing data. Every frame is
sealed with encrypt_oneshot(), so sealed streams are thread-safe.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A lock object created by another user (e.g. by borg running as root) made
borg crash with a PermissionError traceback. Treat it like any other
unreadable lock object: it blocks us (fail closed), is named in the error
message and can be removed with borg break-lock.
stat() does not trigger an automount (autofs, systemd automount units,
ZFS .zfs/snapshot/*), but opening the directory does. borg then saw a
different inode in fstat() than in the stat() before and skipped the
whole tree as a race condition.
Now, if a directory turns into another directory on a different mount
than its parent directory, we accept that as an automount, use the
fstat() result and redo the skip_inodes and --one-file-system decisions.
platform.on_different_mounts() compares the mount ids on Linux (st_dev
is not good enough there, e.g. for btrfs subvolumes) and st_dev elsewhere.
Recursion roots and --paths-from-stdin directories are opened before
taking their stat, so --one-file-system uses the device number of the
mounted filesystem (this also fixes incomplete -x backups of a ZFS
snapshot directory given as a recursion root).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
fstab and autofs entries can only give mount options, not environment
variables, so an encrypted repository could not be mounted from there
without setting BORG_PASSCOMMAND for the whole automount daemon.
-o passcommand=CMD works like BORG_PASSCOMMAND=CMD. It is applied before
the repository is opened (the key is loaded before the other mount
options are processed) and is not passed on to libfuse. Like the
passphrase environment variables, it is mutually exclusive with them.
There is deliberately no mount option for the passphrase itself: mount
options are visible to other users (/etc/fstab, ps output, systemd
mount units, autofs logs).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Same as BORG_MOUNT_ARCHIVE_DIR_FORMAT, but usable from fstab and autofs
entries; if both are given, the mount option is used. As mount options
are separated by commas, the format can not contain a comma.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Directories: same_item() already considered an existing directory with the archived
mode and mtime as complete, but do_extract never passed continue_extraction when
restoring directory metadata, so all directories got their metadata restored again.
--progress: the total includes the files that --continue skips, but their size was
never added to the progress, so the percentage stayed far below the real one.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Like strip_components, this makes --numeric-ids usable from fstab and
autofs entries. Also clarify in the docs that numeric-ids was the
--numeric-ids CLI option, not a mount option.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
fstab and autofs entries can only give mount options, not borg CLI
options, so -o strip_components=N now does what --strip-components N
does (the mount option wins if both are given).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
borg create --sparse seeks over holes, but the archive still does not
record the hole layout, so extract --sparse can not reproduce it exactly.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Haiku has no "root" user, so the name lookup fell back to the name and
the test expected uid 0 in vain.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Repository.__init__ now parses and validates both variables, so a bad
value fails before the store is created, opened or locked. Each must be
a positive integer, and BORG_PACK_MAX_SIZE must be below the new
MAX_PACK_SIZE_LIMIT of 2126512128 (2 GiB - 20 MiB). PackWriter.add keeps
the object that crosses the cap, so packs stay below 2 GiB, clear of OS
bugs with files of 2 GiB or more. A rejected value raises Error with the
variable name and the value.
With only BORG_PACK_MAX_COUNT set, the writer still gets
MAX_PACK_SIZE_LIMIT - 1 as its byte ceiling. pack_max_size returns the
user's size, or DEFAULT_PACK_MAX_SIZE when that variable is unset, so
compaction keeps targeting that value.
The environment-variable help and the packs internals page state the
limit, and repository tests cover rejected values (also on repository
creation), the size one below the limit, count-only mode, the unset
defaults, and both variables set together.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_019wSaUMZrtm6XSzwEcxavkv
Previously, an invalid atime/ctime/mtime PAX header made import-tar crash
with a ValueError from float().
Co-Authored-By: Claude Opus 5.5 <[email protected]>
export-tar wrote the atime/ctime/mtime PAX headers via str(ns / 1e9) and
import-tar parsed them via float(s) * 1e9. A float has ~240 ns resolution
at today's epoch, so the "ns" timestamps of --tar-format=PAX did not
roundtrip exactly (e.g. ...987654321 came back as ...987654400).
Write and parse them as exact decimal strings now.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
GNU tar (--format=posix --acls) writes named ACL entries without the
numeric uid/gid, e.g. "user:nobody:rw-". import-tar stored the text
unchanged, but borg's ACL format has the id as a 4th field, so borg
extract crashed in acl_use_local_uid_gid with an IndexError.
Named entries without an id now get it from a local user/group name
lookup (falling back to the name, like borg create does). Comma
separated entries (as star writes them) and comments are handled, too.
system.posix_acl_* xattrs (GNU tar --xattrs-include='*') are skipped,
like borg create does, because the ACLs are stored separately.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
GIMP source (editable text layers) and a preview of the chosen
rectangular design "C" from the ticket, plus a README with the
font, colours and printing notes.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
acquire() creates its lock object, waits for the race recheck delay and
lists the lock objects again to detect other clients racing for the lock.
With list-after-write consistency (local filesystems, sftp, rest, AWS S3,
MinIO), at least the client that created its lock object last sees the
other's lock in its second listing, so the delay is not needed for
correctness and the 0.01s default stays.
Stores that only list a new object after a lag (e.g. NFS clients caching
directory listings, some cloud storages via rclone) need a delay of at
least that lag. That lag is a property of the storage, not of the request
latency, so it is not measured, but set via BORG_LOCK_RECHECK_DELAY.
Invalid values abort with an error.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- zsh: pass the arguments of _borg_complete_repo_dirs on to _files, so the
completions are described as REPO (SRC_REPOSITORY) again, not as "file".
- fish: give the value being completed to __fish_complete_directories, so
directories are also completed for a value attached to the short option
(-rPATH).
- tests: use TCSH_REPO_URL_RULE instead of a hardcoded list of URL schemes.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
TW