'authenticated-*' in the help text started an unterminated inline emphasis
in the generated rst usage docs and man page.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
SALVAGE_READ_ERROR now covers OSError, BackendConnectionError and
ReadRangeError. Other store backend errors, e.g. BackendMustBeOpen,
PermissionDenied or QuotaExceeded, propagate.
Without the object, the commands silently fell back to the built-in defaults, so
anybody with write access to the store could remove a repository default (e.g. an
"obfuscate" compression) unnoticed: the key protects the content of the object,
not its presence.
Now "borg repo-create" always writes config/defaults (empty if no default was
given), and a missing object is an error (Repository.DefaultsMissing, rc 34) like
one that fails the authentication: the commands using the defaults refuse to run.
"borg check" reports such an object, "borg check --repair" replaces it by empty
defaults, so the repository can be used again (with the built-in defaults).
with_repository reads the defaults only after the security checks (assert_secure).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
On Windows, repo-info output lines end with CRLF, so the assertions
expecting a trailing "\n" failed there.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The moto S3 test server install already pulls in the cryptography
manylinux wheel, so a plain "pip install --no-binary cryptography"
kept that wheel (statically linked OpenSSL) and the dynamic libcrypto
check failed.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
"borg repo-create --compression SPEC --chunker-params PARAMS" stores repository
defaults in the new config/defaults store object.
Compression: the commands that compress data (create, recreate, import-tar,
transfer, repo-compress, debug put-obj) use the repository default if no
compression was given via the command line, the environment or default.yaml;
without a repository default, they use lz4 as before.
Chunker params: "--chunker-params default" now means the repository default,
else the built-in CHUNKER_PARAMS: ChunkerParams("default") returns the
DEFAULT_CHUNKER_PARAMS marker and with_repository replaces it by the real
chunker params. borg create and borg import-tar use the marker as their parser
default, so they use the repository default if no chunker params were given.
borg recreate and borg transfer still only rechunk if --chunker-params is
given; with "default", they rechunk to the repository default.
config/defaults is stored in the key's store object envelope, so it is
authenticated: nobody without the key can change the defaults (e.g. remove an
"obfuscate" compression) without being noticed. config/config can not hold
them, as it is read before the key is known and is not authenticated.
config/defaults is read at most once per Repository object.
borg repo-info shows the defaults (also in --json).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
If an archive's metadata object is missing or invalid while its archives/ entry
still exists, Archives.list() yields a placeholder ArchiveInfo for it, but
opening it as an Archive raises Archive.DoesNotExist.
- ArchiveFormatter: use the placeholder values of the ArchiveInfo (and defaults)
if the archive can not be opened, so repo-list (and prune --list) go on.
- compact: skip such a soft-deleted archive with the "cannot be fully preserved"
warning, as documented in mark_soft_deleted_used(), instead of aborting.
- Archive.DoesNotExist: show a binary archive id as hex.
- check --format: keep using the fallback format for such archives.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Previously, "borg tag" always rewrote the archive metadata (and thus the
archive entry in the repository), even when just listing the tags or when
the requested change was a no-op (including a refused --set).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Previously, borg silently did not set the tags, so it looked like the
given tags were just ignored, and the return code was 0.
Now borg emits a warning naming the special tags that would be removed
and exits with rc 1 (warning).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Without NAME or archive filter options, "borg tag --set/--clear/--add/--remove"
silently changed the tags of all archives in the repository.
Now it aborts like "borg delete" does, unless archives are selected
explicitly, e.g. using -a 'sh:*' for all archives. Just showing the
tags of all archives still works without a selection.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
--tags used nargs="+" and thus swallowed the archive NAME if it was given
after it, e.g. "borg create --tags foo NAME PATH" failed.
Now --tag takes exactly one tag and can be given multiple times,
e.g. "borg create --tag foo --tag bar NAME PATH".
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Special tags like @PROT are kept. --clear can be combined with --add
to replace the normal tags, it is mutually exclusive with --set.
Previously, a bare "--set" cleared all tags, but that is not possible
anymore since --set now takes exactly one tag.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
They used nargs="*" and thus swallowed the archive NAME if it was given
after them, e.g. "borg tag --set keep aid:1ddaae55" failed.
Now each option takes exactly one tag and can be given multiple times,
e.g. "borg tag --add a --add b NAME".
Also add examples to the "borg tag" help.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
ArchiveInfo now carries the parsed ArchiveItem (as an attribute, not a
tuple field, so equality and hashing are unchanged). Archive accepts an
ArchiveInfo instead of a name or id; if it carries the metadata, the
archive is neither looked up again (1 list + 1 load) nor its metadata
loaded again (1 load). Without it, Archive looks the archive up by id as
before, e.g. for the archives of borg 1.x repositories.
ArchiveFormatter (repo-list, prune --list, ...), the per-archive loops
of info, list, tag, find, compact, recreate and analyze, and transfer
(for the source archives) pass the ArchiveInfo. repo-list with the
default format for 1000 archives with 50 ms emulated store latency takes
2.1 s instead of 238 s.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Listing the archives (repo-list and every archive match) read each archive
metadata object with its own repository.get, i.e. one store roundtrip per
archive. Archive metadata objects are small and usually each in a tiny pack
of its own, which is the case Store.gather is made for.
Repository.gather_many reads objects in batches (at most 1000 objects, or
until 16 MiB) with one store.gather per batch, keeping the order of the ids.
Objects in cached packs are sliced from the cache; unknown or still buffered
ids and batches hitting a missing or truncated pack fall back to get().
Archives._infos uses it: repo-list --short for 1000 archives with 50 ms
emulated store latency takes 2.0 s instead of 56 s.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
borgstore 0.7.0 adds Store.gather (many byte ranges with one call, one
roundtrip for REST) and fixes partial loads with a negative offset from
cached namespaces.
Store.defrag now requires all sources to be items of the given namespace,
which borg's defrag calls (hex pack names in "packs") already satisfy.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
In borg 2, the positional argument of borg key import is the key backup
path, not the repository. Like most of the docs, assume BORG_REPO is set.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Update the data structures and security docs: the lock objects are in the
store object envelope, locking needs the key (it is loaded before waiting for
the lock), unreadable lock objects are treated as exclusive locks, and what
the store still sees about the locks. Add a change log entry.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The lock objects below locks/ were stored as plaintext JSON, named by the
store hash of that JSON. Anybody with read access to the store could see
which hosts and processes use the repository, and as the content is easy to
guess, even the names alone could reveal it.
Now the lock content is stored in the repository key's store object
envelope, like the index/ and cache/ objects (encrypted and authenticated in
the encrypting modes, authenticated only in the authenticated-* modes), bound
to the repository id and the locks namespace. The name is the store hash of
the envelope. Lock objects are encrypted with encrypt_oneshot(), as a
LockRefresher thread may refresh the lock while the main thread encrypts.
storelocking stays independent of the crypto code: Lock gets seal/unseal
callables, Repository provides them. A Lock keeps using the key it was made
with, so our own lock objects stay readable for us.
A lock object that can not be unsealed (corrupt, tampered with, written by
an older borg 2 beta, ...) is treated as a foreign exclusive lock (fail
closed): it only expires if the store's clock confirms that it was not
written for longer than the stale timeout; without store-side mtimes, only
break-lock removes it. borg warns once about it.
break-lock now loads the key, too (it shows the lock owners with --info).
repo-delete (also with --force) needs the key, as locking needs it.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Repository.open() now loads the key (via key_factory, or a key_loader given
to Repository) before it acquires the lock, see Repository.acquire_lock().
This is a preparation for protecting the lock objects with the key (#8386),
and it also means that borg asks for the passphrase before waiting for the
lock (and no lock is held while the user types the passphrase).
- key_factory() returns the repository's key if it was loaded already, so
commands loading the key later (Manifest.load, check, ...) do not ask for
the passphrase again.
- the "other" repository (--other-repo) gets a key_loader using the
BORG_OTHER_* settings.
- repo-create locks after it created the key and before it writes the config
(until then, the store is not a repository for other clients).
- tests opening a Repository directly: set a key (or key_loader) before the
lock is taken, or open it with lock=False.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
AEADKeyBase.encrypt() uses the key's current session: the shared cipher and
its IV counter. Two threads calling it concurrently could encrypt two messages
with the same IV, which must never happen with AES-OCB or ChaCha20-Poly1305.
encrypt_oneshot() encrypts in a fresh session (new random session id, IV 0)
and neither uses nor changes the key's current session, so it is safe to call
from any thread. decrypt() reads the result like any other envelope.
This is for rare, small objects that may get written from a background
thread, like the repository lock objects (a LockRefresher thread refreshes
the lock). For the MAC-only (authenticated-*) keys, encrypt() has no state and
encrypt_oneshot() is the same.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Thomas Waldmann