manifest: only write it when its content changed

Without the timestamp, the manifest content only changes when a newer borg
version adds item keys (or when feature flags get set), so rewriting it
after every archive operation just stored the same content again under a
new ciphertext.

Manifest.write() now compares the packed manifest with what load() read
from the repository and skips the store write if they are equal. The
callers stay as they are: repo-create and the check --repair rebuild
still always write (nothing was loaded), and an archive operation writes
only if there is something new to store, e.g. after a borg upgrade or to
drop a legacy "timestamp" entry.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

docs/internals/data-structures.rst

@@ -238,8 +238,11 @@ index. Different from all other repository objects, the chunk id in its object
 header is not the hash of its content, but all-zero
 (``Manifest.MANIFEST_ID``).
 
-The manifest is rewritten each time an archive is created, deleted,
-or modified. It looks like this:
+The manifest is written when the repository is created and by ``borg check
+--repair`` when it rebuilds a lost or corrupted manifest. Commands that modify
+the repository also call ``Manifest.write()``, but that only stores a new
+manifest object if the content changed, e.g. because a newer borg version
+added item keys. It looks like this:
 
 .. code-block:: python
 
@@ -289,7 +292,7 @@ chunks that are not readable with the current feature set. The third
 category are operations that require accurate reference counts, for example
 archive deletion and check.
 
-As the manifest is always updated and always read, it is the ideal place to store
+As the manifest is always read (and rewritten whenever its content changes), it is the ideal place to store
 feature flags, comparable to the super-block of a file system. The only problem
 is to recover from a lost manifest, i.e. how is it possible to detect which feature
 flags are enabled, if there is no manifest to tell. This issue is left open at this time,

src/borg/archive.py

@@ -2769,7 +2769,6 @@ class ArchiveChecker:
                 )
             # drop the in-memory index so close() does not persist it over the index just written.
             self.repository.invalidate_chunk_index()
-            logger.info("Writing Manifest.")
             self.manifest.write()
 
 

src/borg/manifest.py

@@ -526,6 +526,7 @@ class Manifest:
         self.repo_objs = ro_cls(key)
         self.repository = repository
         self.item_keys = frozenset(item_keys) if item_keys is not None else ITEM_KEYS
+        self._loaded_data = None  # the packed manifest as loaded from the repository, see write()
 
     @property
     def id_str(self):
@@ -541,6 +542,7 @@ class Manifest:
             key = key_factory(repository, cdata, other=other, ro_cls=ro_cls)
         manifest = cls(key, repository, ro_cls=ro_cls)
         _, data = manifest.repo_objs.parse(cls.MANIFEST_ID, cdata, ro_type=ROBJ_MANIFEST)
+        manifest._loaded_data = data
         manifest_dict = key.unpack_manifest(data)
         m = ManifestItem(internal_dict=manifest_dict)
         manifest.id = manifest.repo_objs.id_hash(data)
@@ -582,6 +584,14 @@ class Manifest:
         return result
 
     def write(self):
+        """
+        Store the manifest in the repository, but only if its content differs from what was loaded.
+
+        The manifest only holds the item keys known to the repository and (optional) feature flags,
+        so it usually does not change at all: archive operations call this, but it only results in a
+        store write when e.g. a newer borg version added item keys, or when the loaded manifest still
+        had a legacy "timestamp" entry.
+        """
         from .item import ManifestItem
 
         # include checks for limits as enforced by limited unpacker (used by load())
@@ -591,5 +601,10 @@ class Manifest:
         manifest = ManifestItem(version=2, archives=manifest_archives, config=StableDict(self.config))
         data = self.key.pack_metadata(manifest.as_dict())
         self.id = self.repo_objs.id_hash(data)
+        if data == self._loaded_data:
+            logger.debug("manifest unchanged, not writing it.")
+            return
+        logger.debug("writing the manifest.")
         robj = self.repo_objs.format(self.MANIFEST_ID, {}, data, ro_type=ROBJ_MANIFEST)
         self.repository.put_manifest(robj)
+        self._loaded_data = data

src/borg/testsuite/archiver/create_cmd_test.py

@@ -2120,3 +2120,16 @@ def test_files_cache_rebuild_group_by_invalid(archivers, request):
     cmd(archiver, "repo-create", RK_ENCRYPTION)
     output = cmd(archiver, "create", "--group-by", "", "home", "input", exit_code=2)
     assert "At least one group-by key is required" in output
+
+
+def test_create_does_not_rewrite_unchanged_manifest(archivers, request):
+    archiver = request.getfixturevalue(archivers)
+    if archiver.get_kind() != "local":
+        pytest.skip("opens the repository directly")
+    create_regular_file(archiver.input_path, "file1", size=1024)
+    cmd(archiver, "repo-create", RK_ENCRYPTION)
+    with Repository(archiver.repository_path, exclusive=True) as repository:
+        manifest_before = repository.get_manifest()
+    cmd(archiver, "create", "test", "input")
+    with Repository(archiver.repository_path, exclusive=True) as repository:
+        assert repository.get_manifest() == manifest_before