borgbackup/borg
1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2026-10-12 02:11:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

borg help environment: new help topic about env variables, fixes #10061

Browse Source 
Add an "environment" help topic (alias: "env") showing the documentation
about the environment variables borg supports, like borg help already does
for patterns, placeholders, compression and match-archives.

The help topic is now the single source of that documentation:
scripts/make.py build_usage generates docs/usage/general/environment.rst.inc
from it (and skips it on the misc. help docs page, where it would duplicate
the "General" chapter's section), and build_man already generates a
borg-environment(1) man page from any help topic.

The text was adapted a bit so that it renders well both via sphinx and via
nanorst on the terminal: the list-table, note directive and hyperlink
targets were replaced by constructs nanorst understands.

nanorst: end a code block when a following line is less indented than the
code block's contents (instead of the hardcoded "less than 4 spaces" rule),
so code blocks nested inside definition lists do not swallow the text
following them any more. Also, borg help <topic> now resolves :ref:
references like the command epilogs already did.

Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
Thomas Waldmann
2026-08-09 16:31:23 +02:00
co-authored by Claude Fable 5
parent a7a87592e3
commit 39d0e27f56
9 changed files with 530 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/man_intro.rst
+1 -1
View File
@@ -48,7 +48,7 @@ SEE ALSO
`borg-delete(1)`, `borg-prune(1)`, `borg-compact(1)`,
`borg-recreate(1)`
`borg-compression(1)`, `borg-patterns(1)`, `borg-placeholders(1)`
`borg-compression(1)`, `borg-patterns(1)`, `borg-placeholders(1)`, `borg-environment(1)`
* Main web site https://www.borgbackup.org/
* Releases https://github.com/borgbackup/borg/releases
docs/usage/general/environment.rst.inc
+51 -84
View File
@@ -1,6 +1,8 @@
Environment Variables
~~~~~~~~~~~~~~~~~~~~~
.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit!
Borg uses some environment variables for automation:
General:
@@ -61,7 +63,8 @@ General:
You can set the default value for the ``--lock-wait`` option with this, so
you do not need to give it as a command line option.
BORG_LOGGING_CONF
When set, use the given filename as INI_-style logging configuration.
When set, use the given filename as INI-style logging configuration (see
https://docs.python.org/3/library/logging.config.html#configuration-file-format).
A basic example conf can be found at ``docs/misc/logging.conf``.
BORG_RSH
When set, use this command instead of ``ssh``. This can be used to specify ssh options, such as
@@ -294,60 +297,31 @@ Some automatic "answerers" (if set, they automatically answer confirmation quest
answer or ask you interactively, depending on whether retries are allowed (they by default are
allowed). So please test your scripts interactively before making them a non-interactive script.
.. _XDG env var: https://specifications.freedesktop.org/basedir-spec/0.6/ar01s03.html
Directories and files:
Borg 2 uses the platformdirs library (https://pypi.org/project/platformdirs/) to determine
default directory locations. This means that default paths are **platform-specific**:
.. note::
- Linux: XDG Base Directory Specification paths are used (e.g. ``~/.config/borg``,
``~/.cache/borg``, ``~/.local/share/borg``). ``XDG_*`` environment variables are
honoured (see https://specifications.freedesktop.org/basedir-spec/0.6/ar01s03.html).
- macOS: native macOS directories are used by default (e.g. ``~/Library/Application Support/borg``,
``~/Library/Caches/borg``). ``XDG_*`` environment variables are honoured if set.
- Windows: Windows AppData directories are used (e.g. ``C:\Users\<user>\AppData\Roaming\borg``,
``C:\Users\<user>\AppData\Local\borg``). ``XDG_*`` environment variables are **not** honoured.
Borg 2 uses the `platformdirs <https://pypi.org/project/platformdirs/>`_ library to determine
default directory locations. This means that default paths are **platform-specific**:
On all platforms, you can override each directory individually using the specific environment
variables described below. You can also set ``BORG_BASE_DIR`` to force borg to use
``BORG_BASE_DIR/.config/borg``, ``BORG_BASE_DIR/.cache/borg``, etc., regardless of the platform.
- **Linux**: Uses XDG Base Directory Specification paths (e.g., ``~/.config/borg``,
``~/.cache/borg``, ``~/.local/share/borg``). `XDG env var`_ variables are honoured.
- **macOS**: Uses native macOS directories by default (e.g., ``~/Library/Application Support/borg``,
``~/Library/Caches/borg``). `XDG env var`_ variables are honoured if set.
- **Windows**: Uses Windows AppData directories (e.g., ``C:\Users\<user>\AppData\Roaming\borg``,
``C:\Users\<user>\AppData\Local\borg``). `XDG env var`_ variables are **not** honoured.
Default directory locations by platform (when no ``BORG_*`` environment variables are set)::
On all platforms, you can override each directory individually using the specific environment
variables described below. You can also set ``BORG_BASE_DIR`` to force borg to use
``BORG_BASE_DIR/.config/borg``, ``BORG_BASE_DIR/.cache/borg``, etc., regardless of the platform.
Default directory locations by platform (when no ``BORG_*`` environment variables are set):
.. list-table::
:header-rows: 1
:widths: 15 25 30 30
* - Directory
- Linux
- macOS
- Windows
* - Config
- ``~/.config/borg``
- ``~/Library/Application Support/borg``
- ``%APPDATA%\borg``
* - Cache
- ``~/.cache/borg``
- ``~/Library/Caches/borg``
- ``%LOCALAPPDATA%\borg\Cache``
* - Data
- ``~/.local/share/borg``
- ``~/Library/Application Support/borg``
- ``%LOCALAPPDATA%\borg``
* - Runtime
- ``/run/user/<uid>/borg``
- ``~/Library/Caches/TemporaryItems/borg``
- ``%TEMP%\borg``
* - Keys
- ``<config_dir>/keys``
- ``<config_dir>/keys``
- ``<config_dir>\keys``
* - Security
- ``<data_dir>/security``
- ``<data_dir>/security``
- ``<data_dir>\security``
Directory Linux macOS Windows
Config ~/.config/borg ~/Library/Application Support/borg %APPDATA%\borg
Cache ~/.cache/borg ~/Library/Caches/borg %LOCALAPPDATA%\borg\Cache
Data ~/.local/share/borg ~/Library/Application Support/borg %LOCALAPPDATA%\borg
Runtime /run/user/<uid>/borg ~/Library/Caches/TemporaryItems/borg %TEMP%\borg
Keys <config_dir>/keys <config_dir>/keys <config_dir>\keys
Security <data_dir>/security <data_dir>/security <data_dir>\security
BORG_BASE_DIR
Defaults to ``$HOME`` or ``~$USER`` or ``~`` (in that order).
@@ -357,26 +331,26 @@ Directories and files:
BORG_CACHE_DIR
Defaults to the platform-specific cache directory (see table above).
If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.cache/borg``.
On Linux and macOS, `XDG env var`_ ``XDG_CACHE_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
On Linux and macOS, ``XDG_CACHE_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
This directory contains the local cache and might need a lot
of space for dealing with big repositories. Make sure you're aware of the associated
security aspects of the cache location: :ref:`cache_security`
BORG_CONFIG_DIR
Defaults to the platform-specific config directory (see table above).
If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.config/borg``.
On Linux and macOS, `XDG env var`_ ``XDG_CONFIG_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
On Linux and macOS, ``XDG_CONFIG_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
This directory contains all borg configuration directories, see the FAQ
for a security advisory about the data in this directory: :ref:`home_config_borg`
BORG_DATA_DIR
Defaults to the platform-specific data directory (see table above).
If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.local/share/borg``.
On Linux and macOS, `XDG env var`_ ``XDG_DATA_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
On Linux and macOS, ``XDG_DATA_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
This directory contains all borg data directories, see the FAQ
for a security advisory about the data in this directory: :ref:`home_data_borg`
BORG_RUNTIME_DIR
Defaults to the platform-specific runtime directory (see table above).
If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.cache/borg``.
On Linux and macOS, `XDG env var`_ ``XDG_RUNTIME_DIR`` is also honoured if ``BORG_BASE_DIR`` is not set.
On Linux and macOS, ``XDG_RUNTIME_DIR`` is also honoured if ``BORG_BASE_DIR`` is not set.
This directory contains borg runtime files, like e.g. the socket file.
BORG_SECURITY_DIR
Defaults to ``$BORG_DATA_DIR/security``.
@@ -397,7 +371,8 @@ Directories and files:
- you need to point to the correct key file matching the repository the command will operate on.
TMPDIR
This is where temporary files are stored (might need a lot of temporary space for some
operations), see tempfile_ for details.
operations), see https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir
for details.
Building:
BORG_OPENSSL_NAME
@@ -411,38 +386,30 @@ Building:
Adds given prefix directory to the default locations. If a 'include/lz4.h' is found Borg
will be linked against the system liblz4 instead of a bundled implementation. (setup.py)
Automatic option environment variables:
Borg uses jsonargparse (https://jsonargparse.readthedocs.io/) with ``default_env=True``,
which means that every command-line option can also be set via an environment variable.
The environment variable name is derived from the program name (``borg``),
the subcommand (if any), and the option name, all converted to uppercase
with dashes replaced by underscores.
For **top-level options** (not specific to a subcommand), the pattern is::
BORG_<OPTION>
For example, ``--lock-wait`` can be set via ``BORG_LOCK_WAIT``.
For **subcommand options**, the subcommand and option are separated by a
double underscore::
BORG_<SUBCOMMAND>__<OPTION>
For example, ``borg create --comment`` can be set via ``BORG_CREATE__COMMENT``.
Please note:
- Be very careful when using the "yes" sayers, the warnings with prompt exist for your / your data's security/safety.
- Also be very careful when putting your passphrase into a script, make sure it has appropriate file permissions (e.g.
mode 600, root:root).
.. _INI: https://docs.python.org/3/library/logging.config.html#configuration-file-format
.. _tempfile: https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir
Automatically generated Environment Variables (jsonargparse)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Borg uses jsonargparse_ with ``default_env=True``, which means that every
command-line option can also be set via an environment variable.
The environment variable name is derived from the program name (``borg``),
the subcommand (if any), and the option name, all converted to uppercase
with dashes replaced by underscores.
For **top-level options** (not specific to a subcommand), the pattern is::
BORG_<OPTION>
For example, ``--lock-wait`` can be set via ``BORG_LOCK_WAIT``.
For **subcommand options**, the subcommand and option are separated by a
double underscore::
BORG_<SUBCOMMAND>__<OPTION>
For example, ``borg create --comment`` can be set via ``BORG_CREATE__COMMENT``.
.. _jsonargparse: https://jsonargparse.readthedocs.io/
scripts/make.py
+14
View File
@@ -70,6 +70,10 @@ class BuildUsage:
doc.write(".. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit!\n\n")
if command == "help":
for topic in Archiver.helptext:
if topic == "environment":
# not on the misc. help docs page - it is included by the
# "General" docs chapter instead, see below.
continue
params = {"topic": topic, "underline": "~" * len("borg help " + topic)}
doc.write(".. _borg_{topic}:\n\n".format(**params))
doc.write("borg help {topic}\n{underline}\n\n".format(**params))
@@ -93,6 +97,16 @@ class BuildUsage:
doc.write("\n\nDescription\n~~~~~~~~~~~\n")
doc.write(epilog)
if command == "help":
# the "environment" help topic is the single source for the
# "Environment Variables" section of the "General" docs chapter.
# note: the comment must come after the heading - if it came first, it would break
# the association of the ``.. _env_vars:`` label (in the including file) with the heading.
with open("docs/usage/general/environment.rst.inc", "w") as doc:
doc.write("Environment Variables\n~~~~~~~~~~~~~~~~~~~~~\n\n")
doc.write(".. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit!\n")
doc.write(Archiver.helptext["environment"])
if "create" in choices:
common_options = [group for group in choices["create"]._action_groups if group.title == "Common options"][0]
with open("docs/usage/common-options.rst.inc", "w") as doc:
src/borg/archiver/__init__.py
+1
View File
@@ -260,6 +260,7 @@ class Archiver(
match-archives selection patterns for matching archives
placeholders placeholders in repository URLs, archive names, etc.
compression options and specifications for data compression
environment environment variables used by borg (short: env)
"""
)
parser = ArgumentParser(
src/borg/archiver/_common.py
+5
View File
@@ -281,6 +281,11 @@ rst_plain_text_references = {
"key_files": "Internals -> Data structures and file formats -> Key files",
"borg_key_export": "borg key export --help",
"internals_hashindex": "Internals -> Data structures and file formats -> HashIndex",
"borg_serve": "borg serve --help",
"debugging": '"Debugging Facilities"',
"cache_security": 'FAQ -> "Do I need to take security precautions regarding the cache?"',
"home_config_borg": 'FAQ -> "How important is the borg config directory?"',
"home_data_borg": 'FAQ -> "How important is the borg data directory?"',
}
src/borg/archiver/help_cmd.py
+417 -2
View File
@@ -1,5 +1,6 @@
import textwrap
from ._common import rst_plain_text_references
from ..helpers.argparsing import ArgumentParser
from ..constants import * # NOQA
from ..helpers.nanorst import rst_to_terminal
@@ -569,15 +570,429 @@ class HelpMixIn:
borg create --compression obfuscate,2,zstd,6 ...
borg create --compression obfuscate,250,zstd,3 ...\n\n"""
)
helptext["environment"] = textwrap.dedent(
"""
Borg uses some environment variables for automation:
General:
BORG_REPO
When set, use the value to give the default repository location.
Use this so you do not need to type ``--repo /path/to/my/repo`` all the time.
BORG_OTHER_REPO
Similar to BORG_REPO, but gives the default for ``--other-repo``.
BORG_PASSPHRASE (and BORG_OTHER_PASSPHRASE)
When set, use the value to answer the passphrase question for encrypted repositories.
It is used when a passphrase is needed to access an encrypted repo as well as when a new
passphrase should be initially set when initializing an encrypted repo.
See also BORG_NEW_PASSPHRASE.
BORG_PASSCOMMAND (and BORG_OTHER_PASSCOMMAND)
When set, use the standard output of the command (trailing newlines are stripped) to answer the
passphrase question for encrypted repositories.
It is used when a passphrase is needed to access an encrypted repo as well as when a new
passphrase should be initially set when initializing an encrypted repo. Note that the command
is executed without a shell. So variables, like ``$HOME`` will work, but ``~`` won't.
If BORG_PASSPHRASE is also set, it takes precedence.
See also BORG_NEW_PASSPHRASE.
BORG_PASSPHRASE_FD (and BORG_OTHER_PASSPHRASE_FD)
When set, specifies a file descriptor to read a passphrase
from. Programs starting borg may choose to open an anonymous pipe
and use it to pass a passphrase. This is safer than passing via
BORG_PASSPHRASE, because on some systems (e.g. Linux) environment
can be examined by other processes.
If BORG_PASSPHRASE or BORG_PASSCOMMAND are also set, they take precedence.
BORG_NEW_PASSPHRASE
When set, use the value to answer the passphrase question when a **new** passphrase is asked for.
This variable is checked first. If it is not set, BORG_PASSPHRASE and BORG_PASSCOMMAND will also
be checked.
Main use case for this is to fully automate ``borg key change-passphrase``.
BORG_DISPLAY_PASSPHRASE
When set, use the value to answer the "display the passphrase for verification" question when defining a new passphrase for encrypted repositories.
BORG_DEBUG_PASSPHRASE
When set to YES, display debugging information that includes passphrases used and passphrase related env vars set.
BORG_EXIT_CODES
When set to "modern", the borg process will return more specific exit codes (rc).
When set to "legacy", the borg process will return rc 2 for all errors, 1 for all warnings, 0 for success.
Default is "modern".
BORG_HOST_ID
Borg usually computes a host id from the FQDN plus the results of ``uuid.getnode()`` (which usually returns
a unique id based on the MAC address of the network interface. Except if that MAC happens to be all-zero - in
that case it returns a random value, which is not what we want (because it kills automatic stale lock removal).
So, if you have an all-zero MAC address or other reasons to better control the host id externally, just set this
environment variable to a unique value. If all your FQDNs are unique, you can just use the FQDN. If not,
use FQDN@uniqueid.
BORG_HOSTNAME
When set, use this value as the hostname (instead of the auto-detected one), e.g. to run borg
on one host, but impersonate another host. This affects the hostname stored in newly created
archives as well as the ``{hostname}`` placeholder.
BORG_USERNAME
When set, use this value as the username (instead of the auto-detected one), e.g. to run borg
as one user, but impersonate another user. This affects the username stored in newly created
archives as well as the ``{user}`` placeholder.
BORG_LOCK_WAIT
You can set the default value for the ``--lock-wait`` option with this, so
you do not need to give it as a command line option.
BORG_LOGGING_CONF
When set, use the given filename as INI-style logging configuration (see
https://docs.python.org/3/library/logging.config.html#configuration-file-format).
A basic example conf can be found at ``docs/misc/logging.conf``.
BORG_RSH
When set, use this command instead of ``ssh``. This can be used to specify ssh options, such as
a custom identity file ``ssh -i /path/to/private/key``. See ``man ssh`` for other options.
This is the replacement for the removed ``--rsh CMD`` command line option.
borg also gives this to borgstore as ``BORGSTORE_RSH``, except if that is already set.
BORG_REMOTE_PATH
When set, use the given path as borg executable on the remote (defaults to "borg" if unset).
This is the replacement for the removed ``--remote-path PATH`` command line option.
BORG_UNITS
Determines how borg formats sizes in its human-readable output:
- ``si`` (default): decimal units, e.g. ``1.23 MB`` (1kB = 1000B)
- ``iec``: binary units, e.g. ``1.18 MiB`` (1KiB = 1024B)
- ``raw``: exact byte counts, e.g. ``1234567 B``
Use ``raw`` if you want to parse sizes with scripts (e.g. for monitoring),
so you do not have to deal with scaled values and different units.
Alternatively, use a command's ``--json`` output or, for the commands
supporting ``--format``, the size related format keys - sizes are given
as byte counts there anyway.
``BORG_UNITS=iec`` is the replacement for the removed ``BORG_IEC`` environment
variable (and for the ``--iec`` command line option removed before that).
BORG_PROGRESS_FPS
How often the ``--progress`` output is updated at most, in updates per
second (default: 5). Fractional values are allowed, e.g.
``BORG_PROGRESS_FPS=0.1`` limits it to one update every 10 seconds.
Lower values are useful when the output goes into a logfile rather than
to an interactive terminal.
BORG_DEBUG_PROFILE
When set to a filename, write an execution profile in Borg format into that file
(see :ref:`debugging`). If the filename ends with ``.pyprof``, a Python-compatible
profile is written instead.
This is the replacement for the removed ``--debug-profile`` command line option.
Note: every borg invocation writes the profile, so unset it again when you are done.
BORG_REPO_PERMISSIONS
Set repository permissions, see also: :ref:`borg_serve`
BORG_FILES_CACHE_SUFFIX
When set to a value at least one character long, instructs borg to use a specifically named
(based on the suffix) alternative files cache. This can be used to avoid loading and saving
cache entries for backup sources other than the current sources.
BORG_FILES_CACHE_TTL
When set to a numeric value, this determines the maximum "time to live" for the files cache
entries (default: 2). The files cache is used to determine quickly whether a file is unchanged.
BORG_ASSERT_ID
Comma-separated list of the places where borg shall verify that a chunk's content matches
its chunk id (``chunkid == id_hash(content)``) after decrypting and decompressing it.
Verifying costs a full hash pass over everything that is read at such a place.
Default (variable not set)::
BORG_ASSERT_ID=repair,transfer,rechunk
These are the place names that can be listed:
read
Every read that decompresses a chunk: ``borg extract``, ``borg mount``,
``borg export-tar``, ``borg diff``, ... This is by far the most data borg reads, so
this place is **not** in the default, see the explanation below.
repair
``borg check --repair``. It rebuilds archives from the item metadata stream it reads,
re-packing it into new chunks with freshly computed ids, and it recreates manifest and
archives directory entries from what it reads.
transfer
``borg transfer``, for everything it reads from the source repository. Transferring
re-anchors the content in another repository, which is a trust boundary.
rechunk
``borg recreate --chunker-params ...``, i.e. re-chunking reads. Re-chunking computes
new chunk ids from the content it reads, so a violation would not be noticeable any
more afterwards. (Re-chunking in ``borg transfer`` is covered by ``transfer``.)
An unknown place name is an error. An empty value (``BORG_ASSERT_ID=``) verifies at none of
these places, but still where borg always verifies (see below).
Why ``read`` is not in the default: for encrypted repositories (all the AEAD ciphersuites),
the chunk id is part of the AEAD additional authenticated data, so a successful decryption
already proves that a holder of the repository key deliberately stored exactly this
ciphertext for exactly this chunk id. A malicious or buggy **repository** can therefore not
swap, splice or substitute objects, whether the id is verified on read or not. What the id
check adds is the detection of chunks whose content does not match their id, which only a
malicious or compromised **borg client that had your borg key** could have written (e.g. to
poison future deduplication). If that is in your threat model - e.g. because some machines
writing into the repository are not fully trusted - add ``read`` to the list::
BORG_ASSERT_ID=read,repair,transfer,rechunk
Otherwise, running ``borg check --verify-data`` periodically is recommended: it is the
audit that re-certifies the invariant for all chunks in the background, instead of on
every read.
Independent of this variable, borg always verifies the chunk id:
- in ``borg check --verify-data``. That audit is what makes not verifying elsewhere
defensible, so it is not configurable (there is no ``verify_data`` place name).
- for ``authenticated`` and ``none`` mode repositories: there is no AEAD there, so the id
check *is* the read path's integrity check and switching it off would remove it
completely. Same for reading borg 1.x repositories (``borg transfer``).
BORG_BLAKE3_MT_THRESHOLD
When set to a numeric value, chunks of at least that many KiB get their id computed by
multi-threaded BLAKE3, smaller ones single-threaded (default: 256, i.e. 256KiB).
Only relevant for repositories using ``--id-hash blake3``.
Multi-threading only pays off for big enough chunks and the break-even point depends on
the machine's core count, so the default is deliberately conservative.
Run ``scripts/blake3-optimize-mt-threshold.py`` to measure the best value for your
machine - it sweeps input sizes, prints the recommended threshold and the command to
set it, and can optionally show a chart of the measurements in your browser
(``--html --open``).
0 means "always multi-threaded", a very large value effectively disables multi-threading.
BORG_ZSTD_MT_WORKERS
When set to a numeric value, use that many threads to zstd-compress a single chunk
(default: the cpu count). 0 or 1 means single-threaded compression.
Only relevant when compressing with ``zstd``.
Chunks below 768KiB are always compressed single-threaded: libzstd will not use a
compression job smaller than 512KiB, so a small chunk gets split very unevenly and
multi-threading it would be slower than not doing it at all.
Multi-threading trades a little compression ratio for speed (measured at ``zstd,3``:
+0.05% archive size for 1MiB chunks, +0.64% for 8MiB ones, more at higher levels), and
it uses more cpu time in total to reduce the wallclock time. Set it to 1 if you would
rather have the smaller archive, or if borg has to share the cpu with other work.
BORG_FASTCDC_KERNEL / BORG_BUZHASH64_KERNEL
Select the scan kernel the ``fastcdc`` / ``buzhash64`` chunker uses (default:
``scalar``, the plain sequential loop). Accepted values are ``avx512``, ``avx2``,
``neon``, ``blockwise`` and ``scalar``.
All kernels chunk identically - same cut points, same chunk ids - and differ only in
speed, so this is safe to change at any time, also for an existing repository.
Which kernel is fastest is not predictable from the instruction set: it depends on the
cpu and on the compiler that built borg, and the sequential loop wins on some machines.
Nothing is selected automatically, so measure on your own hardware with
``borg benchmark cpu --chunking`` before setting these.
``avx512`` and ``avx2`` exist only on x86-64, ``neon`` only on aarch64, and only if the
compiler that built borg supported them; ``scalar`` and ``blockwise`` are portable C
and always available.
Requesting a kernel that this build or this cpu cannot run is an error rather than a
silent fallback, so a benchmark can not accidentally measure a different kernel.
``borg create --debug`` logs the chunker and the kernel it was created with.
BORG_AES_CHUNKER_KERNEL
Select the scan kernel used by the AES based chunkers - one variable for all three of
``toeplitz-aes``, ``rabin-aes`` and ``goldilocks-aes`` (default: ``evp``, the portable
OpenSSL path). Accepted values are ``vaes``, ``aes-ni``, ``aes-arm64`` and ``evp``.
As with the chunker kernels above, all of them chunk identically and differ only in
speed, nothing is selected automatically, and a kernel that can not run here is an
error rather than a silent fallback.
``vaes`` and ``aes-ni`` exist only on x86-64, ``aes-arm64`` only on aarch64.
``vaes`` additionally needs a compiler that knows it (gcc >= 11 / clang >= 14), so a
cpu supporting VAES is not by itself enough to have that kernel available.
BORG_SHOW_SYSINFO
When set to no (default: yes), system information (like OS, Python version, ...) in
exceptions is not shown.
Please only use for good reasons as it makes issues harder to analyze.
BORG_MSGPACK_VERSION_CHECK
Controls whether Borg checks the ``msgpack`` version.
The default is ``yes`` (strict check). Set to ``no`` to disable the version check and
allow any installed ``msgpack`` version. Use this at your own risk; malfunctioning or
incompatible ``msgpack`` versions may cause subtle bugs or repository data corruption.
BORG_FUSE_IMPL
Choose the low-level FUSE implementation borg shall use for ``borg mount``.
This is a comma-separated list of implementation names, they are tried in the
given order, e.g.:
- ``mfusepy,pyfuse3,llfuse``: default, first try to load mfusepy, then pyfuse3, then llfuse.
- ``llfuse,pyfuse3``: first try to load llfuse, then try to load pyfuse3.
- ``mfusepy``: only try to load mfusepy
- ``pyfuse3``: only try to load pyfuse3
- ``llfuse``: only try to load llfuse
- ``none``: do not try to load an implementation
BORG_SELFTEST
This can be used to influence borg's built-in self-tests. The default is to execute the tests
at the beginning of each borg command invocation.
BORG_SELFTEST=disabled can be used to switch off the tests and rather save some time.
Disabling is not recommended for normal borg users, but large scale borg storage providers can
use this to optimize production servers after at least doing a one-time test borg (with
self-tests not disabled) when installing or upgrading machines/OS/Borg.
BORG_WORKAROUNDS
A list of comma-separated strings that trigger workarounds in borg,
e.g. to work around bugs in other software.
Currently known strings are:
basesyncfile
Use the more simple BaseSyncFile code to avoid issues with sync_file_range.
You might need this to run borg on WSL (Windows Subsystem for Linux) or
in systemd.nspawn containers on some architectures (e.g. ARM).
Using this does not affect data safety, but might result in a more bursty
write-to-disk behavior (not continuously streaming to disk).
retry_erofs
Retry opening a file without O_NOATIME if opening a file with O_NOATIME
caused EROFS. You will need this to make archives from volume shadow copies
in WSL1 (Windows Subsystem for Linux 1).
authenticated_no_key
Work around a lost passphrase or key for an ``authenticated`` mode repository
(these are only authenticated, but not encrypted).
If the key is missing in the repository config, add ``key = anything`` there.
This workaround is **only** for emergencies and **only** to extract data
from an affected repository (read-only access)::
BORG_WORKAROUNDS=authenticated_no_key borg extract --repo repo archive
After you have extracted all data you need, you MUST delete the repository::
BORG_WORKAROUNDS=authenticated_no_key borg delete repo
Now you can init a fresh repo. Make sure you do not use the workaround any more.
Output formatting:
BORG_CHECK_FORMAT
Giving the default value for ``borg check --format=X``.
BORG_LIST_FORMAT
Giving the default value for ``borg list --format=X``.
BORG_REPO_LIST_FORMAT
Giving the default value for ``borg repo-list --format=X``.
BORG_PRUNE_FORMAT
Giving the default value for ``borg prune --format=X``.
Some automatic "answerers" (if set, they automatically answer confirmation questions):
BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no (or =yes)
For "Warning: Attempting to access a previously unknown unencrypted repository"
BORG_RELOCATED_REPO_ACCESS_IS_OK=no (or =yes)
For "Warning: The repository at location ... was previously located at ..."
BORG_CHECK_I_KNOW_WHAT_I_AM_DOING=NO (or =YES)
For "This is a potentially dangerous function..." (check --repair)
BORG_DELETE_I_KNOW_WHAT_I_AM_DOING=NO (or =YES)
For "You requested to DELETE the repository completely *including* all archives it contains:"
Note: answers are case sensitive. setting an invalid answer value might either give the default
answer or ask you interactively, depending on whether retries are allowed (they by default are
allowed). So please test your scripts interactively before making them a non-interactive script.
Directories and files:
Borg 2 uses the platformdirs library (https://pypi.org/project/platformdirs/) to determine
default directory locations. This means that default paths are **platform-specific**:
- Linux: XDG Base Directory Specification paths are used (e.g. ``~/.config/borg``,
``~/.cache/borg``, ``~/.local/share/borg``). ``XDG_*`` environment variables are
honoured (see https://specifications.freedesktop.org/basedir-spec/0.6/ar01s03.html).
- macOS: native macOS directories are used by default (e.g. ``~/Library/Application Support/borg``,
``~/Library/Caches/borg``). ``XDG_*`` environment variables are honoured if set.
- Windows: Windows AppData directories are used (e.g. ``C:\\Users\\<user>\\AppData\\Roaming\\borg``,
``C:\\Users\\<user>\\AppData\\Local\\borg``). ``XDG_*`` environment variables are **not** honoured.
On all platforms, you can override each directory individually using the specific environment
variables described below. You can also set ``BORG_BASE_DIR`` to force borg to use
``BORG_BASE_DIR/.config/borg``, ``BORG_BASE_DIR/.cache/borg``, etc., regardless of the platform.
Default directory locations by platform (when no ``BORG_*`` environment variables are set)::
Directory Linux macOS Windows
Config ~/.config/borg ~/Library/Application Support/borg %APPDATA%\\borg
Cache ~/.cache/borg ~/Library/Caches/borg %LOCALAPPDATA%\\borg\\Cache
Data ~/.local/share/borg ~/Library/Application Support/borg %LOCALAPPDATA%\\borg
Runtime /run/user/<uid>/borg ~/Library/Caches/TemporaryItems/borg %TEMP%\\borg
Keys <config_dir>/keys <config_dir>/keys <config_dir>\\keys
Security <data_dir>/security <data_dir>/security <data_dir>\\security
BORG_BASE_DIR
Defaults to ``$HOME`` or ``~$USER`` or ``~`` (in that order).
If you want to move all borg-specific folders to a custom path at once, all you need to do is
to modify ``BORG_BASE_DIR``: the other paths for cache, config etc. will adapt accordingly
(assuming you didn't set them to a different custom value).
BORG_CACHE_DIR
Defaults to the platform-specific cache directory (see table above).
If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.cache/borg``.
On Linux and macOS, ``XDG_CACHE_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
This directory contains the local cache and might need a lot
of space for dealing with big repositories. Make sure you're aware of the associated
security aspects of the cache location: :ref:`cache_security`
BORG_CONFIG_DIR
Defaults to the platform-specific config directory (see table above).
If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.config/borg``.
On Linux and macOS, ``XDG_CONFIG_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
This directory contains all borg configuration directories, see the FAQ
for a security advisory about the data in this directory: :ref:`home_config_borg`
BORG_DATA_DIR
Defaults to the platform-specific data directory (see table above).
If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.local/share/borg``.
On Linux and macOS, ``XDG_DATA_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
This directory contains all borg data directories, see the FAQ
for a security advisory about the data in this directory: :ref:`home_data_borg`
BORG_RUNTIME_DIR
Defaults to the platform-specific runtime directory (see table above).
If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.cache/borg``.
On Linux and macOS, ``XDG_RUNTIME_DIR`` is also honoured if ``BORG_BASE_DIR`` is not set.
This directory contains borg runtime files, like e.g. the socket file.
BORG_SECURITY_DIR
Defaults to ``$BORG_DATA_DIR/security``.
This directory contains security relevant data.
BORG_KEYS_DIR
Defaults to ``$BORG_CONFIG_DIR/keys``.
This directory contains keys for encrypted repositories.
BORG_KEY_FILE
When set, use the given path as repository key file. Please note that this is only
for rather special applications that externally fully manage the key files:
- this setting only applies to the keyfile modes (not to the repokey modes).
- using a full, absolute path to the key file is recommended.
- all directories in the given path must exist.
- this setting forces borg to use the key file at the given location.
- the key file must either exist (for most commands) or will be created (``borg repo-create``).
- you need to give a different path for different repositories.
- you need to point to the correct key file matching the repository the command will operate on.
TMPDIR
This is where temporary files are stored (might need a lot of temporary space for some
operations), see https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir
for details.
Building:
BORG_OPENSSL_NAME
Defines the subdirectory name for OpenSSL (setup.py).
BORG_OPENSSL_PREFIX
Adds given OpenSSL header file directory to the default locations (setup.py).
BORG_LIBACL_PREFIX
Adds given prefix directory to the default locations. If an 'include/acl/libacl.h' is found
Borg will be linked against the system libacl instead of a bundled implementation. (setup.py)
BORG_LIBLZ4_PREFIX
Adds given prefix directory to the default locations. If a 'include/lz4.h' is found Borg
will be linked against the system liblz4 instead of a bundled implementation. (setup.py)
Automatic option environment variables:
Borg uses jsonargparse (https://jsonargparse.readthedocs.io/) with ``default_env=True``,
which means that every command-line option can also be set via an environment variable.
The environment variable name is derived from the program name (``borg``),
the subcommand (if any), and the option name, all converted to uppercase
with dashes replaced by underscores.
For **top-level options** (not specific to a subcommand), the pattern is::
BORG_<OPTION>
For example, ``--lock-wait`` can be set via ``BORG_LOCK_WAIT``.
For **subcommand options**, the subcommand and option are separated by a
double underscore::
BORG_<SUBCOMMAND>__<OPTION>
For example, ``borg create --comment`` can be set via ``BORG_CREATE__COMMENT``.
Please note:
- Be very careful when using the "yes" sayers, the warnings with prompt exist for your / your data's security/safety.
- Also be very careful when putting your passphrase into a script, make sure it has appropriate file permissions (e.g.
mode 600, root:root).\n\n"""
)
helptext_aliases = {"env": "environment"}
def do_help(self, parser, args):
commands = getattr(parser, "_subcommands_action", None)
commands = commands._name_parser_map if commands else {}
topic = self.helptext_aliases.get(args.topic, args.topic)
if not args.topic:
parser.print_help()
elif args.topic in self.helptext:
print(rst_to_terminal(self.helptext[args.topic]))
elif topic in self.helptext:
print(rst_to_terminal(self.helptext[topic], rst_plain_text_references))
elif args.topic in commands:
if args.epilog_only:
print(commands[args.topic].epilog)
src/borg/helpers/nanorst.py
+23 -12
View File
@@ -19,9 +19,9 @@ class TextPecker:
else:
return self.str[self.i + n - 1 : self.i - 1]
def peekline(self):
def peekline(self, offset=0):
out = ""
i = self.i
i = self.i + offset
while i < len(self.str) and self.str[i] != "\n":
out += self.str[i]
i += 1
@@ -60,6 +60,7 @@ def rst_to_text(text, state_hook=None, references=None):
references = references or {}
state = "text"
inline_mode = "replace"
code_indent = None # indentation of the current code block's contents
text = TextPecker(text)
out = io.StringIO()
@@ -111,6 +112,7 @@ def rst_to_text(text, state_hook=None, references=None):
text.read(2)
state_hook(state, "code-block", out)
state = "code-block"
code_indent = None
out.write(":\n")
continue
if text.peek(-2) in ("\n\n", "") and char == next == ".":
@@ -144,16 +146,25 @@ def rst_to_text(text, state_hook=None, references=None):
state = "text"
text.read(1)
continue
if state == "code-block" and char == next == "\n" and text.peek(5)[1:] != " ":
# Foo::
#
# *stuff* *code* *ignore .. all markup*
#
# More arcane stuff
#
# Regular text...
state_hook(state, "text", out)
state = "text"
if state == "code-block":
if code_indent is None and char == "\n":
line = text.peekline()
if line.strip():
code_indent = len(line) - len(line.lstrip())
if char == next == "\n":
# a blank line ends the code block if the following line is less
# indented than the code block's contents:
# Foo::
#
# *stuff* *code* *ignore .. all markup*
#
# More arcane stuff
#
# Regular text...
following = text.peekline(1)
if following.strip() and len(following) - len(following.lstrip()) < (code_indent or 1):
state_hook(state, "text", out)
state = "text"
out.write(char)
if state == "code-block":
src/borg/testsuite/archiver/help_cmd_test.py
+5 -1
View File
@@ -1,5 +1,6 @@
import pytest
from ...archiver._common import rst_plain_text_references
from ...constants import * # NOQA
from ...helpers.nanorst import RstToTextLazy, rst_to_terminal
from . import Archiver, cmd, exec_cmd
@@ -40,6 +41,8 @@ def test_usage(archiver):
def test_help(archiver):
assert "Borg" in cmd(archiver, "help")
assert "patterns" in cmd(archiver, "help", "patterns")
assert "BORG_PASSPHRASE" in cmd(archiver, "help", "environment")
assert "BORG_PASSPHRASE" in cmd(archiver, "help", "env") # alias
assert "creates a new, empty repository" in cmd(archiver, "help", "repo-create")
assert "positional arguments" not in cmd(archiver, "help", "repo-create", "--epilog-only")
assert "creates a new, empty repository" not in cmd(archiver, "help", "repo-create", "--usage-only")
@@ -54,7 +57,7 @@ def test_help_formatting(command, parser):
@pytest.mark.parametrize("topic", list(Archiver.helptext.keys()))
def test_help_formatting_helptexts(topic):
helptext = Archiver.helptext[topic]
assert str(rst_to_terminal(helptext))
assert str(rst_to_terminal(helptext, rst_plain_text_references))
def test_main_help_epilog(archiver):
@@ -64,6 +67,7 @@ def test_main_help_epilog(archiver):
assert "match-archives" in help_output
assert "placeholders" in help_output
assert "compression" in help_output
assert "environment" in help_output
@pytest.mark.parametrize("command", list(get_all_parsers().keys()))
src/borg/testsuite/helpers/nanorst_test.py
+13
View File
@@ -42,3 +42,16 @@ def test_code_block_end_of_string():
# check that there is no unexpected exception if a code block
# is not followed by blank lines, but the string just ends.
assert rst_to_text("This is a code block::\n\n borg --help") == "This is a code block:\n\n borg --help"
def test_code_block_indented_context():
# a code block inside a definition list: the following text is indented, but less than
# the code block's contents, so it ends the code block and gets markup-processed again.
rst = "term\n Foo::\n\n code\n\n Continued ``text``.\n"
assert rst_to_text(rst) == "term\n Foo:\n\n code\n\n Continued text.\n"
def test_code_block_less_indented_than_four():
# code blocks indented by less than 4 spaces are recognized, too
rst = "Foo::\n\n code\n\nBar ``baz``.\n"
assert rst_to_text(rst) == "Foo:\n\n code\n\nBar baz.\n"