The multi-key feature (#9743) added the "borg key add", "borg key list"
and "borg key remove" subcommands but never wired up their docs:
- scripts/make.py: map key_add/key_list/key_remove to the "key" usage
group, so build_man can locate their examples (it previously aborted
with FileNotFoundError: docs/usage/key_add.rst).
- docs/usage/key.rst: include the three new generated snippets so they
show up on the HTML key page.
- regenerate the affected docs: new key_{add,list,remove}.rst.inc usage
snippets, new borg-key-{add,list,remove}.1 man pages, and borg-key.1
(SYNOPSIS + SEE ALSO now reference the new subcommands).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The authenticated and authenticated-blake3 modes do not encrypt data, but
they still have a real key (id/auth key material) stored as a key blob.
That blob can live as a keyfile or as a repokey just like the encrypted
modes, so make it configurable instead of always forcing repokey storage.
- AuthenticatedKeyBase: set LOCATION_CONFIGURABLE = True so --key-location
(at repo-create) and "borg key change-location" apply.
- key change-location: only copy sessionid/cipher when present (those are
AEAD-only; authenticated keys do not have them).
- repo-info: report the key storage location for authenticated keys too,
and handle the authenticated-blake3 variant (was only "authenticated").
- repo-create help: stop claiming authenticated* has no keyfile/repokey
storage; only "none" truly has no key.
- add change-location round-trip tests for authenticated mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Borg used to read the manifest's key-type byte and then look for the key in
exactly one place (keyfile or repokey) depending on the key class that byte
selected. As a result every crypto suite was duplicated into a keyfile class
and a repokey class that differed only in TYPE, NAME, ARG_NAME and STORAGE.
Now key *location* is independent of the type byte: detection tries keyfiles
first and repokeys afterwards until a passphrase unlocks a key. The type byte
still selects the crypto suite (id hash, MAC, cipher) to instantiate. Where a
key is stored (keyfile vs repokey) is therefore a per-key property
(self.storage), not a separate class, so a repository may even hold a mix of
keyfile- and repo-stored borg keys.
With storage decoupled from class identity, the keyfile/repokey class pairs
collapse into one class per crypto suite:
- modern AEAD: AESOCBKey, CHPOKey, Blake3AESOCBKey, Blake3CHPOKey
- legacy borg 1.x (read-only): AESCTRKey, Blake2AESCTRKey
There is now exactly one type byte per modern crypto suite (the old separate
repokey type bytes 0x11/0x21/0x31/0x41 were removed; borg2 is beta and only
needs to read repos it created). identify_key() matches on TYPES_ACCEPTABLE.
CLI: --encryption selects only the crypto suite (aes-ocb, chacha20-poly1305,
blake3-aes-ocb, blake3-chacha20-poly1305, authenticated*, none); the storage
location is chosen with the new --key-location=repokey|keyfile (default
repokey). The old combined modes (repokey-aes-ocb etc.) were removed.
borg key import also gained --key-location. borg key change-location no longer
swaps key classes or rewrites the manifest; it just re-saves the unlocked key
at the new location.
Keyfile removal (key remove, change-location) now overwrites the keyfile with
random data via secure_erase() before unlinking, consistent with save().
borg 1.x legacy read compatibility is preserved (the legacy class merge is a
behavior-preserving rename; the legacy type bytes incl. PASSPHRASE stay in
TYPES_ACCEPTABLE).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The unix-socket transport (socket:// repositories, the --socket option and
"borg serve" over a socket) was never part of a stable borg 2 release and the
old RPC protocol it relied on is gone, so the remaining code was dead:
- legacy remote: drop the unreachable proto == "socket" connection branch and
the now-unused self.sock handling, "import socket" and get_socket_filename
import (LegacyRemoteRepository is only built for proto == "ssh")
- helpers: remove get_socket_filename() and its export
- parseformat: drop "socket" from local_path_re - socket:// is now treated like
any other unknown scheme (a local path) rather than being special-cased
- tests: drop test_socket and the self.sock check in the legacy reopen helper
- docs: drop the stale --socket entry from the manually maintained
common-options.rst.inc (the auto-generated usage/man docs are left untouched
here and will be rebuilt in a separate commit)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BLAKE3 is generally faster and provides a more modern construction for
keyed hashing (using its internal keyed mode instead of the construction
used for BLAKE2b).
Key types changed:
- authenticated-blake2 -> authenticated-blake3
- {keyfile,repokey}-blake2-aes-ocb -> {keyfile,repokey}-blake3-aes-ocb
- {keyfile,repokey}-blake2-chacha20-poly1305 -> {keyfile,repokey}-blake3-chacha20-poly1305
This also fixes the slightly unusual way how we used blake2b,
it is only supported for importing borg 1.x repos.
New repos either use HMAC-SHA256 or BLAKE3.
for packs, this needs to get implemented differently to perform well.
processing needs to be pack-after-pack and the index needs to be
updated correctly and carefully, e.g. considering interruptions
of repo-compress.
The move to platformdirs and its current usage _does_ honor XDG_*
variables on macOS if they are set. Tests were set up to assume this to
be untrue and the docs matched that.
This commit adds tests asserting that XDG_* variables are used when they
are present on macOS, with default locations still in ~/Library.
This adds the `--paths-from-shell-command` option to the `create` command, enabling the use of shell-specific features like pipes and redirection when specifying input paths. Includes related test coverage.
use borg diff --sort-by=spec1,spec2,spec2 for enhanced sorting.
remove legacy --sort behaviour (sort by path), this was deprecated
since 1.4.2.
Co-authored-by: Daniel Rudolf <github.com@daniel-rudolf.de>
This is a port of #9005 to master branch.
- borg repo-create and borg transfer not only support --repo / --other-repo options,
but also already supported related BORG_REPO and BORG_OTHER_REPO env vars.
- similar to that, the passphrases now come from BORG_[OTHER_]PASSPHRASE, BORG_[OTHER_]PASSCOMMAND or BORG_[OTHER_]PASSPHRASE_FD.
- borg repo-create --repo B --other-repo A does not silently copy the passphrase of key A
to key B anymore, but either asks for the passphrase or reads it from env vars.
Some features like append-only repositories rely on a server-side component
that enforces them (because that shall only be controllable server-side,
not client-side).
So, that can only work, if such a server-side component exists, which is the
case for borg 1.x ssh: repositories (but not for borg 1.x non-ssh: repositories).
For borg2, we currently have:
- fs repos
- sftp: repos
- rclone: repos (enabling many different cloud providers)
- s3/b3: repos
- ssh: repos using client/server rpc code similar as in borg 1.x
So, only for the last method we have a borg server-side process that could enforce some features, but not for any of the other repo types.
For append-only the current idea is that this should not be done within borg,
but solved by a missing repo object delete permission enforced by the storage.
borg create could then use credentials that miss permission to delete,
while borg compact would use credentials that include permission to delete.
