The unix-socket transport (socket:// repositories, the --socket option and
"borg serve" over a socket) was never part of a stable borg 2 release and the
old RPC protocol it relied on is gone, so the remaining code was dead:
- legacy remote: drop the unreachable proto == "socket" connection branch and
the now-unused self.sock handling, "import socket" and get_socket_filename
import (LegacyRemoteRepository is only built for proto == "ssh")
- helpers: remove get_socket_filename() and its export
- parseformat: drop "socket" from local_path_re - socket:// is now treated like
any other unknown scheme (a local path) rather than being special-cased
- tests: drop test_socket and the self.sock check in the legacy reopen helper
- docs: drop the stale --socket entry from the manually maintained
common-options.rst.inc (the auto-generated usage/man docs are left untouched
here and will be rebuilt in a separate commit)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
For sftp/http(s)/s3/b2/rclone repositories, borg only detects the scheme now
and hands the raw URL to borgstore, which parses and validates it - removing
the duplicate parsing borg used to do. Precise field extraction (user/host/
port/path) is kept only for the protocols borg itself reads: file, rest and
legacy ssh.
- drop http_re, s3_re, rclone_re and the sftp arm of the old ssh_or_sftp_re
- add a single scheme-detection pass-through against BORGSTORE_SCHEMES; reject
unknown schemes (e.g. socket://) as before
- canonical_path() returns the processed URL for the delegated protocols, with
embedded credentials stripped so secrets never reach the security state file
or logs
- source local_path_re's scheme exclusions from BORGSTORE_SCHEMES
- create: use proto == "file" instead of "not location.host" for the local
repo-dir inode skip
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
If `borg compact` was interrupted after deleting repository objects but
before writing the updated chunk index, the still-existing cache/chunks.*
kept claiming the deleted objects were present. A later `borg create` would
trust that stale index, skip re-uploading the affected chunks and silently
create an archive with dangling object references that extracts to zero bytes.
Invalidate all cached chunk indexes via delete_chunkindex_cache() before the
first object is deleted, so an interruption is conservative: the next client
rebuilds the index from actual repository contents and re-uploads any deleted
data. The post-deletion save_chunk_index() still writes a fresh, valid index.
Add a regression test covering both compact paths (default and --stats) that
interrupts compaction right before save_chunk_index() and verifies no cached
chunk index survives and a later create+extract reproduces the original bytes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
FilesCacheMixin initialized _newest_cmtime to 0, but _write_files_cache()
only treats None as "no file was chunked this run" (falling back to a
max_time_ns cutoff that keeps all current entries).
When a backup reuses all files from the cache without chunking anything,
_newest_cmtime stayed at 0, so the race-protection cutoff became the unix
epoch and every current (age == 0) entry was discarded. The next backup
then had to re-read, chunk and hash all files again.
Initialize _newest_cmtime to None to match the documented contract in
_write_files_cache(), and make the comparisons in _build_files_cache()
None-safe.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enable the pdf output format on Read the Docs (the LaTeX build config
already existed in docs/conf.py) and add a "Downloads" line to the left
sidebar that links the offline formats (PDF, HTML zip, ePub). The links
are populated from the Read the Docs addons data, reusing the same
mechanism as the version selector, so they are version-correct and hidden
when unavailable. The line is left-aligned with the boxes above and the
table of contents below, with separators above and below it.
Also drop the stale 'resources' entry from latex_appendices (the page was
removed in #2088); it broke the now-enabled PDF build with a doctree
KeyError.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make `borg serve` able to be the server-side component of a rest:// repository,
selected with a new --rest option. Plain `borg serve` (no option) keeps serving
legacy borg-1.x repos and stays command-line compatible with borg 1.x.
- serve_cmd.py: add --rest and --backend. With --rest, serve the given
--backend FILE:<path> on stdio via borgstore.server.rest.serve(); honor
--restrict-to-path/--restrict-to-repository (validated against the FILE path)
and --permissions (mapped via borg_permissions). Without --rest, run the legacy
RepositoryServer as before.
- repository.py: for rest:// locations, build the borgstore REST backend with a
command that runs `borg serve --rest --backend FILE:<path>` (locally via
sys.executable, or over ssh reusing borgstore's ssh_cmd / BORG_REMOTE_PATH),
instead of borgstore's hardcoded `borgstore-server-rest`. So a remote only needs
borg installed. Extracted the permissions string->dict mapping into the reusable
borg_permissions().
- tests: unit tests for the rest serve command builder. The existing
remote_archiver (rest:///) suite now runs against `borg serve --rest`.
- docs: changelog + quickstart updated.
Legacy serve and the legacy ssh client are unchanged (client still spawns plain
`borg serve`).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The modern client/server transport (RemoteRepository served by `borg serve`
over an msgpack RPC protocol) is now redundant for current (borg 2) repos:
its functionality is replaced by rest:// (which can tunnel over ssh to a
remote borgstore REST server).
Remove the modern RemoteRepository (both ssh:// and socket://) entirely.
Legacy v1 (borg 1.x) repos remain reachable over ssh:// via the separate
LegacyRemoteRepository client, and `borg serve` / RepositoryServer is kept,
trimmed to the legacy-only path, so a remote borg2 can still serve a v1 repo
for `borg transfer --from-borg1`.
Details:
- remote.py: delete RemoteRepository, SleepingBandwidthLimiter and the `api`
decorator; trim RepositoryServer to legacy-only (drop modern _rpc_methods,
socket serving, non-legacy open() branch); keep cache_if_remote /
RepositoryCache / RepositoryNoCache (used by all repos).
- get_repository(): non-legacy ssh:// now raises a clear "use rest://" error;
socket:// route and the global --socket option removed.
- parseformat: drop the socket:// scheme (now an invalid location).
- borg serve: keep the command (serves legacy v1 ssh only); update epilog.
- borg version: drop modern remote query; keep legacy ssh path.
- update isinstance/import sites (cache, archive, fuse/hlfuse, analyze/compact,
archiver __init__ -> LegacyRemoteRepository.RPCError).
- tests/docs updated; obsolete socket serve test removed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BLAKE3 is generally faster and provides a more modern construction for
keyed hashing (using its internal keyed mode instead of the construction
used for BLAKE2b).
Key types changed:
- authenticated-blake2 -> authenticated-blake3
- {keyfile,repokey}-blake2-aes-ocb -> {keyfile,repokey}-blake3-aes-ocb
- {keyfile,repokey}-blake2-chacha20-poly1305 -> {keyfile,repokey}-blake3-chacha20-poly1305
This also fixes the slightly unusual way how we used blake2b,
it is only supported for importing borg 1.x repos.
New repos either use HMAC-SHA256 or BLAKE3.
- always have a starting line with FILE_ID repoid
- store repkeys content-addressed, name is sha256(content)
- search by repo id on load
- add keyfile_format / keyfile_parse / is_keyfile helpers
Use BORG_OBJ (8 bytes) as the blob magic and refer to it as OBJ_MAGIC
throughout so the literal and its length appear in one place. Update
the inline blob diagram to the pipe-separated notation Thomas suggested.
Rename PackIndex->ChunkIndex, fix pack path to use pack_id, drop
levels_config detail, fix "keyed MAC"->"ID hash" in Pack ID section,
document chunk_id duplication across unencrypted header and encrypted_meta.
remove forward-looking N>1 references, hardcoded offsets, and stale "currently";
use borgstore vocabulary, medium-sized index files, and simplified recovery prose
for packs, this needs to get implemented differently to perform well.
processing needs to be pack-after-pack and the index needs to be
updated correctly and carefully, e.g. considering interruptions
of repo-compress.
The move to platformdirs and its current usage _does_ honor XDG_*
variables on macOS if they are set. Tests were set up to assume this to
be untrue and the docs matched that.
This commit adds tests asserting that XDG_* variables are used when they
are present on macOS, with default locations still in ~/Library.
This adds a runtime warning when running under MSYS2/Git Bash without the necessary environment variables to disable automatic path translation. The documentation is also updated to explain this behavior and how to mitigate it.
This adds the `--paths-from-shell-command` option to the `create` command, enabling the use of shell-specific features like pipes and redirection when specifying input paths. Includes related test coverage.
