FilesCacheMixin initialized _newest_cmtime to 0, but _write_files_cache()
only treats None as "no file was chunked this run" (falling back to a
max_time_ns cutoff that keeps all current entries).
When a backup reuses all files from the cache without chunking anything,
_newest_cmtime stayed at 0, so the race-protection cutoff became the unix
epoch and every current (age == 0) entry was discarded. The next backup
then had to re-read, chunk and hash all files again.
Initialize _newest_cmtime to None to match the documented contract in
_write_files_cache(), and make the comparisons in _build_files_cache()
None-safe.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In modern borg these were just a pass-through repository wrapper (there
is no RepositoryCache), with one variant doing inline decryption and
returning (csize, plaintext) tuples. Drop both and make all consumers
use the raw repository directly:
- fuse.py: ItemCache / FuseOperations / FuseBackend now take the raw
repository + repo_objs and decrypt via repo_objs.parse(ROBJ_DONTCARE),
matching hlfuse.py. The csize value was discarded at both call sites.
- mount_cmds.py: drop the cache_if_remote wrapper around FuseOperations.
- archive.py (rebuild_archives / check): drop the pass-through wrapper;
robust_iterator now uses self.repository directly.
- repository.py: delete the RepositoryNoCache class and cache_if_remote.
- repository_test.py: remove TestCacheIfRemote and orphaned imports.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Preloading was used only by extract and export-tar, and the modern
borgstore-based Repository.preload() was already a no-op. Remove the
preload calls from both commands and the now-dead supporting code:
- DownloadPipeline.preload_item_chunks / Archive.preload_item_chunks
- hlids_preloaded tracking
- is_preloaded parameter from DownloadPipeline.fetch_many and
Repository.get_many
- the no-op Repository.preload()
Also: remove preload support from borg.legacy
With no remaining callers, drop the legacy-side preload machinery:
LegacyRemoteRepository:
- preload_ids / chunkid_to_msgids state and the pop_preload_msgid helper
- is_preloaded parameter and handling in call_many() (get requests now
always go through the normal send path; MAX_INFLIGHT pipelining of
regular calls is unchanged)
- is_preloaded from get_many() and the preload() method
LegacyRepository:
- is_preloaded from get_many() and the no-op preload() stub
Both legacy repo classes now match the modern Repository interface.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On omniOS the test data now lives on ZFS (via TMPDIR=/var/tmp), and ZFS
rejects the year-2261 os.utime() with EOVERFLOW. Treat that as an
unsupported-filesystem condition and skip, rather than failing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
With TMPDIR=/var/tmp/borg-ci on omniOS, platformdirs' user_runtime_dir
fallback yields /var/tmp/borg-ci/runtime-<uid>/borg. Add it to the
accepted values, like the existing NetBSD CI entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On omniOS /tmp is swap-backed tmpfs (small, RAM-bound), so the pip/cargo
build temps and the pytest temp tree quickly exhaust it ("no space left on
device"). Point TMPDIR at disk-backed /var/tmp instead, mirroring what the
NetBSD job already does.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix PackWriter.flush() to use max_count == 1 (not len == 1) for the pack_id hack,
so final partial packs under max_count > 1 correctly use SHA256. Add covering test.
Move sha256 import to module level in repository_test.
PackWriter buffers (chunk_id, cdata) pairs and flushes as pack files via borgstore.
At N=1 pack_id == chunk_id; UNKNOWN_INT32 (0xFFFFFFFF) placeholders in the index
are replaced by real pack location fields after flush() via update_pack_info().
Update test_chunkindex_add to expect UNKNOWN_INT32 sentinels from add().
Enable the pdf output format on Read the Docs (the LaTeX build config
already existed in docs/conf.py) and add a "Downloads" line to the left
sidebar that links the offline formats (PDF, HTML zip, ePub). The links
are populated from the Read the Docs addons data, reusing the same
mechanism as the version selector, so they are version-correct and hidden
when unavailable. The line is left-aligned with the boxes above and the
table of contents below, with separators above and below it.
Also drop the stale 'resources' entry from latex_appendices (the page was
removed in #2088); it broke the now-enabled PDF build with a doctree
KeyError.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RepositoryServer stored self.permissions but never read it: open() builds
a LegacyRepository without any permissions, and legacy (borg 1.x / v1)
repositories have no permission system at all. Remove the dead __init__
parameter and stop forwarding args.permissions from do_serve.
The --permissions CLI option stays - it applies to the non-legacy
"borg serve --rest" path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remote access is restricted via an SSH forced command in authorized_keys
that hardcodes the restriction, e.g.
command="borg serve --rest --restrict-to-path=/srv/repos"
get_args() merges the forced command with the client's intended command
(SSH_ORIGINAL_COMMAND), copying only allowlisted options from the client.
For legacy serve the repo path travels inside the RPC protocol, so the
server enforces restrictions against it. But a rest:// repo passes the
repo as "--backend FILE:<path>" on the command line, and "backend" was in
neither allow- nor denylist, so under a forced command the client's
--backend was dropped: args.backend ended up None and do_serve_rest failed
with "requires --backend" - restrictions for rest were effectively broken.
Add "backend" to the allowlist so the client chooses which repo while the
forced command pins the restriction and the rest mode; do_serve_rest then
validates the client backend against restrict_to_paths/repositories via
check_rest_restrictions. The --rest mode flag stays out of the allowlist
so a forced legacy serve cannot be flipped to rest by the client.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PathNotAllowed lived in borg.legacy.remote, but borg serve --rest
(non-legacy) now also raises it via check_rest_restrictions, which made
non-legacy code import from the legacy package just for an exception.
It is a generic "repository path not allowed" error, so move it next to
the other cross-cutting Error subclasses in helpers/errors.py and
re-export it from helpers. Pure relocation; exit code stays 83.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A rest:// repository is now served by "borg serve --rest" spawned over ssh
rather than borgstore's "borgstore-server-rest".
CI: chmod o+x $HOME so the rest test's ssh user (sftpuser) can run borg
The rest repo test starts "borg serve --rest" over ssh as sftpuser, which runs
the borg under test from the tox venv under the runner $HOME.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make `borg serve` able to be the server-side component of a rest:// repository,
selected with a new --rest option. Plain `borg serve` (no option) keeps serving
legacy borg-1.x repos and stays command-line compatible with borg 1.x.
- serve_cmd.py: add --rest and --backend. With --rest, serve the given
--backend FILE:<path> on stdio via borgstore.server.rest.serve(); honor
--restrict-to-path/--restrict-to-repository (validated against the FILE path)
and --permissions (mapped via borg_permissions). Without --rest, run the legacy
RepositoryServer as before.
- repository.py: for rest:// locations, build the borgstore REST backend with a
command that runs `borg serve --rest --backend FILE:<path>` (locally via
sys.executable, or over ssh reusing borgstore's ssh_cmd / BORG_REMOTE_PATH),
instead of borgstore's hardcoded `borgstore-server-rest`. So a remote only needs
borg installed. Extracted the permissions string->dict mapping into the reusable
borg_permissions().
- tests: unit tests for the rest serve command builder. The existing
remote_archiver (rest:///) suite now runs against `borg serve --rest`.
- docs: changelog + quickstart updated.
Legacy serve and the legacy ssh client are unchanged (client still spawns plain
`borg serve`).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The existing --from-borg1 transfer tests only use a local v1 repo, so they
exercise LegacyRepository but never the ssh path (LegacyRemoteRepository +
borg serve / RepositoryServer) that this branch preserves.
Add test_transfer_from_borg1_ssh: extract the repo12.tar.gz borg 1.2 repo and
transfer from it via --other-repo=ssh://__testsuite__/<abspath> --from-borg1.
The __testsuite__ host makes the legacy client spawn a local "borg serve"
(no real ssh), driving the full client -> serve -> LegacyRepository chain, then
asserts all archives transferred. Local/non-win32 only, like the sibling tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
borg.remote no longer fit its name: it held the legacy-only borg serve server
plus generic repository cache wrappers used by current repos. Split by purpose
and remove the module:
- Move RepositoryServer into borg.legacy.remote (it only serves legacy v1 ssh
repositories). It reuses the exception classes (PathNotAllowed,
InvalidRPCMethod, UnexpectedRPCDataFormatFromClient) and BORG_VERSION / MSGID
constants already defined there; open() uses the module-level LegacyRepository.
serve_cmd.py now imports RepositoryServer from ..legacy.remote.
- Move RepositoryNoCache and cache_if_remote into borg.repository (they wrap a
Repository and are used by Archive.check and mount of current repos).
archive.py and mount_cmds.py import them from ..repository now.
- Move the cache_if_remote tests into repository_test.py; delete remote_test.py.
- Delete src/borg/remote.py; fix the stale BUFSIZE comment in constants.py.
Pure relocation, no behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
borg.legacy.remote.cache_if_remote (and the RepositoryCache / RepositoryNoCache
classes it returns) are dead code: nothing imports or calls them. Every
cache_if_remote consumer (archive check, mount, tests) uses the non-legacy
borg.remote version, and legacy repos never reach it (Archive.check rejects
legacy repos). The trio was copied wholesale during the borg.legacy split (#9556).
Delete RepositoryNoCache, RepositoryCache and cache_if_remote, plus the imports
that only they used (shutil, struct, tempfile, xxhash.xxh64, compress.Compressor,
helpers.safe_unlink). LegacyRemoteRepository and the rest of the module are
unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After removing the modern RemoteRepository, cache_if_remote always returned
RepositoryNoCache in production (the only RepositoryCache path was the removed
isinstance(RemoteRepository) check; force_cache=True was used only by a test).
Delete the vestigial RepositoryCache class and simplify cache_if_remote: drop
the pack/unpack/force_cache parameters and the LZ4/xxh64 cache-file machinery,
keep building the decrypted_cache -> transform closure, and always return
RepositoryNoCache. Remove the imports that only RepositoryCache used.
Replace the RepositoryCache tests with a focused test of the surviving
cache_if_remote path (plain passthrough and decrypted (csize, plaintext) tuples).
The legacy copy in borg/legacy/remote.py is intentionally left untouched (its
RepositoryCache is still used for LegacyRemoteRepository).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The modern client/server transport (RemoteRepository served by `borg serve`
over an msgpack RPC protocol) is now redundant for current (borg 2) repos:
its functionality is replaced by rest:// (which can tunnel over ssh to a
remote borgstore REST server).
Remove the modern RemoteRepository (both ssh:// and socket://) entirely.
Legacy v1 (borg 1.x) repos remain reachable over ssh:// via the separate
LegacyRemoteRepository client, and `borg serve` / RepositoryServer is kept,
trimmed to the legacy-only path, so a remote borg2 can still serve a v1 repo
for `borg transfer --from-borg1`.
Details:
- remote.py: delete RemoteRepository, SleepingBandwidthLimiter and the `api`
decorator; trim RepositoryServer to legacy-only (drop modern _rpc_methods,
socket serving, non-legacy open() branch); keep cache_if_remote /
RepositoryCache / RepositoryNoCache (used by all repos).
- get_repository(): non-legacy ssh:// now raises a clear "use rest://" error;
socket:// route and the global --socket option removed.
- parseformat: drop the socket:// scheme (now an invalid location).
- borg serve: keep the command (serves legacy v1 ssh only); update epilog.
- borg version: drop modern remote query; keep legacy ssh path.
- update isinstance/import sites (cache, archive, fuse/hlfuse, analyze/compact,
archiver __init__ -> LegacyRemoteRepository.RPCError).
- tests/docs updated; obsolete socket serve test removed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BLAKE3 is generally faster and provides a more modern construction for
keyed hashing (using its internal keyed mode instead of the construction
used for BLAKE2b).
Key types changed:
- authenticated-blake2 -> authenticated-blake3
- {keyfile,repokey}-blake2-aes-ocb -> {keyfile,repokey}-blake3-aes-ocb
- {keyfile,repokey}-blake2-chacha20-poly1305 -> {keyfile,repokey}-blake3-chacha20-poly1305
This also fixes the slightly unusual way how we used blake2b,
it is only supported for importing borg 1.x repos.
New repos either use HMAC-SHA256 or BLAKE3.
working with r1beta5 (from 2024) is just too much pain.
the system packages only have python 3.10.
if one install python 3.11 from HaikuPorts, it has no ssl support.
if one also installs openssl3 from HaikuPorts, creating a venv fails...
also: rust toolchains issues, thread-local storage ("TLS") issues, as seen in #9463.
thus: no haiku CI until they release next beta and cross-platform-actions have it.
