9764 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Thomas Waldmann	caa9fdcd36	tests: add deduplication tests for buzhash64(e) ... this will detect if there is anything going wrong regarding deduplication with the encrypted buzhash mode.	2026-03-06 17:28:01 +01:00
Thomas Waldmann	fbfec8e800	Add chunker comparison script analyzing BuzHash and BuzHash64 quality.	2026-03-06 17:26:08 +01:00
Thomas Waldmann	b8a646fd11	buzhash64: optional rolling hash encryption with AES-128 ... Encrypting the rolling hash should make it better resist against attacks. use aes128-ecb (slow, not optimized)	2026-03-06 17:26:06 +01:00
TW	45c63ff084	Merge pull request #9441 from ThomasWaldmann/relax-recreate-cmd-test-master ... recreate_cmd_test: relax timestamp matching	2026-03-05 23:29:32 +01:00
Thomas Waldmann	14c215396f	recreate_cmd_test: relax timestamp matching	2026-03-05 23:28:31 +01:00
TW	0623ef0e13	Merge pull request #9438 from ThomasWaldmann/improve-coverage-master ... Improve coverage (master)	2026-03-05 21:49:24 +01:00
Thomas Waldmann	408b6e0513	coverage: misc. tweaks ... moved --junitxml parameter to the tox configuration. haiku: add coverage params to pytest invocation (tox not used there). vm_tests: add test_results and coverage uploads. hard-code coverage.xml as coverage filename	2026-03-05 13:42:16 +01:00
Thomas Waldmann	0877b7f48a	coverage: remove some omit entries ... "*/borg/fuse.py" - suspect, let's try what happens if we do not omit. "*/borg/support/*" - directory does not exist anymore. "*/borg/hash_sizes.py" - does not exist anymore.	2026-03-05 12:35:43 +01:00
Thomas Waldmann	9651f16e8d	coverage: remove some excludes ... freebsd coverage is rather a TODO than an exclude. the "unknown platform" exclude has no match in the code.	2026-03-05 12:31:48 +01:00
TW	aea3645c53	Merge pull request #9437 from ThomasWaldmann/macos-pr-coverage-master ... CI: move macOS (ARM) job from "merge" to "PR"	2026-03-04 23:15:20 +01:00
TW	404c06d7bc	Merge pull request #9436 from ThomasWaldmann/three-timestamps ... archive: use 3 timestamps, fixes #9400	2026-03-04 17:45:08 +01:00
Thomas Waldmann	c2b5e28746	CI: move macOS (ARM) job from "merge" to "PR" ... That way we should get macOS coverage visible in PRs. The macOS (intel) job will only run less frequently on merges (likely this runs on older hardware at github).	2026-03-04 17:39:27 +01:00
TW	42196289be	Merge pull request #9419 from mr-raj12/ci-windows-coverage ... fix: add --cov flags to Windows CI pytest command, fixes #9418	2026-03-04 17:34:06 +01:00
TW	1c05ab32db	Merge pull request #9430 from ThomasWaldmann/y2038 ... Y2038+	2026-03-04 14:16:49 +01:00
Thomas Waldmann	9b633cdb68	archive: use 3 timestamps, fixes #9400 ... time: the nominal ts, used for prune, list, sorting, ... start: operation start time (informative) end: operation end time (informative) Often, "time" is the same as "start" (normal borg create). But it can make sense to have a different "time": - borg create --timestamp=... - borg recreate --timestamp=... - borg recreate (will keep "time" as in original archive) - borg transfer (will keep "time" as in original archive) recreate and transfer produce new archives, "start" and "end" will reflect the recreate/transfer operation. Also: remove start_monotonic. start and end are just what the clock shows (including tz), so should be ok to compute duration from that, even for dst switching times.	2026-03-04 13:55:58 +01:00
TW	b1c32bd979	Merge pull request #9389 from mr-raj12/win32-syncfile-write-through ... platform: use FILE_FLAG_WRITE_THROUGH on Windows for SyncFile data durability, fixes #9388	2026-03-04 12:12:56 +01:00
Mrityunjay Raj	0f360f2f1b	platform: use FILE_FLAG_WRITE_THROUGH on Windows for SyncFile data durability, fixes #9388	2026-03-04 15:52:52 +05:30
TW	57af68e1e1	Merge pull request #9434 from ThomasWaldmann/update-changes-master ... update CHANGES	2026-03-04 09:19:45 +01:00
Thomas Waldmann	cb6a7b6fcf	update CHANGES	2026-03-03 14:03:54 +01:00
TW	ff78d00c97	Merge pull request #9431 from borgbackup/dependabot/pip/requirements.d/pip-dependencies-e903f47b53 ... build(deps-dev): bump setuptools from 80.10.1 to 80.10.2 in /requirements.d in the pip-dependencies group	2026-03-03 08:58:50 +01:00
TW	8ef1759343	Merge pull request #9432 from borgbackup/dependabot/github_actions/actions-38c31c5b5d ... build(deps): bump the actions group with 2 updates	2026-03-03 08:33:59 +01:00
dependabot[bot]	5a69ea2c8d	build(deps): bump the actions group with 2 updates ... Bumps the actions group with 2 updates: [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) and [actions/upload-artifact](https://github.com/actions/upload-artifact). Updates `actions/attest-build-provenance` from 3 to 4 - [Release notes](https://github.com/actions/attest-build-provenance/releases) - [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md) - [Commits](https://github.com/actions/attest-build-provenance/compare/v3...v4) Updates `actions/upload-artifact` from 6 to 7 - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v6...v7) --- updated-dependencies: - dependency-name: actions/attest-build-provenance dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions - dependency-name: actions/upload-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com>	2026-03-03 05:46:19 +00:00
dependabot[bot]	71d70fdf92	build(deps-dev): bump setuptools ... Bumps the pip-dependencies group in /requirements.d with 1 update: [setuptools](https://github.com/pypa/setuptools). Updates `setuptools` from 80.10.1 to 80.10.2 - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](https://github.com/pypa/setuptools/compare/v80.10.1...v80.10.2) --- updated-dependencies: - dependency-name: setuptools dependency-version: 80.10.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: pip-dependencies ... Signed-off-by: dependabot[bot] <support@github.com>	2026-03-03 05:43:20 +00:00
Thomas Waldmann	7b69bed8c4	add a borg create/extract timestamp test for y2261. ... in y2262, nanoseconds as signed int64 will overflow.	2026-03-02 23:34:27 +01:00
Thomas Waldmann	eca66a0e3e	y2038: SUPPORT_32BIT_PLATFORMS = False, fixes #9429 ... Not as bad as it sounds: 32bit platforms with 64bit time_t will still work. As of 2026, this is pretty much any platform that can run borg reasonably well.	2026-03-02 23:06:29 +01:00
Thomas Waldmann	f496497d92	time calculations: avoid floating point ... Do not use 1e9 if it is not mean to be a fp calculation anyway, but rather use 10**9 for pure integer calculations.	2026-03-02 23:03:44 +01:00
Thomas Waldmann	04940e3d44	helpers/time: avoid floating point in timestamp handling ... Add utcfromtimestampns() helper that converts nanosecond timestamps to datetime objects using integer arithmetic (timedelta) instead of floating point division. This avoids precision loss and potential overflow on 32bit platforms with old glibc. Use it in safe_timestamp() and timestamp() instead of datetime.fromtimestamp(). Needed to tweak the timestamps in repo12.tar.gz/test_meta/*.json +/- 1us.	2026-03-02 22:49:50 +01:00
TW	ef778332b5	Merge pull request #9428 from borgbackup/new-version-selector-location-master ... docs: move RTD version selector to sidebar top-left, fixes #8204	2026-03-02 21:06:22 +01:00
Thomas Waldmann	9a02947d40	docs: move RTD version selector to sidebar top-left ... That way, right below the docs version number that is currently being displayed, it is easier to find for users. Also: hide the default readthedocs-flyout (bottom right) Co-authored-by: Junie <junie@jetbrains.com> Co-authored-by: Junie <junie@jetbrains.com>	2026-03-02 20:46:26 +01:00
TW	f86c5900e8	Merge pull request #9427 from ThomasWaldmann/simplify-prune ... simplify prune	2026-03-02 19:39:05 +01:00
Thomas Waldmann	ec9e32a54f	prune: print hint to run compact to free space ... similar to what borg delete does. also: - remove "uncommitted" counter, we do not use commits anymore - always call manifest.write() if we deleted something	2026-03-02 18:31:40 +01:00
Thomas Waldmann	1759b6ccfb	prune: use same method to delete archives as delete subcommand, fixes #9424	2026-03-02 18:31:30 +01:00
TW	fb4843fdc6	Merge pull request #9417 from toroleapinc/fix/issue-9414-benchmark-cpu-test-speed ... Speed up benchmark cpu tests with _BORG_BENCHMARK_CPU_TEST env var	2026-03-02 18:06:42 +01:00
Thomas Waldmann	a4fbf7cf36	benchmark cpu test: minor consistency changes	2026-03-02 17:40:32 +01:00
TW	b38107c506	Merge pull request #9422 from ThomasWaldmann/deletion-fixes ... prune/delete fixes	2026-03-01 14:01:11 +01:00
Mrityunjay Raj	cc0ed81e5f	fix: remove extra coverage config, match Linux CI style	2026-03-01 16:07:28 +05:30
Thomas Waldmann	8a4f42d0d2	delete: compute archive_formatted before archive deletion, see #9416 ... here it was not a problem currently, because format_archive(archive_info) does not load the archive from the repo, but only uses the given archive_info contents.	2026-03-01 06:08:14 +01:00
Thomas Waldmann	3c204c31f3	prune: fix Archive.DoesNotExist when using --list, fixes #9416 ... format_item() can trigger lazy loading of archive metadata (e.g. hostname, username, size) from the repository. Previously it was called after archive.delete(), which caused Archive.DoesNotExist for pruned archives. Fix: call formatter.format_item() early, before any deletion takes place. Also added a test.	2026-03-01 05:57:24 +01:00
TW	f3ac2e0d54	Merge pull request #9392 from mr-raj12/fix-syncfile-seek-tell ... cache: add seek()/tell() to SyncFile, use SaveFile in _write_files_cache, fixes #9390	2026-02-28 11:43:01 +01:00
TW	7092d7715d	Merge pull request #9421 from ThomasWaldmann/fuse-fixes-master ... mount: fuse fs performance fix, docstring	2026-02-28 09:31:03 +01:00
Mrityunjay Raj	2357071111	cache: add seek()/tell()/read() to SyncFile, use SaveFile in _write_files_cache, fixes #9390	2026-02-28 13:34:28 +05:30
TW	614d8e33cd	Merge pull request #9420 from ThomasWaldmann/add-prune-stats-master ... added stats to prune command, fixes #9262	2026-02-28 08:58:28 +01:00
Mrityunjay Raj	d4fb70d3b3	fix: add explicit coverage XML report and fix codecov upload for Windows CI	2026-02-28 13:10:19 +05:30
Thomas Waldmann	2b75e98c92	fuse: add thread/async safety warning	2026-02-28 07:27:46 +01:00
Thomas Waldmann	9bf937a485	mount: performance improvement ... The efficiency difference between `meta.extend(bytes(N))` and `meta = meta + bytes(N)` stems from how Python manages memory and objects during these operations. - **`bytearray.extend()`**: This is an **in-place** operation. If the current memory block allocated for the `bytearray` has enough extra capacity (pre-allocated space), Python simply writes the new bytes into that space and updates the length. If it needs more space, it uses `realloc()`, which can often expand the existing memory block without moving the entire data set to a new location. - **Concatenation (`+`)**: This creates a **completely new** `bytearray` object. It allocates a new memory block large enough to hold the sum of both parts, copies the contents of `meta`, copies the contents of `bytes(N)`, and then reassigns the variable `meta` to this new object. - **`bytearray.extend()`**: In the best case (when capacity exists), it is **O(K)**, where K is the number of bytes being added. In the worst case (reallocation), it is **O(N + K)**, but Python uses an over-allocation strategy (growth factor) that amortizes this cost, making it significantly faster on average. - **Concatenation (`+`)**: It is always **O(N + K)** because it must copy the existing `N` bytes every single time. As the `bytearray` grows larger (e.g., millions of items in a backup), this leads to **O(N²)** total time complexity across multiple additions, as you are repeatedly copying an ever-growing buffer. - Concatenation briefly requires memory for **both** the old buffer and the new buffer simultaneously before the old one is garbage collected. This increases the peak memory usage of the process. - `extend()` is more memory-efficient as it minimizes the need for multiple large allocations and relies on the underlying memory manager's ability to resize buffers efficiently. In the context of `borg mount`, where `meta` can grow to be many megabytes or even gigabytes for very large repositories, using concatenation causes a noticeable slowdown as the number of archives or files increases, whereas `extend()` remains performant.	2026-02-28 07:23:23 +01:00
Thomas Waldmann	b2fe60c345	added stats to prune command, fixes #9262 ... -v / --info displays archive counts (total, kept, pruned).	2026-02-28 07:09:51 +01:00
TW	c18184550f	Merge pull request #9393 from mr-raj12/fix-debug-format-obj-robj-type ... debug format-obj: support all repo object types, fixes #9391	2026-02-28 05:44:52 +01:00
Mrityunjay Raj	be42fdf195	debug format-obj: support all repo object types, fixes #9391	2026-02-28 07:16:18 +05:30
Mrityunjay Raj	6439b73c00	fix: add --cov flags to Windows CI pytest command, fixes #9418	2026-02-28 06:42:18 +05:30
edvatar	bb2ca29cba	Speed up benchmark cpu tests with _BORG_BENCHMARK_CPU_TEST env var ... Add _BORG_BENCHMARK_CPU_TEST environment variable (following the existing pattern of _BORG_BENCHMARK_CRUD_TEST) that reduces: - timeit iterations from 100 to 1 (10 to 1 for compression) - KDF iterations from 5 to 1 - random data buffer from 10MB to 100KB Set this env var in test_benchmark_cpu and test_benchmark_cpu_json so they complete quickly in CI while still exercising the full code path. Fixes #9414 Signed-off-by: edvatar <88481784+toroleapinc@users.noreply.github.com>	2026-02-27 11:17:46 -05:00