Commit graph

594 commits

Author SHA1 Message Date
Thomas Waldmann
54c7da3c64
Linux: acl tests: move ACCESS_ACL and DEFAULT_ACL constants
They are only used at one place, move them there rather than having globals.
2024-03-17 18:51:07 +01:00
Thomas Waldmann
b0607909e1
FreeBSD: added tests, only get default ACL from dirs 2024-03-17 18:51:03 +01:00
Thomas Waldmann
d870c58e61
platform tests: misc. minor cleanups
- remove unused global / import
- use is_linux and is_darwin
- rename darwin acl test method
2024-03-16 15:58:50 +01:00
Thomas Waldmann
926b5a6b08
improve are_acls_working function
- ACLs are not working, if ENOTSUP ("Operation not supported") happens
- fix check for macOS
  On macOS borg uses "acl_extended", not "acl_access" and
  also the ACL text format is a bit different.
2024-03-16 15:56:58 +01:00
Thomas Waldmann
5b96d5acc3
create: add the slashdot hack, fixes #4685 2024-01-30 14:01:40 +01:00
Thomas Waldmann
742bd0c097
refactor: use less binascii
our own hex_to_bin / bin_to_hex is more comfortable to use.

also: optimize remaining binascii usage / imports.
2024-01-18 18:13:40 +01:00
kmille
aae959cd6d
borg with-lock: catch exception, print error msg, fixes #8022 2024-01-07 17:14:26 +01:00
Thomas Waldmann
242b8f9f51
refactor (re-)init of exit_code and warnings_list globals
stop directly accessing the variables from other modules.

prefix with underscore to indicate that these shall
only be used within this module and every other user
shall call the respective functions.
2023-12-27 13:49:08 +01:00
Thomas Waldmann
4bb42d2a00
do not return the rc from Archiver methods
this is not needed and getting rid of it makes
the code / behaviour simpler to understand:

if a fatal error is detected, we throw an exception.

if we encounter something warning worthy, we emit and collect the warning.

in a few cases, we directly call set_ec to set the
exit code as needed, e.g. if passing it through
from a subprocess.

also:
- get rid of Archiver.exit_code
- assert that return value of archiver methods is None
- fix a print_warning call to use the correct formatting method
2023-12-27 13:49:06 +01:00
Thomas Waldmann
482ac47ed8
new warnings infrastructure to support modern exit codes
- implement updating exit code based on severity, including modern codes
- extend print_warning with kwargs wc (warning code) and wt (warning type)
- update a global warnings_list with warning_info elements
- create a class hierarchy below BorgWarning class similar to Error class
- diff: change harmless warnings about speed to rc == 0
- delete --force --force: change harmless warnings to rc == 0

Also:

- have BackupRaceConditionError as a more precise subclass of BackupError
2023-12-27 13:47:52 +01:00
Thomas Waldmann
74954faec8
use print_warning also in borg delete ::archive --force --force 2023-12-27 13:46:04 +01:00
Thomas Waldmann
62ad0369ef
update "modern" error RCs (docs and code) 2023-12-27 13:45:48 +01:00
TW
60dcd53130
Merge pull request #8001 from ThomasWaldmann/move-conftest.py-1.4
move conftest.py to src/borg/testsuite, see #6386
2023-12-26 20:59:41 +01:00
Thomas Waldmann
2d97dcf250
get rid of dateutil 2023-12-26 19:12:00 +01:00
Thomas Waldmann
63848349c0
move conftest.py to src/borg/testsuite, see #6386
there, it will be packaged and installed, so
that when one runs the testsuite after installing
the pypi package, it will find the conftest.py.
2023-12-26 18:55:05 +01:00
Thomas Waldmann
8d2fd2a30e
ran pyupgrade --py39-plus ./**/*.py 2023-12-25 02:34:44 +01:00
Thomas Waldmann
ec17b00411
LockRoster.modify: no KeyError if element was already gone, fixes #7937
The intention of LockRoster.modify(key, REMOVE) is to remove self.id.

Using set.discard will just ignore it if self.id is not present there anymore.

Previously, using set.remove triggered a KeyError that has been frequently
seen in tracebacks of teardowns involving Repository.__del__ and Repository.__exit__.

I added a REMOVE2 op to serve one caller that needs to get the KeyError if
self.id was not present.

Thanks to @herrmanntom for the workaround!
2023-11-16 15:59:28 +01:00
TW
686714c366
Merge pull request #7896 from ThomasWaldmann/fix-shadow-index-1.2
fix shadow index update for double-put (1.2-maint)
2023-11-03 14:42:57 +01:00
Thomas Waldmann
48feb28e11
check --repair: test if shadow index is recreated
still failing here, because it is not.
2023-10-29 01:01:17 +02:00
Thomas Waldmann
8e6449f28e
test the shadowing-by-double-put behaviour, see #5661
the new test is currently failing due to a bug in the
repository code.
2023-10-28 17:13:13 +02:00
Thomas Waldmann
05cec8b3b0
docs: OS X -> macOS 2023-10-27 20:31:26 +02:00
Felix Schwarz
90de901ff4
replace "datetime.utcfromtimestamp" with custom helper to avoid deprecation warnings when using Python 3.12 2023-10-09 20:23:10 +02:00
Thomas Waldmann
5e0632a3d0
add tests for archive TAMs, upgrade 2023-08-30 02:52:32 +02:00
Thomas Waldmann
85b173d3d1
TAM msgs: be more specific: archives vs. manifest 2023-08-30 02:52:23 +02:00
Thomas Waldmann
1fd94bd38f
check: rebuild_manifest must verify archive TAM 2023-08-29 21:59:04 +02:00
Thomas Waldmann
453d35fa47
add a test for unreadable parent dir, see #7746 2023-07-29 22:48:13 +02:00
Felix Schwarz
98f547f278 add utcnow() helper function to avoid datetime.utcnow()
`datetime.utcnow()` is deprecated since Python 3.12. This causes additional lines of
output and thus breaks the test suite. I was not sure if all borg internals are
ready to deal with timezone-aware datetime instances so tried to keep the changes
minimal.
2023-07-06 21:44:09 +02:00
Thomas Waldmann
fbf287f825
RepositoryTestCaseBase: remove need for "with" after reopen()
setUp enters the context manager, so let's .reopen() leave it.
then create a fresh Repository instance in self.repository and
enter the context manager again. tearDown then will leave that.
2023-06-27 22:43:00 +02:00
Thomas Waldmann
0ac07b7a3f
RepositoryTestCaseBase: call __exit__
As we call __enter__ in setUp,
let's call __exit__ in tearDown.
2023-06-27 22:34:11 +02:00
Thomas Waldmann
83b2b5cb6c
fix RepositoryTestCaseBase.reopen method
"if self.repository" did not work as expected:
- Repository has a __len__ method, so the boolean evaluation was calling that.
- self.repository is also not set to None anywhere.
2023-06-27 22:33:57 +02:00
Thomas Waldmann
81588989fe
tests: avoid long ids in pytest output
sometimes the automatically computed IDs are just too long,
so rather give IDs directly or avoid them otherwise.
2023-06-27 21:50:51 +02:00
Thomas Waldmann
796d8e2d13
fix test_buzhash_chunksize_distribution
the last chunk can be smaller than 2**min_exp.
2023-06-06 10:46:53 +02:00
Thomas Waldmann
5cc4e3af6a
tests: only warn about "invalid" chunker params, fixes #7590
we previously allowed some weird chunker params, so we just warn
about them instead of rejecting them.

e.g. one could use super small chunk sizes. borg can do that,
but you'll end up with a huge amount of chunks and very large
hash tables (RAM usage) to manage them.

also, one can violate the min <= mask <= max chunker param condition
as in #7590 and it will somehow work, but will likely not dedup as good
as when not violating that.

borg2 **will** reject such strange chunker params, see #7586 for
some ideas how to "fix" your repos.
2023-05-19 18:05:26 +02:00
Thomas Waldmann
965e8a957f
tests: check buzhash chunksize distribution, see #7586 2023-05-19 18:05:20 +02:00
TW
dbc142a32a
Merge pull request #7549 from sashadev-sky/fix-progress-msg-width-1.2
ProgressIndicatorPercent: fix space computation for wide chars, fixes #3027
2023-05-08 19:46:17 +02:00
Thomas Waldmann
21a9458848 ProgressIndicatorPercent: fix space computation for wide chars, fixes #3027
needs to use swidth() in case there are wide chars (like CJK)
in the left part of the msg (e.g. an archive name).

(cherry picked from commit 3dd14f4855)
2023-05-08 12:44:05 -04:00
Thomas Waldmann
3e9668ecbc fix CompressionSpec validator and tests
(cherry picked from commit 0b40e038ad)
2023-05-07 21:03:56 -04:00
Thomas Waldmann
067e36bccc fix ChunkerParams validator and tests
(cherry picked from commit 6d38530ff1)
2023-05-07 20:59:17 -04:00
Michael Deyaso
ea08161fe8 Resolved mode bug and added sleep clause for darwin systems. Fixes #7470 2023-03-23 21:39:55 +03:00
Artem Sheremet
3b7c0605af Add --ignore-zeros flag to import-tar
Fixes #7432.

Backport of #7460.
2023-03-22 18:59:58 +01:00
TW
2ab678ca93
Merge pull request #7414 from Michael-Girma/backport/1.2-maint-7248
Show ctime and mtime on borg diff, fixes #7248 (Backport of #7335)
2023-03-20 14:14:30 +01:00
Michael Deyaso
6c042d7192 Show ctime and mtime on borg diff. Fixes #7248
---

- Added assert_line_exists helper in BaseTestCase
- JSON strings in diff output are now sorted alphabetically
- Modified diff test cases to confirm to new output format
- Added a test case to test ctime and mtime inclusion
- Mode, ctime & mtime are now only displayed on diff if --content-only flag is used
2023-03-07 10:54:29 +03:00
TW
ca1fea37a6
Merge pull request #7410 from snsmac/backport/autocompressortests
"auto" compressor tests: don't assume a specific size (backport to 1.2-maint)
2023-03-05 17:22:45 +01:00
Thomas Waldmann
8cf313900a "auto" compressor tests: do not assume zlib is better than lz4, fixes #7363
while that might be true for many cases, we can not assume it is
always true, as proven by the failing test on S390.
2023-03-05 11:05:47 +01:00
TW
cef5b446ac
Merge pull request #7411 from ThomasWaldmann/debug-id-hash-command-1.2
debug id-hash: implement file content id-hash computation, see #7406
2023-03-04 22:33:20 +01:00
TW
42fb72c022
Merge pull request #7409 from snsmac/backport/hashtablestress
Add hashtable stress test (backport to 1.2-maint)
2023-03-04 22:14:47 +01:00
Thomas Waldmann
eb8ccd22d4
debug id-hash: implement file content id-hash computation, see #7406 2023-03-04 21:47:14 +01:00
snsmac
c339d2e9e2 Update hashtable stress test to work with borg < 2 where NSIndex has 2-tuples as values 2023-03-04 21:41:31 +01:00
Thomas Waldmann
356ba3cc79
debug put-obj: id must be given on commandline, fixes #7290
this is an incompatible change:

before:
borg debug put-obj :: path1 path2 ...
(and borg computed all IDs automatically) (*)

after:
borg debug put-obj :: id path
(id must be given)

(*) the code just using sha256(data) was outdated and incorrect anyway.

also: fix error handling in get-obj.
2023-03-04 20:25:45 +01:00
Thomas Waldmann
8fe980c94e "auto" compressor tests: don't assume a specific size, fixes #7363
The tests assumed a specific compressed results size, which
is bad, because it might vary depending on the zlib implementation.

Now the "auto" compressor tests just check if it is the same size
as when unconditionally using the zlib compressor.
2023-03-04 20:19:08 +01:00