Commit graph

6651 commits

Author SHA1 Message Date
Thomas Waldmann
900a812e9c crypto: bump API_VERSION to 1.3_01 2022-03-26 17:05:57 +01:00
Thomas Waldmann
ce247526c2 docs: update borg init examples 2022-03-26 17:05:57 +01:00
Thomas Waldmann
298c5ee539 docs: security infos only applying to legacy encryption 2022-03-26 17:05:57 +01:00
Thomas Waldmann
b3383a4d53 update borg init docs 2022-03-26 17:05:57 +01:00
Thomas Waldmann
ccf0875053 EVP_DecryptFinal_ex: fix check for return value
seems like the current docs were updated.
was "positive return code".
now specifically mentions 0 and 1.
2022-03-26 17:05:57 +01:00
Thomas Waldmann
e1313ccc05 crypto.low_level: simplify return code checks (legacy) 2022-03-26 17:05:57 +01:00
Thomas Waldmann
948d67efd8 crypto.low_level: simplify return code checks (AEAD) 2022-03-26 17:05:57 +01:00
Thomas Waldmann
f4a6ad080b docs: add new AEAD modes to security docs 2022-03-26 17:05:57 +01:00
Thomas Waldmann
c50e1124b5 also authenticate the chunkid when using the AEAD ciphers (AES-OCB/CHACHA-POLY) 2022-03-26 17:05:57 +01:00
Thomas Waldmann
8bd9477b96 add aad parameter to borg.crypto.low_level api
added it for all classes there, so the caller just give it.

for the legacy AES-CTR based classes, the given aad is completely ignored.
this is to stay compatible with repo data of borg < 1.3.

for the new AEAD based classes:
encrypt: the aad is fed into the auth tag computation
decrypt: same. decrypt will fail on auth tag mismatch.
2022-03-26 17:05:57 +01:00
Thomas Waldmann
d3b78a6cf5 minor key.encrypt api change/cleanup
we already have .decrypt(id, data, ...).
i changed .encrypt(chunk) to .encrypt(id, data).

the old borg crypto won't really need or use the id,
but the new AEAD crypto will authenticate the id in future.
2022-03-26 17:05:57 +01:00
Thomas Waldmann
41b8a04d82 use faster hmac.digest api 2022-03-26 17:05:57 +01:00
Thomas Waldmann
74ecb63671 fix new crypto benchmarks for api change 2022-03-26 17:05:57 +01:00
Thomas Waldmann
0b5a21275f avoid losing the key (old crypto)
if we just have a pointer to a bytes object which might go out of scope, we can lose it.

also: cython can directly assign a bytes object into a same-size char array.
2022-03-26 17:05:57 +01:00
Thomas Waldmann
6d6d3caa24 avoid losing the key
if we just have a pointer to a bytes object which might go out of scope, we can lose it.

also: cython can directly assign a bytes object into a same-size char array.
2022-03-26 17:05:57 +01:00
Thomas Waldmann
41082f558f crypto: add some tests for new key types 2022-03-26 17:05:57 +01:00
Thomas Waldmann
6f2c587080 tests: consistently give iv_int to ciphersuite 2022-03-26 17:05:57 +01:00
Thomas Waldmann
6c7b499d3f set aead auth tag directly before EVP_DecryptFinal_ev
https://wiki.openssl.org/index.php/EVP_Authenticated_Encryption_and_Decryption
2022-03-26 17:05:57 +01:00
Thomas Waldmann
bb949b25ea EVP_DecryptFinal_ex: fix check for return value
seems like the current docs were updated.
was "positive return code".
now specifically mentions 0 and 1.
2022-03-26 17:05:57 +01:00
Thomas Waldmann
c010800f55 header_len=0 fits header=b'' default 2022-03-26 17:05:57 +01:00
Thomas Waldmann
5c66fa4caa crypto: layout updates, low-level does not deal with IV
encrypt used to "patch" the IV into the header,
decrypt used to fetch it from there.

encrypt now takes the header just "as is" and
also decrypt expects that the IV is already set.
2022-03-26 17:05:57 +01:00
Thomas Waldmann
0f6f278b0f crypto: AEAD key classes
also:

cleanup class structure: less inheritance, more mixins.

define type bytes using the 4:4 split

upper 4 bits are ciphersuite:
0 == legacy AES-CTR based stuff
1+ == new AEAD stuff

lower 4 bits are keytype:
legacy: a bit mixed up, as it was...
new stuff: 0=keyfile 1=repokey, ...
2022-03-26 17:05:35 +01:00
Thomas Waldmann
9633273622 crypto: simplify api for new crypto, AEAD only needs 1 key 2022-03-22 20:08:34 +01:00
Thomas Waldmann
3473b17a8d crypto: improve attr naming 2022-03-22 20:07:33 +01:00
Thomas Waldmann
57479fb989 crypto: put the IV into the header, at the end of it
one openssl call less due to simpler layout!

also change default for aad_offset to 0:
by default, we want to authenticate the complete header.
2022-03-22 20:07:33 +01:00
Thomas Waldmann
e647360a0e crypto: better raise NotImplementedError if we have no id_hash 2022-03-22 20:07:33 +01:00
Thomas Waldmann
aff626119a crypto: cleanup, remove references to AES-GCM
A lot of people have concerns about AES-GCM.

Considering we can use AES-OCB, I guess we will
not use AES-GCM anyway, thus no need to talk
about it.
2022-03-22 20:07:33 +01:00
Thomas Waldmann
d3f069cb3b crypto: fix/update borg version comments
new AEAD crypto can be used with borg >= 1.3.
old crypto is used by attic and borg < 1.3.
2022-03-22 20:07:33 +01:00
Andrey Bienkowski
78f041440c
Argon2 the first part: Implement key derivation (was: part 0) (#6468)
add a argon2 based kdf, using argon2-cffi
2022-03-21 21:51:54 +01:00
TW
dfd7ea8171
Merge pull request #6464 from ThomasWaldmann/exclude-checksums-c
fixup: fix some algorithms/* ignores
2022-03-18 01:18:39 +01:00
Thomas Waldmann
1f90d4a77a fixup: fix some algorithms/* ignores 2022-03-17 23:34:26 +01:00
TW
cb0e4fc2b2
Merge pull request #6460 from ThomasWaldmann/move-checksums
remove algorithms package, move checksums module to borg package
2022-03-17 12:55:43 +01:00
Thomas Waldmann
cc3b5c062c remove algorithms package, move checksums module to borg package 2022-03-17 00:24:49 +01:00
TW
c641e31e82
Merge pull request #6454 from bket/fix-benchmark-cpu
`borg benchmark cpu` fails with libressl
2022-03-14 20:13:02 +01:00
Björn Ketelaars
3d86df6649 borg benchmark cpu fails with libressl
`borg benchmark cpu` fails on OpenBSD with the error below, which is
caused by LibreSSL currently not supporting AES256_OCB and
CHACHA20_POLY1305.

Work around this by checking if borg is used with LibreSSL. Tested on
OpenBSD.

```
Chunkers =======================================================
buzhash,19,23,21,4095    1GB        14.294s
fixed,1048576            1GB        0.244s
Non-cryptographic checksums / hashes ===========================
crc32 (libdeflate, used) 1GB        0.724s
crc32 (zlib)             1GB        1.953s
xxh64                    1GB        0.361s
Cryptographic hashes / MACs ====================================
hmac-sha256              1GB        7.039s
blake2b-256              1GB        9.845s
Encryption =====================================================
aes-256-ctr-hmac-sha256  1GB        18.312s
aes-256-ctr-blake2b      1GB        21.213s
Local Exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/borg/archiver.py", line 5241, in main
    exit_code = archiver.run(args)
  File "/usr/local/lib/python3.9/site-packages/borg/archiver.py", line 5172, in run
    return set_ec(func(args))
  File "/usr/local/lib/python3.9/site-packages/borg/archiver.py", line 607, in do_benchmark_cpu
    print(f"{spec:<24} {size:<10} {timeit(func, number=100):.3f}s")
  File "/usr/local/lib/python3.9/timeit.py", line 233, in timeit
    return Timer(stmt, setup, timer, globals).timeit(number)
  File "/usr/local/lib/python3.9/timeit.py", line 177, in timeit
    timing = self.inner(it, self.timer)
  File "<timeit-src>", line 6, in inner
  File "/usr/local/lib/python3.9/site-packages/borg/archiver.py", line 602, in <lambda>
    ("aes-256-ocb", lambda: AES256_OCB(
  File "src/borg/crypto/low_level.pyx", line 636, in borg.crypto.low_level.AES256_OCB.__init__
  File "src/borg/crypto/low_level.pyx", line 633, in borg.crypto.low_level.AES256_OCB.requirements_check
ValueError: AES OCB is not implemented by LibreSSL (yet?).

Platform: OpenBSD gateway.lan 7.1 GENERIC.MP#418 amd64
Borg: 1.2.1.dev98+gebaf0c32  Python: CPython 3.9.10 msgpack: 1.0.3 fuse: None [pyfuse3,llfuse]
PID: 38614  CWD: /storage/8899fc1454db04de.a/home/code/git/ports/sysutils/borg
sys.argv: ['/usr/local/bin/borg', 'benchmark', 'cpu']
SSH_ORIGINAL_COMMAND: None
```
2022-03-14 18:56:01 +01:00
TW
ebaf0c32d3
Merge pull request #6388 from ThomasWaldmann/benchmark-cpu-master
borg benchmark cpu: display benchmarks of cpu bound stuff
2022-03-13 23:25:39 +01:00
Thomas Waldmann
79eee27936 borg benchmark cpu: display benchmarks of cpu bound stuff 2022-03-13 22:49:52 +01:00
TW
a6634c0779
Merge pull request #6387 from ThomasWaldmann/crc32
use crc32 from libdeflate, remove bundled crc32 code
2022-03-13 22:00:28 +01:00
Thomas Waldmann
98a675f62a use crc32 from libdeflate, remove bundled crc32 code
we use zlib.crc32 (macOS) or libdeflate_crc32 (Linux and all others) now.
2022-03-13 21:39:34 +01:00
TW
d9d1e44b67
Merge pull request #6415 from ThomasWaldmann/borg-key
borg key change-location, cleanups
2022-03-12 18:36:48 +01:00
Thomas Waldmann
0b3b78e139 dedup code: assert_type 2022-03-11 23:05:32 +01:00
Thomas Waldmann
b3e7e90c29 improve storage type handling, dedup code 2022-03-11 23:05:32 +01:00
Thomas Waldmann
a63614e35b move key type/storage constants to borg.constants 2022-03-11 23:05:32 +01:00
Thomas Waldmann
2e536bcbe2 borg key change-location 2022-03-11 23:05:32 +01:00
Thomas Waldmann
766d976f46 move passphrase related stuff to borg.helpers.passphrase 2022-03-11 21:05:42 +01:00
Thomas Waldmann
8011fade91 move key loading/saving code to FlexiKey 2022-03-11 21:05:42 +01:00
Thomas Waldmann
ad405892ca rename KeyfileKeyBase -> FlexiKeyBase
it is used as direct base class for both:
- RepoKey
- KeyfileKey
2022-03-11 21:05:42 +01:00
Thomas Waldmann
d42e6f2c41 key: uses key_cls.TYPES_ACCEPTABLE to dispatch
for now only a quite simple change, replacing the hardcoded PassphraseKey -> RepoKey dispatching.
2022-03-11 21:05:42 +01:00
TW
6cd0f6de49
Merge pull request #6431 from mikefarmer01/patch-1
use --numeric-ids in pull doc
2022-03-10 01:29:45 +01:00
Michael Bauer
638774761c
use --numeric-ids in pull doc
replace flag --numeric-owner with --numeric-ids
2022-03-09 21:43:22 +01:00