Buffer: delete support for multiple threads
Thread-local objects are a fairly complex footgun, so avoid them in
the first place. When a Compressor uses a Buffer (for example),
just create multiple Compressor instances for each thread, owned by
each thread. Minimize multiple ownership of objects across
threads if at all possible!
(cherry picked from commit f7ed5d7220)
Integer division is slow, and this improves the speed of all operations on the hashmap.
Benchmarked this patch on the rciorba/master-bench branch:
9e5d61e03c/results.html
(cherry picked from commit 12e0f55991)
a user was (rightfully) complaining that borg does a whole lot of
writing to .cache and was also quite inefficient because that took
rather long and occurred too frequently.
this was caused by big indexes/caches and the relatively high default
checkpoint frequency (every 300s) in 1.0.
we already have increased that value to 1800s in master (which soon will
be 1.1), so this changeset just does the same for 1.0.x.
Split up parsing and filtering for --keep-within
Fixes#2610
Parse --keep-within argument early, via new method within_range passed
to argparse type=, so that better error messages can be given.
Also swallows ValueError stacktrace per the comment in the old code that
including it wasn't desirable.
With the argument specified as unsigned char *, Cython emits
code in the Python wrapper to convert string-like objects to
unsigned char* (essentially PyBytes_AS_STRING).
Because the len(data) call is performed on a cdef'd string-ish type,
Cython emits a strlen() call, on the result of PyBytes_AS_STRING.
This is not correct, since embedded null bytes are entirely possible.
Incidentally, the code generated by Cython was also not correct,
since the Clang Static Analyzer found a path of execution where
passing arguments in a weird way from Python resulted in strlen(NULL).
Formulated like this, Cython emits essentially:
c_buzhash(
PyBytes_AS_STRING(data),
PyObject_Length(data),
...
)
which is correct.
(cherry picked from commit faf2d0b537)
if we always give stdout to extract_item(), it gets into the stdout-
processing branch which only emits data from items that have chunks
and does nothing for items which don't.
(cherry picked from commit 0b00c14c27)
attic.tar.gz contains a repo + corresponding keyfile - all the
upgrader module tests need.
.tar.gz because the .tar was 20x bigger.
(cherry picked from commit fed5873e29)
the cygwin issue that caused these tests to break was fixed in cygwin
at least since cygwin 2.8.0 (maybe even since 2.7.0).
also added a comment to our workaround (os_write wrapper, that is needed
still for people running older cygwin versions) that it can be removed
when cygwin 2.8.0 is considered ancient (and everybody has upgraded to
some fixed version).
(cherry picked from commit 4766d66875)
This should allow us to make sure older borg versions can be cleanly
prevented from doing operations that are no longer safe because of
repository format evolution. This allows more fine grained control than
just incrementing the manifest version. So for example a change that
still allows new archives to be created but would corrupt the repository
when an old version tries to delete an archive or check the repository
would add the new feature to the check and delete set but leave it out
of the write set.
This is somewhat inspired by ext{2,3,4} which uses sets for
compat (everything except fsck), ro-compat (may only be accessed
read-only by older versions) and features (refuse all access).
