borgbackup/src/borg
Thomas Waldmann 945880af47 implement async_response, add wait=True for add_chunk/chunk_decref
Before this changeset, async responses were:
- if not an error: ignored
- if an error: raised as response to the arbitrary/unrelated next command

Now, after sending async commands,  the async_response command must be used
to process outstanding responses / exceptions.

We are avoiding to pile up lots of stuff in cases of high latency, because we do NOT
first wait until ALL responses have arrived, but we just can begin to process responses.
Calls with wait=False will just return what we already have received.
Repeated calls with wait=True until None is returned will fetch all responses.

Async commands now actually could have non-exception non-None results, but
this is not used yet. None responses are still dropped.

The motivation for this is to have a clear separation between a request
blowing up because it (itself) failed and failures unrelated to that request /
to that line in the sourcecode.

also: fix processing for async repo obj deletes

exception_ignored is a special object used that is "not None" (as None is used to signal
"finished with processing async results") but also not a potential async response result value.

Also:

added wait=True to chunk_decref() and add_chunk()

this makes async processing explicit - the default is synchronous and you only
need to be careful and do extra steps for async processing if you explicitly
request async by calling with wait=False (usually for speed reasons).

to process async results, use async_response, see above.
2017-03-26 17:33:19 +02:00
..
_crc32 fix crc32 compilation issues on wheezy 2017-01-15 15:09:40 +01:00
blake2 crypto blake2: don't release the GIL during key hashing 2016-11-11 03:07:05 +01:00
platform fixup: do not access os.POSIX_FADV_* early 2017-01-30 03:11:42 +01:00
testsuite implement async_response, add wait=True for add_chunk/chunk_decref 2017-03-26 17:33:19 +02:00
__init__.py Merge branch '1.0-maint' 2016-10-24 21:41:20 +02:00
__main__.py move borg package to src/ 2016-05-05 20:19:50 +02:00
_chunker.c Merge branch 'master' into move-to-src 2016-05-30 19:38:16 +02:00
_hashindex.c hashindex: rebuild hashtable if we have too little empty buckets, fixes #2246 2017-03-04 03:09:18 +01:00
archive.py implement async_response, add wait=True for add_chunk/chunk_decref 2017-03-26 17:33:19 +02:00
archiver.py use immutable data structure for the compression spec, fixes #2331 2017-03-24 03:09:55 +01:00
blake2-libselect.h crypto: link against system libb2, if possible 2016-11-11 03:08:11 +01:00
cache.py implement async_response, add wait=True for add_chunk/chunk_decref 2017-03-26 17:33:19 +02:00
chunker.pyx merge 1.0-maint into master 2017-01-14 03:07:11 +01:00
compress.pyx Fix compression exceptions (#2224) 2017-03-04 00:01:02 +01:00
constants.py archived file items: add size metadata 2017-02-23 21:24:37 +01:00
crc32.pyx two fast CRC implementations 2017-01-02 15:54:14 +01:00
crypto.pyx merge 1.0-maint into master 2017-01-14 03:07:11 +01:00
fuse.py rename Item.file_size -> get_size 2017-02-23 21:27:05 +01:00
hashindex.pyx fix ChunkIndex.__contains__ assertion for big-endian archs 2017-02-20 07:38:55 +01:00
helpers.py Merge pull request #2340 from jdchristensen/auto-threshold 2017-03-26 04:59:42 +02:00
item.pyx bigint conversion: add compatibility note 2017-03-21 02:27:20 +01:00
key.py use immutable data structure for the compression spec, fixes #2331 2017-03-24 03:09:55 +01:00
keymanager.py Redo key_creator, key_factory, centralise key knowledge (#2272) 2017-03-08 17:08:54 +01:00
locking.py Parse & pass BORG_HOSTNAME_IS_UNIQUE env var to enable stale lock killing 2016-11-07 21:56:40 +01:00
logger.py --log-json: time property on most progress/log objects, remove is_prompt 2017-03-09 21:36:37 +01:00
lrucache.py move borg package to src/ 2016-05-05 20:19:50 +02:00
nonces.py Rename BORG_NONCES_DIR to BORG_SECURITY_DIR 2016-11-29 17:43:06 +01:00
paperkey.html key export: center QR code on the page 2017-02-13 21:24:33 +01:00
remote.py implement async_response, add wait=True for add_chunk/chunk_decref 2017-03-26 17:33:19 +02:00
repository.py implement async_response, add wait=True for add_chunk/chunk_decref 2017-03-26 17:33:19 +02:00
selftest.py Add tertiary authentication for metadata (TAM) 2016-12-20 22:53:53 +01:00
shellpattern.py It's 2017 2016-12-31 16:53:30 +02:00
upgrader.py check: pick better insufficent archives matched warning from TW's merge 2017-01-12 17:04:51 +01:00
version.py version: Add version parsing and formating 2016-11-17 19:56:29 +01:00
xattr.py Merge branch '1.0-maint' into merge/1.0-maint 2017-01-12 15:01:41 +01:00