Commit graph

36 commits

Author SHA1 Message Date
enkore
9a1c1e90c1 Merge pull request #2375 from enkore/buhdf
BORG_HOSTNAME_IS_UNIQUE=yes by default.
2017-04-05 14:09:58 +02:00
Marian Beermann
2ff75d58f2 remove Chunk() 2017-04-04 00:16:15 +02:00
Marian Beermann
bb6b4fde93 BORG_HOSTNAME_IS_UNIQUE=yes by default. 2017-04-01 21:28:41 +02:00
Thomas Waldmann
945880af47 implement async_response, add wait=True for add_chunk/chunk_decref
Before this changeset, async responses were:
- if not an error: ignored
- if an error: raised as response to the arbitrary/unrelated next command

Now, after sending async commands,  the async_response command must be used
to process outstanding responses / exceptions.

We are avoiding to pile up lots of stuff in cases of high latency, because we do NOT
first wait until ALL responses have arrived, but we just can begin to process responses.
Calls with wait=False will just return what we already have received.
Repeated calls with wait=True until None is returned will fetch all responses.

Async commands now actually could have non-exception non-None results, but
this is not used yet. None responses are still dropped.

The motivation for this is to have a clear separation between a request
blowing up because it (itself) failed and failures unrelated to that request /
to that line in the sourcecode.

also: fix processing for async repo obj deletes

exception_ignored is a special object used that is "not None" (as None is used to signal
"finished with processing async results") but also not a potential async response result value.

Also:

added wait=True to chunk_decref() and add_chunk()

this makes async processing explicit - the default is synchronous and you only
need to be careful and do extra steps for async processing if you explicitly
request async by calling with wait=False (usually for speed reasons).

to process async results, use async_response, see above.
2017-03-26 17:33:19 +02:00
Thomas Waldmann
f708183743 Revert "don't do "bigint" conversion for nanosecond mtime"
This reverts commit 8b2e7ec680.

We still need the bigint stuff for borg 1.0 compatibility.

# Conflicts:
#	src/borg/cache.py
2017-03-21 02:21:32 +01:00
Thomas Waldmann
b7a17a6db7 clamp (nano)second values to unproblematic range, fixes #2304
filesystem -> clamp -> archive (create)
2017-03-16 20:31:05 +01:00
Marian Beermann
d5515b6952 add msgid to progress output 2017-02-28 01:19:20 +01:00
enkore
7c9c4b61d7 Merge pull request #2157 from ThomasWaldmann/add-filesize
archived file items: add size metadata
2017-02-27 18:05:43 +01:00
Thomas Waldmann
0721cb1ede files cache: update inode number, fixes #2226 2017-02-27 15:30:55 +01:00
Thomas Waldmann
97bb1b7d9a deduplicate / refactor item (c)size code 2017-02-23 21:27:05 +01:00
Marian Beermann
cc26bdf810 info: add --json option 2017-02-23 11:55:14 +01:00
Marian Beermann
be18418b74 cache: no archive caches => work directly on master cache (no merges) 2016-12-08 22:45:59 +01:00
Marian Beermann
e169510116 cache: don't create Item in fetch_and_build_idx 2016-12-08 22:39:04 +01:00
Marian Beermann
8b2e7ec680 don't do "bigint" conversion for nanosecond mtime
2**63 nanoseconds are 292 years, so this change is good until 2262.
See also https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_2262

I expect that we will have plenty of time to revert this commit in time
for 2262.

timespec := time_t + long, so it's probably only 64 bits on some platforms
anyway.
2016-12-03 11:52:49 +01:00
TW
99c38e8fc0 Merge pull request #1892 from Abogical/sync-progress
Add cache.sync progress display
2016-11-29 21:52:07 +01:00
Abogical
f3ce6be30b Add cache.sync progress display 2016-11-29 22:15:27 +02:00
Marian Beermann
f62a22392e Implement security dir perks
Key type, location and manifest timestamp checks now survive cache
deletion. This also means that you can now delete your cache and avoid
previous warnings, since Borg can still tell it's safe.
2016-11-29 17:52:32 +01:00
Thomas Waldmann
c83a124e65 Merge branch '1.0-maint' (into master) 2016-11-28 02:23:32 +01:00
Abogical
be6341b956 Add begin_txn progress message 2016-11-26 19:38:00 +02:00
Abogical
f9b3d28c19 Add progress messages for cache.commit 2016-11-24 15:44:00 +02:00
Marian Beermann
bd4a0fe23b
Clarify cache/repository README file 2016-11-11 21:24:17 +01:00
Marian Beermann
676e69cac4 Parse & pass BORG_HOSTNAME_IS_UNIQUE env var to enable stale lock killing 2016-11-07 21:56:40 +01:00
Oleg Drokin
d490292be3 Detect and delete stale locks when it's safe
If BORG_UNIQUE_HOSTNAME shell variable is set, stale locks
in both cache and repository are deleted.

Stale lock is defined as a lock that's originating from the same
hostname as us, and correspond to a pid that no longer exists.

This fixes #562
2016-11-07 21:54:16 +01:00
Thomas Waldmann
8a15916284 Merge branch '1.0-maint' into merge-1.0-maint
# Conflicts:
#	docs/changes.rst
#	docs/usage/help.rst.inc
#	src/borg/cache.py
#	src/borg/remote.py
#	src/borg/testsuite/__init__.py
#	src/borg/testsuite/archiver.py
2016-10-31 18:24:13 +01:00
Thomas Waldmann
b0cc4fd74a Merge branch '1.0-maint' into merge-1.0-maint 2016-10-13 19:10:04 +02:00
Thomas Waldmann
2a864be84f Merge branch '1.0-maint' into merge-1.0-maint
# Conflicts:
#	setup.py
#	src/borg/archiver.py
#	src/borg/helpers.py
2016-09-27 21:31:41 +02:00
Thomas Waldmann
c355f3617a Merge branch '1.0-maint' 2016-08-20 00:04:55 +02:00
Thomas Waldmann
1f056d9e8a more safe interface for manifest.archives 2016-08-15 15:03:05 +02:00
Thomas Waldmann
c8922c8b3d use ArchiveItem 2016-08-15 01:11:33 +02:00
Thomas Waldmann
dbe862f3d8 Merge branch '1.0-maint' into merge-1.0-maint 2016-08-06 01:24:22 +02:00
Marian Beermann
2e3fc9ddfc SyncFile/SaveFile: default binary=False, just like open() 2016-07-26 22:49:25 +02:00
Marian Beermann
f4be2b3523 Use platform.SaveFile for repository, cache and key files
Fixes #1060
2016-07-09 21:10:46 +02:00
Thomas Waldmann
60da32123a refactor to use Item class, fixes #1071 2016-06-04 17:24:55 +02:00
Thomas Waldmann
d3deec9fc9 cleanup / reorder imports
import stdlib
from stdlib import x

import 3rd party

import borg stuff
2016-05-31 01:18:03 +02:00
Thomas Waldmann
49484d6d67 split helpers import into multiple ones 2016-05-31 00:33:13 +02:00
Thomas Waldmann
d1ea925a5b move borg package to src/ 2016-05-05 20:19:50 +02:00
Renamed from borg/cache.py (Browse further)