Commit graph

224 commits

Author SHA1 Message Date
Antoine Beaupré
1c61f87da3 remove debugging code and fix all have_cython calls 2015-10-08 17:20:52 -04:00
Antoine Beaupré
80e53fb66d it's a function, call it as such 2015-10-08 17:16:11 -04:00
Antoine Beaupré
a1dad8c9da try to mock msgpack altogether to fix RTD again
it seems that msgpack is a hard depends in archive...
2015-10-08 17:06:48 -04:00
Antoine Beaupré
a869ab0702 try to fix RTD build *again* *again* 2015-10-08 17:03:35 -04:00
Antoine Beaupré
e8ae96b54e try to fix RTD build *again* 2015-10-08 17:01:42 -04:00
Antoine Beaupré
f2c56fb890 try to fix build on RTD *again* 2015-10-08 16:57:36 -04:00
Antoine Beaupré
423ff45d81 rename cython detection function
"have" has clearer semantics than "detect"
2015-10-08 15:34:44 -04:00
Antoine Beaupré
ff483fe485 fix logical inversion in the semantics of detect_cython() 2015-10-08 15:31:46 -04:00
Antoine Beaupré
f98998f042 fix logical inversion in the semantics of detect_cython() 2015-10-08 15:26:40 -04:00
Antoine Beaupré
6f9e04bc21 generalise the cython check hack
instead of applying this only to usage generation, use it as a generic
mechanism to disable loading of Cython code.

it may be incomplete: there may be other places where Cython code is
loaded that is not checked, but that is sufficient to build the usage
docs. the environment variable used is documented as such in the
docs/usage.rst.

we also move the check to a helper function and document it
better. this has the unfortunate side effect of moving includes
around, but I can't think of a better way.
2015-10-08 08:56:02 -04:00
Antoine Beaupré
9cbc868764 make tests and build work again in usage using environment
this is such a crude hack it is totally embarrassing....

the proper solution would probably be to move the `build_parser()`
function out of `Archiver` completely, but this is such an undertaking
that i doubt it is worth doing since we're looking at switching to
click anyways.

the main problem in moving build_parser() out is that it references
`self` all the time, so it *needs* an archiver context that it can
reuse. we could make the function static and pass self in there by
hand, but it seems like almost a worse hack... and besides, we would
need to load the archiver in order to do that, which would break usage
all over again...
2015-10-07 22:26:59 -04:00
Antoine Beaupré
13d3568548 move usage generation to setup.py
this is an unfortunate rewrite of the manpage creation code mentionned
in #208. ideally, this would be rewritten into a class that can
generate both man pages and .rst files.
2015-10-07 21:07:15 -04:00
Thomas Waldmann
a4967ec582 ssh_cmd: fix wrong caller, fixes #255 2015-10-07 03:32:55 +02:00
Thomas Waldmann
8ddc448f41 make sure to always give segment and offset in repo IntegrityError exception messages
this was only handled correctly at one place, by adding the segment number afterwards.
now the segment number is always included.
2015-10-06 20:35:22 +02:00
anarcat
7908c29180 Merge pull request #249 from anarcat/ssh-env
complete test coverage for SSH args parsing
2015-10-05 19:53:17 -04:00
Antoine Beaupré
a7b70d87cd complete test coverage for SSH args parsing 2015-10-05 19:22:33 -04:00
TW
745f9b89f8 Merge pull request #247 from anarcat/xdg
respect XDG_CACHE_HOME
2015-10-06 01:20:48 +02:00
TW
974dd58c23 Merge pull request #248 from anarcat/ssh-env
add support for arbitrary SSH commands
2015-10-06 01:20:23 +02:00
Antoine Beaupré
8f0de2cab7 fix tests on travis, which seem to set BORG_CACHE_DIR 2015-10-05 19:05:27 -04:00
Antoine Beaupré
a0ef4e25dd add support for arbitrary SSH commands (attic#99)
while SSH options can be specified through `~/.ssh/config`, some users
may want to use a completely different SSH command for their backups,
without overriding their $PATH variable. it may also be easier to do
ad-hoc configuration and tests that way.

plus, the POLA tells us that users expects something like this to be
supported by commands that talk to ssh. it is supported by rsync, git
and so on.
2015-10-05 18:54:00 -04:00
Antoine Beaupré
43a65933f7 move ssh generation code to a stub, add unit test 2015-10-05 18:51:20 -04:00
Antoine Beaupré
de2a811606 move RemoteRepository defaults to the class
the reasoning behind this is that we may need to test a
RemoteRepository setup outside of the main archiver routines, which
the current default location makes impossible

by moving the umask and remote_path remotes into the RemoteRepository
the (reasonable) defaults are available regardless of the (currently
obscure) initialisation routine, and make unit tests easier to develop
and support
2015-10-05 18:45:57 -04:00
Antoine Beaupré
427ddd64a6 respect XDG_CACHE_HOME
fixes attic#181
2015-10-05 17:50:46 -04:00
Thomas Waldmann
c50f32426b do not crash on empty lock.roster, fixes #232 2015-10-05 23:23:59 +02:00
Thomas Waldmann
6f637bed2f LoggedIO: deduplicated code, improved checks and error handling in read()
Code shared by read() and iter_objects() was moved into _read().

Compared to read()'s previous state, this improved:
- fixed size check to avoid read with negative size
- exception handler for struct unpack
- checking for short read
- more precise exception messages
2015-10-05 02:27:24 +02:00
Thomas Waldmann
51dc66d05f implement borg delete --cache-only repo, attic #123
it deletes just the local cache for the given repository, not the repo itself.
2015-10-03 19:29:45 +02:00
TW
adac324b6c Merge pull request #240 from ThomasWaldmann/cache-config-check
fix multiple issues with the cache config version check, fixes #234
2015-10-03 19:19:22 +02:00
TW
23bfe4d1bc Merge pull request #238 from ThomasWaldmann/index-archive-optional
temporary hack to avoid using lots of disk space for chunks.archive.d
2015-10-03 19:12:52 +02:00
Antoine Beaupré
5409cbaa67 also copy files cache verbatim
it seems the file cache does *not* have the ATTIC magic header (nor
does it have one in borg), so we don't need to edit the file - we just
copy it like a regular file.

while i'm here, simplify the cache conversion loop: it's no use
splitting the copy and the edition since the latter is so fast, just
do everything in one loop, which makes it much easier to read.
2015-10-03 12:56:03 -04:00
Antoine Beaupré
fded2219a8 mention borg delete borg
this makes it clear how to start from scratch, in case the chunk cache
was failed to be copied and so on.
2015-10-03 12:46:23 -04:00
Antoine Beaupré
c91c5d0029 rename convert command to upgrade
convert is too generic for the Attic conversion: we may have other
converters, from other, more foreign systems that will require
different options and different upgrade mechanisms that convert could
never cover appropriately. we are more likely to use an approach
similar to "git fast-import" instead here, and have the conversion
tools be external tool that feed standard data into borg during
conversion.

upgrade seems like a more natural fit: Attic could be considered like
a pre-historic version of Borg that requires invasive changes for borg
to be able to use the repository. we may require such changes in the
future of borg as well: if we make backwards-incompatible changes to
the repository layout or data format, it is possible that we require
such changes to be performed on the repository before it is usable
again. instead of scattering those conversions all over the code, we
should simply have assertions that check the layout is correct and
point the user to upgrade if it is not.

upgrade should eventually automatically detect the repository format
or version and perform appropriate conversions. Attic is only the
first one. we still need to implement an adequate API for
auto-detection and upgrade, only the seeds of that are present for now.

of course, changes to the upgrade command should be thoroughly
documented in the release notes and an eventual upgrade manual.
2015-10-03 12:36:52 -04:00
Antoine Beaupré
48b7c8cea3 avoid checking for non-existent files
if there's no attic cache, it's no use checking for individual files

this also makes the code a little clearer

also added comments
2015-10-03 11:52:12 -04:00
Antoine Beaupré
690541264e style fixes (pep8, append, file builtin) 2015-10-03 11:49:01 -04:00
Antoine Beaupré
3773681f00 rewire cache copy mechanisms
we separate the conversion and the copy in order to be able to copy
arbitrary files from attic without converting them. this allows us to
copy the config file cleanly without attempting to rewrite its magic
number
2015-10-03 11:07:38 -04:00
Antoine Beaupré
2c66e7c233 make percentage a real percentage 2015-10-03 10:49:29 -04:00
Thomas Waldmann
893242ead4 fix multiple issues with the cache config version check, fixes #234
- issue #234: handle exception when config file is empty is really not a borg cache config
- there was a unused %s in the Exception string
- error msg was wrong when version check failed - this IS a borg cache, but not of expected version
2015-10-02 18:11:10 +02:00
Thomas Waldmann
8978515991 temporary hack to avoid using lots of disk space for chunks.archive.d 2015-10-02 16:56:31 +02:00
Antoine Beaupré
ea5d00436c also document the cache locations 2015-10-02 10:12:13 -04:00
Antoine Beaupré
ad85f64842 whitespace 2015-10-02 10:10:50 -04:00
Antoine Beaupré
69040588cd update docs to reflect that cache is converted 2015-10-02 10:10:43 -04:00
Antoine Beaupré
d4d1b414b5 remove needless autouse 2015-10-02 09:44:53 -04:00
Antoine Beaupré
41e9942efe follow naming of tested module 2015-10-02 09:43:51 -04:00
Antoine Beaupré
081b91bea0 remove needless paren 2015-10-02 09:43:10 -04:00
Antoine Beaupré
3e7fa0d633 also copy the cache config file to workaround #234 2015-10-01 16:41:17 -04:00
Antoine Beaupré
8022e563a9 don't clobber existing borg cache 2015-10-01 16:27:19 -04:00
Antoine Beaupré
55f79b4999 complete cache conversion code
we need to create the borg cache directory

dry run was ignored, fixed.

process cache before segment, because we want to do the faster stuff first
2015-10-01 16:24:28 -04:00
Antoine Beaupré
28a033d1d3 remove debug output that clobbers segment spinner 2015-10-01 16:03:52 -04:00
Antoine Beaupré
4f9a411ad8 remove unneeded fixture decorator 2015-10-01 16:01:17 -04:00
Antoine Beaupré
022de5be47 untested file/chunks cache conversion
i couldn't figure out how to generate a cache set directly, Archiver is a pain...
2015-10-01 16:01:01 -04:00
Antoine Beaupré
7c32f555ac repository index conversion 2015-10-01 15:43:16 -04:00