Commit graph

59 commits

Author SHA1 Message Date
Will
242424ecae Move sync_file_range to its own extension 2020-03-01 00:06:04 -05:00
Thomas Waldmann
b43d3bb7fa new BORG_WORKAROUNDS mechanism, basesyncfile, fixes #4710
remove WSL autodetection. if WSL still has this problem, you need to
set BORG_WORKAROUNDS=basesyncfile in the borg process environment to
work around it.

(cherry picked from commit beb948fc71)
2019-08-27 19:37:40 +05:30
Jürg Rast
6b426d08d7 Initial work to build and run borg under windows
- Created a batch file to build borg on windows
- Adjusted setup.py to be runnable on windows and build the windows
extension
- Extracted the free space check to a function in the platform module
- Created the minimal needed (dummy) functions for the windows platform
module
2019-08-24 10:17:18 +02:00
Thomas Waldmann
f66c102cc5 bump platform api version 2019-03-18 21:51:02 +01:00
Thomas Waldmann
0686237484 make freebsd xattr platform code api compatible with linux, fixes #3952
i.e. prefix the keys with the namespace, so it is ns.key like on
linux.

still only dealing with the "user" namespace, like before.
in the "system" namespaces there are ACLs (we deal with them via the acl
api, so no problem) and stuff from pnfsd (not sure what exactly).

this change is needed because FreeBSD's FUSE code expects the xattr
keys to be in that format.

it is also needed for cross-platform data exchange, so e.g. if one wants to:
- create archive on linux, extract on freebsd - with "user.xxx" xattrs.
- or vice versa.

archives made with older borg versions on freebsd will still extract correctly
on freebsd (not on linux though) even though they do not have the
namespace prefixes in the archived metadata (it will be interpreted in
same way as if they were prefixed by "user." as we do not support any
other namespace anyway).

they do not have "
2019-03-18 21:40:46 +01:00
Björn Ketelaars
c5e3f3b4d4 acl_get() and acl_set() should accept a fd. Related to #4403
Fixes:

self = <borg.archive.MetadataCollector object at 0x14ffb70b28d0>
st = os.stat_result(st_mode=16895, st_ino=246007, st_dev=65280, st_nlink=3, st_uid=100
0, st_gid=0, st_size=512, st_atime=1551123978, st_mtime=1551123979, st_ctime=155112397
9)
path = 'input', fd = 16

    def stat_ext_attrs(self, st, path, fd=None):
        attrs = {}
        bsdflags = 0
        with backup_io('extended stat'):
            if not self.nobsdflags:
                bsdflags = get_flags(path, st, fd=fd)
            xattrs = xattr.get_all(fd or path, follow_symlinks=False)
>           acl_get(path, attrs, st, self.numeric_owner, fd=fd)
E           TypeError: acl_get() got an unexpected keyword argument 'fd'

src/borg/archive.py:1023: TypeError
2019-02-26 17:06:58 +01:00
Thomas Waldmann
a65cefb7bb bump API_VERSIONs to 1.2_xx 2019-02-24 19:45:41 +01:00
Thomas Waldmann
b960d3cd23 linux: acl_(get|set) - adapt to more FD usage / default acl for dirs
acl_get:

remove assumption that having an FD means it is a regular file, we try
to use FDs a much as possible.

only get the default acl for directories - other fs objects are not
expected to have a default acl.

the path needs to be encoded also for the case when we have an fd,
it is needed to get the default acl for directories.

also: micro-opt: encode path later, not needed for ISLNK check.

acl_set:

remove the "if False" branch, it is the same here: the fd-based api
only supports access ACLs, but not default ACLs, so we always need
to use the path-based api here.
2019-02-14 09:20:04 +01:00
Thomas Waldmann
a9d562de17 work around some Microsoft WSL issues
see there:

https://github.com/borgbackup/borg/issues/1961

and especially there (not implemented sync_file_range):

https://github.com/Microsoft/WSL/issues/645
2019-02-05 15:15:50 +01:00
Thomas Waldmann
111c34f92e make "hostname" short, even on misconfigured systems, fixes #4262 2019-01-29 19:51:51 +01:00
Thomas Waldmann
ca6a7d1500 on linux, symlinks can't have ACLs, see #4044
(cherry picked from commit ca1928c925)
2018-12-15 21:32:27 +01:00
Emmo Emminghaus
e5c22eac0d remove accidental change 2018-11-10 23:39:28 +01:00
Emmo Emminghaus
733a2bfa30 Introduce borg.platformflags.is_<os> 2018-11-10 23:34:43 +01:00
Emmo Emminghaus
973921df5a remove unneded code and remarks, added needed except clause 2018-11-10 21:56:39 +01:00
Emmo Emminghaus
558ca61d20 remove posix issues and fixup for unsupported methodes 2018-11-10 21:48:46 +01:00
Emmo Emminghaus
b997d5ba5b move code from borg.helpers.usergroup to borg.platform.posix 2018-11-10 21:43:45 +01:00
Thomas Waldmann
90348c1de9 add BORG_HOST_ID, fixes #3985 2018-08-04 15:01:48 +02:00
Thomas Waldmann
a51fca49bf process_alive: add some assertions
(cherry picked from commit 2bf1b29e8e)
2018-07-17 19:06:26 +02:00
Ivan Shapovalov
074a984111 xattr: use sys/xattr.h as per setxattr(2), attr/xattr.h is absent on my system 2018-07-10 13:08:59 +03:00
Thomas Waldmann
44b4f9645d xattrs: use follow_symlinks=False as default, more tests
it's like we do it everywhere else because this is what we usually need.
2018-07-08 15:08:01 +02:00
Thomas Waldmann
018b62c845 bsdflags: use fd instead of path
this optimization is only needed for linux, the bsd-like platforms
do not need an open file to run a ioctl against, but have bsdflags
in the stat result already.

on linux, this optimization saves 1 file open/close per input file.
2018-07-07 17:30:17 +02:00
Thomas Waldmann
7e47e68e29 acls: use fd instead of path 2018-07-07 17:02:37 +02:00
Thomas Waldmann
5bf5f12be1 acls: only calls os.fsencode for str paths 2018-07-07 16:54:19 +02:00
Thomas Waldmann
34a51eb958 xattr: fix dummy base getxattr, must raise for any given name 2018-07-07 15:47:56 +02:00
Thomas Waldmann
c29c3063b0 xattr: use bytes typed path for listxattr, getxattr, setxattr 2018-07-07 15:47:56 +02:00
Thomas Waldmann
9deb90db71 xattr: use bytes typed names for listxattr, getxattr, setxattr 2018-07-07 15:47:56 +02:00
Thomas Waldmann
b5a9ac5682 xattr: use bytes typed values for listattr, getxattr, setxattr
- getxattr should only return bytes, not None
- setxattr should not get a None value, just bytes
- remove unneeded tmp vars
2018-07-07 15:47:56 +02:00
Thomas Waldmann
99149684bf xattr: move to platform package, use cython, fixes #2495
this code used to live in borg.xattr and used ctypes
(and was the only ctypes-using code in borg).

the low level code now was converted to cython and
the platform code moved to platform package.

got rid of the code that tried to find the libc.
2018-07-07 15:47:56 +02:00
Thomas Waldmann
b76661ab67 acl platform code: fix acl set return type (master) 2018-07-05 22:56:54 +02:00
Thomas Waldmann
b419098fa4 make swidth available on all posix platforms, fixes #2667
it is assumed that posix == everything else than win32.
2018-06-12 21:25:17 +02:00
Thomas Waldmann
9b0d0f3127 use patched version of socket.getfqdn(), fixes #3471 2017-12-24 04:46:20 +01:00
Thomas Waldmann
5e4df7782b refactor/move hostname/fqdn related funcs, see #3471
- move stuff to platform.base (should be platform independent according
  to the docs).
- bump platform API version
- parseformat: import fqdn from platform instead of recomputing it

This is not yet fixing #3471, just a preparation for it.
2017-12-24 04:12:02 +01:00
Thomas Waldmann
a6ee4e9aed bsdflags support: do not open BLK/CHR/LNK files, fixes #3130
opening a device file for a non-existing device can be very slow.
symlinks will make the open() call fail as it is using O_NOFOLLOW.

also: lstat -> stat(..., follow_symlinks=False) like everywhere else.
2017-10-13 23:53:22 +02:00
Thomas Waldmann
6f94949a36 migrate locks to child PID when daemonize is used
also:

increase platform api version due to change in get_process_id behaviour.
2017-08-08 03:46:44 +02:00
Marian Beermann
2fe37dba7f umount: try fusermount, then try umount 2017-07-24 13:55:32 +02:00
Marian Beermann
ed0a5c798f platform.SaveFile: truncate_and_unlink temporary
SaveFile is typically used for small files where this is not
necessary. The sole exception is the files cache.
2017-06-06 18:13:20 +02:00
Marian Beermann
7b519e4769 platform.linux: get rid of "resource" module 2017-04-19 11:31:40 +02:00
Thomas Waldmann
7f2a108c94 fixup: do not access os.POSIX_FADV_* early
before we know posix_fadvise support exists on the platform.
2017-01-30 03:11:42 +01:00
Thomas Waldmann
c0dc644ef6 Merge branch '1.0-maint' into merge-1.0-maint
# Conflicts:
#	MANIFEST.in
#	Vagrantfile
#	docs/changes.rst
#	docs/usage/mount.rst.inc
#	src/borg/archiver.py
#	src/borg/fuse.py
#	src/borg/repository.py
2017-01-29 05:49:53 +01:00
Thomas Waldmann
e4c5db4efc posix platform module: only build / import on non-win32 platforms, fixes #2041
rather use a inverted check like "not windows".
also: add a base implementation for this stuff, just raising NotImplementedError
2017-01-14 14:48:49 +01:00
Thomas Waldmann
e431d60cc5 merge 1.0-maint into master
# Conflicts:
#	src/borg/crypto.pyx
#	src/borg/hashindex.pyx
#	src/borg/helpers.py
#	src/borg/platform/__init__.py
#	src/borg/platform/darwin.pyx
#	src/borg/platform/freebsd.pyx
#	src/borg/platform/linux.pyx
#	src/borg/remote.py
2017-01-14 03:07:11 +01:00
Marian Beermann
b9770c348f posix: use fully-qualified hostname + node ID
The node ID is usually the 48 bit MAC of the primary network
interface.
2017-01-12 00:54:17 +01:00
Thomas Waldmann
afab1cc9b0 Merge branch '1.0-maint'
# Conflicts:
#	borg/platform.py
#	src/borg/archiver.py
#	src/borg/helpers.py
#	src/borg/platform/darwin.pyx
#	src/borg/platform/freebsd.pyx
#	src/borg/platform/linux.pyx
#	src/borg/testsuite/__init__.py
2016-11-17 23:56:34 +01:00
Marian Beermann
c380d91805 fixes for flake 3.1.1 2016-11-14 21:50:20 +01:00
Marian Beermann
676e69cac4 Parse & pass BORG_HOSTNAME_IS_UNIQUE env var to enable stale lock killing 2016-11-07 21:56:40 +01:00
Marian Beermann
cc14975f2d Add tests for stale lock killing and platform.process_alive 2016-11-07 21:56:07 +01:00
Marian Beermann
2bd8ac7762 platform: bump API version (and check consistency) 2016-11-07 21:54:16 +01:00
Marian Beermann
c562f7750c Move platform-dependent code to platform package 2016-11-07 21:54:16 +01:00
TW
02e229b255 Merge pull request #1377 from enkore/issue/1060
Use atomic file writing and updating for configuration and key files
2016-07-30 16:06:01 +02:00
Lee Bousfield
b652f4039c
Remove trailing whitespace 2016-07-28 10:35:41 -04:00