Commit graph

191 commits

Author SHA1 Message Date
Julius Härtl
4e57146617
Use unique combination of hostname/bucket/key for external storages
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-11-04 11:34:13 +01:00
Julius Härtl
9356124700
Merge pull request #29021 from nextcloud/work/smb/additional-logging 2021-10-25 13:25:04 +02:00
Robin Appelman
a1ca901e58
cache versioning enabled status
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 15:03:22 +02:00
Robin Appelman
eb6e6e3a85
minor directory detect improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 14:58:40 +02:00
Robin Appelman
247e12da96
always set Key field in headObject
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:43:50 +02:00
Robin Appelman
4bd08af2ad
more reliable hasUpdated for s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:02:57 +02:00
Robin Appelman
34637697e1
remove old migration method
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:19 +02:00
Robin Appelman
d3bd0b5a1b
optimize filetype for s3 directories a bit
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:16 +02:00
Robin Appelman
294af4275c
more reliable directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:09 +02:00
Robin Appelman
5e3c8b3af2
doesDirectoryExist fixes
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:06 +02:00
Robin Appelman
09ffac5e6d
s3 external storage listing rework
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:02 +02:00
Robin Appelman
fadeae8c8a
Merge pull request #29185 from nextcloud/revert-29165-bug/noid/s3-delete-folder-versions
Revert "Delete S3 versions in rmdir"
2021-10-15 13:30:51 +00:00
John Molakvoæ
739b67031b
Merge pull request #27194 from nextcloud/new-ftp 2021-10-14 10:29:24 +02:00
Vincent Petry
d02936bfb3
Revert "Delete S3 versions in rmdir" 2021-10-12 17:06:02 +02:00
Vincent Petry
09ab7a40fe
Delete S3 versions in rmdir
When deleting a complete folder in a bucket that has versioning enabled,
also make sure to delete all associated versions and delete markers

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-10-11 13:45:02 +02:00
Robin Appelman
a3f1d10f4d
ftp ci improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-07 18:47:13 +02:00
Robin Appelman
7d582cb489
update copyright header
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-07 17:36:17 +02:00
Robin Appelman
6b80ae9d44
teardown
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-07 17:19:36 +02:00
Robin Appelman
ccb24416ac
add new ftp backend
this uses the raw `ftp_` functions instead of the stream wrapper

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-07 17:19:23 +02:00
Carl Schwan
365f5e425f
Add additional logging on SMB failure to read folder content
Currently we discard the error comming from the SMB library and create a
new exception for nextcloud. This patch makes sure that the
access/permission exception from the SMB library are logged correctly.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-10-01 17:54:27 +02:00
Louis Chemineau
735fd94cc0 Fix folder size contained in S3 buckets
If 'filesystem_check_changes' was set to never, the cached size was alway set to -1 (Pending) on every access

Signed-off-by: Louis Chemineau <louis@chmn.me>
2021-08-19 15:03:46 +02:00
Julius Härtl
8a14f64ac2 Delete object to cleanup leftover of paths when removing directories
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-08-13 16:34:48 +00:00
Richard Steinmetz
2e53cf4c1f
Gracefully handle smb acls for users without a domain
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2021-08-13 09:43:50 +02:00
John Molakvoæ (skjnldsv)
215aef3cbd
Update php licenses
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +02:00
Robin Appelman
effb7dc8ba
set mimetype for objects uploaded to object storages
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-04-21 15:25:58 +02:00
Julius Härtl
8f01bc65f1
Continue iterating over diretory contents and just hide file if denied by acl
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-04-20 17:17:09 +02:00
Robin Appelman
43f70e5817
ignore readonly flag for directories
see http://support.microsoft.com/kb/326549

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-25 12:03:09 +01:00
Robin Appelman
5ffaa94fbc
add (hidden) option to always show smb root as writable
some smb servers are very insistent in reporting that the root of the share is readonly, even if it isn't.

This works around the problem by adding a hidden option to overwrite the permissions of the root of the share.

This can be enabled using

```bash
occ files_external:config <mount id> root_force_writable true
```

where you can find your mount id using

```bash
occ files_external:list
```

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 15:35:35 +01:00
Robin Appelman
b844293536 catch notfound and forbidden exception in smb::getmetadata
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-04 21:34:54 +01:00
Julius Härtl
e9ae943bba
Properly handle SMB ACL blocking scanning a directory
This makes sure that a possible ForbiddenException is properly passed
through the storage as a ForbiddenException and can be catched when
trying to fetch the quota info of a parent folder

Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-01-29 11:26:22 +01:00
Christoph Wurst
4373afeae1
Bump phpseclib/phpseclib from 2.0.25 to 2.0.30
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2021-01-19 21:19:02 +01:00
Christoph Wurst
73c7d0dc81
Bump icewind/streams from 0.7.1 to 0.7.2
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-29 09:07:36 +01:00
Christoph Wurst
d89a75be0b
Update all license headers for Nextcloud 21
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-16 18:48:22 +01:00
Christoph Wurst
fd649afb1f
Remove the deprecated update.php
* It was documented as deprecated.
* The app code checker warned about it
* It's been three years

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-11-27 13:02:59 +01:00
Roeland Jago Douma
b7be09ab2c
Check if array elements exist before using them
It seems that in some recent upgrade here. Not always the entries we
expect are returned. So we should first check if they exist. As to not
spam the log.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-10-25 22:03:40 +01:00
Christoph Wurst
d9015a8c94
Format code to a single space around binary operators
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-10-05 20:25:24 +02:00
Robin Appelman
43b3049a4f
improve handling of out of space errors for smb
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-17 17:23:33 +02:00
Julius Härtl
2004ae4603
Set delimiter so that the CommonPrefixes response can properly be determined
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-10 08:11:55 +02:00
Robin Appelman
fc96fa0cc7
fix s3 doesDirectoryExist check for empty directories
if a directory is empty, only the 'marker' object `/` exists.

since not all s3 implementations return just the prefix when listing objects by prefix,
when listing objects by the folder prefix, nothing will be returned if the directory is empty.
by not including the trailing slash in the prefix, the folder marked will always be returned if it exists

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-10 08:11:55 +02:00
Roeland Jago Douma
bb06b6cce4
Fix reading empty files from objectstorage
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-09-09 20:45:13 +02:00
Christoph Wurst
2a054e6c04
Update the license headers for Nextcloud 20
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-24 14:54:25 +02:00
Morris Jobke
ac73982395
Merge pull request #21734 from nextcloud/external-s3-no-touch
disable touch to update mtime for external s3 storage
2020-08-20 16:07:59 +02:00
Morris Jobke
fedf9c69d9
Use matching parameter names form interfaces and implementations
Found by Psalm 3.14.1

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-08-19 18:16:35 +02:00
Daniel Kesselberg
ba3a855ad8
Remove unused method uploadFile
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2020-08-13 08:02:28 +02:00
Morris Jobke
638e9438a9
Merge pull request #21728 from nextcloud/bugfix/noid/s3-external-mtime-cache
Reuse cache for directory mtime/size if filesystem changes can be ignored
2020-07-15 18:26:48 +02:00
Robin Appelman
e357d46863
filter files containing a hash in the path for ftp storages
the php ftp streamwrapper doesn't handle hashes correctly and will break when it tries to enter a path containing a hash.

By filtering out paths containing a hash we can at least stop the external storage from breaking completely

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-10 14:18:40 +02:00
Robin Appelman
0d6910ac13
disable touch to update mtime for external s3 storage
since updating the mtime requires doing a copy this can have a significant overhead and it messes with versioning

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-07 15:55:38 +02:00
Julius Härtl
f5cc34c9c2
Reuse cache for directory mtime/size if filesystem changes can be ignored
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-07-07 10:22:22 +02:00
Julius Härtl
69f4c50032
Remove rescanDelay from directory mtime
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-06-22 14:15:57 +02:00
Robin Appelman
6f2e920a5b
normalize sftp path in read and write stream
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-05-25 15:26:04 +02:00