Commit graph

82 commits

Author SHA1 Message Date
James Guo
bb173e092b
Update AmazonS3.php
Signed-off-by: James Guo <i@ze3kr.com>
2023-04-06 14:34:08 +08:00
François Ménabé
710f3fd405 Support of s3 storage class in External Storage application
Signed-off-by: François Ménabé <francois.menabe@gmail.com>
2023-01-12 14:59:13 +01:00
Robin Appelman
ed8fedc685
fix s3 directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-11-15 17:17:59 +01:00
Carl Schwan
a5ea677370 Rename file1 and file2 to source and target in Storage abstraction
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 17:21:47 +02:00
Carl Schwan
732badf552 Additional type hinting found by psalm
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-18 12:28:34 +02:00
Carl Schwan
cf1381a6f9 Port away from deprecated clearBucket call
This was removed a long time ago in the library that we use, but never
detected since this was inside a try catch block...

Remove the call for now and we should investigate newer api in the
future.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-16 23:39:49 +02:00
Carl Schwan
1d90f84141 Port to LoggerInterface
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-10-16 23:39:28 +02:00
Matthieu Gallien
54c05bcdb9 handle AccessDenied error when checking if external s3 support versions
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
2022-07-28 09:20:12 +00:00
Carl Schwan
d5c23dbb9f Move CappedMemoryCache to OCP
This is an helpful helper that should be used in more place than just
server and this is already the case with groupfodlers, deck, user_oidc
and more using it, so let's make it public

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-07-14 15:54:31 +02:00
Carl Schwan
ec6b83cc18 Add stricter psalm type for CappedMemoryCache
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-12 15:06:18 +02:00
Carl Schwan
f945c0cbc6 Add a public replacement for OC::$server->get
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-10 18:51:12 +02:00
Erik van Velzen
17f3e5e4b5 AmazonS3: allow not implemented versioning
In case the S3 implementation does not implement versioning, set it to
false. Versioning was introduced in Nexcloud in commit
09ffac5e6d

This is needed for compatibility with the Storj gateway.

Signed-off-by: Erik van Velzen <erik@evanv.nl>
2022-03-15 00:40:38 +01:00
Julius Härtl
4e57146617
Use unique combination of hostname/bucket/key for external storages
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-11-04 11:34:13 +01:00
Robin Appelman
a1ca901e58
cache versioning enabled status
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 15:03:22 +02:00
Robin Appelman
eb6e6e3a85
minor directory detect improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-19 14:58:40 +02:00
Robin Appelman
247e12da96
always set Key field in headObject
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:43:50 +02:00
Robin Appelman
4bd08af2ad
more reliable hasUpdated for s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:02:57 +02:00
Robin Appelman
34637697e1
remove old migration method
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:19 +02:00
Robin Appelman
d3bd0b5a1b
optimize filetype for s3 directories a bit
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:16 +02:00
Robin Appelman
294af4275c
more reliable directory copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:09 +02:00
Robin Appelman
5e3c8b3af2
doesDirectoryExist fixes
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:06 +02:00
Robin Appelman
09ffac5e6d
s3 external storage listing rework
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:02 +02:00
Vincent Petry
d02936bfb3
Revert "Delete S3 versions in rmdir" 2021-10-12 17:06:02 +02:00
Vincent Petry
09ab7a40fe
Delete S3 versions in rmdir
When deleting a complete folder in a bucket that has versioning enabled,
also make sure to delete all associated versions and delete markers

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-10-11 13:45:02 +02:00
Louis Chemineau
735fd94cc0 Fix folder size contained in S3 buckets
If 'filesystem_check_changes' was set to never, the cached size was alway set to -1 (Pending) on every access

Signed-off-by: Louis Chemineau <louis@chmn.me>
2021-08-19 15:03:46 +02:00
Julius Härtl
8a14f64ac2 Delete object to cleanup leftover of paths when removing directories
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-08-13 16:34:48 +00:00
John Molakvoæ (skjnldsv)
215aef3cbd
Update php licenses
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +02:00
Robin Appelman
effb7dc8ba
set mimetype for objects uploaded to object storages
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-04-21 15:25:58 +02:00
Christoph Wurst
d89a75be0b
Update all license headers for Nextcloud 21
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-16 18:48:22 +01:00
Christoph Wurst
fd649afb1f
Remove the deprecated update.php
* It was documented as deprecated.
* The app code checker warned about it
* It's been three years

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-11-27 13:02:59 +01:00
Roeland Jago Douma
b7be09ab2c
Check if array elements exist before using them
It seems that in some recent upgrade here. Not always the entries we
expect are returned. So we should first check if they exist. As to not
spam the log.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-10-25 22:03:40 +01:00
Julius Härtl
2004ae4603
Set delimiter so that the CommonPrefixes response can properly be determined
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-10 08:11:55 +02:00
Robin Appelman
fc96fa0cc7
fix s3 doesDirectoryExist check for empty directories
if a directory is empty, only the 'marker' object `/` exists.

since not all s3 implementations return just the prefix when listing objects by prefix,
when listing objects by the folder prefix, nothing will be returned if the directory is empty.
by not including the trailing slash in the prefix, the folder marked will always be returned if it exists

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-10 08:11:55 +02:00
Roeland Jago Douma
bb06b6cce4
Fix reading empty files from objectstorage
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-09-09 20:45:13 +02:00
Christoph Wurst
2a054e6c04
Update the license headers for Nextcloud 20
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-24 14:54:25 +02:00
Morris Jobke
ac73982395
Merge pull request #21734 from nextcloud/external-s3-no-touch
disable touch to update mtime for external s3 storage
2020-08-20 16:07:59 +02:00
Robin Appelman
0d6910ac13
disable touch to update mtime for external s3 storage
since updating the mtime requires doing a copy this can have a significant overhead and it messes with versioning

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-07 15:55:38 +02:00
Julius Härtl
f5cc34c9c2
Reuse cache for directory mtime/size if filesystem changes can be ignored
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-07-07 10:22:22 +02:00
Julius Härtl
69f4c50032
Remove rescanDelay from directory mtime
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-06-22 14:15:57 +02:00
Christoph Wurst
caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst
14c996d982
Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Christoph Wurst
1a9330cd69
Update the license headers for Nextcloud 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-31 14:52:54 +02:00
Christoph Wurst
b80ebc9674
Use the short array syntax, everywhere
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-26 16:34:56 +01:00
Christoph Wurst
5bf3d1bb38
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-05 15:38:45 +01:00
Roeland Jago Douma
68748d4f85
Some php-cs fixes
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-11-22 20:52:10 +01:00
Daniel Kesselberg
230f685987
Remove superfluous fclose
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-10-21 11:31:07 +02:00
Daniel Kesselberg
72d22a4828
Use local variable for directory name
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg
75b8944061
Don't strip path from directory prefix
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg
80340a8d46
Fill directory cache from opendir
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg
12863e0d24
Change visibility to private (from protected)
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00