upstream/nextcloud
1
0
Fork 0
mirror of https://github.com/nextcloud/server.git synced 2026-02-03 20:41:22 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions 25
77085 commits 651 branches 1179 tags 8.1 GiB
Commit graph

262 commits

Author SHA1 Message Date
Stephan Orbaugh
f25acfb799
Merge pull request #47986 from cfiehe/fix_move_on_same_bucket 
perf(ObjectStoreStorage): Improve (slow) move on same object bucket
 2024-09-17 16:35:19 +02:00
provokateurin
d2d1e3284d
fix(Storage): Fix method docs inheritance 
Signed-off-by: provokateurin <kate@provokateurin.de>
 2024-09-17 10:20:38 +02:00
provokateurin
8ca6fcace7
fix(Storage): Document getOwner() can return false 
Signed-off-by: provokateurin <kate@provokateurin.de>
 2024-09-17 10:10:50 +02:00
Côme Chilliet
492e6997d8
chore: Fix psalm issues, put back private versions of getter in private Storage interface 
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
 2024-09-16 18:11:38 +02:00
Christoph Fiehe
9597072ada perf(ObjectStoreStorage): Improve (slow) move on same object bucket 
This commit fixes the issue #47856. When you upload a file into a group folder and when you use a single S3 bucket as primary storage, the final move operation hangs for a long time. In the background, Nextcloud initiates a copy-delete sequence from the bucket into the bucket, with causes a lot unnecessary overhead. Nextcloud thinks that the file must be imported to another storage and does not recognize that everything is done on the same object bucket. In that case, the import step can be completely skipped, which saves time, network bandwidth and reduces the load on the object storage.

The behavior improves a lot with https://github.com/nextcloud/server/pull/46013. However, there are still some put messages that are being sent to the object storage when you use an object storage as primary storage and upload files into a group folder.

Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Signed-off-by: Christoph Fiehe <c.fiehe@eurodata.de>
 2024-09-16 15:11:09 +02:00
provokateurin
7c1c02d6a5
fix(S3ConfigTrait): Allow proxy field to take false 
Signed-off-by: provokateurin <kate@provokateurin.de>
 2024-09-15 15:41:58 +02:00
Robin Appelman
508d8e7c01
fix: make swift connect exception message more informative 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-09 17:15:01 +02:00
Julius Härtl
6d2253e692
Merge pull request #46013 from nextcloud/obj-store-move-from-storage-preserve-fileid 
fix: write object to the correct urn when moving from another storage to object store
 2024-08-26 13:47:27 +02:00
Daniel Kesselberg
af6de04e9e
style: update codestyle for coding-standard 1.2.3 
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
 2024-08-25 19:34:58 +02:00
Robin Appelman
bd740ac0b0
fix: write object to the correct urn when moving from another storage to object store 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-08-23 15:22:16 +02:00
Git'Fellow
c84e76a749 fix(s3): Don't wait indefinitely for S3 to return 
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: lint

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: use AwsException

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: Throw on connection failure

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: Wrap all in try catch block

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: use RequestTimeout error message

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

log: use OCP Server class

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: Handle connect timeout only

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: Handle errors more generically

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
 2024-07-05 16:24:32 +02:00
Ferdinand Thiessen
ae1f5cc56d
fix(S3): Adjust usage of guzzle promise 
`Promise\promise_for` was deprecated and is now removed and replaced with the static API (`Create::promiseFor`).

Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-06-16 15:54:44 +02:00
Josh
63886147ba fix(ObjectStore): handle empty S3 hostname 
Fixes #45637

The support for s3-accelerate added in #44496 introduced a regression in AWS S3 environments when `hostname` is blank (which is a valid configuration w/ AWS since the hostname gets auto-generated).

Signed-off-by: Josh <josh.t.richards@gmail.com>
 2024-06-12 15:43:20 -04:00
Andy Scherzinger
dae7c159f7
chore: Add SPDX header 
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
 2024-05-24 13:11:22 +02:00
Robin Appelman
40d9a3fcab
feat: add option to enable s3 acceleration 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-04-26 14:40:24 +02:00
Joas Schilling
cdb40a3110
fix(openstack): Fix psalm issue with updated openstack version 
Signed-off-by: Joas Schilling <coding@schilljs.com>
 2024-04-24 07:01:57 +02:00
Côme Chilliet
ec5133b739 fix: Apply new coding standard to all files 
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
 2024-04-02 14:16:21 +02:00
Ferdinand Thiessen
7880086de8
fix(S3): Adjust typing for 32bit compatibility 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-03-26 13:07:53 +01:00
Ferdinand Thiessen
32dee2f84a
feat(ObjectStore): Make S3 MultipartUpload concurrency configurable 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-03-21 16:25:49 +01:00
Ferdinand Thiessen
69c5e8ebde
fix(ObjectStore): Use common S3 trait so config and connection trait members are in sync 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-03-21 16:25:42 +01:00
Joas Schilling
951c330d39
Revert "don't perform an explicit setup when using the file cache" 2024-03-07 14:38:02 +01:00
John Molakvoæ
b7088e7071
Merge pull request #32163 from nextcloud/file-cache-gc-no-full-setup 
don't perform an explicit setup when using the file cache
 2024-03-07 11:09:19 +01:00
Robin Appelman
4fba4cd14c fix: fix user folder init 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-03-06 17:52:24 +01:00
Robin Appelman
c54f7bd761 fix: disable the internal s3sdk multi part copy logic 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-03-06 15:32:14 +01:00
Côme Chilliet
c0ce272e9c chore: Migrate away from OC::$server->getLogger 
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
 2024-02-13 17:32:30 +01:00
Thomas Citharel
30798eb6c2
refactor(objectstorage): cleanup types 
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
 2024-02-02 14:59:21 +01:00
Thomas Citharel
3be3dbdb3b
refactor(objectstorage): move $user property from ObjectStoreStorage class to HomeObjectStoreStorage 
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
 2024-02-02 14:59:21 +01:00
Thomas Citharel
4c618c4bf1
fix(objectstorage): cleanup HomeObjectStoreStorage 
Fix invalid signature for getUser() method and change occurences of OC\User\User for OCP\IUser

Signed-off-by: Thomas Citharel <tcit@tcit.fr>
 2024-02-02 14:59:20 +01:00
Robin Appelman
b777304a77
also updated parent etags when a changed etag is detected during scanning 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-01-31 18:06:18 +01:00
Julius Härtl
e4054370b1
fix(s3): Add config option to disable multipart copy for certain s3 providers 
Signed-off-by: Julius Härtl <jus@bitgrid.net>
 2023-12-28 15:31:38 +01:00
Robin Appelman
1043c21b35
only do a multipart s3 copy when above the regular copy limit 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-12-28 15:31:37 +01:00
Ferdinand Thiessen
5172baaf8b
fix(ObjectStore): Make copying behavior consistent with local storage 
Drop file permissions on copy like we do on local storage.

Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2023-11-17 12:54:37 +01:00
Tobias Zimmerer
cdf96fab92 fix(s3): support SSE-C headers for the MultipartCopy call 
Signed-off-by: Tobias Zimmerer <3228193+ir0nhide@users.noreply.github.com>
 2023-11-14 21:19:59 +01:00
Hamid Dehnavi
ea06cf2f39 Convert isset ternary to null coalescing operator 
Signed-off-by: Hamid Dehnavi <hamid.dev.pro@gmail.com>
 2023-09-28 17:44:19 +03:30
Côme Chilliet
dc347b39c1
Merge pull request #40501 from nextcloud/fix/fix-int-size-in-files-scanner 
Store size in int|float for 32bit support
 2023-09-25 16:32:32 +02:00
Robin Appelman
eab8dd3ca0 use multipart copy for s3 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-09-20 17:28:48 +02:00
Côme Chilliet
5090848b25
Store size in int|float for 32bit support 
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
 2023-09-19 13:43:01 +02:00
Julius Härtl
88a96e4db7 fix: Only store unencrypted_size if path should be encrypted 
Signed-off-by: Julius Härtl <jus@bitgrid.net>
 2023-09-06 07:25:28 +02:00
Robin Appelman
64f62f7cfd
Merge pull request #39863 from nextcloud/sharing-mask-wrapper 
move share permission logic to storage wrapper
 2023-09-04 14:05:14 +02:00
Thomas Citharel
3f28fc58aa fix(s3): fix handling verify_bucket_exists parameter 
If 'verify_bucket_exists' is set to false in the config.php s3 configuration, it's supposed to avoid
verifying that the bucket exists. However empty(falsy) will  always return true, so this condition
would not work.

Signed-off-by: Thomas Citharel <tcit@tcit.fr>
 2023-08-31 21:36:03 +02:00
Robin Appelman
a031bc4788 more share permission logic to storage wrapper 
this way we only have to determine the share permissions once

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-08-14 18:17:27 +02:00
Tobias Zimmerer
23f85b5ac2 fix(s3): add SSE-C parameters to headObject call 
Signed-off-by: Tobias Zimmerer <3228193+ir0nhide@users.noreply.github.com>
 2023-07-30 22:54:58 +02:00
Julius Härtl
77f249726d
Merge pull request #38934 from nextcloud/bugfix/noid/ssec-multipart 2023-07-20 11:00:08 +02:00
Robin Appelman
39a0554c09
Merge pull request #38858 from nextcloud/scanner-performance3 
file scanner performance improvements
 2023-06-26 17:01:18 +02:00
Julius Härtl
ea575e342f
fix(s3): Pass SSE-C parameters for multipart upload 
Signed-off-by: Julius Härtl <jus@bitgrid.net>
 2023-06-21 18:32:24 +02:00
Robin Appelman
de85610385 use source cache when listing folder during recursive copy 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-06-19 16:40:15 +02:00
Robin Appelman
561536fc58
file scanner performance improvements 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-06-16 16:31:48 +02:00
Robin Appelman
ea88ec1350
add utility command for object store objects 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-06-09 18:08:18 +02:00
Robin Appelman
9f1d497a0b
Merge pull request #38261 from fsamapoor/replace_strpos_calls_in_lib_private 
Refactors "strpos" calls in  lib/private to improve code readability.
 2023-06-01 23:10:00 +02:00
Robin Appelman
cc0ad25c98
don't always check if we need to setup the object store root 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-05-23 17:44:40 +02:00