upstream/nextcloud
1
0
Fork 0
mirror of https://github.com/nextcloud/server.git synced 2026-02-16 09:19:43 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
78909 commits 690 branches 1185 tags 6.5 GiB
Commit graph

284 commits

Author SHA1 Message Date
Robin Appelman
7eb607c8b9
perf: set the folder size in the same query as we create it 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-06-02 12:00:59 +02:00
Louis Chemineau
1c80a98720 fix(S3): Use original folder size during copy 
This prevent having copied folders with a wrongly set size of 0KB.

- Fix https://github.com/nextcloud/server/issues/51916

Signed-off-by: Louis Chemineau <louis@chmn.me>
 2025-06-02 09:56:15 +00:00
Kate
119727d74a
Merge pull request #52759 from nextcloud/backport/49352/stable30 2025-05-19 06:44:24 +02:00
Robin Appelman
86206e9bac fix: keep IObjectStoreMetaData in private namespace for backports 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-15 18:00:35 +02:00
Robin Appelman
9da4ebd650 feat: add command to list objects 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-15 14:13:06 +02:00
Robin Appelman
042934a99f feat: add command to get object metadata 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-15 14:13:06 +02:00
Robin Appelman
87a4bff4ee chore: cleanup leftover debug statement 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-12 11:59:53 +02:00
Robin Appelman
eade5703ba fix: get object size from stream where possible 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-12 11:59:47 +02:00
Robin Appelman
6632669157 fix: don't perform the extra buffering in s3 stream write when the stream size is known 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-12 11:58:34 +02:00
Robin Appelman
b1ad63666f fix: use php://temp instead of php://memory for multi-part upload buffer 
this should reduce potential memory issues if the limit is set very high

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-05-12 11:58:23 +02:00
Ferdinand Thiessen
832f79ac93
chore: apply code style 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2025-04-30 19:04:59 +02:00
Anna Larch
cae5c4f02b fix(objectstorage): add retry attempts to S3 connection 
Signed-off-by: Anna Larch <anna@nextcloud.com>
 2025-04-23 18:40:07 +00:00
Louis Chemineau
9bed21a8d7 fix: Transfer ownership with S3 as primary 
When using S3 as primary storage, transferring ownership with the `--move` option fail with the following error:

`SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8-45b963397aa40d4a0063e0d85e4fe7a1' for key 'fs_storage_path_hash'`

The `--move` option moves the entire home folder from one account to another.
The error means that the move failed because the destination folder already exist in `oc_filecache`.

- With S3 as primary storage, folders only exists as entries in `oc_filecache`.
- With S3 as primary storage, `moveFromStorage(...)` only moves the cache entry, as nothing needs to be moved on disk. This cache move does not delete potentially pre-existing destination folder.
- With Local storage, `moveFromStorage(...)` calls `rename(...)` which delete pre-existing folder.

- `transfer(...)`: 687a4d9ac7/apps/files/lib/Service/OwnershipTransferService.php (L112)
- `oneTimeUserSetup(...)`: 687a4d9ac7/lib/private/Files/SetupManager.php (L261-L262)
- `mkdir(...)`: 687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L91-L135)
- `moveFromStorage(...)`: 687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L635-L636)

Delete pre-existing folder in `moveFromStorage(...)`

Signed-off-by: Louis Chemineau <louis@chmn.me>
 2025-04-16 16:18:42 +02:00
Andy Scherzinger
682c8e6a61
Merge pull request #50796 from nextcloud/backport/50781/stable30 
[stable30] perf(files): faster query to fetch incomplete directories
 2025-02-17 19:15:58 +01:00
Benjamin Gaussorgues
765b4921ff
perf(files): faster query to fetch incomplete directories 
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
 2025-02-14 10:57:26 +01:00
Robin Appelman
da334737e4 fix: rework move into object store to better preserve fileids 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-02-13 17:31:51 +01:00
Robin Appelman
e85d5df5f0 fix: preserve fileid when moving from objectstore to non-objectstore 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-02-13 17:31:51 +01:00
Robin Appelman
9fd7dfd1c2 fix: ensure source folder is removed from cache when moving to objectstore 
otherwise this causes confusion down the line as it's contents will be moved to the new cache

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-02-13 17:31:51 +01:00
Richard Steinmetz
b081f3b7c4 fix(s3): treat empty sse_c_key as not set 
Co-authored-by: Christian Becker <christian@dabecka.de>
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
 2025-01-29 17:34:47 +00:00
Robin Appelman
3522819624
fix: throw correct exception type when we can't verify if an s3 bucket exists 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-11-26 18:49:56 +01:00
Kate
61f3a09dcd
Merge pull request #48233 from nextcloud/backport/48222/stable30 2024-09-20 23:02:25 +02:00
Kate
ae7241f6c8
Merge pull request #48221 from nextcloud/backport/30/fix_move_on_same_bucket 2024-09-20 10:06:14 +02:00
Robin Appelman
e8c8ab58f7 perf: reuse cache info we already have when moving to object store 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-20 07:50:23 +00:00
Christoph Fiehe
82be489f53 perf(ObjectStoreStorage): Improve (slow) move on same object bucket 
This commit fixes the issue #47856. When you upload a file into a group folder and when you use a single S3 bucket as primary storage, the final move operation hangs for a long time. In the background, Nextcloud initiates a copy-delete sequence from the bucket into the bucket, with causes a lot unnecessary overhead. Nextcloud thinks that the file must be imported to another storage and does not recognize that everything is done on the same object bucket. In that case, the import step can be completely skipped, which saves time, network bandwidth and reduces the load on the object storage.

The behavior improves a lot with https://github.com/nextcloud/server/pull/46013. However, there are still some put messages that are being sent to the object storage when you use an object storage as primary storage and upload files into a group folder.

Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Signed-off-by: Christoph Fiehe <c.fiehe@eurodata.de>
 2024-09-19 18:53:36 +02:00
Robin Appelman
dd68f678b2 fix: ensure parent folder exists when writing a file to object storage 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-19 13:59:08 +02:00
Robin Appelman
1e03bd5d18 fix: fix object store id for test object store 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-19 13:59:08 +02:00
Robin Appelman
e8c7216d5b fix: cleanup objectstore file_put_content 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-19 13:59:08 +02:00
Robin Appelman
9b07b7d9c1 fix: create intermediate directories for objectstore moveFromStorage 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-19 13:59:08 +02:00
Robin Appelman
5bcd74ae87 fix: get source file before moving the cache item in objectstore moveFromStorage 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-19 13:59:08 +02:00
provokateurin
69b01a265c fix(S3ConfigTrait): Allow proxy field to take false 
Signed-off-by: provokateurin <kate@provokateurin.de>
 2024-09-15 17:19:40 +00:00
Robin Appelman
7206e5e086 fix: make swift connect exception message more informative 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-10 14:47:23 +00:00
Robin Appelman
05f79f23cd fix: write object to the correct urn when moving from another storage to object store 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-09-04 23:17:04 +02:00
Git'Fellow
c84e76a749 fix(s3): Don't wait indefinitely for S3 to return 
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: lint

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: use AwsException

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: Throw on connection failure

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: Wrap all in try catch block

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: use RequestTimeout error message

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

log: use OCP Server class

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: Handle connect timeout only

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

fix: Handle errors more generically

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
 2024-07-05 16:24:32 +02:00
Ferdinand Thiessen
ae1f5cc56d
fix(S3): Adjust usage of guzzle promise 
`Promise\promise_for` was deprecated and is now removed and replaced with the static API (`Create::promiseFor`).

Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-06-16 15:54:44 +02:00
Josh
63886147ba fix(ObjectStore): handle empty S3 hostname 
Fixes #45637

The support for s3-accelerate added in #44496 introduced a regression in AWS S3 environments when `hostname` is blank (which is a valid configuration w/ AWS since the hostname gets auto-generated).

Signed-off-by: Josh <josh.t.richards@gmail.com>
 2024-06-12 15:43:20 -04:00
Andy Scherzinger
dae7c159f7
chore: Add SPDX header 
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
 2024-05-24 13:11:22 +02:00
Robin Appelman
40d9a3fcab
feat: add option to enable s3 acceleration 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-04-26 14:40:24 +02:00
Joas Schilling
cdb40a3110
fix(openstack): Fix psalm issue with updated openstack version 
Signed-off-by: Joas Schilling <coding@schilljs.com>
 2024-04-24 07:01:57 +02:00
Côme Chilliet
ec5133b739 fix: Apply new coding standard to all files 
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
 2024-04-02 14:16:21 +02:00
Ferdinand Thiessen
7880086de8
fix(S3): Adjust typing for 32bit compatibility 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-03-26 13:07:53 +01:00
Ferdinand Thiessen
32dee2f84a
feat(ObjectStore): Make S3 MultipartUpload concurrency configurable 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-03-21 16:25:49 +01:00
Ferdinand Thiessen
69c5e8ebde
fix(ObjectStore): Use common S3 trait so config and connection trait members are in sync 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-03-21 16:25:42 +01:00
Joas Schilling
951c330d39
Revert "don't perform an explicit setup when using the file cache" 2024-03-07 14:38:02 +01:00
John Molakvoæ
b7088e7071
Merge pull request #32163 from nextcloud/file-cache-gc-no-full-setup 
don't perform an explicit setup when using the file cache
 2024-03-07 11:09:19 +01:00
Robin Appelman
4fba4cd14c fix: fix user folder init 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-03-06 17:52:24 +01:00
Robin Appelman
c54f7bd761 fix: disable the internal s3sdk multi part copy logic 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-03-06 15:32:14 +01:00
Côme Chilliet
c0ce272e9c chore: Migrate away from OC::$server->getLogger 
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
 2024-02-13 17:32:30 +01:00
Thomas Citharel
30798eb6c2
refactor(objectstorage): cleanup types 
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
 2024-02-02 14:59:21 +01:00
Thomas Citharel
3be3dbdb3b
refactor(objectstorage): move $user property from ObjectStoreStorage class to HomeObjectStoreStorage 
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
 2024-02-02 14:59:21 +01:00
Thomas Citharel
4c618c4bf1
fix(objectstorage): cleanup HomeObjectStoreStorage 
Fix invalid signature for getUser() method and change occurences of OC\User\User for OCP\IUser

Signed-off-by: Thomas Citharel <tcit@tcit.fr>
 2024-02-02 14:59:20 +01:00