Commit graph

67 commits

Author SHA1 Message Date
Carl Schwan
b6313f68d3
perf(s3): Expose pre-signed urls for S3
This is faster than going back to nextcloud to download the files.

This is an opt-in setting that can be enabled by setting
use_presigned_url in the object store config.

Additionally add support for the proxy config which is needed in a
docker setup. See https://github.com/juliusknorr/nextcloud-docker-dev/pull/431

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-12-09 11:21:35 +01:00
Ferdinand Thiessen
d6d6747a73 refactor: apply rector rules for PHPUnit 10
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2025-10-27 21:56:04 +01:00
Carl Schwan
324b54b863 refactor(preview): Cleanup the implementation of the new preview backend
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Côme Chilliet
8107f614d8
fix(tests): Ignore expected warning for s3 tests
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2025-09-27 13:05:35 +02:00
Robin Appelman
0c1d5e8c54
fix: ensure all object store configuration have distict bucket names
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-09-05 18:45:32 +02:00
Robin Appelman
02f4a82088
fix: make bucket mapper work with new multi-object-store config
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-28 17:51:25 +02:00
Carl Schwan
5835599fa1 chore(rector): Apply current rector config
Signed-off-by: Carl Schwan <carl.schwan@nextclound.com>
2025-08-26 16:27:26 +02:00
Robin Appelman
b3c53c7436 feat: allow object store configuration aliases for easier migrations
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-08-14 13:06:51 +02:00
Robin Appelman
aa15f9d16d
chore: run rector
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-07-01 22:45:52 +02:00
Ferdinand Thiessen
5981b7eb51
chore: apply new CSFixer rules
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>

# Conflicts:
#	apps/settings/lib/SetupChecks/PhpOpcacheSetup.php
2025-07-01 16:26:50 +02:00
Robin Appelman
29e39c0a2e
chore: run rector on tests
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-06-12 18:31:58 +02:00
Robin Appelman
e21ce793a5 test: add test that object store folder copy preserves folder size
Signed-off-by: Robin Appelman <robin@icewind.nl>
2025-05-28 15:52:00 +02:00
Joas Schilling
5f9117b939
test: Fix coding standards
Signed-off-by: Joas Schilling <coding@schilljs.com>
2025-05-15 08:48:13 +02:00
Joas Schilling
5283e9a5e2
test: Cleanup tests/lib/Files/*
Signed-off-by: Joas Schilling <coding@schilljs.com>
2025-05-15 08:21:22 +02:00
Ferdinand Thiessen
b48ee2e924
fix: Harden files scanner for invalid null access
Co-authored-by: Ferdinand Thiessen <opensource@fthiessen.de>
Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2025-01-28 20:08:46 +01:00
provokateurin
9836e9b164
chore(deps): Update nextcloud/coding-standard to v1.3.1
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-19 14:21:20 +02:00
Robin Appelman
74adaebf96 test: fix incorrect ltrim usage in test
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-18 14:11:27 +02:00
Côme Chilliet
cfca7e7911
fix(tests): Fix most obvious errors in ObjectStore tests
Some are still failing

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2024-09-17 19:20:13 +02:00
Christoph Fiehe
9597072ada perf(ObjectStoreStorage): Improve (slow) move on same object bucket
This commit fixes the issue #47856. When you upload a file into a group folder and when you use a single S3 bucket as primary storage, the final move operation hangs for a long time. In the background, Nextcloud initiates a copy-delete sequence from the bucket into the bucket, with causes a lot unnecessary overhead. Nextcloud thinks that the file must be imported to another storage and does not recognize that everything is done on the same object bucket. In that case, the import step can be completely skipped, which saves time, network bandwidth and reduces the load on the object storage.

The behavior improves a lot with https://github.com/nextcloud/server/pull/46013. However, there are still some put messages that are being sent to the object storage when you use an object storage as primary storage and upload files into a group folder.

Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Signed-off-by: Christoph Fiehe <c.fiehe@eurodata.de>
2024-09-16 15:11:09 +02:00
Christoph Wurst
49dd79eabb
refactor: Add void return type to PHPUnit test methods
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-09-15 22:32:31 +02:00
Daniel Kesselberg
af6de04e9e
style: update codestyle for coding-standard 1.2.3
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2024-08-25 19:34:58 +02:00
Joas Schilling
e1caa198e3
ci: Skip flaky test on PHP 8.3
Signed-off-by: Joas Schilling <coding@schilljs.com>
2024-08-21 13:13:12 +02:00
Andy Scherzinger
1f7e2ba599
chore: Add SPDX header
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
2024-05-13 17:41:36 +02:00
Côme Chilliet
ec5133b739 fix: Apply new coding standard to all files
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2024-04-02 14:16:21 +02:00
Thomas Citharel
30798eb6c2
refactor(objectstorage): cleanup types
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2024-02-02 14:59:21 +01:00
Joas Schilling
5850695284
fix(tests): Add messages to assertions to explain what failed
Signed-off-by: Joas Schilling <coding@schilljs.com>
2024-01-26 20:46:14 +01:00
Ferdinand Thiessen
8e02014b55
chore(tests): Migrate object storage unit tests from drone to GitHub Actions
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2024-01-26 14:54:23 +01:00
Ferdinand Thiessen
fceb781058
fix(tests): Fix deprecation errors in S3 tests
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2024-01-26 14:53:33 +01:00
Ferdinand Thiessen
5172baaf8b
fix(ObjectStore): Make copying behavior consistent with local storage
Drop file permissions on copy like we do on local storage.

Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2023-11-17 12:54:37 +01:00
Robin Appelman
2ea41dab93
repair -1 folder sizes for object store background scan
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-05-10 19:33:26 +02:00
Côme Chilliet
f5c361cf44
composer run cs:fix
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-20 11:45:08 +01:00
Robin Appelman
3357c8e783 test disabling write check
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-10-26 18:45:59 +00:00
Robin Appelman
d8961ed10f
fix using FSEEK_END with SeekableHttpStream to get file size
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-09-15 17:23:16 +02:00
Robin Appelman
1374cbee3e store unencrypted size in the unencrypted_size column
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:54:26 +02:00
Carl Schwan
6312c0df69
Check style update
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-13 00:19:07 +01:00
John Molakvoæ
6ab2feaa54
Implement multibucket shift for ObjectStore
Signed-off-by: John Molakvoæ <skjnldsv@protonmail.com>
2021-11-11 15:09:28 +01:00
Bernd Rederlechner
3866f388b1
Refactor writeObject to only use MultipartUpload when required
Signed-off-by: Bernd Rederlechner <Bernd.Rederlechner@t-systems.com>

Co-authored-by: Julius Härtl <jus@bitgrid.net>
2021-08-20 17:02:25 +02:00
Julius Härtl
3f44fd00e2
Run s3 tests again
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-06-30 14:30:04 +02:00
Robin Appelman
effb7dc8ba
set mimetype for objects uploaded to object storages
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-04-21 15:25:58 +02:00
Robin Appelman
c87b1a50d9 apply object store copy optimization when 'cross storage' copy is within the same object store
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-09 20:31:27 +01:00
Robin Appelman
9d4848e863
use in objectstore copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-11-24 15:16:58 +01:00
Christoph Wurst
1584c9ae9c
Add visibility to all methods and position of static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:51:06 +02:00
Christoph Wurst
caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst
2a529e453a
Use a blank line after the opening tag
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 11:50:14 +02:00
Christoph Wurst
41b5e5923a
Use exactly one empty line after the namespace declaration
For PSR2

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 11:48:10 +02:00
Robin Appelman
3ba46f3b50
add basic tests for s3 seeking and add some error handling if reopen return the wrong range
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-01 15:23:05 +02:00
Christoph Wurst
463b388589
Merge pull request #20170 from nextcloud/techdebt/remove-unused-imports
Remove unused imports
2020-03-27 17:14:08 +01:00
Christoph Wurst
b80ebc9674
Use the short array syntax, everywhere
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-26 16:34:56 +01:00
Christoph Wurst
74936c49ea
Remove unused imports
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-25 22:08:08 +01:00
Roeland Jago Douma
c007ca624f
Make phpunit8 compatible
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-11-27 13:34:41 +01:00