upstream/nextcloud
1
0
Fork 0
mirror of https://github.com/nextcloud/server.git synced 2026-02-03 20:41:22 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions 25
85577 commits 651 branches 1179 tags 8.1 GiB
Commit graph

63 commits

Author SHA1 Message Date
Carl Schwan
2ea1bd4cdc
feat(movie-preview): Use getDirectDownloadById for generating preview 
Allow to speed-up considerably the creation of previews for movies
stored on S3.

Signed-off-by: Carl Schwan <carlschwan@kde.org>
 2026-01-02 17:13:41 +01:00
Carl Schwan
b6313f68d3
perf(s3): Expose pre-signed urls for S3 
This is faster than going back to nextcloud to download the files.

This is an opt-in setting that can be enabled by setting
use_presigned_url in the object store config.

Additionally add support for the proxy config which is needed in a
docker setup. See https://github.com/juliusknorr/nextcloud-docker-dev/pull/431

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
 2025-12-09 11:21:35 +01:00
Robin Appelman
8194be795d
fix: encode s3 metadata as base64 if needed 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-11-18 16:06:57 +01:00
Louis Chmn
00ec57efd6 fix(S3): Only append streams if non-seekable 
Later, when we rewind the stream in `writeMultiPart` during retry, both streams were rewinded, so the resulting stream was bigger than expected.

Inspired by https://github.com/aws/aws-sdk-php/blob/master/src/S3/ObjectUploader.php#L136-L146

Signed-off-by: Louis Chmn <louis@chmn.me>
 2025-10-10 09:54:05 +02:00
Robin Appelman
398b106f0c fix: validate written size for s3 multipart uploads 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-08-05 19:17:58 +02:00
Kent Delante
06b5ce5859 fix(s3): retry failed multipart uploads with decreased concurrency 
Signed-off-by: Kent Delante <kent.delante@proton.me>
 2025-06-10 17:39:59 +08:00
Robin Appelman
8aae332622
feat: store original storage id and path in object store metadata 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-03-31 15:10:58 +02:00
Robin Appelman
94114b99f7
feat: more generic way of passing metadata to object storage backends for new objects 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2025-03-31 14:55:18 +02:00
Robin Appelman
ca547bb913
fix: don't perform the extra buffering in s3 stream write when the stream size is known 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-12-05 14:07:40 +01:00
Robin Appelman
7da8951c57
fix: use php://temp instead of php://memory for multi-part upload buffer 
this should reduce potential memory issues if the limit is set very high

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-12-05 14:07:37 +01:00
Daniel Kesselberg
af6de04e9e
style: update codestyle for coding-standard 1.2.3 
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
 2024-08-25 19:34:58 +02:00
Andy Scherzinger
dae7c159f7
chore: Add SPDX header 
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
 2024-05-24 13:11:22 +02:00
Côme Chilliet
ec5133b739 fix: Apply new coding standard to all files 
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
 2024-04-02 14:16:21 +02:00
Ferdinand Thiessen
32dee2f84a
feat(ObjectStore): Make S3 MultipartUpload concurrency configurable 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-03-21 16:25:49 +01:00
Ferdinand Thiessen
69c5e8ebde
fix(ObjectStore): Use common S3 trait so config and connection trait members are in sync 
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
 2024-03-21 16:25:42 +01:00
Robin Appelman
c54f7bd761 fix: disable the internal s3sdk multi part copy logic 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2024-03-06 15:32:14 +01:00
Julius Härtl
e4054370b1
fix(s3): Add config option to disable multipart copy for certain s3 providers 
Signed-off-by: Julius Härtl <jus@bitgrid.net>
 2023-12-28 15:31:38 +01:00
Robin Appelman
1043c21b35
only do a multipart s3 copy when above the regular copy limit 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-12-28 15:31:37 +01:00
Tobias Zimmerer
cdf96fab92 fix(s3): support SSE-C headers for the MultipartCopy call 
Signed-off-by: Tobias Zimmerer <3228193+ir0nhide@users.noreply.github.com>
 2023-11-14 21:19:59 +01:00
Robin Appelman
eab8dd3ca0 use multipart copy for s3 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-09-20 17:28:48 +02:00
Robin Appelman
ea88ec1350
add utility command for object store objects 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2023-06-09 18:08:18 +02:00
Julius Härtl
159a0c8411
feat(s3): Add option to specify an SSE-C customer provided key 
Signed-off-by: Julius Härtl <jus@bitgrid.net>
 2023-01-24 10:36:03 +01:00
François Ménabé
cf9bdc0274 Manage s3 storage class in objectstore 
Signed-off-by: François Ménabé <francois.menabe@gmail.com>
 2023-01-12 14:58:27 +01:00
Maxence Lange
cc639a8877 compare cached filesize on download 
Signed-off-by: Maxence Lange <maxence@artificial-owl.com>
 2022-12-01 15:42:15 -01:00
Julius Härtl
214a50d071
Avoid allocating too much memory for the buffer 
Signed-off-by: Julius Härtl <jus@bitgrid.net>
 2022-10-11 15:34:27 +02:00
Robin Appelman
61fdd91dcf
don't set null as a bundle path 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2022-07-27 12:58:07 +02:00
Robin Appelman
f1486890d7
only use nextcloud bundle when explicitly enabled 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2022-07-05 16:06:55 +02:00
Robin Appelman
bffa67c48b
also use nextcloud certificate bundle when downloading from s3 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2022-07-05 15:57:54 +02:00
Louis Chemineau
bc2c02f451 Wrap S3 multipart upload exception 
Signed-off-by: Louis Chemineau <louis@chmn.me>
 2022-02-16 14:56:14 +01:00
Louis Chemineau
0b768c17d2 Wrap S3 multipart upload exception 
Signed-off-by: Louis Chemineau <louis@chmn.me>
 2022-02-15 17:35:12 +01:00
Julius Härtl
2aac757805
Make max size for single put uploads configurable 
Signed-off-by: Julius Härtl <jus@bitgrid.net>
 2021-12-29 13:30:22 +01:00
Robin Appelman
09ffac5e6d
s3 external storage listing rework 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-10-15 15:35:02 +02:00
Bernd Rederlechner
3866f388b1
Refactor writeObject to only use MultipartUpload when required 
Signed-off-by: Bernd Rederlechner <Bernd.Rederlechner@t-systems.com>

Co-authored-by: Julius Härtl <jus@bitgrid.net>
 2021-08-20 17:02:25 +02:00
Julius Härtl
3e67637a4a
Merge pull request #26463 from Worteks/fix-s3proxy 2021-07-14 23:32:05 +02:00
John Molakvoæ (skjnldsv)
215aef3cbd
Update php licenses 
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
 2021-06-04 22:02:41 +02:00
Robin Appelman
effb7dc8ba
set mimetype for objects uploaded to object storages 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-04-21 15:25:58 +02:00
Maxime Besson
547438527d fix(proxy): reaching s3 storage behind some http proxy 
Signed-off-by: Maxime Besson <maxime.besson@worteks.com>
 2021-04-09 10:15:57 +02:00
Samuel
03fe74b95e fix(proxy): reaching s3 storage behind some http proxy 
Signed-off-by: Maxime Besson <maxime.besson@worteks.com>
 2021-04-09 10:15:57 +02:00
Robin Appelman
3eb764e65d
remove explicit fclose from S3->writeStream 
streams get closed automatically when dropped, and in some cases the stream seems to be already closed by the S3 library, in which case trying to close it again will raise an error

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-03-11 18:05:04 +01:00
Robin Appelman
9d4848e863
use in objectstore copy 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2020-11-24 15:16:58 +01:00
Christoph Wurst
2a054e6c04
Update the license headers for Nextcloud 20 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-08-24 14:54:25 +02:00
Florent
3594ba6971 Upload part size as S3 parameter instead of constant value 
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize

Signed-off-by: Florent <florent@coppint.com>
 2020-06-09 09:18:42 +02:00
Christoph Wurst
cb057829f7
Update license headers for 19 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-04-29 11:57:22 +02:00
Christoph Wurst
1584c9ae9c
Add visibility to all methods and position of static keyword 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-04-10 16:51:06 +02:00
Robin Appelman
7b07e7251c
make seekable s3 stream generic 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2020-04-01 15:21:05 +02:00
Lukas Stabe
14401efb0f
Enable fseek for files in S3 storage 
Signed-off-by: Lukas Stabe <lukas@stabe.de>
 2020-04-01 15:21:01 +02:00
Christoph Wurst
5bf3d1bb38
Update license headers 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2019-12-05 15:38:45 +01:00
Morris Jobke
e45fb5fa3e
Fix typo in comment 
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2019-07-29 14:55:43 +02:00
Roeland Jago Douma
e953205908
Use HTTP1.1 to read S3 objects 
Some of the READs otherwise use HTTP/1.0 which is not always supported
by all backends. HTTP/1.1 is there since 1999 way longer than S3 so safe
to assume it is always there IMO.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-07-10 11:42:22 +02:00
Roeland Jago Douma
61d66d7954
Fix empty file uploads to S3 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-02-15 08:41:39 +01:00