upstream/nextcloud
1
0
Fork 0
mirror of https://github.com/nextcloud/server.git synced 2026-02-11 23:04:22 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions 13
60190 commits 687 branches 1182 tags 8.3 GiB
Commit graph

171 commits

Author SHA1 Message Date
Robin Appelman
09ffac5e6d
s3 external storage listing rework 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-10-15 15:35:02 +02:00
Bernd Rederlechner
3866f388b1
Refactor writeObject to only use MultipartUpload when required 
Signed-off-by: Bernd Rederlechner <Bernd.Rederlechner@t-systems.com>

Co-authored-by: Julius Härtl <jus@bitgrid.net>
 2021-08-20 17:02:25 +02:00
Julius Härtl
3e67637a4a
Merge pull request #26463 from Worteks/fix-s3proxy 2021-07-14 23:32:05 +02:00
blizzz
62675eb5c2
Merge pull request #27586 from nextcloud/bugfi/noid/objectstore-checksum 
Reset checksum when writing files to object store
 2021-07-01 17:30:54 +02:00
Maxime Besson
388a458cd5 fix(proxy): use accessor 
Signed-off-by: Maxime Besson <maxime.besson@worteks.com>
 2021-06-30 09:25:04 +02:00
Christoph Wurst
6d5cfe0c66
Move DateTime::RFC2822 to DateTimeInterface::2822 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2021-06-23 15:30:43 +02:00
Julius Härtl
21db463bc9
Reset checksum when writing files to object store 
Signed-off-by: Julius Härtl <jus@bitgrid.net>
 2021-06-21 12:04:49 +02:00
John Molakvoæ (skjnldsv)
215aef3cbd
Update php licenses 
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
 2021-06-04 22:02:41 +02:00
Morris Jobke
333665b43d
Merge pull request #27040 from nextcloud/FlorentCoppint-master 
Avoid reading ~/.aws/config when using S3 provider
 2021-05-25 10:19:22 +02:00
Florent
d4444f2472
Avoid reading ~/.aws/config when using S3 provider 
When using S3 storage, we sometimes have open_basedir errors due to aws-sdk-php trying to read ~/.aws/config which is out of open_basedir restrictions.
Christoph Wurst already added csm=false config in #21406 but it wasn't enough, we also need to set use_arn_region=false, added in this commit.

Signed-off-by: Florent <florent@coppint.com>
 2021-05-20 12:21:15 +02:00
Robin Appelman
ef6f2e68f0
explicitly close source stream on object store upload even if count wrapper isn't needed 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-04-29 17:01:19 +02:00
Robin Appelman
effb7dc8ba
set mimetype for objects uploaded to object storages 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-04-21 15:25:58 +02:00
Maxime Besson
547438527d fix(proxy): reaching s3 storage behind some http proxy 
Signed-off-by: Maxime Besson <maxime.besson@worteks.com>
 2021-04-09 10:15:57 +02:00
Samuel
03fe74b95e fix(proxy): reaching s3 storage behind some http proxy 
Signed-off-by: Maxime Besson <maxime.besson@worteks.com>
 2021-04-09 10:15:57 +02:00
Robin Appelman
ad636ae12d
use returned file id for object store copy 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-03-12 17:22:40 +01:00
Robin Appelman
32551b9ff7
Merge pull request #25722 from nextcloud/objectstore-copy-cross 
apply object store copy optimization when 'cross storage' copy is wit…
 2021-03-12 16:10:18 +00:00
Robin Appelman
3eb764e65d
remove explicit fclose from S3->writeStream 
streams get closed automatically when dropped, and in some cases the stream seems to be already closed by the S3 library, in which case trying to close it again will raise an error

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-03-11 18:05:04 +01:00
Robin Appelman
c87b1a50d9 apply object store copy optimization when 'cross storage' copy is within the same object store 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2021-03-09 20:31:27 +01:00
dependabot-preview[bot]
eb502c02ff
Bump nextcloud/coding-standard from 0.3.0 to 0.5.0 
Bumps [nextcloud/coding-standard](https://github.com/nextcloud/coding-standard) from 0.3.0 to 0.5.0.
- [Release notes](https://github.com/nextcloud/coding-standard/releases)
- [Changelog](https://github.com/nextcloud/coding-standard/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nextcloud/coding-standard/compare/v0.3.0...v0.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2021-02-18 13:31:24 +01:00
Roeland Jago Douma
137636b651
Merge pull request #24700 from Imajie/s3-creds 
Resolves #24699, Support ES2 and ECS instance providers for S3 buckets
 2021-02-02 21:48:22 +01:00
Christoph Wurst
9ce3ea3368
Update license headers 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-12-30 14:07:05 +01:00
Christoph Wurst
73c7d0dc81
Bump icewind/streams from 0.7.1 to 0.7.2 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-12-29 09:07:36 +01:00
Julius Härtl
5094e29ebd
Merge pull request #24594 from kofemann/dcache 2020-12-22 09:26:07 +01:00
Tigran Mkrtchyan
4f2dc18f58 storage: update IStorage#file_put_contents docs to match usage 
The current phpdoc of IStorage#file_put_contents doesnt corresponds to
it's actual usage in code, e.g.

Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
 2020-12-21 19:04:18 +01:00
Christoph Wurst
d89a75be0b
Update all license headers for Nextcloud 21 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-12-16 18:48:22 +01:00
James Letendre
45a02ee30c Fix failing tests 
Signed-off-by: James Letendre <james.letendre@gmail.com>
 2020-12-15 19:06:06 -05:00
James Letendre
ad95e51cbe Add web identity provider to S3 connection chain 
Signed-off-by: James Letendre <james.letendre@gmail.com>
 2020-12-15 16:37:46 -05:00
James Letendre
5b756a9fb0 Replace defaultProvider with explicit calls to exclude user home directory lookup 
Signed-off-by: James Letendre <james.letendre@gmail.com>
 2020-12-14 16:22:41 -05:00
James Letendre
85aa77539b Resolves #24699, Support ES2 and ECS instance providers for S3 buckets 
Signed-off-by: James Letendre <james.letendre@gmail.com>
 2020-12-14 12:55:46 -05:00
Robin Appelman
9d4848e863
use in objectstore copy 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2020-11-24 15:16:58 +01:00
Roeland Jago Douma
65141d4864
Allow config to specify the bucket exists 
In the 99% case the bucket is just always there. And if it is not the
read/write will fail hard anyways. Esp on big instances the Objectstore
is not always fast and this can save a few hundered ms of each request
that acess the objectstore.

In short it is adding

'verify_bucket_exists' => false

To the S3 config part

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-11-06 11:08:38 +01:00
Christoph Wurst
d9015a8c94
Format code to a single space around binary operators 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-10-05 20:25:24 +02:00
Roeland Jago Douma
bb06b6cce4
Fix reading empty files from objectstorage 
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-09-09 20:45:13 +02:00
Roeland Jago Douma
9fdeed8cc0
Run stat less often for objectstorages 
When we want to get the permissions we now do stat at least 5 times for
each entry. Which is a bit much. Especially since the permssions are all
just in the database already.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-09-08 14:38:36 +02:00
Roeland Jago Douma
789b33aba4
Only update the filecache entry once the file has been written to S3 
If we already update before we have no way to revert if the upload
fails.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-08-31 12:28:04 +02:00
Roeland Jago Douma
6ffd7173f9
Don't lose filecache entry on s3 overwrite error 
If the object store errors we should not always delete the filecache
entry. As this might lead to people losing access to their files.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-08-31 12:25:20 +02:00
Christoph Wurst
2a054e6c04
Update the license headers for Nextcloud 20 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-08-24 14:54:25 +02:00
Stephen Cuppett
5ef0f86ce7
Resolves #19790, Provides Support for IAM Credentials 
Includes support for either leveraging environment variables
passed to the PHP runtime or IAM instance profile present
on the host being used. The default and first choice is
still the parameter file as documented.

See also: https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_provider.html#chaining-providers

Signed-off-by: Stephen Cuppett <steve@cuppett.com>
 2020-08-20 15:54:33 +02:00
Morris Jobke
fedf9c69d9
Use matching parameter names form interfaces and implementations 
Found by Psalm 3.14.1

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2020-08-19 18:16:35 +02:00
Morris Jobke
438ac23e2a
Distribute preview folders in appdata in multibucket setup to multiple buckets 
* introduces a new IRootMountProvider to register mount points inside the root storage
* adds a AppdataPreviewObjectStoreStorage to handle the split between preview folders and bucket number

Ref #22033

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2020-08-06 22:19:20 +02:00
Christoph Wurst
d3a4bc91f7
Fix static method call for s3 bucket compat check 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-07-03 09:02:58 +02:00
Daniel Kesselberg
8aa99aa8ff
Update presign method to match with interface again. 
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
 2020-06-25 22:56:28 +02:00
Christoph Wurst
9d392891be
Disable Client-Side Monitoring on AWS storage 
The S3 client enables this by default and then tries to read
`.aws/config`. This causes `open_basedir` restriction related error for
some setups. So this patch disables the CSM because it's most likely
unused anyway.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-06-15 09:13:08 +02:00
Florent
3594ba6971 Upload part size as S3 parameter instead of constant value 
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize

Signed-off-by: Florent <florent@coppint.com>
 2020-06-09 09:18:42 +02:00
Roeland Jago Douma
d5850eb28f
Merge pull request #18955 from adrb/swift_upload_large_objects 
Large Object support for OpenStack Swift
 2020-05-06 13:03:24 +02:00
Robin Appelman
4e6d3d1880
Fix getDirectoryContent implementation for Jail wrapper 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2020-04-30 15:27:09 +02:00
Roeland Jago Douma
f6b435d0d9
Run the cs fixer for green ci 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-04-30 10:28:44 +02:00
Roeland Jago Douma
a1c1b354fc
Merge pull request #18883 from adrb/optimize_swift_14116 
Optimize Openstack Swift files download
 2020-04-30 09:06:19 +02:00
Christoph Wurst
cb057829f7
Update license headers for 19 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-04-29 11:57:22 +02:00
Adrian Brzezinski
e107519295 Large Object support for OpenStack Swift. 
Until now, you wouldn't be able to create
objects larger that 5GB.

It's somewhat related with pull #18883

Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
 2020-04-27 14:23:59 +02:00