upstream/nextcloud
1
0
Fork 0
mirror of https://github.com/nextcloud/server.git synced 2026-03-03 05:51:07 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions 10
53352 commits 698 branches 1186 tags 6.5 GiB
Commit graph

114 commits

Author SHA1 Message Date
Roeland Jago Douma
af6b57cd0d Fix reading empty files from objectstorage 
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-09-09 19:47:30 +00:00
Roeland Jago Douma
55ce30ad53 Only update the filecache entry once the file has been written to S3 
If we already update before we have no way to revert if the upload
fails.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-08-31 17:08:35 +00:00
Roeland Jago Douma
f909d2a39a Don't lose filecache entry on s3 overwrite error 
If the object store errors we should not always delete the filecache
entry. As this might lead to people losing access to their files.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-08-31 17:08:34 +00:00
Christoph Wurst
5fdcb5a1e3 Disable Client-Side Monitoring on AWS storage 
The S3 client enables this by default and then tries to read
`.aws/config`. This causes `open_basedir` restriction related error for
some setups. So this patch disables the CSM because it's most likely
unused anyway.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2020-06-17 09:02:20 +00:00
Morris Jobke
1d816add44
Merge pull request #20965 from nextcloud/backport/20033/stable18 
[stable18] Enable fseek for files in S3 storage
 2020-05-25 13:38:04 +02:00
Robin Appelman
8434d0af9f
make seekable s3 stream generic 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2020-05-20 22:57:36 +02:00
Lukas Stabe
7a62514c31
Enable fseek for files in S3 storage 
Signed-off-by: Lukas Stabe <lukas@stabe.de>
 2020-05-20 22:57:30 +02:00
Julien Lutran
1a990b9487 Fix issues with Keystone auth v3 in files_external app 
Signed-off-by: Julien Lutran <julien.lutran@corp.ovh.com>
 2020-05-15 15:19:36 +00:00
Roeland Jago Douma
ebba16a219 Get correct mimetype on objectstores 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2020-03-12 18:59:58 +00:00
Christoph Wurst
5bf3d1bb38
Update license headers 
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
 2019-12-05 15:38:45 +01:00
Roeland Jago Douma
68748d4f85
Some php-cs fixes 
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-11-22 20:52:10 +01:00
Robin Appelman
733d4b6cca
dont delete cache entries if deleting an object from object store failed 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-09-25 18:09:45 +02:00
Morris Jobke
e45fb5fa3e
Fix typo in comment 
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2019-07-29 14:55:43 +02:00
Marcel Klehr
d46744e2f1 Fix File#putContents(string) on ObjectStorage 
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
 2019-07-17 14:58:56 +02:00
Roeland Jago Douma
e953205908
Use HTTP1.1 to read S3 objects 
Some of the READs otherwise use HTTP/1.0 which is not always supported
by all backends. HTTP/1.1 is there since 1999 way longer than S3 so safe
to assume it is always there IMO.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-07-10 11:42:22 +02:00
Morris Jobke
96e892770d
Merge pull request #15715 from nextcloud/s3-bucket-name 
Allow other code to use the s3 client from object store
 2019-05-24 15:03:22 +02:00
Robin Appelman
834db3af2c
add method to get bucket name from s3 storage 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-05-24 14:02:03 +02:00
Robin Appelman
06a119242e
make ObjectStoreStorage::getURN public 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-05-22 12:17:33 +02:00
Robin Appelman
439b341e89
add method to get the used object store from the storage 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-05-21 17:18:00 +02:00
Roeland Jago Douma
dedb84acb0
Always use a temp file for swift writes 
Apparently the if statement doesn't work in all cases (even if I could
not reproduce it). So for the time being we will just not directly
stream to swift.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-04-18 21:04:27 +02:00
Roeland Jago Douma
1b27e9578d
Merge pull request #14424 from nextcloud/fix/13554/swift_to_tmp 
Use a tmp file for swift writes
 2019-03-06 15:33:15 +01:00
Morris Jobke
debd32b461
Merge pull request #14175 from skydiablo/master 
Fix ObjectStorage expired token (v3)
 2019-03-05 18:59:45 +01:00
Roeland Jago Douma
1153123b90
Use a tmp file for swift writes 
Else this leads to a seekable stream error with chunked uploads

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-03-04 13:11:59 +01:00
Roeland Jago Douma
df55781814
Merge pull request #14254 from sparrowjack63/master 
Dont check Bucket Name in Nextcloud
 2019-03-01 13:46:04 +01:00
S. Cat
e3142a91b2 Bucket name - Alert in debug mode 
Alert in debug mode only when trying to mount non compatible DNS bucket name (in order to not flood the logs)

Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
 2019-03-01 11:28:54 +01:00
S. Cat
381ae44d38 Correct code indent 
Just correct code indent.

Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
 2019-03-01 11:28:54 +01:00
S. Cat
77473b0228 Warning on S3 not compatible DNS bucket name 
The new created bucket should respect the DNS compatibility, nevertheless, Nextcloud should accept to mount "old created" buckets that does not respect DNS compatibility (Backward compatibility, or compatibility with CEPH).

Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
 2019-03-01 11:28:54 +01:00
Sebastien Cat
59c3d8d85f Dont check Bucket Name in Nextcloud 
Signed-off-by: Sebastien Cat <sparrow.jack63@gmail.com>
Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
 2019-03-01 11:28:54 +01:00
Roeland Jago Douma
f6f002e2ad
Merge pull request #14210 from nextcloud/fix/14192/fix_empty_uploads 
Fix empty file uploads to S3 (and other streaming storages)
 2019-02-18 15:39:17 +01:00
Roeland Jago Douma
104616c1da
Use CountWrapper instead of almost identical CountReadStream 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-02-15 13:24:58 +01:00
Roeland Jago Douma
61d66d7954
Fix empty file uploads to S3 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-02-15 08:41:39 +01:00
Volker
d06c25b9c8
Update SwiftFactory.php 
Auth-Service (in explizit v3) will recheck the cached-token and will end in an "token expired exception".
 2019-02-13 12:26:27 +01:00
William Pain
c26bdd4731 Fix: Check if $this->params['user'] is an array 
Signed-off-by: William Pain <pain.william@gmail.com>
 2019-02-05 09:56:42 +01:00
Robin Appelman
1d322d3b37
always use multipart uploader for s3 uploads 
the multipart uploader handles non seekable streams while `upload` does not

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-01-28 14:51:19 +01:00
Morris Jobke
5a27e54f4b
Merge pull request #13032 from nextcloud/objectstore-write-exists 
upload new files in objectstore to a .part path first
 2019-01-14 11:26:29 +01:00
Robin Appelman
615fb8cd77
Cache tokens when using swift's v2 authentication 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-01-13 19:59:48 +01:00
Robin Appelman
d6bf5d4384
upload new files in objectstore to a .part path first 
This prevent the object store and cache from getting out of sync
when an objectstore silently fails or the php process get's killed
during the upload without giving us the chance to cleanup

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-12-12 15:24:40 +01:00
Robin Appelman
6c9f2644cf
Add objectExists to objectstore interface 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-12-12 14:17:19 +01:00
Morris Jobke
cef8eadf2a
Add PHPDoc for thrown exception 
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2018-11-19 15:34:07 +01:00
Robin Appelman
35251928d5
forward object not found error in switch as dav 404 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-11-19 11:34:38 +01:00
Robin Appelman
93de63777e
extend storage api to allow directly writing a stream to storage 
this removes the need for temporary storages with some external storage backends.
The new method is added to a separate interface to maintain compatibility with
storage backends implementing the storage interface directly (without inheriting common)

Currently the interface is implemented for objectstorage based storages and local storage
and used by webdav uploads

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-10-31 21:10:44 +01:00
Robin Appelman
47fd3499a3
Fix listing s3 objects when using google cloud aws compatibility 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-06-11 17:10:57 +02:00
Robin Appelman
ac26175a17
add azure unit tests with azurite 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-06-04 16:18:03 +02:00
Robin Appelman
60b7cbc0f0
add azure object storage backend 
Signed-off-by: Robin Appelman <robin@icewind.nl>
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-06-04 10:57:52 +02:00
Roeland Jago Douma
a0e038f912
generateTokenFromCache is only valid for V3 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-04-26 20:51:05 +02:00
Roeland Jago Douma
7318882357
json_decode an associative array for the swift cached token 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-04-26 20:47:47 +02:00
Arthur Schiwon
38a90130ce
move log constants to ILogger 
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
 2018-04-26 10:45:52 +02:00
Roeland Jago Douma
1ceb081c9b
Merge pull request #9280 from nextcloud/bugfix/noid/swift-v3-scope 
Swift v3 requires setting a scope
 2018-04-24 09:49:27 +02:00
Roeland Jago Douma
ba71918ad2
Swift v3 requires setting a scope 
Unscoped auth is not guaranteed to return a catalog. So require the
scope to be set properly.

See https://developer.openstack.org/api-ref/identity/v3/index.html#password-authentication-with-unscoped-authorization

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-04-23 19:34:09 +02:00
Morris Jobke
431ccb6386
Fix undefined variable - found by phan 
Added in https://github.com/nextcloud/server/pull/8314/files#diff-a55c10804586cd797b7f3f16556c1457R157

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2018-04-23 11:34:41 +02:00