Commit graph

964 commits

Author SHA1 Message Date
Joas Schilling
b6e8aba4c8 Don't set up full filesystem to check for certificates
Signed-off-by: Joas Schilling <coding@schilljs.com>
2022-03-02 10:46:25 +00:00
Vincent Petry
64f376e4b9
Merge pull request #31098 from nextcloud/backport/30942/stable23
[stable23] only setup part of the filesystem for appdata requests
2022-03-01 16:00:45 +01:00
Louis Chemineau
5043a9dc6b Wrap S3 multipart upload exception
Signed-off-by: Louis Chemineau <louis@chmn.me>
2022-02-21 14:25:54 +00:00
Louis Chemineau
7a2697a100 Wrap S3 multipart upload exception
Signed-off-by: Louis Chemineau <louis@chmn.me>
2022-02-21 14:25:52 +00:00
Robin Appelman
417ead0a6b fix fs cache test user setup
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-02-16 10:46:48 +00:00
Robin Appelman
fa43e4ecf2 pass user object during fs init
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-02-16 10:46:48 +00:00
Robin Appelman
19d214294c only setup part of the filesystem for appdata requests
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-02-16 10:46:48 +00:00
Carl Schwan
d69531ac8e
Fix psalm not running
The issue was that we were using psalm/phar instead of vimeo/psalm. This
caused issue with the custom psalm plugin in buildd/psalm.

This is using the opportunity to also update the psalm version from 3.8
to 3.17 and the php-cs-fixer too.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-27 17:20:00 +01:00
John Molakvoæ
2645b6cd69
Merge pull request #29798 from nextcloud/stable23-disk 2022-01-20 10:36:40 +01:00
Carl Schwan
253d502471
Merge pull request #30684 from nextcloud/backport/30531/stable23
[stable23] Optimize FileSystemTags workflow for groupfolder
2022-01-17 10:15:26 +01:00
Jonas Meurer
9da4620ff8 updateEncryptedVersion: cleanup on target if cache already got renamed
When moving a file to trash with encryption enabled, the cache gets
moved before the actual file. According to @icewind1991 this is in order
to not break object storage.

When moving a file from an unencrypted storage (e.g. a collectives
storage) to the encrypted trashbin storage, this causes errors, see

This commit fixes it by doing `updateEncryptedVersion()` on the target
cache entry *if* the source cache entry doesn't exist anymore, but the
corresponding target cache entry does exist already.

Fixes: #26544

Signed-off-by: Jonas Meurer <jonas@freesources.org>
2022-01-14 21:41:10 +00:00
Carl Schwan
61624c2213 Fix psalm issues
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-14 14:17:11 +00:00
Carl Schwan
c3fcb2c199 Add helper method in Wrapper
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-14 14:17:06 +00:00
John Molakvoæ
05a13d3dc6
Implement multibucket shift for ObjectStore
Signed-off-by: John Molakvoæ <skjnldsv@protonmail.com>
2022-01-14 08:31:38 +01:00
Vincent Petry
9eeaed0051 Fix fail when keys/files folder already exists
Fixes an issue with transfer ownership in move mode where the folder
"files_encryption/keys/files" already exists.

Instead of failing, its existence is checked before calling mkdir.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
Signed-off-by: nextcloud-command <nextcloud-command@users.noreply.github.com>
2022-01-11 08:20:41 +00:00
Vincent Petry
1291abc130 Check resource before closing in encryption wrapper
In case of error there is no guarantee that $source or $target is set or
is a resource when handling an error.

Without this fix, there's a risk that fclose will fail and the actual
exception will not be thrown, making it impossible to find out about the
root cause.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-12-10 00:03:08 +00:00
Robin Appelman
c9db02a921 background scan the source storage when a background scan on a storage jail is triggered
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-12-02 17:47:05 +00:00
Joas Schilling
4736bba7f6 Check for invalid characters before trimming
Signed-off-by: Joas Schilling <coding@schilljs.com>
2021-11-25 11:49:54 +00:00
acsfer
913e64c81a
[stable23] Check for disk_free_space
Backport of https://github.com/nextcloud/server/pull/29758
2021-11-19 20:03:34 +01:00
Vincent Petry
f0661de4e5 Normalize directory entries in Encoding wrapper
Directory entry file names are now normalized in getMetaData(),
getDirectoryContents() and opendir().

This makes the scanner work properly as it assumes pre-normalized names.

In case the names were not normalized, the scanner will now skip the
entries and display a warning when applicable.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-19 14:51:38 +00:00
Vincent Petry
516b10de33 Move storage encoding compatibility warning logic
The encoding check for file names is now happening the Scanner, and an
event will be emitted only if the storage doesn't contain the encoding
compatibility wrapper.

The event is listened to by the occ scan command to be able to display a
warning in case of file name mismatches when they have NFD encoding.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-19 14:51:38 +00:00
Vincent Petry
106950d493 Normalize file name before existence check in scanner
The scanner would not find a NFD-encoded file name in an
existing file list that is normalized.

This normalizes the file name before scanning.

Fixes issues where scanning repeatedly would make NFD files flicker in
and out of existence in the file cache.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-19 14:51:38 +00:00
Vincent Petry
93fb33d863
Update lib/private/Files/Storage/Wrapper/Availability.php add void
Signed-off-by: Vincent Petry <vincent@nextcloud.com>

Co-authored-by: Carl Schwan <carl@carlschwan.eu>
2021-11-05 10:28:41 +01:00
Vincent Petry
db29fd29ee
Return false in hasUpdated when storage is not available
Technically, saying that a storage has no updates when it's not
available is correct.

This makes it possible to retrieve the cache entry for the mount point
and also to list and remove unavailable federated shares.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-05 09:48:52 +01:00
John Molakvoæ
d6152bf98d
Fix psalm
Signed-off-by: John Molakvoæ <skjnldsv@protonmail.com>
2021-11-04 10:24:13 +01:00
John Molakvoæ
44c06a9641
Merge pull request #29502 from nextcloud/revert-27378-dav-recalc-checksum 2021-11-01 13:50:52 +01:00
Robin Appelman
118c647f61
properly handle cases where cache wrappers block access
`CacheWrapper::formatCacheEntry` can return false for files that should be filtered out

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-11-01 11:48:01 +01:00
John Molakvoæ
63d3931e80
Merge pull request #29281 from vijfhoek/master 2021-11-01 11:29:37 +01:00
John Molakvoæ
d6a26b2536
Revert "add dav plugin to trigger recalculating of checksums" 2021-11-01 11:26:17 +01:00
John Molakvoæ
e1bf5c1cd3
Merge pull request #27378 from nextcloud/dav-recalc-checksum 2021-11-01 11:25:21 +01:00
Carl Schwan
df4e6bab69
Merge pull request #29115 from nextcloud/work/carl/correct-permissions-when-copying
Fix permissions when copying from ObjectStorage
2021-10-28 14:38:20 +02:00
Carl Schwan
bfa60aaf27
Fix permissions when copying from ObjectStorage
Make sure that when a user copy a file from a directory they don't have
all permissions to a directory where they have more permissions, the
permissions are correctly set to the one from the parent taget folder.

This was caused by the ObjectStoreStorage::copyFromStorage using
the jailed storage and cache entry instead of the unjailed one like other
storages (the local one).
Steps to reproduce

+ Use object storage
+ Create a groupfolder with one group having full permission and another one
  who can just read files.
+ With an user who is in the second group, copy a file from the groupfolder to
  the home folder of this user.
+ The file in the home folder of the user will be read only and can't be deleted
  even though it is in their home folder and they are the owner. In oc_filecache,
  the permissions stored for this file are 1 (READ)

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-10-28 13:29:50 +02:00
alanmeeson
16f70e8647 Add type hint to fread_block $blockSize param & apply cs:fix.
Signed-off-by: alanmeeson <alan@carefullycalculated.co.uk>
2021-10-23 15:11:23 +01:00
Alan Meeson
44c332a46e Fix truncation of files upon read when using object store and encryption.
When using and object store as primary storage and using the default
encryption module at the same time,  any encrypted file would be truncated
when read, and a text error message added to the end.

This was caused by a combination of the reliance of the read functions on
on knowing the unencrypted file size,  and a bug in the function which
calculated the unencrypted file size for a given file.

In order to calculate the unencrypted file size,  the function would first
skip the header block, then use fseek to skip to the last encrypted block
in the file.  Because there was a corresponence between the encrypted and
unencrypted blocks, this would also be the last encrypted block.  It would
then read the final block and decrypt it to get the unencrypted length of
the last block.  With that, the number of blocks, and the unencrypted block
size, it could calculate the unencrypted file size.

The trouble was that when using an object store, an fread call doesn't
always get you the number of bytes you asked for, even if they are
available.  To resolve this I adapted the stream_read_block function from
lib/private/Files/Streams/Encryption.php to work here.  This function
wraps the fread call in a loop and repeats until it has the entire set of
bytes that were requested,  or there are no more to get.

This fixes the imediate bug, and should (with luck) allow people to get
their encrypted files out of Nextcloud now.  (The problem was purely on
the decryption side).  In the future it would be nice to do some
refactoring here.

I have tested this with image files ranging from 1kb to 10mb using
Nextcloud version 22.1.0 (the nextcloud:22.1-apache docker image), with
sqlite and a Linode object store as the primary storage.

Signed-off-by: Alan Meeson <alan@carefullycalculated.co.uk>
2021-10-23 15:11:23 +01:00
John Molakvoæ
f4e4a85fcf
Merge pull request #27440 from nextcloud/is-file-handle 2021-10-23 11:18:52 +02:00
Julius Härtl
6b099ecfe5
Merge pull request #25392 from nextcloud/imountpoint-ocp-storage 2021-10-22 20:14:22 +02:00
John Molakvoæ
2be0eda47c
Merge pull request #29220 from nextcloud/s3-external-list 2021-10-22 12:06:05 +02:00
John Molakvoæ
e673ec0b87
Merge branch 'master' into imountpoint-ocp-storage 2021-10-22 11:47:37 +02:00
John Molakvoæ
0a35937662
Merge pull request #26347 from J0WI/clean-path-regex 2021-10-22 11:42:30 +02:00
Carl Schwan
9408f8ae69
Fix security issues when copying groupfolder with advanced ACL
Using advanced ACL, it is possible that an user has access to a
directory but not to a subdirectory, so the copying use
Common::copyFromStorage instead of Local::copyFromStorage.

Fix https://github.com/nextcloud/groupfolders/issues/1692

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-10-21 17:11:52 +02:00
John Molakvoæ
a1c8b6e9eb
Merge pull request #28768 from nextcloud/check-file-exists 2021-10-18 09:54:22 +02:00
Sijmen Schoon
34600c78bb
Move query outside the loop and reduce chunk size to 1000
This involved changing CacheQueryBuilder\whereParentIn to take a
parameter name, renaming the function accordingly.

Signed-off-by: Sijmen Schoon <me@sijmenschoon.nl>
2021-10-17 20:57:28 +02:00
Sijmen Schoon
c959bf2b42
Limit parameter count per query in Cache.removeChildren
Signed-off-by: Sijmen Schoon <me@sijmenschoon.nl>
2021-10-17 19:10:58 +02:00
Robin Appelman
55346b5d6c
more reliable return value for Watcher::checkUpdate
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:03:18 +02:00
Robin Appelman
09ffac5e6d
s3 external storage listing rework
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 15:35:02 +02:00
John Molakvoæ
739b67031b
Merge pull request #27194 from nextcloud/new-ftp 2021-10-14 10:29:24 +02:00
Daniel
9187e986e1
Merge pull request #28907 from nextcloud/bug/noid/close-stream-local
explicitly close source stream on local / encryption storage
2021-10-08 10:02:01 +02:00
Robin Appelman
b3766fc99b
make param names consistent with interface for copy directory polyfill
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-07 17:19:44 +02:00
Vincent Petry
f518710c52
Merge pull request #28608 from nextcloud/path-prefix-filter
generate a better optimized query for path prefix search filters
2021-10-07 17:16:39 +02:00
acsfer
98eac0fc05
Simplify :) 2021-10-04 17:21:37 +02:00