Commit graph

219 commits

Author SHA1 Message Date
Jonas
aec0899efd Check whether entry is of type ICacheEntry in Cache->remove()
In some scenarios (file not in cache, but partial data of it in the
object), Cache->get() might return an array, which leads to errors like
"Call to a member function getId() on array".

So check whether the returned entry is of type ICacheEntry before doing
operations on it in Cache->remove().

Fixes: #33023

Signed-off-by: Jonas <jonas@freesources.org>
2022-06-30 12:25:26 +00:00
Robin Appelman
fd2afe604e
store unencrypted size in the unencrypted_size column
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-06-03 15:43:50 +02:00
Robin Appelman
32cd1bed38 tell mysql to ignore the sort index for search queries
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-05-16 09:53:43 +00:00
Robin Appelman
9834350ac3
Merge pull request #31713 from nextcloud/fed-performance
Federated share performance improvements
2022-04-07 18:53:36 +00:00
Joas Schilling
a44671d396
Merge pull request #31816 from nextcloud/bugfix/noid/deduplicate-storage-id-before-reusing
Deduplicate storage ids in list before reusing
2022-04-04 19:00:22 +02:00
Robin Appelman
cc5e26bb14
Merge pull request #31776 from nextcloud/storage-id-cache-bi-directional
cache storage id mapping both ways
2022-04-04 14:50:23 +00:00
Robin Appelman
5e2dd29aaf
only request free space once for external shares
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-04 16:48:27 +02:00
Robin Appelman
89d3cd1447
don't overwrite the etag from storage backends that already provide "good" etags
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-04 16:48:18 +02:00
Joas Schilling
d3b9d49c22
Deduplicate storage ids in list before reusing
Signed-off-by: Joas Schilling <coding@schilljs.com>
2022-04-04 07:07:48 +02:00
Robin Appelman
ac657b0f06
cache storage id mapping both ways
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-03-31 14:35:42 +02:00
Côme Chilliet
ea23523c70
Adapt more code to migration to LoggerInterface
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:25 +01:00
Côme Chilliet
6be7aa112f
Migrate from ILogger to LoggerInterface in lib/private
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:25 +01:00
Robin Appelman
84f464550a
some file scanner performance improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-03-17 22:21:16 +01:00
Robin Appelman
b0ce876f41
return dummy availability if storage is not found in cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-02-25 15:25:13 +01:00
acsfer
074a9a5e88
Switch logical operators
Unless there is a good reason to keep actual ones 👀
2021-12-03 16:05:19 +01:00
Robin Appelman
f5fe887b4b
background scan the source storage when a background scan on a storage jail is triggered
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-12-02 17:34:58 +01:00
Côme Chilliet
113756db30
Fix ArrayAccess and JsonSerializable return types
First round of modifications for PHP 8.1

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2021-11-23 09:28:56 +01:00
Vincent Petry
c92a0e4152
Normalize directory entries in Encoding wrapper
Directory entry file names are now normalized in getMetaData(),
getDirectoryContents() and opendir().

This makes the scanner work properly as it assumes pre-normalized names.

In case the names were not normalized, the scanner will now skip the
entries and display a warning when applicable.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-17 09:43:25 +01:00
Vincent Petry
67ebe75d0e
Move storage encoding compatibility warning logic
The encoding check for file names is now happening the Scanner, and an
event will be emitted only if the storage doesn't contain the encoding
compatibility wrapper.

The event is listened to by the occ scan command to be able to display a
warning in case of file name mismatches when they have NFD encoding.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-17 09:24:13 +01:00
Vincent Petry
39fe43b914
Normalize file name before existence check in scanner
The scanner would not find a NFD-encoded file name in an
existing file list that is normalized.

This normalizes the file name before scanning.

Fixes issues where scanning repeatedly would make NFD files flicker in
and out of existence in the file cache.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-17 09:24:13 +01:00
Robin Appelman
118c647f61
properly handle cases where cache wrappers block access
`CacheWrapper::formatCacheEntry` can return false for files that should be filtered out

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-11-01 11:48:01 +01:00
John Molakvoæ
63d3931e80
Merge pull request #29281 from vijfhoek/master 2021-11-01 11:29:37 +01:00
Carl Schwan
df4e6bab69
Merge pull request #29115 from nextcloud/work/carl/correct-permissions-when-copying
Fix permissions when copying from ObjectStorage
2021-10-28 14:38:20 +02:00
Carl Schwan
bfa60aaf27
Fix permissions when copying from ObjectStorage
Make sure that when a user copy a file from a directory they don't have
all permissions to a directory where they have more permissions, the
permissions are correctly set to the one from the parent taget folder.

This was caused by the ObjectStoreStorage::copyFromStorage using
the jailed storage and cache entry instead of the unjailed one like other
storages (the local one).
Steps to reproduce

+ Use object storage
+ Create a groupfolder with one group having full permission and another one
  who can just read files.
+ With an user who is in the second group, copy a file from the groupfolder to
  the home folder of this user.
+ The file in the home folder of the user will be read only and can't be deleted
  even though it is in their home folder and they are the owner. In oc_filecache,
  the permissions stored for this file are 1 (READ)

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-10-28 13:29:50 +02:00
Sijmen Schoon
34600c78bb
Move query outside the loop and reduce chunk size to 1000
This involved changing CacheQueryBuilder\whereParentIn to take a
parameter name, renaming the function accordingly.

Signed-off-by: Sijmen Schoon <me@sijmenschoon.nl>
2021-10-17 20:57:28 +02:00
Sijmen Schoon
c959bf2b42
Limit parameter count per query in Cache.removeChildren
Signed-off-by: Sijmen Schoon <me@sijmenschoon.nl>
2021-10-17 19:10:58 +02:00
Robin Appelman
55346b5d6c
more reliable return value for Watcher::checkUpdate
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-10-15 16:03:18 +02:00
Robin Appelman
7351059326
generate a better optimized query for path prefix search filters
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-08-26 19:07:52 +02:00
Vincent Petry
e32d293346
Merge pull request #28476 from nextcloud/search-jail-insensitive
use case insensitive like when limiting search to jail
2021-08-24 08:59:35 +02:00
Robin Appelman
c688703117
use getGetUnjailedRoot to determine if jailed search needs the path filter
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-08-23 16:27:53 +02:00
Robin Appelman
c48f9ffcdd
use case insensitive like when limiting search to jail
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-08-17 13:51:45 +02:00
Carl Schwan
28970563a2
Remove some mentions of ownCloud from our api documentation
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-07-29 15:56:30 +02:00
Robin Appelman
97aadbe46c
dont apply jail search filter is on the root
the extra '/' breaks things and the filter wouldn't do anything anyway except making the databases job harder

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-07-26 17:44:52 +02:00
Robin Appelman
362cb2a11f
inject SearchBuilder
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-06-14 16:23:08 +02:00
Robin Appelman
66e10718c6
split of query building bits from searchhelper
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-06-14 16:11:40 +02:00
Robin Appelman
f938daa6e4
update tests and fix some edge cases around new search
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-06-14 16:11:36 +02:00
Robin Appelman
27e50b24aa
some cleanup and documentation
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-06-14 16:11:32 +02:00
Robin Appelman
5d1d53c42e
perform file search in a single query
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-06-14 16:11:29 +02:00
Robin Appelman
9774fb1573
use searchoperation for storage filter instead of db expression
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-06-14 16:11:25 +02:00
Robin Appelman
e198dc1b20
rework search api to allow searching on multiple caches at once
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-06-14 16:11:22 +02:00
John Molakvoæ (skjnldsv)
215aef3cbd
Update php licenses
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +02:00
Robin Appelman
7ee528fc5a
properly use limit and offset for search in Jail wrapper
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-05-21 15:37:38 +02:00
Robin Appelman
ed2396b045
better cleanup of filecache when deleting an external storage
this way it can delete the cache entries even with per-user credentials

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-05-19 17:08:20 +02:00
Morris Jobke
81fef4ddee
Log when a storage is marked as unavailable
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2021-03-25 16:19:08 +01:00
John Molakvoæ (skjnldsv)
2074d87d0c
Catch invalid cache source storage path
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-03-24 10:36:51 +01:00
Robin Appelman
50e374c12f
only require user to be set in a query that handles tags
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-18 17:16:28 +01:00
Robin Appelman
f1e2fb426e
Merge pull request #26013 from nextcloud/object-store-copy-cache-id
Return the fileid from `copyFromCache` and use it instead of doing an extra query
2021-03-16 16:11:17 +00:00
Vincent Petry
e559afb8d4
Merge pull request #25136 from nextcloud/cachejail-search-filter
do cachejail search filtering in sql
2021-03-15 17:45:30 +01:00
Robin Appelman
d7748e2b4d
verify that cache entry is valid
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-12 17:23:02 +01:00
Robin Appelman
c2fc3195dd
also implement for FailedCache and NullCache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-12 17:22:43 +01:00