Commit graph

247 commits

Author SHA1 Message Date
Arthur Schiwon
0612ad250e
refactor: remove where specification from SELECT getter
- search constraints are now fully in control of
  SystemTagsInFilesDetector::detectAssignedSystemTagsIn(), avoids
  duplication of a WHERE statement

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-16 12:59:10 +02:00
Arthur Schiwon
1e5de8aff4
refactor: remove SystemTag logic from Folder into QuerySearchHelper
- adds OC\SystemTag\SystemTagsInFilesDetector where the search logic is
  moved to

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-16 12:59:04 +02:00
Arthur Schiwon
de008e1109
feat: specify media type via url path: systemtags-current/$mediaType
- only the media part of the mime type can be search, but not the full
  mime type. It can be added, should it become necessary.
- thus fixes previously hardcoded selector for image/ types
- also fixes a return type hint
- adds a return type hint

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-16 12:58:47 +02:00
Arthur Schiwon
8ec4760645
PoC: SystemTags endpoint to return tags used by a user with meta data
Target case is photos app: when visiting the tags category, all systemtags
of the whole cloud are retrieved. In subequent steps the next tag is
requested until the browser view is filled with tag tiles (i.e. previews
are requested just as well).

With this approach, we incorpoate the dav search and look for user related
tags that are used by them, and already returns the statistics (number of
files tagged with the respective tag) as well as a file id for the purpose
to load the preview. This defaults to the file with the highest id.

Call:
curl -s -u 'user:password' \
  'https://my.nc.srv/remote.php/dav/systemtags-current' \
  -X PROPFIND -H 'Accept: text/plain' \
  -H 'Accept-Language: en-US,en;q=0.5'  -H 'Depth: 1' \
  -H 'Content-Type: text/plain;charset=UTF-8' \
  --data @/home/doe/request-systemtag-props.xml

With request-systemtag-props.xml:
<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:">
        <d:prop xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
                <oc:id/>
                <oc:display-name/>
                <oc:user-visible/>
                <oc:user-assignable/>
                <oc:can-assign/>
                <nc:files-assigned/>
                <nc:reference-fileid/>
        </d:prop>
</d:propfind>

Example output:
  …
  <d:response>
    <d:href>/master/remote.php/dav/systemtags/84</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>84</oc:id>
        <oc:display-name>Computer</oc:display-name>
        <oc:user-visible>true</oc:user-visible>
        <oc:user-assignable>true</oc:user-assignable>
        <oc:can-assign>true</oc:can-assign>
        <nc:files-assigned>42</nc:files-assigned>
        <nc:reference-fileid>924022</nc:reference-fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/systemtags/97</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>97</oc:id>
        <oc:display-name>Bear</oc:display-name>
        <oc:user-visible>true</oc:user-visible>
        <oc:user-assignable>true</oc:user-assignable>
        <oc:can-assign>true</oc:can-assign>
        <nc:files-assigned>1</nc:files-assigned>
        <nc:reference-fileid>923422</nc:reference-fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  …

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-16 12:58:42 +02:00
Julius Härtl
2788dfc4f2 fix: Check for wrapped retriable exceptions
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-05-13 22:23:05 +00:00
Robin Appelman
5c8a8df114 ignore errors while trying to update parent storage_mtime
in the worst case this should only cause an extra rescan later

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-21 17:41:47 +00:00
Robin Appelman
034f5a17ee clear encrypted flag when moving away from encrypted storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-04-01 16:22:42 +00:00
Robin Appelman
5c743ac135 escape path prefix when doing cache jail search
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-01-11 16:58:54 +00:00
Andy Xheli
5200b0e5f2 Fix Executing a query: SQLSTATE[42000]
Signed-off-by: Andy Xheli <axheli@axtsolutions.com>

Fix https://github.com/nextcloud/server/issues/32007#issuecomment-1329405245

Signed-off-by: Andy Xheli <axheli@axtsolutions.com>
2022-11-30 11:48:43 +00:00
Robin Appelman
5733fccdd1 fix null error in getUnencryptedSize
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-10-21 14:59:40 +00:00
raul
7ed5679f2c Fix unencrypted size calculation for files created before the Encryption storage is enabled
Signed-off-by: raul <raul@nextcloud.com>
2022-10-03 14:29:01 +00:00
raul
380aaef96e Fix: Prevent deadlocks during mtime/size/etag propagation
Signed-off-by: raul <raul@nextcloud.com>
2022-10-03 14:29:01 +00:00
Robin Appelman
b6f8b8da60 directly build the search filter for shared storage instead of setting up the source cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-17 11:58:57 +02:00
Vincent Petry
312b719acf
Merge pull request #32943 from nextcloud/unencrypted-size-revert-interface-changes
Revert interface changes from "store unencrypted size in the unencrypted_size column"
2022-08-17 11:36:16 +02:00
Vincent Petry
11d5a1d3df
Merge pull request #33550 from nextcloud/jail-search-post
optimize search post-processing for jail wrapper
2022-08-17 10:55:41 +02:00
Vincent Petry
082432e01c
Merge pull request #33551 from nextcloud/scanner-dont-update-same-size
don't update the folder size if we know it hasn't changed
2022-08-17 09:21:30 +02:00
Robin Appelman
457822c144 don't update the folder size if we know it hasn't changed
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:56:51 +02:00
Robin Appelman
de63f6363f fix updating size when folder is empty
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:54:26 +02:00
Robin Appelman
1374cbee3e store unencrypted size in the unencrypted_size column
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:54:26 +02:00
Robin Appelman
9d6c2ce301 optimize search post-processing for jail wrapper
don't both asking the wrapped cache if we know it's not in our jail anyway

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 13:36:52 +02:00
Robin Appelman
5e375d9092 Revert "store unencrypted size in the unencrypted_size column"
This reverts commit 8238582e59.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-08-16 12:07:14 +02:00
David
fe3f4afd77
fixed the cache searchBuilder negative map
Signed-off-by: David <37280718+yeyulantu@users.noreply.github.com>
2022-08-16 17:59:02 +08:00
Côme Chilliet
7ae6791737
Document all getIncomplete implementations as returning string|false
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-08-02 14:23:14 +02:00
Côme Chilliet
cb271b759e
Fix dynamic property creations in test files
This fixes warnings in PHP 8.2

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-08-01 15:07:53 +02:00
luz paz
368f83095d Fix typos in lib/private subdirectory
Found via `codespell -q 3 -S l10n -L jus ./lib/private`

Signed-off-by: luz paz <luzpaz@github.com>
2022-07-27 08:52:17 -04:00
Jonas
7d07e06bfe Check whether entry is of type ICacheEntry in Cache->remove()
In some scenarios (file not in cache, but partial data of it in the
object), Cache->get() might return an array, which leads to errors like
"Call to a member function getId() on array".

So check whether the returned entry is of type ICacheEntry before doing
operations on it in Cache->remove().

Fixes: #33023

Signed-off-by: Jonas <jonas@freesources.org>
2022-06-29 10:47:07 +00:00
Carl Schwan
f326b54e53 Search without join on filecache_extended
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-06-23 16:23:52 +02:00
Robin Appelman
8238582e59
store unencrypted size in the unencrypted_size column
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-06-02 16:07:00 +02:00
Marcel Klehr
f755ee0868 Files: Extend search to also cover tags
fixes #326

Signed-off-by: Marcel Klehr <mklehr@gmx.net>
2022-05-17 21:05:09 +02:00
Côme Chilliet
069477e608
Migrate more classes of lib/private to LoggerInterface
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-04-26 16:52:52 +02:00
Robin Appelman
a4e120c203
tell mysql to ignore the sort index for search queries
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-22 13:42:33 +02:00
Robin Appelman
9834350ac3
Merge pull request #31713 from nextcloud/fed-performance
Federated share performance improvements
2022-04-07 18:53:36 +00:00
Joas Schilling
a44671d396
Merge pull request #31816 from nextcloud/bugfix/noid/deduplicate-storage-id-before-reusing
Deduplicate storage ids in list before reusing
2022-04-04 19:00:22 +02:00
Robin Appelman
cc5e26bb14
Merge pull request #31776 from nextcloud/storage-id-cache-bi-directional
cache storage id mapping both ways
2022-04-04 14:50:23 +00:00
Robin Appelman
5e2dd29aaf
only request free space once for external shares
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-04 16:48:27 +02:00
Robin Appelman
89d3cd1447
don't overwrite the etag from storage backends that already provide "good" etags
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-04 16:48:18 +02:00
Joas Schilling
d3b9d49c22
Deduplicate storage ids in list before reusing
Signed-off-by: Joas Schilling <coding@schilljs.com>
2022-04-04 07:07:48 +02:00
Robin Appelman
ac657b0f06
cache storage id mapping both ways
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-03-31 14:35:42 +02:00
Côme Chilliet
ea23523c70
Adapt more code to migration to LoggerInterface
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:25 +01:00
Côme Chilliet
6be7aa112f
Migrate from ILogger to LoggerInterface in lib/private
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:25 +01:00
Robin Appelman
84f464550a
some file scanner performance improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-03-17 22:21:16 +01:00
Robin Appelman
b0ce876f41
return dummy availability if storage is not found in cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-02-25 15:25:13 +01:00
acsfer
074a9a5e88
Switch logical operators
Unless there is a good reason to keep actual ones 👀
2021-12-03 16:05:19 +01:00
Robin Appelman
f5fe887b4b
background scan the source storage when a background scan on a storage jail is triggered
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-12-02 17:34:58 +01:00
Côme Chilliet
113756db30
Fix ArrayAccess and JsonSerializable return types
First round of modifications for PHP 8.1

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2021-11-23 09:28:56 +01:00
Vincent Petry
c92a0e4152
Normalize directory entries in Encoding wrapper
Directory entry file names are now normalized in getMetaData(),
getDirectoryContents() and opendir().

This makes the scanner work properly as it assumes pre-normalized names.

In case the names were not normalized, the scanner will now skip the
entries and display a warning when applicable.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-17 09:43:25 +01:00
Vincent Petry
67ebe75d0e
Move storage encoding compatibility warning logic
The encoding check for file names is now happening the Scanner, and an
event will be emitted only if the storage doesn't contain the encoding
compatibility wrapper.

The event is listened to by the occ scan command to be able to display a
warning in case of file name mismatches when they have NFD encoding.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-17 09:24:13 +01:00
Vincent Petry
39fe43b914
Normalize file name before existence check in scanner
The scanner would not find a NFD-encoded file name in an
existing file list that is normalized.

This normalizes the file name before scanning.

Fixes issues where scanning repeatedly would make NFD files flicker in
and out of existence in the file cache.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-11-17 09:24:13 +01:00
Robin Appelman
118c647f61
properly handle cases where cache wrappers block access
`CacheWrapper::formatCacheEntry` can return false for files that should be filtered out

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-11-01 11:48:01 +01:00
John Molakvoæ
63d3931e80
Merge pull request #29281 from vijfhoek/master 2021-11-01 11:29:37 +01:00