Commit graph

312 commits

Author SHA1 Message Date
Git'Fellow
394cd75322
Merge pull request #41272 from nextcloud/fixOccScans
fix: reduce memory consumption of scans
2023-11-28 11:36:59 +01:00
Joas Schilling
aa5f037af7
chore: apply changes from Nextcloud coding standards 1.1.1
Signed-off-by: Joas Schilling <coding@schilljs.com>
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2023-11-23 10:36:13 +01:00
Maxence Lange
ee787cd1c6
Merge pull request #41459 from nextcloud/enh/noid/search-metadata-null
implements search on null/notnull metadata
2023-11-23 07:52:27 -01:00
Maxence Lange
7467422f57 implements search on null/notnull metadata
Signed-off-by: Maxence Lange <maxence@artificial-owl.com>
2023-11-22 12:43:27 -01:00
Christoph Wurst
ea8f9a7e84 refactor: Repalce array_search with in_array in lib/
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2023-11-22 11:42:53 +01:00
Maxence Lange
761a688ec9 fix psalm
Signed-off-by: Maxence Lange <maxence@artificial-owl.com>
2023-11-14 10:11:24 -01:00
Maxence Lange
3af12b2a50 ignore metadata if table is empty
Signed-off-by: Maxence Lange <maxence@artificial-owl.com>
2023-11-14 09:28:58 -01:00
Maxence Lange
380b81315a moving metadataquery
Signed-off-by: Maxence Lange <maxence@artificial-owl.com>
2023-11-13 22:25:37 -01:00
Benjamin Gaussorgues
a75a93af8e
feat(files): Allow advanced search for files
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2023-11-10 09:21:16 +01:00
Louis Chemineau
d3a313f192
Support getting and setting metadata in DAV requests
Signed-off-by: Louis Chemineau <louis@chmn.me>
2023-11-08 16:23:53 +01:00
Maxence Lange
f497d8b6e5 IFilesMetadata
Signed-off-by: Maxence Lange <maxence@artificial-owl.com>
2023-11-07 12:43:08 -01:00
Maxence Lange
e62e9e3dbf IFilesMetadata
Signed-off-by: Maxence Lange <maxence@artificial-owl.com>
2023-11-07 00:21:38 -01:00
Git'Fellow
b4a3773115 fix: reduce memory consumption of scans
Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>

Fix lint

Signed-off-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
2023-11-03 18:07:42 +01:00
Robin Appelman
460344336e
optimize cache jail creation
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-10-19 19:27:00 +02:00
Hamid Dehnavi
ea06cf2f39 Convert isset ternary to null coalescing operator
Signed-off-by: Hamid Dehnavi <hamid.dev.pro@gmail.com>
2023-09-28 17:44:19 +03:30
Côme Chilliet
5090848b25
Store size in int|float for 32bit support
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-09-19 13:43:01 +02:00
Julius Härtl
13b9dd2267 fix: Only read unencrypted_size when file is actually encrypted
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-09-06 07:25:28 +02:00
Robin Appelman
9ea03a9534 reuse dependencies from wrapped cache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-08-18 11:14:52 +02:00
Ferdinand Thiessen
9c04c07622 fix: Prevent PHP warnings when optional CacheEntry attributes are unset
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2023-08-16 14:12:51 +02:00
Robin Appelman
658aed27ea select the fileid first when looking for incomplete files
this seems to improve mariadbs index selection

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-08-14 18:08:01 +02:00
John Molakvoæ
7b7d1e3c09
fix: simplify sourceData check
Co-authored-by: Git'Fellow <12234510+solracsf@users.noreply.github.com>
Signed-off-by: John Molakvoæ <skjnldsv@users.noreply.github.com>
2023-08-05 12:05:36 +02:00
Josh Richards
deecb48e54
Catch more invalid cache source storage paths
OC\Files\Cache\Cache::get can return string|false|null, not just string|false.

- nextcloud/server#26270 added handling of false, but null is needed too.
- Well, or we change the default $resullt to false, but I'm not sure if that has other ramifications and the real need here is to simply catch situations where the cache source storage path is not valid for whatever reason

Related: nextcloud/server#19009

Signed-off-by: Josh Richards <josh.t.richards@gmail.com>
2023-08-03 09:53:46 -04:00
Robin Appelman
c912a3f0f6
Merge pull request #38860 from nextcloud/scanner-change-summary
add summary of detected changes to files:scan output
2023-07-20 18:37:07 +02:00
Christoph Wurst
14719110b9 chore: Replace \OC::$server->query with \OCP\Server::get in /lib
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2023-07-06 15:21:22 +02:00
Julius Härtl
eddb64f8c3
Merge pull request #37969 from nextcloud/poc/noid/systemtags-perf-tag-endpoint 2023-06-28 07:53:35 +02:00
Robin Appelman
39a0554c09
Merge pull request #38858 from nextcloud/scanner-performance3
file scanner performance improvements
2023-06-26 17:01:18 +02:00
Arthur Schiwon
49db546f78
fix: include invisible tags for admins
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 20:28:01 +02:00
Arthur Schiwon
783e32a64d
fix: favorites view and universal search against tags
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 18:19:08 +02:00
Arthur Schiwon
b0d1cf5730
fix: change if with conditionless else to switch; and a parameter value
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 18:19:08 +02:00
Arthur Schiwon
58f7fd2370
use efficient tag retrieval on DAV report request
- uses DAV search approach against valid files joined by systemtag selector
- reduced table join for tag/systemtag search
- supports pagination
- no changes to the output formats or similar

Example request body:

<?xml version="1.0"?>
<oc:filter-files xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns" xmlns:ocs="http://open-collaboration-services.org/ns">
  <d:prop>
    <d:getcontentlength/>
    <d:getcontenttype/>
    <d:getetag/>
    <d:getlastmodified/>
    <d:resourcetype/>
    <nc:face-detections/>
    <nc:file-metadata-size/>
    <nc:has-preview/>
    <nc:realpath/>
    <oc:favorite/>
    <oc:fileid/>
    <oc:permissions/>
    <nc:nbItems/>
  </d:prop>
  <oc:filter-rules>
    <oc:systemtag>32</oc:systemtag>
  </oc:filter-rules>
  <d:limit>
    <d:nresults>50</d:nresults>
    <nc:firstresult>0</nc:firstresult>
  </d:limit>
</oc:filter-files>

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 18:19:08 +02:00
Arthur Schiwon
71e41ffeeb
refactor: declare getMount() and getMountsIn() at IRootFolder
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 16:53:20 +02:00
Arthur Schiwon
f6ac874dbf
chore: ugly type juggling
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 16:53:20 +02:00
Arthur Schiwon
eb50b6c6aa
fix: expect interface, not a specific implementation
- fixes a regression when deleting folders while music app was enabled,
  for a LazyRoot was passed to this method.

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-06-21 16:53:13 +02:00
Robin Appelman
292c0e53f8
add summary of detected changes to files:scan output
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-06-16 17:56:34 +02:00
Robin Appelman
561536fc58
file scanner performance improvements
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-06-16 16:31:48 +02:00
Christoph Wurst
872c181c74
chore: Drop dead private methods in /lib
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2023-06-06 11:01:58 +02:00
Simon L
702196dd86
Merge pull request #38478 from nextcloud/backport/38440/master
[master] fix: Catch Deadlock properly as execute throws Doctrine exceptions not our wrapped ones
2023-06-05 23:22:40 +02:00
Robin Appelman
9f1d497a0b
Merge pull request #38261 from fsamapoor/replace_strpos_calls_in_lib_private
Refactors "strpos" calls in  lib/private to improve code readability.
2023-06-01 23:10:00 +02:00
Julius Härtl
9453c2d40d fix: execute throws docrine exceptions not our wrapped ones
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-05-26 06:56:55 +00:00
Côme Chilliet
8362eea14e
Merge pull request #38196 from nextcloud/fix/fix-32bits-freespace-and-sizes
Get rid of more int casts in file size manipulations
2023-05-15 17:05:46 +02:00
Faraz Samapoor
e7cc7653b8 Refactors "strpos" calls in lib/private to improve code readability.
Signed-off-by: Faraz Samapoor <fsamapoor@gmail.com>
2023-05-15 15:17:19 +03:30
Côme Chilliet
ae525e1935
Fix return types in phpdoc
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-05-15 12:24:42 +02:00
Julius Härtl
e18f97fc95
Merge pull request #37709 from nextcloud/bugfix/deadlock 2023-05-13 13:13:04 +02:00
Côme Chilliet
a90581b405
Get rid of more int casts in file size manipulations
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-05-11 12:46:16 +02:00
Arthur Schiwon
b6c034ac57
Merge pull request #37961 from nextcloud/poc/noid/systemtags-perf
SystemTags endpoint to return tags used by a user with meta data
2023-05-11 10:16:03 +02:00
Robin Appelman
2ea41dab93
repair -1 folder sizes for object store background scan
Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-05-10 19:33:26 +02:00
Arthur Schiwon
df662f50bd
refactor: remove where specification from SELECT getter
- search constraints are now fully in control of
  SystemTagsInFilesDetector::detectAssignedSystemTagsIn(), avoids
  duplication of a WHERE statement

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-10 18:48:06 +02:00
Arthur Schiwon
dbfd2f936a
refactor: remove SystemTag logic from Folder into QuerySearchHelper
- adds OC\SystemTag\SystemTagsInFilesDetector where the search logic is
  moved to

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-09 23:51:52 +02:00
Arthur Schiwon
2489a2d63a
feat: specify media type via url path: systemtags-current/$mediaType
- only the media part of the mime type can be search, but not the full
  mime type. It can be added, should it become necessary.
- thus fixes previously hardcoded selector for image/ types
- also fixes a return type hint
- adds a return type hint

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-09 23:51:51 +02:00
Arthur Schiwon
6bff6a5667
PoC: SystemTags endpoint to return tags used by a user with meta data
Target case is photos app: when visiting the tags category, all systemtags
of the whole cloud are retrieved. In subequent steps the next tag is
requested until the browser view is filled with tag tiles (i.e. previews
are requested just as well).

With this approach, we incorpoate the dav search and look for user related
tags that are used by them, and already returns the statistics (number of
files tagged with the respective tag) as well as a file id for the purpose
to load the preview. This defaults to the file with the highest id.

Call:
curl -s -u 'user:password' \
  'https://my.nc.srv/remote.php/dav/systemtags-current' \
  -X PROPFIND -H 'Accept: text/plain' \
  -H 'Accept-Language: en-US,en;q=0.5'  -H 'Depth: 1' \
  -H 'Content-Type: text/plain;charset=UTF-8' \
  --data @/home/doe/request-systemtag-props.xml

With request-systemtag-props.xml:
<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:">
        <d:prop xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
                <oc:id/>
                <oc:display-name/>
                <oc:user-visible/>
                <oc:user-assignable/>
                <oc:can-assign/>
                <nc:files-assigned/>
                <nc:reference-fileid/>
        </d:prop>
</d:propfind>

Example output:
  …
  <d:response>
    <d:href>/master/remote.php/dav/systemtags/84</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>84</oc:id>
        <oc:display-name>Computer</oc:display-name>
        <oc:user-visible>true</oc:user-visible>
        <oc:user-assignable>true</oc:user-assignable>
        <oc:can-assign>true</oc:can-assign>
        <nc:files-assigned>42</nc:files-assigned>
        <nc:reference-fileid>924022</nc:reference-fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/systemtags/97</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>97</oc:id>
        <oc:display-name>Bear</oc:display-name>
        <oc:user-visible>true</oc:user-visible>
        <oc:user-assignable>true</oc:user-assignable>
        <oc:can-assign>true</oc:can-assign>
        <nc:files-assigned>1</nc:files-assigned>
        <nc:reference-fileid>923422</nc:reference-fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  …

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-05-09 23:51:48 +02:00