Commit graph

287 commits

Author SHA1 Message Date
Benjamin Gaussorgues
30e35a8411
fix(perf): use index when deleting old previews
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2026-02-18 17:27:36 +01:00
Carl Schwan
2391d58eb0
Merge pull request #58107 from nextcloud/local-preview-mkdir-exists
fix: don't try to create preview folder if it already exists
2026-02-13 15:10:24 +01:00
Carl Schwan
3da621d7fb
Merge pull request #58230 from nextcloud/carl/fix-appdata-scanning
fix(preview): Fix files:app-data-scan for previews
2026-02-12 00:57:28 +01:00
Carl Schwan
3d18cd7cc5
fix(preview): Fix files:app-data-scan for previews
And add unit tests for both migrated files and non-migrated files

Signed-off-by: Carl Schwan <carlschwan@kde.org>
2026-02-12 00:26:20 +01:00
Carl Schwan
4722f56778 fix(preview): Handle unique constraints
Signed-off-by: Carl Schwan <carlschwan@kde.org>
2026-02-11 19:56:11 +01:00
provokateurin
fe9e43c165
feat(preview): Expire previews
Signed-off-by: provokateurin <kate@provokateurin.de>
2026-02-10 15:35:38 +01:00
Andy Scherzinger
7da7f50203
Merge pull request #58209 from nextcloud/carl/local-preview-fix
fix(preview): Fix scanning preview
2026-02-10 12:38:58 +01:00
Carl Schwan
543b46f3aa fix(preview): Fix scanning preview
Make sure we set the mimetype string representation and not int
representation.

Signed-off-by: Carl Schwan <carlschwan@kde.org>
2026-02-09 22:49:59 +01:00
Anna Larch
99a1150ec2 fix(snowflake): cast lastId to int
Signed-off-by: Anna Larch <anna@nextcloud.com>
2026-02-09 21:36:20 +01:00
provokateurin
f12cecb684
feat(rector): Enable SafeDeclareStrictTypesRector
Signed-off-by: provokateurin <kate@provokateurin.de>
2026-02-09 10:59:31 +01:00
Carl Schwan
c96ece0bcb
refactor: Add more typing
- repairs job
- database
- redis

And remove Helpertest which was unused outside of some tests.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2026-02-06 13:55:39 +01:00
Carl Schwan
7b6078875b
refactor: Run rector on lib/private
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2026-02-06 13:50:18 +01:00
Robin Appelman
2e9f6c37d5
fix: don't try to create preview folder if it already exists
Signed-off-by: Robin Appelman <robin@icewind.nl>
2026-02-05 15:22:54 +01:00
Carl Schwan
c6c11d474b
Merge pull request #55649 from nextcloud/carl/on-demand-preview-migration
feat(preview): On demand preview migration
2026-01-28 22:33:48 +01:00
Ferdinand Thiessen
4b015568fc test(preview): properly test postscript preview provider
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
2026-01-28 01:09:27 +01:00
Carl Schwan
7a025ffb0b
feat(preview): Make it possible to disable on preview migration
Signed-off-by: Carl Schwan <carlschwan@kde.org>
2026-01-14 14:09:07 +01:00
Carl Schwan
6149168129
feat(preview): On demand preview migration
When requesting previews, which we don't find in oc_previews, search in
IAppData first before creating them.

Move the logic from MovepreviewJob to PreviewMigrationService and reuse
that in the Preview Generator.

At the same time rename MovePreviewJob to PreviewMigrationJob as it is a
better name.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2026-01-12 14:20:09 +01:00
Carl Schwan
7c1a8a4060
feat: Adapt a bit the snowflake ids API
- Rename setId() -> generateId() in SnowflakeAwareEntity

Signed-off-by: Carl Schwan <carlschwan@kde.org>
2026-01-06 12:57:17 +01:00
Anna Larch
5f797ebc32
refactor: move existing usages of snoflake IDs SnowflakeAwareEntity
Signed-off-by: Anna Larch <anna@nextcloud.com>
2026-01-06 12:57:17 +01:00
Anna Larch
f546daada7
refactor: Rename Snowflake Generator and Decoder
And introduce the Snowflake DTO

Signed-off-by: Anna Larch <anna@nextcloud.com>
2026-01-06 12:57:04 +01:00
Carl Schwan
2ea1bd4cdc
feat(movie-preview): Use getDirectDownloadById for generating preview
Allow to speed-up considerably the creation of previews for movies
stored on S3.

Signed-off-by: Carl Schwan <carlschwan@kde.org>
2026-01-02 17:13:41 +01:00
invario
ffe91b48dc
feat(previews): allow ffmpeg to connect direct for AWS S3 buckets
Co-authored-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2026-01-02 10:48:39 +01:00
Carl Schwan
06b6a5bc05
fix(preview): Correctly insert preview version
Use createNamedParameter otherwise values are not correctly escapted.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-12-17 18:23:53 +01:00
Carl Schwan
247b66c5ee
fix(preview-cleanup): Also delete previews stored in the oc_previews table
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-12-10 13:49:24 +01:00
Carl Schwan
503a65ddd0 refactor: Run rector
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-11-18 17:45:54 +01:00
Carl Schwan
336cc3fa35 feat(Db): Use SnowflakeId for previews
Allow to get an id for the storing the preview on disk before inserting
the preview on the DB.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-28 17:50:03 +01:00
Carl Schwan
1dc363581f fix: get all available previews on Oracle
Split the DB requests in chunk of 1000 and use INNER JOIN instead of IN
as this is better supported on all DB.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-23 14:10:15 +02:00
Carl Schwan
9adf6cce55 fix: Fetching previews on Oracle
We need an explicit cast on Oracle as Oracle returns number as string.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-23 09:39:05 +02:00
Carl Schwan
d6889e9220 fix(preview): Fix deleting dummy preview in object store
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-22 15:40:26 +02:00
Stephan Orbaugh
567a988c6c
Merge pull request #55100 from invario/previews-better-logging
feat(previews): improved logging for movie previews
2025-10-15 10:54:09 +02:00
invario
13500be1d1
feat(previews): improved logging for movie previews
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2025-10-14 13:17:09 -04:00
Carl Schwan
cb82a9ef04 refactor(preview): Cleanup a bit the public interface
* Remove old IProvider interface, it's been deprecated since 17.0.0 (8
  years)
* Add type hinting to the IPreview interface and mark it as consumeable
  only
* Remove unused arguments from GeneratorHelper

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-09 17:41:13 +02:00
Carl Schwan
fed7a33d1f refactor(preview-object-store): Refactor object store backend
Simplify logic

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-08 14:07:07 +02:00
Carl Schwan
bef3996c3e fix(preview): Make version column a string
And move it to a different table so that we don't have to pay the
storage cost when not using it (most of the times).

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-08 14:07:06 +02:00
Carl Schwan
66f50bd585 refactor(preview): Use same mimetype ids as filecache
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
bd001c9524 refactor: Use Override annotation in new preview code
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
58023782b6 feat(preview): Store original file mimetype in preview table
Allow to quickly query all the files from a specific mimetype like in
the ResetRenderedTexts command.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
bfc7d5dd9f feat(preview): Implement scanning for previews
This work similarly to the move preview job to migrate the previews to
the new DB table and also reuse some code.

So when we are finding files in appdata/preview, try adding them to the
oc_previews table and delete them from the oc_filecache table.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
324b54b863 refactor(preview): Cleanup the implementation of the new preview backend
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
6f56dcf73e fix(preview): Fix some tests
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
b0357663b9 perf(preview): Optimize migration and simplify DB layout
* Simplify migration by not moving the actual files and just updating
  the DB
* Don't store the storageid in the preview table as it is not needed
* Start adding tests

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
bba9667882 perf(preview): Adapt BackgroundCleanupJob to new previews table
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
6008852232 feat(preview): Support multibucket storage
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
13c35c0f17 perf(preview): Migrate previews to the new optimized table
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
656e33e8da perf(preview): Add support for multibucket storage
Signed-off-by: Carl Schwan <carl.schwan@nextclound.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
18fbacdd8d perf(preview): Split preview data to new table
The new oc_previews table is optimized for storing previews and should
decrease significantly the space taken by previews in the filecache
table.

This attend to reuse the IObjectStore abstraction over S3/Swift/Azure
but currently only support one single bucket configuration.

Signed-off-by: Carl Schwan <carl.schwan@nextclound.com>
2025-10-06 13:37:15 +02:00
invario
ba51caf5f4
Fix(previews): prevent infinite loop in case of bad video file
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2025-09-05 09:32:37 -04:00
invario
97a0dde160 feat(previews): previews and tests for large remote videos w/o full download
Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Co-authored-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2025-09-05 10:08:19 +02:00
provokateurin
4b2a93cf0a
chore(Preview): Remove avconv support
Signed-off-by: provokateurin <kate@provokateurin.de>
2025-08-27 20:51:08 +02:00
Ferdinand Thiessen
5981b7eb51
chore: apply new CSFixer rules
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>

# Conflicts:
#	apps/settings/lib/SetupChecks/PhpOpcacheSetup.php
2025-07-01 16:26:50 +02:00