a setup can have multiple bucket without having `multibucket` enabled trough things like per-groupfolder buckets
Signed-off-by: Robin Appelman <robin@icewind.nl>
This is faster than going back to nextcloud to download the files.
This is an opt-in setting that can be enabled by setting
use_presigned_url in the object store config.
Additionally add support for the proxy config which is needed in a
docker setup. See https://github.com/juliusknorr/nextcloud-docker-dev/pull/431
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
Add retriesMaxAttempts parameter to S3 objectstore configuration
to allow customization of AWS SDK retry behavior for handling
unreliable network conditions or proxy issues.
Defaults to 5 retries (AWS SDK default) if not specified.
Signed-off-by: nfebe <fenn25.fn@gmail.com>
Otherwise, we call doesBucketExist all the time which does a network
request to the S3 server adding some non-trivial latency when creating a
S3 connection object.
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
Fix https://github.com/nextcloud/server/issues/56077
This commit makes the configuration settings 'request_checksum_calculation' and 'response_checksum_validation' of the S3Client from the AWS SDK for PHP configurable.
Signed-off-by: Fiehe Christoph <c.fiehe@eurodata.de>
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
to avoid 'object::%' to be considered as a column
(`la colonne \\u00ab object::% \\u00bb n'existe pas` on PG)
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
* Simplify migration by not moving the actual files and just updating
the DB
* Don't store the storageid in the preview table as it is not needed
* Start adding tests
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
The new oc_previews table is optimized for storing previews and should
decrease significantly the space taken by previews in the filecache
table.
This attend to reuse the IObjectStore abstraction over S3/Swift/Azure
but currently only support one single bucket configuration.
Signed-off-by: Carl Schwan <carl.schwan@nextclound.com>
The hardcoded connection timeout of 5 seconds may not be enough in some
cases, so now it is got from the ObjectStore arguments in Nextcloud
configuration, falling back to 5 if not set.
The connection timeout is set in seconds, but decimal precision can be
used for subsecond accuracy (for example, 4.2 for 4200 milliseconds).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
When using S3 as primary storage, transferring ownership with the `--move` option fail with the following error:
`SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '8-45b963397aa40d4a0063e0d85e4fe7a1' for key 'fs_storage_path_hash'`
The `--move` option moves the entire home folder from one account to another.
The error means that the move failed because the destination folder already exist in `oc_filecache`.
- With S3 as primary storage, folders only exists as entries in `oc_filecache`.
- With S3 as primary storage, `moveFromStorage(...)` only moves the cache entry, as nothing needs to be moved on disk. This cache move does not delete potentially pre-existing destination folder.
- With Local storage, `moveFromStorage(...)` calls `rename(...)` which delete pre-existing folder.
- `transfer(...)`: 687a4d9ac7/apps/files/lib/Service/OwnershipTransferService.php (L112)
- `oneTimeUserSetup(...)`: 687a4d9ac7/lib/private/Files/SetupManager.php (L261-L262)
- `mkdir(...)`: 687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L91-L135)
- `moveFromStorage(...)`: 687a4d9ac7/lib/private/Files/ObjectStore/ObjectStoreStorage.php (L635-L636)
Delete pre-existing folder in `moveFromStorage(...)`
Signed-off-by: Louis Chemineau <louis@chmn.me>
