Commit graph

82 commits

Author SHA1 Message Date
Christoph Wurst
d52ebaa7cd
Merge pull request #43297 from nextcloud/fix/db/dirty-read-cool-off
fix(db): Let dirty writes cool off
2024-02-19 10:53:05 +01:00
Christoph Wurst
1f46e4b854
fix(db): Let dirty writes cool off
We can assume that after a few seconds a read will be clean again.
This is helpful for false warnings in long running processes.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-02-19 10:13:24 +01:00
Christoph Wurst
cf17a20835 fix(db): Give dirty read exception a message
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-02-15 09:14:47 +01:00
Christoph Wurst
aef28e191a
fix(db): Unify long transaction log/exception message
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-02-13 08:17:49 +01:00
Christoph Wurst
97152de9bf
fix(db): Execute dirty reads on the primary node
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-01-30 12:17:39 +01:00
Christoph Wurst
911ab393c0
feat(db): Make dirty query logging available in production
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-01-30 10:31:03 +01:00
Christoph Wurst
033a654389
Merge pull request #42929 from nextcloud/fix/db/transacted-read-not-dirty
fix(db): Do not log transacted reads as dirty read
2024-01-23 09:34:20 +01:00
Christoph Wurst
ab0d7c007d fix(db): Remove very verbose dirty query logs
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-01-18 18:33:07 +01:00
Christoph Wurst
f54b08c224
fix(db): Do not log transacted reads as dirty read
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2024-01-18 13:37:36 +01:00
Julius Härtl
296096e069
fix: Add reconnect check in case of timeouts on the db side
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2024-01-12 14:38:01 +01:00
Julius Härtl
c17c42a0b2 feat: First attempt to track dirty tables after writes and switch back to replicas if reads go to other tables
Signed-off-by: Julius Härtl <jus@bitgrid.net>

debug: error log

Signed-off-by: Julius Härtl <jus@bitgrid.net>
2024-01-12 11:02:50 +01:00
Julius Härtl
3e60092b7d feat: Add logging for transaction time
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2024-01-12 11:02:50 +01:00
Julius Härtl
79c4986354
enh: Implement PrimaryReadReplicaConnection
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2023-12-16 20:27:37 +01:00
Joas Schilling
aa5f037af7
chore: apply changes from Nextcloud coding standards 1.1.1
Signed-off-by: Joas Schilling <coding@schilljs.com>
Signed-off-by: Benjamin Gaussorgues <benjamin.gaussorgues@nextcloud.com>
2023-11-23 10:36:13 +01:00
Joas Schilling
160298c556
fix(mysql): Remove custom MySQL workaround from 2015
Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-10-06 15:29:25 +02:00
Joas Schilling
b202b139dd
fix(postgres): Remove old Postgres 9.4 workaround
Postgres 10 is the minimum in the meantime
and doctrine/dbal fixed this in 2.6.0 already
ref https://github.com/doctrine/dbal/pull/2614

Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-10-06 15:29:25 +02:00
Joas Schilling
ad839dbb0a
fix(sqlite): Remove no longer required autoincrement fix
- I installed current master and exported the schema as SQL
- Then I went to this branch, removed the content of the run() method (so made it no-op)
- I installed again and exported the schema as SQL
- The files are exactly the same, so whatever we tried to fix was fixed since 2015 in doctrine dbal

Signed-off-by: Joas Schilling <coding@schilljs.com>
2023-10-06 15:29:25 +02:00
Carl Schwan
e966cfbc4b
Cleanup plsam issues in DB/ContactsManager and Console
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2023-09-05 09:20:26 +02:00
Arthur Schiwon
997c2a2a79
fix DBAL exception handling in setValues
This seems to be a left over after abstracting DBAL. Nowadays,
IQueryBuilder::executeStatement() only throws a \OCP\DB\Exception, where
previously original DBAL exceptions where thrown. These are now wrapped,
the orignal classes are now mapped to a reason.

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2023-03-31 17:01:17 +02:00
Côme Chilliet
8a16e7e8a1
Fix more psalm issues in OC\DB
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-11-21 11:30:20 +01:00
Côme Chilliet
378630f8a0
Fix psalm issues following doctrine/dbal bump
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-11-21 10:56:15 +01:00
Côme Chilliet
a2a7150d6d
Migrate Repair events to OCP\EventDispatcher\Event
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-08-25 16:15:48 +02:00
Robin Appelman
1537c3d23b
record backtrace when profiling db requests
Signed-off-by: Robin Appelman <robin@icewind.nl>
2022-04-04 18:12:30 +02:00
Carl Schwan
7d272c54d0 Add a built-in profiler inside Nextcloud
The webui is provided by a seperate application named profiler

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-04-04 10:28:26 +02:00
Côme Chilliet
ea23523c70
Adapt more code to migration to LoggerInterface
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-03-24 16:21:25 +01:00
Joas Schilling
0acd4b5f82
Merge pull request #31235 from nextcloud/techdebt/noid/extract-request-id
Extract request id handling to dedicated class so it can be injected without DB dependency
2022-03-22 12:08:45 +01:00
Julius Härtl
83717a7800
Add event logging for db and redis connection
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-02-28 11:24:41 +01:00
Louis Chemineau
405c5eb813 Add --dry-run option for db:add-missing-* cmd
Signed-off-by: Louis Chemineau <louis@chmn.me>
2022-02-24 16:42:08 +01:00
Joas Schilling
1c138d3ae2
Allow to prefix the Query log with the request id
Signed-off-by: Joas Schilling <coding@schilljs.com>
2022-02-23 11:01:58 +01:00
Carl Schwan
52760a95d9
Check if directoy is writable instead of possibly missing file
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2021-12-20 15:36:52 +01:00
Joas Schilling
168c673755
Allow to log DB, redis and LDAP requests into files
Signed-off-by: Joas Schilling <coding@schilljs.com>
2021-12-01 22:33:41 +01:00
Joas Schilling
5bb49feef8
Allow NULL as well for limit, not integer only
Signed-off-by: Joas Schilling <coding@schilljs.com>
2021-11-15 18:51:28 +01:00
John Molakvoæ (skjnldsv)
215aef3cbd
Update php licenses
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +02:00
Morris Jobke
ab48d5e8cb
Cleanup unneeded code around database.xml
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2021-03-24 22:15:44 +01:00
Robin Appelman
111fa47f10
remove ReconnectWrapper
dbal now handles it's own reconnections: https://github.com/doctrine/dbal/blob/3.0.x/UPGRADE.md#the-pingableconnection-interface-is-removed

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-02-09 16:40:09 +01:00
Christoph Wurst
2c9cdc1cdb
Add our own DB exception abstraction
Right now our API exports the Doctrine/dbal exception. As we've seen
with the dbal 3 upgrade, the leakage of 3rdparty types is problematic as
a dependency update means lots of work in apps, due to the direct
dependency of what Nextcloud ships. This breaks this dependency so that
apps only need to depend on our public API. That API can then be vendor
(db lib) agnostic and we can work around future deprecations/removals in
dbal more easily.

Right now the type of exception thrown is transported as "reason". For
the more popular types of errors we can extend the new exception class
and allow apps to catch specific errors only. Right now they have to
catch-check-rethrow. This is not ideal, but better than the dependnecy
on dbal.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2021-01-12 16:38:23 +01:00
Christoph Wurst
8b64e92b92
Bump doctrine/dbal from 2.12.0 to 3.0.0
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2021-01-08 11:45:19 +01:00
Christoph Wurst
d89a75be0b
Update all license headers for Nextcloud 21
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-16 18:48:22 +01:00
Joas Schilling
fbda2d1d25
Don't try to update on NotNullConstraintViolationException, only on unique or foreign key
Signed-off-by: Joas Schilling <coding@schilljs.com>
2020-11-10 15:36:27 +01:00
Joas Schilling
1aa9c9164d
Fix comparing the empty string for global credentials
Signed-off-by: Joas Schilling <coding@schilljs.com>
2020-11-10 15:36:26 +01:00
Julius Härtl
f49630f48c
Replace prefix in executeUpdate
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-11-03 11:03:02 +01:00
Roeland Jago Douma
fe46149560
Psalm fixes
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-10-30 19:27:53 +01:00
Roeland Jago Douma
28be188fbc
Update internal connection class
executeUpdate is deprecated in favor of executeStatement.
We overwrote the old one hence the prefix was no longer replaced.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-10-30 19:27:53 +01:00
Morris Jobke
f12e7e0e95
Fetch the logger and system config once for all query builder instances
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-10-29 22:35:06 +01:00
Christoph Wurst
d9015a8c94
Format code to a single space around binary operators
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-10-05 20:25:24 +02:00
Joas Schilling
3212c074b9
Log the number of queries built and executed
Signed-off-by: Joas Schilling <coding@schilljs.com>
2020-09-25 14:55:53 +02:00
Morris Jobke
99c9423766
Remove @suppress SqlInjectionChecker
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-09-16 15:53:56 +02:00
Christoph Wurst
caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst
14c996d982
Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Christoph Wurst
44577e4345
Remove trailing and in between spaces
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:07:47 +02:00