Commit graph

242 commits

Author SHA1 Message Date
Julian Brost
9946fa5902
Merge pull request #767 from Icinga/icingadb-migrate-null-name-i766
icingadb-migrate: Mitigate NULL names in {comment,downtime}_query
2024-07-31 10:29:16 +02:00
Alexander A. Klimov
b6839ce9f0 cmd/icingadb-migrate: replace magic number 20 with sha1.Size 2024-07-09 13:30:08 +00:00
Alvar Penning
b7483eb6f8
icingadb-migrate: Verify env ID length
An env ID with the wrong length, either due to a copy-paste error or
human error during testing, results in a SQL CHECK CONSTRAINT violation
that is retried multiple times until it finally fails.
2024-07-09 13:24:01 +02:00
Alvar Penning
59685cd623
icingadb-migrate: Mitigate NULL names in {comment,downtime}_query
Historical data from an older Icinga 2 installation contained NULL
values for the name column in some rows of the icinga_commenthistory and
icinga_downtimehistory tables.

Normally this field contains something like
${name1}!${name2}!${unique_value} where the $unique_value is based on a
timestamp for older entries and a UUID for newer ones. For a concrete
example, this could be "host.example.com!ping6!123…".

Unfortunately, using an empty string for these NULL values will cause an
error later because the new primary key will be calculated based on it.
Therefore, a new deterministic name is generated based on the primary
keys and the known name1 and name2 values.

Closes #766.
2024-07-09 13:05:36 +02:00
Eric Lippmann
7c068d4adf Use icinga-go-library 2024-05-24 09:56:28 +02:00
Eric Lippmann
a73940e60a Introduce GetAddr() for database#DB and redis#Client 2024-05-23 12:42:28 +02:00
Eric Lippmann
9f8354466a config#ParseFlags(): Parse result into value pointed to by the passed argument 2024-05-23 10:08:48 +02:00
Eric Lippmann
957e610508 Replace utils.Fatal() with utils.PrintErrorThenExit() 2024-05-23 09:46:25 +02:00
Eric Lippmann
7b73b8788a Introduce logging#NewLoggingFromConfig() 2024-05-22 11:51:22 +02:00
Eric Lippmann
c070615e64 Move Redis related code to redis 2024-05-22 11:51:22 +02:00
Eric Lippmann
8beb8616ad Move database related code from internal to database 2024-05-22 11:51:22 +02:00
Eric Lippmann
1ddaa70aab Move special types to icingadb/types 2024-05-22 11:51:21 +02:00
Eric Lippmann
9eddcac7fb icingadb-migrate: Don't alias types import 2024-05-22 11:51:21 +02:00
Eric Lippmann
f54a142815 Move icingadb#DB to database#DB 2024-05-22 11:51:21 +02:00
Eric Lippmann
490c18fd5f Move database#DB.CheckSchema() to icingadb#CheckSchema() 2024-05-22 11:51:21 +02:00
Eric Lippmann
ed69f42462 Move utils#TableName() to database#TableName() 2024-05-22 11:51:21 +02:00
Eric Lippmann
75501e11f8 Move database related contracts to database/contracts 2024-05-22 11:51:21 +02:00
Eric Lippmann
7b537661ee Move icingadb/objectpacker to objectpacker 2024-05-22 11:51:21 +02:00
Eric Lippmann
9e70aa808c Move config to internal/config 2024-05-22 11:51:21 +02:00
Alexander A. Klimov
2681634105 Unify check attempt data type to uint32 already used somewhere
A float isn't necessary as in Icinga 2 Checkable#max_check_attempts and
check_attempt are ints. But uint8 isn't enough for e.g. 1 check/s to get
HARD after 5m (300s > 255).
2024-04-08 16:01:53 +02:00
Julian Brost
d9dc16d54a
Merge pull request #711 from Icinga/drop-custom-driver-registry
Try setting `wsrep_sync_wait` for mysql connections
2024-04-05 15:07:04 +02:00
Alvar Penning
779afd1da3 Enhance HA "Taking over", "Handing over" logging
The reason for a switch in the HA roles was not always directly clear.
This change now introduces additional debug logging, indicating the
reasoning for either taking over or handing over the HA responsibility.

First, some logic was moved from the SQL query selecting active Icinga
DB instances to Go code. This allowed distinguishing between no
available responsible instances and responsible instances with an
expired heartbeat.

As the HA's peer timeout is logically bound to the Redis timeout, it
will now reference this timeout with an additional grace timeout. Doing
so eliminates a race between a handing over and a "forceful" take over.

As the old code indicated a takeover on the fact that no other instance
is active, it will now additionally check if it is already being the
active/responsible node. In this case, the takeover logic - which will
be interrupted at a later point as the node is already responsible - can
be skipped.

Next to the additional logging messages, both the takeover and handover
channel are now transporting a string to communicate the reason instead
of an empty struct{}. By doing so, both the "Taking over" and "Handing
over" log messages are enriched with reason.

This also required a change in the suppressed logging handling of the
HA.realize method, which got its logging enabled through the shouldLog
parameter. Now, there are both recurring events, which might be
suppressed, as well as state changing events, which should be logged.
Therefore, and because the logTicker's functionality was not clear to me
on first glance, I renamed it to routineLogTicker.

While dealing with the code, some function signature documentation were
added, to ease both mine as well as the understanding of future readers.

Additionally, the error handling of the SQL query selecting active
Icinga DB instances was changed slightly to also handle wrapped
sql.ErrNoRows errors.

Closes #688.
2024-04-02 13:23:11 +02:00
Yonas Habteab
eaf9744f16 Move pkg/driver to pkg/icingadb/driver.go 2024-03-28 13:19:44 +01:00
Julian Brost
2c468302ae
Merge pull request #657 from Icinga/Flatten-no-exponent
Flatten(): render even large numbers as-is, not using scientific notation
2024-03-25 16:32:32 +01:00
Alexander A. Klimov
10afc562ce Use types.MakeString() instead of manual initialization (refactor) 2024-03-25 14:52:54 +01:00
Eric Lippmann
e31b101f4f Upgrade go-redis to v9
Co-Authored-By: Alvar Penning <alvar.penning@icinga.com>
2024-03-22 15:32:15 +01:00
Alvar Penning
3e5ca98beb cmd/icingadb: Log Icinga DB version during startup
To better recognize the Icinga DB version used in case of errors, it is
now logged at startup.

If VCS is available during build, the current commit is included.
- Dirty VCS directory:
> 2024-03-11T14:36:29.317+0100    INFO    icingadb        Starting Icinga DB daemon (1.1.1-g0e9810c-dirty)
- Clean VCS directory:
> 2024-03-11T14:38:09.664+0100    INFO    icingadb        Starting Icinga DB daemon (1.1.1-geed8589)
- Build with `-buildvcs=false`:
> 2024-03-11T14:38:56.554+0100    INFO    icingadb        Starting Icinga DB daemon (1.1.1)

Closes #689.
2024-03-11 14:46:45 +01:00
Julian Brost
336ee4a8ab
Merge pull request #554 from Icinga/553
convertFlappingRows(): fix foreign key error history -> flapping_history
2023-07-31 15:00:39 +02:00
Julian Brost
71c1d2fa4d Migration: refactor output/processing of converted entities
This commit simplifies the `icingaDbOutputStage` type to contain only one
entity slice to be insert/upsert. This allows to simplify the handling in
`migrateOneType()` by removing nested loops.

Additionally, a bit of code inside that function is outsourced into a new
`utils.ChanFromSlice()` function. This makes the body of the loop over the
insert/upsert operation (the loop using the `op` variable) simple enough so
that it can just be unrolled which saves the inline struct and slice definition
for that loop.
2023-07-31 11:10:42 +02:00
Alexander A. Klimov
78327c1f34 cmd/icingadb-migrate: ignore icinga_downtimehistory#was_started=0 rows
Such appear e.g. for scheduled but never triggered flexible Downtimes.
Not triggered = no events.
2023-07-27 17:29:33 +02:00
Julian Brost
5ea6831ebc
Merge pull request #551 from Icinga/550
icingadb-migrate: handle NULL event time columns
2023-07-25 16:38:55 +02:00
Alexander A. Klimov
a163694dc6 Config parsing: fail on unknown fields and print them
Useful against config validation or runtime failures
caused by wrong field spelling or YAML indentation.
2023-06-19 17:38:52 +02:00
Alexander A. Klimov
3d571548e8 icingadb-migrate: handle NULL event time columns
Ignore such rows.
2023-05-26 12:50:41 +02:00
Alexander A. Klimov
03b0e84808 convertFlappingRows(): fix foreign key error history -> flapping_history
Don't INSERT IGNORE everything, then UPSERT everything. Instead INSERT IGNORE
flapping_history, UPSERT flapping_history and finally INSERT IGNORE history.
2022-12-19 16:07:55 +01:00
Alexander A. Klimov
a778fc8d34 migrateOneType(): allow multiple stages of INSERT IGNORE, UPSERT
Refactor the converters so they can signal the caller to behave not as usual
i.e. not to INSERT IGNORE everything, then UPSERT everything.
2022-12-19 16:07:53 +01:00
Julian Brost
de78af332f
Merge pull request #542 from Icinga/unix-port
Log UNIX socket address w/o port number
2022-11-09 16:04:11 +01:00
Alexander A. Klimov
ab14413393 Log UNIX socket address w/o port number
E.g. not
Connecting to database at '/var/lib/mysql/mysql.sock:0'
Connecting to Redis at '/run/icingadb-redis/icingadb-redis-server.sock:6380'
2022-11-09 11:03:50 +01:00
Alexander A. Klimov
cfaee2e3d4 cmd/icingadb-migrate: avoid unnecessary config option
for columns which are NULLable anyway.
2022-11-07 12:35:31 +01:00
Alexander A. Klimov
6a5f3fd02f cmd/: rename ido2icingadb/ to icingadb-migrate/ 2022-10-26 11:18:52 +02:00
Alexander A. Klimov
0ff1728434 cmd/ido2icingadb: move docs to the main tree 2022-10-26 10:24:40 +02:00
Alexander A. Klimov
70a0bfb867 cmd/ido2icingadb: use built-in decorators for the progress bar 2022-10-24 16:44:38 +02:00
Alexander A. Klimov
649f7e0e3d cmd/ido2icingadb: build cache from [0,toId], not just [fromId,toId]
Just to be sure.
2022-10-14 16:58:14 +02:00
Alexander A. Klimov
016a5ddae1 cmd/ido2icingadb: don't tell how to mess up everything 2022-10-11 16:32:22 +02:00
Alexander A. Klimov
5ebb2b0798 cmd/ido2icingadb: Keep It Simple Stupid 2022-10-11 16:32:22 +02:00
Alexander A. Klimov
f233c63209 cmd/ido2icingadb: assert schema 2022-10-11 16:32:22 +02:00
Alexander A. Klimov
f7d132ccfa Make checkDbSchema() reusable as DB#CheckSchema() 2022-10-11 16:32:22 +02:00
Alexander A. Klimov
7ff52c5c21 cmd/ido2icingadb: handle custom input data time range changes 2022-10-11 16:32:22 +02:00
Alexander A. Klimov
2c0e927a49 cmd/ido2icingadb: allow custom input data time range 2022-10-11 16:32:22 +02:00
Alexander A. Klimov
dbf394fb0d cmd/ido2icingadb: make sure not to return false-positive nil (i.e. EOF) checkpoints 2022-10-11 12:46:48 +02:00
Alexander A. Klimov
8402eb266f cmd/ido2icingadb: show ops/s 2022-10-11 12:46:48 +02:00