mirror of
https://github.com/postgres/postgres.git
synced 2026-04-10 11:37:33 -04:00
This allows data checksums to be enabled, or disabled, in a running cluster without restricting access to the cluster during processing. Data checksums could prior to this only be enabled during initdb or when the cluster is offline using the pg_checksums app. This commit introduce functionality to enable, or disable, data checksums while the cluster is running regardless of how it was initialized. A background worker launcher process is responsible for launching a dynamic per-database background worker which will mark all buffers dirty for all relation with storage in order for them to have data checksums calculated on write. Once all relations in all databases have been processed, the data_checksums state will be set to on and the cluster will at that point be identical to one which had data checksums enabled during initialization or via offline processing. When data checksums are being enabled, concurrent I/O operations from backends other than the data checksums worker will write the checksums but not verify them on reading. Only when all backends have absorbed the procsignalbarrier for setting data_checksums to on will they also start verifying checksums on reading. The same process is repeated during disabling; all backends write checksums but do not verify them until the barrier for setting the state to off has been absorbed by all. This in-progress state is used to ensure there are no false negatives (or positives) due to reading a checksum which is not in sync with the page. A new testmodule, test_checksums, is introduced with an extensive set of tests covering both online and offline data checksum mode changes. The tests which run concurrent pgbdench during online processing are gated behind the PG_TEST_EXTRA flag due to being very expensive to run. Two levels of PG_TEST_EXTRA flags exist to turn on a subset of the expensive tests, or the full suite of multiple runs. This work is based on an earlier version of this patch which was reviewed by among others Heikki Linnakangas, Robert Haas, Andres Freund, Tomas Vondra, Michael Banck and Andrey Borodin. During the work on this new version, Tomas Vondra has given invaluable assistance with not only coding and reviewing but very in-depth testing. Author: Daniel Gustafsson <daniel@yesql.se> Author: Magnus Hagander <magnus@hagander.net> Co-authored-by: Tomas Vondra <tomas@vondra.me> Reviewed-by: Tomas Vondra <tomas@vondra.me> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/CABUevExz9hUUOLnJVr2kpw9Cx=o4MCr1SVKwbupzuxP7ckNutA@mail.gmail.com Discussion: https://postgr.es/m/20181030051643.elbxjww5jjgnjaxg@alap3.anarazel.de Discussion: https://postgr.es/m/CABUevEwE3urLtwxxqdgd5O2oQz9J717ZzMbh+ziCSa5YLLU_BA@mail.gmail.com
123 lines
3.4 KiB
C
123 lines
3.4 KiB
C
/*-------------------------------------------------------------------------
|
|
* auxprocess.c
|
|
* functions related to auxiliary processes.
|
|
*
|
|
*
|
|
* Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
|
*
|
|
* IDENTIFICATION
|
|
* src/backend/postmaster/auxprocess.c
|
|
*-------------------------------------------------------------------------
|
|
*/
|
|
#include "postgres.h"
|
|
|
|
#include <unistd.h>
|
|
#include <signal.h>
|
|
|
|
#include "access/xlog.h"
|
|
#include "miscadmin.h"
|
|
#include "pgstat.h"
|
|
#include "postmaster/auxprocess.h"
|
|
#include "storage/condition_variable.h"
|
|
#include "storage/ipc.h"
|
|
#include "storage/proc.h"
|
|
#include "storage/procsignal.h"
|
|
#include "utils/memutils.h"
|
|
#include "utils/ps_status.h"
|
|
#include "utils/wait_event.h"
|
|
|
|
|
|
static void ShutdownAuxiliaryProcess(int code, Datum arg);
|
|
|
|
|
|
/*
|
|
* AuxiliaryProcessMainCommon
|
|
*
|
|
* Common initialization code for auxiliary processes, such as the bgwriter,
|
|
* walwriter, walreceiver, and the startup process.
|
|
*/
|
|
void
|
|
AuxiliaryProcessMainCommon(void)
|
|
{
|
|
Assert(IsUnderPostmaster);
|
|
|
|
/* Release postmaster's working memory context */
|
|
if (PostmasterContext)
|
|
{
|
|
MemoryContextDelete(PostmasterContext);
|
|
PostmasterContext = NULL;
|
|
}
|
|
|
|
init_ps_display(NULL);
|
|
|
|
Assert(GetProcessingMode() == InitProcessing);
|
|
|
|
IgnoreSystemIndexes = true;
|
|
|
|
/*
|
|
* As an auxiliary process, we aren't going to do the full InitPostgres
|
|
* pushups, but there are a couple of things that need to get lit up even
|
|
* in an auxiliary process.
|
|
*/
|
|
|
|
/*
|
|
* Create a PGPROC so we can use LWLocks and access shared memory.
|
|
*/
|
|
InitAuxiliaryProcess();
|
|
|
|
BaseInit();
|
|
|
|
ProcSignalInit(NULL, 0);
|
|
|
|
/*
|
|
* Initialize a local cache of the data_checksum_version, to be updated by
|
|
* the procsignal-based barriers.
|
|
*
|
|
* This intentionally happens after initializing the procsignal, otherwise
|
|
* we might miss a state change. This means we can get a barrier for the
|
|
* state we've just initialized - but it can happen only once.
|
|
*
|
|
* The postmaster (which is what gets forked into the new child process)
|
|
* does not handle barriers, therefore it may not have the current value
|
|
* of LocalDataChecksumVersion value (it'll have the value read from the
|
|
* control file, which may be arbitrarily old).
|
|
*
|
|
* NB: Even if the postmaster handled barriers, the value might still be
|
|
* stale, as it might have changed after this process forked.
|
|
*/
|
|
InitLocalDataChecksumState();
|
|
|
|
/*
|
|
* Auxiliary processes don't run transactions, but they may need a
|
|
* resource owner anyway to manage buffer pins acquired outside
|
|
* transactions (and, perhaps, other things in future).
|
|
*/
|
|
CreateAuxProcessResourceOwner();
|
|
|
|
|
|
/* Initialize backend status information */
|
|
pgstat_beinit();
|
|
pgstat_bestart_initial();
|
|
pgstat_bestart_final();
|
|
|
|
/* register a before-shutdown callback for LWLock cleanup */
|
|
before_shmem_exit(ShutdownAuxiliaryProcess, 0);
|
|
|
|
SetProcessingMode(NormalProcessing);
|
|
}
|
|
|
|
/*
|
|
* Begin shutdown of an auxiliary process. This is approximately the equivalent
|
|
* of ShutdownPostgres() in postinit.c. We can't run transactions in an
|
|
* auxiliary process, so most of the work of AbortTransaction() is not needed,
|
|
* but we do need to make sure we've released any LWLocks we are holding.
|
|
* (This is only critical during an error exit.)
|
|
*/
|
|
static void
|
|
ShutdownAuxiliaryProcess(int code, Datum arg)
|
|
{
|
|
LWLockReleaseAll();
|
|
ConditionVariableCancelSleep();
|
|
pgstat_report_wait_end();
|
|
}
|