postgresql/contrib
Thomas Munro fb540b6aa5 Fix corruption when relation truncation fails.
RelationTruncate() does three things, while holding an
AccessExclusiveLock and preventing checkpoints:

1. Logs the truncation.
2. Drops buffers, even if they're dirty.
3. Truncates some number of files.

Step 2 could previously be canceled if it had to wait for I/O, and step
3 could and still can fail in file APIs.  All orderings of these
operations have data corruption hazards if interrupted, so we can't give
up until the whole operation is done.  When dirty pages were discarded
but the corresponding blocks were left on disk due to ERROR, old page
versions could come back from disk, reviving deleted data (see
pgsql-bugs #18146 and several like it).  When primary and standby were
allowed to disagree on relation size, standbys could panic (see
pgsql-bugs #18426) or revive data unknown to visibility management on
the primary (theorized).

Changes:

 * WAL is now unconditionally flushed first
 * smgrtruncate() is now called in a critical section, preventing
   interrupts and causing PANIC on file API failure
 * smgrtruncate() has a new parameter for existing fork sizes,
   because it can't call smgrnblocks() itself inside a critical section

The changes apply to RelationTruncate(), smgr_redo() and
pg_truncate_visibility_map().  That last is also brought up to date with
other evolutions of the truncation protocol.

The VACUUM FileTruncate() failure mode had been discussed in older
reports than the ones referenced below, with independent analysis from
many people, but earlier theories on how to fix it were too complicated
to back-patch.  The more recently invented cancellation bug was
diagnosed by Alexander Lakhin.  Other corruption scenarios were spotted
by me while iterating on this patch and earlier commit 75818b3a.

Back-patch to all supported releases.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reported-by: rootcause000@gmail.com
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/18146-04e908c662113ad5%40postgresql.org
Discussion: https://postgr.es/m/18426-2d18da6586f152d6%40postgresql.org
2024-12-20 23:57:34 +13:00
..
adminpack Use has_privs_for_roles for predefined role checks 2022-03-28 15:10:04 -04:00
amcheck Backport BackgroundPsql perl test module 2024-06-27 19:01:13 +03:00
auth_delay Disallow setting bogus GUCs within an extension's reserved namespace. 2022-02-21 14:10:43 -05:00
auto_explain Make new auto_explain test safe for log_error_verbosity = verbose. 2022-07-31 12:29:44 -04:00
basebackup_to_shell basebackup_to_shell: Check for a NULL return from OpenPipeStream. 2023-04-12 11:51:09 -04:00
basic_archive Replace durable_rename_excl() by durable_rename(), take two 2024-12-20 09:11:03 +09:00
bloom Count contrib/bloom index scans in pgstat view. 2024-11-12 20:57:39 -05:00
bool_plperl Fix broken ruleutils support for function TRANSFORM clauses. 2021-01-25 13:03:43 -05:00
btree_gin btree_gin: Fix calculation of leftmost interval value. 2023-10-29 11:14:35 +00:00
btree_gist Pre-beta mechanical code beautification. 2022-05-12 15:17:30 -04:00
citext Skip citext_utf8 test on Windows. 2024-08-02 10:13:16 +12:00
cube Indent C code in flex and bison files 2022-05-13 07:17:29 +02:00
dblink Rename SetSingleFuncCall() to InitMaterializedSRF() 2022-10-18 10:22:40 +09:00
dict_int Update copyright for 2022 2022-01-07 19:04:57 -05:00
dict_xsyn Update copyright for 2022 2022-01-07 19:04:57 -05:00
earthdistance Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
file_fdw Add header matching mode to COPY FROM 2022-03-30 09:02:31 +02:00
fuzzystrmatch Ensure Soundex difference() function handles empty input sanely. 2023-05-16 10:53:42 -04:00
hstore hstore: Tighten key/value parsing check for whitespaces 2023-06-12 09:14:13 +09:00
hstore_plperl Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
hstore_plpython In hstore_plpython, avoid crashing when return value isn't a mapping. 2023-04-27 11:55:06 -04:00
intagg Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
intarray Fix integer-overflow problem in intarray's g_int_decompress(). 2024-01-07 15:19:50 -05:00
isn Update copyright for 2022 2022-01-07 19:04:57 -05:00
jsonb_plperl Expose internal function for converting int64 to numeric 2020-09-09 20:16:28 +02:00
jsonb_plpython plpython: Code cleanup related to removal of Python 2 support. 2022-03-07 18:30:28 -08:00
lo Fix bogus CALLED_AS_TRIGGER() defenses. 2020-04-03 11:24:56 -04:00
ltree Validate ltree siglen GiST option to be int-aligned 2023-04-23 14:30:51 +03:00
ltree_plpython plpython: Code cleanup related to removal of Python 2 support. 2022-03-07 18:30:28 -08:00
oid2name Improve frontend error logging style. 2022-04-08 14:55:14 -04:00
old_snapshot Update copyright for 2022 2022-01-07 19:04:57 -05:00
pageinspect Fix contrib/pageinspect's test for sequences. 2024-09-13 10:16:40 -05:00
passwordcheck Remove non-functional code for unloading loadable modules. 2022-05-11 15:30:30 -04:00
pg_buffercache Remove support for upgrading extensions from "unpackaged" state. 2020-02-19 16:59:14 -05:00
pg_freespacemap Avoid instabilities with the regression tests of pg_freespacemap 2022-03-29 13:52:49 +09:00
pg_prewarm Add a new shmem_request_hook hook. 2022-05-13 09:31:06 -04:00
pg_stat_statements pg_stat_statements: Fix second comment related to entry resets 2023-06-29 09:17:30 +09:00
pg_surgery Remove xloginsert.h from xlog.h 2022-01-30 12:25:24 -03:00
pg_trgm Fix arrays comparison in CompareOpclassOptions() 2024-11-12 01:54:30 +02:00
pg_visibility Fix corruption when relation truncation fails. 2024-12-20 23:57:34 +13:00
pg_walinspect Limit memory usage of pg_walinspect functions. 2023-02-20 11:29:31 -08:00
pgcrypto pgcrypto: Fix check for buffer size 2024-01-30 11:15:46 +01:00
pgrowlocks Adjust the order of the prechecks in pgrowlocks() 2023-10-31 16:43:01 +13:00
pgstattuple Reintroduce support for sequences in pgstattuple and pageinspect. 2024-09-12 16:31:29 -05:00
postgres_fdw Fix Y2038 issues with MyStartTime. 2024-10-07 13:51:03 -05:00
seg Replace last PushOverrideSearchPath() call with set_config_option(). 2023-05-08 06:14:11 -07:00
sepgsql Adjust sepgsql expected output for 681d9e462 et al. 2023-05-08 11:24:47 -04:00
spi Remove extraneous blank lines before block-closing braces 2022-04-13 19:16:02 +02:00
sslinfo contrib/sslinfo needs a fix too to make hamerkop happy. 2021-11-07 11:33:53 -05:00
start-scripts Remove contrib/start-scripts/osx/. 2017-11-17 12:53:20 -05:00
tablefunc Remove all traces of tuplestore_donestoring() in the C code 2022-02-17 09:52:02 +09:00
tcn Update copyright for 2022 2022-01-07 19:04:57 -05:00
test_decoding Fix invalidation of local pgstats references for entry reinitialization 2024-12-09 10:46:10 +09:00
tsm_system_rows Update copyright for 2022 2022-01-07 19:04:57 -05:00
tsm_system_time Update copyright for 2022 2022-01-07 19:04:57 -05:00
unaccent unaccent: Tweak value of PYTHON when building without Python support 2023-09-27 14:41:21 +09:00
uuid-ossp Reject bogus output from uuid_create(3). 2022-09-09 12:41:36 -04:00
vacuumlo Improve frontend error logging style. 2022-04-08 14:55:14 -04:00
xml2 Replace usages of xmlXPathCompile() with xmlXPathCtxtCompile(). 2024-09-15 13:33:09 -04:00
contrib-global.mk Respect TEMP_CONFIG when pg_regress_check and friends are called 2016-02-27 12:28:21 -05:00
Makefile Add contrib/pg_walinspect. 2022-04-08 00:26:44 -07:00
README Rename 'gmake' to 'make' in docs and recommended commands 2014-02-12 17:29:19 -05:00

The PostgreSQL contrib tree
---------------------------

This subtree contains porting tools, analysis utilities, and plug-in
features that are not part of the core PostgreSQL system, mainly
because they address a limited audience or are too experimental to be
part of the main source tree.  This does not preclude their
usefulness.

User documentation for each module appears in the main SGML
documentation.

When building from the source distribution, these modules are not
built automatically, unless you build the "world" target.  You can
also build and install them all by running "make all" and "make
install" in this directory; or to build and install just one selected
module, do the same in that module's subdirectory.

Some directories supply new user-defined functions, operators, or
types.  To make use of one of these modules, after you have installed
the code you need to register the new SQL objects in the database
system by executing a CREATE EXTENSION command.  In a fresh database,
you can simply do

    CREATE EXTENSION module_name;

See the PostgreSQL documentation for more information about this
procedure.