postgresql/contrib
Thomas Munro 2280912165 Fix corruption when relation truncation fails.
RelationTruncate() does three things, while holding an
AccessExclusiveLock and preventing checkpoints:

1. Logs the truncation.
2. Drops buffers, even if they're dirty.
3. Truncates some number of files.

Step 2 could previously be canceled if it had to wait for I/O, and step
3 could and still can fail in file APIs.  All orderings of these
operations have data corruption hazards if interrupted, so we can't give
up until the whole operation is done.  When dirty pages were discarded
but the corresponding blocks were left on disk due to ERROR, old page
versions could come back from disk, reviving deleted data (see
pgsql-bugs #18146 and several like it).  When primary and standby were
allowed to disagree on relation size, standbys could panic (see
pgsql-bugs #18426) or revive data unknown to visibility management on
the primary (theorized).

Changes:

 * WAL is now unconditionally flushed first
 * smgrtruncate() is now called in a critical section, preventing
   interrupts and causing PANIC on file API failure
 * smgrtruncate() has a new parameter for existing fork sizes,
   because it can't call smgrnblocks() itself inside a critical section

The changes apply to RelationTruncate(), smgr_redo() and
pg_truncate_visibility_map().  That last is also brought up to date with
other evolutions of the truncation protocol.

The VACUUM FileTruncate() failure mode had been discussed in older
reports than the ones referenced below, with independent analysis from
many people, but earlier theories on how to fix it were too complicated
to back-patch.  The more recently invented cancellation bug was
diagnosed by Alexander Lakhin.  Other corruption scenarios were spotted
by me while iterating on this patch and earlier commit 75818b3a.

Back-patch to all supported releases.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reported-by: rootcause000@gmail.com
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/18146-04e908c662113ad5%40postgresql.org
Discussion: https://postgr.es/m/18426-2d18da6586f152d6%40postgresql.org
2024-12-20 23:58:08 +13:00
..
adminpack Read until EOF vice stat-reported size in read_binary_file 2020-07-04 06:28:21 -04:00
amcheck Backport BackgroundPsql perl test module 2024-06-27 19:01:23 +03:00
auth_delay Update copyrights for 2020 2020-01-01 12:21:45 -05:00
auto_explain Fix ancient memory leak in contrib/auto_explain. 2021-02-02 13:49:08 -05:00
bloom Count contrib/bloom index scans in pgstat view. 2024-11-12 20:57:35 -05:00
bool_plperl Fix broken ruleutils support for function TRANSFORM clauses. 2021-01-25 13:03:11 -05:00
btree_gin btree_gin: Fix calculation of leftmost interval value. 2023-10-29 11:14:34 +00:00
btree_gist Fix results of index-only scans on btree_gist char(N) indexes. 2022-01-08 14:54:39 -05:00
citext CREATE INDEX: use the original userid for more ACL checks. 2022-06-25 09:07:45 -07:00
cube Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
dblink postgres_fdw: Fix unexpected reporting of empty message. 2021-12-03 17:37:14 +09:00
dict_int Preserve integer and float values accurately in (de)serialize_deflist. 2020-03-10 12:30:02 -04:00
dict_xsyn Remove support for upgrading extensions from "unpackaged" state. 2020-02-19 16:59:14 -05:00
earthdistance Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
file_fdw Remove leftover comments, left behind by removal of WITH OIDS. 2020-11-30 10:29:18 +02:00
fuzzystrmatch Ensure Soundex difference() function handles empty input sanely. 2023-05-16 10:53:42 -04:00
hstore hstore: Tighten key/value parsing check for whitespaces 2023-06-12 09:14:17 +09:00
hstore_plperl Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
hstore_plpython In hstore_plpython, avoid crashing when return value isn't a mapping. 2023-04-27 11:55:06 -04:00
intagg Make contrib modules' installation scripts more secure. 2020-08-10 10:44:42 -04:00
intarray Fix integer-overflow problem in intarray's g_int_decompress(). 2024-01-07 15:19:50 -05:00
isn Remove support for upgrading extensions from "unpackaged" state. 2020-02-19 16:59:14 -05:00
jsonb_plperl Mark some contrib modules as "trusted". 2020-02-13 15:02:35 -05:00
jsonb_plpython In jsonb_plpython.c, suppress warning message from gcc 10. 2020-01-30 18:26:12 -05:00
lo Fix bogus CALLED_AS_TRIGGER() defenses. 2020-04-03 11:24:56 -04:00
ltree Validate ltree siglen GiST option to be int-aligned 2023-04-23 14:00:16 +03:00
ltree_plpython Improve error messages in ltree_in and lquery_in. 2020-03-31 11:14:42 -04:00
oid2name Report the true database name on connection errors 2021-01-26 16:42:13 -03:00
pageinspect Fix contrib/pageinspect's test for sequences. 2024-09-13 10:16:40 -05:00
passwordcheck Update copyrights for 2020 2020-01-01 12:21:45 -05:00
pg_buffercache Remove support for upgrading extensions from "unpackaged" state. 2020-02-19 16:59:14 -05:00
pg_freespacemap Remove support for upgrading extensions from "unpackaged" state. 2020-02-19 16:59:14 -05:00
pg_prewarm Replace RelationOpenSmgr() with RelationGetSmgr(). 2022-11-17 16:54:30 -05:00
pg_standby pg_standby: Don't use HAVE_WORKING_LINK 2020-03-03 08:54:44 +01:00
pg_stat_statements pg_stat_statements: fetch stmt location/length before it disappears. 2022-11-01 12:48:01 -04:00
pg_trgm Fix arrays comparison in CompareOpclassOptions() 2024-11-12 01:54:38 +02:00
pg_visibility Fix corruption when relation truncation fails. 2024-12-20 23:58:08 +13:00
pgcrypto pgcrypto: Fix check for buffer size 2024-01-30 11:15:46 +01:00
pgrowlocks Adjust the order of the prechecks in pgrowlocks() 2023-10-31 16:44:00 +13:00
pgstattuple Reintroduce support for sequences in pgstattuple and pageinspect. 2024-09-12 16:31:29 -05:00
postgres_fdw Restrict accesses to non-system views and foreign tables during pg_dump. 2024-08-05 06:05:20 -07:00
seg Replace last PushOverrideSearchPath() call with set_config_option(). 2023-05-08 06:14:12 -07:00
sepgsql Adjust sepgsql expected output for 681d9e462 et al. 2023-05-08 11:24:47 -04:00
spi Remove support for upgrading extensions from "unpackaged" state. 2020-02-19 16:59:14 -05:00
sslinfo Remove support for upgrading extensions from "unpackaged" state. 2020-02-19 16:59:14 -05:00
start-scripts Remove contrib/start-scripts/osx/. 2017-11-17 12:53:20 -05:00
tablefunc Remove support for upgrading extensions from "unpackaged" state. 2020-02-19 16:59:14 -05:00
tcn Mark some contrib modules as "trusted". 2020-02-13 15:02:35 -05:00
test_decoding Fix possibility of logical decoding partial transaction changes. 2024-07-11 22:48:10 +09:00
tsm_system_rows Mark some contrib modules as "trusted". 2020-02-13 15:02:35 -05:00
tsm_system_time Mark some contrib modules as "trusted". 2020-02-13 15:02:35 -05:00
unaccent unaccent: Tweak value of PYTHON when building without Python support 2023-09-27 14:41:26 +09:00
uuid-ossp Reject bogus output from uuid_create(3). 2022-09-09 12:41:36 -04:00
vacuumlo Report the true database name on connection errors 2021-01-26 16:42:13 -03:00
xml2 Replace usages of xmlXPathCompile() with xmlXPathCtxtCompile(). 2024-09-15 13:33:09 -04:00
contrib-global.mk Respect TEMP_CONFIG when pg_regress_check and friends are called 2016-02-27 12:28:21 -05:00
Makefile Create contrib/bool_plperl to provide a bool transform for PL/Perl[U]. 2020-03-06 17:11:23 -05:00
README Rename 'gmake' to 'make' in docs and recommended commands 2014-02-12 17:29:19 -05:00

The PostgreSQL contrib tree
---------------------------

This subtree contains porting tools, analysis utilities, and plug-in
features that are not part of the core PostgreSQL system, mainly
because they address a limited audience or are too experimental to be
part of the main source tree.  This does not preclude their
usefulness.

User documentation for each module appears in the main SGML
documentation.

When building from the source distribution, these modules are not
built automatically, unless you build the "world" target.  You can
also build and install them all by running "make all" and "make
install" in this directory; or to build and install just one selected
module, do the same in that module's subdirectory.

Some directories supply new user-defined functions, operators, or
types.  To make use of one of these modules, after you have installed
the code you need to register the new SQL objects in the database
system by executing a CREATE EXTENSION command.  In a fresh database,
you can simply do

    CREATE EXTENSION module_name;

See the PostgreSQL documentation for more information about this
procedure.