Alvaro Herrera 1df0a914d5 Fix WAL replay in presence of an incomplete record ... Physical replication always ships WAL segment files to replicas once they are complete. This is a problem if one WAL record is split across a segment boundary and the primary server crashes before writing down the segment with the next portion of the WAL record: WAL writing after crash recovery would happily resume at the point where the broken record started, overwriting that record ... but any standby or backup may have already received a copy of that segment, and they are not rewinding. This causes standbys to stop following the primary after the latter crashes: LOG: invalid contrecord length 7262 at A8/D9FFFBC8 because the standby is still trying to read the continuation record (contrecord) for the original long WAL record, but it is not there and it will never be. A workaround is to stop the replica, delete the WAL file, and restart it -- at which point a fresh copy is brought over from the primary. But that's pretty labor intensive, and I bet many users would just give up and re-clone the standby instead. A fix for this problem was already attempted in commit 515e3d84a0, but it only addressed the case for the scenario of WAL archiving, so streaming replication would still be a problem (as well as other things such as taking a filesystem-level backup while the server is down after having crashed), and it had performance scalability problems too; so it had to be reverted. This commit fixes the problem using an approach suggested by Andres Freund, whereby the initial portion(s) of the split-up WAL record are kept, and a special type of WAL record is written where the contrecord was lost, so that WAL replay in the replica knows to skip the broken parts. With this approach, we can continue to stream/archive segment files as soon as they are complete, and replay of the broken records will proceed across the crash point without a hitch. Because a new type of WAL record is added, users should be careful to upgrade standbys first, primaries later. Otherwise they risk the standby being unable to start if the primary happens to write such a record. A new TAP test that exercises this is added, but the portability of it is yet to be seen. This has been wrong since the introduction of physical replication, so backpatch all the way back. In stable branches, keep the new XLogReaderState members at the end of the struct, to avoid an ABI break. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: Nathan Bossart <bossartn@amazon.com> Discussion: https://postgr.es/m/202108232252.dh7uxf6oxwcy@alvherre.pgsql		2021-09-29 11:21:51 -03:00
..
editors	Make Emacs perl-mode indent more like perltidy.	2019-01-13 11:32:31 -08:00
findoidjoins	Create the infrastructure for planner support functions.	2019-02-09 18:08:48 -05:00
ifaddrs	Update copyright for 2019	2019-01-02 12:44:25 -05:00
make_diff	Remove useless whitespace at end of lines	2010-11-23 22:34:55 +02:00
msvc	Add fallback implementation for setenv()	2021-06-01 09:27:31 +09:00
perlcheck	Fix whitespace	2018-06-27 08:03:54 +02:00
pginclude	Make cpluspluscheck more portable.	2019-06-02 13:45:01 -04:00
pgindent	Fix WAL replay in presence of an incomplete record	2021-09-29 11:21:51 -03:00
ccsym	tools/ccsym: update for modern versions of gcc	2015-01-20 13:02:58 -05:00
check_bison_recursion.pl	Update copyright for 2019	2019-01-02 12:44:25 -05:00
codelines	Fix remaining stray references to CVS.	2010-09-22 19:51:39 -04:00
copyright.pl	Update copyright for 2019	2019-01-02 12:44:25 -05:00
FAQ2txt	Remove cvs keywords from all files.	2010-09-20 22:08:53 +02:00
find_badmacros	Remove cvs keywords from all files.	2010-09-20 22:08:53 +02:00
find_static	Fix omission of -X (--no-psqlrc) in some psql invocations.	2015-12-28 11:46:43 -05:00
find_typedef	Refer to OS X as "macOS", except for the port name which is still "darwin".	2016-09-25 15:40:57 -04:00
fix-old-flex-code.pl	Replace @postgresql.org with @lists.postgresql.org for mailinglists	2019-01-19 19:06:35 +01:00
gen_keywordlist.pl	Initial pgperltidy run for v12.	2019-05-22 13:36:19 -04:00
git-external-diff	Preserve information on use of git-external-diff	2018-05-24 23:45:31 +09:30
git_changelog	Stamp HEAD as 12devel	2018-06-30 12:47:59 -04:00
make_ctags	Improve missing-program error handling in make_ctags and make_etags.	2019-01-13 13:33:50 -05:00
make_etags	Improve missing-program error handling in make_ctags and make_etags.	2019-01-13 13:33:50 -05:00
make_mkid	Add another pgdefine path check, and a cvs-git change.	2011-08-26 21:52:35 -04:00
PerfectHash.pm	Make our perfect hash functions be valid C++.	2019-05-31 10:40:00 -04:00
pgtest	pgtest: run clean, build, and check stages separately	2018-07-28 15:34:06 -04:00
RELEASE_CHANGES	Create a script that can renumber manually-assigned OIDs.	2019-03-12 10:50:48 -04:00
testint128.c	Update copyright for 2019	2019-01-02 12:44:25 -05:00
valgrind.supp	Avoid need for valgrind suppressions for pg_atomic_init_u64 on some platforms.	2020-06-08 20:02:52 -07:00
version_stamp.pl	Remove bug.template file	2019-05-20 08:58:21 +02:00
win32tzlist.pl	Update Windows timezone name list to include currently-known zones.	2020-04-24 17:53:23 -04:00