This backpatches commits beb4e9ba16 and 1fb17b1903 (originally
appearing in previously in REL_15_STABLE) to REL_14_STABLE. Performance
of the WAL archiver can become pretty critical at times, and reports
exist of users getting in serious trouble (hours of downtime, loss of
replicas) because of lack of this optimization.
We'd like to backpatch these to REL_13_STABLE too, but because of the
very invasive changes made by commit d75288fb27 in the 14 timeframe,
we deem it too risky :-(
Original commit messages appear below.
Discussion: https://postgr.es/m/202411131605.m66syq5i5ucl@alvherre.pgsql
commit beb4e9ba16
Author: Robert Haas <rhaas@postgresql.org>
AuthorDate: Thu Nov 11 15:02:53 2021 -0500
Improve performance of pgarch_readyXlog() with many status files.
Presently, the archive_status directory was scanned for each file to
archive. When there are many status files, say because archive_command
has been failing for a long time, these directory scans can get very
slow. With this change, the archiver remembers several files to archive
during each directory scan, speeding things up.
To ensure timeline history files are archived as quickly as possible,
XLogArchiveNotify() forces the archiver to do a new directory scan as
soon as the .ready file for one is created.
Nathan Bossart, per a long discussion involving many people. It is
not clear to me exactly who out of all those people reviewed this
particular patch.
Discussion: http://postgr.es/m/CA+TgmobhAbs2yabTuTRkJTq_kkC80-+jw=pfpypdOJ7+gAbQbw@mail.gmail.com
Discussion: http://postgr.es/m/620F3CE1-0255-4D66-9D87-0EADE866985A@amazon.com
commit 1fb17b1903
Author: Tom Lane <tgl@sss.pgh.pa.us>
AuthorDate: Wed Dec 29 17:02:50 2021 -0500
Fix issues in pgarch's new directory-scanning logic.
The arch_filenames[] array elements were one byte too small, so that
a maximum-length filename would get corrupted if another entry
were made after it. (Noted by Thomas Munro, fix by Nathan Bossart.)
Move these arrays into a palloc'd struct, so that we aren't wasting
a few kilobytes of static data in each non-archiver process.
Add a binaryheap_reset() call to make it plain that we start the
directory scan with an empty heap. I don't think there's any live
bug of that sort, but it seems fragile, and this is very cheap
insurance.
Cleanup for commit beb4e9ba1, so no back-patch needed.
Discussion: https://postgr.es/m/CA+hUKGLHAjHuKuwtzsW7uMJF4BVPcQRL-UMZG_HM-g0y7yLkUg@mail.gmail.com
4abf615cc8