The existing comment explained that a user-created slot could reuse the
same shared memory as 'local_slot' during the window between selecting a
slot to drop and locking its database, and that we therefore recheck
before dropping. It did not, however, spell out the fuller consequence:
because local_slot points to a reusable slot-array entry, its fields may
already describe a replacement slot, so the earlier drop decision and the
slot_database used for locking could relate to an unrelated slot/database.
Expand the comment to describe this, and note that the recheck prevents
us from dropping a user-created replacement slot while the residual risk
(such as briefly locking an unrelated database) is confined to the cycle
and is acceptable given the race is rare and non-fatal.
No functional change.
Author: Fujii Masao <masao.fujii@gmail.com>
Author: Xuneng Zhou <xunengzhou@gmail.com>
Author: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwGGyEDL3dh7uJ6qPsGvnq4QK_R8+U=12CaprnzwrwaLGA@mail.gmail.com
Discussion: https://postgr.es/m/CAHGQGwHqQ1PPVFfYKVxLfRyC-byRdwSN0NeaHj9SLYV97oO5cw@mail.gmail.com
The jsonpath .decimal(precision[, scale]) method built its numeric
typmod by calling numerictypmodin() through DirectFunctionCall1(), which
can throw a hard error for an incorrect set of precision and/or scale
vaulues. This breaks the silent mode supported by this function, that
should not fail.
Most of the jsonpath code uses the soft error reporting to bypass
errors, which is what this fix does by avoiding a direct use of
numerictypmodin(). Its code is refactored to use a new routine called
make_numeric_typmod_safe(), able to take an error context in input.
numerictypmodin() sets no context, mapping to its previous behavior.
The jsonpath code sets or not a context depending on the use of the
silent mode. This result leads to some nice simplifications:
numerictypmodin() feeds on an array, we can now pass directly values for
the scale and precision.
Oversight in 66ea94e8e6.
Author: Ewan Young <kdbase.hack@gmail.com>
Discussion: https://postgr.es/m/CAON2xHMaigKABiyPBBq3Sjd3gp7uWMJXnnMHt=s85V1ij3KP1w@mail.gmail.com
Backpatch-through: 17
Until now, logical replication conflicts were only written as plain text
to the server log, which is hard to query, analyze, or feed into external
monitoring and automation.
This commit adds a conflict_log_destination option to CREATE SUBSCRIPTION
and ALTER SUBSCRIPTION that controls where conflicts are recorded. It
accepts 'log' (the existing behavior), 'table', or 'all'.
When table logging is enabled ('table' or 'all'), an internal log table
named pg_conflict_log_<subid> is created automatically in a dedicated,
system-managed pg_conflict namespace. Using a separate namespace avoids
collisions with user table names and lets the table be shielded from
direct modification. The table is tied to the subscription through an
internal dependency, so it is dropped automatically when the subscription
is removed.
The conflict details, including the local and remote tuples, are stored in
JSON columns, so a single table layout can accommodate rows from tables
with different schemas. The table also records the local and remote
transaction IDs, LSNs, commit timestamps, and the conflict type, providing
a complete record for post-mortem analysis.
A per-subscription table was chosen over a single global log because it
aligns table ownership with the subscription lifecycle. This keeps
permission management simple: the subscription owner can perform the
permitted maintenance operations without the security concerns or
Row-Level Security that a shared table would require.
Because the table is system-managed, it is protected from direct
manipulation: DDL (such as ALTER, DROP, CREATE INDEX, and adding a
trigger, rule, policy, or extended statistics), use as an inheritance
parent or a foreign-key target, and manual INSERT, UPDATE, MERGE, or row
locking are all rejected. Only DELETE and TRUNCATE are permitted, so that
users can prune old conflict rows.
Conflict log tables are also excluded from publications, even those
defined with FOR ALL TABLES or FOR TABLES IN SCHEMA.
This commit only establishes the conflict log table along with its
creation, cleanup, and protection; recording the conflicts detected
during apply into the table will be handled in a follow-up commit.
Author: Dilip Kumar <dilipbalaut@gmail.com>
Author: Nisha Moond <nisha.moond412@gmail.com>
Author: Amit Kapila <akapila@postgresql.org>
Reviewed-by: Shveta Malik <shveta.malik@gmail.com>
Reviewed-by: Vignesh C <vignesh21@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAFiTN-u5D5o_AGNbHRZHaOqAMWkxLf%2BhSk_r9X3gv6HbLOB5%2Bg%40mail.gmail.com
This commit adds support for pg_stat_kind_info, that exposes at SQL
level data about the statistics kinds registered into a backend:
- Meta-data of a stats kind (built-in or custom, some properties).
- Number of entries, if tracking is enabled.
We have discussed the possibility of more fields (like shared memory
size for a single entry); this adds the minimum agreed on.
This is in spirit similar to pg_get_loaded_modules() for custom stats
kinds, this view providing detailed information about the stats kinds
when registered through shared_preload_libraries.
Bump catalog version.
Author: Tristan Partin <tristan@partin.io>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Sami Imseih <samimseih@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/DI6OFGHJ1B69.25YVDEP3BABRH@partin.io
The uuid type already has a full set of comparison operators and a
btree operator class, so it is totally ordered. min() and max() were
the only common aggregates missing for it. Add the uuid_larger() and
uuid_smaller() support functions and register the min(uuid) and
max(uuid) aggregates that use them.
uuid values are compared lexicographically over their 128 bits. For
UUIDv7, whose most significant bits encode a Unix timestamp, this
coincides with chronological order, so min() and max() return the
oldest and newest values.
Bump catalog version.
Author: Tristan Partin <tristan@partin.io>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/DJGML0T9FCDV.3VA29JLODXEHZ@partin.io
Some platforms provide a definition of unreachable() in standard C
headers, leading to a conflict with unreachable() in the new
timezone code. It seems best for our purposes to conform to what
pg_unreachable() does, so #undef away the platform version.
Reported-by: Tristan Partin <tristan@partin.io>
Discussion: https://postgr.es/m/DJNDN9UQS9GP.11L4NJ1HHE1ZJ@partin.io
If the user doesn't specify a name for an index, it's generated
based on the names chosen for the index columns (which the user
has no direct control over). For index columns that are just
columns of the base relation, the index column name is the same as
the base column name; but for index columns that are expressions,
it's less clear what to do. Up to now, what we have done is
equivalent to the heuristics used to choose SELECT output column
names, except that we fall back to "expr" not "?column?" in the
numerous cases that FigureColname doesn't know what to do with.
This is not tremendously helpful. More, it frequently leads to
collisions of generated index names, which we can handle but
only at the cost of user confusion; also there's some risk of
concurrent index creations trying to use the same name.
Let's try to do better.
Messing with the FigureColname heuristics would have a very
large blast radius, since that affects the column headings
that applications see. That doesn't seem wise, but fortunately
SQL queries are seldom directly concerned with index names.
So we should be able to change the index-name generation rules
as long as we decouple them from FigureColname.
The method used in this patch is to dig through the expression,
extract the names of Vars, the string representations of Consts,
and the names of functions, and run those together with underscores
between. Other expression node types are ignored but descended
through. We could work harder by handling more node types, but
it seems like this is likely to be sufficient to arrive at unique
index names in many cases.
Notably, this rule ignores the names of operators, for example
both "a + b" and "a * b" will be rendered as "a_b". This choice
was made to reduce the probability of having to double-quote
the index name.
I've also chosen to strip Const representations down to only
alphanumeric characters (plus non-ASCII characters, which our
parser treats as alphabetic anyway). So for example "x + 1.0"
would be represented as "x_10". This likewise avoids possible
quoting problems. I also considered limiting how many characters
we'd take from each Const, but didn't do that here.
We might tweak these rules some more after we get some experience
with this patch. It's being committed at the start of a
development cycle to provide as much time as possible to gather
feedback.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/876799.1757987810@sss.pgh.pa.us
Discussion: https://postgr.es/m/18959-f63b53b864bb1417@postgresql.org
This was moderately tedious, because upstream has been busy
since we last did this in 2020.
Notably, they changed the signatures of both tzload() and tzparse(),
which we'd unwisely exposed as API for callers to use. I concluded
that the best answer was to change them both back to "static" and
instead expose a new API function of our own choosing, pg_tzload().
That change may be a sufficient reason not to back-patch this update,
as I'd normally want to do. There's probably not a good reason for
extensions to be calling those functions, but on the other hand
there are few pressing reasons for a back-patch. The one bug we have
run into (a Valgrind uninitialized-data complaint about zic) appears
to have no field-visible consequences.
A few of the files generated by this version of zic are not
byte-for-byte the same as before, but "zdump -v" avers that
they represent the same sets of transitions.
Discussion: https://postgr.es/m/2294297.1780270682@sss.pgh.pa.us
Previously, if a base backup failed after it had started streaming
files, pg_stat_progress_basebackup could continue to show a stale
progress entry even though the backup was no longer running. This could
be observed when the client kept the replication connection open after
the error. It is normally not observable when using pg_basebackup,
because the client disconnects after the error.
The problem was that progress reporting was cleared only after
successful completion.
This commit moves the progress reporting cleanup into the progress
sink's cleanup callback so that it is cleared after both successful
and failed backups.
Backpatch to v15. v14 has the same issue, but the fix does not apply
cleanly because it lacks the base backup sink infrastructure. Since
the bug does not affect the backup itself and is normally not
observable when using pg_basebackup, skip the v14 backpatch.
Author: Chao Li <lic@highgo.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/EA1A6CD2-EFA6-462B-9A02-03003555AB4A@gmail.com
Backpatch-through: 15
Commit bc60ee860 added a connection warning after successful MD5
authentication, but only for the md5 authentication method. A role with
an MD5-encrypted password can also authenticate via the password method,
which left that path without the same deprecation warning.
Emit the MD5 deprecation connection warning after successful
password authentication as well, when the stored password is
MD5-encrypted.
Backpatch to v19, where the MD5 connection warning was introduced.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Reviewed-by: Japin Li <japinli@hotmail.com>
Discussion: https://postgr.es/m/CAHGQGwGkWfn5rtHzvdRbVk+PCefQU3gun3hc7QnaMXHFa5Bu3w@mail.gmail.com
Backpatch-through: 19
In fe_memutils.h, we have various allocation functions beginning with
either pg_ or p. The pg_ functions have a matching pg_free() for
freeing memory, while the p functions use pfree(). In some cases, we
were allocating memory with one set of functions while using the wrong
deallocation functions. This creates a tiny bit of mental overhead
when reading code. Matching up allocation and deallocation functions
makes it easier to analyze memory handling in a code path.
Author: Tristan Partin <tristan@partin.io>
Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com>
Discussion: https://www.postgresql.org/message-id/flat/DIBZE2B6SVF2.28R3EQTYJSWIG@partin.io
Fixup for commit c05dee1911. It fits better with the style and APIs
to print separate primary and a detail messages instead of one
multiline message.
Reviewed-by: Euler Taveira <euler@eulerto.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CAHut+PsvQJQnQO0KT0S2oegenkvJ8FUuY-QS5syyqmT24R2xFQ@mail.gmail.com
off_t is most likely a 64-bit integer, so casting it to a 32-bit type
for output could lose data. There are more issues like this in the
tree, but this is an instance where this could actually happen in
practice, since base backups are routinely larger than 4 GB. So this
is separated out as a bug fix.
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/20ce62fa-47fc-457b-b504-12f3c1651726%40eisentraut.org
Replace pg_attribute_aligned with C11 alignas, for consistency with
current conventions.
(These new uses were added by commit fbc57f2bc2, which was developed
concurrently with the switch from pg_attribute_aligned to C11 standard
alignas, and it ended up being committed with the "old" style.)
Reviewed-by: John Naylor <johncnaylorls@gmail.com>
Discussion: https://postgr.es/m/CANWCAZaKhE+RD5KKouUFoxx1EbUNrNhcduM1VQ=DkSDadNEFng@mail.gmail.com
A UNION (not UNION ALL) removes duplicates, so its output has no more
rows than its input. The planner did not account for this: it set the
set-op relation's row count to the total size of the appended input,
as though dedup removed nothing. That inflated estimate then
propagated to every node above the UNION, leading to poor plan choices
such as a hash join with a full table scan where an index nested loop
would have been cheaper.
This patch estimates the number of distinct output rows as the sum of
the per-child distinct-group estimates instead. This relies on the
fact that:
distinct(A union B) <= distinct(A) + distinct(B)
that is, the union cannot have more distinct rows than its children do
in total. And because each child's distinct-group estimate never
exceeds that child's row-count estimate, this sum is never larger than
the old estimate, so it only tightens the previous over-estimate.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Chengpeng Yan <chengpeng_yan@outlook.com>
Discussion: https://postgr.es/m/CAMbWs48Fu1nhGXPa60oc+adj7ge4dn0nHhqngqKvOVVQP61duA@mail.gmail.com
MultiXactOffsetStorageSize() and GetMultiXactInfo() are called to gather
the information reported by the function, but were wasteful for the case
where a role does not have the privileges of pg_read_all_stats, where we
return a set of NULLs. These calls are moved to the code path where
their results are used.
Author: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/CAEudQAonQh7be=wOR-CJFW=bgMBz5wW_bv4t0OFxbgn-794JCQ@mail.gmail.com
Backpatch-through: 19
Commit 4035cd5d4 added LZ4 compression for full-page writes in WAL, and
retained "on" as a backward-compatible way to specify the builtin PGLZ
method. Document this meaning of "on" and update postgresql.conf.sample
to make the equivalence clear.
Author: Christoph Berg <myon@debian.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/akJDHRtXwGLTppsQ@msg.df7cb.de
Backpatch-through: 15
Commit a1643d40b removed RADIUS authentication, but apparently
overlooked initdb's list of accepted authentication methods. As a
result, initdb still accepted radius for --auth, --auth-host, and
--auth-local, allowing it to create a pg_hba.conf that the server could
not load.
Remove radius from initdb's local and host authentication method lists.
Backpatch-through: 19
Author: Chao Li <lic@highgo.com>
Discussion: https://postgr.es/m/983F946B-A7CE-4C93-B5F0-665616F72254%40gmail.com
We previously allowed this, but it leads to odd behaviors, basically
because putting a SRF there is inconsistent with the principle that a
window function doesn't change the number of rows in the query result.
There doesn't seem to be a strong reason to try to make such cases
behave consistently. Users should put their SRFs in lateral FROM
clauses instead.
This issue has been sitting on the back burner for multiple years
now, partially because it didn't seem wise to back-patch such a
change. Let's squeeze it into v19 before it's too late.
Bug: #17502
Bug: #19535
Reported-by: Daniel Farkaš <daniel.farkas@datoris.com>
Reported-by: Qifan Liu <imchifan@163.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/17502-281a7aaacfaa872a@postgresql.org
Discussion: https://postgr.es/m/19535-376081d7cc07c86d@postgresql.org
Backpatch-through: 19
In various places we depend on compiler-defined macros like __x86_64__
to guard CPU-type-specific code. However, those macros aren't very
well standardized; in particular, it emerges that MSVC doesn't define
any of the ones gcc does, but has its own. We were not coping with
that consistently, with the result that we're missing some useful
CPU-dependent optimizations in MSVC builds. There are also some
places that are checking randomly-different spellings that may
have been the only ones recognized by some old compilers, but we
weren't doing that consistently either.
Let's standardize on using gcc's long-form spellings (with trailing
underscores), after putting a stanza into c.h that ensures that these
spellings are defined even when the compiler provides some other one.
I put an "#else #error" branch into the c.h addition so that we'll
get an error if the compiler provides none of the symbols we're
expecting. That might be best removed in the end, since it might
annoy people trying to port to some new CPU type. But for testing
this it seems like a good idea, in case we've missed some common
variant spelling.
In addition to enabling some optimizations we previously missed on
MSVC, this cleans up a thinko. Several places used "_M_X64" in the
apparent belief that that's MSVC's equivalent to __x86_64__, but
it's not: it will also get defined on some but not all ARM64 builds.
Also, guard the x86_feature_available() stuff in pg_cpu.[hc] with
#if defined(__x86_64__) || defined(__i386__)
which seems like a more natural way of specifying what it applies to.
This builds on some previous work by Thomas Munro, but it requires
much less code churn because it re-uses gcc's names for the CPU-type
macros instead of inventing our own.
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA+hUKGL8Hs-phHPugrWM=5dAkcT897rXyazYzLw-Szxnzgx-rA@mail.gmail.com
Discussion: https://postgr.es/m/3035145.1780503430@sss.pgh.pa.us
This changes the argtypes argument of SPI_prepare(),
SPI_prepare_cursor(), SPI_cursor_open_with_args(), and
SPI_execute_with_args() from Oid *argtypes to const Oid *argtypes.
The underlying functions were already receptive to that, so this
doesn't require any significant changes beyond the function signatures
and some internal variables.
Commit 28972b6fc3 recently introduced a case where a const had to be
cast away before calling these functions. This is fixed here.
In passing, make a very similar const addition to SPI_modifytuple().
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Ewan Young <kdbase.hack@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/86b5162f-c472-40fa-997b-0450dece1dec%40eisentraut.org
Fixup for commit 8a27d418f8, which converted many functions to use
"const Datum *" instead of "Datum *", including some SPI functions.
For SPI_cursor_open(), the code was updated but not the documentation.
For SPI_cursor_open_with_args(), the documentation was updated but not
the code. (Possibly, these two were confused with each other.) Also,
SPI_execp() and SPI_modifytuple() were not updated, even though they
are closely related to the functions touched by the previous commit
and now look inconsistent. Fix all these.
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/86b5162f-c472-40fa-997b-0450dece1dec%40eisentraut.org
This commit adds per-backend lock statistics, providing the same
information as pg_stat_lock. It is now possible to retrieve those stats
(lock wait counts, wait times, and fast-path exceeded count) on a
per-backend basis.
This data can be retrieved with a new system function called
pg_stat_get_backend_lock(), that returns one tuple per lock type based
on the PID provided in input. Like pg_stat_get_backend_io(), this is
useful if joined with pg_stat_activity to get a live picture of the
locks behavior for each running backend.
pgstat_flush_backend() gains a new flag value, able to control the flush
of the lock stats.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61 (backend-level pgstats).
Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Tatsuya Kawata <kawatatatsuya0913@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Tristan Partin <tristan@partin.io>
Reviewed-by: Rui Zhao <zhaorui126@gmail.com>
Discussion: https://postgr.es/m/aiAzEY+cMQb/W8yu@bdtpg
This commit extracts the tuple-building logic from pg_stat_get_lock()
into a new static helper pg_stat_lock_build_tuples(). This is in
preparation for a follow-up patch, to add support for backend-level lock
stats, which will reuse the same helper.
This change follows the pattern established by pg_stat_io_build_tuples()
for IO stats and pg_stat_wal_build_tuple() for WAL stats.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Tatsuya Kawata <kawatatatsuya0913@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Tristan Partin <tristan@partin.io>
Reviewed-by: Rui Zhao <zhaorui126@gmail.com>
Discussion: https://postgr.es/m/aiAzEY+cMQb/W8yu@bdtpg
A placeholder %s is now used instead of the GUC names in the error
string of this routine. This is going to be useful for a follow-up
patch, where we will be able to reuse the same string, hence reducing
the translation work.
Based on a suggestion by me.
Author: Baji Shaik <baji.pgdev@gmail.com>
Discussion: https://postgr.es/m/ajnhfw84reaXgjfO@paquier.xyz
Other statistics views (pg_stat_io, pg_stat_database, etc.) use float8
for all measured-time columns, the new pg_stat_lock standing out as an
outlier by using bigint.
This commit aligns pg_stat_lock with the other stats views for
consistency. Like pg_stat_io, the time is stored in microseconds, and
is displayed in milliseconds with a conversion done when the view is
queried.
While on it, replace a use of "long" by PgStat_Counter, the former could
overflow for large wait times where sizeof(long) is 4 bytes (aka WIN32).
Bump catalog version.
Author: Tatsuya Kawata <kawatatatsuya0913@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAHza6qerEiQehrbW5xaXyxvR0qJe3KBX1R4kocDz1+7Ygu8x-g@mail.gmail.com
Backpatch-through: 19
LockBufferForCleanup() acquires the exclusive content lock, checks the
buffer's shared pin count, and, if other pins remain, registers itself
as the BM_PIN_COUNT_WAITER before waiting for an unpin notification.
Since commits 5310fac6e0 and c75ebc657f, however, a shared buffer
pin can be released while BM_LOCKED is set, introducing the following
race:
- LockBufferForCleanup() observes a refcount greater than one.
- Before it sets BM_PIN_COUNT_WAITER, another backend releases the
last conflicting pin.
- Since BM_PIN_COUNT_WAITER is not yet set, no wakeup is sent.
- LockBufferForCleanup() then sets BM_PIN_COUNT_WAITER and goes to
sleep, even though only its own pin remains.
As a result, LockBufferForCleanup() can sleep indefinitely because
the wakeup corresponding to the last conflicting unpin has already been
missed.
Fix this by setting BM_PIN_COUNT_WAITER while holding the buffer
header lock, then rechecking the refcount before releasing the content
lock. If only our pin remains, clear the waiter state and proceed
without sleeping. Otherwise, wait as before.
This issue was reported by buildfarm member skink, where it manifested
as intermittent timeouts in 048_vacuum_horizon_floor.pl.
Backpatch to v19, where commits 5310fac6e0 and c75ebc657f
introduced the race.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Author: Xuneng Zhou <xunengzhou@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/7685519a-0bf9-4e17-93ca-7e3aa10fa29c@gmail.com
Backpatch-through: 19
Previously, if an unlogged sequence was created on the primary and
replicated to a standby, reading the sequence after promoting the
standby (for example, with nextval()) could trigger the following
assertion failure:
TRAP: failed Assert("((const PageHeaderData *) page)->pd_special >= SizeOfPageHeaderData")
In non-assert builds, the same operation could instead fail with an
error such as:
ERROR: bad magic number in sequence
The problem was that seq_redo() updated the init fork page in shared
buffers but did not flush it to disk. During promotion,
ResetUnloggedRelations() recreates the main fork of unlogged
relations by copying the init fork from disk, bypassing shared
buffers. As a result, the main fork could be recreated from a stale
init fork instead of the WAL-replayed page.
Fix this by introducing a helper to flush init fork buffers
immediately, and make seq_redo() use it. As a result, the main fork
of an unlogged sequence is recreated from the up-to-date init fork on
disk, allowing the unlogged sequence to be read successfully after
standby promotion.
Backpatch to v15, where unlogged sequences were introduced.
Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwH1Ssze3XM6wjoTjSLVOR041c6xP+vsdLP951=w8oG8bA@mail.gmail.com
Backpatch-through: 15
statatt_build_stavalues() and array_in_safe() have been relying on
InitFunctionCallInfoData() with a locally-filled state to call a data
type input function. InputFunctionCallSafe() can be used to achieve the
same job, simplifying some code.
This fixes an over-allocation of FunctionCallInfoBaseData done in
statatt_build_stavalues(), where there was space for 8 elements but only
3 were needed. The over-allocation exists since REL_18_STABLE, and was
harmless in practice.
While on it, fix some comments for both routines, where elemtypid was
mentioned.
Backpatch down to v19. This code has been reworked during the last
development cycle while working on the restore of extended statistics,
so this keeps the code consistent across all branches.
Author: Jian He <jian.universality@gmail.com>
Author: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CACJufxEGah9PaiTQ=cG14GMMBsUQ3ohGct9tdSwbMQPQ0-nbbQ@mail.gmail.com
Backpatch-through: 19
This is required before the creation of a new branch. pgindent is
clean, as well as is reformat-dat-files.
perltidy version is v20230309, as documented in pgindent's README.
It is not clear how the implicit condition of FOR PORTION OF should
interact with the use of a cursor. Normally, we forbid combining
WHERE CURRENT OF with other WHERE conditions. The SQL standard only
includes FOR PORTION OF with <update statement: searched> and <delete
statement: searched>, not <update statement: positioned> or <delete
statement: positioned>, so it is easy for us to exclude the
functionality, at least for now.
Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/flat/CA%2BrenyUEKPexUYsH4qeU8_o1jqKsUkEWca1keS6n21shgG1g%2BA%40mail.gmail.com
Treat copy_file_range() return value of zero as an error: it indicates
that no bytes could be copied (perhaps the source file is shorter than
expected), and the existing retry loop would otherwise spin forever
since nwritten would never reach BLCKSZ.
The other uses of copy_file_range() in the tree don't have this
problem.
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Yingying Chen <cyy9255@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/3208cf7a-c7f3-41eb-92f6-33cbeff4df40%40eisentraut.org
PL/Python and its hstore and jsonb transforms build SQL values from
Python containers by calling Python C API functions that can return
NULL, and in several places the result was used without first checking
it.
On the sequence side, PySequence_GetItem() is used when converting a
returned sequence into a SQL array or composite value, when reading
the argument list passed to plpy.execute() or plpy.cursor(), and when
reading the list of type names given to plpy.prepare(). On the
mapping side, the hstore and jsonb transforms call PyMapping_Size()
and PyMapping_Items() and then index the result with PyList_GetItem()
and PyTuple_GetItem().
All of these return NULL (or -1), with a Python exception set, for a
broken object: for example one whose __getitem__() or items() raises,
or which reports a length that disagrees with what it actually yields.
The unchecked result was then dereferenced, crashing the backend.
Fix this by checking the result of each call and reporting a regular
error if it failed, so that the underlying Python exception is
surfaced instead of taking down the session.
Author: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: https://postgr.es/m/CAMbWs49BKM9wP6m8bCXEpHwQKp7usvOGV6Jf=J7FYr_BCpxLqg@mail.gmail.com
Backpatch-through: 14
Commit b7b27eb41a, which added foreign-key fast-path batching to
ri_triggers.c, registered ri_FastPathXactCallback() via
RegisterXactCallback() to clear the fast-path batching state at end of
transaction. RegisterXactCallback() is documented as intended for
dynamically loaded modules; built-in code is supposed to hardwire its
end-of-xact hooks into xact.c, mainly so callback ordering can be
controlled where it matters (see the header comment on
RegisterXactCallback()).
Convert the callback into a plain AtEOXact_RI() function and call it
directly from CommitTransaction(), PrepareTransaction() and
AbortTransaction(), alongside the other AtEOXact_* cleanup steps, and
drop the RegisterXactCallback() registration.
Like the other AtEOXact_* routines, AtEOXact_RI() takes an isCommit
argument and treats the two paths differently. On commit or prepare
the fast-path cache must already have been flushed and torn down by
the after-trigger batch callback, so a surviving cache indicates a
trigger batch was never flushed -- which would have silently skipped
FK checks -- and draws an Assert plus a WARNING. On abort a surviving
cache is expected (a flush may have errored out partway) and is simply
reset.
There is no ordering dependency here: AtEOXact_RI() only resets
backend-local static state (the cache pointer, the
callback-registered flag, and the in-flush guard). It touches no
relations, locks, buffers or catalogs, so its position relative to
ResourceOwnerRelease() and the surrounding AtEOXact_* calls does not
matter. On a normal commit the fast-path cache has already been
flushed and torn down by ri_FastPathEndBatch() (an
AfterTriggerBatchCallback fired from AfterTriggerFireDeferred(), well
before any end-of-xact callback), so the reset is a no-op; its real
job is the abort path, where teardown may not have run and the static
pointers would otherwise dangle into the next transaction. The cache
memory itself lives in TopTransactionContext and is freed by the
end-of-transaction memory-context reset on both paths.
The companion RegisterSubXactCallback() use from b7b27eb41a was
already removed by commit 4113873a, which confined fast-path batching
to the top transaction level, so only the RegisterXactCallback() use
remained.
Reported-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/ajypPeEWceXRGAEW@bdtpg
If a "char" column has a statistics histogram, scalarineqsel()
would fail with "cache lookup failed for collation 0". Avoid
the failing lookup by acting as though the collation is "C".
Prior to commit 06421b084, this code didn't fail because
lc_collate_is_c() intentionally didn't spit up on InvalidOid.
It did act differently though: it would take the non-C-collation
code path and hence apply strxfrm using libc's prevailing locale.
But that seems like the wrong thing for a non-collatable comparison,
so let's not resurrect that aspect.
Author: Feng Wu <wufengwufengwufeng@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CACK3muq6s-O1Wc3w4dRL1Fe8YQ-Fz1zJbezeQwhuLgNxGNEFiA@mail.gmail.com
Backpatch-through: 18
Replace the VARIADIC text[] alternating key/value option interface with
typed named boolean parameters for pg_get_role_ddl(), pg_get_tablespace_ddl(),
and pg_get_database_ddl(), as added by commit 4881981f92 and friends.
The new signatures are:
pg_get_role_ddl(role regrole,
pretty boolean DEFAULT false,
memberships boolean DEFAULT true)
pg_get_tablespace_ddl(tablespace oid/name,
pretty boolean DEFAULT false,
owner boolean DEFAULT true)
pg_get_database_ddl(db regdatabase,
pretty boolean DEFAULT false,
owner boolean DEFAULT true,
tablespace boolean DEFAULT true)
This provides type safety at the SQL level, allows named-argument calling
syntax (pretty => true), removes the runtime string-parsing machinery
(DdlOption, parse_ddl_options) in favour of direct PG_GETARG_BOOL() calls,
and allows the functions to be marked STRICT.
While we're here, I added an extra TAP test for pg_get_database(owner =>
false, ...)
Catalog version bumped.
Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/DHM6C7SLS4BN.1WW9Z4PRPN0VJ@jeltef.nl
(and on Discord)
Disallow marking an inherited CHECK constraint as NOT ENFORCED when an
equivalent parent constraint remains ENFORCED. This prevents ALTER
CONSTRAINT from producing a child constraint that is weaker than one of
its inherited parent definitions.
When recursively altering a CHECK constraint to NOT ENFORCED, collect the
corresponding constraints in the affected inheritance subtree and ignore
those parent constraints while checking descendants. If a descendant also
inherits an equivalent ENFORCED constraint from a parent outside the
current ALTER, keep the descendant ENFORCED by merging to the stricter
state.
This was missed in commit 342051d73b, which introduced the ability to
alter CHECK constraint enforceability.
Add regression coverage for direct child ALTER, ONLY ALTER, mixed-parent
inheritance, and a common-ancestor diamond where all equivalent inherited
constraints can be changed together.
Author: Chao Li <lic@highgo.com>
Reviewed-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/E74C57FA-1DD0-4C8E-8FB1-538034752592@gmail.com
When COPY TO with FORMAT json is given an explicit column list that
names all columns in a different order, the JSON output incorrectly
used the table's physical column order instead of the requested order.
This happened because BeginCopyTo() only built a restricted TupleDesc
when list_length(attnumlist) < tupDesc->natts. When all columns are
listed (just reordered), this condition was false and no projected
TupleDesc was built, causing CopyToJsonOneRow() to emit columns in
physical order.
Fix by also building the projected TupleDesc when an explicit column
list was provided (attnamelist != NIL), even if it names all columns.
Author: Baji Shaik <baji.pgdev@gmail.com>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/CA+fm-ROd4cNKM524n6EdgtZ9xOzOHJDNv8J_9Mvr2+2t1qWSDw@mail.gmail.com
We should defer validating FDW usage until after analysis. We have to
guard against not just the topmost table, but also individual child
partitions. Added the check to CheckValidResultRel, because it is
called after looking up child partitions (accounting for pruning), but
before the FDW can run a DirectModify update, which would bypass
per-tuple executor work.
Author: jian he <jian.universality@gmail.com>
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Discussion: https://www.postgresql.org/message-id/flat/CA%2BrenyUte0_UJsJiDJQi82oaBsMJn%3Dcct0Wn%3DvOqXtuDn%3DYYJA%40mail.gmail.com
