Commit graph

7698 commits

Author SHA1 Message Date
Tom Lane
e57296ed48 Log more info when wait-for-catchup tests time out.
Cluster.pm's wait_for_catchup and allied subroutines don't provide
enough information to diagnose the problem when a wait times out.
In hopes of debugging some intermittent buildfarm failures, let's
dump the ending state of the relevant system view when that happens.

Add this to v17 too, but not stable branches.

Discussion: https://postgr.es/m/352068.1723422725@sss.pgh.pa.us
2024-08-12 13:18:36 -04:00
Tom Lane
2b8d33f66c Allow adjusting session_authorization and role in parallel workers.
The code intends to allow GUCs to be set within parallel workers
via function SET clauses, but not otherwise.  However, doing so fails
for "session_authorization" and "role", because the assign hooks for
those attempt to set the subsidiary "is_superuser" GUC, and that call
falls foul of the "not otherwise" prohibition.  We can't switch to
using GUC_ACTION_SAVE for this, so instead add a new GUC variable
flag GUC_ALLOW_IN_PARALLEL to mark is_superuser as being safe to set
anyway.  (This is okay because is_superuser has context PGC_INTERNAL
and thus only hard-wired calls can change it.  We'd need more thought
before applying the flag to other GUCs; but maybe there are other
use-cases.)  This isn't the prettiest fix perhaps, but other
alternatives we thought of would be much more invasive.

While here, correct a thinko in commit 059de3ca4: when rejecting
a GUC setting within a parallel worker, we should return 0 not -1
if the ereport doesn't longjmp.  (This seems to have no consequences
right now because no caller cares, but it's inconsistent.)  Improve
the comments to try to forestall future confusion of the same kind.

Despite the lack of field complaints, this seems worth back-patching.
Thanks to Nathan Bossart for the idea to invent a new flag,
and for review.

Discussion: https://postgr.es/m/2833457.1723229039@sss.pgh.pa.us
2024-08-10 15:51:28 -04:00
Tom Lane
81a12a4477 Fix "failed to find plan for subquery/CTE" errors in EXPLAIN.
To deparse a reference to a field of a RECORD-type output of a
subquery, EXPLAIN normally digs down into the subquery's plan to try
to discover exactly which anonymous RECORD type is meant.  However,
this can fail if the subquery has been optimized out of the plan
altogether on the grounds that no rows could pass the WHERE quals,
which has been possible at least since 3fc6e2d7f.  There isn't
anything remaining in the plan tree that would help us, so fall back
to printing the field name as "fN" for the N'th column of the record.
(This will actually be the right thing some of the time, since it
matches the column names we assign to RowExprs.)

In passing, fix a comment typo in create_projection_plan, which
I noticed while experimenting with an alternative fix for this.

Per bug #18576 from Vasya B.  Back-patch to all supported branches.

Richard Guo and Tom Lane

Discussion: https://postgr.es/m/18576-9feac34e132fea9e@postgresql.org
2024-08-09 11:21:39 -04:00
Alvaro Herrera
344f9f5e2b
Refuse ATTACH of a table referenced by a foreign key
Trying to attach a table as a partition which is already on the
referenced side of a foreign key on the partitioned table that it is
being attached to, leads to strange behavior: we try to clone the
foreign key from the parent to the partition, but this new FK points to
the partition itself, and the mix of pg_constraint rows and triggers
doesn't behave well.

Rather than trying to untangle the mess (which might be possible given
sufficient time), I opted to forbid the ATTACH.  This doesn't seem a
problematic restriction, given that we already fail to create the
foreign key if you do it the other way around, that is, having the
partition first and the FK second.

Backpatch to all supported branches.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Discussion: https://postgr.es/m/18541-628a61bc267cd2d3@postgresql.org
2024-08-08 19:35:13 -04:00
Alvaro Herrera
28b953f263
Refactor error messages to reduce duplication
I also took the liberty of changing

	errmsg("COPY DEFAULT only available using COPY FROM")
to
	errmsg("COPY %s cannot be used with %s", "DEFAULT", "COPY TO")

because the original wording is unlike all other messages that indicate
option incompatibility.  This message was added by commit 9f8377f7a2
(16-era), in whose development thread there was no discussion on this
point.

Backpatch to 17.
2024-08-08 15:17:11 -04:00
Alvaro Herrera
899f39ea25
Refactor/reword some error messages to avoid duplicates
Also, remove brackets around "EMPTY [ ARRAY ]".  An error message is
not the place to state that a keyword is optional.

Backpatch to 17.
2024-08-07 11:30:36 -04:00
Masahiko Sawada
fdf218f1d5 Restrict accesses to non-system views and foreign tables during pg_dump.
When pg_dump retrieves the list of database objects and performs the
data dump, there was possibility that objects are replaced with others
of the same name, such as views, and access them. This vulnerability
could result in code execution with superuser privileges during the
pg_dump process.

This issue can arise when dumping data of sequences, foreign
tables (only 13 or later), or tables registered with a WHERE clause in
the extension configuration table.

To address this, pg_dump now utilizes the newly introduced
restrict_nonsystem_relation_kind GUC parameter to restrict the
accesses to non-system views and foreign tables during the dump
process. This new GUC parameter is added to back branches too, but
these changes do not require cluster recreation.

Back-patch to all supported branches.

Reviewed-by: Noah Misch
Security: CVE-2024-7348
Backpatch-through: 12
2024-08-05 06:05:30 -07:00
Tom Lane
630b81d5cc Revert "Allow parallel workers to cope with a newly-created session user ID."
This reverts commit 5887dd4894.

Some buildfarm animals are failing with "cannot change
"client_encoding" during a parallel operation".  It looks like
assign_client_encoding is unhappy at being asked to roll back a
client_encoding setting after a parallel worker encounters a
failure.  There must be more to it though: why didn't I see this
during local testing?  In any case, it's clear that moving the
RestoreGUCState() call is not as side-effect-free as I thought.
Given that the bug f5f30c22e intended to fix has gone unreported
for years, it's not something that's urgent to fix; I'm not
willing to risk messing with it further with only days to our
next release wrap.
2024-07-31 20:54:31 -04:00
Tom Lane
5887dd4894 Allow parallel workers to cope with a newly-created session user ID.
Parallel workers failed after a sequence like
	BEGIN;
	CREATE USER foo;
	SET SESSION AUTHORIZATION foo;
because check_session_authorization could not see the uncommitted
pg_authid row for "foo".  This is because we ran RestoreGUCState()
in a separate transaction using an ordinary just-created snapshot.
The same disease afflicts any other GUC that requires catalog lookups
and isn't forgiving about the lookups failing.

To fix, postpone RestoreGUCState() into the worker's main transaction
after we've set up a snapshot duplicating the leader's.  This affects
check_transaction_isolation and check_transaction_deferrable, which
think they should only run during transaction start.  Make them
act like check_transaction_read_only, which already knows it should
silently accept the value when InitializingParallelWorker.

Per bug #18545 from Andrey Rachitskiy.  Back-patch to all
supported branches, because this has been wrong for awhile.

Discussion: https://postgr.es/m/18545-feba138862f19aaa@postgresql.org
2024-07-31 18:54:10 -04:00
Andrew Dunstan
0d57dc2f91 Preserve tz when converting to jsonb timestamptz
This removes an inconsistency in the treatment of different datatypes by
the jsonpath timestamp_tz() function. Conversions from data types that
are not timestamp-aware, such as date and timestamp, are now treated
consistently with conversion from those that are such as timestamptz.

Author: David Wheeler
Reviewed-by: Junwang Zhao and Jeevan Chalke

Discussion: https://postgr.es/m/7DE080CE-6D8C-4794-9BD1-7D9699172FAB%40justatheory.com

Backpatch to release 17.
2024-07-30 07:57:16 -04:00
Andrew Dunstan
787ea8c9d8 Stabilize xid_wraparound tests
The tests had a race condition if autovacuum was set to off. Instead we
create all the tables we are interested in with autovacuum disabled, so
they are only ever touched when in danger of wraparound.

Discussion: https://postgr.es/m/3e2cbd24-f45e-4b2b-ba83-8149214f0a4d@dunslane.net

Masahiko Sawada (slightly tweaked by me)

Backpatch to release 17 where these tests were introduced.
2024-07-30 06:26:36 -04:00
Amit Langote
f95c5090d9 SQL/JSON: Fix casting for integer EXISTS columns in JSON_TABLE
The current method of coercing the boolean result value of
JsonPathExists() to the target type specified for an EXISTS column,
which is to call the type's input function via json_populate_type(),
leads to an error when the target type is integer, because the
integer input function doesn't recognize boolean literal values as
valid.

Instead use the boolean-to-integer cast function for coercion in that
case so that using integer or domains thereof as type for EXISTS
columns works. Note that coercion for ON ERROR values TRUE and FALSE
already works like that because the parser creates a cast expression
including the cast function, but the coercion of the actual result
value is not handled by the parser.

Tests by Jian He.

Reported-by: Jian He <jian.universality@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Author: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/CACJufxEo4sUjKCYtda0_qt9tazqqKPmF1cqhW9KBOUeJFqQd2g@mail.gmail.com
Backpatch-through: 17
2024-07-30 10:39:28 +09:00
Amit Langote
847ee701bd SQL/JSON: Some fixes to JsonBehavior expression casting
1. Remove the special case handling when casting the JsonBehavior
   expressions to types with typmod, like 86d33987 did for the casting
   of SQL/JSON constructor functions.

2. Fix casting for fixed-length character and bit string types by
   using assignment-level casts.  This is again similar to what
   86d33987 did, but for ON ERROR / EMPTY expressions.

3. Use runtime coercion for the boolean ON ERROR constants so that
   using fixed-length character string types, for example, for an
   EXISTS column doesn't cause a "value too long for type
   character(n)" when the parser tries to coerce the default ON ERROR
   value "false" to that type, that is, even when clause is not
   specified.

4. Simplify the conditions of when to use runtime coercion vs
   creating the cast expression in the parser itself.  jsonb-valued
   expressions are now always coerced at runtime and boolean
   expressions too if the target type is a string type for the
   reasons mentioned above.

New tests are from a patch that Jian He posted.  Outputs of some
existing tests change because the coercion now happens at runtime
instead of at parse time.

Reported-by: Jian He <jian.universality@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Author: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/CACJufxEo4sUjKCYtda0_qt9tazqqKPmF1cqhW9KBOUeJFqQd2g@mail.gmail.com
Backpatch-through: 17
2024-07-30 10:37:56 +09:00
David Rowley
1e020258e5 Fix incorrect return value for pg_size_pretty(bigint)
pg_size_pretty(bigint) would return the value in bytes rather than PB
for the smallest-most bigint value.  This happened due to an incorrect
assumption that the absolute value of -9223372036854775808 could be
stored inside a signed 64-bit type.

Here we fix that by instead storing that value in an unsigned 64-bit type.

This bug does exist in versions prior to 15 but the code there is
sufficiently different and the bug seems sufficiently non-critical that
it does not seem worth risking backpatching further.

Author: Joseph Koshakow <koshy44@gmail.com>
Discussion: https://postgr.es/m/CAAvxfHdTsMZPWEHUrZ=h3cky9Ccc3Mtx2whUHygY+ABP-mCmUw@mail.gmail.com
Backpatch-through: 15
2024-07-28 22:23:32 +12:00
Amit Langote
3c3ccd4ca8 SQL/JSON: Respect OMIT QUOTES when RETURNING domains over jsonb
populate_domain() didn't take into account the omit_quotes flag passed
down to json_populate_type() by ExecEvalJsonCoercion() and that led
to incorrect behavior when the RETURNING type is a domain over
jsonb.  Fix that by passing the flag by adding a new function
parameter to populate_domain().

Reported-by: Jian He <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxEo4sUjKCYtda0_qt9tazqqKPmF1cqhW9KBOUeJFqQd2g@mail.gmail.com
Backpatch-through: 17
2024-07-26 16:36:10 +09:00
Amit Langote
d1dc4ae560 SQL/JSON: Improve error-handling of JsonBehavior expressions
Instead of returning a NULL when the JsonBehavior expression value
could not be coerced to the RETURNING type, throw the error message
informing the user that it is the JsonBehavior expression that caused
the error with the actual coercion error message shown in its DETAIL
line.

Discussion: https://postgr.es/m/CACJufxEo4sUjKCYtda0_qt9tazqqKPmF1cqhW9KBOUeJFqQd2g@mail.gmail.com
Backpatch-through: 17
2024-07-26 16:36:06 +09:00
Amit Langote
79fa052e78 SQL/JSON: Fix error-handling of some JsonBehavior expressions
To ensure that the errors of executing a JsonBehavior expression that
is coerced in the parser are caught instead of being thrown directly,
pass ErrorSaveContext to ExecInitExprRec() when initializing it.
Also, add a EEOP_JSONEXPR_COERCION_FINISH step to handle the errors
that are caught that way.

Discussion: https://postgr.es/m/CACJufxEo4sUjKCYtda0_qt9tazqqKPmF1cqhW9KBOUeJFqQd2g@mail.gmail.com
Backpatch-through: 17
2024-07-26 16:36:02 +09:00
Alvaro Herrera
2b22543a44
Reset relhassubclass upon attaching table as a partition
We don't allow inheritance parents as partitions, and have checks to
prevent this; but if a table _was_ in the past an inheritance parents
and all their children are removed, the pg_class.relhassubclass flag
may remain set, which confuses the partition pruning code (most
obviously, it results in an assertion failure; in production builds it
may be worse.)

Fix by resetting relhassubclass on attach.

Backpatch to all supported versions.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18550-d5e047e9a897a889@postgresql.org
2024-07-24 12:38:18 +02:00
Nathan Bossart
657e54a058 Detect integer overflow in array_set_slice().
When provided an empty initial array, array_set_slice() fails to
check for overflow when computing the new array's dimensions.
While such overflows are ordinarily caught by ArrayGetNItems(),
commands with the following form are accepted:

	INSERT INTO t (i[-2147483648:2147483647]) VALUES ('{}');

To fix, perform the hazardous computations using overflow-detecting
arithmetic routines.  As with commit 18b585155a, the added test
cases generate errors that include a platform-dependent value, so
we again use psql's VERBOSITY parameter to suppress printing the
message text.

Reported-by: Alexander Lakhin
Author: Joseph Koshakow
Reviewed-by: Jian He
Discussion: https://postgr.es/m/31ad2cd1-db94-bdb3-f91a-65ffdb4bef95%40gmail.com
Backpatch-through: 12
2024-07-23 21:59:02 -05:00
Melanie Plageman
1a3e90948b Revert "Test that vacuum removes tuples older than OldestXmin"
This reverts commit 80c34692e8.

This test proved to be unstable on the buildfarm, timing out before the
standby could catch up on 32-bit machines where more rows were required
and failing to reliably trigger multiple index vacuum rounds on 64-bit
machines where fewer rows should be required.

Because the instability is only known to be present on versions of
Postgres with TIDStore used for dead TID storage by vacuum, this is only
being reverted on master and REL_17_STABLE.

As having this coverage may be valuable, there is a discussion on the
thread of possible ways to stabilize the test. If that happens, a fixed
test can be committed again.

Backpatch-through: 17
Reported-by: Tom Lane

Discussion: https://postgr.es/m/614152.1721580711%40sss.pgh.pa.us
2024-07-22 16:57:44 -04:00
Tom Lane
041a00c480 Correctly check updatability of columns targeted by INSERT...DEFAULT.
If a view has some updatable and some non-updatable columns, we failed
to verify updatability of any columns for which an INSERT or UPDATE
on the view explicitly specifies a DEFAULT item (unless the view has
a declared default for that column, which is rare anyway, and one
would almost certainly not write one for a non-updatable column).
This would lead to an unexpected "attribute number N not found in
view targetlist" error rather than the intended error.

Per bug #18546 from Alexander Lakhin.  This bug is old, so back-patch
to all supported branches.

Discussion: https://postgr.es/m/18546-84a292e759a9361d@postgresql.org
2024-07-20 13:40:15 -04:00
Nathan Bossart
3764ee47f7 Add overflow checks to money type.
None of the arithmetic functions for the the money type handle
overflow.  This commit introduces several helper functions with
overflow checking and makes use of them in the money type's
arithmetic functions.

Fixes bug #18240.

Reported-by: Alexander Lakhin
Author: Joseph Koshakow
Discussion: https://postgr.es/m/18240-c5da758d7dc1ecf0%40postgresql.org
Discussion: https://postgr.es/m/CAAvxfHdBPOyEGS7s%2Bxf4iaW0-cgiq25jpYdWBqQqvLtLe_t6tw%40mail.gmail.com
Backpatch-through: 12
2024-07-19 11:52:32 -05:00
Melanie Plageman
80c34692e8 Test that vacuum removes tuples older than OldestXmin
If vacuum fails to prune a tuple killed before OldestXmin, it will
decide to freeze its xmax and later error out in pre-freeze checks.

Add a test reproducing this scenario to the recovery suite which creates
a table on a primary, updates the table to generate dead tuples for
vacuum, and then, during the vacuum, uses a replica to force
GlobalVisState->maybe_needed on the primary to move backwards and
precede the value of OldestXmin set at the beginning of vacuuming the
table.

This commit is separate from the fix in case there are test stability
issues.

Author: Melanie Plageman
Reviewed-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com
2024-07-19 12:12:03 -04:00
Andrew Dunstan
5d797c896f Avoid error in recovery test if history file is not yet present
Error was detected when testing use of libpq sessions instead of psql
for polling queries.

Discussion: https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db886@dunslane.net

Backpatch to all live branches
2024-07-17 10:44:14 -04:00
Jeff Davis
b4da732fd6 When creating materialized views, use REFRESH to load data.
Previously, CREATE MATERIALIZED VIEW ... WITH DATA populated the MV
the same way as CREATE TABLE ... AS.

Instead, reuse the REFRESH logic, which locks down security-restricted
operations and restricts the search_path. This reduces the chance that
a subsequent refresh will fail.

Reported-by: Noah Misch
Backpatch-through: 17
Discussion: https://postgr.es/m/20240630222344.db.nmisch@google.com
2024-07-16 15:41:22 -07:00
Fujii Masao
6a700da46e Fix unstable tests in partition_merge.sql and partition_split.sql.
The tests added by commit c086896625 were unstable due to
missing schema names when checking pg_tables and pg_indexes.

Backpatch to v17.

Reported by buildfarm.
2024-07-15 14:11:49 +09:00
Fujii Masao
929390e4d8 Fix tablespace handling in MERGE/SPLIT partition commands.
As commit ca4103025d stated, new partitions without a specified tablespace
should inherit the parent relation's tablespace. However, previously,
ALTER TABLE MERGE PARTITIONS and ALTER TABLE SPLIT PARTITION commands
always created new partitions in the default tablespace, ignoring
the parent's tablespace. This commit ensures new partitions inherit
the parent's tablespace.

Backpatch to v17 where these commands were introduced.

Author: Fujii Masao
Reviewed-by: Masahiko Sawada
Discussion: https://postgr.es/m/abaf390b-3320-40a5-8815-ef476db5cfe7@oss.nttdata.com
2024-07-15 13:14:20 +09:00
Tom Lane
cf588e10f6 Avoid unhelpful internal error for incorrect recursive-WITH queries.
checkWellFormedRecursion would issue "missing recursive reference"
if a WITH RECURSIVE query contained a single self-reference but
that self-reference was inside a top-level WITH, ORDER BY, LIMIT,
etc, rather than inside the second arm of the UNION as expected.
We already intended to throw more-on-point errors for such cases,
but those error checks must be done before examining the UNION arm
in order to have the desired results.  So this patch need only
move some code (and improve the comments).

Per bug #18536 from Alexander Lakhin.  Back-patch to all supported
branches.

Discussion: https://postgr.es/m/18536-0a342ec07901203e@postgresql.org
2024-07-14 13:49:46 -04:00
Andrew Dunstan
5ea9f66616 Use correct collate.windows.win1252.out
I inadvertently missed backporting this to Release 17 from commit 291c420747

per offlist reminder from Alexander Lakhin.
2024-07-13 16:24:01 -04:00
Noah Misch
0d3b35c367 Fix new assertion for MERGE view_name ... DO NOTHING.
Such queries don't expand automatically updatable views, and ModifyTable
uses the wholerow attribute unconditionally.  The user-visible behavior
is fine, so change to more-specific assertions.  Commit
d5f788b41d added the wrong assertion.
Back-patch to v17, where commit 5f2e179bd3
introduced MERGE view_name.

Reported by Alexander Lakhin.

Discussion: https://postgr.es/m/e4b40a88-c134-6926-3196-bc4501cb87a2@gmail.com
2024-07-13 08:09:36 -07:00
Noah Misch
f5bb46fb2e Don't lose partitioned table reltuples=0 after relhassubclass=f.
ANALYZE sets relhassubclass=f when a partitioned table no longer has
partitions.  An ANALYZE doing that proceeded to apply the inplace update
of pg_class.reltuples to the old pg_class tuple instead of the new
tuple, losing that reltuples=0 change if the ANALYZE committed.
Non-partitioning inheritance trees were unaffected.  Back-patch to v14,
where commit 375aed36ad introduced
maintenance of partitioned table pg_class.reltuples.

Reported by Alexander Lakhin.

Discussion: https://postgr.es/m/a295b499-dcab-6a99-c06e-01cf60593344@gmail.com
2024-07-13 08:09:36 -07:00
Alvaro Herrera
0340eefd9b
Add ORDER BY to new test query
Per buildfarm.
2024-07-12 13:44:19 +02:00
Alvaro Herrera
30ca4e1ab1
Fix ALTER TABLE DETACH for inconsistent indexes
When a partitioned table has an index that doesn't support a constraint,
but a partition has an equivalent index that does, then a DETACH
operation would misbehave: a crash in assertion-enabled systems (because
we fail to find the constraint in the parent that we expect to), or a
broken coninhcount value (-1) in production systems (because we blindly
believe that we've successfully detached the parent).

While we should reject an ATTACH of a partition with such an index, we
have failed to do so in existing releases, so adding an error in stable
releases might break the (unlikely) existing applications that rely on
this behavior.  At this point I don't even want to reject them in
master, because it'd break pg_upgrade if such databases exist, and there
would be no easy way to fix existing databases without expensive index
rebuilds.

(Later on we could add ALTER TABLE ... ADD CONSTRAINT USING INDEX to
partitioned tables, which would allow the user to fix such patterns.  At
that point we could add more restrictions to prevent the problem from
its root.)

Also, add a test case that leaves one table in this condition, so that
we can verify that pg_upgrade continues to work if we later decide to
change the policy on the master branch.

Backpatch to all supported branches.

Co-authored-by: Tender Wang <tndrwang@gmail.com>
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Tender Wang <tndrwang@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/18500-62948b6fe5522f56@postgresql.org
2024-07-12 12:54:01 +02:00
Tom Lane
a9747be272 Make our back branches compatible with libxml2 2.13.x.
This back-patches HEAD commits 066e8ac6e, 6082b3d5d, e7192486d,
and 896cd266f into supported branches.  Changes:

* Use xmlAddChildList not xmlAddChild in XMLSERIALIZE
(affects v16 and up only).  This was a flat-out coding mistake
that we got away with due to lax checking in previous versions
of xmlAddChild.

* Use xmlParseInNodeContext not xmlParseBalancedChunkMemory.
This is to dodge a bug in xmlParseBalancedChunkMemory in libxm2
releases 2.13.0-2.13.2.  While that bug is now fixed upstream and
will probably never be seen in any production-oriented distro, it is
currently a problem on some more-bleeding-edge-friendly platforms.

* Suppress "chunk is not well balanced" errors from libxml2,
unless it is the only error.  This eliminates an error-reporting
discrepancy between 2.13 and older releases.  This error is
almost always redundant with previous errors, if not flat-out
inappropriate, which is why 2.13 changed the behavior and why
nobody's likely to miss it.

Erik Wienhold and Tom Lane, per report from Frank Streitzig.

Discussion: https://postgr.es/m/trinity-b0161630-d230-4598-9ebc-7a23acdb37cb-1720186432160@3c-app-gmx-bap25
Discussion: https://postgr.es/m/trinity-361ba18b-541a-4fe7-bc63-655ae3a7d599-1720259822452@3c-app-gmx-bs01
2024-07-10 20:15:52 -04:00
Andrew Dunstan
4506d18a98 Use diff's --strip-trailing-cr flag where appropriate on Windows
Test result files might be checked out using Unix or Windows style line
endings, depening on git flags, so on Windows we use the
--strip-trailing-cr flag to tell diff to ignore line endings
differences.

The flag is added to the diff invocation for the test_json_parser module
tests and the pg_bsd_indent tests. in pg_regress.c we replace the
current use of the "-w" flag, which ignore all white space differences,
with this one which only ignores line end differences.

Discussion: https://postgr.es/m/20240707052030.r77hbdkid3mwksop@awork3.anarazel.de
2024-07-10 10:05:00 -04:00
Andrew Dunstan
e72f787ea6 Prevent CRLF conversion of inputs in json_parser test module
Do this by opening the file in PG_BINARY_R mode. This prevents us from
getting wrong byte count from stat().

Per complaint from Andres Freund

Discussion: https://postgr.es/m/20240707052030.r77hbdkid3mwksop@awork3.anarazel.de

Backpatch to rlease 17 where this code was introduced
2024-07-09 17:34:47 -04:00
Andrew Dunstan
e68cf81bfe Choose ports for test servers less likely to result in conflicts
If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee056f2@gmail.com

Backpatch to all live branches (12 and up)
2024-07-08 11:40:58 -04:00
Andrew Dunstan
e4754f780f Force nodes for SSL tests to start in TCP mode
Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee056f2@gmail.com

Backpatch to all live branches (12 and up).
2024-07-08 11:40:58 -04:00
Dean Rasheed
7a8977d258 Fix scale clamping in numeric round() and trunc().
The numeric round() and trunc() functions clamp the scale argument to
the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much
smaller than the actual allowed range of type numeric. As a result,
they return incorrect results when asked to round/truncate more than
2000 digits before or after the decimal point.

Fix by using the correct upper and lower scale limits based on the
actual allowed (and documented) range of type numeric.

While at it, use the new NUMERIC_WEIGHT_MAX constant instead of
SHRT_MAX in all other overflow checks, and fix a comment thinko in
power_var() introduced by e54a758d24 -- the minimum value of
ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this
doesn't affect the point being made in the comment, that the resulting
local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000).

Back-patch to all supported branches.

Dean Rasheed, reviewed by Joel Jacobson.

Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com
2024-07-08 17:51:23 +01:00
Richard Guo
cccab85c2b Fix right-anti-joins when the inner relation is proven unique
For an inner_unique join, we always assume that the executor will stop
scanning for matches after the first match.  Therefore, for a mergejoin
that is inner_unique and whose mergeclauses are sufficient to identify a
match, we set the skip_mark_restore flag to true, indicating that the
executor need not do mark/restore calls.  However, merge-right-anti-join
did not get this memo and continues scanning the inner side for matches
after the first match.  If there are duplicates in the outer scan, we
may incorrectly skip matching some inner tuples, which can lead to wrong
results.

Here we fix this issue by ensuring that merge-right-anti-join also
advances to next outer tuple after the first match in inner_unique
cases.  This also saves cycles by avoiding unnecessary scanning of inner
tuples after the first match.

Although hash-right-anti-join does not suffer from this wrong results
issue, we apply the same change to it as well, to help save cycles for
the same reason.

Per bug #18522 from Antti Lampinen, and bug #18526 from Feliphe Pozzer.
Back-patch to v16 where right-anti-join was introduced.

Author: Richard Guo
Discussion: https://postgr.es/m/18522-c7a8956126afdfd0@postgresql.org
2024-07-08 10:17:12 +09:00
Michael Paquier
fa1a63dffc Re-enable autoruns for for cmd.exe on Windows
This acts as a revert of b83747a8a6 and 9886744a36.  As pointed out
by Noah, HEAD and REL_17_STABLE are in a weird state where the code
paths adding /D would limit the spawn of child processes, but we still
have code paths where the spawn of more than one child process would be
possible.

Let's remove these /D switches for now, to bring back the code into a
state consistent with how autorun is configured on a Windows host.

Reported-by: Noah Misch
Discussion: https://postgr.es/m/20240630021211.f3.nmisch@google.com
Backpatch-through: 17
2024-07-08 09:43:35 +09:00
Amit Kapila
0f934b0739 Rename standby_slot_names to synchronized_standby_slots.
The standby_slot_names GUC allows the specification of physical standby
slots that must be synchronized before the logical walsenders associated
with logical failover slots. However, for this purpose, the GUC name is
too generic.

Author: Hou Zhijie
Reviewed-by: Bertrand Drouvot, Masahiko Sawada
Backpatch-through: 17
Discussion: https://postgr.es/m/ZnWeUgdHong93fQN@momjian.us
2024-07-01 11:33:55 +05:30
Michael Paquier
7dcc6f8e6d Run pgperltidy
This is required before the creation of a new branch.  pgindent is
clean, as well as is reformat-dat-files.

perltidy version is v20230309, as documented in pgindent's README.
2024-07-01 07:35:01 +09:00
Tomas Vondra
35a7b288b9 Add PG_TEST_PG_COMBINEBACKUP_MODE
Introduces an environment variable PG_TEST_PG_COMBINEBACKUP_MODE, that
determines copy mode used by pg_combinebackup in TAP tests. Defaults to
"--copy" but may be set to "--clone" or "--copy-file-range" to use the
alternative stategies.

Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/48da4a1f-ccd9-4988-9622-24f37b1de2b4%40eisentraut.org
2024-06-30 20:53:39 +02:00
Noah Misch
db0c96cc18 Fix .gitignore for new injection suite.
Commit c35f419d6e missed this.
2024-06-28 11:17:50 -07:00
Noah Misch
b99414de90 Remove configuration-dependent output from new inplace-inval test.
Per buildfarm members prion and trilobite.  Back-patch to v12 (all
supported versions), like commit
0844b39689.

Strategy reviewed by Tom Lane.

Discussion: https://postgr.es/m/20240628051353.a0.nmisch@google.com
2024-06-28 09:33:40 -07:00
Amit Langote
716bd12d22 SQL/JSON: Always coerce JsonExpr result at runtime
Instead of looking up casts at parse time for converting the result
of JsonPath* query functions to the specified or the default
RETURNING type, always perform the conversion at runtime using either
the target type's input function or the function
json_populate_type().

There are two motivations for this change:

1. json_populate_type() coerces to types with typmod such that any
   string values that exceed length limit cause an error instead of
   silent truncation, which is necessary to be standard-conforming.

2. It was possible to end up with a cast expression that doesn't
   support soft handling of errors causing bugs in the of handling
   ON ERROR clause.

JsonExpr.coercion_expr which would store the cast expression is no
longer necessary, so remove.

Bump catversion because stored rules change because of the above
removal.

Reported-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: Discussion: https://postgr.es/m/202405271326.5a5rprki64aw%40alvherre.pgsql
2024-06-28 21:58:13 +09:00
Amit Langote
c2d93c3802 SQL/JSON: Fix coercion of constructor outputs to types with typmod
Ensure SQL/JSON constructor functions that allow specifying the
target type using the RETURNING clause perform implicit cast to
that type.  This ensures that output values that exceed the specified
length produce an error rather than being  silently truncated. This
behavior conforms to the SQL standard.

Reported-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: https://postgr.es/m/202405271326.5a5rprki64aw%40alvherre.pgsql
2024-06-28 21:48:44 +09:00
Amit Langote
55e56c84da SQL/JSON: Validate values in ON ERROR/EMPTY clauses
Currently, the grammar allows any supported values in the ON ERROR
and ON EMPTY clauses for SQL/JSON functions, regardless of whether
the values are appropriate for the function. This commit ensures
that during parse analysis, the provided value is checked for
validity for the given function and throws a syntax error if it is
not.

While at it, this fixes some omissions in the documentation of the
ON ERROR/EMPTY clauses for JSON_TABLE().

Reported-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Jian He <jian.universality@gmail.com>
Discussion: https://postgr.es/m/CACJufxFgWGqpESSYzyJ6tSurr3vFYBSNEmCfkGyB_dMdptFnZQ%40mail.gmail.com
2024-06-28 14:01:43 +09:00
Amit Langote
e3c1393efc SQL/JSON: Prevent ON EMPTY for EXISTS columns in JSON_TABLE()
Due to an oversight in de3600452b, the ON EMPTY clause was
incorrectly allowed in the EXISTS column. Fix the grammar to prevent
this.

Discussion: https://postgr.es/m/CA%2BHiwqHh3YDXTpccgAo4CdfV9Mhy%2Bmg%3Doh6t8rfM5uLW1BJN4g%40mail.gmail.com
2024-06-28 14:01:43 +09:00