Commit graph

4489 commits

Author SHA1 Message Date
Noah Misch
ec6ba191ce Account for IPC::Run::result() Windows behavior change.
This restores compatibility with the not-yet-released successor of
version 20220807.0.  Back-patch to 9.4, which introduced this code.

Reviewed by Andrew Dunstan.

Discussion: https://postgr.es/m/20221117061805.GA4020280@rfd.leadboat.com
2022-11-17 07:39:20 -08:00
Tom Lane
23e2a06acf Yet further fixes for multi-row VALUES lists for updatable views.
DEFAULT markers appearing in an INSERT on an updatable view
could be mis-processed if they were in a multi-row VALUES clause.
This would lead to strange errors such as "cache lookup failed
for type NNNN", or in older branches even to crashes.

The cause is that commit 41531e42d tried to re-use rewriteValuesRTE()
to remove any SetToDefault nodes (that hadn't previously been replaced
by the view's own default values) appearing in "product" queries,
that is DO ALSO queries.  That's fundamentally wrong because the
DO ALSO queries might not even be INSERTs; and even if they are,
their targetlists don't necessarily match the view's column list,
so that almost all the logic in rewriteValuesRTE() is inapplicable.

What we want is a narrow focus on replacing any such nodes with NULL
constants.  (That is, in this context we are interpreting the defaults
as being strictly those of the view itself; and we already replaced
any that aren't NULL.)  We could add still more !force_nulls tests
to further lobotomize rewriteValuesRTE(); but it seems cleaner to
split out this case to a new function, restoring rewriteValuesRTE()
to the charter it had before.

Per bug #17633 from jiye_sw.  Patch by me, but thanks to
Richard Guo and Japin Li for initial investigation.
Back-patch to all supported branches, as the previous fix was.

Discussion: https://postgr.es/m/17633-98cc85e1fa91e905@postgresql.org
2022-10-11 18:24:15 -04:00
Alvaro Herrera
8bf4705272
Ensure all perl test modules are installed
PostgreSQL::Test::Cluster and ::Utils were not being installed.  This is
very hard to notice, as it only seems to affect external modules that
want to run tests from 15 back in earlier versions.  Oversight in
b235d41d96.

This applies only to branches 14 and back, because 15 had already been
made correct in commit b3b4d8e68a.

Discussion: https://postgr.es/m/20221010093415.poplkyn7pjeiv2y7@alvherre.pgsql
2022-10-11 09:56:13 +02:00
Tom Lane
19a00ea56f In back branches, fix conditions for pullup of FROM-less subqueries.
In branches before commit 4be058fe9, we have to prevent flattening
of subqueries with empty jointrees if the subqueries' output
columns might need to be wrapped in PlaceHolderVars.  That's
because the empty jointree would result in empty phrels for the
PlaceHolderVars, meaning we'd be unable to figure out where to
evaluate them.  However, we've failed to keep is_simple_subquery's
check for this hazard in sync with what pull_up_simple_subquery
actually does.  The former is checking "lowest_outer_join != NULL",
whereas the conditions pull_up_simple_subquery actually uses are

	if (lowest_nulling_outer_join != NULL)
	if (containing_appendrel != NULL)
	if (parse->groupingSets)

So the outer-join test is overly conservative, while we missed out
checking for appendrels and groupingSets.  The appendrel omission
is harmless, because in that case we also check is_safe_append_member
which will also reject such subqueries.  The groupingSets omission
is a bug though, leading to assertion failures or planner errors
such as "variable not found in subplan target lists".

is_simple_subquery has access to none of the three variables used
in the correct tests, but its callers do, so I chose to have them
pass down a bool corresponding to the OR of these conditions.
(The need for duplicative conditions would be a maintenance
hazard in actively-developed code, but I'm not too concerned
about it in branches that have only ~ 1 year to live.)

Per bug #17614 from Wei Wei.  Patch v10 and v11 only, since we have a
better answer to this in v12 and later (indeed, the faulty check in
is_simple_subquery is gone entirely).

Discussion: https://postgr.es/m/17614-8ec20c85bdecaa2a@postgresql.org
2022-09-15 15:21:35 -04:00
Tom Lane
174c929e3b Further fixes for MULTIEXPR_SUBLINK fix.
Some more things I didn't think about in commits 3f7323cbb et al:

MULTIEXPR_SUBLINK subplans might have been converted to initplans
instead of regular subplans, in which case they won't show up in
the modified targetlist.  Fortunately, this would only happen if
they have no input parameters, which means that the problem we
originally needed to fix can't happen with them.  Therefore, there's
no need to clone their output parameters, and thus it doesn't hurt
that we'll fail to see them in the first pass over the targetlist.
Nonetheless, this complicates matters greatly, because now we have
to distinguish output Params of initplans (which shouldn't get
renumbered) from those of regular subplans (which should).
This also breaks the simplistic scheme I used of assuming that the
subplans found in the targetlist have consecutive subLinkIds.
We really can't avoid the need to know the subplans' subLinkIds in
this code.  To fix that, add subLinkId as the last field of SubPlan.
We can get away with that change in back branches because SubPlan
nodes will never be stored in the catalogs, and there's no ABI
break for external code that might be looking at the existing
fields of SubPlan.

Secondly, rewriteTargetListIU might have rolled up multiple
FieldStores or SubscriptingRefs into one targetlist entry,
breaking the assumption that there's at most one Param to fix
per targetlist entry.  (That assumption is OK I think in the
ruleutils.c code I stole the logic from in 18f51083c, because
that only deals with pre-rewrite query trees.  But it's
definitely not OK here.)  Abandon that shortcut and just do a
full tree walk on the targetlist to ensure we find all the
Params we have to change.

Per bug #17606 from Andre Lin.  As before, only v10-v13 need the
patch.

Discussion: https://postgr.es/m/17606-e5c8ad18d31db96a@postgresql.org
2022-09-06 16:38:18 -04:00
Tom Lane
42d0d46f98 Fix oversight in recent MULTIEXPR_SUBLINK fix.
Commits 3f7323cbb et al missed the possibility that the Params
they are looking for could be buried under implicit coercions,
as well as other stuff that processIndirection() could add to
the original targetlist entry.  Copy the code in ruleutils.c
that deals with such cases.  (I thought about refactoring so
that there's just one copy; but seeing that we only need this
in old back branches, it seems not worth the trouble.)

Per off-list report from Andre Lin.  As before, only v10-v13
need the patch.

Discussion: https://postgr.es/m/17596-c5357f61427a81dc@postgresql.org
2022-09-02 14:54:41 -04:00
Tom Lane
e1ea6f3745 Repair rare failure of MULTIEXPR_SUBLINK subplans in inherited updates.
Prior to v14, if we have a MULTIEXPR SubPlan (that is, use of the syntax
UPDATE ... SET (c1, ...) = (SELECT ...)) in an UPDATE with an inherited
or partitioned target table, inheritance_planner() will clone the
targetlist and therefore also the MULTIEXPR SubPlan and the Param nodes
referencing it for each child target table.  Up to now, we've allowed
all the clones to share the underlying subplan as well as the output
parameter IDs -- that is, the runtime ParamExecData slots.  That
technique is borrowed from the far older code that supports initplans,
and it works okay in that case because the cloned SubPlan nodes are
essentially identical.  So it doesn't matter which one of the clones
the shared ParamExecData.execPlan field might point to.

However, this fails to hold for MULTIEXPR SubPlans, because they can
have nonempty "args" lists (values to be passed into the subplan), and
those lists could get mutated to different states in the various clones.
In the submitted reproducer, as well as the test case added here, one
clone contains Vars with varno OUTER_VAR where another has INNER_VAR,
because the child tables are respectively on the outer or inner side of
the join.  Sharing the execPlan pointer can result in trying to evaluate
an args list that doesn't match the local execution state, with mayhem
ensuing.  The result often is to trigger consistency checks in the
executor, but I believe this could end in a crash or incorrect updates.

To fix, assign new Param IDs to each of the cloned SubPlans, so that
they don't share ParamExecData slots at runtime.  It still seems fine
for the clones to share the underlying subplan, and extra ParamExecData
slots are cheap enough that this fix shouldn't cost much.

This has been busted since we invented MULTIEXPR SubPlans in 9.5.
Probably the lack of previous reports is because query plans in which
the different clones of a MULTIEXPR mutate to effectively-different
states are pretty rare.  There's no issue in v14 and later, because
without inheritance_planner() there's never a reason to clone
MULTIEXPR SubPlans.

Per bug #17596 from Andre Lin.  Patch v10-v13 only.

Discussion: https://postgr.es/m/17596-c5357f61427a81dc@postgresql.org
2022-08-27 12:11:20 -04:00
Amit Kapila
f01e16b198 Back-Patch "Add wait_for_subscription_sync for TAP tests."
This was originally done in commit 0c20dd33db for 16 only, to eliminate
duplicate code and as an infrastructure that makes it easier to write
future tests. However, it has been suggested that it would be good to
back-patch this testing infrastructure to aid future tests in
back-branches.

Backpatch to all supported versions.

Author: Masahiko Sawada
Reviewed by: Amit Kapila, Shi yu
Discussion: https://postgr.es/m/CAD21AoC-fvAkaKHa4t1urupwL8xbAcWRePeETvshvy80f6WV1A@mail.gmail.com
Discussion: https://postgr.es/m/E1oJBIf-0006sw-SA@gemulon.postgresql.org
2022-08-12 10:18:26 +05:30
Tom Lane
1446612c5b Fix handling of R/W expanded datums that are passed to SQL functions.
fmgr_sql must make expanded-datum arguments read-only, because
it's possible that the function body will pass the argument to
more than one callee function.  If one of those functions takes
the datum's R/W property as license to scribble on it, then later
callees will see an unexpected value, leading to wrong answers.

From a performance standpoint, it'd be nice to skip this in the
common case that the argument value is passed to only one callee.
However, detecting that seems fairly hard, and certainly not
something that I care to attempt in a back-patched bug fix.

Per report from Adam Mackler.  This has been broken since we
invented expanded datums, so back-patch to all supported branches.

Discussion: https://postgr.es/m/WScDU5qfoZ7PB2gXwNqwGGgDPmWzz08VdydcPFLhOwUKZcdWbblbo-0Lku-qhuEiZoXJ82jpiQU4hOjOcrevYEDeoAvz6nR0IU4IHhXnaCA=@mackler.email
Discussion: https://postgr.es/m/187436.1660143060@sss.pgh.pa.us
2022-08-10 13:37:25 -04:00
Tom Lane
2d7d8f4397 Stabilize output of new regression test.
Per buildfarm, the output order of \dx+ isn't consistent across
locales.  Apply NO_LOCALE to force C locale.  There might be a
more localized way, but I'm not seeing it offhand, and anyway
there is nothing in this test module that particularly cares
about locales.

Security: CVE-2022-2625
2022-08-08 12:16:01 -04:00
Tom Lane
5919bb5a59 In extensions, don't replace objects not belonging to the extension.
Previously, if an extension script did CREATE OR REPLACE and there was
an existing object not belonging to the extension, it would overwrite
the object and adopt it into the extension.  This is problematic, first
because the overwrite is probably unintentional, and second because we
didn't change the object's ownership.  Thus a hostile user could create
an object in advance of an expected CREATE EXTENSION command, and would
then have ownership rights on an extension object, which could be
modified for trojan-horse-type attacks.

Hence, forbid CREATE OR REPLACE of an existing object unless it already
belongs to the extension.  (Note that we've always forbidden replacing
an object that belongs to some other extension; only the behavior for
previously-free-standing objects changes here.)

For the same reason, also fail CREATE IF NOT EXISTS when there is
an existing object that doesn't belong to the extension.

Our thanks to Sven Klemm for reporting this problem.

Security: CVE-2022-2625
2022-08-08 11:12:31 -04:00
Alvaro Herrera
8e5874964b
Remove unportable use of timezone in recent test
Per buildfarm member snapper

Discussion: https://postgr.es/m/129951.1659812518@sss.pgh.pa.us
2022-08-07 10:19:40 +02:00
Alvaro Herrera
ad0e083947
Improve recently-added test reliability
Commit 59be1c942a already tried to make
src/test/recovery/t/033_replay_tsp_drops more reliable, but it wasn't
enough.  Try to improve on that by making this use of a replication slot
to be more like others.  Also, don't drop the slot.

Make a few other stylistic changes while at it.  It's still quite slow,
which is another thing that we need to fix in this script.

Backpatch to all supported branches.

Discussion: https://postgr.es/m/349302.1659191875@sss.pgh.pa.us
2022-08-06 15:52:10 +02:00
Alvaro Herrera
e797c7a6f7
BRIN: mask BRIN_EVACUATE_PAGE for WAL consistency checking
That bit is unlogged and therefore it's wrong to consider it in WAL page
comparison.

Add a test that tickles the case, as branch testing technology allows.

This has been a problem ever since wal consistency checking was
introduced (commit a507b86900 for pg10), so backpatch to all supported
branches.

Author: 王海洋 (Haiyang Wang) <wanghaiyang.001@bytedance.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/CACciXAD2UvLMOhc4jX9VvOKt7DtYLr3OYRBhvOZ-jRxtzc_7Jg@mail.gmail.com
Discussion: https://postgr.es/m/CACciXADOfErX9Bx0nzE_SkdfXr6Bbpo5R=v_B6MUTEYW4ya+cg@mail.gmail.com
2022-08-05 18:00:17 +02:00
Tom Lane
9820032daa Reduce test runtime of src/test/modules/snapshot_too_old.
The sto_using_cursor and sto_using_select tests were coded to exercise
every permutation of their test steps, but AFAICS there is no value in
exercising more than one.  This matters because each permutation costs
about six seconds, thanks to the "pg_sleep(6)".  Perhaps we could
reduce that, but the useless permutations seem worth getting rid of
in any case.  (Note that sto_using_hash_index got it right already.)

While here, clean up some other sloppiness such as an unused table.

This doesn't make too much difference in interactive testing, since the
wasted time is typically masked by parallelization with other tests.
However, the buildfarm runs this as a serial step, which means we can
expect to shave ~40 seconds from every buildfarm run.  That makes it
worth back-patching.

Discussion: https://postgr.es/m/2515192.1659454702@sss.pgh.pa.us
2022-08-03 11:14:55 -04:00
Andrew Dunstan
c308003d22 Fix new recovery test for log_error_verbosity=verbose case
The new test is from commit 9e4f914b5e.

With this setting messages have SQL error numbers included, so that
needs to be provided for in the pattern looked for.

Backpatch to all live branches like the original.
2022-07-29 18:17:55 -04:00
Alvaro Herrera
6ffaf75a80
Fix test instability
On FreeBSD, the new test fails due to a WAL file being removed before
the standby has had the chance to copy it.  Fix by adding a replication
slot to prevent the removal until after the standby has connected.

Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reported-by: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAEze2Wj5nau_qpjbwihvmXLfkAWOZ5TKdbnqOc6nKSiRJEoPyQ@mail.gmail.com
2022-07-29 12:50:47 +02:00
Alvaro Herrera
084318c33a
Fix replay of create database records on standby
Crash recovery on standby may encounter missing directories
when replaying database-creation WAL records.  Prior to this
patch, the standby would fail to recover in such a case;
however, the directories could be legitimately missing.
Consider the following sequence of commands:

    CREATE DATABASE
    DROP DATABASE
    DROP TABLESPACE

If, after replaying the last WAL record and removing the
tablespace directory, the standby crashes and has to replay the
create database record again, crash recovery must be able to continue.

A fix for this problem was already attempted in 49d9cfc68b, but it
was reverted because of design issues.  This new version is based
on Robert Haas' proposal: any missing tablespaces are created
during recovery before reaching consistency.  Tablespaces
are created as real directories, and should be deleted
by later replay.  CheckRecoveryConsistency ensures
they have disappeared.

The problems detected by this new code are reported as PANIC,
except when allow_in_place_tablespaces is set to ON, in which
case they are WARNING.  Apart from making tests possible, this
gives users an escape hatch in case things don't go as planned.

Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Author: Asim R Praveen <apraveen@pivotal.io>
Author: Paul Guo <paulguo@gmail.com>
Reviewed-by: Anastasia Lubennikova <lubennikovaav@gmail.com> (older versions)
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com> (older versions)
Reviewed-by: Michaël Paquier <michael@paquier.xyz>
Diagnosed-by: Paul Guo <paulguo@gmail.com>
Discussion: https://postgr.es/m/CAEET0ZGx9AvioViLf7nbR_8tH9-=27DN5xWJ2P9-ROH16e4JUA@mail.gmail.com
2022-07-28 08:26:05 +02:00
Tom Lane
6bceacfe87 Fix ruleutils issues with dropped cols in functions-returning-composite.
Due to lack of concern for the case in the dependency code, it's
possible to drop a column of a composite type even though stored
queries have references to the dropped column via functions-in-FROM
that return the composite type.  There are "soft" references,
namely FROM-clause aliases for such columns, and "hard" references,
that is actual Vars referring to them.  The right fix for hard
references is to add dependencies preventing the drop; something
we've known for many years and not done (and this commit still doesn't
address it).  A "soft" reference shouldn't prevent a drop though.
We've been around on this before (cf. 9b35ddce9, 2c4debbd0), but
nobody had noticed that the current behavior can result in dump/reload
failures, because ruleutils.c can print more column aliases than the
underlying composite type now has.  So we need to rejigger the
column-alias-handling code to treat such columns as dropped and not
print aliases for them.

Rather than writing new code for this, I used expandRTE() which already
knows how to figure out which function result columns are dropped.
I'd initially thought maybe we could use expandRTE() in all cases, but
that fails for EXPLAIN's purposes, because the planner strips a lot of
RTE infrastructure that expandRTE() needs.  So this patch just uses it
for unplanned function RTEs and otherwise does things the old way.

If there is a hard reference (Var), then removing the column alias
causes us to fail to print the Var, since there's no longer a name
to print.  Failing seems less desirable than printing a made-up
name, so I made it print "?dropped?column?" instead.

Per report from Timo Stolz.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/5c91267e-3b6d-5795-189c-d15a55d61dbb@nullachtvierzehn.de
2022-07-21 13:56:02 -04:00
Dean Rasheed
8ace122d43 Fix alias matching in transformLockingClause().
When locking a specific named relation for a FOR [KEY] UPDATE/SHARE
clause, transformLockingClause() finds the relation to lock by
scanning the rangetable for an RTE with a matching eref->aliasname.
However, it failed to account for the visibility rules of a join RTE.

If a join RTE doesn't have a user-supplied alias, it will have a
generated eref->aliasname of "unnamed_join" that is not visible as a
relation name in the parse namespace. Such an RTE needs to be skipped,
otherwise it might be found in preference to a regular base relation
with a user-supplied alias of "unnamed_join", preventing it from being
locked.

In addition, if a join RTE doesn't have a user-supplied alias, but
does have a join_using_alias, then the RTE needs to be matched using
that alias rather than the generated eref->aliasname, otherwise a
misleading "relation not found" error will be reported rather than a
"join cannot be locked" error.

Backpatch all the way, except for the second part which only goes back
to 14, where JOIN USING aliases were added.

Dean Rasheed, reviewed by Tom Lane.

Discussion: https://postgr.es/m/CAEZATCUY_KOBnqxbTSPf=7fz9HWPnZ5Xgb9SwYzZ8rFXe7nb=w@mail.gmail.com
2022-07-07 13:07:51 +01:00
Noah Misch
ca590a4e95 Fix PostgreSQL::Test aliasing for Perl v5.10.1.
This Perl segfaults if a declaration of the to-be-aliased package
precedes the aliasing itself.  Per buildfarm members lapwing and wrasse.
Like commit 20911775de, back-patch to v10
(all supported versions).

Discussion: https://postgr.es/m/20220625171533.GA2012493@rfd.leadboat.com
2022-06-25 14:16:00 -07:00
Noah Misch
38790408b0 For PostgreSQL::Test compatibility, alias entire package symbol tables.
Remove the need to edit back-branch-specific code sites when
back-patching the addition of a PostgreSQL::Test::Utils symbol.  Replace
per-symbol, incomplete alias lists.  Give old and new package names the
same EXPORT and EXPORT_OK semantics.  Back-patch to v10 (all supported
versions).

Reviewed by Andrew Dunstan.

Discussion: https://postgr.es/m/20220622072144.GD4167527@rfd.leadboat.com
2022-06-25 09:07:46 -07:00
Peter Eisentraut
59fb360c48 Fix whitespace 2022-06-08 13:23:17 +02:00
Amit Kapila
37fd181d6d Ignore heap rewrites for materialized views in logical replication.
If you have a publication that specifies FOR ALL TABLES clause, a REFRESH
MATERIALIZED VIEW can break your setup with the following message

ERROR: logical replication target relation "public.pg_temp_NNN" does not exist

Commit 1a499c2520 introduces a heuristic to skip table writes that look
like they are from a heap rewrite for a FOR ALL TABLES publication.
However, it forgot to exclude materialized views (heap rewrites occur when
you execute REFRESH MATERIALIZED VIEW). Since materialized views are not
supported in logical decoding, it is appropriate to filter them too.

As explained in the commit 1a499c2520, a more robust solution was included
in version 11 (commit 325f2ec555), hence only version 10 is affected.

Reported-by: Euler Taveira
Author: Euler Taveira
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/bc557ebe-92dc-4afa-b6bb-285a9eeaa614@www.fastmail.com
2022-06-03 08:15:00 +05:30
Tom Lane
7686403b43 Show 'AS "?column?"' explicitly when it's important.
ruleutils.c was coded to suppress the AS label for a SELECT output
expression if the column name is "?column?", which is the parser's
fallback if it can't think of something better.  This is fine, and
avoids ugly clutter, so long as (1) nothing further up in the parse
tree relies on that column name or (2) the same fallback would be
assigned when the rule or view definition is reloaded.  Unfortunately
(2) is far from certain, both because ruleutils.c might print the
expression in a different form from how it was originally written
and because FigureColname's rules might change in future releases.
So we shouldn't rely on that.

Detecting exactly whether there is any outer-level use of a SELECT
column name would be rather expensive.  This patch takes the simpler
approach of just passing down a flag indicating whether there *could*
be any outer use; for example, the output column names of a SubLink
are not referenceable, and we also do not care about the names exposed
by the right-hand side of a setop.  This is sufficient to suppress
unwanted clutter in all but one case in the regression tests.  That
seems like reasonable evidence that it won't be too much in users'
faces, while still fixing the cases we need to fix.

Per bug #17486 from Nicolas Lutic.  This issue is ancient, so
back-patch to all supported branches.

Discussion: https://postgr.es/m/17486-1ad6fd786728b8af@postgresql.org
2022-05-21 14:45:58 -04:00
Alvaro Herrera
8c47622bb3
Fix mis-merge of result file
Failed to "git add" this file in this branch.
2022-05-20 19:05:55 +02:00
Alvaro Herrera
70f70d7d3f
Fix DDL deparse of CREATE OPERATOR CLASS
When an implicit operator family is created, it wasn't getting reported.
Make it do so.

This has always been missing.  Backpatch to 10.

Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reported-by: Leslie LEMAIRE <leslie.lemaire@developpement-durable.gouv.fr>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Michael Paquiër <michael@paquier.xyz>
Discussion: https://postgr.es/m/f74d69e151b22171e8829551b1159e77@developpement-durable.gouv.fr
2022-05-20 18:52:55 +02:00
Alvaro Herrera
fa51cc86c6
Backpatch regression tests added by 2d689babe3
A new plpgsql test function was added in 14 and up to cover for a bugfix
that was not backpatchable.  We can add it to older versions as a way to
cover other bits of DDL event triggers, with an exception clause to
avoid the problematic corner case.

Originally authored by Michaël Paquier.

Backpatch: 10 through 13.

Discussion: https://postgr.es/m/202205201523.7m5jbfvyanmj@alvherre.pgsql
2022-05-20 17:52:16 +02:00
Alvaro Herrera
29d1115187
Update xml_1.out and xml_2.out
Commit 0fbf011200 should have updated them but didn't.
2022-05-18 23:19:53 +02:00
Alvaro Herrera
16cb7db34f
Check column list length in XMLTABLE/JSON_TABLE alias
We weren't checking the length of the column list in the alias clause of
an XMLTABLE or JSON_TABLE function (a "tablefunc" RTE), and it was
possible to make the server crash by passing an overly long one.  Fix it
by throwing an error in that case, like the other places that deal with
alias lists.

In passing, modify the equivalent test used for join RTEs to look like
the other ones, which was different for no apparent reason.

This bug came in when XMLTABLE was born in version 10; backpatch to all
stable versions.

Reported-by: Wang Ke <krking@zju.edu.cn>
Discussion: https://postgr.es/m/17480-1c9d73565bb28e90@postgresql.org
2022-05-18 20:28:31 +02:00
Tom Lane
b53442f6fe Make pull_var_clause() handle GroupingFuncs exactly like Aggrefs.
This follows in the footsteps of commit 2591ee8ec by removing one more
ill-advised shortcut from planning of GroupingFuncs.  It's true that
we don't intend to execute the argument expression(s) at runtime, but
we still have to process any Vars appearing within them, or we risk
failure at setrefs.c time (or more fundamentally, in EXPLAIN trying
to print such an expression).  Vars in upper plan nodes have to have
referents in the next plan level, whether we ever execute 'em or not.

Per bug #17479 from Michael J. Sullivan.  Back-patch to all supported
branches.

Richard Guo

Discussion: https://postgr.es/m/17479-6260deceaf0ad304@postgresql.org
2022-05-12 11:31:46 -04:00
Tom Lane
4eabaffcad Fix core dump in transformValuesClause when there are no columns.
The parser code that transformed VALUES from row-oriented to
column-oriented lists failed if there were zero columns.
You can't write that straightforwardly (though probably you
should be able to), but the case can be reached by expanding
a "tab.*" reference to a zero-column table.

Per bug #17477 from Wang Ke.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/17477-0af3c6ac6b0a6ae0@postgresql.org
2022-05-09 14:15:37 -04:00
Tom Lane
86a21803c7 Revert "Disallow infinite endpoints in generate_series() for timestamps."
This reverts commit eafdf9de06
and its back-branch counterparts.  Corey Huinker pointed out that
we'd discussed this exact change back in 2016 and rejected it,
on the grounds that there's at least one usage pattern with LIMIT
where an infinite endpoint can usefully be used.  Perhaps that
argument needs to be re-litigated, but there's no time left before
our back-branch releases.  To keep our options open, restore the
status quo ante; if we do end up deciding to change things, waiting
one more quarter won't hurt anything.

Rather than just doing a straight revert, I added a new test case
demonstrating the usage with LIMIT.  That'll at least remind us of
the issue if we forget again.

Discussion: https://postgr.es/m/3603504.1652068977@sss.pgh.pa.us
Discussion: https://postgr.es/m/CADkLM=dzw0Pvdqp5yWKxMd+VmNkAMhG=4ku7GnCZxebWnzmz3Q@mail.gmail.com
2022-05-09 11:40:42 -04:00
Noah Misch
f26d570285 In REFRESH MATERIALIZED VIEW, set user ID before running user code.
It intended to, but did not, achieve this.  Adopt the new standard of
setting user ID just after locking the relation.  Back-patch to v10 (all
supported versions).

Reviewed by Simon Riggs.  Reported by Alvaro Herrera.

Security: CVE-2022-1552
2022-05-09 08:35:13 -07:00
Noah Misch
ef792f7856 Make relation-enumerating operations be security-restricted operations.
When a feature enumerates relations and runs functions associated with
all found relations, the feature's user shall not need to trust every
user having permission to create objects.  BRIN-specific functionality
in autovacuum neglected to account for this, as did pg_amcheck and
CLUSTER.  An attacker having permission to create non-temp objects in at
least one schema could execute arbitrary SQL functions under the
identity of the bootstrap superuser.  CREATE INDEX (not a
relation-enumerating operation) and REINDEX protected themselves too
late.  This change extends to the non-enumerating amcheck interface.
Back-patch to v10 (all supported versions).

Sergey Shinderuk, reviewed (in earlier versions) by Alexander Lakhin.
Reported by Alexander Lakhin.

Security: CVE-2022-1552
2022-05-09 08:35:13 -07:00
Andres Freund
390b489eff Disable 031_recovery_conflict.pl until after minor releases.
f40d362a66 disabled part of 031_recovery_conflict.pl due to instability
that's not trivial to fix in the back branches. That fixed most of the
issues. But there was one more failure (on lapwing / REL_10_STABLE).

That failure looks like it might be caused by a genuine problem. Disable the
test until after the set of releases, to avoid packagers etc potentially
having to fight with a test failure they can't do anything about.

Discussion: https://postgr.es/m/3447060.1652032749@sss.pgh.pa.us
Backpatch: 10-14
2022-05-08 18:06:42 -07:00
Andres Freund
441fa7a63a Temporarily skip recovery deadlock test in back branches.
The recovery deadlock test has a timing issue that was fixed in 5136967f1e in
HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1)
adjust_conf() doesn't exist, which is easy enough to work around 2) a restart
cleares the recovery conflict stats < 15.

These issues can be worked around, but given the upcoming set of minor
releases, skip the problematic test for now. The buildfarm doesn't show
failures in other parts of 031_recovery_conflict.pl.

Discussion: https://postgr.es/m/20220506155827.dfnaheq6ufylwrqf@alap3.anarazel.de
Backpatch: 10-14
2022-05-06 09:07:44 -07:00
Andres Freund
14f9d8b7e3 Revert "Fix timing issue in deadlock recovery conflict test."
This reverts commit 3197e0f5ae.
2022-05-04 14:15:20 -07:00
Andres Freund
3197e0f5ae Fix timing issue in deadlock recovery conflict test.
Per buildfarm members longfin and skink.

Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-
2022-05-04 12:57:32 -07:00
Andres Freund
b1f35a36e6 Backpatch 031_recovery_conflict.pl.
The prior commit showed that the introduction of recovery conflict tests was a
good idea. Without these tests it's hard to know that the fix didn't break
something...

031_recovery_conflict.pl was introduced in 9f8a050f68 and extended in
21e184403b.

Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-14
2022-05-02 18:30:38 -07:00
Andres Freund
e8a0cf9b20 Backpatch addition of wait_for_log(), pump_until().
These were originally introduced in a2ab9c06ea and a2ab9c06ea, as they are
needed by a about-to-be-backpatched test.

Discussion: https://postgr.es/m/20220413002626.udl7lll7f3o7nre7@alap3.anarazel.de
Backpatch: 10-14
2022-05-02 18:09:44 -07:00
Andrew Dunstan
5d223e4e06 Support new perl module namespace in stable branches
Commit b3b4d8e68a moved our perl test modules to a better namespace
structure, but this has made life hard for people wishing to backpatch
improvements in the TAP tests. Here we alleviate much of that difficulty
by implementing the new module names on top of the old modules, mostly
by using a little perl typeglob aliasing magic, so that we don't have a
dual maintenance burden. This should work both for the case where a new
test is backpatched and the case where a fix to an existing test that
uses the new namespace is backpatched.

Reviewed by Michael Paquier

Per complaint from Andres Freund

Discussion: https://postgr.es/m/20220418141530.nfxtkohefvwnzncl@alap3.anarazel.de

Applied to branches 10 through 14
2022-04-21 09:38:20 -04:00
Peter Geoghegan
a903895b3f Fix CLUSTER tuplesorts on abbreviated expressions.
CLUSTER sort won't use the datum1 SortTuple field when clustering
against an index whose leading key is an expression.  This makes it
unsafe to use the abbreviated keys optimization, which was missed by the
logic that sets up SortSupport state.  Affected tuplesorts output tuples
in a completely bogus order as a result (the wrong SortSupport based
comparator was used for the leading attribute).

This issue is similar to the bug fixed on the master branch by recent
commit cc58eecc5d.  But it's a far older issue, that dates back to the
introduction of the abbreviated keys optimization by commit 4ea51cdfe8.

Backpatch to all supported versions.

Author: Peter Geoghegan <pg@bowt.ie>
Author: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA+hUKG+bA+bmwD36_oDxAoLrCwZjVtST2fqe=b4=qZcmU7u89A@mail.gmail.com
Backpatch: 10-
2022-04-20 17:17:33 -07:00
Tom Lane
a1e4782a0b Disallow infinite endpoints in generate_series() for timestamps.
Such cases will lead to infinite loops, so they're of no practical
value.  The numeric variant of generate_series() already threw error
for this, so borrow its message wording.

Per report from Richard Wesley.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/91B44E7B-68D5-448F-95C8-B4B3B0F5DEAF@duckdblabs.com
2022-04-20 18:08:15 -04:00
Alvaro Herrera
9ade3c09a5
Fix "missing continuation record" after standby promotion
Invalidate abortedRecPtr and missingContrecPtr after a missing
continuation record is successfully skipped on a standby. This fixes a
PANIC caused when a recently promoted standby attempts to write an
OVERWRITE_RECORD with an LSN of the previously read aborted record.

Backpatch to 10 (all stable versions).

Author: Sami Imseih <simseih@amazon.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/44D259DE-7542-49C4-8A52-2AB01534DCA9@amazon.com
2022-03-23 18:22:10 +01:00
Michael Paquier
8138bd4a56 Fix failures in SSL tests caused by out-of-tree keys and certificates
This issue is environment-sensitive, where the SSL tests could fail in
various way by feeding on defaults provided by sslcert, sslkey,
sslrootkey, sslrootcert, sslcrl and sslcrldir coming from a local setup,
as of ~/.postgresql/ by default.  Horiguchi-san has reported two
failures, but more advanced testing from me (aka inclusion of garbage
SSL configuration in ~/.postgresql/ for all the configuration
parameters) has showed dozens of failures that can be triggered in the
whole test suite.

History has showed that we are not good when it comes to address such
issues, fixing them locally like in dd87799, and such problems keep
appearing.  This commit strengthens the entire test suite to put an end
to this set of problems by embedding invalid default values in all the
connection strings used in the tests.  The invalid values are prefixed
in each connection string, relying on the follow-up values passed in the
connection string to enforce any invalid value previously set.  Note
that two tests related to CRLs are required to fail with certain pre-set
configurations, but we can rely on enforcing an empty value instead
after the invalid set of values.

Reported-by: Kyotaro Horiguchi
Reviewed-by: Andrew Dunstan, Daniel Gustafsson, Kyotaro Horiguchi
Discussion: https://postgr.es/m/20220316.163658.1122740600489097632.horikyota.ntt@gmail.com
backpatch-through: 10
2022-03-22 13:21:58 +09:00
Tom Lane
2afa031ac5 Fix assorted missing logic for GroupingFunc nodes.
The planner needs to treat GroupingFunc like Aggref for many purposes,
in particular with respect to processing of the argument expressions,
which are not to be evaluated at runtime.  A few places hadn't gotten
that memo, notably including subselect.c's processing of outer-level
aggregates.  This resulted in assertion failures or wrong plans for
cases in which a GROUPING() construct references an outer aggregation
level.

Also fix missing special cases for GroupingFunc in cost_qual_eval
(resulting in wrong cost estimates for GROUPING(), although it's
not clear that that would affect plan shapes in practice) and in
ruleutils.c (resulting in excess parentheses in pretty-print mode).

Per bug #17088 from Yaoguang Chen.  Back-patch to all supported
branches.

Richard Guo, Tom Lane

Discussion: https://postgr.es/m/17088-e33882b387de7f5c@postgresql.org
2022-03-21 17:44:29 -04:00
Tom Lane
e6fd4a3daf Fix incorrect xmlschema output for types timetz and timestamptz.
The output of table_to_xmlschema() and allied functions includes
a regex describing valid values for these types ... but the regex
was itself invalid, as it failed to escape a literal "+" sign.

Report and fix by Renan Soares Lopes.  Back-patch to all
supported branches.

Discussion: https://postgr.es/m/7f6fabaa-3f8f-49ab-89ca-59fbfe633105@me.com
2022-03-18 16:01:42 -04:00
Tom Lane
205214c8b8 Revert applying column aliases to the output of whole-row Vars.
In commit bf7ca1587, I had the bright idea that we could make the
result of a whole-row Var (that is, foo.*) track any column aliases
that had been applied to the FROM entry the Var refers to.  However,
that's not terribly logically consistent, because now the output of
the Var is no longer of the named composite type that the Var claims
to emit.  bf7ca1587 tried to handle that by changing the output
tuple values to be labeled with a blessed RECORD type, but that's
really pretty disastrous: we can wind up storing such tuples onto
disk, whereupon they're not readable by other sessions.

The only practical fix I can see is to give up on what bf7ca1587
tried to do, and say that the column names of tuples produced by
a whole-row Var are always those of the underlying named composite
type, query aliases or no.  While this introduces some inconsistencies,
it removes others, so it's not that awful in the abstract.  What *is*
kind of awful is to make such a behavioral change in a back-patched
bug fix.  But corrupt data is worse, so back-patched it will be.

(A workaround available to anyone who's unhappy about this is to
introduce an extra level of sub-SELECT, so that the whole-row Var is
referring to the sub-SELECT's output and not to a named table type.
Then the Var is of type RECORD to begin with and there's no issue.)

Per report from Miles Delahunty.  The faulty commit dates to 9.5,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/2950001.1638729947@sss.pgh.pa.us
2022-03-17 18:18:05 -04:00
Noah Misch
7f2e1befc0 Introduce PG_TEST_TIMEOUT_DEFAULT for TAP suite non-elapsing timeouts.
Slow hosts may avoid load-induced, spurious failures by setting
environment variable PG_TEST_TIMEOUT_DEFAULT to some number of seconds
greater than 180.  Developers may see faster failures by setting that
environment variable to some lesser number of seconds.  In tests, write
$PostgreSQL::Test::Utils::timeout_default wherever the convention has
been to write 180.  This change raises the default for some briefer
timeouts.  Back-patch to v10 (all supported versions).

Discussion: https://postgr.es/m/20220218052842.GA3627003@rfd.leadboat.com
2022-03-04 18:53:17 -08:00