Commit graph

7329 commits

Author SHA1 Message Date
Michael Paquier
ea15816928 psql: Fix category of \parse in output of --help=commands and \?
\parse was listed under the category "Connection", which was incorrect.
Let's move it to "General" like the other meta-commands of the same type
(\bind, \bind_named and \close).

Oversight in commit d55322b0da.

Discussion: https://postgr.es/m/Zz_x-NEKNeeRlAVc@paquier.xyz
2024-11-22 14:04:21 +09:00
Michael Paquier
768dfd8e65 psql: Include \pset xheader_width in --help=commands|variables
psql's --help was missed the description of the \pset variable
xheader_width, that should be listed when using \? or --help=commands,
and described for --help=variables.

Oversight in a45388d6e0.

Author: Pavel Luzanov
Discussion: https://postgr.es/m/1e3e06d6-0807-4e62-a9f6-c11481e6eb10@postgrespro.ru
Backpatch-through: 16
2024-11-22 12:17:37 +09:00
Álvaro Herrera
b5be29ecaf
Fix newly introduced 010_keep_recycled_wals.pl
It failed to set the archive_command as it desired because of a syntax
problem.  Oversight in commit 90bcc7c2db.

This bug doesn't cause the test to fail, because the test only checks
pg_rewind's output messages, not the actual outcome (and the outcome in
both cases is that the file is kept, not deleted).  But in either case
the message about the file being kept is there, so it's hard to get
excited about doing much more.

Reported-by: Antonin Houska <ah@cybertec.at>
Author: Alexander Kukushkin <cyberdemn@gmail.com>
Discussion: https://postgr.es/m/7822.1732167825@antos
2024-11-21 17:04:26 +01:00
Bruce Momjian
f722dd32de clarify --no-comments option in --help and SGML files
The previous commit, b38bac26e2, missed these cases for dump/restore.

Reported-by: Tom Lane

Discussion: https://postgr.es/m/3495698.1731968093@sss.pgh.pa.us

Backpatch-through: master
2024-11-20 14:48:31 -05:00
Álvaro Herrera
90bcc7c2db
Avoid deleting critical WAL segments during pg_rewind
Previously, in unlucky cases, it was possible for pg_rewind to remove
certain WAL segments from the rewound demoted primary.  In particular
this happens if those files have been marked for archival (i.e., their
.ready files were created) but not yet archived; the newly promoted node
no longer has such files because of them having been recycled, but they
are likely critical for recovery in the demoted node.  If pg_rewind
removes them, recovery is not possible anymore.

Fix this by maintaining a hash table of files in this situation in the
scan that looks for a checkpoint, which the decide_file_actions phase
can consult so that it knows to preserve them.

Backpatch to 14.  The problem also exists in 13, but that branch was not
blessed with commit eb00f1d4bf, so this patch is difficult to apply
there.  Users of older releases will just have to continue to be extra
careful when rewinding.

Co-authored-by: Полина Бунгина (Polina Bungina) <bungina@gmail.com>
Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Discussion: https://postgr.es/m/CAAtGL4AhzmBRsEsaDdz7065T+k+BscNadfTqP1NcPmsqwA5HBw@mail.gmail.com
2024-11-15 12:53:12 +01:00
Álvaro Herrera
38c18710b3
Fix pg_upgrade's cross-version tests when old < 18
Because in the 18 cycle we turned checksums on by default with commit
04bec894a0, and pg_upgrade fails if the setting doesn't match in old
and new clusters, the built-in cross-version pg_upgrade test is failing
if the old version is older than 18.  Fix the script so that it creates
the old cluster with checksums enabled (-k) in cross-version scenarios.

This went unnoticed because the buildfarm doesn't use the same test code
for cross-version testing.

Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/202411071838.7fgkb7uvavvz@alvherre.pgsql
2024-11-13 11:06:44 +01:00
Álvaro Herrera
14e87ffa5c
Add pg_constraint rows for not-null constraints
We now create contype='n' pg_constraint rows for not-null constraints on
user tables.  Only one such constraint is allowed for a column.

We propagate these constraints to other tables during operations such as
adding inheritance relationships, creating and attaching partitions and
creating tables LIKE other tables.  These related constraints mostly
follow the well-known rules of conislocal and coninhcount that we have
for CHECK constraints, with some adaptations: for example, as opposed to
CHECK constraints, we don't match not-null ones by name when descending
a hierarchy to alter or remove it, instead matching by the name of the
column that they apply to.  This means we don't require the constraint
names to be identical across a hierarchy.

The inheritance status of these constraints can be controlled: now we
can be sure that if a parent table has one, then all children will have
it as well.  They can optionally be marked NO INHERIT, and then children
are free not to have one.  (There's currently no support for altering a
NO INHERIT constraint into inheriting down the hierarchy, but that's a
desirable future feature.)

This also opens the door for having these constraints be marked NOT
VALID, as well as allowing UNIQUE+NOT NULL to be used for functional
dependency determination, as envisioned by commit e49ae8d3bc.  It's
likely possible to allow DEFERRABLE constraints as followup work, as
well.

psql shows these constraints in \d+, though we may want to reconsider if
this turns out to be too noisy.  Earlier versions of this patch hid
constraints that were on the same columns of the primary key, but I'm
not sure that that's very useful.  If clutter is a problem, we might be
better off inventing a new \d++ command and not showing the constraints
in \d+.

For now, we omit these constraints on system catalog columns, because
they're unlikely to achieve anything.

The main difference to the previous attempt at this (b0e96f3119) is
that we now require that such a constraint always exists when a primary
key is in the column; we didn't require this previously which had a
number of unpalatable consequences.  With this requirement, the code is
easier to reason about.  For example:

- We no longer have "throwaway constraints" during pg_dump.  We needed
  those for the case where a table had a PK without a not-null
  underneath, to prevent a slow scan of the data during restore of the
  PK creation, which was particularly problematic for pg_upgrade.

- We no longer have to cope with attnotnull being set spuriously in
  case a primary key is dropped indirectly (e.g., via DROP COLUMN).

Some bits of code in this patch were authored by Jian He.

Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Author: Bernd Helmle <mailings@oopsware.de>
Reviewed-by: 何建 (jian he) <jian.universality@gmail.com>
Reviewed-by: 王刚 (Tender Wang) <tndrwang@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Discussion: https://postgr.es/m/202408310358.sdhumtyuy2ht@alvherre.pgsql
2024-11-08 13:28:48 +01:00
Amit Kapila
7054186c4e Replicate generated columns when 'publish_generated_columns' is set.
This patch builds on the work done in commit 745217a051 by enabling the
replication of generated columns alongside regular column changes through
a new publication parameter: publish_generated_columns.

Example usage:
CREATE PUBLICATION pub1 FOR TABLE tab_gencol WITH (publish_generated_columns = true);

The column list takes precedence. If the generated columns are specified
in the column list, they will be replicated even if
'publish_generated_columns' is set to false. Conversely, if generated
columns are not included in the column list (assuming the user specifies a
column list), they will not be replicated even if
'publish_generated_columns' is true.

Author: Vignesh C, Shubham Khanna
Reviewed-by: Peter Smith, Amit Kapila, Hayato Kuroda, Shlok Kyal, Ajin Cherian, Hou Zhijie, Masahiko Sawada
Discussion: https://postgr.es/m/B80D17B2-2C8E-4C7D-87F2-E5B4BE3C069E@gmail.com
2024-11-07 08:58:49 +05:30
Daniel Gustafsson
f638aafd1e Find invalid databases during upgrade check stage
Before continuing with the check start by checking that all databases
allow connections to avoid a hard fail without proper error reporting.

Inspired by a larger patch by Thomas Krennwallner.

Discussion: https://postgr.es/m/f9315bf0-e03e-4490-9f0d-5b6f7a6d9908@postsubmeta.net
2024-11-06 15:40:52 +01:00
Peter Eisentraut
ecb5af7798 Remove unused #include's from bin .c files
as determined by IWYU

Similar to commit dbbca2cf29, but for bin and some related files.

Discussion: https://www.postgresql.org/message-id/flat/0df1d5b1-8ca8-4f84-93be-121081bde049%40eisentraut.org
2024-11-06 11:11:52 +01:00
Tom Lane
b1008c1f01 pg_basebackup, pg_receivewal: fix failure to find password in ~/.pgpass.
Sloppy refactoring in commit cca97ce6a caused these programs
to pass dbname = NULL to libpq if there was no "--dbname" switch
on the command line, where before "replication" would be passed.
This didn't break things completely, because the source server doesn't
care about the dbname specified for a physical replication connection.
However, it did cause libpq to fail to match a ~/.pgpass entry that
has "replication" in the dbname field.  Restore the previous behavior
of passing "replication".

Also, closer inspection shows that if you do specify a dbname
in the connection string, that is what will be matched to ~/.pgpass,
not "replication".  This was the pre-existing behavior so we should
not change it, but the SGML docs were pretty misleading about it.
Improve that.

Per bug #18685 from Toshi Harada.  Back-patch to v17 where the
error crept in.

Discussion: https://postgr.es/m/18685-fee2dd142b9688f1@postgresql.org
Discussion: https://postgr.es/m/2702546.1730740456@sss.pgh.pa.us
2024-11-04 14:36:11 -05:00
Tom Lane
350e6b8ea8 pg_dump: provide a stable sort order for rules.
Previously, we sorted rules by schema name and then rule name;
if that wasn't unique, we sorted by rule OID.  This can be
problematic for comparing dumps from databases with different
histories, especially since certain rule names like "_RETURN"
are very common.  Let's make the sort key schema name, rule name,
table name, which should be unique.  (This is the same behavior
we've long used for triggers and RLS policies.)

Andreas Karlsson

Discussion: https://postgr.es/m/b4e468d8-0cd6-42e6-ac8a-1d6afa6e0cf1@proxel.se
2024-11-04 13:31:12 -05:00
Robert Haas
0b1765d959 pg_combinebackup: Error if incremental file exists in full backup.
Suppose that you run a command like "pg_combinebackup b1 b2 -o output",
but both b1 and b2 contain an INCREMENTAL.$something file in a directory
that is expected to contain relation files. This is an error, but the
previous code would not detect the problem and instead write a garbage
full file named $something to the output directory. This commit adds
code to detect the error and a test case to verify the behavior.

It's difficult to imagine that this will ever happen unless someone
is intentionally trying to break incremental backup, but per discussion,
let's consider that the lack of adequate sanity checking in this area is
a bug and back-patch to v17, where incremental backup was introduced.

Patch by me, reviewed by Bertrand Drouvot and Amul Sul.

Discussion: http://postgr.es/m/CA+TgmoaD7dBYPqe7kMtO0dyto7rd0rUh7joh=JPUSaFszKY6Pg@mail.gmail.com
2024-11-04 10:11:05 -05:00
Robert Haas
6c24801b17 pg_combinebackup: When reconstructing, avoid double slash in filename.
This function is always called with a relative_path that ends in a
slash, so there's no need to insert a second one. So, don't. Instead,
add an assertion to verify that nothing gets broken in the future, and
adjust the comments.

While this is not a critical bug, the duplicate slash is visible in
error messages, which could create confusion, so back-patch to v17.
This is also better in that it keeps the code consistent across
branches.

Patch by me, reviewed by Bertrand Drouvot and Amul Sul.

Discussion: http://postgr.es/m/CA+TgmoaD7dBYPqe7kMtO0dyto7rd0rUh7joh=JPUSaFszKY6Pg@mail.gmail.com
2024-11-04 09:55:02 -05:00
Amit Kapila
1bf1140be8 Change the default value of the streaming option to 'parallel'.
Previously the default value of streaming option for a subscription was
'off'. The parallel option indicates that the changes in large
transactions (greater than logical_decoding_work_mem) are to be applied
directly via one of the parallel apply workers, if available.

The parallel mode was introduced in 16, but we refrain from enabling it by
default to avoid seeing any unpleasant behavior in the existing
applications. However we haven't found any such report yet, so this is a
good time to enable it by default.

Reported-by: Vignesh C
Author: Hayato Kuroda, Masahiko Sawada, Peter Smith, Amit Kapila
Discussion: https://postgr.es/m/CALDaNm1=MedhW23NuoePJTmonwsMSp80ddsw+sEJs0GUMC_kqQ@mail.gmail.com
2024-10-28 08:42:05 +05:30
Tatsuo Ishii
7175ef870e pgbench: Fix typo.
Fix typo in commit cae0f3c405.

Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/70eaa41b-805b-ce19-6004-5a0dccd3f731%40gmail.com
2024-10-25 15:06:16 +09:00
Fujii Masao
9272bdeac8 Remove unused code for unlogged materialized views.
Commit 3bf3ab8c56 initially introduced support for unlogged
materialized views, but this was later disallowed by commit 3223b25ff7.
Additionally, commit d25f519107 added more code for handling
unlogged materialized views. This commit cleans up all unused
code related to them.

If unlogged materialized views had been supported in any official
release, psql would need to retain code to handle them for compatibility
with older servers. However, since they were never included in
an official release, this code is no longer necessary.

Author: Pixian Shi
Reviewed-by: Yugo Nagata, Fujii Masao
Discussion: https://postgr.es/m/CAAccyYKRZ=OvAvgowiSH+OELbStLP=p2Ht=R3CgT=OaNSH5DAA@mail.gmail.com
2024-10-18 17:18:57 +09:00
Peter Eisentraut
04bec894a0 initdb: Change default to using data checksums.
Checksums are now on by default.  They can be disabled by the
previously added option --no-data-checksums.

Author: Greg Sabino Mullane <greg@turnstep.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://www.postgresql.org/message-id/flat/CAKAnmmKwiMHik5AHmBEdf5vqzbOBbcwEPHo4-PioWeAbzwcTOQ@mail.gmail.com
2024-10-16 08:48:10 +02:00
Peter Eisentraut
e7d0cf42b1 Allow TAP tests to force checksums off when calling init()
TAP tests can write

    $node->init(no_data_checksums => 1);

to initialize a cluster explicitly without checksums.  Currently, this
is the default, but this change allows running all tests with
checksums enabled, like

    PG_TEST_INITDB_EXTRA_OPTS=--data-checksums meson test ...

And this also prepares the tests for when we switch the default to
checksums enabled.

The pg_checksums tests need to disable checksums so it can test its
own functionality of enabling checksums.  The amcheck/pg_amcheck tests
need to disable checksums because they manually introduce corruption
that they want to detect, but with checksums enabled, the checksum
verification will fail before they even get to their work.

Author: Greg Sabino Mullane <greg@turnstep.com>
Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/CAKAnmmKwiMHik5AHmBEdf5vqzbOBbcwEPHo4-PioWeAbzwcTOQ@mail.gmail.com
2024-10-14 11:25:03 +02:00
Michael Paquier
9f34cae142 psql: Fix \watch when using interval values less than 1ms
Attempting to use an interval of time less than 1ms would cause \watch
to hang.  This was confusing, so let's change the logic so as an
interval lower than 1ms behaves the same as 0.

Comments are added to mention that the internals of do_watch() had
better rely on "sleep_ms", the interval value in milliseconds.  While on
it, this commit adds a test to check the behavior of interval values
less than 1ms.

\watch hanging for interval values less than 1ms existed before
6f9ee74d45, that has changed the code to support an interval value of
0.

Reported-by: Heikki Linnakangas
Author: Andrey M. Borodin, Michael Paquier
Discussion: https://postgr.es/m/88445e0e-3156-4b9d-afae-9a1a7b1631f6@iki.fi
Backpatch-through: 16
2024-10-14 12:27:51 +09:00
Tatsuo Ishii
cae0f3c405 pgbench: Improve result outputs related to failed transactions.
Previously, per-script statistics were never output when all
transactions failed due to serialization or deadlock errors.  However,
it is reasonable to report such information if there are ones even
when there are no successful transaction since these failed
transactions are now objects to be reported.

Meanwhile, if the total number of successful, skipped, and failed
transactions is zero, we don't have to report the number of failed
transactions as similar to the number of skipped transactions, which
avoids to print "NaN%" in lines on failed transaction reports.

Also, the number of transactions in per-script results now includes
skipped and failed transactions. It prevents to print "total of NaN%"
when any transactions are not successfully processed. The number of
transactions actually processed per-script and TPS based on it are now
output explicitly in a separate line.

Author: Yugo Nagata
Reviewed-by: Tatsuo Ishii
Discussion: https://postgr.es/m/20240921003544.2436ef8da9c5c8cb963c651b%40sraoss.co.jp
2024-10-11 13:40:23 +09:00
Tom Lane
c75c6f8d28 Don't hard-code the input file name in gen_tabcomplete.pl's output.
Use $ARGV[0], that is the specified input file name, in #line
directives generated by gen_tabcomplete.pl.  This makes code
coverage reports work properly in the meson build system (where
the input file name will be a relative path).

Also fix up brain fade in the meson build rule for tab-complete.c:
we only need to write the input file name once not twice.

Jacob Champion (some cosmetic adjustments by me)

Discussion: https://postgr.es/m/CAOYmi+=+oWAoi8pqnH0MJQqsSn4ddzqDhqRQJvyiN2aJSWvw2w@mail.gmail.com
2024-10-10 17:02:08 -04:00
Tom Lane
95eb4cd4ff Avoid possible segfault in psql's tab completion.
Fix oversight in bd1276a3c: the "words_after_create" stanza in
psql_completion() requires previous_words_count > 0, since it uses
prev_wd.  This condition was formerly assured by the if-else chain
above it, but no more.  If there were no previous words then we'd
dereference an uninitialized pointer, possibly causing a segfault.

Report and patch by Anthonin Bonnefoy.

Discussion: https://postgr.es/m/CAO6_XqrSRE7c_i+D7Hm07K3+6S0jTAmMr60RY41XzaA29Ae5uA@mail.gmail.com
2024-10-10 16:17:38 -04:00
Tom Lane
c01fd93088 Silence buildfarm warning chatter from bd1276a3c.
Buildfarm members using -Wextra complained about "warning: suggest
braces around empty body in an 'if' statement".  Do it gcc's way,
though I see no actual readability benefit in this.
2024-10-08 11:15:16 -04:00
Nathan Bossart
8318f2b170 vacuumdb: Schema-qualify operator in catalog query's WHERE clause.
Commit 1ab67c9dfa, which modified this catalog query so that it
doesn't return temporary relations, forgot to schema-qualify the
operator.  A comment earlier in the function implores us to fully
qualify everything in the query:

	 * Since we execute the constructed query with the default search_path
	 * (which could be unsafe), everything in this query MUST be fully
	 * qualified.

This commit fixes that.  While at it, add a newline for consistency
with surrounding code.

Reviewed-by: Noah Misch
Discussion: https://postgr.es/m/ZwQJYcuPPUsF0reU%40nathan
Backpatch-through: 12
2024-10-07 16:49:20 -05:00
Nathan Bossart
5d6187d2a2 Fix Y2038 issues with MyStartTime.
Several places treat MyStartTime as a "long", which is only 32 bits
wide on some platforms.  In reality, MyStartTime is a pg_time_t,
i.e., a signed 64-bit integer.  This will lead to interesting bugs
on the aforementioned systems in 2038 when signed 32-bit integers
are no longer sufficient to store Unix time (e.g., "pg_ctl start"
hanging).  To fix, ensure that MyStartTime is handled as a 64-bit
value everywhere.  (Of course, users will need to ensure that
time_t is 64 bits wide on their system, too.)

Co-authored-by: Max Johnson
Discussion: https://postgr.es/m/CO1PR07MB905262E8AC270FAAACED66008D682%40CO1PR07MB9052.namprd07.prod.outlook.com
Backpatch-through: 12
2024-10-07 13:51:03 -05:00
Tom Lane
f391d9dc93 Convert tab-complete's long else-if chain to a switch statement.
Rename tab-complete.c to tab-complete.in.c, create the preprocessor
script gen_tabcomplete.pl, and install Makefile/meson.build rules
to create tab-complete.c from tab-complete.in.c.  The preprocessor
converts match_previous_words' else-if chain into a switch and
populates tcpatterns[] with the data needed by the driver loop.

The initial HeadMatches/TailMatches/Matches test in each else-if arm
is now performed in a table-driven loop.  Where we get a match, the
corresponding switch case is invoked to see if the match succeeds.
(It might not, if there were additional conditions in the original
else-if test.)

The total number of string comparisons done is just about the
same as it was in the previous coding; however, now that we
have table-driven logic underlying the handmade rules, there
is room to improve that.  For now I haven't bothered because
tab completion is still plenty fast enough for human use.
If the number of rules keeps increasing, we might someday
need to do more in that area.

The immediate benefit of all this thrashing is that C compilers
frequently don't deal well with long else-if chains.  On gcc 8.5.0,
this reduces the compile time of tab-complete.c by about a factor of
four, while MSVC is reported to crash outright with the previous
coding.

Discussion: https://postgr.es/m/2208466.1720729502@sss.pgh.pa.us
2024-10-07 12:22:10 -04:00
Tom Lane
bd1276a3c9 Prepare tab-complete.c for preprocessing.
Separate out psql_completion's giant else-if chain of *Matches
tests into a new function.  Add the infrastructure needed for
table-driven checking of the initial match of each completion
rule.  As-is, however, the code continues to operate as it did.
The new behavior applies only if SWITCH_CONVERSION_APPLIED
is #defined, which it is not here.  (The preprocessor added
in the next patch will add a #define for that.)

The first and last couple of bits of psql_completion are not
based on HeadMatches/TailMatches/Matches tests, so they stay
where they are; they won't become part of the switch.

This patch also fixes up a couple of if-conditions that didn't meet
the conditions enumerated in the comment for match_previous_words().
Those restrictions exist to simplify the preprocessor.

Discussion: https://postgr.es/m/2208466.1720729502@sss.pgh.pa.us
2024-10-07 12:19:12 -04:00
Tom Lane
ef0938f7bd Invent "MatchAnyN" option for tab-complete.c's Matches/MatchesCS.
This argument matches any number (including zero) of previous words.
Use it to replace the common coding pattern

	if (HeadMatches("A", "B") && TailMatches("X", "Y"))

with

	if (Matches("A", "B", MatchAnyN, "X", "Y"))

In itself this feature doesn't do much except (arguably) make the
code slightly shorter and more readable.  However, it reduces the
number of complex if-condition patterns that have to be dealt with
in the next commits in this series.

While here, restructure the *Matches implementation functions so
that the actual work is done in functions that take a char **
array of pattern strings, and the versions taking variadic arguments
are thin wrappers around the array ones.  This simplifies the
new Matches logic considerably.  At the end of this patch series,
the array functions will be the only ones that are material to
performance, so having the variadic ones be wrappers makes sense.

Discussion: https://postgr.es/m/2208466.1720729502@sss.pgh.pa.us
2024-10-07 12:13:02 -04:00
Thomas Munro
adbb27ac89 Reject non-ASCII locale names.
Commit bf03cfd1 started scanning all available BCP 47 locale names on
Windows.  This caused an abort/crash in the Windows runtime library if
the default locale name contained non-ASCII characters, because of our
use of the setlocale() save/restore pattern with "char" strings.  After
switching to another locale with a different encoding, the saved name
could no longer be understood, and setlocale() would abort.

"Turkish_Türkiye.1254" is the example from recent reports, but there are
other examples of countries and languages with non-ASCII characters in
their names, and they appear in Windows' (old style) locale names.

To defend against this:

1.  In initdb, reject non-ASCII locale names given explicity on the
command line, or returned by the operating system environment with
setlocale(..., ""), or "canonicalized" by the operating system when we
set it.

2.  In initdb only, perform the save-and-restore with Windows'
non-standard wchar_t variant of setlocale(), so that it is not subject
to round trip failures stemming from char string encoding confusion.

3.  In the backend, we don't have to worry about the save-and-restore
problem because we have already vetted the defaults, so we just have to
make sure that CREATE DATABASE also rejects non-ASCII names in any new
databases.  SET lc_XXX doesn't suffer from the problem, but the ban
applies to it too because it uses check_locale().  CREATE COLLATION
doesn't suffer from the problem either, but it doesn't use
check_locale() so it is not included in the new ban for now, to minimize
the change.

Anyone who encounters the new error message should either create a new
duplicated locale with an ASCII-only name using Windows Locale Builder,
or consider using BCP 47 names like "tr-TR".  Users already couldn't
initialize a cluster with "Turkish_Türkiye.1254" on PostgreSQL 16+, but
the new failure mode is an error message that explains why, instead of a
crash.

Back-patch to 16, where bf03cfd1 landed.  Older versions are affected
in theory too, but only 16 and later are causing crash reports.

Reviewed-by: Andrew Dunstan <andrew@dunslane.net> (the idea, not the patch)
Reported-by: Haifang Wang (Centific Technologies Inc) <v-haiwang@microsoft.com>
Discussion: https://postgr.es/m/PH8PR21MB3902F334A3174C54058F792CE5182%40PH8PR21MB3902.namprd21.prod.outlook.com
2024-10-05 13:50:02 +13:00
Fujii Masao
e7834a1a25 Add log_verbosity = 'silent' support to COPY command.
Previously, when the on_error option was set to ignore, the COPY command
would always log NOTICE messages for input rows discarded due to
data type incompatibility. Users had no way to suppress these messages.

This commit introduces a new log_verbosity setting, 'silent',
which prevents the COPY command from emitting NOTICE messages
when on_error = 'ignore' is used, even if rows are discarded.
This feature is particularly useful when processing malformed files
frequently, where a flood of NOTICE messages can be undesirable.

For example, when frequently loading malformed files via the COPY command
or querying foreign tables using file_fdw (with an upcoming patch to
add on_error support for file_fdw), users may prefer to suppress
these messages to reduce log noise and improve clarity.

Author: Atsushi Torikoshi
Reviewed-by: Masahiko Sawada, Fujii Masao
Discussion: https://postgr.es/m/ab59dad10490ea3734cf022b16c24cfd@oss.nttdata.com
2024-10-03 15:55:37 +09:00
Michael Paquier
e2bab2d792 Remove support for unlogged on partitioned tables
The following commands were allowed on partitioned tables, with
different effects:
1) ALTER TABLE SET [UN]LOGGED did not issue an error, and did not update
pg_class.relpersistence.
2) CREATE UNLOGGED TABLE was working with pg_class.relpersistence marked
as initially defined, but partitions did not inherit the UNLOGGED
property, which was confusing.

This commit causes the commands mentioned above to fail for partitioned
tables, instead.

pg_dump is tweaked so as partitioned tables marked as UNLOGGED ignore
the option when dumped from older server versions.  pgbench needs a
tweak for --unlogged and --partitions=N to ignore the UNLOGGED option on
the partitioned tables created, its partitions still being unlogged.

Author: Michael Paquier
Reviewed-by: Nathan Bossart
Discussion: https://postgr.es/m/ZiiyGFTBNkqcMQi_@paquier.xyz
2024-10-03 10:55:02 +09:00
Robert Haas
d94cf5ca7f File size in a backup manifest should use uint64, not size_t.
size_t is the size of an object in memory, not the size of a file on disk.

Thanks to Tom Lane for noting the error.

Discussion: http://postgr.es/m/1865585.1727803933@sss.pgh.pa.us
2024-10-02 09:59:04 -04:00
Daniel Gustafsson
9c73395104 Add fastpaths for when no objects are found
If there are no objects found, there is no reason to inspect the
result columns and mallocing a zero-sized  (which will be 1 byte
in reality) heap buffer for it.  Add a fast-path for immediately
returning like how other object inspection functions are already
doing it.

Reviewed-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/C2F05B3C-1414-45DD-AE09-6FEE4D0F89BD@yesql.se
2024-10-02 13:08:55 +02:00
Daniel Gustafsson
1a123e3b13 Remove superfluous PQExpBuffer resetting
Since the buffer was just created, there is no reason to immediately
reset it.

Reviewed-by: Ranier Vilela <ranier.vf@gmail.com>
Discussion: https://postgr.es/m/C2F05B3C-1414-45DD-AE09-6FEE4D0F89BD@yesql.se
2024-10-02 13:07:31 +02:00
Peter Eisentraut
983a588e0b initdb: Add new option "--no-data-checksums"
Right now this does nothing except override any earlier
--data-checksums option.  But the idea is that --data-checksums could
become the default, and then this option would allow forcing it off
instead.

Author: Greg Sabino Mullane <greg@turnstep.com>
Discussion: https://www.postgresql.org/message-id/flat/CAKAnmmKwiMHik5AHmBEdf5vqzbOBbcwEPHo4-PioWeAbzwcTOQ@mail.gmail.com
2024-10-01 10:50:30 -04:00
Peter Eisentraut
10b721821d Use macro to define the number of enum values
Refactoring in the interest of code consistency, a follow-up to 2e068db56e.

The argument against inserting a special enum value at the end of the enum
definition is that a switch statement might generate a compiler warning unless
it has a default clause.

Aleksander Alekseev, reviewed by Michael Paquier, Dean Rasheed, Peter Eisentraut

Discussion: https://postgr.es/m/CAJ7c6TMsiaV5urU_Pq6zJ2tXPDwk69-NKVh4AMN5XrRiM7N%2BGA%40mail.gmail.com
2024-10-01 09:30:24 -04:00
Robert Haas
fc1b2ce0ee Fix some pg_verifybackup issues reported by Coverity.
Commit 8dfd312902 introduced a few
problems. verify_tar_file() forgot to free a buffer; the leak can't
add up to anything material, but might as well fix it.
precheck_tar_backup_file() intended to return after reporting an
error but didn't actually do so. member_copy_control_data() could
try to copy zero bytes (and maybe Coverity thinks it can even be
trying to copy a negative number of bytes).

Per discussion with Tom Lane.

Discussion: http://postgr.es/m/1240823.1727629418@sss.pgh.pa.us
2024-10-01 08:36:54 -04:00
Tom Lane
7702337489 Do not treat \. as an EOF marker in CSV mode for COPY IN.
Since backslash is (typically) not special in CSV data, we should
not be treating \. as special either.  The server historically did
this to keep CSV and TEXT modes more alike and to support V2 protocol;
but V2 protocol is long dead, and the inconsistency with CSV standards
is annoying.  Remove that behavior in CopyReadLineText, and make some
minor consequent code simplifications.

On the client side, we need to fix psql so that it does not check
for \. except when reading data from STDIN (that is, the script
source).  We must do that regardless of TEXT/CSV mode or there is
no way to end the COPY short of script EOF.  Also, be careful
not to send the \. to the server in that case.

This is a small compatibility break in that other applications
beside psql may need similar adjustment.  Also, using an older
version of psql with a v18 server may result in misbehavior
during CSV-mode COPY IN.

Daniel Vérité, reviewed by vignesh C, Robert Haas, and myself

Discussion: https://postgr.es/m/ed659f37-a9dd-42a7-82b9-0da562cc4006@manitou-mail.org
2024-09-30 17:57:12 -04:00
Tom Lane
04c64e3fb3 Remove incorrect entries in pg_walsummary's getopt_long call.
For some reason this listed "-f" and "-w" as valid switches, though
the code doesn't implement any such thing nor do the docs mention
them.  The effect of this was that if you tried to use one of these
switches, you'd get an unhelpful error message.

Yusuke Sugie

Discussion: https://postgr.es/m/68e72a2a70f4d84c1c7847b13bcdaef8@oss.nttdata.com
2024-09-30 12:06:54 -04:00
Fujii Masao
20cfec896c reindexdb: Skip reindexing temporary tables and indexes.
Reindexing temp tables or indexes of other sessions is not allowed.
However, reindexdb in parallel mode previously listed them as
the objects to process, leading to failures.

This commit ensures reindexdb in parallel mode skips temporary tables
and indexes by adding a condition based on the relpersistence column
in pg_class to the object listing queries, preventing these issues.

Note that this commit does not affect reindexdb when temporary tables
or indexes are explicitly specified using the -t or -j options;
reindexdb in that case still does not skip them and can cause an error.

Back-patch to v13 where parallel mode was introduced in reindexdb.

Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/5f37ee56-14fb-44fe-9150-9eb97e10538b@oss.nttdata.com
2024-09-30 11:13:55 +09:00
Robert Haas
7f7474a8e4 Reindent pg_verifybackup.c. 2024-09-27 11:14:31 -04:00
Robert Haas
8dfd312902 pg_verifybackup: Verify tar-format backups.
This also works for compressed tar-format backups. However, -n must be
used, because we use pg_waldump to verify WAL, and it doesn't yet know
how to verify WAL that is stored inside of a tarfile.

Amul Sul, reviewed by Sravan Kumar and by me, and revised by me.
2024-09-27 08:40:24 -04:00
Fujii Masao
8410f738ad Fix typo in pg_walsummary/nls.mk.
Author: Koki Nakamura
Discussion: https://postgr.es/m/485c613d1db8de2e8169d5afd43e7f9e@oss.nttdata.com
2024-09-27 10:20:22 +09:00
Nathan Bossart
9726653185 Improve style of pg_upgrade task callback functions.
I wanted to avoid adjusting this code too much when converting
these tasks to use the new parallelization framework (see commit
40e2e5e92b), which is why this is being done as a follow-up commit.
These stylistic adjustments result in fewer lines of code and fewer
levels of indentation in some places.

While at it, add names to the UpgradeTaskSlotState enum and the
UpgradeTaskSlot struct.  I'm not aware of any established project
policy in this area, but let's at least be consistent within the
same file.

Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/ZunW7XHLd2uTts4f%40nathan
2024-09-26 13:54:37 -05:00
Nathan Bossart
2ceeb638b7 Remove extra whitespace in pg_upgrade status message.
There's no need to add another level of indentation to this status
message.  pg_log() will put it in the right place.

Oversight in commit 347758b120.

Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/ZunW7XHLd2uTts4f%40nathan
Backpatch-through: 17
2024-09-25 11:18:56 -05:00
Michael Paquier
1ab67c9dfa vacuumdb: Skip temporary tables in query to build list of relations
Running vacuumdb with a non-superuser while another user has created a
temporary table would lead to a mid-flight permission failure,
interrupting the operation.  vacuum_rel() skips temporary relations of
other backends, and it makes no sense for vacuumdb to know about these
relations, so let's switch it to ignore temporary relations entirely.

Adding a qual in the query based on relpersistence simplifies the
generation of its WHERE clause in vacuum_one_database(), per se the
removal of "has_where".

Author: VaibhaveS, Michael Paquier
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/CAM_eQjwfAR=y3G1fGyS1U9FTmc+FyJm9amNfY2QCZBnDDbNPZg@mail.gmail.com
Backpatch-through: 12
2024-09-25 14:43:16 +09:00
Tom Lane
1591b38d17 Fix psql describe commands' handling of ACL columns for old servers.
Commit d1379ebf4 carelessly broke printACLColumn for pre-9.4 servers,
by using the cardinality() function which we introduced in 9.4.
We expect psql's describe-related commands to work back to 9.2, so
this is bad.  Use the longstanding array_length() function instead.

Per report from Christoph Berg.  Back-patch to v17.

Discussion: https://postgr.es/m/ZvLXYglRS6hMMhtr@msg.df7cb.de
2024-09-24 17:21:38 -04:00
Nathan Bossart
afb03e2ebf Alphabetize #include directives in pg_checksums.c.
Author: Michael Banck
Discussion: https://postgr.es/m/66edaed0.050a0220.32a9ba.42c8%40mx.google.com
2024-09-20 15:18:42 -05:00
Alexander Korotkov
a094e8b9e4 Fix order of includes in src/bin/pg_upgrade/info.c 2024-09-19 14:34:00 +03:00