Base de données relationnelle
Find a file
Michael Paquier babbbb595d Add support for LZ4 compression in pg_receivewal
pg_receivewal gains a new option, --compression-method=lz4, available
when the code is compiled with --with-lz4.  Similarly to gzip, this
gives the possibility to compress archived WAL segments with LZ4.  This
option is not compatible with --compress.

The implementation uses LZ4 frames, and is compatible with simple lz4
commands.  Like gzip, using --synchronous ensures that any data will be
flushed to disk within the current .partial segment, so as it is
possible to retrieve as much WAL data as possible even from a
non-completed segment (this requires completing the partial file with
zeros up to the WAL segment size supported by the backend after
decompression, but this is the same as gzip).

The calculation of the streaming start LSN is able to transparently find
and check LZ4-compressed segments.  Contrary to gzip where the
uncompressed size is directly stored in the object read, the LZ4 chunk
protocol does not store the uncompressed data by default.  There is
contentSize that can be used with LZ4 frames by that would not help if
using an archive that includes segments compressed with the defaults of
a "lz4" command, where this is not stored.  So, this commit has taken
the most extensible approach by decompressing the already-archived
segment to check its uncompressed size, through a blank output buffer in
chunks of 64kB (no actual performance difference noticed with 8kB, 16kB
or 32kB, and the operation in itself is actually fast).

Tests have been added to verify the creation and correctness of the
generated LZ4 files.  The latter is achieved by the use of command
"lz4", if found in the environment.

The tar-based WAL method in walmethods.c, used now only by
pg_basebackup, does not know yet about LZ4.  Its code could be extended
for this purpose.

Author: Georgios Kokolatos
Reviewed-by: Michael Paquier, Jian Guo, Magnus Hagander, Dilip Kumar
Discussion: https://postgr.es/m/ZCm1J5vfyQ2E6dYvXz8si39HQ2gwxSZ3IpYaVgYa3lUwY88SLapx9EEnOf5uEwrddhx2twG7zYKjVeuP5MwZXCNPybtsGouDsAD1o2L_I5E=@pm.me
2021-11-05 11:33:25 +09:00
config Adjust configure to insist on Perl version >= 5.8.3. 2021-10-07 14:26:17 -04:00
contrib Avoid some other O(N^2) hazards in list manipulation. 2021-11-01 16:24:39 -04:00
doc Add support for LZ4 compression in pg_receivewal 2021-11-05 11:33:25 +09:00
src Add support for LZ4 compression in pg_receivewal 2021-11-05 11:33:25 +09:00
.dir-locals.el Make Emacs perl-mode indent more like perltidy. 2019-01-13 11:32:31 -08:00
.editorconfig Add .editorconfig 2019-12-18 09:13:13 +01:00
.git-blame-ignore-revs Add another old commit to git-blame-ignore-revs. 2021-11-03 17:34:19 -07:00
.gitattributes gitattributes: Add new entry to silence whitespace error 2021-06-05 07:57:31 +02:00
.gitignore Support for optimizing and emitting code in LLVM JIT provider. 2018-03-22 11:05:22 -07:00
aclocal.m4 Remove configure-time probe for DocBook DTD. 2020-11-30 15:24:13 -05:00
configure Make configure check for minimum required version of IPC::Run. 2021-10-11 16:49:49 -04:00
configure.ac Make configure check for minimum required version of IPC::Run. 2021-10-11 16:49:49 -04:00
COPYRIGHT Update copyright for 2021 2021-01-02 13:06:25 -05:00
GNUmakefile.in add missing tag from commit b8c4261e5e 2021-07-01 15:47:46 -04:00
HISTORY Canonicalize some URLs 2020-02-10 20:47:50 +01:00
Makefile Don't unset MAKEFLAGS in non-GNU Makefile. 2019-06-25 09:36:21 +12:00
README Canonicalize some URLs 2020-02-10 20:47:50 +01:00
README.git Canonicalize some URLs 2020-02-10 20:47:50 +01:00

PostgreSQL Database Management System
=====================================

This directory contains the source code distribution of the PostgreSQL
database management system.

PostgreSQL is an advanced object-relational database management system
that supports an extended subset of the SQL standard, including
transactions, foreign keys, subqueries, triggers, user-defined types
and functions.  This distribution also contains C language bindings.

PostgreSQL has many language interfaces, many of which are listed here:

	https://www.postgresql.org/download/

See the file INSTALL for instructions on how to build and install
PostgreSQL.  That file also lists supported operating systems and
hardware platforms and contains information regarding any other
software packages that are required to build or run the PostgreSQL
system.  Copyright and license information can be found in the
file COPYRIGHT.  A comprehensive documentation set is included in this
distribution; it can be read as described in the installation
instructions.

The latest version of this software may be obtained at
https://www.postgresql.org/download/.  For more information look at our
web site located at https://www.postgresql.org/.