postgresql/src
Tom Lane 0e6185283c Adopt the GNU convention for handling tar-archive members exceeding 8GB.
The POSIX standard for tar headers requires archive member sizes to be
printed in octal with at most 11 digits, limiting the representable file
size to 8GB.  However, GNU tar and apparently most other modern tars
support a convention in which oversized values can be stored in base-256,
allowing any practical file to be a tar member.  Adopt this convention
to remove two limitations:
* pg_dump with -Ft output format failed if the contents of any one table
exceeded 8GB.
* pg_basebackup failed if the data directory contained any file exceeding
8GB.  (This would be a fatal problem for installations configured with a
table segment size of 8GB or more, and it has also been seen to fail when
large core dump files exist in the data directory.)

File sizes under 8GB are still printed in octal, so that no compatibility
issues are created except in cases that would have failed entirely before.

In addition, this patch fixes several bugs in the same area:

* In 9.3 and later, we'd defined tarCreateHeader's file-size argument as
size_t, which meant that on 32-bit machines it would write a corrupt tar
header for file sizes between 4GB and 8GB, even though no error was raised.
This broke both "pg_dump -Ft" and pg_basebackup for such cases.

* pg_restore from a tar archive would fail on tables of size between 4GB
and 8GB, on machines where either "size_t" or "unsigned long" is 32 bits.
This happened even with an archive file not affected by the previous bug.

* pg_basebackup would fail if there were files of size between 4GB and 8GB,
even on 64-bit machines.

* In 9.3 and later, "pg_basebackup -Ft" failed entirely, for any file size,
on 64-bit big-endian machines.

In view of these potential data-loss bugs, back-patch to all supported
branches, even though removal of the documented 8GB limit might otherwise
be considered a new feature rather than a bug fix.
2015-11-21 20:21:32 -05:00
..
backend Adopt the GNU convention for handling tar-archive members exceeding 8GB. 2015-11-21 20:21:32 -05:00
bin Adopt the GNU convention for handling tar-archive members exceeding 8GB. 2015-11-21 20:21:32 -05:00
common Unlink static libraries before rebuilding them. 2015-03-01 13:06:39 -05:00
include Adopt the GNU convention for handling tar-archive members exceeding 8GB. 2015-11-21 20:21:32 -05:00
interfaces Fix unwanted flushing of libpq's input buffer when socket EOF is seen. 2015-11-12 13:03:53 -05:00
makefiles Don't use deprecated dllwrap on Cygwin. 2014-02-01 16:13:32 -05:00
pl PL/Python: Make tests pass with Python 3.5 2015-11-14 13:44:09 -05:00
port Adopt the GNU convention for handling tar-archive members exceeding 8GB. 2015-11-21 20:21:32 -05:00
template AIX: Test the -qlonglong option before use. 2015-07-17 03:02:03 -04:00
test Fix handling of inherited check constraints in ALTER COLUMN TYPE (again). 2015-11-20 14:55:29 -05:00
timezone Update time zone data files to tzdata release 2015g. 2015-10-02 19:16:06 -04:00
tools Turn install.bat into a pure one line wrapper fort he perl script. 2015-07-07 16:37:16 +03:00
tutorial Remove tabs after spaces in C comments 2014-05-06 11:26:28 -04:00
.gitignore Convert cvsignore to gitignore, and add .gitignore for build targets. 2010-09-22 12:57:04 +02:00
bcc32.mak Autoconfiscate selection of 64-bit int type for 64-bit large object API. 2012-10-07 21:52:43 -04:00
DEVELOPERS Replace a couple of references to files that no longer exist in the source 2009-05-04 08:08:47 +00:00
Makefile Create libpgcommon, and move pg_malloc et al to it 2013-02-12 11:21:05 -03:00
Makefile.global.in Make numeric form of PG version number readily available in Makefiles. 2015-07-05 12:01:01 -04:00
Makefile.shlib Unlink static libraries before rebuilding them. 2015-03-01 13:06:39 -05:00
nls-global.mk NLS: Use msgmerge --previous option 2012-12-13 23:12:12 -05:00
win32.mak Autoconfiscate selection of 64-bit int type for 64-bit large object API. 2012-10-07 21:52:43 -04:00