Commit graph

28 commits

Author SHA1 Message Date
Andrew Dunstan
3b5e03dca2 Provide a FORCE NULL option to COPY in CSV mode.
This forces an input field containing the quoted null string to be
returned as a NULL. Without this option, only unquoted null strings
behave this way. This helps where some CSV producers insist on quoting
every field, whether or not it is needed. The option takes a list of
fields, and only applies to those columns. There is an equivalent
column-level option added to file_fdw.

Ian Barwick, with some tweaking by Andrew Dunstan, reviewed by Payal
Singh.
2014-03-04 17:31:59 -05:00
Bruce Momjian
4deb57de7d Issue ERROR if FREEZE mode can't be honored by COPY
Previously non-honored FREEZE mode was ignored.  This also issues an
appropriate error message based on the cause of the failure, per
suggestion from Tom.  Additional regression test case added.
2013-01-26 13:33:24 -05:00
Simon Riggs
1f023f9297 Optimize COPY FREEZE with CREATE TABLE also.
Jeff Davis, additional test by me
2012-12-07 13:26:52 +00:00
Simon Riggs
5457a130d3 Reduce scope of changes for COPY FREEZE.
Allow support only for freezing tuples by explicit
command. Previous coding mistakenly extended
slightly beyond what was agreed as correct on -hackers.
So essentially a partial revoke of earlier work,
leaving just the COPY FREEZE command.
2012-12-02 20:52:52 +00:00
Simon Riggs
ddf509eb4a Tweak tests in COPY FREEZE 2012-12-01 13:46:41 +00:00
Simon Riggs
8de72b66a2 COPY FREEZE and mark committed on fresh tables.
When a relfilenode is created in this subtransaction or
a committed child transaction and it cannot otherwise
be seen by our own process, mark tuples committed ahead
of transaction commit for all COPY commands in same
transaction. If FREEZE specified on COPY
and pre-conditions met then rows will also be frozen.
Both options designed to avoid revisiting rows after commit,
increasing performance of subsequent commands after
data load and upgrade. pg_restore changes later.

Simon Riggs, review comments from Heikki Linnakangas, Noah Misch and design
input from Tom Lane, Robert Haas and Kevin Grittner
2012-12-01 12:54:20 +00:00
Tom Lane
e8476f46fc Fix COPY FROM for null marker strings that correspond to invalid encoding.
The COPY documentation says "COPY FROM matches the input against the null
string before removing backslashes".  It is therefore reasonable to presume
that null markers like E'\\0' will work ... and they did, until someone put
the tests in the wrong order during microoptimization-driven rewrites.
Since then, we've been failing if the null marker is something that would
de-escape to an invalidly-encoded string.  Since null markers generally
need to be something that can't appear in the data, this represents a
nontrivial loss of functionality; surprising nobody noticed it earlier.

Per report from Jeff Davis.  Backpatch to 8.4 where this got broken.
2012-03-25 23:17:22 -04:00
Itagaki Takahiro
3cba8240a1 Add ENCODING option to COPY TO/FROM and file_fdw.
File encodings can be specified separately from client encoding.
If not specified, client encoding is used for backward compatibility.

Cases when the encoding doesn't match client encoding are slower
than matched cases because we don't have conversion procs for other
encodings. Performance improvement would be be a future work.

Original patch by Hitoshi Harada, and modified by me.
2011-02-21 14:32:40 +09:00
Tom Lane
50a90fac40 Stamp HEAD as 9.0devel, and update various places that were referring to 8.5
(hope I got 'em all).  Per discussion, this release will be 9.0 not 8.5.
2010-02-17 04:19:41 +00:00
Tom Lane
923413ac6d Define a new, more extensible syntax for COPY options.
This is intentionally similar to the recently revised syntax for EXPLAIN
options, ie, (name value, ...).  The old syntax is still supported for
backwards compatibility, but we intend that any options added in future
will be provided only in the new syntax.

Robert Haas, Emmanuel Cecchet
2009-09-21 20:10:21 +00:00
Andrew Dunstan
de7531a971 Allow * as parameter for FORCE QUOTE for COPY CSV. Itagaki Takahiro. 2009-07-25 00:07:14 +00:00
Tom Lane
f51c9a6665 Change test tables in copy2 regression test to be temporary tables.
This prevents autovacuum from reclaiming free space in them and causing
the test's output row order to change, which is causing intermittent
bogus failure reports in the buildfarm.

Backpatch to 8.3.  The issue exists further back, but since autovacuum was
not on by default before 8.3, it's not a problem for buildfarm testing.
2009-06-14 00:00:24 +00:00
Peter Eisentraut
7f4f42fa10 Clean up CREATE FUNCTION syntax usage in contrib and elsewhere, in
particular get rid of single quotes around language names and old WITH ()
construct.
2006-02-27 16:09:50 +00:00
Bruce Momjian
54e5caa440 Add COPY CSV test that tests CSV output of \. 2005-12-28 15:07:02 +00:00
Bruce Momjian
87289ff35c Add regression tests for CSV and \., and add automatic quoting of a
single column dump that has a \. value, so the load works properly.  I
also added documentation describing this issue.
2005-12-28 03:25:32 +00:00
Bruce Momjian
bb3cce4ec9 Add E'' syntax so eventually normal strings can treat backslashes
literally.

Add GUC variables:

        "escape_string_warning" - warn about backslashes in non-E strings
        "escape_string_syntax" - supports E'' syntax?
        "standard_compliant_strings" - treats backslashes literally in ''

Update code to use E'' when escapes are used.
2005-06-26 03:04:37 +00:00
Neil Conway
167bc6c621 Add regression test for consecutive newlines in COPY CSV mode. (There is
no bug related to this functionality in HEAD, but it's worth adding a test
for anyway.) From Andrew Dunstan.
2005-05-13 06:33:40 +00:00
Neil Conway
5df3fc67a7 This patch updates the regression tests to allow "make installcheck" to
pass if "default_with_oids" is set to false. I took the approach of
explicitly adding WITH OIDS to the CREATE TABLEs where necessary, rather
than tweaking the default_with_oids GUC var.
2005-01-22 05:12:33 +00:00
Tom Lane
45616f5bbb Clean up generation of default names for constraints, indexes, and serial
sequences, as per recent discussion.  All these names are now of the
form table_column_type, with digits added if needed to make them unique.
Default constraint names are chosen to be unique across their whole schema,
not just within the parent object, so as to be more SQL-spec-compatible
and make the information schema views more useful.
2004-06-10 17:56:03 +00:00
Bruce Momjian
659660c116 Add regression tests for CSV COPY format. 2004-06-09 23:39:24 +00:00
Bruce Momjian
962987a1c2 Add NULL regression tests.
Manfred Koizar
2003-10-10 03:46:51 +00:00
Tom Lane
f85f43dfb5 Backend support for autocommit removed, per recent discussions. The
only remnant of this failed experiment is that the server will take
SET AUTOCOMMIT TO ON.  Still TODO: provide some client-side autocommit
logic in libpq.
2003-05-14 03:26:03 +00:00
Bruce Momjian
5c6a5fe18b Make regression tests safe for autocommit = 'off'. 2002-10-19 01:35:43 +00:00
Tom Lane
b663f3443b Add a bunch of pseudo-types to replace the behavior formerly associated
with OPAQUE, as per recent pghackers discussion.  I still want to do some
more work on the 'cstring' pseudo-type, but I'm going to commit the bulk
of the changes now before the tree starts shifting under me ...
2002-08-22 00:01:51 +00:00
Tom Lane
38bb77a5d1 ALTER TABLE DROP COLUMN works. Patch by Christopher Kings-Lynne,
code review by Tom Lane.  Remaining issues: functions that take or
return tuple types are likely to break if one drops (or adds!)
a column in the table defining the type.  Need to think about what
to do here.

Along the way: some code review for recent COPY changes; mark system
columns attnotnull = true where appropriate, per discussion a month ago.
2002-08-02 18:15:10 +00:00
Bruce Momjian
874148fe34 IMPROVED VERSION APPLIED:
The attached patch completes the following TODO item:

    * Generate failure on short COPY lines rather than pad NULLs

I also restructed a lot of the existing COPY code, did some code
review on the column list patch sent in by Brent Verner a little
while ago, and added some regression tests. I also added an
explicit check (and resultant error) for extra data before
the end-of-line.

Neil Conway
2002-07-30 16:55:06 +00:00
Bruce Momjian
776fc8cc5c New depend code doesn't need sequence drop. 2002-07-18 22:31:44 +00:00
Bruce Momjian
2c48b3db95 here are the copy2.sql and copy2.out files for the new regression
tests

Brent Verner
2002-07-18 04:45:51 +00:00