postgresql/src/include
Andres Freund 86b85044e8 tableam: Add table_multi_insert() and revamp/speed-up COPY FROM buffering.
This adds table_multi_insert(), and converts COPY FROM, the only user
of heap_multi_insert, to it.

A simple conversion of COPY FROM use slots would have yielded a
slowdown when inserting into a partitioned table for some
workloads. Different partitions might need different slots (both slot
types and their descriptors), and dropping / creating slots when
there's constant partition changes is measurable.

Thus instead revamp the COPY FROM buffering for partitioned tables to
allow to buffer inserts into multiple tables, flushing only when
limits are reached across all partition buffers. By only dropping
slots when there've been inserts into too many different partitions,
the aforementioned overhead is gone. By allowing larger batches, even
when there are frequent partition changes, we actuall speed such cases
up significantly.

By using slots COPY of very narrow rows into unlogged / temporary
might slow down very slightly (due to the indirect function calls).

Author: David Rowley, Andres Freund, Haribabu Kommi
Discussion:
    https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
    https://postgr.es/m/20190327054923.t3epfuewxfqdt22e@alap3.anarazel.de
2019-04-04 16:28:18 -07:00
..
access tableam: Add table_multi_insert() and revamp/speed-up COPY FROM buffering. 2019-04-04 16:28:18 -07:00
bootstrap Update copyright for 2019 2019-01-02 12:44:25 -05:00
catalog Reduce overhead of pg_mcv_list (de)serialization 2019-04-03 21:23:40 +02:00
commands Allow VACUUM to be run with index cleanup disabled. 2019-04-04 15:04:43 -04:00
common Add support for partial TOAST decompression 2019-04-02 12:35:32 -04:00
datatype Update copyright for 2019 2019-01-02 12:44:25 -05:00
executor Generated columns 2019-03-30 08:15:57 +01:00
fe_utils Unified logging system for command-line programs 2019-04-01 20:01:35 +02:00
foreign Store tuples for EvalPlanQual in slots, rather than as HeapTuples. 2019-03-01 10:37:57 -08:00
jit Renaming for new subscripting mechanism 2019-02-01 12:50:32 -03:00
lib Add IntegerSet, to hold large sets of 64-bit ints efficiently. 2019-03-22 13:21:45 +02:00
libpq Move the be_gssapi_get_* prototypes 2019-04-04 11:11:46 -04:00
mb Update copyright for 2019 2019-01-02 12:44:25 -05:00
nodes tableam: Add table_multi_insert() and revamp/speed-up COPY FROM buffering. 2019-04-04 16:28:18 -07:00
optimizer Refactor create_limit_path() to share cost adjustment code with FDWs. 2019-04-02 19:55:12 +09:00
parser Generated columns 2019-03-30 08:15:57 +01:00
partitioning Collations with nondeterministic comparison 2019-03-22 12:12:43 +01:00
port Fix previous MinGW fix. 2019-02-16 15:23:02 +00:00
portability Update copyright for 2019 2019-01-02 12:44:25 -05:00
postmaster Refactor the fsync queue for wider use. 2019-04-04 23:38:38 +13:00
regex Partial implementation of SQL/JSON path language 2019-03-16 12:16:48 +03:00
replication Add walreceiver API to get remote server version 2019-03-15 10:16:26 +01:00
rewrite Update copyright for 2019 2019-01-02 12:44:25 -05:00
snowball Update copyright for 2019 2019-01-02 12:44:25 -05:00
statistics Fix deserialization of pg_mcv_list values 2019-03-28 20:03:14 +01:00
storage Refactor the fsync queue for wider use. 2019-04-04 23:38:38 +13:00
tcop Update copyright for 2019 2019-01-02 12:44:25 -05:00
tsearch Update copyright for 2019 2019-01-02 12:44:25 -05:00
utils Allow VACUUM to be run with index cleanup disabled. 2019-04-04 15:04:43 -04:00
.gitignore Refactor dlopen() support 2018-09-06 11:33:04 +02:00
c.h Add macro to cast away volatile without allowing changes to underlying type 2019-03-25 09:37:03 +01:00
fmgr.h Change function call information to be variable length. 2019-01-26 14:17:52 -08:00
funcapi.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
getaddrinfo.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
getopt_long.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
Makefile Get rid of jsonpath_gram.h and jsonpath_scanner.h 2019-03-20 11:13:34 +03:00
miscadmin.h Convert [autovacuum_]vacuum_cost_delay into floating-point GUCs. 2019-03-10 15:01:39 -04:00
pg_config.h.in Make use of compiler builtins and/or assembly for CLZ, CTZ, POPCNT. 2019-02-15 23:22:33 -05:00
pg_config.h.win32 Make use of compiler builtins and/or assembly for CLZ, CTZ, POPCNT. 2019-02-15 23:22:33 -05:00
pg_config_ext.h.in Autoconfiscate selection of 64-bit int type for 64-bit large object API. 2012-10-07 21:52:43 -04:00
pg_config_ext.h.win32 Autoconfiscate selection of 64-bit int type for 64-bit large object API. 2012-10-07 21:52:43 -04:00
pg_config_manual.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
pg_getopt.h Use our own getopt() on OpenBSD. 2019-01-18 15:06:26 -05:00
pg_trace.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
pgstat.h GSSAPI encryption support 2019-04-03 15:02:33 -04:00
pgtar.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
pgtime.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
port.h Cygwin and Mingw floating-point fixes. 2019-02-16 01:50:16 +00:00
postgres.h Change function call information to be variable length. 2019-01-26 14:17:52 -08:00
postgres_ext.h Phase 2 of pgindent updates. 2017-06-21 15:19:25 -04:00
postgres_fe.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
rusagestub.h Update copyright for 2019 2019-01-02 12:44:25 -05:00
windowapi.h Update copyright for 2019 2019-01-02 12:44:25 -05:00