postgresql/src/backend/nodes
Tom Lane 54eac6e8c5 Force rescanning of parallel-aware scan nodes below a Gather[Merge].
The ExecReScan machinery contains various optimizations for postponing
or skipping rescans of plan subtrees; for example a HashAgg node may
conclude that it can re-use the table it built before, instead of
re-reading its input subtree.  But that is wrong if the input contains
a parallel-aware table scan node, since the portion of the table scanned
by the leader process is likely to vary from one rescan to the next.
This explains the timing-dependent buildfarm failures we saw after
commit a2b70c89c.

The established mechanism for showing that a plan node's output is
potentially variable is to mark it as depending on some runtime Param.
Hence, to fix this, invent a dummy Param (one that has a PARAM_EXEC
parameter number, but carries no actual value) associated with each Gather
or GatherMerge node, mark parallel-aware nodes below that node as dependent
on that Param, and arrange for ExecReScanGather[Merge] to flag that Param
as changed whenever the Gather[Merge] node is rescanned.

This solution breaks an undocumented assumption made by the parallel
executor logic, namely that all rescans of nodes below a Gather[Merge]
will happen synchronously during the ReScan of the top node itself.
But that's fundamentally contrary to the design of the ExecReScan code,
and so was doomed to fail someday anyway (even if you want to argue
that the bug being fixed here wasn't a failure of that assumption).
A follow-on patch will address that issue.  In the meantime, the worst
that's expected to happen is that given very bad timing luck, the leader
might have to do all the work during a rescan, because workers think
they have nothing to do, if they are able to start up before the eventual
ReScan of the leader's parallel-aware table scan node has reset the
shared scan state.

Although this problem exists in 9.6, there does not seem to be any way
for it to manifest there.  Without GatherMerge, it seems that a plan tree
that has a rescan-short-circuiting node below Gather will always also
have one above it that will short-circuit in the same cases, preventing
the Gather from being rescanned.  Hence we won't take the risk of
back-patching this change into 9.6.  But v10 needs it.

Discussion: https://postgr.es/m/CAA4eK1JkByysFJNh9M349u_nNjqETuEnY_y1VUc_kJiU0bxtaQ@mail.gmail.com
2017-08-30 09:29:56 -04:00
..
bitmapset.c Support hashed aggregation with grouping sets. 2017-03-27 04:20:54 +01:00
copyfuncs.c Force rescanning of parallel-aware scan nodes below a Gather[Merge]. 2017-08-30 09:29:56 -04:00
equalfuncs.c Use MINVALUE/MAXVALUE instead of UNBOUNDED for range partition bounds. 2017-07-21 09:20:47 +01:00
extensible.c Update copyright via script for 2017 2017-01-03 13:48:53 -05:00
list.c Phase 2 of pgindent updates. 2017-06-21 15:19:25 -04:00
Makefile Introduce extensible node types. 2016-02-12 09:38:11 -05:00
makefuncs.c Phase 2 of pgindent updates. 2017-06-21 15:19:25 -04:00
nodeFuncs.c Code review for NextValueExpr expression node type. 2017-07-14 15:25:43 -04:00
nodes.c Update copyright via script for 2017 2017-01-03 13:48:53 -05:00
outfuncs.c Force rescanning of parallel-aware scan nodes below a Gather[Merge]. 2017-08-30 09:29:56 -04:00
params.c Phase 3 of pgindent updates. 2017-06-21 15:35:54 -04:00
print.c Add infrastructure to support EphemeralNamedRelation references. 2017-03-31 23:17:18 -05:00
read.c Update copyright via script for 2017 2017-01-03 13:48:53 -05:00
readfuncs.c Force rescanning of parallel-aware scan nodes below a Gather[Merge]. 2017-08-30 09:29:56 -04:00
README Fix various common mispellings. 2016-06-03 16:08:45 +01:00
tidbitmap.c Phase 3 of pgindent updates. 2017-06-21 15:35:54 -04:00
value.c Update copyright via script for 2017 2017-01-03 13:48:53 -05:00

src/backend/nodes/README

Node Structures
===============

Andrew Yu (11/94)

Introduction
------------

The current node structures are plain old C structures. "Inheritance" is
achieved by convention. No additional functions will be generated. Functions
that manipulate node structures reside in this directory.


FILES IN THIS DIRECTORY (src/backend/nodes/)

    General-purpose node manipulation functions:
	copyfuncs.c	- copy a node tree
	equalfuncs.c	- compare two node trees
	outfuncs.c	- convert a node tree to text representation
	readfuncs.c	- convert text representation back to a node tree
	makefuncs.c	- creator functions for some common node types
	nodeFuncs.c	- some other general-purpose manipulation functions

    Specialized manipulation functions:
	bitmapset.c	- Bitmapset support
	list.c		- generic list support
	params.c	- Param support
	tidbitmap.c	- TIDBitmap support
	value.c		- support for Value nodes

FILES IN src/include/nodes/

    Node definitions:
	nodes.h		- define node tags (NodeTag)
	primnodes.h	- primitive nodes
	parsenodes.h	- parse tree nodes
	plannodes.h	- plan tree nodes
	relation.h	- planner internal nodes
	execnodes.h	- executor nodes
	memnodes.h	- memory nodes
	pg_list.h	- generic list


Steps to Add a Node
-------------------

Suppose you want to define a node Foo:

1. Add a tag (T_Foo) to the enum NodeTag in nodes.h.  (If you insert the
   tag in a way that moves the numbers associated with existing tags,
   you'll need to recompile the whole tree after doing this.  It doesn't
   force initdb though, because the numbers never go to disk.)
2. Add the structure definition to the appropriate include/nodes/???.h file.
   If you intend to inherit from, say a Plan node, put Plan as the first field
   of your struct definition.
3. If you intend to use copyObject, equal, nodeToString or stringToNode,
   add an appropriate function to copyfuncs.c, equalfuncs.c, outfuncs.c
   and readfuncs.c accordingly.  (Except for frequently used nodes, don't
   bother writing a creator function in makefuncs.c)  The header comments
   in those files give general rules for whether you need to add support.
4. Add cases to the functions in nodeFuncs.c as needed.  There are many
   other places you'll probably also need to teach about your new node
   type.  Best bet is to grep for references to one or two similar existing
   node types to find all the places to touch.


Historical Note
---------------

Prior to the current simple C structure definitions, the Node structures
used a pseudo-inheritance system which automatically generated creator and
accessor functions. Since every node inherited from LispValue, the whole thing
was a mess. Here's a little anecdote:

    LispValue definition -- class used to support lisp structures
    in C.  This is here because we did not want to totally rewrite
    planner and executor code which depended on lisp structures when
    we ported postgres V1 from lisp to C. -cim 4/23/90