upstream/postgresql
1
0
Fork 0
mirror of https://github.com/postgres/postgres.git synced 2026-02-03 20:40:14 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
postgresql/src/backend/executor/nodeTableFuncscan.c
Tom Lane 1cff1b95ab Represent Lists as expansible arrays, not chains of cons-cells. 
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link.  We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells.  That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.

In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links.  Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header.  (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)

Of course this is not without downsides.  The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.

For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state.  We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value.  (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.

The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext().  The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell().  However, we no longer
need a previous-cell pointer to delete a list cell efficiently.  Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)

There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents.  To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen.  This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).

There are two notable API differences from the previous code:

* lnext() now requires the List's header pointer in addition to the
current cell's address.

* list_delete_cell() no longer requires a previous-cell argument.

These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.

Programmers should be aware of these significant performance changes:

* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.

* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.)  Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long.  Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.

* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index.  These have the
data-movement penalty explained above, but there's no search penalty.

* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists.  The second argument is
now declared "const List *" to reflect that it isn't changed.

This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests.  As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.

Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit.  Code using those should have been gone a dozen years ago.

Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.

Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 13:41:58 -04:00

526 lines
14 KiB
C
Raw Blame History

/*-------------------------------------------------------------------------
*
* nodeTableFuncscan.c
* Support routines for scanning RangeTableFunc (XMLTABLE like functions).
*
* Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
*
* IDENTIFICATION
* src/backend/executor/nodeTableFuncscan.c
*
*-------------------------------------------------------------------------
*/
/*
* INTERFACE ROUTINES
* ExecTableFuncscan scans a function.
* ExecFunctionNext retrieve next tuple in sequential order.
* ExecInitTableFuncscan creates and initializes a TableFuncscan node.
* ExecEndTableFuncscan releases any storage allocated.
* ExecReScanTableFuncscan rescans the function
*/
#include "postgres.h"
#include "nodes/execnodes.h"
#include "executor/executor.h"
#include "executor/nodeTableFuncscan.h"
#include "executor/tablefunc.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/xml.h"
static TupleTableSlot *TableFuncNext(TableFuncScanState *node);
static bool TableFuncRecheck(TableFuncScanState *node, TupleTableSlot *slot);
static void tfuncFetchRows(TableFuncScanState *tstate, ExprContext *econtext);
static void tfuncInitialize(TableFuncScanState *tstate, ExprContext *econtext, Datum doc);
static void tfuncLoadRows(TableFuncScanState *tstate, ExprContext *econtext);
/* ----------------------------------------------------------------
* Scan Support
* ----------------------------------------------------------------
*/
/* ----------------------------------------------------------------
* TableFuncNext
*
* This is a workhorse for ExecTableFuncscan
* ----------------------------------------------------------------
*/
static TupleTableSlot *
TableFuncNext(TableFuncScanState *node)
{
TupleTableSlot *scanslot;
scanslot = node->ss.ss_ScanTupleSlot;
/*
* If first time through, read all tuples from function and put them in a
* tuplestore. Subsequent calls just fetch tuples from tuplestore.
*/
if (node->tupstore == NULL)
tfuncFetchRows(node, node->ss.ps.ps_ExprContext);
/*
* Get the next tuple from tuplestore.
*/
(void) tuplestore_gettupleslot(node->tupstore,
true,
false,
scanslot);
return scanslot;
}
/*
* TableFuncRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
static bool
TableFuncRecheck(TableFuncScanState *node, TupleTableSlot *slot)
{
/* nothing to check */
return true;
}
/* ----------------------------------------------------------------
* ExecTableFuncscan(node)
*
* Scans the function sequentially and returns the next qualifying
* tuple.
* We call the ExecScan() routine and pass it the appropriate
* access method functions.
* ----------------------------------------------------------------
*/
static TupleTableSlot *
ExecTableFuncScan(PlanState *pstate)
{
TableFuncScanState *node = castNode(TableFuncScanState, pstate);
return ExecScan(&node->ss,
(ExecScanAccessMtd) TableFuncNext,
(ExecScanRecheckMtd) TableFuncRecheck);
}
/* ----------------------------------------------------------------
* ExecInitTableFuncscan
* ----------------------------------------------------------------
*/
TableFuncScanState *
ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
{
TableFuncScanState *scanstate;
TableFunc *tf = node->tablefunc;
TupleDesc tupdesc;
int i;
/* check for unsupported flags */
Assert(!(eflags & EXEC_FLAG_MARK));
/*
* TableFuncscan should not have any children.
*/
Assert(outerPlan(node) == NULL);
Assert(innerPlan(node) == NULL);
/*
* create new ScanState for node
*/
scanstate = makeNode(TableFuncScanState);
scanstate->ss.ps.plan = (Plan *) node;
scanstate->ss.ps.state = estate;
scanstate->ss.ps.ExecProcNode = ExecTableFuncScan;
/*
* Miscellaneous initialization
*
* create expression context for node
*/
ExecAssignExprContext(estate, &scanstate->ss.ps);
/*
* initialize source tuple type
*/
tupdesc = BuildDescFromLists(tf->colnames,
tf->coltypes,
tf->coltypmods,
tf->colcollations);
/* and the corresponding scan slot */
ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc,
&TTSOpsMinimalTuple);
/*
* Initialize result type and projection.
*/
ExecInitResultTypeTL(&scanstate->ss.ps);
ExecAssignScanProjectionInfo(&scanstate->ss);
/*
* initialize child expressions
*/
scanstate->ss.ps.qual =
ExecInitQual(node->scan.plan.qual, &scanstate->ss.ps);
/* Only XMLTABLE is supported currently */
scanstate->routine = &XmlTableRoutine;
scanstate->perTableCxt =
AllocSetContextCreate(CurrentMemoryContext,
"TableFunc per value context",
ALLOCSET_DEFAULT_SIZES);
scanstate->opaque = NULL; /* initialized at runtime */
scanstate->ns_names = tf->ns_names;
scanstate->ns_uris =
ExecInitExprList(tf->ns_uris, (PlanState *) scanstate);
scanstate->docexpr =
ExecInitExpr((Expr *) tf->docexpr, (PlanState *) scanstate);
scanstate->rowexpr =
ExecInitExpr((Expr *) tf->rowexpr, (PlanState *) scanstate);
scanstate->colexprs =
ExecInitExprList(tf->colexprs, (PlanState *) scanstate);
scanstate->coldefexprs =
ExecInitExprList(tf->coldefexprs, (PlanState *) scanstate);
scanstate->notnulls = tf->notnulls;
/* these are allocated now and initialized later */
scanstate->in_functions = palloc(sizeof(FmgrInfo) * tupdesc->natts);
scanstate->typioparams = palloc(sizeof(Oid) * tupdesc->natts);
/*
* Fill in the necessary fmgr infos.
*/
for (i = 0; i < tupdesc->natts; i++)
{
Oid in_funcid;
getTypeInputInfo(TupleDescAttr(tupdesc, i)->atttypid,
&in_funcid, &scanstate->typioparams[i]);
fmgr_info(in_funcid, &scanstate->in_functions[i]);
}
return scanstate;
}
/* ----------------------------------------------------------------
* ExecEndTableFuncscan
*
* frees any storage allocated through C routines.
* ----------------------------------------------------------------
*/
void
ExecEndTableFuncScan(TableFuncScanState *node)
{
/*
* Free the exprcontext
*/
ExecFreeExprContext(&node->ss.ps);
/*
* clean out the tuple table
*/
if (node->ss.ps.ps_ResultTupleSlot)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecClearTuple(node->ss.ss_ScanTupleSlot);
/*
* Release tuplestore resources
*/
if (node->tupstore != NULL)
tuplestore_end(node->tupstore);
node->tupstore = NULL;
}
/* ----------------------------------------------------------------
* ExecReScanTableFuncscan
*
* Rescans the relation.
* ----------------------------------------------------------------
*/
void
ExecReScanTableFuncScan(TableFuncScanState *node)
{
Bitmapset *chgparam = node->ss.ps.chgParam;
if (node->ss.ps.ps_ResultTupleSlot)
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
ExecScanReScan(&node->ss);
/*
* Recompute when parameters are changed.
*/
if (chgparam)
{
if (node->tupstore != NULL)
{
tuplestore_end(node->tupstore);
node->tupstore = NULL;
}
}
if (node->tupstore != NULL)
tuplestore_rescan(node->tupstore);
}
/* ----------------------------------------------------------------
* tfuncFetchRows
*
* Read rows from a TableFunc producer
* ----------------------------------------------------------------
*/
static void
tfuncFetchRows(TableFuncScanState *tstate, ExprContext *econtext)
{
const TableFuncRoutine *routine = tstate->routine;
MemoryContext oldcxt;
Datum value;
bool isnull;
Assert(tstate->opaque == NULL);
/* build tuplestore for the result */
oldcxt = MemoryContextSwitchTo(econtext->ecxt_per_query_memory);
tstate->tupstore = tuplestore_begin_heap(false, false, work_mem);
/*
* Each call to fetch a new set of rows - of which there may be very many
* if XMLTABLE is being used in a lateral join - will allocate a possibly
* substantial amount of memory, so we cannot use the per-query context
* here. perTableCxt now serves the same function as "argcontext" does in
* FunctionScan - a place to store per-one-call (i.e. one result table)
* lifetime data (as opposed to per-query or per-result-tuple).
*/
MemoryContextSwitchTo(tstate->perTableCxt);
PG_TRY();
{
routine->InitOpaque(tstate,
tstate->ss.ss_ScanTupleSlot->tts_tupleDescriptor->natts);
/*
* If evaluating the document expression returns NULL, the table
* expression is empty and we return immediately.
*/
value = ExecEvalExpr(tstate->docexpr, econtext, &isnull);
if (!isnull)
{
/* otherwise, pass the document value to the table builder */
tfuncInitialize(tstate, econtext, value);
/* initialize ordinality counter */
tstate->ordinal = 1;
/* Load all rows into the tuplestore, and we're done */
tfuncLoadRows(tstate, econtext);
}
}
PG_CATCH();
{
if (tstate->opaque != NULL)
routine->DestroyOpaque(tstate);
PG_RE_THROW();
}
PG_END_TRY();
/* clean up and return to original memory context */
if (tstate->opaque != NULL)
{
routine->DestroyOpaque(tstate);
tstate->opaque = NULL;
}
MemoryContextSwitchTo(oldcxt);
MemoryContextReset(tstate->perTableCxt);
return;
}
/*
* Fill in namespace declarations, the row filter, and column filters in a
* table expression builder context.
*/
static void
tfuncInitialize(TableFuncScanState *tstate, ExprContext *econtext, Datum doc)
{
const TableFuncRoutine *routine = tstate->routine;
TupleDesc tupdesc;
ListCell *lc1,
*lc2;
bool isnull;
int colno;
Datum value;
int ordinalitycol =
((TableFuncScan *) (tstate->ss.ps.plan))->tablefunc->ordinalitycol;
/*
* Install the document as a possibly-toasted Datum into the tablefunc
* context.
*/
routine->SetDocument(tstate, doc);
/* Evaluate namespace specifications */
forboth(lc1, tstate->ns_uris, lc2, tstate->ns_names)
{
ExprState *expr = (ExprState *) lfirst(lc1);
Value *ns_node = (Value *) lfirst(lc2);
char *ns_uri;
char *ns_name;
value = ExecEvalExpr((ExprState *) expr, econtext, &isnull);
if (isnull)
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("namespace URI must not be null")));
ns_uri = TextDatumGetCString(value);
/* DEFAULT is passed down to SetNamespace as NULL */
ns_name = ns_node ? strVal(ns_node) : NULL;
routine->SetNamespace(tstate, ns_name, ns_uri);
}
/* Install the row filter expression into the table builder context */
value = ExecEvalExpr(tstate->rowexpr, econtext, &isnull);
if (isnull)
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("row filter expression must not be null")));
routine->SetRowFilter(tstate, TextDatumGetCString(value));
/*
* Install the column filter expressions into the table builder context.
* If an expression is given, use that; otherwise the column name itself
* is the column filter.
*/
colno = 0;
tupdesc = tstate->ss.ss_ScanTupleSlot->tts_tupleDescriptor;
foreach(lc1, tstate->colexprs)
{
char *colfilter;
Form_pg_attribute att = TupleDescAttr(tupdesc, colno);
if (colno != ordinalitycol)
{
ExprState *colexpr = lfirst(lc1);
if (colexpr != NULL)
{
value = ExecEvalExpr(colexpr, econtext, &isnull);
if (isnull)
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("column filter expression must not be null"),
errdetail("Filter for column \"%s\" is null.",
NameStr(att->attname))));
colfilter = TextDatumGetCString(value);
}
else
colfilter = NameStr(att->attname);
routine->SetColumnFilter(tstate, colfilter, colno);
}
colno++;
}
}
/*
* Load all the rows from the TableFunc table builder into a tuplestore.
*/
static void
tfuncLoadRows(TableFuncScanState *tstate, ExprContext *econtext)
{
const TableFuncRoutine *routine = tstate->routine;
TupleTableSlot *slot = tstate->ss.ss_ScanTupleSlot;
TupleDesc tupdesc = slot->tts_tupleDescriptor;
Datum *values = slot->tts_values;
bool *nulls = slot->tts_isnull;
int natts = tupdesc->natts;
MemoryContext oldcxt;
int ordinalitycol;
ordinalitycol =
((TableFuncScan *) (tstate->ss.ps.plan))->tablefunc->ordinalitycol;
/*
* We need a short-lived memory context that we can clean up each time
* around the loop, to avoid wasting space. Our default per-tuple context
* is fine for the job, since we won't have used it for anything yet in
* this tuple cycle.
*/
oldcxt = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
/*
* Keep requesting rows from the table builder until there aren't any.
*/
while (routine->FetchRow(tstate))
{
ListCell *cell = list_head(tstate->coldefexprs);
int colno;
CHECK_FOR_INTERRUPTS();
ExecClearTuple(tstate->ss.ss_ScanTupleSlot);
/*
* Obtain the value of each column for this row, installing them into
* the slot; then add the tuple to the tuplestore.
*/
for (colno = 0; colno < natts; colno++)
{
Form_pg_attribute att = TupleDescAttr(tupdesc, colno);
if (colno == ordinalitycol)
{
/* Fast path for ordinality column */
values[colno] = Int32GetDatum(tstate->ordinal++);
nulls[colno] = false;
}
else
{
bool isnull;
values[colno] = routine->GetValue(tstate,
colno,
att->atttypid,
att->atttypmod,
&isnull);
/* No value? Evaluate and apply the default, if any */
if (isnull && cell != NULL)
{
ExprState *coldefexpr = (ExprState *) lfirst(cell);
if (coldefexpr != NULL)
values[colno] = ExecEvalExpr(coldefexpr, econtext,
&isnull);
}
/* Verify a possible NOT NULL constraint */
if (isnull && bms_is_member(colno, tstate->notnulls))
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("null is not allowed in column \"%s\"",
NameStr(att->attname))));
nulls[colno] = isnull;
}
/* advance list of default expressions */
if (cell != NULL)
cell = lnext(tstate->coldefexprs, cell);
}
tuplestore_putvalues(tstate->tupstore, tupdesc, values, nulls);
MemoryContextReset(econtext->ecxt_per_tuple_memory);
}
MemoryContextSwitchTo(oldcxt);
}
Reference in a new issue View git blame Copy permalink