Skip to content

Chapter 18.1 · Query Planning and Execution · 26 min read

The Cloudberry Executor: Framework and Lifecycle

ExecutorStart, ExecutorRun and ExecutorEnd, the plan-state tree, tuple slots, and the pull-based volcano model that drives every node.

By Tushar Pednekar · · Verified against the Apache Cloudberry source tree, read September 2026

Every node in every plan obeys the same three-call contract. Once you see it, EXPLAIN ANALYZE output reads differently.

Chapters 15–17 produced a PlannedStmt: a tree of instructions, and a pile of predictions about how many rows each step would emit. This chapter runs it. src/backend/executor/ is 90 908 lines of C, and almost all of it is one shape repeated eighty-odd times — a node that knows how to produce its next tuple. This section describes the frame those nodes hang in: how the state tree is built and torn down, who calls whom for each tuple, where the memory and the snapshot live, how a tuple is handed from one operator to the next without either knowing the other’s storage format, and — the part everyone actually needs and almost nobody is told — what the numbers in EXPLAIN ANALYZE mean when there are three segments doing the work.

18.1.1 The demand-pull model and the two trees

A Plan node is inert. It has no cursor, no hash table, no open file — it is a description, which is why it can be serialized and shipped to a segment (§15.4, §17.5, §13.3). Execution therefore needs a second tree: for every Plan node, one PlanState node holding the mutable state that operator needs while it runs. The two trees have the same shape and are built together by ExecInitNode (src/backend/executor/execProcnode.c:192), a recursive switch on the plan’s node tag — case T_SeqScan: result = (PlanState *) ExecInitSeqScan(...) at :283, and so on for every node type in the book. Each state node keeps a back-pointer to its plan (ps->plan) and a pointer to the one shared EState (ps->state).

Plan tree — optimizer output, serialized, dispatched PlanState tree — built by ExecInitNode, one per executor AggFinalize, AGG_PLAIN MotionGather 3:1, slice 1 AggPartial SeqScanscanrelid 1 lefttree AggStatetransValues, ExprContext MotionStatetuple queue, EOS flags AggStatehashtable or transValues SeqScanStateTableScanDesc, slot, pin ps->plan ps->state EState snapshot, range table, slice table, per-tuple ExprContext solid lefttree / righttree child links exist in both trees · dashed = each state node knows its plan · green = all state nodes share one EState
The two trees for SELECT count(*) FROM t: same shape, three different kinds of edge.

Tuples move by demand, not by push. A parent asks its child for one tuple by calling ExecProcNode — an inline function in src/include/executor/executor.h:299 that tests chgParam and then makes one indirect call through the child’s own function pointer. The child returns a TupleTableSlot holding exactly one tuple, or a NULL/empty slot meaning end of stream. That is the whole contract. It is the classic volcano, or iterator, model, and the vocabulary is worth fixing: a node is pulled, never pushed; NULL means finished, not empty; and nothing between two nodes is batched or materialised unless some node deliberately chooses to buffer (Material, Sort, Hash — §18.3, §18.4, §18.6).

Seq ScanAggregateExecutePlan loopSeq ScanAggregateExecutePlan loopone tuple per call — no batching, no materialisationExecProcNodeExecProcNodeslot holding row 1ExecProcNodeslot holding row 2ExecProcNodeNULL slot — end of streamslot holding the countExecProcNodeNULL slot
One tuple per call. The loop at the top pulls until the root returns NULL.

Two consequences carry the rest of the chapter. First, memory is bounded by what the blocking operators choose to hold, not by the size of the intermediate results — which is why work_mem is an operator property and spilling is an operator decision (§18.3). Second, if the consumer stops asking, the producer stops working: LIMIT is cheap for free. In a single-process PostgreSQL that second property is merely an optimisation. In Cloudberry, where the producer may be a different process on a different host with a full send queue, it becomes a correctness problem — and the answer is ExecSquelchNode, in §18.1.7.

18.1.2 Start, Run, Finish, End

Every execution passes through four functions in src/backend/executor/execMain.c, each a thin wrapper that calls an extension hook if one is installed and otherwise the standard_ implementation — ExecutorStart (:231, hook tested at :243, standard_ExecutorStart at :250), ExecutorRun (:830/:858), ExecutorFinish (:1152/:1161) and ExecutorEnd (:1240/:1249). This is the seam pg_stat_statements and friends hang off. ExecutorStart calls CreateExecutorState (:351), registers the snapshot (:435), builds the slice table on the coordinator (InitSliceTable, :455) and then calls InitPlan (:1788), which opens the result relations, initialises the init-plans (:1995) and finally calls ExecInitNode on the plan tree (:2018).

a cursor calls Run once per FETCH
ExecutorStart

CreateExecutorState builds the EState

InitPlan: range table, snapshot, slice table

ExecInitNode recurses, building the PlanState tree
ExecutorRun

ExecutePlan loops: reset the per-tuple context,

ExecProcNode on the root node,

hand each slot to the DestReceiver
ExecutorFinish

ExecPostprocessPlan, AfterTriggerEndQuery,

then ExecSquelchNode over the whole tree, forced
ExecutorEnd

ExecEndNode recurses: close relations,

free the per-query memory context
The four phases. A cursor re-enters phase 2 once per FETCH.

src/backend/executor/execMain.c:2892

	for (;;)
	{
		ResetPerTupleExprContext(estate);        /* :2895 */
		slot = ExecProcNode(planstate);          /* :2900 */
		if (TupIsNull(slot))
		{
			estate->es_got_eos = true;
			(void) ExecShutdownNode(planstate);  /* :2916 */
			break;
		}
		if (!dest->receiveSlot(slot, dest))      /* :2947, junk filter elided */
			break;

That loop, in ExecutePlan (:2830), is ExecutorRun: reset the per-tuple memory, pull one tuple from the root, ship it to the DestReceiver — which on the coordinator writes the client protocol (§13.1) and on a segment feeds the sending side of a Motion. ExecutorFinish then runs after-triggers and, at :1196, calls ExecSquelchNode(planstate, true) over the whole tree: the executor’s last act before teardown is to tell everybody nobody is reading any more. ExecutorEnd calls ExecEndNode (execProcnode.c:797), the mirror image of ExecInitNode, which recurses down closing relations and freeing state.

18.1.3 One statement, N+1 executors

On the coordinator this lifecycle is driven by the portal (§13.1): pquery.c:246 calls ExecutorStart, :251 ExecutorRun, :258 ExecutorFinish, :260 ExecutorEnd. On a segment the same four functions are reached from a completely different door: the 'M' message (tcop/postgres.c:527, :5942) carries a serialized plan and slice table to exec_mpp_query (:1174), which builds a QueryDesc and runs exactly this lifecycle. So for one user statement there are N+1 executors, one per segment plus the coordinator’s, each with its own EState, its own PlanState tree, its own instrumentation — all running at the same time, all convinced they are executing the query.

Four processes, one sess_id, one statement. Segment −1 is the QD.

-- session A, kept busy for 9 s; the subquery runs once per segment
SELECT count(*) FROM (SELECT pg_sleep(9) FROM gp_dist_random('gp_id')) x;

-- session B, while A is still running: who is executing it?
SELECT gp_execution_segment() AS seg, pid, sess_id, state
  FROM gp_dist_random('pg_stat_activity') WHERE query LIKE 'SELECT count(%'
UNION ALL
SELECT -1, pid, sess_id, state
  FROM pg_stat_activity WHERE query LIKE 'SELECT count(%'
ORDER BY 1;
 seg |  pid  | sess_id | state
-----+-------+---------+--------
  -1 | 20203 |    6338 | active
   0 | 20205 |    6338 | active
   1 | 20206 |    6338 | active
   2 | 20207 |    6338 | active
(4 rows)

Each of those executors is handed the whole plan but is responsible for one slice of it. standard_ExecutorStart asks getGpExecIdentity (execMain.c:715) which of three roles it plays — GP_ROOT_SLICE (it drives the top of the plan), GP_NON_ROOT_ON_QE (it drives a slice whose root is a sending Motion, :731) or GP_IGNORE (it has nothing to do, :723) — and EState.eliminateAliens lets it skip initialising the alien nodes that belong to other slices altogether. Which slices exist, and which segments each runs on, was decided in §16.3; how the tuples actually cross between them is Chapter 19.

18.1.4 EState and ExprContext

EState (src/include/nodes/execnodes.h:635) is the per-query container every node reaches through ps->state. It is a long struct and reading it front to back is not illuminating; what matters is that it holds the four things a node cannot invent for itself — what is visible, what relations exist, where memory comes from, and which executor am I.

EState fieldWhat it is for
es_snapshot, es_crosscheck_snapshotThe visibility this query sees — on a QE it derives from the distributed snapshot that arrived with the dispatch (§10.1, §10.5).
es_range_table, es_relationsThe RTE list, and the array of Relations opened during InitPlan; a scan node finds its table by indexing this with scanrelid.
es_result_relations, es_output_cidTargets of an INSERT/UPDATE/DELETE, and the command id their tuples are stamped with (§9.2).
es_query_cxt, es_per_tuple_exprcontextThe two memory lifetimes — everything a node allocates lives in one or the other.
es_instrumentThe INSTRUMENT_* bitmask. This is the single flag EXPLAIN ANALYZE sets, and everything in §18.1.8 follows from it.
es_sliceTable, currentSliceId — CloudberryThe slice table from §16.3, and which slice this executor is currently running.
eliminateAliens, locallyExecutableSubplans — CloudberryPermission to skip initialising the parts of the plan that belong to other slices.
showstatctx, dispatcherState — CloudberryWhere EXPLAIN ANALYZE statistics arriving from the QEs are accumulated, and the handle on the dispatched gangs (§13.3).
es_got_eos — CloudberryWhether end-of-stream was reached, so a cursor’s ExecutorEnd tears down correctly.

Sitting one level down is ExprContext (execnodes.h:258), the per-tuple container. Its ecxt_scantuple, ecxt_innertuple and ecxt_outertuple are the slots an expression reads its Vars from; ecxt_per_query_memory and ecxt_per_tuple_memory are the two lifetimes again. The second one is the reason the executor does not leak: expression results are palloc’d in the short-lived context, and the whole context is reset once per tuple — that is the ResetPerTupleExprContext at the top of the loop above, and the same discipline is repeated inside nodes that iterate. §18.5 lives in this struct. One Cloudberry addition is worth flagging: with explain_memory_verbosity = detail, ExecInitNode gives every node its own MemoryContext (execProcnode.c:222, stored in PlanState.node_context) purely so that memory can be attributed per node — at the cost of losing all cross-node reuse.

18.1.5 The ExecProcNode trampoline

PlanState carries two function pointers: ExecProcNode, which callers use, and ExecProcNodeReal, the node’s actual method. The indirection buys a one-time trick. Every node registers its method through ExecSetExecProcNode, which does not install it — it installs a stub.

src/backend/executor/execProcnode.c:586

void ExecSetExecProcNode(PlanState *node, ExecProcNodeMtd function)
{
	node->ExecProcNodeReal = function;
	node->ExecProcNode = ExecProcNodeFirst;        /* :595 */
}

static TupleTableSlot *ExecProcNodeFirst(PlanState *node)   /* :604 */
{
	check_stack_depth();
	node->ExecProcNode = ExecProcNodeGPDB;         /* :631 — retarget */
	return node->ExecProcNode(node);
}
1 · after ExecInitNode PlanState ExecProcNode ExecProcNodeReal ExecProcNodeFirst ExecSeqScan Upstream PostgreSQL would install the real method here, or ExecProcNodeInstr when instrumenting — in Cloudberry that branch sits inside #if 0 at execProcnode.c:627, so the wrapper always stays. first call only: check_stack_depth(); then node->ExecProcNode = ExecProcNodeGPDB; and return node->ExecProcNode(node); 2 · steady state, for every tuple after the first PlanState ExecProcNode ExecProcNodeReal ExecProcNodeGPDB ExecSeqScan calls ExecProcNodeReal
The pointer rewrites itself on the first call. Every later tuple pays one indirect call and no test.

This is where Cloudberry parts company with upstream, and it is worth knowing because the folklore is the other way round. In PostgreSQL, ExecProcNodeFirst retargets the pointer to ExecProcNodeReal when instrumentation is off and to ExecProcNodeInstr when it is on — so EXPLAIN ANALYZE literally changes the call path and an un-instrumented query pays nothing per tuple. In this tree that optimisation is disabled: the retarget at :627 and the whole of ExecProcNodeInstr (:696) are bracketed by #if 0, and :631 unconditionally installs ExecProcNodeGPDB instead, because Cloudberry has per-tuple work that must happen on every node regardless.

src/backend/executor/execProcnode.c:637

	if (QueryFinishPending && !IsA(node, MotionState))
		return NULL;
	if (node->squelched)
		elog(ERROR, "cannot execute squelched plan node of type: %d",
			 (int) nodeTag(node));
	if (node->instrument)
		InstrStartNode(node->instrument);
	result = node->ExecProcNodeReal(node);
	if (node->instrument)
		InstrStopNode(node->instrument, TupIsNull(result) ? 0.0 : 1.0);

18.1.6 TupleTableSlot: one tuple, four physical forms

A Seq Scan over a heap table hands a tuple to a Sort, which hands it to a Hash Join, which hands it to a Motion. None of those operators wants to know that the first one came out of a shared-buffer page while the second spooled it to a temp file. TupleTableSlot (src/include/executor/tuptable.h:118) is the abstraction that makes this possible: a fixed header — tts_flags, tts_nvalid, tts_tupleDescriptor, the tts_values/tts_isnull arrays — plus a const TupleTableSlotOps *tts_ops vtable, and a subtype-specific tail holding whatever physical tuple the slot actually wraps. execTuples.c defines exactly four such vtables.

tts_opsPhysical form behind the slotWhere you meet it
TTSOpsVirtual (execTuples.c:1043)No physical tuple at all — only the tts_values/tts_isnull arrays, whose Datums may point into somebody else’s memory. .get_heap_tuple and .get_minimal_tuple are deliberately NULL.Result slots of every projecting node — and, notably, the slot type table_slot_callbacks hands back for AO (appendonlyam_handler.c:508), AOCO (aocsam_handler.c:550) and PAX (pax_access_handle.cc:437).
TTSOpsHeapTuple (:1063)A palloc’d HeapTuple that the slot may own and free (TTS_FLAG_SHOULDFREE).Tuples fetched by TID, trigger tuples, anything freshly built by heap_form_tuple (§4.2).
TTSOpsMinimalTuple (:1080)A MinimalTuple — the same data layout without the 23-byte header, because nothing about visibility survives the trip anyway.Tuplestores, tuplesort runs, hash-join batch files: everything that spools tuples (§18.3, §18.4).
TTSOpsBufferHeapTuple (:1097)A HeapTuple pointing directly into a shared buffer page, with a pin held on that buffer for as long as the slot holds the tuple.heapam_slot_callbacks (heapam_handler.c:71) — so every heap scan (§3.4, §18.2).

Two things follow. First, that the AO/AOCO/PAX handlers all answer TTSOpsVirtual is the whole story of how a columnar table joins a heap table without a single node knowing: the AM decodes its own MemTuple or column stripes (§5.1, §6.2) straight into tts_values, and from there up the tree the tuple is indistinguishable. Second, deforming is lazy: slot_getattr (tuptable.h:352) only calls slot_getsomeattrs when tts_nvalid < attnum, so a slot deforms attributes 1..n and stops. Because a heap tuple’s attributes can only be located by walking past the ones before them, the cost of reading a column grows with its ordinal position — which is directly measurable.

m181_w: 30 int columns, 2 000 000 rows, heap. Same row count, same predicate, different column.

\timing on
SELECT count(c01) FROM m181_w;
SELECT count(c30) FROM m181_w;
SELECT count(c01) FROM m181_w;
SELECT count(c30) FROM m181_w;
  count
---------
 2000000
(1 row)
Time: 254.458 ms
  count
---------
 2000000
(1 row)
Time: 375.425 ms
  count
---------
 2000000
(1 row)
Time: 257.727 ms
  count
---------
 2000000
(1 row)
Time: 303.286 ms

18.1.7 Rescan, and the Cloudberry-only idea of squelch

Some subtrees run more than once. ExecReScan (src/backend/executor/execAmi.c:95) resets a node so it can be read again: it ends the current instrumentation cycle (InstrEndLoop), clears node->squelched, propagates chgParam to children, and dispatches to the node’s own ExecReScanXxx. That is what the inner side of a nested loop gets once per outer row (§18.4), and it is the only source of loops greater than one in EXPLAIN ANALYZE.

Nested loop, PostgreSQL planner, hash and merge joins disabled. The inner Materialize is rescanned once per outer row.

SET optimizer=off;
SET enable_hashjoin=off; SET enable_mergejoin=off;
EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF)
  SELECT count(*) FROM m181_d d, m181_skew s WHERE s.grp = d.grp;
 Finalize Aggregate (actual rows=1 loops=1)
   ->  Gather Motion 3:1  (slice1; segments: 3) (actual rows=3 loops=1)
         ->  Partial Aggregate (actual rows=1 loops=1)
               ->  Nested Loop (actual rows=90000 loops=1)
                     Join Filter: (d.grp = s.grp)
                     Rows Removed by Join Filter: 180000
                     ->  Seq Scan on m181_skew s (actual rows=90000 loops=1)
                     ->  Materialize (actual rows=3 loops=90000)
                           ->  Seq Scan on m181_d d (actual rows=3 loops=1)
 ...
 Optimizer: Postgres query optimizer

The opposite operation has no upstream equivalent at all. ExecSquelchNode (execAmi.c:756, with a long explanatory comment at :720) tells a subtree that has not yet returned end-of-data that nobody will ever read from it again. The comment states the real motive plainly: “This is necessary, to avoid deadlock with Motion nodes.” A receiving Motion in that subtree has senders on other segments; if a sender’s send queue is full it will block forever waiting for a reader that has quietly walked away, and sibling QEs waiting on that same sender starve with it. Squelching pushes the news down the tree — node->squelched = true and recursion into both children at :812–:814, with node-specific handlers for the cases that need them (ExecSquelchMotion, ExecSquelchAppend, ExecSquelchSequence, …). The secondary benefit is memory: most handlers call an ExecEagerFree* so a finished operator’s work_mem goes back to operators still running. A squelched node is not dead, merely closed for business — ExecReScan un-squelches it, while calling ExecProcNode on it raises cannot execute squelched plan node. LIMIT over a Motion is the classic trigger.

LIMIT 3 above a Hash Join whose inner input is a Broadcast Motion. The sender in slice2 scanned 4 501 of its 90 000 rows and then stopped.

SET optimizer=off;
EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF)
  SELECT a.id, b.grp FROM m181_t a JOIN m181_skew b ON a.grp = b.grp LIMIT 3;
 Limit (actual rows=3 loops=1)
   ->  Gather Motion 3:1  (slice1; segments: 3) (actual rows=3 loops=1)
         ->  Limit (actual rows=3 loops=1)
               ->  Hash Join (actual rows=3 loops=1)
                     Hash Cond: (b.grp = a.grp)
                     ->  Broadcast Motion 3:3  (slice2; segments: 3) (actual rows=1 loops=1)
                           ->  Seq Scan on m181_skew b (actual rows=4501 loops=1)
                     ->  Hash (actual rows=100172 loops=1)
                           Buckets: 524288  Batches: 1  Memory Usage: 8009kB
                           ->  Seq Scan on m181_t a (actual rows=100172 loops=1)
 ...
 Optimizer: Postgres query optimizer

18.1.8 Instrumentation: where EXPLAIN ANALYZE’s numbers come from

EXPLAIN ANALYZE sets EState.es_instrument, which makes ExecInitNode attach an Instrumentation struct (src/include/executor/instrument.h:75) to every node. ExecProcNodeGPDB then brackets each call with InstrStartNode (instrument.c:94) and InstrStopNode (:168): the first takes a timestamp, the second accumulates the elapsed time into counter and bumps tuplecount by one if a tuple came back. InstrEndLoop (:186) closes a cycle — it folds counter into total, tuplecount into ntuples, and increments nloops. It is called from ExecReScan and at shutdown. This matters for reading the output, because explain.c:2316 computes rows = instrument->ntuples / nloops: actual rows is an average per loop, not a total. The Materialize above really produced 270 000 tuples; it reports 3.

Ground truth first, then the plan. m181_skew is deliberately skewed: 90 000 rows on seg0, 10 000 on seg1, none at all on seg2.

SELECT gp_segment_id, count(*) FROM m181_skew GROUP BY 1 ORDER BY 1;

SET optimizer=off;
SET gp_enable_explain_allstat=on;
EXPLAIN (ANALYZE, COSTS OFF) SELECT count(*) FROM m181_skew;
 gp_segment_id | count
---------------+-------
             0 | 90000
             1 | 10000
(2 rows)

 Finalize Aggregate (actual time=15.545..15.551 rows=1 loops=1)
   ->  Gather Motion 3:1  (slice1; segments: 3) (actual time=1.620..15.518 rows=3 loops=1)
         ->  Partial Aggregate (actual time=13.947..13.952 rows=1 loops=1)
               allstat: seg_firststart_total_ntuples/seg0_2.296 ms_14 ms_1/seg1_2.260 ms_1.559 ms_1/seg2_2.362 ms_0.030 ms_1//end
               ->  Seq Scan on m181_skew (actual time=0.070..9.088 rows=90000 loops=1)
                     allstat: seg_firststart_total_ntuples/seg0_2.298 ms_9.088 ms_90000/seg1_2.262 ms_1.057 ms_10000/seg2_2.364 ms_0.018 ms_0//end
 ...
 Optimizer: Postgres query optimizer
 Execution Time: 16.813 ms

There is the rule, established rather than assumed. The table holds 100 000 rows; the Seq Scan reports 90 000. It is not a sum and not an average — it is one segment’s own count, the count of the segment that produced the most tuples at that node. gp_enable_explain_allstat prints the raw per-segment triples underneath and proves it (90 000 / 10 000 / 0). The mechanism is in explain_gp.c: each QE ships its Instrumentation to the QD, cdbexplain_depStatAcc_upd (:972) tracks which segment held the maximum, and the QD then copies that one segment’s entire instrumentation record into the PlanState that EXPLAIN will print.

src/backend/commands/explain_gp.c:1179

	if (ntuples.agg.vcnt > 0)
		nodeAcc = &ntuples;
	else if (nloops.agg.vcnt > 0)
		nodeAcc = &nloops;
	if (nodeAcc)
	{
		instr->counter    = nodeAcc->nsimax->counter;
		instr->ntuples    = nodeAcc->nsimax->ntuples;
		instr->nloops     = nodeAcc->nsimax->nloops;
		instr->nfiltered1 = nodeAcc->nsimax->nfiltered1;
		instr->workmemused = nodeAcc->nsimax->workmemused;
		/* ... every other counter, from the same winner ... */

Read that carefully, because it has a consequence nobody expects: the winner is chosen per node, independently. Two adjacent lines of one plan can be reporting two different segments — and the timings, the filter counts and the Extra Text on a line all come from that line’s winner, not from a globally chosen “worst” segment. When a node produced no tuples anywhere, the tie is broken by nloops and effectively by arrival order.

What you are readingWhose number it actually is
actual rows, loops on a segment sliceOne segment’s own figure — the segment with the most tuples at that node. Never a sum. And actual rows is itself ntuples / nloops, a per-loop average (explain.c:2316).
actual time, Rows Removed by Filter, Sort Method, Extra Text: (segN)Copied from the same winner as that node’s row count (explain_gp.c:1183–:1201). Different lines may be different segments; Extra Text names its segment, which is how you can tell.
Anything in slice0 — the top of the plan, Gather MotionA true count, because slice0 has exactly one worker: the coordinator.
per-node Executor Memory: X Segments: N Max: Y (segment S) (explain_memory_verbosity = detail)Here the convention flips: X is the sum over the N reporting workers, Y is the per-segment maximum and S names it.
(sliceN) Executor memory: A avg x Nx(M) workers, B max (segS)Peak executor memory in that slice: A averaged over the N segments reporting, B the maximum, segS where it happened. M is the count of PostgreSQL parallel workers launched — 0 unless enable_parallel is on (§18.2).
Work_mem: X max, Y wantedMaximum over all nodes and all workers of the slice (explain_gp.c:2206). Y wanted appears only when some operator would have avoided spilling given more memory.
The leading * on a slice lineSet by the same branch that prints wanted: es->str->data[flag] = '*' (explain_gp.c:2214) reaches back and overwrites a placeholder character at the start of the line. It means exactly “this slice spilled”.
Memory used: / Memory wanted:Statement level, not slice level: what the resource manager granted this statement as query_mem (:1913), and what it would have taken to avoid every spill (:1934). Chapter 21 owns the granting side.

Squeeze the statement and the whole footer wakes up. Note that Rows Removed by Filter and the Seq Scan row count come from different segments — m181_t holds 99 877 / 99 951 / 100 172 rows on seg0/1/2.

SET optimizer=off;
SET statement_mem='2MB';
EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF)
  SELECT grp, count(*) FROM m181_t GROUP BY grp, id HAVING count(*) > 1;
 Gather Motion 3:1  (slice1; segments: 3) (actual rows=0 loops=1)
   ->  HashAggregate (actual rows=0 loops=1)
         Group Key: grp, id
         Filter: (count(*) > 1)
         Rows Removed by Filter: 99877
         Extra Text: (seg0)   hash table(s): 1; 99877 groups total in 5 batches, 294688 spill partitions; disk usage: 2816KB; chain length 2.2 avg, 7 max; using 99877 of 327680 buckets; total 0 expansions.
         ->  Seq Scan on m181_t (actual rows=100172 loops=1)
 Planning Time: 1.558 ms
   (slice0)    Executor memory: 1565K bytes.
 * (slice1)    Executor memory: 2951K bytes avg x 3x(0) workers, 2951K bytes max (seg0).  Work_mem: 5777K bytes max, 13841K bytes wanted.
 Memory used:  2048kB
 Memory wanted:  14040kB
 Optimizer: Postgres query optimizer
 Execution Time: 149.575 ms

explain_memory_verbosity = detail adds a per-node line — and demonstrates the sum-versus-max convention: 330 kB total over 3 segments, 110 kB each.

SET optimizer=off;
SET explain_memory_verbosity='detail';
EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF) SELECT count(*) FROM m181_skew;
 Finalize Aggregate (actual rows=1 loops=1)
   Executor Memory: 125kB  Segments: 1  Max: 125kB (segment -1)
   ->  Gather Motion 3:1  (slice1; segments: 3) (actual rows=3 loops=1)
         Executor Memory: 118kB  Segments: 1  Max: 118kB (segment -1)
         ->  Partial Aggregate (actual rows=1 loops=1)
               Executor Memory: 672kB  Segments: 3  Max: 224kB (segment 0)
               ->  Seq Scan on m181_skew (actual rows=90000 loops=1)
                     Executor Memory: 330kB  Segments: 3  Max: 110kB (segment 0)
 ...
 Optimizer: Postgres query optimizer

TIMING OFF versus the default, 6 runs each: 9 M rows, ~3 M per segment, only Execution Time shown. Ratios on one machine, assert-enabled build — not a benchmark.

-- x6:
EXPLAIN (ANALYZE, TIMING OFF, COSTS OFF) SELECT count(*) FROM m181_big WHERE id > 0;
-- x6:
EXPLAIN (ANALYZE,            COSTS OFF) SELECT count(*) FROM m181_big WHERE id > 0;
-- TIMING OFF
 Execution Time: 1002.310 ms
 Execution Time: 984.674 ms
 Execution Time: 991.330 ms
 Execution Time: 971.711 ms
 Execution Time: 963.955 ms
 Execution Time: 969.497 ms
-- TIMING ON (the default)
 Execution Time: 1109.578 ms
 Execution Time: 997.255 ms
 Execution Time: 991.228 ms
 Execution Time: 1079.157 ms
 Execution Time: 916.240 ms
 Execution Time: 960.514 ms

TIMING OFF exists because InstrStartNode/InstrStopNode take a timestamp per tuple per node, and on some platforms that is genuinely expensive. On this box it is not measurable: six runs each over three million tuples per segment overlap completely, and the fastest run of all is a TIMING ON one. Two reasons. INSTR_TIME_SET_CURRENT here is clock_gettime through the vDSO — tens of nanoseconds, no syscall — and, per §18.1.5, Cloudberry pays the ExecProcNodeGPDB wrapper on every tuple whether or not instrumentation is on, so the marginal cost of the timestamps is a small share of an overhead that is already there. Treat TIMING OFF as insurance, not as a habit; actual rows survives it, and that is the number this chapter cares about.

Everything above is optimizer-independent: the same plan run under GPORCA produces byte-for-byte the same instrumentation vocabulary (only the Optimizer: footer changes), because §17.5 hands the executor the same PlannedStmt the PostgreSQL planner does. Two knobs are worth remembering from this section: gp_enable_explain_allstat, which turns the max-across-segments guesswork into printed per-segment truth, and explain_memory_verbosity (suppress / summary / detail), where summary adds Vmem reserved per slice and detail costs you a memory context per node. From here the chapter descends into the nodes themselves: scans in §18.2, sort and aggregation in §18.3, joins in §18.4, expressions in §18.5, subqueries and the long tail in §18.6, the Cloudberry-only operators in §18.7 — and Motion, the one node that turns this single-process machine into an MPP one, in Chapter 19.

More in Query Planning and Execution