Reference
Apache Cloudberry Internals
How the engine actually works, one subsystem at a time — read from the source tree rather than
summarised from the manual. Every code path carries a file and line reference, and every
psql
session is real captured output.
Most of it applies to Greenplum too. The lineage is shared, the catalogs and vocabulary are largely the same, and where Cloudberry has diverged the chapters say so explicitly.
Start here
Apache Cloudberry Architecture: How a Query Runs
Coordinator, segments, gangs and slices — how an MPP query is planned, dispatched and executed across an Apache Cloudberry cluster.
Part 1
Storage and Access Methods
Everything Cloudberry stores ends up as bytes in files on a segment. This part follows that path from the access methods down to the storage manager.
- 2 Data Files and the Storage Manager in Cloudberry
- 3 The Cloudberry Buffer Cache: Pinning and Clock Sweep
- 4 Heap Storage: Page Layout, Tuples, HOT and Pruning
- 5 Append-Optimized Tables in Cloudberry: AO and AOCS
- 6 PAX: Apache Cloudberry's Native Columnar Storage
- 7 Indexes in Cloudberry: B-Tree, GIN, BRIN and Bitmap
- 8 The Cloudberry System Catalog and Distributed Tables
Part 2
Transactions, MVCC and Locking
A single-node transaction is hard enough. This part covers what changes when a transaction spans every segment in the cluster.
Part 3
Query Planning and Execution
The longest part of the book, and the one most performance questions land in: two optimizers, an executor, and the interconnect that moves tuples between segments.
- 13 From SQL to Segments: Cloudberry Query Processing
- 14 Statistics and Selectivity Estimation in Cloudberry
- 15 The Postgres Planner in Cloudberry, Step by Step
- 16 Distributed Planning: Locus, Motion and Direct Dispatch
- 17.1 ORCA Optimizer: Entry Points and Fallback Rules
- 17.2 Inside ORCA's Cascades Engine: Groups and Jobs
- 17.3 ORCA Transformation Rules and the Search Space
- 17.4 ORCA's Metadata Layer and Cost Model Explained
- 17.5 DXL: Translating Queries Between ORCA and Cloudberry
- 17.6 Steering ORCA: GUCs, Traceflags and Minidumps
- 18.1 The Cloudberry Executor: Framework and Lifecycle
- 18.2 Executor Scan Nodes: Seq, Index, Bitmap and AO
- 18.3 Sort and Aggregate Nodes in the Cloudberry Executor
- 18.4 Join Execution: Nested Loop, Hash and Merge Joins
- 18.5 Expression Evaluation in the Cloudberry Executor
- 18.6 Subplans, CTEs and the Rest of the Executor Nodes
- 18.7 Executor Nodes That Exist Only in Cloudberry
- 19.1 How Distributed Query Execution Works in Cloudberry
- 19.2 The Dispatcher: Gangs, Gang Reuse and Async Dispatch
- 19.3 Slice Tables: Mapping Plan Slices to Gangs
- 19.4 Motion Nodes: Gather, Broadcast and Redistribute
- 19.5 The Cloudberry Interconnect: UDPIFC, TCP and Proxy
- 19.6 Parallel Retrieve Cursors and Endpoints in Cloudberry
Part 4
High Availability and Recovery
What keeps a cluster alive when a host dies, and what happens in the seconds after it does.
Part 5
Resource Management
Two managers, one question: who may use how much. Both are documented here from the source, including where each one falls short.