Lesson 07 · The decision

Choosing: query shape picks the engine

Six species later, the pattern behind the whole field guide should be visible: every database is a bet about your access patterns. Relational bets you'll ask unforeseen questions; key-value bets you always know the name; document bets you read whole aggregates; wide-column bets you'll write torrentially along planned paths; graph bets your questions traverse; the specialists each bet on one structural property. Choosing well means knowing which bet your workload lets you win. First, the whole zoo on one page:

speciesunit of datamade cheapmade expensive / impossiblereach for it when…
relationalrow (fact)ad-hoc queries, joins, constraints, transactionshuge write streams; objects reassembled per read; scale-out (classically)in doubt. Unknown future queries, invariants that must hold: the default for systems of record
key-valueopaque blob by keyGET/PUT by name at extreme speed & scaleany query by value or relationshipyou always know the key: sessions, carts, flags, cache, counters
documentself-contained treeread/write whole aggregate; per-doc atomicity; evolving fieldscross-document consistency; queries across the grain of the treeaggregate-shaped data, one-doc-at-a-time access, fast-moving schema
wide-columnpartitioned, sorted rowsmassive sustained writes; range reads inside a partitionad-hoc queries (declined!); multi-partition transactions; joinswrite-heavy at scale with a known, finite query list: events, telemetry, feeds
graphnodes + edgesvariable-depth traversal; path queries; relationship propertiesbulk scans/aggregation; sharding; being the only storethe core queries walk connections at depth ≥2: access analysis, fraud, recommendations
time-seriestimestamped pointsordered ingest, 10–20× compression, range/rollup queries, retentionupdates to old data; entity-centric queriesmetrics, sensors, ticks: anything shaped like (time, value)
searchanalyzed documentfull-text lookup + relevance rankingbeing a source of truth (relaxed durability/consistency)humans type words into a box; always as a CDC-fed secondary
vectorembedding pointnearest-neighbor by meaningexact matches, filters at scale (improving), source-of-truth dutysemantic similarity: search-by-meaning, recommendations, RAG

The framework, in four questions

1. What shape are the queries?: not the data; the data can be reshaped. By-name → KV. Aggregate-at-a-time → document. Ad-hoc/analytical → relational. Path-at-depth → graph. Time-ranged → TS. By-words / by-meaning → search / vector. 2. What's the read:write ratio and volume?: write-torrential with known queries → wide-column (LSM); read-heavy with rich queries → relational (B-tree) + a KV cache. 3. What must be true at every instant?: strong invariants (money, stock, uniqueness) → ACID at the core; tolerance for staleness buys you every relaxation in this guide. 4. What don't you know yet?: early products don't know their queries; that uncertainty is itself an argument for the model that doesn't require knowing (Lesson 1), until measurement says otherwise. Now pressure-test yourself:

The scenario gauntlet

Eight real workloads. Pick the primary engine for each: the feedback argues back. (Several have defensible seconds; the reasoning is the product.)

scenario 1 / 8 · score 0

Polyglot persistence: and its bill

Real systems end up combining species: the e-commerce cliché is PostgreSQL as system of record, Redis caching sessions and hot pages, Elasticsearch powering the search box, a vector index for recommendations, ClickHouse or a warehouse for analytics: each fed from the core by CDC (replication course, lesson 5), so there's exactly one writable truth and N disposable, rebuildable projections of it. That architecture is right: eventually. The bill is real: every added engine is another thing to operate, secure, monitor, upgrade, and reason about during incidents, another consistency lag to explain, another skill on the hiring page. So the discipline that separates senior from clever: add a species only when a measured workload demands it, keep one system of record, and let everything else be a projection you could delete and rebuild. Boring first; specialized under duress; never two sources of truth.

The course in one paragraph: storage engines don't have opinions, they have physics. B-trees buy reads with write cost; LSM buys writes with read cost; hashing buys speed with order; embedding buys locality with duplication; adjacency buys traversal with sharding pain; inverted and approximate indexes buy their one query with everything else. A database choice is just deciding which physics your access patterns get to enjoy: which is why the first question is never "which database is best?" but always "what will we ask, how often, and what must never be wrong?"

Final check

1. A startup's PM says: "We don't know our access patterns yet: so let's pick the scalable NoSQL option to be safe." What does this course say?

Wide-column demands the query list up front (Lesson 4's methodology); documents bake access patterns into aggregate shape (Lesson 3); key-value bets you'll never query by value (Lesson 2). Uncertainty is a workload property, and the engine that prices it lowest is the one with joins, indexes you can add later, and constraints. "Scale" solves a problem most products are lucky to ever have; flexibility solves the problem every product has on day one.

2. Orders live in PostgreSQL; the search box is served by Elasticsearch. A product name shows the old value in search for ~2 seconds after an edit. The team proposes writing to both stores in the request handler to fix it. What's the verdict?

This is the dual-writes-vs-CDC lesson meeting polyglot persistence. The CDC pipeline gives crash-consistent, ordered, replayable sync at the price of small lag; dual writes give the illusion of no lag at the price of silent divergence under partial failure. And 2PC across Postgres and ES isn't realistically on offer. Search indexes are projections: bounded staleness is their contract, same as Lesson 2's TTL.

3. Which single question from the four-question framework most reliably disqualifies candidate engines fastest?

Query shape tells you where each engine shines; invariants tell you which engines are forbidden: and forbidden beats shiny. A perfectly aggregate-shaped checkout flow still needs its stock decrement and payment record atomic, which is why the system of record so often stays boring while the specialists orbit it. Correctness constraints first, physics second, fashion never.