Single machines get a luxury they never notice: one clock, one instruction stream, and therefore a free, total, obvious order of events. Distribute the system and that luxury evaporates. Each machine's quartz crystal drifts (seconds per day, unsynchronized); NTP corrects the drift but only to within milliseconds — and sometimes steps a clock backwards. Meanwhile your application logic quietly assumes timestamps mean order: "show comments chronologically," "last write wins," "this event caused that one."
Here's the assumption failing:
When "later" comes first
Break causality with 40 milliseconds of skew
Alex posts a question (handled by server A). Miljan sees it and posts the answer (handled by server B). Each server stamps events with its own clock. Drag B's skew and rebuild the "sort by timestamp" feed.
feed will render here
Nothing exotic happened: a reply that provably came second (it was caused by the question) carried the smaller timestamp. Now recall Lesson 2 of the replication course — last-writer-wins conflict resolution compares exactly these timestamps across exactly such servers. Clock skew doesn't just disorder a comment feed; it silently decides which committed write survives. Google's Spanner attacks the problem with atomic clocks and GPS receivers (TrueTime exposes the uncertainty interval and waits it out); everyone without a hardware budget uses the fix below instead.
Lamport's move: stop measuring, start counting
Leslie Lamport's 1978 insight: for consistency, we don't need when events happened — we need an order that respects causality: if event a could have influenced event b (same process, or a sent a message b received), then a must order before b. That relation — happened-before — needs no clock at all. Just counters and two rules: increment on every local event; on receiving a message, set your counter to max(local, received) + 1.
Drive three processes by hand
Fire local events and send messages between the processes. Watch the max(local, received)+1 rule drag lagging counters forward, so every message lands "after" its send.
The guarantee you just built: if a happened-before b, then L(a) < L(b) — every causal chain climbs. But run the arrow the other way and it breaks: L(a) < L(b) does not mean a caused b. Two events on processes that never exchanged messages can carry any counters at all; Lamport timestamps order them arbitrarily and confidently. They give you a consistent total order (great for tie-breaking, log merging, fairness) but cannot answer the question replication conflict-detection desperately needs: were these two writes concurrent?
Vector clocks: counters that remember who knew what
Give each of n processes a whole vector of n counters — its best knowledge of everyone's progress. Increment your own slot on local events; on receive, take the element-wise max and increment your slot. Now comparison is information-rich: if every slot of V(a) ≤ V(b), then a happened-before b. If each vector beats the other in some slot — concurrent, provably, no clock consulted.
The concurrency detector
Two replicas of a user profile each carry a vector clock. Apply writes on either side, sync them, and before each sync the detector rules: causal update or true conflict?
replica X
[X:0, Y:0]
name="Alex"
replica Y
[X:0, Y:0]
name="Alex"
Where you've already met all three: wall-clock timestamps drive LWW in DynamoDB and Cassandra (with exactly the risk from the first lab); Lamport-style logical counters order writes inside Raft's log and version chains; vector clocks (and their leaner cousins, version vectors and dotted version vectors) are how Riak and Dynamo-lineage stores detect the concurrent writes whose resolution the replication course's multi-leader lesson agonized over. The ladder: physical time for humans, Lamport time for order, vector time for causality.
Check yourself
1. Perfect NTP sync (0ms skew) across the fleet. Does "sort by timestamp" now equal causal order?
Even with magically perfect clocks, a timestamp records when, not because-of-what. Two independent writes one microsecond apart get a confident but meaningless order — which is fine for a feed and disastrous for conflict resolution. Only tracking message flow (Lamport/vector machinery) captures causality. In practice, of course, clocks are also never perfectly synced.
2. Two events have Lamport timestamps L(a)=4 and L(b)=9. What do we know?
The implication runs one way: causality forces L(a)<L(b), so L(b)>L(a) merely fails to rule causality out. This one-wayness is the entire reason vector clocks exist — they make the implication run both directions at the cost of O(n) counters per event.
3. Replica X carries [X:3, Y:1], replica Y carries [X:2, Y:4]. The verdict?
Happened-before requires every slot of one vector ≤ the other's. A split verdict is the mathematical definition of concurrency: each replica saw writes the other hadn't. Downstream, this is precisely the signal that triggers siblings/merge logic in Dynamo-style stores rather than a silent overwrite.