2.2The Problem: How Incoherence Could Possibly Occur

One root cause

The possibility of incoherence incoherence Accessing a stale copy of a datum after another actor has written the original. defined in Chapter 1 — open in glossary arises from exactly one fundamental issue: there exist multiple actors with access to caches and memory. In modern systems those actors are processor cores, DMA engines, and external devices that can read and/or write caches and memory. This primer focuses on cores as the actors, but keep the others in mind — they will matter (coherent DMA returns in Chapter 9).

No second ingredient is required. No reordering, no speculation, no weak memory model — §1.1’s subtleties are a different problem. Perfectly simple, in-order cores are enough to produce incoherence, as the canonical example shows.

The canonical example (Table 2.1)

Initially, memory location A holds 42 — in memory and in both cores’ private caches:

At time 1, C1 changes A from 42 to 43 in its own cache — remember from §2.1 that the private caches are write-back, so the new value goes no further. C2’s cached copy of A is now stale, and C2’s loop keeps loading that stale 42 from its own cache, hit after hit, forever. C1’s store has simply never been made visible to C2. Step through it:

The (only) way out

To prevent incoherence, the system must implement a cache coherence protocol coherence protocol The set of rules implemented by a system's distributed actors (cache and memory controllers) that prevents access to stale data. defined in Chapter 1 — open in glossary that makes the store from C1 visible to C2. The design and implementation of these protocols — how they detect the write, whom they notify, what state they track — is the main topic of chapters 6–9. Everything there is, at heart, a machine for ending exactly this kind of stale-copy standoff.