Pinned loads: taming speculative loads in secure processors

Defences against Spectre-style attacks work by making a load wait until it can no longer be cancelled, and the condition that keeps a load cancellable longest turns out not to be branches but memory consistency — a load can be thrown out at any moment because another core wrote to the same cache line; Pinned Loads (Illinois and Tel Aviv) makes that impossible in advance by having the core refuse incoming invalidations and refuse to evict the line, so the load becomes uncancellable far earlier and every defence built on top gets roughly twice as cheap.

Why this problem is worth solving

Every defence in this area is organised around one moment.

The last of those conditions is enforced by a mechanism most people never think about.

What people did before, and where it broke

• Put a fence before every speculative load. The simplest defence, and completely effective: the load cannot execute until the fence lets it through. Where it broke. It costs 113 percent on average under the comprehensive threat model, roughly doubling execution time.
• Stall only speculative loads that miss in the first-level cache. A cheaper idea: a hit disturbs nothing an attacker can observe from outside, so only misses need holding back. Where it broke. Overhead falls to 36 percent, which is much better and still substantial, and it is concentrated in exactly the applications whose working sets do not fit.
• Stall only loads whose inputs came from speculatively read data. The most refined of the three: track which values are tainted by speculation and protect only the loads that consume them. Where it broke. Overhead falls to 25 percent, the best of the three, and the remaining cost is still large enough to matter for a mechanism that must always be on.
• Execute speculative loads invisibly, then repeat them later. Rather than stalling, run the load in a way that leaves no observable trace, and issue it properly once it is safe. Where it broke. The load is performed twice, so the work is real even though the stall is avoided, and the underlying question of when the load becomes safe is unchanged.
• Allow load reordering without squashes by blocking writers in the directory. A prior design achieves a similar goal by having the directory buffer a rejected write, hand out single-use uncacheable copies to other readers, and refuse to evict the blocked entry. Where it broke. It requires a new transient directory state, writes that switch between three-hop and four-hop transactions, reads that become uncacheable depending on directory state, and reads that bypass directory allocation entirely — parallel cached and uncached paths that are difficult to verify.
• The gap. Every defence accepts the visibility point as a given and competes on what to do before it arrives. Nobody had asked which of the four conditions actually delays it, or whether that condition could be removed rather than waited out.

How a load stops being able to be squashed

The idea

Measuring which safety condition actually costs is the move that makes the rest obvious. Under a threat model covering every source of cancellation, a load must clear four hurdles before it is safe, and the natural assumption is that branch resolution dominates, since branch misprediction is what the original attacks exploited. The experiment that opens this paper removes the protective fence at four progressively later points and compares the resulting overhead, and the answer is that waiting for the guarantee of no memory consistency violation costs more than the other three conditions together. That is not obvious, and it reframes the problem entirely: the expensive part of being safe against speculative attacks turns out to be a mechanism that exists for an unrelated reason and whose cost is normally invisible. Attacking it makes every defence cheaper at once, which is why the same technique halves the overhead of three schemes that otherwise have nothing in common. Finding that the thing genuinely holding up progress is not the thing everyone was optimising is the same discovery the critical-slice prefetching work covered elsewhere on this site makes about instruction scheduling, where the load actually on the critical path waits politely behind work that does not matter.

A conservative detector can be turned into an active guarantee by refusing the events it detects. The hardware does not know whether a reordering actually violated the ordering rules, so it assumes the worst: if a line a pending load has read gets invalidated or evicted, throw the load out. That is a detection mechanism, and every defence has been waiting for the window in which detection is possible to close. The reframing here is that both triggers are events the core participates in rather than merely observes. An invalidation is a request from another core that this core answers; an eviction is a decision this cache makes. So answer no to both. Decline the invalidation and tell the writer to retry; refuse the eviction and pick a different victim, updating the replacement state so the line is not chosen again soon. The load is now provably uncancellable, which is exactly what the visibility point means, and it reaches that point without waiting for anything. Changing what the defence acts upon rather than how hard it acts is the same shift the row-hammer mitigation work covered elsewhere on this site makes when it stops refreshing the victims of an attack and starts relocating the aggressor.

Making a promise costs resources, and most of the design is the machinery that keeps the promise keepable. Declining an invalidation is only sustainable until the load retires, so the core must be certain the load can retire, which means every older store needs somewhere to go — and if the write buffer is full while another core’s write is being blocked, two cores can deadlock waiting for each other, a case the paper constructs explicitly. So a load is only pinned when the count of older incomplete stores fits the write buffer. Refusing evictions has a parallel requirement: all pinned lines must actually fit, so their number per set cannot exceed the first-level associativity, and each core is allotted a reserved share of ways in the shared cache and directory. And refusing writes creates starvation, since each reader decides locally and none of them can see a writer being denied over and over — fixed by having a retried write carry a marker that makes every sharer record the line as temporarily unpinnable, so the readers drain and the write goes through. Establishing a window in which a hazard is impossible, and finding that the real work is proving nothing stands in the way, is the same shape as the live kernel update work covered elsewhere on this site, where halting the machine is easy and demonstrating that no thread is inside the code being replaced is the contribution.

Following one load from address to pin

Without this check two cores can deadlock: each has a pinned load blocking the other core write, and neither load can retire because an older store cannot leave a full write buffer.

Why it works

• The costly condition is the one nobody was looking at. Under a comprehensive threat model a load must clear branches, aliasing, exceptions and consistency before it is safe → and measuring each separately shows consistency costs more than the other three together, because it is unresolved until the load nears the head of the reorder buffer → so attacking it alone recovers about half the overhead of every defence, whatever that defence does.
• Both triggers of the consistency hazard are things the core answers, not observes. A load is squashed when its line is invalidated by another core or evicted from the cache → and an invalidation is a request this core replies to while an eviction is a choice this cache makes → so both can simply be declined, turning a hazard the defence was waiting out into one that cannot occur.
• Being uncancellable is exactly what the visibility point means. Defences do not require a load to be complete, only to be beyond cancellation → so a load whose line cannot be invalidated or evicted has met the condition by construction rather than by elapsed time → and the defence stops protecting it immediately.
• Every load that reaches the point releases the next one. The conditions for a younger load depend on older loads having reached their own points → so pinning compounds down the instruction stream rather than helping one load at a time → which is why the aggressive design reaches performance close to the much weaker threat model on several benchmarks.
• Checking resources in advance is what allows pinning before the data arrives. If pinning requires seeing the line actually arrive in the cache, then every load waits for memory before releasing the next, and independent loads serialise → but a small table tracking where pinned lines map can answer the space question ahead of time → letting loads issue in parallel, which is worth 15 points of overhead on the fence-based defence.

What it costs (trade-offs and scope)

• A substantial gap to the weaker threat model remains. The best configuration brings the fence-based defence from 113 percent overhead to 51, against 35 for the same defence under the narrow threat model. That 15-point absolute gap is unrecovered consistency overhead, and the paper reports it directly.
• The simpler of the two designs barely helps one of the defences. Late pinning takes the miss-stalling defence from 36 percent to 32, because that scheme’s cost comes from cache misses and late pinning cannot pin until the data returns. The headline halving is really about the design that needs the extra table.
• Dependent loads defeat the aggressive design structurally. If a load’s address comes from an earlier load’s value, it cannot be pinned until that value arrives and the address translates, and because pinning is strictly in order everything younger stalls behind it. One benchmark remains far above the weaker model for exactly this reason.
• It changes the coherence protocol so that a core can refuse another core’s write. That requires new message types and a starvation-prevention table, and the paper identifies a corner case — a writer arriving repeatedly while the readers’ tables are full — whose fix is described as a more advanced design with a queue, and is not built.
• Deadlock is genuinely possible and prevented by a conservative check. The paper constructs a two-core case where each pinned load blocks the other core’s write while an older store cannot leave a full write buffer. The remedy restricts when pinning may happen.
• Ways in the shared cache and directory are reserved per core. Two per set and slice by default; reducing to one costs up to 3.4 points of overhead. This is a static agreement among cores, and its behaviour with many cores or with asymmetric workloads is not examined.
• The analysis is specific to x86 ordering. The whole argument rests on how that model conservatively detects load reordering. Architectures with weaker ordering squash under different conditions, and neither the measurement nor the mechanism transfers directly.
• Everything is simulated, in a cycle-level model with area and energy for the added table estimated from a modelling tool rather than synthesised.
• The comparison to the closest prior mechanism is qualitative. That design is argued to be too complex to verify — new transient states, uncached paths, transactions that skip directory allocation — rather than implemented and measured against.
• Only loads are treated as leaking instructions. The paper argues they are the most important, and other instructions whose timing or resource use depends on operands are out of scope.
• A new observable is created and argued away rather than tested. Whether a write is deferred now depends on whether another core holds a pinned load. The argument that this is safe is careful — a pinned load is guaranteed to retire, so its effects reveal nothing speculative — and non-speculative channels, including coherence-state timing, are explicitly out of scope.

The evaluation

Check yourself

Sources

The paper is “Pinned Loads: Taming Speculative Loads in Secure Processors” by Zirui Neil Zhao, Houxiang Ji, Adam Morrison, Darko Marinov and Josep Torrellas, published at ASPLOS 2022, pages 314-328, DOI 10.1145/3503222.3507724. The authors are at the University of Illinois Urbana-Champaign and Tel Aviv University.

This explainer is written from the full 15-page text. Everything above comes from the paper’s own sections: the background on transient instructions and leaking instructions, on the visibility point and the conditions defining it under two threat models, and on how x86 ordering conservatively detects load reordering by squashing on invalidation or eviction; the motivating measurement that separates the four conditions and finds the consistency condition dominant; the worked comparison of how loads overlap in the reorder buffer under an unprotected processor, a protected one, and each of the two designs, including the case where a dependent load blocks younger independent ones; the threat model and the argument that the technique changes when the visibility point is reached rather than what it means, and adds no new speculative channel; the mechanisms covering the modified write transaction with its refusal and abort messages, the write buffer accounting with its constructed deadlock example, the denial of evictions with replacement-state update, the guarantee of cache and directory space, and the starvation-prevention protocol with its marked invalidations and clearing messages; the implementation aspects covering two ways to record pinned lines and why the load queue is chosen, the shadow table with its hashing, stale-entry handling and identifier wraparound, the cannot-pin table, and the behaviour when any structure fills; the qualitative comparison against a prior write-blocking protocol; and the evaluation covering the simulated architecture, the three defence schemes and four configurations, the per-benchmark and average results on both suites, the breakdown of remaining overhead by source, the network traffic measurement, and the sensitivity studies on table sizes and reserved ways.