Necro-reaper: Pruning away Dead Memory Traffic in Warehouse-Scale Computers

Sotiris Apostolakis, Chris Kennelly, Xinliang David Li and Parthasarathy Ranganathan

Inside the giant computers that run web services, the link between processors and memory has become a bottleneck, and this work from Google finds that a surprising share of the data shuttled over it is pure waste — writing freed, dead data back to memory, and fetching old data that is about to be completely overwritten; their system, Necro-reaper, adds a few new processor instructions that let the compiler and memory allocator tell the hardware when data is dead or will be overwritten, so it skips those pointless transfers, cutting memory traffic by about 26 percent and speeding the processors by about 6 percent across ten real workloads, with no special hardware tracking.

The argument in one glance

Problem

Memory bandwidth is a bottleneck, and much traffic is wasted

In warehouse-scale computers, the fleets of machines that run web services, the bandwidth between processors and memory is becoming a critical bottleneck, so data moved needlessly steals speed from real work. This work finds that a surprising share of memory traffic is unnecessary: freed data written back to memory, and old data fetched only to be immediately overwritten.
What people did

Track data liveness in hardware

Prior work reduces such dead traffic by having the hardware itself track which memory is dead or uninitialized, so it can skip the needless transfers. This adds dedicated tracking machinery to the processor.
Why it fell short

Hardware tracking is costly and hard to deploy

Building hardware to track memory liveness adds complexity and area to the processor, and it is a heavy change to justify. Meanwhile the waste is acute in warehouse-scale code, where short-lived heap allocations larger than a cache line are everywhere, generating both needless writebacks of freed data and needless fetches of soon-overwritten data.
This paper's idea

Let software tell the hardware what is dead (Necro-reaper)

Necro-reaper is a software-hardware co-design that avoids dead memory traffic without hardware tracking. New instructions let software tell the processor that a region is dead (do not write it back) or will be fully overwritten (do not fetch it). A profile-guided compiler and a customized memory allocator place these instructions where production traffic is dead. Across ten warehouse-scale workloads it cuts memory traffic by about 26 percent and raises instructions-per-cycle by about 6 percent.

Why this problem is worth solving

The bottleneck has shifted from computing to moving data, so wasted movement is wasted machine.

A processor is fast, but it can only run as fast as memory can feed it data. In modern warehouse-scale computers, the fleets of servers behind web services, the bandwidth of the link between the processors and main memory has become a critical bottleneck: there is more compute available than there is memory bandwidth to keep it fed, so every byte moved needlessly over that link is a byte of useful work not done. Data moves between the processor and memory in fixed-size chunks called cache lines. This work reveals that a surprising fraction of that traffic is dead — it carries data that no one will ever use. Two patterns cause it. The first is needless writebacks of deallocated data. When a program frees a block of memory, the values in it are dead, but the processor’s cache does not know that, so when those cache lines are later evicted, it dutifully writes the dead data back to main memory. The second is needless fetches of uninitialized data. When a program writes into a freshly allocated block, a standard cache first fetches the old contents of each line from memory before overwriting them — even when the program is about to overwrite the entire line, making that fetch pointless. In both cases, memory bandwidth is spent transferring data that is dead on arrival. Removing these transfers frees bandwidth for real work.

The setting is what makes the problem both severe and solvable.

The waste is especially large in warehouse-scale code. Such software constantly allocates buffers on the heap, uses them briefly, and frees them, and these short-lived allocations are frequently larger than a single cache line. Every such allocation is an opportunity for a needless fetch when it is first written, and every deallocation is an opportunity for a needless writeback when its lines are later evicted. Multiplied across a fleet of machines, this dead traffic is a real cost. What makes warehouse-scale computing special, though, is not only the severity of the problem but the tools available to fix it. A warehouse-scale operator controls the whole stack: it can change the hardware, it uses profile-guided compilation, which recompiles programs using measurements of how they actually behave in production, and it deploys customized memory allocators shared across its software. This vertical integration means a fix can span layers that, in a normal setting, belong to different vendors. That is the lever this work uses: rather than making the hardware clever enough to detect dead memory on its own, it lets the software, which already knows when memory is freed or fully overwritten, simply tell the hardware.

What people did before, and where it broke

Standard caches, oblivious to liveness. Treat every dirty line as needing writeback and every write miss as needing a fetch. Where it broke. The cache has no idea which data is dead or about to be overwritten, so it faithfully performs both the needless writebacks and the needless fetches, wasting bandwidth.
Hardware liveness tracking. Add dedicated hardware that tracks which memory is dead or uninitialized, so the processor can skip the dead transfers. Where it broke. This adds real complexity and area to the processor to track liveness, a heavy hardware change that is hard to justify and deploy.
Ignore it, and buy more bandwidth. Accept the waste and provision more memory bandwidth. Where it broke. Memory bandwidth is exactly the scarce, expensive resource that is bottlenecking these machines, so spending more of it on dead data is the opposite of what is needed.
The gap. No approach removed the dead traffic cheaply, without costly hardware tracking, by using the knowledge software already has. Doing that — with a few new instructions plus a profile-guided compiler and allocator that emit them — is what Necro-reaper does.

How Necro-reaper works

From bandwidth wasted on dead data to software-directed skipping

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Recognize that a lot of memory traffic is provably dead, and stop paying for it. The starting insight is that a meaningful fraction of the data crossing the memory bus carries values no one will ever read: the writeback of memory that has been freed, and the fetch of memory that is about to be entirely overwritten. Both are pure waste, and removing them changes no result. Finding the work a system performs that turns out not to be needed, and simply not doing it, is the same instinct the compatibility-measurement work covered elsewhere on this site applies in a different domain, measuring which operations a program truly requires and skipping the large share that can be left out with no effect on the outcome.

Let the layer that knows the truth tell the layer that acts on it. The reason dead traffic persists is that the hardware, which performs the transfers, does not know which data is dead, while the software, which frees and overwrites memory, does. Prior work tried to make the hardware smart enough to detect this itself, at real cost. Necro-reaper instead adds a thin channel — new instructions — so the software can hand the hardware the knowledge it already has, keeping the hardware simple. Pairing a lightweight hardware primitive with software that supplies the intelligence, rather than building complex tracking into the hardware, is the same division the manycore work-stealing work covered elsewhere on this site chooses, adding a small hardware extension and letting software decide when to use it rather than paying for heavyweight hardware machinery.

Never move data that no one will use. The concrete payoff is that the two pointless round-trips to memory simply do not happen: a freed block is dropped rather than written back, and a fully-overwritten block is written without first fetching its stale contents. The system avoids the data movement entirely by knowing, ahead of time, that the moved bytes are not needed. Arranging things so that a needless transfer of data nobody will read is skipped rather than performed is the same win the analytics-storage work covered elsewhere on this site secures, laying out data so a query runs where it lives and avoids shuttling bytes across the system that would only be thrown away.

Following a short-lived allocation through Necro-reaper

Removing the dead traffic of a temporary buffer

A high-level trace of the approach as described in the abstract; the exact instructions and internals are beyond the abstract this page is based on.

Step 1 of 5

A buffer is allocated and soon freed

A service allocates a buffer larger than a cache line, fills it, uses it briefly, and frees it — a pattern the paper finds pervasive in warehouse-scale code.

Being larger than a cache line means whole lines are written and later freed.

Why it works

Memory bandwidth is the bottleneck. Because warehouse-scale processors have more compute than memory bandwidth to feed it → every byte moved needlessly to or from memory steals bandwidth from useful work → so cutting dead traffic directly speeds the machine.
A lot of the traffic is provably dead. Because freeing memory leaves dirty lines that still get written back, and writing new memory first fetches soon-overwritten data → these transfers carry values no one will read → so they can be removed without changing any result.
Short-lived large allocations make it common. Because warehouse-scale code constantly allocates buffers bigger than a cache line, uses them briefly, and frees them → both dead writebacks and needless fetches happen constantly → so the waste is large and worth attacking.
Software knows what hardware cannot cheaply track. Because the compiler and allocator know when a region is freed or fully overwritten → they can tell the hardware directly through a new instruction → so the hardware needs no expensive liveness tracking of its own.
Vertical integration makes it deployable. Because a warehouse operator controls the compiler, allocator, and hardware together → profile-guided compilation can place the instructions exactly where production traffic is dead → so the fix is pragmatic and pays off at scale, about 26 percent less traffic and 6 percent more instructions per cycle.

What it costs (trade-offs and scope)

It requires changes across the whole stack. Necro-reaper needs new processor instructions, a modified compiler, and an extended allocator. This is feasible for a vertically integrated warehouse-scale operator, but not something an ordinary user can adopt piecemeal.
The compiler must correctly identify dead and overwritten regions. The instructions are only safe where a region truly is dead or truly will be fully overwritten. The analysis and profiling must be accurate, because telling the hardware to drop data that is actually still needed would corrupt results.
It depends on profiles and workload patterns. The gains come from the prevalence of short-lived, larger-than-a-cache-line allocations, identified from production profiles. Workloads with different memory behavior, or without good profiles, would benefit less.
The benefit is bounded by how much traffic is truly dead. Necro-reaper reclaims the specific dead traffic it targets; it does not reduce necessary memory traffic, so its ceiling is the amount of waste present, which the paper measures as substantial but finite.
New instructions must be supported by the hardware. The approach assumes processors that implement the new instructions. It avoids hardware liveness tracking, but it is not free of hardware change.
This page is written from the abstract. The full text was not available when this explainer was prepared, so the exact new instructions, the compiler and allocator internals, and the detailed evaluation are summarized only at the level the abstract states, without invented specifics.

The evaluation

Three ways to handle dead memory traffic

Property	Standard caching	Hardware liveness tracking
Writes back freed, dead data		Avoided
Fetches soon-overwritten data		Avoided
Needs hardware liveness tracking	Not applicable
How it learns what is dead	It does not	Hardware tracks it
Memory traffic reduction	Baseline	Some, at hardware cost

Three ways to deal with dead memory traffic in warehouse-scale computers, as described in the paper's abstract. A standard cache is oblivious to liveness, so it faithfully writes back freed dead data and fetches lines it is about to fully overwrite, wasting scarce memory bandwidth. Prior work adds hardware to track liveness and skip the dead transfers, at the cost of processor complexity. Necro-reaper instead adds new instructions and lets software — a profile-guided compiler and a customized memory allocator — tell the hardware when a region is dead or will be fully overwritten, so the hardware skips the pointless transfers without tracking liveness itself. Across a diverse set of ten warehouse-scale workloads, Necro-reaper achieves a geometric-mean memory traffic reduction of about 26 percent and a geometric-mean instructions-per-cycle increase of about 6 percent. These figures are from the abstract; the full paper was not available for this explainer.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer is written from the paper’s abstract, because the full text was not available when it was prepared, so it stays at the level the abstract supports and adds only standard, well-known background about caches and memory. The core finding is striking and clear: in warehouse-scale computers, where memory bandwidth is a bottleneck, a surprising share of the traffic is dead — the writeback of freed data and the fetch of data about to be fully overwritten — and this is especially common because such software constantly allocates and frees short-lived buffers larger than a cache line. Necro-reaper’s response is a pragmatic software-hardware co-design: rather than building costly hardware to detect dead memory, it adds a few new instructions and lets the software that already knows the truth — a profile-guided compiler and a customized allocator — tell the hardware what to skip. The reported results are strong, a geometric-mean 26 percent cut in memory traffic and a 6 percent gain in instructions-per-cycle across ten workloads, and they lean on exactly the vertical integration a warehouse-scale operator has. Treat the specifics — the exact instructions, how the compiler and allocator identify and mark dead regions, and the full evaluation — as beyond what this summary can responsibly claim. The durable idea is that when one layer of the stack knows something valuable that another layer needs, a thin interface to pass that knowledge can beat building expensive machinery to rediscover it.

Background

Why passing knowledge across layers beats rediscovering it

It helps to place Necro-reaper in a broader pattern: much of the waste in a computer comes from one layer of the stack not knowing what another layer already knows, and the cheapest fix is often to pass that knowledge across the boundary rather than to have the ignorant layer reconstruct it. Here the software knows which memory is dead or about to be overwritten, and the hardware does not; a few instructions carry that knowledge across, and a large amount of pointless data movement disappears. The cooperative-scheduling work covered elsewhere on this site makes the same kind of argument in the virtualization setting, where a guest operating system schedules badly because it cannot see state the hypervisor holds, and exposing that state through a light channel unlocks large gains. In both, the enabling condition is the ability to co-design across a boundary that is usually opaque — and warehouse-scale computing, where one operator controls compiler, allocator, and hardware, is an unusually good place to do it. The general lesson is that the interfaces between layers are not sacred; when they hide information that would let a lower layer avoid useless work, widening them by a little — a new instruction, a shared table, a profile — can reclaim efficiency that neither layer could find alone.

If you remember only three things

1. In warehouse-scale computers, memory bandwidth is a critical bottleneck, and this work finds that a surprising fraction of the memory traffic is dead — it moves data nobody will use. Two patterns cause it: writing freed, deallocated data back to memory (no one will read it again), and fetching a cache line's old contents just before the program fully overwrites it. The waste is acute in warehouse-scale software, which constantly allocates short-lived heap buffers larger than a cache line, creating a needless fetch on each write and a needless writeback on each free.
2. Necro-reaper is a software-hardware co-design that removes this dead traffic without the hardware liveness tracking prior work needed. It adds new processor instructions that let software declare a region dead (do not write it back) or about to be fully overwritten (do not fetch it). A profile-guided compiler, using measurements of production behavior, emits these instructions where the opportunities are, and a customized memory allocator marks freed and freshly allocated regions. The hardware, told what is dead, simply skips the pointless transfers — relying on the vertical integration a warehouse-scale operator has over compiler, allocator, and hardware.
3. Across a diverse set of ten warehouse-scale workloads, Necro-reaper reports a geometric-mean memory traffic reduction of about 26 percent and a geometric-mean instructions-per-cycle increase of about 6 percent. Its scope: it requires changes across the whole stack (instructions, compiler, allocator), the analysis must correctly identify dead and overwritten regions or risk corrupting data, the gains depend on workload patterns and good profiles, and the benefit is bounded by how much traffic is truly dead. Note: this explainer is written from the abstract, as the full text was not available, so the exact instructions and evaluation details are summarized only at the level the abstract states.

Check yourself

Check your understanding

1. What is "dead memory traffic," and why does it matter in warehouse-scale computers?
2. How does Necro-reaper avoid dead traffic differently from prior work?
3. Why is a warehouse-scale computing environment particularly suited to this approach?
4. What is the key correctness risk in telling the hardware to skip a transfer, and why does it matter?

Sources

The paper is “Necro-reaper: Pruning away Dead Memory Traffic in Warehouse-Scale Computers” by Sotiris Apostolakis, Chris Kennelly, Xinliang David Li, and Parthasarathy Ranganathan, published at ASPLOS 2025, DOI 10.1145/3676641.3716007. All authors are at Google.

This explainer is written from the paper’s abstract, because the full text was not available at the time of writing: although the paper is open access under a Creative Commons Attribution license, the publisher’s copy was behind automated-access protection, no preprint was available, and no separately hosted copy could be retrieved. The problem statement (memory bandwidth as a bottleneck in warehouse-scale computing, and the finding that a significant portion of memory traffic is unnecessary, consisting of needless writebacks of deallocated data and fetches of uninitialized data, especially where short-lived heap allocations larger than a cache line are prevalent) and the description of the approach (a pragmatic software-hardware co-design named Necro-reaper that avoids dead memory traffic without the hardware tracking of prior work, using new instruction-set instructions to let hardware skip dead traffic and a profile-guided compiler and customized memory allocator to optimize their use) come directly from the abstract, as do the results (evaluation across ten diverse warehouse-scale workloads, a geometric-mean memory traffic reduction of about 26 percent, and a geometric-mean instructions-per-cycle increase of about 6 percent). The surrounding explanation of caches, cache lines, writebacks, write-allocation fetches, and why memory bandwidth bounds warehouse-scale machines is standard, widely-known background provided to make the abstract’s claims understandable, and is not specific to this paper. The exact new instructions, the compiler and allocator internals, and the detailed evaluation are not reproduced here because they were not available.