Skip It: Take Control of Your Cache!

Programs must sometimes force freshly-modified data out of the processor's cache and into main memory — to survive a crash, or to hand data to a device — but these cache-writeback operations are slow, are baked into secret undocumented hardware researchers cannot experiment with, and are frequently wasted on data that has not actually changed; this work from ETH Zurich adds new writeback instructions to an open-source RISC-V processor and builds Skip It, which lets the hardware notice when a line is unchanged and skip the pointless write, running in about 100 cycles and matching or beating the software tricks used today.

The argument in one glance

Problem

Forcing data out of the cache is essential, costly, and often wasted

To make data durable before a crash, or to hand it to a device, a program must force a modified cache line out to main memory with a cache-writeback operation. These are fundamental to correctness and performance, but they are slow, they live in undocumented proprietary hardware that researchers cannot experiment with, and they are frequently issued on lines that have not actually changed — pure waste.
What people did

Use the writeback instructions the hardware provides

Processors expose cache-management instructions that write a dirty line back to memory, and programmers use them carefully wherever durability or coherence demands. Software techniques try to avoid issuing the ones that are unnecessary.
Why it fell short

They are slow, opaque, and redundantly issued

Each writeback still costs many cycles even when used carefully. The mechanisms are proprietary and undocumented, so studying or improving them is limited to slow software simulation rather than real hardware. And a program often writes back a line that was not modified since its last writeback, wasting cycles and memory bandwidth on data that is already in memory.
This paper's idea

Add open writeback instructions, and Skip It (Skip It)

This work adds microarchitectural extensions to the open-source BOOM RISC-V processor that give software two types of user-controlled cache writeback, so the mechanism can finally be studied on real hardware. On top of them, Skip It tracks whether a line has actually changed and skips the writeback when it has not, removing redundant writes. The operations run in about 100 cycles — competitive with commercial server platforms — and Skip It matches or beats state-of-the-art software techniques.

Why this problem is worth solving

Controlling what sits in the cache is a routine necessity, and each such control operation is expensive.

A processor keeps recently-used data in caches, small fast memories close to the compute, and it writes changes there first for speed. A cache line that has been modified but whose new value has not yet reached main memory is called dirty. Normally the hardware writes dirty data back to memory on its own schedule, but many situations require a program to force it out at a specific moment. The clearest case is persistent memory: to guarantee that an update will survive a power failure, the program must ensure the modified data has actually reached the durable main memory before it declares the update complete, which means explicitly writing back the dirty cache line. Another case is handing data to a device that reads memory directly, where the program must push its writes out of the cache so the device sees them. Processors provide cache-management instructions for exactly this — writing a dirty line back to memory on demand. These operations are fundamental to both correctness and performance, and systems that rely on persistence or on device interaction issue them constantly. The trouble is that each one is costly, taking many processor cycles, so their frequent use imposes a real performance tax that careful programming only partly relieves.

Beyond being slow, these mechanisms are hidden and frequently used for nothing.

Two further problems compound the cost. First, the mechanisms are opaque. Cache management is implemented deep in the processor, in proprietary hardware whose exact behavior vendors do not document. That means researchers who want to understand these operations, optimize them, or propose new designs cannot easily experiment on real hardware — they are largely limited to slow, simplified software simulations that may not reflect how real silicon behaves. Progress on better cache-management mechanisms is throttled by this lack of an open, real platform. Second, and just as importantly, a great many of these writebacks are redundant. A program that must guarantee durability will issue a writeback for a cache line, but very often that line has not been modified since the last time it was written back — the program is being cautious, or the code path repeats, so it asks the hardware to push out data that is already sitting in main memory. That writeback achieves nothing, yet it still pays the full cost in cycles and memory bandwidth. Software techniques exist to track which writebacks are truly needed and skip the rest, but they add their own bookkeeping overhead. So the operation is not only expensive per use, it is used far more than necessary.

What people did before, and where it broke

Issue a writeback wherever durability or coherence might require it. Conservatively push out cache lines whenever correctness could depend on it. Where it broke. Many of these writebacks are redundant, on lines that have not changed since the last writeback, so they waste cycles and memory bandwidth pushing out data already in memory.
Track needed writebacks in software. Have the program keep track of which lines are actually dirty and only write those back. Where it broke. This avoids some redundant writebacks, but it adds software bookkeeping overhead, so it trades one cost for another.
Study cache management in software simulation. Research new writeback designs on simulators, since real hardware is proprietary. Where it broke. Simulations are slow and simplified, and may not reflect real silicon, so proposed optimizations are hard to validate and progress is limited.
The gap. No open, real hardware exposed user-controlled writebacks and eliminated the redundant ones cheaply, in the microarchitecture itself. Providing exactly that on an open-source processor is what this work does.

How Skip It works

From blind, opaque, redundant writebacks to open, hardware-skipped ones

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Give software direct, explicit control of writebacks, on open hardware. The first move is to expose cache writebacks as first-class, user-controlled operations, built into an open processor rather than hidden in proprietary silicon. This gives programs precise control over when data is pushed to memory, and gives researchers a real platform to measure and improve the mechanism instead of guessing from simulation. Exposing fine-grained control over a hardware mechanism to software, so that the layer that knows what it needs can direct the hardware directly, is the same move the userspace-storage work covered elsewhere on this site makes, giving software direct control of the data path rather than leaving it buried in a fixed, opaque layer.

Skip the writeback when there is nothing to write. The heart of Skip It is the observation that a large share of writebacks are redundant — the line has not changed since it was last written back, so pushing it out again moves data already in memory. Skip It has the hardware track whether a line is actually dirty and simply skip the memory write when it is clean, so a needless operation becomes almost free. Recognizing that an expensive operation is often unnecessary and refusing to perform it when nothing has changed is the same principle the warehouse-scale memory-traffic work covered elsewhere on this site is built on, eliminating the needless writebacks of data that no longer matters so that scarce memory bandwidth is not spent moving it.

Let a cheap hardware bit decide, instead of expensive software bookkeeping. Software can also avoid redundant writebacks, but only by tracking dirtiness itself, which costs the program overhead. Skip It instead relies on the fact that the cache already knows whether a line is dirty, so a tiny amount of hardware state can make the skip-or-write decision precisely and cheaply, with no software tracking. Letting a small, accurate hardware flag decide whether the expensive operation is needed, rather than paying to track the condition in software, is the same trade the copy-on-write correctness work covered elsewhere on this site makes, using a single hardware flag to decide whether a page must be copied instead of maintaining costly bookkeeping about it.

Following a persistent update through Skip It

Making a record durable without redundant writebacks

A high-level trace of the approach as described in the abstract; the exact microarchitecture and the two writeback types are beyond the abstract this page is based on.

Step 1 of 5

Update a record in persistent memory

A program updates a record that must survive a crash, so before declaring the update done it must ensure the modified cache line has actually reached durable main memory.

Durability requires the dirty data to leave the cache.

Why it works

Cache management is essential but costly. Because forcing data out of the cache is required for durability and device coherence, yet each such operation costs many cycles → programs that use it heavily pay a large tax → so making these operations cheaper directly speeds real systems.
Many writebacks are redundant. Because a program often writes back a line that has not changed since its last writeback → that operation moves data that is already in memory → so it is pure waste that can be removed.
Hardware can tell what software only guesses. Because the cache already knows whether a line is dirty → the hardware can decide precisely whether a writeback would move new data → so it skips the redundant ones without the overhead software spends tracking dirtiness itself.
Open hardware makes the design real. Because the mechanism is built into the open BOOM RISC-V core rather than simulated → its cost is measured on real hardware at about 100 cycles → so the design is validated where proprietary hardware left researchers guessing.
It matches or beats software at lower cost. Because Skip It removes redundant writebacks in hardware while software techniques track dirtiness in the program → it achieves the same or better savings with less overhead → so it is competitive with the state of the art and with commercial platforms.

What it costs (trade-offs and scope)

It requires hardware changes. Skip It is a microarchitectural extension to the processor, not a software update. Its benefits require building the mechanism into the core, which is feasible on an open processor but is a hardware change.
The benefit tracks how redundant the writebacks are. Skip It saves work by eliminating writebacks on unchanged lines. A workload whose writebacks are almost always on genuinely dirty lines has fewer redundant ones to skip, so it gains less.
It targets writebacks specifically. The work addresses forcing modified data out to memory. Other cache-management operations, and the broader costs of persistence or device coherence, are separate concerns.
Correctness still rests on issuing writebacks where needed. Skip It only removes writebacks that are provably redundant because the line is unchanged; a program must still issue writebacks wherever durability or coherence requires, and the skipping must be exactly right so no needed write is dropped.
The results are on one open core. The reported roughly hundred-cycle cost and the comparison to software techniques come from an implementation on the open BOOM RISC-V processor; other cores and memory systems could shift the figures.
This page is written from the abstract. The full text was not available when this explainer was prepared, so the two specific writeback types, the exact tracking mechanism, and the full evaluation are summarized only at the level the abstract states, without invented specifics.

The evaluation

Three ways to handle a redundant cache writeback

Property	Blind writeback	Software dirtiness tracking
Skips redundant writebacks		Yes
Where the decision is made	Not applicable
Overhead of deciding	None, but wasteful
Open to study on real hardware		Not applicable
Operation cost	Full writeback	Depends on tracking

Three ways to deal with a cache-writeback operation that may be redundant, as described in the paper's abstract. A blind writeback always pushes the line to memory, wasting cycles when the line is unchanged. Software techniques track which lines are actually dirty and skip the rest, but add their own bookkeeping overhead. Skip It, built as a microarchitectural extension on the open-source BOOM RISC-V processor, exposes two types of user-controlled writeback and has the hardware track whether a line changed since its last writeback, skipping the memory write when it did not. The reported operation cost is about 100 cycles, which favorably compares to similar operations on commercial server-class platforms, and Skip It removes redundant writebacks as well as or better than state-of-the-art software techniques, while also giving researchers a real, open platform to study these mechanisms. These figures are from the abstract; the full paper was not available for this explainer.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

Skip It is a practical hardware contribution aimed at a real and under-served pain: the operations that force data out of the cache — needed for persistence and for talking to devices — are costly, are locked inside proprietary undocumented hardware that researchers cannot experiment with, and are frequently issued when they are not needed. This work addresses all three at once. It adds user-controlled writeback support to the open-source BOOM RISC-V processor, giving both programs precise control and researchers a real platform instead of slow simulation. And it builds Skip It, which has the hardware track whether a line actually changed and skip the writeback when it did not, removing redundant writes cheaply using a hardware bit rather than software bookkeeping. The reported results are strong: operations run in about 100 cycles, competitive with commercial servers, and Skip It matches or beats state-of-the-art software techniques for avoiding unnecessary writebacks. Read the specifics with care. The exact two writeback types, how the dirty-tracking is implemented, and the full evaluation are in the paper and not reproduced here. As trade-offs, it is a hardware change rather than a software fix, its benefit depends on how many writebacks are actually redundant, it targets writebacks specifically, and the numbers come from one open core. The durable idea is that cache management deserves to be a first-class, open, and precisely-controlled part of the processor, and that an operation which is often unnecessary should be able to recognize that and cost almost nothing.

Background

Why eliminating unnecessary memory writes keeps paying off

It helps to see Skip It within a recurring theme: a surprising amount of what a computer moves to and from memory is unnecessary, and eliminating it — ideally in hardware, ideally for free — is a repeatable win. Here the unnecessary work is writing back a cache line that has not changed. The warehouse-scale memory-traffic work covered elsewhere on this site pursues the same goal from a related angle, cutting the needless writebacks of dead data and the fetches of data about to be overwritten, so scarce memory bandwidth is spent only on transfers that matter. In both, the insight is that the hardware, or a thin cooperation between software and hardware, can know when a memory operation is pointless and simply not perform it. A second thread in Skip It is just as important: it does this on an open-source processor, so the mechanism can be measured and improved on real silicon rather than in simulation — part of a broader shift in which open RISC-V cores let researchers study and change microarchitecture that used to be sealed inside proprietary designs. The general lesson is that memory movement is where much of a system’s time and energy goes, that a large fraction of it is avoidable, and that giving hardware the small amount of state it needs to skip the avoidable part — on platforms open enough to build and test — is a durable source of efficiency.

If you remember only three things

1. Systems constantly need to force modified data out of the processor's cache into main memory — a cache writeback — to make data durable before a crash or to hand it to a device. These operations are fundamental to correctness and performance but have three problems: each one is slow (many cycles), they live in proprietary undocumented hardware so researchers can only study them in slow software simulation, and they are frequently redundant because a program writes back a line that has not been modified since its last writeback, moving data already in memory for no benefit.
2. This work adds microarchitectural extensions to the open-source BOOM out-of-order RISC-V processor that provide two types of user-controlled cache writeback, giving software precise control and giving researchers a real hardware platform. On top of them it builds Skip It, which has the hardware track whether a line has actually changed since its last writeback and skip the memory write entirely when the line is unchanged — removing redundant writebacks using a cheap hardware bit rather than the software bookkeeping that prior techniques required.
3. The operations run in about 100 cycles, which the paper reports favorably compares to similar operations on commercial server-class platforms, and Skip It performs as well as or better than state-of-the-art software techniques for avoiding unnecessary writebacks. Its scope: it is a hardware change rather than a software fix, its benefit depends on how many writebacks are actually redundant, it targets writebacks specifically, and the numbers come from one open core. Note: this explainer is written from the abstract, as the full text was not available, so the two writeback types, the exact tracking mechanism, and the full evaluation are summarized only at the level the abstract states.

Check yourself

Check your understanding

1. Why must a program sometimes explicitly force a cache line back to main memory?
2. What does Skip It do, and why is that a win?
3. Why did the authors implement this on the open-source BOOM RISC-V processor rather than in simulation?
4. A workload issues writebacks that are almost always on genuinely modified (dirty) lines. What does Skip It's design imply for it?

Sources

The paper is “Skip It: Take Control of Your Cache!” by Shashank Anand, Michal Friedman, Michael Giardino, and Gustavo Alonso, published at ASPLOS 2024, DOI 10.1145/3620665.3640407. The authors are at ETH Zurich.

This explainer is written from the paper’s abstract, because the full text was not available at the time of writing: the publisher’s copy was behind automated-access protection, the paper’s open-access status was closed, and no preprint or separately hosted author copy could be retrieved. The problem statement (that mechanisms to explicitly manage the presence of data in caches are fundamental for correctness and performance, that these operations incur significant performance penalties even when carefully used, and that they are implemented in proprietary and often undocumented hardware, limiting research to slow, simplified software simulations) and the description of the approach (microarchitectural extensions supporting two types of user-controlled cache writebacks to main memory, and Skip It, a mechanism built on top that substantially reduces redundant writebacks, implemented on the open-source BOOM out-of-order RISC-V CPU) come directly from the abstract, as do the results (a hardware performance of about 100 cycles that favorably compares to similar operations on commercially available server-class platforms, and Skip It performing as well as or better than state-of-the-art software techniques for avoiding unnecessary writebacks). The surrounding explanation of caches, dirty cache lines, writebacks, persistent memory, device coherence, and open-source RISC-V cores is standard, widely-known background provided to make the abstract’s claims understandable, and is not specific to this paper. The two specific writeback types, the exact dirty-tracking mechanism, and the detailed evaluation are not reproduced here because they were not available.