CPU-Oblivious Offloading of Failure-Atomic Transactions for Disaggregated Memory

Cheng Chen, Chencheng Ye, Yuanchao Xu, Xipeng Shen, Xiaofei Liao, Hai Jin and 2 others

When memory is disaggregated — a shared pool of remote memory reached over a CXL interconnect — making updates crash-consistent means waiting for a log to reach the far-away persistent memory, and that wait (hundreds to thousands of CPU cycles) stalls the processor on every transaction; this paper (Huazhong University of Science and Technology, UC Santa Cruz, NC State, UC Central Florida — Cheng Chen, Chencheng Ye, and colleagues) presents Fanmem, which puts a small processing unit inside the CXL switch to take over the slow log-persistence work and pairs it with an asynchronous transaction model, so the CPU acknowledges early and keeps working while durability completes off the critical path — 1.2 to 1.7 times faster than the state of the art, with no changes to the processor.

The argument in one glance

Problem

Disaggregated memory needs crash consistency, but persisting to it is painfully slow

Datacenters are moving toward memory disaggregation: instead of every server owning its own memory, a shared pool of memory sits at the other end of a fast interconnect (CXL) and many compute servers draw from it. This improves memory utilization and cost, but it splits a machine into parts that can fail separately — a compute server or the interconnect can crash mid-update, leaving the shared memory pool in a half-written, inconsistent state. To stay reliable, applications wrap updates in failure-atomic transactions, borrowing the machinery of persistent memory: write a log describing the change, make sure that log is durable, and only then commit. The problem is that in disaggregated memory the persistent memory is far away. Reaching remote CXL-attached memory can take hundreds to thousands of CPU cycles (for example, around 729 nanoseconds median for one memory-semantic SSD). Every transaction has to force its log all the way out to that remote persistent memory before it can commit, and the instruction that waits for this — a persist barrier — stalls the CPU pipeline the whole time. So the very step that makes disaggregated memory reliable also makes it slow: the processor spends much of its time frozen, waiting for a distant write to become durable.
What people did

Speed persistence up in software, or bake transaction logic into the CPU

Two families of fixes exist. Software-only approaches keep improving the logging protocol — undo logs, redo logs, speculative logging — to reduce how much must be persisted and when. Hardware-assisted approaches go further and build transaction and persistence logic directly into the CPU core, so the processor itself can manage logging more efficiently.
Why it fell short

Software still stalls on the fence, and CPU changes are impractical

Neither family fits CXL disaggregated memory well. Even the best software schemes still end up with a persist barrier that must wait for the log to travel all the way to the remote memory server and come back durable; the smarter logging reduces the number of such waits but cannot remove the fundamental latency of reaching far-away persistent memory, so the CPU still stalls. The hardware-assisted approaches avoid some of that stall, but at a steep cost: they require non-trivial modifications to the processor core. Changing the CPU is slow to standardize and deploy, ties the solution to particular processors, and is exactly what a disaggregated, heterogeneous datacenter — full of different CPUs, NICs, and accelerators all sharing memory over CXL — cannot assume. What was missing was a way to take the slow persistence work off the CPU's critical path without modifying the CPU at all: a CPU-oblivious design that fits the disaggregated architecture as it actually is.
This paper's idea

Offload log persistence into the CXL switch, and commit asynchronously

Fanmem is the first failure-atomic transaction system built specifically for CXL disaggregated memory, and its idea is to move the slow persistence work off the CPU and into the network. It has two coupled parts. First, an asynchronous transaction model splits a transaction into execution, offloading, and commit stages: the application writes its log and issues the persist barrier, but that barrier now only has to reach the CXL switch, not the far-off memory server — so the switch acknowledges immediately and the CPU proceeds to other work (even new transactions) while durability finishes in the background. Second, a small processing unit embedded in the CXL switch actually carries out the persistence: it intercepts log writes on their way through the fabric, forwards them to the memory server, and tracks their durability using a per-thread sequential log with a hardware cursor — a single "high-water mark" that says how much of each log is now durable. To commit, the application just reads that cursor and checks whether its last log address is covered, then a timestamp-based protocol commits transactions in execution order so recovery is always consistent. Because the acceleration lives in the switch — a natural interception point for all memory traffic — Fanmem needs no changes to the CPU, and it delivers 1.2 to 1.7 times higher throughput than the state-of-the-art persistent memory transaction system on two representative disaggregated architectures.

Why this problem is worth solving

Disaggregated memory is where large datacenters are heading, and reliability is a precondition for running real applications on it.

Memory disaggregation lets many compute servers share one large, remote memory pool over a fast interconnect like CXL, which fixes a chronic waste problem: memory stranded on machines that are not using it. That makes it attractive for the big, memory-hungry applications datacenters run. But pooling memory across separate components multiplies the ways a system can fail: the compute server can crash, the interconnect or switch can drop, all while an update to the shared pool is only half done. Unlike a single machine that fails as a unit, a disaggregated system can lose the compute side while the memory side survives holding a partially-written structure. If applications cannot recover a consistent state from that, the whole architecture is unusable for anything that must not corrupt data — databases, key-value stores, filesystems. So failure-atomic transactions, which guarantee an update either fully happens or fully does not, are not a luxury here; they are what makes disaggregated memory safe to build on. The catch is that providing them cheaply is hard precisely because the persistent memory is remote.

The reason it is slow is that durability sits on the critical path, and the path to remote memory is long.

A failure-atomic transaction is only safe once its log is truly durable — written all the way to persistent memory — because recovery replays or reverts based on that log. To enforce this, the CPU issues a persist barrier (a fence) that will not let the program continue until the log write is durable. On a local machine this wait is short. In disaggregated memory the persistent memory is at the far end of a CXL interconnect, so a single durable write can take hundreds to thousands of CPU cycles, and the fence stalls the pipeline for that entire time, on every transaction. Clever logging schemes reduce how often you must fence and how much you must write, but they cannot shrink the physical latency of reaching remote memory, so the stall remains the dominant cost. The key realization is that the CPU does not actually need to sit idle during this wait — it only needs a guarantee that persistence will complete and a cheap way to check later whether it has. If the slow durability work could be handed to something else and verified asynchronously, the processor could keep doing useful work instead of freezing. Turning that realization into a correct, deployable system is the problem.

What people did before, and where it broke

Improve the software logging protocol (undo, redo, speculative). Reduce how much data is persisted and how often a fence is needed. Where it broke. However efficient the log, the transaction still ends in a persist barrier that must wait for the log to reach remote persistent memory, so the CPU keeps stalling on that unavoidable latency.
Persist speculatively or asynchronously in software. Overlap some persistence with computation to hide latency. Where it broke. A single fence to confirm durability still remains on the critical path before commit, and in disaggregated memory that one fence is expensive enough to dominate.
Build transaction and persistence logic into the CPU core. Let hardware inside the processor manage logging and durability. Where it broke. This requires intrusive, non-trivial modifications to the CPU, which are slow to standardize, tie the solution to specific processors, and are impractical in a heterogeneous disaggregated datacenter.
Keep memory local to sidestep remote persistence entirely. Avoid disaggregation for reliability-critical workloads. Where it broke. That forfeits the whole point of disaggregation — high memory utilization, on-demand provisioning, and sharing a large pool across many servers — which modern datacenters increasingly need.
The gap. No system provided failure-atomic transactions for CXL disaggregated memory that took persistence off the CPU’s critical path without modifying the CPU.

How Fanmem works

Taking persistence off the CPU by offloading it into the switch

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Put the acceleration logic in the network fabric, not the CPU. Fanmem’s defining choice is where the work runs: instead of touching the processor, it embeds a small processing unit in the CXL switch, the point every memory write already passes through, and lets that in-fabric unit take over persistence. Moving a recurring, latency-critical operation off the endpoints and into the switch that all the traffic flows through anyway is the same undertaking the work on in-network aggregation with transport transparency for distributed training covered elsewhere on this site pursues, exploiting the network element as a natural place to do shared work. Fanmem does this for log persistence, which keeps the design CPU-oblivious and therefore deployable across the heterogeneous processors of a disaggregated datacenter.

Decouple durability from execution so the slow step overlaps useful work. The asynchronous transaction model is Fanmem’s way of getting the persist wait off the critical path: the application does not block on durability, it hands the log off, gets an early acknowledgment, and continues, verifying persistence later. Restructuring a computation so a slow, dependent operation runs in the background while the processor pushes ahead with independent work is the same principle the work on overlapping communication with dependent computation covered elsewhere on this site is built on, converting a stall into overlapped progress. Fanmem overlaps remote persistence with subsequent transactions, and only pays a cheap cursor read when it finally needs to confirm durability.

Guarantee that whatever survives a crash is consistent. Underneath the speedups, Fanmem is a correctness system: its timestamp-based commit protocol and sequential logs exist so that after any failure — of the compute server, the switch, or the memory server — recovery can replay the durable log and restore a state that never violates atomicity. Making crash consistency a first-class, provable property of a persistent-memory design, rather than an afterthought, is the same idea the work on checking crash consistency of non-volatile memory covered elsewhere on this site develops, treating what remains after a failure as something that must be reasoned about carefully. Fanmem builds this in by committing transactions in execution order, so recovery from the last durable commit timestamp always yields a consistent system.

Following a transaction through Fanmem

Committing a failure-atomic transaction without stalling

A trace, drawn from the paper, of how Fanmem runs one transaction across the compute server, the CXL switch, and the memory server.

Step 1 of 5

The application executes and logs its changes

The transaction begins, modifies its data, and generates log records with the new values (speculative logging). It writes those records sequentially into its pre-allocated, contiguous log area in disaggregated memory.

The append-only log means durability can be summarized by a single high-water mark.

Why it works

Offloading to the switch removes remote latency from the fence. The persist barrier normally waits for the log to reach far-off memory → but the in-switch unit acknowledges as soon as the write reaches the switch → so the fence retires quickly and the CPU stops stalling on remote persistence.
Asynchrony overlaps durability with real work. Waiting for durability wastes CPU cycles → but the asynchronous model lets the CPU proceed to other transactions while persistence completes in the background → so the slow step overlaps useful work instead of blocking it.
A sequential log plus a cursor makes checking durability trivial. Tracking many scattered log writes would be complex → but an append-only log means one high-water-mark cursor summarizes durability → so checking persistence is a single load and comparison, not a costly synchronization.
Committing in execution order keeps recovery consistent. Out-of-order durable logs could leave an inconsistent state → but the timestamp protocol commits transactions in the order they executed → so recovery from the last durable commit timestamp always restores a valid state.
Living in the switch keeps the design CPU-oblivious. Baking logic into the CPU ties a solution to specific processors → but the switch is where all memory traffic already passes and needs no core changes → so Fanmem deploys across the heterogeneous hardware of a disaggregated datacenter.

What it costs (trade-offs and scope)

It requires a smarter CXL switch. Fanmem moves the burden from the CPU to the network by adding a processing unit and small tables (a partitioned log-area table and a log-write-status table with cursors) inside the switch; this is hardware that must be built into the fabric, even though it avoids touching the CPU.
It fits log-based schemes best, and undo logging is awkward. Fanmem naturally accelerates redo and speculative logging, which already tolerate asynchronous persistence, but undo logging demands a strict order between a data write and its log write, which would require more complex dependency-tracking logic in the switch.
Commit is delayed, not instantaneous. Because durability is verified asynchronously, a transaction commits only after its log is confirmed persistent and its ordered dependencies commit; this is excellent for throughput and for workloads that can proceed past a pending commit, but a transaction cannot be declared durable the instant it finishes executing.
Gains depend on how persistence-bound the workload is. The speedup is largest when logging time is a big share of the work (the paper sees more benefit on the higher-latency architecture and on log-heavy applications) and smaller when persistence is already a minor cost, so the 1.2 to 1.7 times average varies across applications.
It targets CXL disaggregated memory with a switch on the path. The design assumes memory traffic flows through a CXL switch that can be augmented; other interconnect topologies or integration points would need the acceleration logic placed differently, which the paper discusses but does not fully evaluate.

The evaluation

Failure-atomic transactions on CXL disaggregated memory

Property	Software persistent-memory transactions	CPU-modifying hardware schemes	Fanmem
Where the persist barrier waits
CPU stall on commit
Changes to the CPU
Checking durability
Throughput vs state of the art

A comparison of failure-atomic transaction approaches on CXL disaggregated memory. By offloading log persistence to a processing unit in the CXL switch and verifying durability asynchronously with a per-log cursor, Fanmem takes the persist barrier off the CPU's critical path without any CPU changes, reaching 1.2 to 1.7 times higher throughput than the state-of-the-art persistent memory transaction system on two representative architectures (a fast switch-based one and a higher-latency rack-scale one).Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer is based on the paper’s full text (the authors’ copy hosted by their research group, of the ASPLOS 2026 paper). The system described is the paper’s: Fanmem is a failure-atomic transaction system for CXL-based disaggregated memory. Its two key innovations are an asynchronous transaction model (dividing a transaction into execution, offloading, and commit stages, so the persist barrier need only reach the CXL switch) and the integration of a lightweight processing unit within the CXL switch that offloads log-persistence operations, intercepting log writes, forwarding them to the memory server, and tracking durability with a per-thread append-only log and a hardware cursor. The application checks persistence with a single load of the cursor and a comparison, and a timestamp-based commit protocol commits transactions in execution order so that recovery from a compute-server, switch, or memory-server failure restores a consistent state; all transaction semantics remain on the CPU, keeping the switch oblivious. The paper reports that reaching remote CXL memory can take up to thousands of cycles (about 729 nanoseconds median for one device), cites a 2.5 percent failure rate for datacenter Ethernet switches as motivation, and reports average performance improvements of 1.2 times and 1.7 times over a state-of-the-art speculative persistent-memory transaction system on two disaggregated architectures (a fast switch-based CXL-F and a higher-latency rack-scale CXL-S), with additional results for a redo-logging scheme. Exact hardware structures, the commit and recovery protocols, and full results should be read from the paper.

Background

Why moving durability off the critical path keeps recurring in memory systems

It helps to see Fanmem within a long line of work on making durable, reliable memory fast. Whenever a system must guarantee that data survives a crash, it faces the same tension: the act of making data durable is slow, but correctness demands you know it happened before you rely on it. Local persistent memory already struggles with this — persist barriers and flushes stall processors — and disaggregation makes it far worse, because the persistent memory now sits at the end of an interconnect, hundreds to thousands of cycles away. The recurring escape is to stop treating durability as a synchronous wait and instead pipeline it: let the processor race ahead while the durability machinery runs in the background, and provide a cheap, reliable way to confirm completion after the fact. Disaggregated memory sharpens this further by separating who computes from who stores, which is both the source of the latency and, as Fanmem shows, an opportunity — the switch in between becomes a place to do the work. The broader move to disaggregate memory and give it persistence for demanding applications is the same terrain the work on persistent memory disaggregation for cloud-native relational databases covered elsewhere on this site occupies, wrestling with durable state in a pooled-memory world. Fanmem’s contribution is to show that the interconnect itself, not the CPU, can be the place that absorbs the cost of durability, keeping the processor free and the design deployable.

If you remember only three things

1. In CXL-based disaggregated memory, a shared memory pool sits across an interconnect from the compute servers, and any component failure can leave it inconsistent, so applications need failure-atomic transactions. But making a transaction durable means forcing its log all the way to the remote persistent memory before committing, and that persist barrier stalls the CPU for hundreds to thousands of cycles on every transaction. Software logging cannot remove the remote latency, and prior hardware schemes required impractical modifications to the CPU core.
2. Fanmem takes persistence off the critical path with two coupled ideas. An asynchronous transaction model (execution, offloading, commit) lets the persist barrier reach only the CXL switch, which acknowledges early so the CPU keeps working; and a processing unit embedded in the switch does the actual persistence, forwarding log writes to the memory server and tracking durability with a per-thread append-only log and a single hardware cursor. The application confirms durability with one cursor read and a comparison, and a timestamp-based protocol commits transactions in execution order so recovery is always consistent. Because the logic lives in the switch, the CPU is untouched — a CPU-oblivious design.
3. On two representative disaggregated architectures, Fanmem achieves 1.2 to 1.7 times higher throughput than the state-of-the-art persistent memory transaction system, with larger gains on the higher-latency, rack-scale architecture and on log-heavy workloads. The trade-offs: it needs an augmented CXL switch, fits redo and speculative logging naturally while undo logging would need more switch-side dependency tracking, delays commit until durability is verified asynchronously, and its benefit scales with how persistence-bound the workload is.

Check yourself

Check your understanding

1. Why is providing failure-atomic transactions expensive in CXL-based disaggregated memory?
2. What does it mean that Fanmem is "CPU-oblivious," and why does that matter?
3. How does Fanmem let an application check whether its transaction is durable without a costly synchronization?
4. Predict: two workloads run on Fanmem — one spends a large fraction of its time writing logs to persistent memory, the other spends almost all its time on computation with tiny, infrequent logs. How will Fanmem's speedup compare?

Sources

The paper is “CPU-Oblivious Offloading of Failure-Atomic Transactions for Disaggregated Memory” by Cheng Chen, Chencheng Ye, Yuanchao Xu, Xipeng Shen, Xiaofei Liao, Hai Jin, Wenbin Jiang, and Yan Solihin (Huazhong University of Science and Technology, University of California Santa Cruz, North Carolina State University, and University of Central Florida), published at ASPLOS 2026, DOI 10.1145/3779212.3790146. It is licensed CC BY, with an author copy hosted by the PICTure research group.

This explainer is written from the paper’s full text (the authors’ hosted copy). Fanmem is a failure-atomic transaction system designed specifically for CXL-based disaggregated memory, ensuring data consistency in the presence of compute-server, switch, or memory-server failures. Its key innovations are an asynchronous transaction model — dividing a transaction into execution, offloading, and commit stages so the persist barrier need only reach the CXL switch — and the integration of a lightweight processing unit within the CXL switch that offloads time-consuming log-persistence operations, intercepting log writes, forwarding them to the memory server, and tracking durability with a per-thread append-only sequential log and a hardware cursor (a “high-water mark”). The application verifies persistence with a single load of the cursor and a comparison, and a timestamp-based commit protocol commits transactions in execution order so post-failure recovery restores a consistent state, while all transaction semantics stay on the CPU (a CPU-oblivious design). The paper reports that reaching remote CXL memory can take up to thousands of CPU cycles, and that Fanmem achieves average performance improvements of 1.2 times and 1.7 times over a state-of-the-art persistent memory transaction system on two representative disaggregated architectures. Numbers are the paper’s own; exact hardware structures, protocols, and full results should be confirmed in the paper.