EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation

Memory disaggregation — putting compute and memory in separate boxes wired together by a network — is held back not by bandwidth but by how long a remote memory access takes over Ethernet, where the frame-based MAC layer bloats tiny read requests and blocks them behind big frames; this paper (Purdue — Weigao Su and Vishal Shrivastav) presents EDM, which bypasses the MAC entirely by running the whole remote-memory protocol inside Ethernet's physical layer using 66-bit blocks (enabling, for the first time, preempting a big frame mid-transmission), and adds a switch scheduler that reserves virtual circuits so memory traffic never queues, reaching about 300 nanoseconds per remote access — an order of magnitude below RoCEv2 and on par with CXL.

Why this problem is worth solving

The bottleneck is not the wire but the layer the protocol sits on.

The competing answer, a dedicated memory fabric, trades one problem for several.

What people did before, and where it broke

• TCP/IP over Ethernet for remote memory. Use the general-purpose network stack to move memory messages. Where it broke. TCP/IP’s host stack is heavy and runs on the MAC layer, adding microseconds of latency and huge relative overhead for tiny memory messages, far above what memory access can tolerate.
• RDMA over Converged Ethernet (RoCEv2). Offload remote memory access to the NIC to bypass the CPU and cut latency, the current state of the art over Ethernet. Where it broke. It still sits on top of the MAC layer, so it inherits the minimum frame size, the inter-frame gap, and the inability to preempt a large frame; its unloaded latency is still an order of magnitude above intra-server memory, and it queues at the switch under load.
• PCIe/CXL memory fabrics. Abandon Ethernet for a dedicated low-latency memory interconnect. Where it broke. CXL matches the latency goal when idle but scales poorly in bandwidth, distance, cost, and power, and its congestion handling is weak, so its latency degrades sharply as network load rises.
• Lossless Ethernet with switch queuing and flow control. Add priority flow control and switch buffering to avoid drops for memory traffic. Where it broke. Queuing at the switch is itself a latency source, and reactive flow control responds after congestion has already formed, so tail latency under load stays high.
• The gap. No fabric delivered CXL-class remote-memory latency while keeping Ethernet’s bandwidth, reach, cost, and congestion resilience — which requires escaping the MAC layer’s frame granularity on the data path and preventing queuing at the switch proactively rather than reacting to it.

How EDM works

The idea

Drop below the MAC layer, because the MAC’s frame granularity is the disease. The core realization is that the latency problems of remote memory over Ethernet — padding a tiny request to a 64-byte frame, the inter-frame gap, and above all being unable to interrupt a big frame in flight — are all consequences of one thing: the protocol operates at whole-frame granularity because it lives on the MAC layer. EDM’s answer is not to optimize the MAC but to bypass it, implementing the entire remote-memory protocol in the Physical Coding Sublayer, which naturally works at 66-bit blocks. At that granularity the small request is no longer bloated, the inter-frame gap becomes usable capacity, and, decisively, a memory message can be interleaved into the middle of a large frame’s transmission, so it never has to wait for unrelated bulk traffic to drain. EDM runs alongside the normal stack rather than replacing it, so ordinary traffic is untouched. Pushing a latency-critical function down into the hardware’s lowest layer, where the granularity matches the work, rather than accepting the overheads of the layer it conventionally sits on, is the same co-design instinct the hardware-software co-designed disaggregated-memory work covered elsewhere on this site is built on, moving remote-memory handling into purpose-built hardware to strip away the general-purpose stack’s latency.

Prevent congestion by scheduling it away, instead of reacting after it forms. The second latency source is the switch, where ordinary Ethernet queues packets and processes them at layer 2, and where reactive flow control only responds once congestion has already built up. EDM removes this by putting a centralized scheduler in the switch PHY that, using the classic Parallel Iterative Matching algorithm, computes a matching between compute and memory nodes and reserves each granted pair a virtual circuit through the switch. Because a matching admits at most one sender per destination, there is by construction no contention, hence no queuing and no drops — the congestion is designed out rather than absorbed. A forwarded message needs no header parsing or table lookup because its circuit is already reserved, and the host no longer needs a heavy transport layer for reliability and congestion control, further shortening the path. Moving the decision of who may send into the switch itself, so the fabric proactively arranges non-conflicting transfers rather than letting endpoints collide and recover, is the same in-network-control philosophy the in-network aggregation work covered elsewhere on this site relies on, doing the coordinating work inside the switch where it can see and shape all the traffic at once.

A clever scheduler is useless if it cannot keep up with the wire, so make it constant-time. Reserving circuits only helps if the scheduler can compute matchings at line rate; a matching algorithm that stalls the switch would reintroduce the very latency it was meant to remove. The obstacle is that a naive priority-based matching iteration, picking the highest-priority request among many for each port, costs on the order of the logarithm of the number of requests in clock cycles. EDM instead completes each iteration in a constant number of cycles through a custom hardware design that deliberately trades silicon for time — a constant-time ordered-list structure supporting parallel reads, comparisons, and writes, plus a fast priority encoder. With that, the scheduler runs at line rate while still honoring message priorities, letting EDM implement SRPT for heavy-tailed traffic and FCFS for light-tailed traffic to get near-optimal completion times. Turning an algorithm that is too slow in software into a fixed-latency hardware datapath that keeps pace with the network is the same move the work on compiling network programs into line-rate NIC hardware covered elsewhere on this site makes, converting per-packet logic into a pipeline that never falls behind the wire.

Following a remote read through EDM

The request also implicitly declares the size of the reply it will generate, which the switch will use to schedule the response.

Why it works

• Block granularity fits memory messages that frame granularity bloats. The MAC pads an 8-byte request to a 64-byte frame and adds an inter-frame gap → but the PHY’s 66-bit blocks carry the small message as-is and even reuse the gap → so tiny memory messages stop wasting bandwidth and latency.
• Intra-frame preemption removes head-of-line blocking by bulk traffic. On the MAC a memory message must wait behind a large frame already transmitting → but EDM interleaves memory blocks into that frame at 66-bit granularity → so a latency-critical access is never stuck behind unrelated storage or IP data.
• Matching designs congestion out instead of reacting to it. Reactive flow control only responds after queues build at the switch → but Parallel Iterative Matching admits at most one sender per destination → so there is no contention, no queuing, and no drops for memory traffic by construction.
• Reserved virtual circuits eliminate switch processing. Normal layer-2 forwarding parses headers and looks up tables per packet → but a granted match is already a virtual PHY circuit → so the data forwards with zero switch processing delay, and hosts can shed their heavy transport stack.
• Constant-cycle scheduling keeps the fabric at line rate. A naive priority matching iteration costs on the order of log-n cycles and would stall the switch → but EDM’s ordered-list-plus-priority-encoder hardware does each iteration in constant cycles → so scheduling never becomes the bottleneck, and latency stays within 1.3 times unloaded even at high load.

What it costs (trade-offs and scope)

• It requires custom hardware in both NICs and switches. EDM is not a software update: it reimplements the remote-memory stack inside the Ethernet PHY at the hosts and puts a scheduler in the switch PHY, demonstrated by extending 25-gigabit Ethernet on FPGAs. Deploying it means EDM-capable NICs and switches, a real barrier compared with a protocol that runs on existing hardware.
• Writes do not get their demand for free. The elegant zero-overhead demand estimation works for reads because the request states the reply size; one-sided writes must send an explicit demand message to the switch first, adding a half-round-trip of latency and some bandwidth. EDM keeps this small with batching and argues it is nominal at rack or cluster scale, but it is a real asymmetry.
• The scheduler trades silicon for speed. Achieving constant-cycle matching relies on a hardware design that spends extra logic — a parallel ordered-list structure and a fast priority encoder — to buy time; the area and power of this scale with the port and request counts, so very large switches face a resource cost the paper’s testbed does not fully stress.
• Centralized scheduling fits a rack or cluster, not the whole datacenter. The virtual-circuit approach and the half-round-trip write cost are argued to be reasonable at rack or cluster scale; extending EDM across many switches and longer distances raises scheduling and coordination questions that are outside the demonstrated scope.
• Evidence is an FPGA testbed plus simulation. The roughly-300-nanosecond and 1.3-times-under-load results come from a small FPGA hardware testbed, a cycle-accurate FPGA simulator on YCSB workloads, and a C network simulator on traces from real disaggregated applications; behavior on production-scale silicon, higher line rates, and mixed real workloads is inferred rather than exhaustively measured.

The evaluation

Check yourself

Sources

The paper is “EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation” by Weigao Su and Vishal Shrivastav (Purdue University), published at ASPLOS 2025, DOI 10.1145/3669940.3707221. It is licensed CC BY.

This explainer is based on the paper’s full text, obtained from the authors’ arXiv preprint (arXiv:2411.08300). The account of the mechanism — the latency (not bandwidth) bottleneck of memory disaggregation over Ethernet; the MAC layer’s frame-granularity overheads (64-byte minimum frame, inter-frame gap, no intra-frame preemption); EDM’s implementation of the entire remote-memory protocol in the Physical Coding Sublayer using custom 66-bit PHY blocks, with repurposed inter-frame gaps and the first intra-frame preemption; the centralized switch-PHY scheduler based on Parallel Iterative Matching that reserves virtual circuits to eliminate queuing and layer-2 processing; the free read-demand estimation from the request versus the explicit batched demand messages for one-sided writes; the constant-cycle scheduling hardware (a constant-time ordered-list structure plus a fast priority encoder) that runs at line rate; and the SRPT and FCFS priority policies for near-optimal completion times — follows the paper’s own description. The reported results cited here are the paper’s own: about 300 nanoseconds unloaded remote-access latency; read latency 3.7, 6.8, and 12.7 times lower than raw Ethernet, RoCEv2, and hardware-offloaded TCP/IP (write latency 1.9, 3.4, and 6.4 times lower); comparable to an unloaded single-hop CXL fabric and intra-server NUMA; and average latency within 1.3 times unloaded under high load. Results come from an FPGA testbed on 25-gigabit Ethernet, a cycle-accurate FPGA simulator on YCSB workloads, and a C network simulator on real disaggregated-application traces. Exact per-experiment numbers, the scheduler hardware details, and the write-demand overhead should be read from the paper itself.