eHDL: Turning eBPF/XDP Programs into Hardware Designs for the NIC

Alessandro Rivitti, Roberto Bifulco, Angelo Tulumello, Marco Bonola and Salvatore Pontarelli

Network cards now run at 100 gigabits and the only way to keep up is to do packet processing on the card, but that means designing hardware, which network programmers cannot do and which vendors take months to deliver for each new function; this paper (Rome, NEC and Axbryd) takes an ordinary unmodified Linux eBPF program — the small filters people already write to run inside the kernel — and compiles its bytecode straight into an FPGA pipeline that forwards at line rate with a microsecond of latency, using a tenth of the chip, with no pragmas and no hardware expertise anywhere in the process.

The argument in one glance

Problem

The card has to do the work, and the card is not yours to program

Network ports are past 100 gigabits per second, which is more packet processing than a host processor can absorb, so the work has to happen on the network card itself. Cards have become programmable, with configurable parsers and forwarding tables, but those features are shaped around a small number of high-volume uses such as offloading a virtual switch. Anything else is either unsupported or takes a large bespoke engineering effort, which is why buying a networking chip company reliably comes with an equally large investment in the software needed to actually use it.
What people did

Let people describe the function and synthesise the hardware

High-level synthesis exists for exactly this. Write what the function should do in a high-level language and let a tool produce the circuit. There are two established flavours. Network-specific tools take P4, a language built for describing packet parsing and classification, and target a matching hardware architecture. General tools take C or C plus plus and will synthesise essentially anything, with far more expressive power and no assumptions about the domain.
Why it fell short

One is too narrow and the other assumes you are a hardware engineer

The network-specific route inherits its architecture from programmable switching chips, which are excellent at header parsing and poor at carrying state between packets. The authors simply could not build a dynamic address translator with it, because there is no way to express updating the translation table. The general route can express that easily, but it does not abstract the hardware away — it just makes describing hardware more convenient. You annotate variables with pragmas that choose memory types and datapath widths, and you interface with modules through bus protocols whose implications require real expertise.
This paper's idea

The program people already write is the specification you need

This paper (Rome Tor Vergata, NEC and Axbryd — Rivitti, Bifulco, Tulumello, Bonola and Pontarelli) notices that Linux already lets people inject small programs into the kernel to process packets at the earliest driver hook, and that operators use them for load balancing, monitoring, intrusion detection and policy. Those programs are already a precise statement of a function somebody wants at the card. So take the bytecode unmodified, unroll it into a hardware pipeline where each stage carries the packet and the program state, and let the language own restrictions do the work that a hardware engineer would otherwise have to.

Why this problem is worth solving

The whole approach rests on a language whose limitations were designed for a completely different reason, so it is worth being precise about what those limitations are.

EBPF lets a user load a small program into the Linux kernel to customise behaviour that used to be fixed. XDP is the hook at the earliest point in the network driver, before the packet reaches the kernel network stack, and people use it for load balancing, security filtering, monitoring and deep packet inspection. The program runs on a small virtual machine — a register machine with eleven 64-bit registers and a 512-byte stack. The kernel imposes two restrictions so that it can guarantee a loaded program terminates and behaves. It is TIME-BOUNDED, meaning loop counts are known at compile time and backward branches only occur in bounded loops. And it is MEMORY-BOUNDED, with no dynamic allocation and a sharp split between memory that lives for one packet — the packet buffer and the stack — and memory that persists across packets, which lives only in structures called MAPS. Apart from map access through a fixed set of helper functions, a program has no side effects at all

Those two restrictions map onto exactly what a hardware pipeline needs.

A hardware pipeline is a fixed sequence of stages through which work flows forward. Turning a program into one requires knowing, at compile time, how long the sequence is — which is precisely what a bound on loops guarantees, since bounded loops can be UNROLLED into stages while an unbounded loop could not be. It also requires knowing which values must travel with the packet and which live outside the pipeline entirely, because anything shared between packets is where race conditions come from — which is precisely what the separation between stack and maps declares. And it requires knowing that nothing else is going on: because a program has no side effects beyond map access, the evolution of every value, including the packet itself, is fully described by the program text. None of these properties were designed for hardware. They exist because the kernel refuses to load a program it cannot vouch for

What people did before, and where it broke

Ship a card with fixed offload features. Vendors build in the functions most customers want, such as virtual switch offloading, and those work very well.
But everything else needs bespoke engineering. The paper notes that vendors take months to deliver a new accelerated function, and many use cases are simply never supported.
Synthesise hardware from P4. A domain-specific language for packet parsing and classification, with tool support targeting both FPGAs and programmable switching chips.
But it inherits an architecture built for switches. State that persists across packets is poorly supported, and the authors could not express a dynamic address translator at all, because the language gives no way to update the translation table from the data plane.
Synthesise hardware from C with a general tool. Far more expressive, handles loops and classes, and avoids writing clock-cycle-level descriptions.
But it does not remove the hardware expertise, it repackages it. Inputs arrive as a stream of bus words that the programmer must reassemble into packets by hand, variables need pragmas selecting memory types and datapath widths, and data consistency is the programmer’s problem.
Run eBPF on a small processor embedded in the card. Both a custom eBPF-tailored processor and an off-the-shelf card with Arm cores can do this, running the programs unmodified.
But a processor handles one packet at a time. Measured here at 0.9 to 5.4 million packets per second against a line rate requirement of 148 million.
The gap. Expressiveness and accessibility were being traded against each other, when the constraints that make a language safe to load into a kernel are the same constraints that make it safe to turn into a pipeline.

How a kernel filter becomes a circuit

From unmodified eBPF bytecode to a pipeline running at line rate

Write the packet processing function as an ordinary kernel program

A network programmer writes C, compiles it to eBPF bytecode, and today loads it into the Linux kernel at the earliest driver hook. This step is unchanged and is what people already do — the input to everything that follows is that bytecode, not a rewritten version of it.
↓ →
Work out which memory each instruction touches new

Build the control-flow and data-dependency graphs, then label every load and store by the region it accesses. The stack pointer lives in a fixed read-only register, the packet address arrives in another at program start, and a map pointer lands in a third after every lookup call, so following what derives from each identifies all three regions.
↓ →
Unroll the instructions into pipeline stages new

Treat the program as a sequence of transformations on a state made of the packet, the registers and the stack, with each stage handing the transformed state to the next. Instructions with no dependency on each other within the same control block share a stage, so the width of each stage is whatever that point in the program can use.
↓ →
Handle control flow by disabling stages rather than jumping new

Every packet walks through every stage. Jump results drive enable signals, and a disabled stage simply passes the packet along untouched. Backward jumps are turned into forward ones during scheduling, which is always possible because the language forbids unbounded loops, so the pipeline only ever feeds forward.
↓ →
Deal with the races that unrolling creates new

Persistent state lives in maps, and unrolling means several packets touch a map at once. Delay each write until the later reads that raced it have finished. For per-flow state, detect a stale read and flush the pipeline, gambling that two packets of the same flow rarely arrive together. For counters, use in-place atomic hardware instead.
↓ →
Shrink the pipeline until it fits comfortably new

Carrying a full packet, a full stack and eleven registers per stage would cost over 2 kilobytes each across hundreds of stages. Instead, chop the packet into frames and let a stage reach backwards for frames still upstream, and prune each stage down to only the registers and stack bytes actually live there.

Prototyped on a Xilinx Alveo U50 100 gigabit FPGA card integrated with an open-source NIC shell, evaluated on five unmodified real programs including a Linux router, a tunnel, a dynamic address translator and an intrusion detection system.

The idea

The restrictions that made the language safe are what make it synthesisable. This is the argument the paper is really making, and it explains why the two existing routes both fail. P4 is too restrictive because it was designed around a switching chip, so it is good at headers and bad at memory that survives a packet. General C synthesis is too permissive, so the tool cannot infer what the programmer meant and demands pragmas instead. Kernel-loadable programs happen to sit exactly in between, and not by anyone’s design for this purpose. Bounded loops exist so the kernel can prove termination, and they are also what lets a program be unrolled into a fixed-length pipeline. No dynamic allocation exists so the kernel can bound memory, and it is also what lets stage sizes be computed at compile time. The hard separation between per-packet memory and persistent maps exists so the kernel can reason about lifetime, and it is also exactly the line between values that travel with the packet and values that can race. Finding that a constraint imposed for safety turns out to be the precondition for an entirely different capability is the same the-restriction-is-the-asset argument covered elsewhere on this site where a deliberately limited description is what allows a tool to take over work a human would otherwise have to do by hand.

Generating hardware per program deletes the trade-offs a fixed design must make. There is a lovely recurring observation running through the design section. When you build a processor you constantly weigh whether a feature pays for itself across the whole workload mix: the eBPF-tailored processor they compare against added six-byte load and store instructions, which help only when a program modifies MAC addresses and are pure wasted area otherwise. Same story for parallelism — provide few parallel issue slots and you cap what fast code can do, provide many and they sit idle whenever dependencies bite, because instruction-level parallelism varies through a program. When the hardware is generated for one specific program, neither question arises. A fused instruction gets its circuit instantiated only where that program uses it. A stage is exactly as wide as the instructions that can run there, growing and shrinking along the pipeline: in their worked example, nine stages hold a single register, six hold two, one holds three, and the whole thing would have carried eleven per stage without analysis. Replacing a design-time trade-off with a per-instance decision, so that generality costs nothing because nothing general is built, is the same specialise-away-the-average move covered elsewhere on this site where a tailored datapath beats a programmable one simply by omitting everything the workload does not need.

Borrow the processor’s tricks, then check the gamble against real traffic. Having built a pipeline, they find they need the whole classical repertoire. Control flow becomes PREDICATION, with every packet walking every stage and disabled stages passing it through. Write-after-read races are fixed by delaying the write in a buffer until the racing read completes. Read-after-write races on per-flow state are handled by speculating and flushing, with a dedicated block tracking the addresses of unconfirmed reads. Reaching a packet frame still upstream is operand bypassing; needing one not yet in the pipeline is solved with inserted no-op stages. What makes this more than an analogy is that the speculation is a bet about traffic, and they test it: on real captured traces the pipeline drops nothing and flushes fewer than 350,000 times a second, but forcing every packet to touch the same map entry collapses throughput from 29 to 12 million packets per second. They also explain why they rejected stalling instead, which would waste less work but requires knowing the write address before the read stage. Validating a speculative policy against the distribution it assumes, and publishing the adversarial case where it fails, is the same honest-about-the-workload-assumption discipline covered elsewhere on this site where a design bets on a common case and reports what happens when the bet is wrong.

Watching one program turn into twenty stages

A twenty-three instruction packet counter becomes a hardware pipeline

The paper worked example counts received packets by Ethernet protocol type, with all figures from its generated design and evaluation.

Step 1 of 5

Start from bytecode nobody wrote for hardware

The input is compiled eBPF bytecode for an ordinary kernel program: read two bytes of the Ethernet header, compare against three protocol constants, set a key accordingly, look up a counter in a map and atomically increment it, then transmit. Compiling from bytecode rather than source means the programmer can write in C or Rust and it makes no difference.

the bounds-check instructions the kernel requires vanish entirely, because the generated hardware checks packet length when it accesses a frame
the program is heavy on branching, so at most two instructions ever run in parallel and only three of the twenty stages exploit even that

Why it works

Kernel-loadable programs have bounded loops and no dynamic allocation → the instruction sequence can be unrolled into a fixed-length pipeline with compile-time stage sizes → so no programmer annotation is needed to describe the hardware structure.
Every stage holds its own copy of the packet and program state → the pipeline contains as many packets in flight as it has stages → giving line rate where a processor running the same bytecode manages a few million packets per second.
The hardware is generated for one specific program → a fused instruction or a wide stage is instantiated only where that program uses it → so resources land 2 to 4 times below a tool that builds generic parsers and tables.
Persistent state is confined to maps by the language itself → the only possible races are known at compile time and can be given delay buffers, flush logic or atomic units → so consistency is handled without the programmer thinking about it.
Real traffic spreads across many flows and packets are mostly larger than minimum → speculating that same-flow packets rarely collide almost always pays → costing fewer than 350,000 flushes per second and no dropped packets on captured traces.

What it costs (trade-offs and scope)

Changing the program means resynthesising the chip. Kernel programs load in milliseconds; a generated pipeline takes hours to synthesise, and even a pre-built one requires taking the card out of service to reflash it.
The flush gamble has an adversarial case. Forcing every packet to the same map entry drops throughput from 29 to 12 million packets per second. Traffic that concentrates on one flow is exactly what breaks it.
Maps that stay on the host cost round trips. Programs run unchanged either way, but an operator has to know where each map lives, and coordinating that across a heterogeneous system is left open.
The clock was capped at 250 megahertz to match the port. The authors believe higher is achievable but note that pushing frequency risks the design failing to meet timing, which would undermine the no-expertise-required premise.
Five programs on one FPGA card. The evaluation is thorough within that scope, and the authors say plainly it is unclear whether this approach is useful to chip designers as opposed to FPGA users.

The evaluation

Five unmodified real programs: two shipped with the Linux kernel sources, a stateful firewall tracking bidirectional connection setup, a dynamic address translator that picks ports in the data plane, and the eBPF filters generated by an open-source intrusion detection system. Traffic comes from a directly connected machine generating line rate with minimum-size packets, with flow counts from one to over a hundred thousand, and everything is measured with the processing staying entirely on the card, since sending packets to the host would make the bus the bottleneck. Three comparison points: a P4 synthesis tool on the same FPGA, a custom eBPF processor on the same FPGA, and a commercial card with Arm cores.

Property	Synthesis from P4	eBPF on an on-card processor	Synthesis from eBPF bytecode
What the programmer supplies
State that persists across packets
Throughput at 10,000 flows
Chip resources used
Hardware expertise needed

Forwarding latency is about one microsecond, comparable to the on-card processor because both exploit the same instruction-level parallelism; the throughput gap comes from the pipeline holding one packet per stage rather than one in total. On captured traffic traces a flush-prone application drops no packets, with under 350,000 flushes per second. Disabling state pruning costs 46 percent more logic cells, 66 percent more flip-flops and 123 percent more block memory.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

The deployment story is the weak point and the paper says so. Kernel programs are loaded at runtime in milliseconds, which is much of why people like them, whereas synthesising a generated pipeline takes hours and even loading an already-synthesised one means taking the card out of service to reflash. The authors want dynamic partial reconfiguration and do not have it, so the workflow this enables is closer to building a specialised appliance than to the load-and-unload rhythm eBPF users expect. The consistency handling rests on a bet about traffic. Speculating that two packets of the same flow rarely occupy the pipeline together holds up on real captured traces, with no drops and under 350,000 flushes per second, but they report the adversarial case honestly: directing every packet at the same map entry drops throughput from 29 to 12 million packets per second. They also considered stalling instead of flushing, which wastes less work, and rejected it because it needs the write address known at the read stage. Two smaller limits are worth holding. The clock was capped at 250 megahertz to match a 100 gigabit port, and the authors note that pushing it higher risks timing closure failures — which would quietly reintroduce the hardware expertise the whole design exists to avoid. And maps that remain on the host still cost bus round trips, so an operator has to reason about placement even though the program itself runs unchanged.

Background

When a language built for one purpose turns out to be a hardware specification

This paper sits in the site’s networking and hardware-synthesis clusters and rhymes with several ideas covered elsewhere on this site. Its central argument — that a deliberately restricted description is more useful than a fully general one because the restrictions let a tool infer what a human would otherwise annotate — is the same bargain the portable custom instruction page strikes when it insists a description say only what an instruction computes and never when, and the same one the vectorizing compiler generation page relies on when an instruction-set specification plus a cost model turns out to be enough input to build a compiler from. Its finding that generating hardware per program dissolves trade-offs a fixed design cannot escape rhymes with the reconfigurable-acceleration and dataflow-synthesis pages, where the value of specialisation is mostly in what gets omitted. Its wholesale borrowing of processor pipeline machinery — predication, speculation with flush, operand bypassing, hazard detection — for a pipeline that is not a processor at all is a nice inversion of the per-packet machine learning page, which pushes general computation into the data plane rather than pulling data plane work into a generated circuit. And its willingness to publish the traffic pattern that defeats its own speculation is the same name-the-adversarial-case honesty the cache and prefetching pages practise.

If you remember only three things

1. Network ports past 100 gigabits per second force packet processing onto the card, but programming a card means designing hardware. The two existing synthesis routes fail in opposite directions: a network-specific language inherits an architecture built for switching chips and cannot express state that survives a packet, so a dynamic address translator is simply impossible, while general synthesis from C is expressive but keeps the hardware expertise, demanding pragmas that pick memory types and datapath widths and leaving data consistency to the programmer.
2. The insight is that Linux kernel filters already sit between those extremes, and not by design for this. They must have bounded loops so the kernel can prove termination, which is also what lets a program be unrolled into a fixed-length pipeline. They cannot allocate dynamically, which is also what makes stage sizes computable at compile time. And they must keep anything persisting across packets in separate structures called maps, which is also exactly the boundary between values that ride along with a packet and values that can race. Restrictions imposed for kernel safety turn out to be the preconditions for automatic hardware generation.
3. The generated design unrolls each instruction into a pipeline stage carrying the packet, registers and stack, then borrows the classical processor repertoire — predication for control flow, delayed writes for write-after-read races, speculate-and-flush for read-after-write, operand bypassing to reach packet frames still upstream. Framing the packet and pruning dead state cuts the largest stage from over 2 kilobytes to 88 bytes. All five real programs forward at line rate, 148 million packets per second, with about a microsecond of latency on 6.5 to 13.3 percent of the chip, against 0.9 to 5.4 million for the same programs on an on-card processor.

Check yourself

Check your understanding

1. Synthesising hardware from a high-level description is a mature field, yet the paper argues neither existing route works here. Predict what makes kernel filter programs a better starting point than either P4 or C.
2. The same eBPF programs run on a processor embedded in the card, and that processor achieves comparable per-packet latency. Predict why the generated pipelines are 10 to 100 times faster in throughput anyway.
3. Unrolling a program into a pipeline means several packets touch the same persistent map simultaneously, creating classic read-after-write races. Predict how the design handles them, given that flushing wastes work.
4. A processor design must weigh whether adding an instruction or more parallel issue slots pays for itself. Predict why that calculation disappears when hardware is generated per program.

Sources

Alessandro Rivitti, Roberto Bifulco, Angelo Tulumello, Marco Bonola and Salvatore Pontarelli. “eHDL: Turning eBPF/XDP Programs into Hardware Designs for the NIC.” ASPLOS 2023, Vancouver. University of Roma Tor Vergata, NEC Laboratories Europe and Axbryd. DOI 10.1145/3582016.3582035.
Full text of the paper was used for this explainer, including the comparison with network-specific and general high-level synthesis in Section 2, the program analysis, parallelisation, hardware primitives and control flow enforcement in Section 3, the map consistency handling, packet framing and state pruning in Section 4, the throughput, latency, resource, flushing and pruning measurements in Section 5, and the stated limitations around deployment and host map access in Section 6.