Domain specific run time optimization for software data planes

Sebastiano Miano, Alireza Sanaee, Fulvio Risso, Gábor Rétvári and Gianni Antichi

Software network functions — routers, firewalls, load balancers running in software — must process millions of packets a second, but the static compiler that builds them has to keep the code fully general, full of branches and memory lookups for every protocol and configuration, because it cannot see two things that are actually stable at run time — the control-plane rules (routing and firewall tables that change only every few seconds) and the traffic itself (where just 5% of flows carry 95% of the packets); Morpheus (Polytechnic of Turin and Queen Mary) is a domain-specific JIT that watches the running data plane, bakes those runtime facts into specialized code — turning table lookups into inlined constants, deleting handlers for protocols that never appear, and fast-pathing the heavy-hitter flows — hot-swaps it in without dropping a packet, and guards each specialization so it safely falls back the instant an assumption breaks, doubling throughput and cutting tail latency by up to 123% where a general profile-guided optimizer managed only 4%.

The argument in one glance

Problem

Packet-processing software must be blazing fast, but its compiler is flying blind

A software data plane — the router, firewall, NAT, or load-balancer code that forwards and filters packets in software (eBPF/XDP, DPDK) — runs at the very edge of what a CPU can do: millions of packets per second, a budget of only nanoseconds per packet, so any wasted instruction is multiplied millions of times a second. Yet the static compiler (LLVM/Clang) that builds this code optimizes with no knowledge of the conditions it will actually run under. So it emits GENERIC code: it keeps handlers for every protocol even ones that never appear in a deployment, it issues memory loads to fetch configuration values that in practice never change, and it takes hard-to-predict branches on data it cannot know. The code is correct and general, and slower than it needs to be — and the generic profile-guided optimizers meant to help barely move the needle.
What people did

Apply a general profile-guided optimizer

The standard way to close the gap between generic code and the actual workload is profile-guided optimization: run the program, collect a profile of which branches and paths are hot, and recompile with that information — reordering branches, inlining hot functions, laying out code for the common case. Industry tools like AutoFDO and BOLT do exactly this at scale. Point one at the data-plane binary, feed it a profile of real traffic, and let it optimize the hot paths the way it would for any program.
Why it fell short

Generic PGO does not understand packets, and misses the real invariants

General PGO barely helps: applied to a DPDK firewall, AutoFDO and BOLT improved throughput by only 4.2%. The reason is that they optimize at the level of generic branches and code layout, but the performance being left on the table in a data plane is domain-specific and lives in two places a general optimizer does not understand. First, the CONTROL PLANE — routing tables, ACL rules, load-balancer backends — changes only on human timescales (seconds to minutes), so between changes its contents are effectively CONSTANT, yet the compiled code still treats every lookup as a variable memory access. Second, the TRAFFIC is wildly skewed: empirically just 5% of the active flows carry 95% of the packets, so the code could fast-path a handful of heavy hitters, but a generic profiler sees only branch frequencies, not that a specific set of flow values dominates. General PGO cannot bake a routing table into constants or specialize for the heavy-hitter flows, because it does not know those are the opportunities.
This paper's idea

A domain-specific JIT that specializes the data plane to its live conditions

Morpheus (Polytechnic of Turin and Queen Mary University of London — Miano, Sanaee, Risso, Rétvári, Antichi) is a run-time optimizer that works alongside the static compiler and continuously RE-SPECIALIZES the hot networking code to what it actually sees. It tracks the two runtime invariants the static compiler misses — the current control-plane configuration and the live traffic distribution — and bakes them into specialized code: it turns table lookups into inlined constants and if-then-else chains, deletes handlers for protocols that never appear, and builds a fast path for the heavy-hitter flows. Cheap ADAPTIVE INSTRUMENTATION observes the traffic without drowning the data plane, a domain-specific optimization TOOLBOX rewrites the code, and GUARDS protect every specialization so it safely deoptimizes back to the general code the instant an assumption breaks. Recompilation runs on a separate core and the new code is hot-swapped in atomically, with no dropped packets — for up to 2x throughput and 123% lower tail latency, where general PGO managed 4%.

Why this problem is worth solving

Software data planes are where a growing fraction of networking happens — flexible, upgradable packet processing on commodity servers instead of fixed-function switches — and at line rate their efficiency is everything, so the gap between generic and specialized code is expensive at scale. The gap exists because the compiler must be conservative about facts it cannot know, and the deepest of those facts is the

the configuration that governs how packets are handled — the routing table, the firewall ACL, the set of load-balancer backends — which is set by the control plane and changes only on human timescales, as an operator adds a rule or a route flaps, seconds to minutes apart. Between those changes, the configuration is effectively CONSTANT: the same routing table, the same ACL, the same backends, for millions of packets. A static compiler, though, has no way to know what the configuration will be, so it compiles a general lookup — fetch the table from memory, index into it, branch on the result — that pays the full cost of generality on every packet even though the answer is, for long stretches, fixed. If the compiler could see that the table currently has two entries, it could turn the lookup into two inlined comparisons with the values baked in as constants, erasing the memory access entirely

. The static compiler leaves this on the table not out of weakness but because the information does not exist at compile time.

The second missed fact is about the data itself, and it is just as exploitable. Network traffic is not uniform but heavily

the empirical fact that a tiny fraction of network flows carries the overwhelming majority of the packets — in the paper’s measurements, about 5% of the active 5-tuple flows account for 95% of the traffic. This is a strong, persistent skew: at any moment a handful of “heavy hitter” flows dominate, and the rest is a long tail of flows seen rarely. It means the code spends almost all of its time processing a small, identifiable set of flow values, so a fast path specialized to exactly those values — with their lookups resolved and their handling inlined — would accelerate the common case dramatically, while the general code still handles the rare tail. But which flows are the heavy hitters changes with the workload and is invisible at compile time; only by observing the live traffic can you discover and specialize for them

. Together these two invariants — a configuration that is constant between control-plane changes, and a traffic distribution that is skewed and observable — are a large, domain-specific pool of optimization opportunity that a static compiler cannot reach and a generic profiler does not recognize. The reason to build Morpheus is that capturing this requires watching the data plane run and re-specializing continuously, safely, and cheaply enough not to undo the gains it produces.

What people did before, and where it broke

Static compilation (LLVM/Clang). Compile the data plane once, ahead of time. Correct and general, but with no runtime knowledge it keeps dead protocol handlers, does memory loads for effectively-constant config, and branches on unknown data — leaving large, domain-specific gains untaken.
General profile-guided optimization (AutoFDO, BOLT). Recompile using a profile of hot branches and code layout. Helps generic programs, but on a data plane it improves a firewall only 4.2%, because it optimizes branch frequencies rather than the real levers — constant config values and heavy-hitter flows — which it does not understand.
Policy-only / control-plane optimizers (ESwitch). Specialize based on the control-plane configuration alone. A real gain when config is exploitable, but blind to traffic skew, so it misses the heavy-hitter opportunity — Morpheus beats it 5-10x on high-locality traffic.
Hand-tuned or single-target data-plane optimizers (PacketMill). Aggressively optimize one framework (DPDK). Effective there, but tied to a platform and a static view — Morpheus, tracking live traffic, beats PacketMill by 469% on a 500-rule router with skewed traffic.
The gap. No system continuously specialized the data plane to BOTH its live control-plane state and its observed traffic distribution, with cheap instrumentation and safe fallback, in a data-plane-agnostic way. Morpheus does, treating packet processing as a domain a JIT can exploit.

How Morpheus works

Analyze, watch cheaply, specialize with a domain toolbox, guard, and hot-swap

See what the static compiler cannot new

Morpheus targets two runtime invariants a static compiler misses: the control-plane configuration, which is constant between changes, and the traffic distribution, which is skewed toward a few heavy-hitter flows. Both are stable enough to specialize for, yet unknowable at compile time — so a runtime optimizer is the only way to reach them.
↓ →
Analyze the code to find and classify opportunities new

Before running, Morpheus statically analyzes the program's control and data flow to find every table (map) access, and classifies each as READ-ONLY (never modified by the data plane, so it can be aggressively specialized) or READ-WRITE (updated per packet, so specializations need per-access guards). For Katran this marks the VIP and backend maps read-only and the connection table read-write.
↓ →
Instrument adaptively to watch traffic cheaply new

Logging every packet lookup would be catastrophic (naive instrumentation costs up to 900% overhead), so Morpheus samples selectively — by map size, access dynamics, per-CPU locality, and call site — recording heavy-hitter values into small caches. This adaptive instrumentation costs only 0.9-9% while revealing which config values are constant and which flows dominate, with a sweet-spot sampling rate of 5-25%.
↓ →
Specialize with a domain-specific optimization toolbox new

Morpheus rewrites the hot code: JIT-compile small tables into if-then-else chains with their entries inlined as CONSTANTS (erasing memory loads); delete handlers for protocols the traffic never shows (dead-code elimination); build a JIT fast-path cache for the heavy-hitter flows; swap a longest-prefix table that has degenerated to exact-match for a faster hash table; and inject cheap guards before expensive lookups.
↓ →
Guard every specialization for safe fallback new

Each specialization rests on an invariant — a table's contents, a traffic pattern — protected by a GUARD. If the invariant changes (a new rule, a new heavy connection), the guard triggers DEOPTIMIZATION, reverting to the original general code until the next recompilation. Read-only tables need only one program-level guard at entry; read-write tables keep per-access guards.
↓ →
Recompile off the critical path and hot-swap atomically new

The specialized LLVM IR is regenerated on a SEPARATE core while the data plane keeps forwarding, taking 62-600 milliseconds; control-plane updates are queued meanwhile. When ready, a single program-level guard is flipped atomically and the queued changes applied — the new code takes over with no dropped packets, at about 5% average compiler-core CPU.

Evaluated on eBPF/XDP network functions (L2 switch, router, NAT, BPF-iptables), Facebook's Katran L4 load balancer (10 VIPs, 100 backends each), and a DPDK FastClick router (20 and 500 rules), on synthetic and real (CAIDA) traffic. About 6,000 lines of C++; eBPF fully supported, DPDK partially.

The idea

Recover the performance the static compiler leaves behind because it cannot see run time. A compiler must be conservative about facts it does not know, so it emits general code: full lookups for configuration that is actually constant, handlers for protocols that never appear, branches on data it cannot predict. Morpheus’s central move is to observe those facts as the data plane runs and bake them into specialized code — a domain-specific just-in-time optimizer that turns a routing table’s current contents into inlined constants and the heavy-hitter flows into a fast path. This is the specialize-to-the- runtime-reality discipline the site has been circling from both sides: the on-chip-allocation page could plan offline because its problem was static and known, and the sparse-tiling page had to decide at runtime because its data was irregular and unknowable ahead — and Morpheus is squarely in the second camp, exploiting invariants that exist at run time but not at compile time. The insight is that “the compiler cannot know this” is not the end of the story; a JIT that watches the program run can.

Optimize the common case aggressively, and guard it so the rare case stays correct. The two invariants Morpheus exploits are both common-case bets: the configuration is USUALLY unchanged, and the traffic is MOSTLY a few heavy hitters. Betting on them lets Morpheus inline constants and fast-path hot flows for enormous speedups — but a bet must be safe, so every specialization is protected by a guard that detects when its assumption breaks (a rule changes, a new flow turns hot) and deoptimizes back to the general code. The common case runs the fast specialized path; the rare case falls back to correct general code. This is the optimistic-common-case-fast-with-safe-fallback discipline the site rewards — the persistent-memory database reading optimistically and repairing the rare stale case, the noisy-quantum page betting on structured errors — where the aggression is only acceptable because a cheap check plus a correct fallback bounds the downside. Guards are what let a data plane speculate without ever being wrong.

Do not let the watching cost more than the specialization saves. The whole scheme depends on observing the traffic, and naive per-packet profiling would cost up to 900% overhead — annihilating the performance it is trying to improve. Morpheus’s adaptive instrumentation samples selectively across several dimensions (map size, access dynamics, locality, call site) to get the same visibility for 0.9-9%, and it runs the expensive recompilation on a separate core off the packet path, hot-swapping atomically. This is the same observer-effect discipline the site just met in the tiered-memory page, whose CPU-side monitoring degraded the very workload it optimized, and the datacenter-IO page, which split a cheap per-request path from an occasional expensive one — here the measurement is made cheap enough, and the compilation moved off the critical path, so the optimization is a net win rather than a self-inflicted wound.

Baking the runtime into the code

From a generic data plane to one specialized to its live traffic

Optimizing a running software firewall/load balancer. Numbers from Morpheus's evaluation.

Step 1 of 5

The generic compiled data plane

The statically-compiled firewall or load balancer carries branches and table lookups for every protocol and every rule, because the compiler had to keep it general. On each packet it fetches configuration from memory, indexes tables, and takes data-dependent branches — paying the full cost of generality millions of times a second, even where the answers barely change.

general profile-guided optimizers improve this only ~4.2% — they optimize branches, not the real levers
the wasted work is domain-specific: constant config treated as variable, dead protocol handlers, cold branches

Why it works

The static compiler cannot know the control-plane config or the traffic, so it emits general code → Morpheus observes both at run time and bakes them in → constant config becomes inlined constants and heavy-hitter flows become a fast path.
Configuration is constant between control-plane changes → JIT-ing small tables into if-then-else with values inlined erases the memory loads and data-dependent branches → up to 96% fewer cache misses and ~50% fewer instructions per packet.
Traffic is skewed, with 5% of flows carrying 95% of packets → a specialized fast path for the heavy hitters accelerates the common case while the general code handles the tail → ~42% on the firewall, ~24% on the load balancer.
Aggressive specialization would be unsafe if assumptions changed → guards detect changed invariants and deoptimize to the general code → the fast path is used only while correct, and the worst case never raises latency.
Naive profiling (900% overhead) would destroy the gains → adaptive sampling (0.9-9%) plus recompilation on a separate core with atomic hot-swap → the measurement is cheap, the compilation is off the packet path, and no packets are dropped.

What it costs

Stateful, churning workloads can regress. When a program updates state per packet (a NAT connection table) and the traffic has no locality, guards fire and recompilation churns, costing about 6% — so Morpheus falls back to traffic-independent optimizations and, in the worst case, needs the stateful code’s optimization disabled manually.
Adaptation is bounded by recompile latency. If traffic changes faster than the roughly one-second recompilation period, the traffic-dependent specializations cannot keep up and only the config-based gains remain.
Platform coverage is partial. The eBPF path is fully implemented (via Polycube), but the DPDK path disables dynamic optimizations for stateful FastClick elements, and other frameworks (netmap, AF_XDP) are not yet supported.
It stops at the IR, missing machine-level gains. Operating on LLVM IR for portability across eBPF and DPDK means Morpheus forgoes native peephole, vectorization, and SIMD optimizations a machine-code specializer could add.
Read-write tables limit aggressiveness. Because per-access guards are needed on tables the data plane mutates, the downstream constant-propagation and dead-code elimination that make read-only specialization so effective apply only weakly there.

The evaluation

eBPF/XDP network functions (L2 switch, router, NAT, BPF-iptables), Facebook’s Katran L4 load balancer (10 VIPs, 100 backends each), and a DPDK FastClick router (20 and 500 rules), driven by synthetic traffic of varying locality and a real CAIDA trace, measured for throughput, tail latency, micro-architecture, and instrumentation and recompilation overhead, against static compilation, general PGO (AutoFDO, BOLT), a policy-only optimizer (ESwitch), and a DPDK optimizer (PacketMill).

Optimizing a software data plane

Property	Static compiler (LLVM/Clang)	General PGO (AutoFDO, BOLT)	Morpheus (domain-specific JIT)
Uses control-plane config?
Uses traffic skew?
Specializes safely?	n/a — always general
Adapts at run time?
Speedup	Reference

Morpheus wins because it exploits the two domain-specific invariants a general optimizer cannot — constant control-plane config and heavy-hitter traffic — while a guard mechanism keeps the aggressive specialization safe and adaptive instrumentation keeps the watching cheap, so it delivers order-of-magnitude gains over policy-only and general PGO where they see only single-digit percentages.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

Morpheus’s gains depend on there being exploitable invariants, so its weak spot is fully stateful code under churny, locality-free traffic (a NAT whose connection table updates every packet): there guards fire and recompilation churns, costing about 6%, and the stateful optimization may have to be disabled by hand. Adaptation is bounded by the roughly one-second recompile period, so traffic that shifts faster than that keeps only the config-based gains. The eBPF path is fully supported but the DPDK path disables dynamic optimization for stateful elements, and other frameworks are unimplemented; operating at LLVM IR for portability forgoes machine-level optimizations; and read-write tables, needing per-access guards, cannot get the aggressive constant-propagation and dead-code elimination that make read-only specialization so effective. None of this dulls the core insight — that a static compiler must emit general packet-processing code because it cannot see the control-plane configuration (constant between changes) or the traffic distribution (skewed toward a few heavy-hitter flows), and a domain-specific run-time optimizer can watch the data plane, bake those invariants into specialized code (inlined constants, deleted dead handlers, heavy-hitter fast paths), guard each specialization so it deoptimizes safely when an assumption breaks, and hot-swap it in off the critical path — turning a 4%-margin generic optimization into a 2x one.

Background

Specialize to the runtime you can see, bet on the common case, and keep the watching cheap

Morpheus sits in the site’s networking-and-systems cluster and rhymes with several ideas covered elsewhere on this site. Its governing move is to specialize to runtime facts a static compiler cannot see — the control-plane config that is constant between changes and the traffic skew toward heavy hitters — which places it opposite the on-chip-memory allocation page (whose problem is static and known, so it plans offline) and alongside the sparse-tiling page (whose data is irregular and unknowable ahead, so it decides at runtime): the same static-versus- dynamic axis, and Morpheus is firmly on the exploit-what-only-exists- at-run-time side, a domain-specific JIT where the general profile-guided optimizers fail because, like the GNN-operator and PCIe-collectives pages argue, a specialized optimizer that understands the domain beats a general one that does not. Its second move is the optimistic-common- case-fast-with-safe-fallback discipline: it bets aggressively on the common case (constant config, dominant flows) and protects each bet with a guard that deoptimizes when the assumption breaks — the same speculation-plus-cheap-verification the persistent-memory database uses reading optimistically and repairing the rare stale case, and the noisy-quantum page uses betting on structured errors. Its third move is the observer-effect discipline the site just sharpened: naive profiling would cost 900% and destroy the very performance it seeks, so adaptive instrumentation makes the watching cheap (0.9-9%) and the recompilation runs off the packet path with an atomic hot-swap — the same don’t-let- monitoring-cost-more-than-it-saves and keep-the-hot-path-cheap lessons the tiered-memory and datacenter-IO-control pages turn on.

If you remember only three things

1. Software data planes (routers, firewalls, load balancers in eBPF/XDP or DPDK) must process millions of packets a second, but their static compiler must emit generic code — dead protocol handlers, memory loads for effectively-constant configuration, data-dependent branches — because it cannot see two runtime invariants: the control-plane configuration (routing/ACL/backend tables that change only every few seconds, so are constant between changes) and the traffic distribution (about 5% of flows carrying 95% of packets); general profile-guided optimizers barely help (4.2% on a firewall) because they optimize branch layout rather than these domain-specific levers.
2. Morpheus is a domain-specific run-time optimizer that continuously re-specializes the hot code to its live conditions: static analysis classifies each table as read-only or read-write, adaptive instrumentation samples the traffic cheaply (0.9-9% overhead versus 900% for naive logging) to learn constant values and heavy hitters, a toolbox of optimizations bakes them in (inline table entries as constants, delete handlers for absent protocols, JIT a fast path for heavy-hitter flows, swap degenerate longest-prefix tables for hash tables), and a guard mechanism deoptimizes back to the general code the instant an invariant changes — with recompilation on a separate core and atomic hot-swap, so no packet is dropped.
3. The result roughly doubles throughput on high-locality traffic (up to 469% over a DPDK optimizer, 5-10x over a policy-only one), cuts 99th-percentile latency up to 123% while never raising it, and yields up to 96% fewer cache misses and ~50% fewer instructions per packet — holding ~30% even on low-locality traffic and re-specializing within about a second when traffic shifts. Its honest limits are stateful, churny workloads (which can regress ~6%), an adaptation speed bounded by recompile latency, and IR-level (not machine-level) optimization. The durable lessons: specialize to the runtime facts a static compiler cannot see, bet on the common case but guard it for safe fallback, and keep the measurement cheap enough not to erase the gains.

Check yourself

Check your understanding

1. General profile-guided optimizers (AutoFDO, BOLT) improve a data-plane firewall by only about 4.2%, while Morpheus achieves up to 2x. Predict why the general tools leave so much on the table.
2. Morpheus aggressively specializes code by inlining the current control-plane config as constants and fast-pathing the current heavy-hitter flows — but both of these can change. Predict how it specializes so aggressively while staying correct.
3. Naive per-packet profiling to learn the traffic distribution would cost up to 900% overhead. Predict why Morpheus's handling of this measurement problem is essential to the whole approach.
4. Morpheus can plan its specializations only by observing the data plane at run time, whereas the on-chip-memory allocator in a sibling paper plans entirely offline. Predict what determines which approach each problem needs.

Sources

ACM DL: Domain Specific Run Time Optimization for Software Data Planes — ASPLOS 2022 (the Morpheus system; full text read via the author-hosted PDF; no arXiv preprint).
Morpheus source code (GitHub) — open-source implementation.