Data centers offload network functions — firewalls, encryption, packet processors — onto SmartNICs to spare host CPUs, but running several on one card makes them fight over the card's onboard accelerators and memory, so each one slows down unpredictably, and operators cannot promise performance without a way to forecast that slowdown; this paper (Chinese University of Hong Kong and Microsoft Research — Shaofeng Wu, Qiang Su, Zhixiong Niu, Hong Xu) presents Yala, a predictor built on SmartNIC-specific knowledge — a white-box queueing model for the round-robin accelerator queues, a learned model for memory contention, composed according to whether each function runs as a pipeline or run-to-completion, and augmented with the incoming traffic's flow count and packet size — cutting average prediction error to 3.7 percent (a 78.8 percent improvement over the prior best) and reducing SLA violations by 92.2 percent.
The argument in one glance
-
Problem
Network functions sharing a SmartNIC slow each other down, and no one can say by how much
Modern data centers offload network functions — firewalls, encryption, load balancing, packet inspection — from the host CPU onto SmartNICs, programmable network cards with their own processor cores, hardware accelerators, and memory. This saves precious host CPU cycles and lets operators program the network flexibly. To use these cards efficiently, operators pack several network functions onto the same SmartNIC. But co-located functions contend for the card's limited onboard resources, so they interfere: a function that hit some throughput running alone drops when it shares the card, and the measured drop ranges from a few percent to more than sixty percent depending on what it runs alongside. To promise customers a performance level (a service-level agreement, or SLA) and still pack functions tightly, operators must be able to predict how much a function will slow down under contention. Without an accurate predictor, they either overprovision and waste the card, or pack tightly and break SLAs.
-
What people did
Reuse the performance predictors built for CPU servers
There is a mature line of work on predicting performance interference for workloads co-located on CPU servers, such as SLOMO. These predictors profile how workloads stress shared resources and learn a model that forecasts the slowdown when they run together. The natural move is to point the same machinery at network functions running on a SmartNIC.
-
Why it fell short
CPU-oriented predictors miss the SmartNIC's resources and the role of traffic
The paper shows these predictors are inaccurate for on-NIC functions because they lack SmartNIC-specific knowledge and are barely traffic-aware. On a SmartNIC, contention is not for one resource but several at once — the onboard hardware accelerators and the memory subsystem — and different functions use those resources with different execution patterns: one runs as a pipeline for throughput, another waits for each dispatched request to finish (run-to-completion) for low latency. So naively composing single-resource models — summing their predicted losses, or taking the maximum — gives the wrong answer, because how a resource's contention affects end-to-end performance depends on the execution pattern. On top of that, the external traffic matters: the number of flows and the packet size change how hard each resource is stressed, and a predictor blind to traffic loses accuracy the moment the traffic shifts. CPU-world predictors capture none of this SmartNIC structure.
-
This paper's idea
Model each onboard resource with SmartNIC knowledge, compose by pattern, add traffic
Yala is a performance predictor built specifically for on-NIC network functions, on the observation that co-located functions contend for multiple onboard resources and that traffic attributes change the contention. It builds a per-resource model for each contended resource: for the hardware accelerators, a white-box queueing model, justified by the observation that functions reach an accelerator through their own queues coordinated by round-robin scheduling; for the memory subsystem, a black-box ensemble machine-learning model as prior work does. It then joins them with execution-pattern-based composition, which combines the per-resource effects according to whether a function runs as a pipeline or run-to-completion, rather than blindly summing. Layered on top is traffic-aware augmentation: it folds traffic attributes such as flow count and packet size into the per-resource models — expressing the queueing model's parameters as functions of traffic for the accelerators, and fusing traffic features into the memory model — and uses adaptive profiling to prune attribute dimensions and sample the performance-critical ranges so profiling stays cheap. On BlueField-2 SmartNICs, Yala reaches an average prediction error of 3.7 percent, a 78.8 percent improvement over the state of the art, and cuts SLA violations by 92.2 percent.
Why this problem is worth solving
SmartNICs have become central infrastructure, and the whole economic case for them rests on packing them efficiently. A network function is a piece of packet-processing logic — a firewall filtering traffic, an encryptor securing it, a load balancer steering it, a monitor inspecting it. Traditionally these run on the host CPU, consuming cores that the operator would rather sell to tenants for their applications. A SmartNIC is a network card with its own general-purpose cores, specialized hardware accelerators (for tasks like encryption or compression), and memory, so the network functions can be offloaded onto the card itself. This frees host CPU cycles and gives a programmable place to run network logic close to the wire. But a SmartNIC is expensive and has finite onboard resources, so running a single lightly loaded function on each card would waste most of its capacity and defeat the cost savings that justified buying it. The economic point is therefore to consolidate: run many network functions on each SmartNIC, keeping it busy. That consolidation is exactly what creates the interference problem — the more functions you pack together, the more they compete for the card’s accelerators and memory, and the more each one slows down. So the value of the SmartNIC and the danger of interference are two sides of the same coin, and being able to predict the interference is what lets an operator capture the value without violating the performance guarantees it sold.
The reason prediction is hard, and worth a dedicated system, is that a SmartNIC is not a small server. Performance-interference prediction is well studied for CPU servers, where the shared resources are things like the last-level cache and memory bandwidth, and a learned model can forecast slowdown reasonably well. A SmartNIC breaks the assumptions those models rest on in two ways. First, the contended resources are different and plural: on-NIC functions fight over the card’s hardware accelerators and its memory subsystem simultaneously, and these behave very differently from a CPU cache. Worse, functions use them with different execution styles — a throughput-oriented function pipelines its work across stages, while a latency-oriented one dispatches a request to an accelerator and waits for it to complete before moving on — so the same amount of contention on the same resource produces different end-to-end effects depending on the function’s structure. That makes the obvious trick of building one model per resource and adding up their predictions unreliable. Second, the workload is traffic: the packets flowing through change how hard each resource is exercised — many small packets stress different parts than few large ones — so a prediction that ignores the current traffic attributes drifts out of date as soon as traffic changes, which in a network it constantly does. A predictor that treats a SmartNIC like a CPU, or ignores the traffic, is therefore doomed to be inaccurate, which is why capturing the card’s specific structure is the crux.
What people did before, and where it broke
- CPU interference predictors (SLOMO and similar). Profile how co-located workloads stress shared resources and learn to predict their slowdown, as done for CPU servers. Where it broke. They lack SmartNIC-specific knowledge of the onboard accelerators and memory subsystem and are barely traffic-aware, so their accuracy for on-NIC network functions is poor.
- Single-resource modeling. Model contention for one dominant resource and predict from that. Where it broke. On a SmartNIC, functions contend for several resources at once (accelerators and memory), so a one-resource model misses most of the interference.
- Sum composition of per-resource models. Build a model per resource and add up the predicted throughput losses. Where it broke. It ignores the execution pattern, so it overcounts or miscounts, because how each resource’s contention affects end-to-end performance depends on whether the function is a pipeline or run-to-completion.
- Min (maximum-loss) composition. Take the largest single-resource predicted loss as the overall prediction. Where it broke. It also ignores execution patterns and the interplay of resources, reducing error over summing but still failing to be accurate across cases.
- The gap. No predictor combined SmartNIC-specific per-resource models — a white-box queueing model for the round-robin accelerator queues and a learned model for memory — composed according to each function’s execution pattern and augmented with the current traffic attributes, which is what accurate on-NIC prediction actually requires.
How Yala works
Predicting an on-NIC function's slowdown from resource models, execution pattern, and traffic
-
-
new
-
new
-
new
-
new
-
new
The idea
Use the right model for each resource, because the SmartNIC’s structure is known where it counts. The first move in Yala is to reject a single monolithic black box and instead model each contended resource with the approach that fits it. The onboard accelerators have an exploitable structure — functions reach them through per-function queues arbitrated by round-robin scheduling — so Yala builds a white-box queueing model that derives the slowdown from that mechanism, which is more accurate and more interpretable than learning it blindly. The memory subsystem lacks such clean structure, so it gets a learned ensemble model. Building a faithful predictive model of how a shared service behaves under load, so operators can reason about performance before committing resources, is the same goal the work on end-to-end application cloning for networked cloud services covered elsewhere on this site pursues, constructing an accurate stand-in for the real system’s performance. Yala’s distinctive choice is to make that model white-box exactly where the hardware’s scheduling discipline is known, and learned only where it is not.
Compose the pieces by how the function actually runs. Having a model per resource is not enough, because the naive ways to combine them — add the losses, or take the biggest — assume the resources’ effects are independent of the function’s structure, and they are not. A pipelined function overlaps stages so a bottleneck in one resource may be partly hidden, while a run-to-completion function serializes on each dispatched request so contention hits it directly; the same contention yields different end-to-end slowdowns. Yala’s execution-pattern-based composition therefore combines the per-resource predictions according to whether the function pipelines or runs to completion. Recognizing that co-located workloads interfere through shared hardware and that predicting the outcome requires understanding how each workload uses that hardware is the same concern the work on QoS-aware, resource-efficient multi-stage serving on shared accelerators covered elsewhere on this site is built on, matching the interference model to how work flows through the system. Yala turns that matching into an explicit composition step keyed on the execution pattern.
Make the model track the traffic, without paying to profile everything. The last piece is that a network function’s performance is not a fixed property; it depends on the packets flowing through it, so the predictor must be traffic-aware. Yala folds traffic attributes like flow count and packet size into its models — parameterizing the white-box queue by traffic and adding traffic features to the black-box memory model — so predictions stay accurate as traffic varies. But traffic attributes multiply the space to profile, so Yala uses adaptive profiling to prune dimensions that do not matter and sample densely only where performance is sensitive. Coping with the constant variability of a shared, noisy operating environment by profiling and adapting intelligently rather than exhaustively is the same discipline the work on tuning applications in noisy clouds by playing tournaments covered elsewhere on this site pursues, spending measurement effort where it changes the outcome. Yala applies that thrift to the traffic dimensions of on-NIC performance, keeping the model current at an affordable profiling cost.
Following a prediction through Yala
Forecasting a firewall's throughput when it shares a SmartNIC
A trace, drawn from the paper, of how Yala predicts a target network function's performance under contention and given the current traffic.
Step 1 of 5
Identify the target function, its neighbors, and the traffic
An operator wants to know how a target network function (say a firewall) will perform when co-located with other functions on a SmartNIC, under the current traffic. Yala takes the set of co-running functions and the traffic attributes — flow count, packet size — as inputs.
The prediction is for the target under this specific contention and traffic, not in isolation.
Why it works
- A white-box queue matches the accelerators’ real behavior. A black-box model must learn the accelerator’s sharing dynamics from scratch → but the functions actually contend through round-robin queues, a known mechanism → so a queueing model captures the slowdown accurately and interpretably instead of approximating it.
- Per-resource models cover the plural contention. A single-resource predictor misses that on-NIC functions fight over several resources → but modeling the accelerators and the memory subsystem separately covers both → so the prediction reflects all the major sources of interference, not one.
- Composition by execution pattern captures the real end-to-end effect. Summing or maxing per-resource losses ignores how the function is structured → but composing them by pipeline versus run-to-completion reflects how contention becomes slowdown → so the combined prediction is right where naive composition is wrong.
- Traffic-aware augmentation keeps predictions valid as traffic shifts. A model fixed at one traffic mix drifts when flows and packet sizes change → but folding traffic attributes into both the queueing and memory models tracks the change → so accuracy holds under the varying traffic a network actually sees.
- Adaptive profiling makes traffic-awareness affordable. Profiling every combination of traffic attributes would be prohibitively expensive → but pruning unimportant dimensions and sampling the performance-critical ranges cuts the cost → so the richer, traffic-aware models can be built without exhaustive measurement.
What it costs (trade-offs and scope)
- It needs offline profiling of the functions and resources. Yala builds its per-resource models and traffic augmentation from profiling runs, so deploying it requires measuring the network functions on the SmartNIC beforehand; adaptive profiling reduces but does not eliminate this cost, and new functions need profiling to be predicted.
- The white-box model assumes the accelerators’ queueing discipline. The queueing model’s accuracy rests on the observation that functions reach accelerators through their own round-robin-scheduled queues. Hardware whose accelerators are shared by a different mechanism would need the model reconsidered, so the white-box benefit is tied to that architectural assumption.
- It is specific to on-NIC network functions and this class of SmartNIC. Yala is designed around SmartNIC resources (onboard accelerators and memory) and evaluated on BlueField-2 with common network functions; the exact models and numbers are tied to that setting, and generalization to other cards or non-NF workloads is not the paper’s claim.
- It predicts throughput under contention, not every metric. The system targets NF throughput prediction under multi-resource contention and traffic variation; other performance aspects, or contention sources outside the modeled accelerators and memory, are outside the modeled scope.
- The reported gains are relative to specific baselines. The 3.7 percent average error, 78.8 percent accuracy improvement over SLOMO, and 92.2 percent reduction in SLA violations are measured against particular state-of-the-art and greedy baselines on the paper’s testbed and workloads; other setups would shift the numbers.
The evaluation
Predicting on-NIC network function performance under contention
| Property | CPU-oriented predictor | Naive per-resource composition | Yala |
|---|---|---|---|
| SmartNIC-specific knowledge | |||
| Handles multi-resource contention | |||
| Accounts for execution pattern | |||
| Traffic awareness | |||
| Reported outcome |
Check yourself
Check your understanding
1. Why do operators need to predict on-NIC network function performance, and why is it hard?
2. What is Yala's core approach?
3. Why does Yala compose per-resource models by execution pattern instead of just summing their predicted losses?
4. What did Yala achieve, and what are its main assumptions and limits?
Sources
The paper is “Performance Prediction of On-NIC Network Functions with Multi-Resource Contention and Traffic Awareness” by Shaofeng Wu, Qiang Su, Zhixiong Niu, and Hong Xu (The Chinese University of Hong Kong and Microsoft Research), published at ASPLOS 2025, DOI 10.1145/3669940.3707232. The system is named Yala. It is licensed CC BY.
This explainer is written from the paper’s full text (arXiv:2405.05529, the extended version of the ASPLOS 2025 paper). The paper addresses performance prediction for network functions offloaded onto SmartNICs, where co-located functions cause interference by contending for onboard resources, and existing solutions lack SmartNIC-specific knowledge and traffic awareness. Yala builds per-resource contention models — a white-box queueing model for hardware accelerators (motivated by the observation that functions interact with accelerators through their own queues coordinated by round-robin scheduling) and a black-box ensemble machine-learning model for the memory subsystem — and joins them with execution-pattern-based composition that accounts for whether a function runs as a pipeline or run-to-completion, because the execution pattern determines how resource contention affects end-to-end performance. It adds traffic-aware augmentation, integrating traffic attributes such as flow count and packet size into the per-resource models (parameterizing the white-box queueing model by traffic and fusing traffic features into the black-box memory model), and uses adaptive profiling to prune attribute dimensions and sample performance-critical ranges to curb profiling cost. Evaluated on BlueField-2 SmartNICs over nine common network functions, Yala reports an average prediction error of 3.7 percent (a 78.8 percent improvement over the state-of-the-art SLOMO) and reduces SLA violations by up to 92.2 percent in NF placement (88.5 percent versus greedy approaches), enabling contention-aware scheduling and performance diagnosis. No numbers beyond the paper’s have been introduced; exact model formulations and the full evaluation should be confirmed in the paper, and an artifact is available.