Enabling Fast Networking in the Public Cloud

Alireza Sanaee, Vahab Jabrayilov, Ilias Marinos, Farbod Shahinfar, Divyanshu Saxena, Gianni Antichi and 1 other

Fast "kernel-bypass" networking stacks can cut the latency of talking between machines, but they only work if the network card exposes special features — and the virtual network cards that cloud tenants get in their VMs deliberately hide those features, so a decade of fast-networking research has been unusable for most cloud users; this paper (Cambridge, Columbia, NVIDIA, Politecnico di Milano, UT Austin, Queen Mary University of London — Alireza Sanaee, Kostis Kaffes, Gianni Antichi, and colleagues) presents Machnet, a userspace network stack that relies only on the minimal feature set every cloud offers and uses a microkernel-style design for flexibility, getting nearly the latency of stacks that need special hardware.

The argument in one glance

Problem

Fast networking research assumes hardware that cloud tenants cannot touch

Cloud applications constantly talk to other machines — distributed databases, caches, stream processors — and that inter-machine communication needs to be fast. To cut latency, developers reach for kernel-bypass (or "userspace") networking stacks like mTCP, eRPC, and Demikernel, which skip the operating system's slow network path and let the application talk almost directly to the network card. These stacks are the product of a decade of research and can be very fast. But there is a catch that makes most of them unusable in the setting where most cloud software actually runs: a virtual machine (VM) in the public cloud. The reason is the network card. A cloud tenant does not get a real, bare-metal network card; it gets a virtual network card (vNIC), and cloud providers deliberately expose only a restricted, uniform feature set on those vNICs — both because they must virtualize the network for many tenants, and because they must present a consistent interface across a fleet of physical cards spanning more than a decade of hardware generations. Crucially, the fast userspace stacks depend on advanced card features — flow steering (mapping network flows to specific CPU cores), configurable receive-side scaling, remote DMA, deep receive queues — that cloud vNICs simply do not offer. So the research that was supposed to make networking fast assumes hardware capabilities that the typical cloud tenant will never have.
What people did

Use existing userspace stacks, built for bare-metal cards

The natural move is to take one of the existing high-performance userspace stacks and run it in the cloud VM. These stacks are built around a bare-metal network card and its advanced features: flow steering to spread flows across cores, direct memory access from application memory, remote DMA, and large receive queues. They also come in two architectural styles — a library operating system that links the stack into the application, or a tightly co-designed sidecar that squeezes out performance by fusing the scheduler, queues, and stack together.
Why it fell short

Missing card features and rigid designs both block cloud use

Both the hardware assumptions and the software architecture fail in the cloud. On the hardware side, the vNIC lacks the features these stacks require. Most tellingly, no leading cloud provider's vNIC supports flow steering, a feature standard on bare-metal cards since 2009 and used by the stacks to map flows to cores so they can scale across a machine's CPUs; without it, a stack cannot spread its work over cores the way it was designed to. Newer cards might support these features, but the fleet is full of older cards that are financially impractical to replace, so the features will not be universally available soon. On the software side, the rigid execution models do not fit real cloud applications. The library-operating-system approach makes one application "own" the card and ties each thread to one of the few available card queues, but a cloud server often runs many processes and many threads — one workload hosts thousands of database instances on a single VM, far more than there are cores — so a model that limits threads to queues, and requires constant polling, simply does not work. The heavily co-designed sidecars, meanwhile, are hard to deploy and tune for non-experts. What was missing was a fast userspace stack that needs only the features every cloud vNIC actually provides and that flexibly supports the messy reality of cloud applications.
This paper's idea

Build on the least common denominator, with a microkernel sidecar

Machnet is a high-performance, easy-to-use userspace network stack designed for public-cloud VMs, taking the stance of a tenant who cannot modify the vNIC. It rests on three ideas. First, it defines a Least Common Denominator (LCD) vNIC model: the minimal feature set that any cloud vNIC provides, which the paper shows is just plain Ethernet packet I/O, opaque receive-side scaling (the card hashes flows to queues but you cannot see or set the hash), at most one transmit/receive queue per CPU core, and at most 256 descriptors per queue. This LCD is effectively the practical "network device instruction set" for the public cloud, and Machnet uses only it, so it runs on every major provider. Second, because the LCD cannot do flow steering, Machnet introduces RSS--, a flow-to-core mapping technique that works with only opaque receive-side scaling: since it cannot control which core a flow lands on, it repeatedly tries different source and destination ports during connection setup until a combination happens to hash to the desired core (about 25 to 55 setup packets for four to eight cores, with a timeout-and-retry fallback), then remembers the successful ports. Third, Machnet adopts a microkernel-style sidecar architecture: a separate sidecar process owns the card and mediates access for many application processes and threads through a thin shim library and shared memory, supports blocking (interrupt-driven) receive so threads can sleep and be woken, offers a BSD-socket-like API with high-level language bindings, and deploys as a container. The extra hop through the sidecar costs about one microsecond, negligible against real cloud-network round trips. Evaluated on AWS, Azure, and Google Cloud with a key-value store, an HTTP server, and a state-machine replication system, Machnet achieves latency almost as low as stacks that require the advanced hardware cloud tenants cannot get, and it can opportunistically use advanced features when a card exposes them.

Why this problem is worth solving

Almost all cloud software runs in VMs and depends on fast inter-machine communication, so a fast stack that actually works there unlocks a decade of research for the majority of users.

The default way to deploy software in the public cloud is inside a virtual machine, and the applications that matter most — distributed data stores, caches, stream processors, the backends of productivity tools — are constantly communicating across machines, where networking latency directly shapes performance. A great deal of research has produced userspace stacks that make this communication fast, but almost none of it is usable by an ordinary cloud tenant, because it assumes hardware the tenant cannot access. That is a striking gap: the people with the most to gain from fast networking, and the setting where most computing happens, are exactly the ones the research has left behind. Closing the gap means those tenants can get low-latency communication without renting special bare-metal instances or waiting for the entire cloud fleet to upgrade its network cards. Because the LCD feature set is common to every major provider, a stack built on it is portable across clouds, so the benefit is broad rather than tied to one vendor. Making fast networking practical for the common cloud deployment is therefore a large, widely-applicable win.

The difficulty is a double mismatch — with the virtualized hardware and with how cloud applications are actually structured — and both must be solved together.

There are two independent reasons the existing stacks fail in the cloud, and fixing only one is not enough. The hardware mismatch is that cloud vNICs deliberately expose a minimal, uniform feature set to keep network virtualization manageable across a fleet of cards from many vendors and generations, so the advanced features the fast stacks rely on — above all flow steering, which lets a stack decide which core each flow goes to — are simply not there. A stack that assumes those features cannot even scale across the VM’s cores. The software mismatch is that the fast stacks impose rigid execution models: the library-operating-system style makes a single application own the card and binds threads to a tiny number of card queues, and requires busy-polling, none of which fits a cloud server running many processes and far more threads than cores, some mostly idle. You cannot get around this by attaching more vNICs, because providers cap that at a handful. So a practical stack has to both live within the minimal hardware interface and support a flexible execution model where many applications and threads share the card and can sleep when idle. These two constraints pull on different parts of the design — the hardware interface and the process architecture — which is why the paper needs both a least-common-denominator model and a microkernel-style sidecar, not just one.

What people did before, and where it broke

Run a bare-metal userspace stack (mTCP, eRPC, Demikernel) in the cloud VM. Reuse an existing fast stack unchanged. Where it broke. It depends on advanced vNIC features — flow steering, configurable receive-side scaling, remote DMA, deep queues — that cloud vNICs do not expose, so it cannot run or cannot scale.
Use flow steering to map flows to cores. Scale across CPUs by steering each flow to a chosen core. Where it broke. No leading cloud provider’s vNIC supports flow steering, so this core-scaling mechanism is unavailable to tenants entirely.
Use the library-operating-system model. Statically link the stack into the application for speed. Where it broke. One application owns the card and each thread needs a card queue, but cloud servers run many processes and far more threads than the few queues or cores allow, and busy-polling wastes those cores.
Use a heavily co-designed sidecar. Fuse the scheduler, queues, and stack to extract maximum performance. Where it broke. Such systems are hard to deploy and tune for non-experts and often assume specifics of the application, which is impractical for typical cloud tenants.
The gap. No fast userspace stack used only the features every cloud vNIC provides while flexibly supporting the diverse process-and-thread structure of real cloud applications.

How Machnet works

A fast userspace stack that lives within the cloud's minimal interface

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Get high performance using only the commodity capabilities everyone actually has. Machnet’s founding decision is to assume nothing the tenant cannot get: it distills the minimal feature set common to every cloud vNIC and builds a fast stack on that alone, rather than requiring special hardware. Achieving a demanding goal on ordinary, widely-deployed hardware, without depending on special-purpose features, is the same undertaking the work on software radiation protection for commodity hardware in space (RadShield) covered elsewhere on this site pursues, delivering a hard capability on commodity parts instead of exotic ones. Machnet does this for cloud networking, treating the least common denominator vNIC as the fixed substrate and engineering performance within it, which is what makes it portable across every major provider.

Choose a flexible architecture so one design fits many uses. Rather than fuse everything for peak speed on one workload, Machnet runs the stack as a separate microkernel-style sidecar that many applications and threads share, trading a tiny overhead for the flexibility to serve the diverse, messy reality of cloud software. Structuring a system as a separated, modular component to gain flexibility and isolation, rather than statically fusing it into the application, is the same principle the work on flexible operating-system isolation (FlexOS) covered elsewhere on this site is built on, letting architecture provide flexibility instead of a rigid monolith. Machnet’s sidecar lets many processes share the card, supports blocking receive so idle threads sleep, and exposes a familiar API, none of which a library-operating-system design could offer.

Wring fast inter-machine communication out of a constrained setting. The whole point of Machnet is to make talking between machines fast despite the cloud’s restrictions, cutting latency where a naive path through the operating system would be slow. Extracting efficient inter-machine communication in a datacenter, working within what the environment allows rather than assuming ideal primitives, is the same idea the work on combining collective and one-sided communication for efficiency (Two-Face) covered elsewhere on this site develops, squeezing performance from the communication substrate at hand. Machnet delivers this with kernel bypass on the LCD vNIC plus RSS— for core scaling, reaching latency close to stacks that require hardware cloud tenants cannot have.

Following a connection through Machnet

Setting up a fast, core-scaled connection on a cloud vNIC

A trace, drawn from the paper, of how Machnet establishes and runs a connection using only least-common-denominator features.

Step 1 of 5

An application asks the sidecar to connect

The application uses Machnet's BSD-socket-like API (connect, send, recv) through a thin shim library. The request goes to the Machnet sidecar, the separate process that owns the vNIC and does all kernel-bypass packet I/O.

The sidecar mediates the card for many applications and threads at once.

Why it works

Building on the least common denominator makes the stack portable. Bare-metal stacks assume features cloud vNICs lack → but Machnet uses only the minimal set every provider offers → so the same stack runs unmodified across AWS, Azure, and Google Cloud.
RSS— replaces flow steering with a randomized search. The cloud card will not let you control which core a flow hits → but trying ports until one hashes to the target core, then remembering it → gives stable flow-to-core mapping using only opaque receive-side scaling.
A sidecar lets many applications share the few card queues. A library-operating-system stack lets one app own the card and binds threads to queues → but a shared sidecar mediates the card for all processes and threads → so the scarce queues no longer cap how many applications and threads a VM can run.
Blocking receive frees threads from busy-polling. Polling ties the number of threads to the number of cores and burns idle cores → but the sidecar wakes sleeping threads on message arrival → so a VM can host far more threads than cores, most of them cheap.
The sidecar hop is negligible in a real cloud. Adding an inter-process hop sounds costly → but it is about one microsecond against cloud round trips of hundreds of microseconds → so the flexibility comes at almost no latency cost where it matters.

What it costs (trade-offs and scope)

RSS— adds connection-setup overhead and is probabilistic. Because it searches for ports that hash correctly, setting up a connection takes tens of extra packets (about 25 to 55 for four to eight cores), and a small fraction of flows need a timeout and retry; this cost is paid once per connection at setup, not per message, but it is real and grows with the number of cores.
It deliberately sacrifices peak message rate for latency and portability. Machnet targets low latency and broad compatibility rather than the tens-of-millions-of-messages-per-second that some bare-metal stacks chase, on the observation that cloud applications are usually far less message-rate-demanding; a workload that truly needs that extreme throughput is not the target.
The sidecar hop costs about a microsecond. The microkernel architecture adds an inter-process hop; this is negligible in a real cloud but is a meaningful fraction of latency on tiny bare-metal testbeds, so the design is tuned for cloud-scale networks rather than the smallest local setups.
It uses only least-common-denominator features by design. Machnet forgoes zero-copy DMA, remote DMA, and other advanced capabilities to stay portable; it can opportunistically use them when a card exposes them, but its baseline gives up the extra performance those features could provide.
It assumes the vNIC offers kernel-bypass packet I/O. Machnet builds on the LCD feature set (via DPDK or AF_XDP) available across major clouds; settings whose virtual cards do not provide even that minimal kernel-bypass interface would fall outside its scope.

The evaluation

Userspace networking for public-cloud VMs

Property	Bare-metal userspace stacks	Linux kernel networking	Machnet
Runs on cloud vNICs
Scales across cores
Execution flexibility
Latency
Portability across clouds

A comparison of networking options for public-cloud VMs. By building only on the least-common-denominator vNIC feature set, mapping flows to cores with RSS-- (opaque receive-side scaling plus a randomized port search instead of flow steering), and using a shared microkernel sidecar for flexible execution, Machnet runs across AWS, Azure, and Google Cloud with latency almost as low as bare-metal userspace stacks that require hardware cloud tenants cannot access. Specific per-application figures should be read from the paper.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer is based on the paper’s full text (the authors’ copy hosted at Columbia, of the ASPLOS 2026 paper). The system described is the paper’s: Machnet is a high-performance, flexible userspace network stack for public-cloud VMs that uses only a minimal set of vNIC features supported by any major cloud provider and a microkernel architecture for flexible application execution. It defines a Least Common Denominator (LCD) vNIC model — plain Ethernet packet I/O, opaque receive-side scaling, at most one transmit/receive queue per CPU core, and at most 256 descriptors per queue — and introduces RSS—, a flow-to-core mapping technique that works with only opaque receive-side scaling by trying different source and destination ports at connection setup until one hashes to the desired queue (the paper reports about 25 packets for four engines and 55 for eight to reach roughly 95 percent connection-setup probability, with timeout-and-retry for the rest). Machnet’s sidecar process owns the card and mediates access for many processes and threads via a shim library and shared memory, supports interrupt-based blocking receive, exposes a BSD-socket-like API with high-level language bindings, and deploys as a container; the extra inter-process hop is reported at about one microsecond, small relative to cloud round trips. The paper evaluates Machnet on AWS, Azure, and Google Cloud with a key-value store, an HTTP server, and a state-machine replication system, reporting latency almost as low as stacks that rely on advanced hardware, and improvements over Linux TCP (for example, reported reductions in median latency for the web server). Machnet is open-sourced. Exact mechanisms, the full API, and all quantitative results should be read from the paper.

Background

Why the cloud keeps forcing a rethink of systems built for bare metal

It helps to see Machnet within a broad pattern: techniques that work beautifully on hardware you fully control often break when that hardware is virtualized and shared, and the public cloud is where this bites hardest. Virtualization exists to multiplex one physical machine among many tenants and to present a stable interface over a churning fleet of hardware, and both goals push providers to expose a narrower, more uniform interface than the raw hardware offers. Systems research, meanwhile, tends to chase peak performance by exploiting the newest, most specialized hardware features — which are exactly the features virtualization is most likely to hide. The result is a recurring gap between what the literature assumes and what a cloud tenant can actually use, and closing it requires rethinking the system around the constrained, portable interface rather than the ideal one. This is not unique to networking; it recurs wherever cloud abstractions sit between software and hardware. The broader effort to understand and optimize how network functions behave within the realities of cloud infrastructure is the same terrain the work on performance prediction of on-NIC network functions covered elsewhere on this site occupies, grappling with networking under the cloud’s constraints. Machnet’s contribution is to show that fast userspace networking need not be a bare-metal privilege: by naming the least common denominator interface and engineering cleverly within it — most strikingly, replacing hardware flow steering with a randomized port search — a stack can be both fast and portable across every major cloud.

If you remember only three things

1. A decade of fast kernel-bypass (userspace) network stacks stays impractical for public-cloud tenants, for two reasons. The virtual network cards tenants get in their VMs deliberately expose only a minimal, uniform feature set — no flow steering (mapping flows to cores, standard on bare-metal since 2009), no configurable receive-side scaling, no remote DMA — that the fast stacks depend on. And those stacks use rigid execution models (one application owns the card, threads bound to a few card queues, constant polling) that do not fit cloud servers running many processes and far more threads than cores.
2. Machnet is a fast, flexible userspace stack for cloud VMs built on three ideas. It defines a Least Common Denominator vNIC model — plain Ethernet packet I/O, opaque receive-side scaling, one transmit/receive queue per core, at most 256 descriptors per queue — and uses only that, so it runs on every major cloud. It replaces flow steering with RSS--, which maps flows to cores by trying source and destination ports at connection setup until one hashes to the target core (about 25 to 55 setup packets for four to eight cores), then remembers it. And it uses a microkernel-style sidecar that owns the card and lets many applications and threads share it, supports blocking receive so idle threads sleep, and offers a BSD-socket-like API.
3. Across AWS, Azure, and Google Cloud, evaluated on a key-value store, an HTTP server, and a state-machine replication system, Machnet achieves latency almost as low as bare-metal userspace stacks that need hardware cloud tenants cannot access, improving over Linux TCP, with the sidecar hop costing about one microsecond (negligible against cloud round trips). The trade-offs: RSS-- adds per-connection setup overhead and is probabilistic, Machnet deliberately favors latency and portability over peak message rate, and it forgoes advanced features by default (using them only opportunistically when a card exposes them). It is open-sourced at github.com/microsoft/machnet.

Check yourself

Check your understanding

1. Why are fast userspace (kernel-bypass) network stacks usually unusable for public-cloud tenants?
2. What is the "Least Common Denominator" (LCD) vNIC model, and why does Machnet build on it?
3. How does Machnet map network flows to CPU cores without flow steering, using RSS--?
4. Machnet runs its network stack in a separate "sidecar" process, adding an inter-process hop of about one microsecond. Why is this a good trade-off for cloud VMs?

Sources

The paper is “Enabling Fast Networking in the Public Cloud” by Alireza Sanaee, Vahab Jabrayilov, Ilias Marinos, Farbod Shahinfar, Divyanshu Saxena, Gianni Antichi, and Kostis Kaffes (University of Cambridge and Huawei, Columbia University, NVIDIA, Politecnico di Milano, The University of Texas at Austin, and Queen Mary University of London), published at ASPLOS 2026, DOI 10.1145/3779212.3790158. It is licensed CC BY, with an author copy hosted at Columbia and the system open-sourced at github.com/microsoft/machnet.

This explainer is written from the paper’s full text (the authors’ hosted copy). Machnet is a high-performance, flexible userspace network stack for public-cloud VMs that addresses two root causes of prior stacks’ impracticality: reliance on specialized NIC features absent in commodity cloud vNICs, and rigid execution models ill-suited to diverse application needs. It models a Least Common Denominator (LCD) vNIC — plain Ethernet packet I/O, opaque receive-side scaling, at most one transmit/receive queue per CPU core, and at most 256 descriptors per queue — and uses only these features. To scale across cores without flow steering (which no leading cloud vNIC supports), it introduces RSS—, which tries different source and destination ports at connection setup until one hashes to the desired queue (about 25 packets for four engines and 55 for eight to reach roughly 95 percent probability, with timeout-and-retry). It adopts a microkernel sidecar that owns the card and mediates access for many processes and threads, supports interrupt-based blocking receive, exposes a BSD-socket-like API with high-level language bindings, and deploys as a container, at an extra inter-process cost of about one microsecond. Evaluated across AWS, Azure, and Google Cloud on a key-value store, an HTTP server, and a state-machine replication system, the paper reports Machnet achieves latency almost as low as stacks requiring advanced hardware, and improves over Linux TCP. Numbers are the paper’s own; exact mechanisms, the API, and full results should be confirmed in the paper.