FUYAO: DPU-enabled Direct Data Transfer for Serverless Computing

Guowei Liu, Laiping Zhao, Yiming Li, Zhaolin Duan, Sheng Chen, Yitao Hu and 2 others

Serverless apps are built from tiny functions chained into workflows, and the output of one function is the input of the next — but because functions are isolated sandboxes that don't know where each other live, today's platforms pass that data the slow way, through storage or the orchestration controller, which copies it twice, saturates the controller's CPU, and can eat over 70% of an app's total latency; this paper (Tianjin University and USC) builds FUYAO, which decouples control flow from data flow and lets function pairs open persistent DIRECT transfer channels that bypass the middleman — on the same machine via shared memory, and across machines via a DPU (a SmartNIC with its own processor and RDMA) that shuttles data straight into the destination function's memory without ever touching the host CPU — cutting end-to-end latency from 50 milliseconds to under 13, slashing control-plane CPU by 80 to 95%, and raising throughput by one to two orders of magnitude.

The argument in one glance

Problem

Chained serverless functions must pass data, but they cannot hand it over directly

Serverless computing (Functions-as-a-Service) lets you build an application as a chain of tiny, short-lived functions that the cloud runs on demand — no servers to manage. A serverless WORKFLOW threads these functions together: function A produces a result that function B consumes, and so on through a pipeline (a real media-processing app might chain eight functions with eleven data hand-offs). The snag is how the data gets from A to B. Functions are deliberately ISOLATED — each runs in its own sandbox — and STATELESS, and they do not even know where each other are running. So they cannot simply pass data to one another; every hand-off has to go somewhere in between.
What people did

Route the data through storage or the controller

Today's platforms move the intermediate data through a THIRD PARTY. Function A writes its output to remote storage (like S3 or Redis) or hands it to the orchestration controller; the controller then notifies function B that data is ready; and B reads the data back from storage. This works with the isolation and statelessness of functions — nobody holds a persistent connection, and the controller stays in charge of orchestration — and it is how mainstream serverless frameworks (OpenFaaS and others) pass data along a workflow.
Why it fell short

The middleman copies twice, saturates the controller, and burns host CPU

Routing every hand-off through a third party is triply wasteful. First, REDUNDANT COPIES: the data travels sender to storage to receiver, two copies where zero were needed. Second, a CONTROL-PLANE BOTTLENECK: the controller forwards every transfer and becomes the throughput ceiling — at 500 requests per second, an OpenFaaS controller consumes more than 6 CPU cores and 86% of the CPU, and its forwarding latency balloons to 52 milliseconds. Third, HOST-CPU OVERHEAD: moving each byte burns the host cores that should be running functions. The result is that data transfer, not computation, dominates — over 70% of end-to-end latency in a real media pipeline, and a small transfer through storage takes over 3 milliseconds where a direct transfer needs 0.07. This is fatal for serverless, whose whole premise is short, cheap functions: the transfer overhead dwarfs the work. The deep flaw is that the design COUPLES control flow (orchestration messages) with data flow (the actual bytes), forcing the data through the orchestrator.
This paper's idea

Decouple control from data, and transfer directly — through shared memory and a DPU

This paper (Tianjin University and USC — Liu, Zhao, Li, and colleagues) builds FUYAO, which DECOUPLES control flow from data flow so the bytes never go through the orchestrator. Once two functions are introduced, they open a persistent DIRECT TRANSFER CHANNEL (DTC) — set up by a one-time handshake, then reused for every later invocation until released — over which data flows straight from sender to receiver. On the SAME machine, the DTC is a shared-memory FIFO: A writes, B reads, no controller, no copies, in under 72 microseconds regardless of data size. ACROSS machines, FUYAO enlists a DPU (a SmartNIC with its own ARM cores, memory, and RDMA): its inter-node engine runs ON the DPU, polls remote memory over one-sided RDMA, orchestrates the handshake, and then the source function writes data directly into the destination's registered memory via RDMA — all without touching the host CPU. A zero-copy allocator hides the RDMA plumbing, and the runtime automatically picks among four transfer methods, building a DTC only when a function pair is called often enough to amortize the handshake. Across real workflows, FUYAO cuts end-to-end latency from 50 milliseconds to under 13, slashes control-plane CPU by 80 to 95%, and raises throughput by one to two orders of magnitude.

Why this problem is worth solving

Serverless is one of the fastest-growing ways to build cloud applications — Amazon reports over 100 trillion monthly serverless calls across its analytics services — yet it stumbles on latency-sensitive workflows for a reason that is entirely about plumbing. Understanding that plumbing comes first. A

a serverless WORKFLOW chains Functions-as-a-Service into a pipeline where each function’s output feeds the next. The functions are isolated (each in its own sandbox or container), stateless (they keep nothing between invocations), and location-unaware (they do not know which machine a peer runs on), which is what lets the platform schedule and scale them freely. The cost of that freedom is that a function cannot hand data straight to the next one; the platform routes it through a THIRD PARTY — external storage or the orchestration controller. So a single hand-off becomes: sender writes the data to storage, the controller signals the receiver, the receiver reads the data back. That indirection is convenient for orchestration but ruinous for performance, because the same bytes are copied twice, every transfer passes through the controller (which becomes a CPU and throughput bottleneck), and the host CPU is spent moving data rather than running functions

is the setup. Because functions are short, the transfer overhead is not a rounding error — it is the majority of the latency, which is why fixing the data path is what makes serverless viable for real-time services.

The hardware that makes a direct path possible without spending host CPU is worth understanding too. A

a DPU (Data Processing Unit), also called a SmartNIC, is a network card that is also a small computer: it carries its own multi-core processor (typically ARM), its own memory, an RDMA-capable network adapter, and direct access to host memory over PCIe. The key capability is RDMA (Remote Direct Memory Access), which lets one machine read or write another machine’s memory directly over the network WITHOUT involving the remote CPU — a one-sided operation. A DPU can therefore move data at near-network speed and run offloaded networking or control logic on its own cores, sparing the expensive host CPU. The important caveat is that a DPU’s cores are comparatively weak, so it pays to offload only low-compute work to it; putting a compute-heavy component on the DPU can actually make things slower

is the enabling technology. Its promise for serverless is exactly the opposite of the current design: move data directly, off the critical host CPU, and let the DPU do the cross-machine orchestration that the central controller does today.

What people did before, and where it broke

Storage-based forwarding (OpenFaaS and mainstream platforms). Pass data through remote storage or the controller. Simple and compatible with function isolation, but two redundant copies, a controller that saturates (over 6 cores at 500 requests per second), and host-CPU cost make it slow — data transfer becomes over 70% of latency.
Shared-memory intra-node transfer (Nightcore). For co-located functions, use inter-process communication and a shared in-memory volume to avoid network round-trips and copies. Fast on the same machine, but it does not accelerate cross-machine transfers (falling back to TCP at over 5 seconds for 512 MB).
RDMA between hosts (host-driven). Use RDMA to move data across machines. Faster than storage, but a host-driven RDMA path still spends host CPU on orchestration and connection management, and does not decouple control from data.
Coupling control and data flow. Every prior approach routes data through the orchestrator that also handles control, so the controller is on the data path and cannot escape being a bottleneck.
The gap. No system gave a direct, zero-copy data path both within and across machines, decoupled from the controller and off the host CPU, while preserving function isolation. That is the gap FUYAO fills.

How FUYAO transfers data directly

Decouple control from data, open a reusable direct channel, and let the DPU carry it across machines

The third-party data path and why it is slow

Isolated, stateless, location-unaware functions route every data hand-off through storage or the controller, causing two redundant copies, a controller that saturates, and host-CPU cost for every byte. Data transfer ends up dominating latency because functions are short.
↓ →
Decouple control flow from data flow new

FUYAO separates orchestration messages (control) from the actual bytes (data). The controller still decides what runs when, but the data no longer passes through it — once two functions are introduced, they can exchange data directly, taking the controller off the data path.
↓ →
Open a persistent Direct Transfer Channel new

A one-time handshake establishes a Direct Transfer Channel between a function pair, which is then REUSED for every later invocation until the functions are released. The handshake cost (about 6 to 9 milliseconds) is paid once and amortized over many transfers.
↓ →
Same machine: a shared-memory FIFO new

For co-located functions, the DTC is a full-duplex shared-memory FIFO buffer: the sender writes and the receiver reads directly, with no controller involvement and no copies. Latency is under 72 microseconds regardless of data size, from a few bytes to hundreds of megabytes.
↓ →
Across machines: DPU-orchestrated RDMA new

For functions on different nodes, the inter-node engine runs ON the DPU. It polls remote RDMA memory blocks with one-sided RDMA reads, queries the gateway for the target's location, and publishes the handshake to the target node with an immediate-notification RDMA write. Then the source function writes data straight into the destination's registered memory over RDMA — never touching the host CPU.
↓ →
Auto-select the method, and build a DTC only when it pays new

A zero-copy allocator hides RDMA memory registration, and the runtime automatically chooses among four transfer methods (intra- or inter-node, direct or third-party) by data size and location. A DTC is created only when a decision rule predicts the pair will be called often enough to amortize the handshake; sparse pairs use third-party forwarding.

Host-side gateway and intra-node engine on the CPU, inter-node engine offloaded to an NVIDIA BlueField-class DPU, and a per-function sidecar managing the channel lifecycle. Evaluated on real serverless workflows (media service, function chains, FINRA fan-out) against OpenFaaS, Nightcore, and Pheromone.

The idea

Decouple control flow from data flow so the data leaves the orchestrator’s path. The root inefficiency is that serverless couples two different things — the control messages that decide what runs when, and the actual bytes that flow between functions — and routes both through the central controller, which then cannot help being a bottleneck. FUYAO separates them: the controller keeps orchestrating, but once two functions are introduced, their data flows directly between them, and the controller is off the data path. Untying two concerns that were conflated for convenience is a move the site rewards repeatedly — the FHE-accelerator page decouples residue size from precision, and the coherent-NIC page separates signaling from data placement — and here it is the whole unlock, because the coupling is exactly what put the controller in the data’s way. Once control and data are separate, the data path is free to be as direct as the hardware allows.

Offload the data path to a smart NIC near the network, bypassing the host CPU. The direct path across machines is built on a DPU doing the work the host CPU used to do: its own cores run the inter-node engine, and one-sided RDMA moves data straight into the destination’s memory without waking either host CPU. Pushing data movement onto a processor near the network fabric, so the host cores are spared, is the same bypass-the-CPU-and-use-the-fabric discipline the site’s coherent-NIC page builds on when it makes the NIC a first-class memory participant, the microsecond-BFT page uses when one-sided RDMA reaches remote memory without the remote CPU, and the in-memory-computing pages share when they move computation to where the data lives. The subtlety FUYAO adds is selectivity: because the DPU’s cores are weak, it offloads only the low-compute inter-node engine and keeps the compute-heavy gateway on the host — offloading the gateway would slow it down — the same match-the-work-to-the-processor lesson the heterogeneity-aware pages teach.

Amortize a one-time setup over reuse, but only when reuse justifies it. A direct channel is not free to open — the handshake costs several milliseconds — so building one blindly would hurt rarely-used pairs. FUYAO pays that setup cost once and reuses the persistent channel for every subsequent call, and it gates the decision on a rule that predicts whether the pair will be invoked often enough for the savings to exceed the handshake. Paying a fixed cost to unlock a cheap steady state, but only when the workload will reuse it enough to come out ahead, is the same amortize-and-gate discipline the site’s virtual-domain page uses when it builds a mapping only when it will be reused, and the broader optimize-the-common-case-when-it-pays reasoning behind adaptive systems. The evidence is in the tail: on sparse, cold-start-heavy workloads the handshake is barely worth it, so the rule correctly leans on third-party forwarding there and reserves direct channels for the hot pairs.

Getting data from one function to the next

From a triple-copied hand-off to a direct channel

Passing intermediate data between serverless functions. Numbers from the paper's evaluation.

Step 1 of 5

Why the middleman is slow

Isolated, stateless functions route every hand-off through storage or the controller: the data is copied twice, the controller saturates (over 6 cores and 86% of CPU at 500 requests per second, with 52-millisecond forwarding latency), and host CPU is spent per byte. Because functions are short, this transfer overhead dominates — over 70% of end-to-end latency.

a small transfer through storage takes over 3 milliseconds where a direct one needs 0.07
the deep flaw is coupling control flow with data flow, forcing the bytes through the orchestrator

Why it works

Isolated, stateless functions route data through a third party, copying it twice and saturating the controller → decoupling control from data and connecting functions directly → the bytes leave the controller’s path, removing the copies and the bottleneck.
Establishing a direct channel costs a one-time handshake → the channel is persistent and reused for every later invocation → the setup is amortized, and a decision rule builds a channel only for pairs called often enough to pay for it.
Moving data across machines normally spends host CPU on orchestration → the DPU runs the inter-node engine on its own cores and uses one-sided RDMA → data goes straight into the destination’s memory without touching either host CPU.
The DPU’s cores are weak → FUYAO offloads only the low-compute inter-node engine and keeps the compute-heavy gateway on the host → the offload helps rather than hurts, since a heavy component on the DPU would be slower.
Because functions are short, transfer overhead dominated their latency → making the transfer direct and off-CPU shrinks it dramatically → end-to-end latency drops from 50 to 12 milliseconds and control-plane CPU falls 80 to 95%.

What it costs (trade-offs and scope)

The handshake only pays off for frequent pairs. Opening a channel costs 6 to 9 milliseconds, so on sparse, unpredictable, cold-start-heavy workflows the benefit is small (only 6.6% latency reduction on one production trace with 24.5% cold starts). The decision rule must predict reuse correctly, and a wrong guess wastes the handshake.
It requires DPU hardware in every node. FUYAO depends on BlueField-class SmartNICs with RDMA, a real hardware cost, and a single DPU supports a limited number of RDMA connections — though 2000 connections raised latency only from 102 to 120 microseconds, so the impact is minor at their scale.
The host CPU is not fully freed. Because the DPU’s cores are weak, only the low-compute inter-node engine is offloaded; the compute-heavy gateway stays on the host CPU (offloading it can raise latency by up to 780%), so FUYAO frees the data plane, not the whole control plane.
It adds machinery on the critical path. A per-function sidecar (a ten-state state machine), shared-memory-block management, per-pair RDMA registration, and the auto-selection logic are all new moving parts, and isolation rests on per-pair RDMA-region registration and validation being correct rather than on a formal proof.
The gain is proportional to how much the data path was costing. The large wins come on communication-heavy workflows where transfer was over 70% of latency; a compute-bound workflow with little inter-function data would benefit far less.

The evaluation

FUYAO on a serverless testbed with NVIDIA BlueField-class DPUs, evaluated on real workflows — a media service (eight functions, eleven data transfers), configurable function chains, and a FINRA fan-out workflow (10 to 80 parallel functions) — plus microbenchmarks and Azure Functions production traces, against OpenFaaS (storage-based), Nightcore (shared-memory), and Pheromone.

Passing data between serverless functions

Property	Storage-based (OpenFaaS)	Shared-memory (Nightcore)	FUYAO (this work)
Intra-node path
Inter-node path (512 MB)
Redundant copies
Control-plane CPU (500 QPS)
End-to-end (media workflow)

Storage-based transfer copies twice and saturates the controller; shared-memory helps only within a node. FUYAO decouples control from data and transfers directly — shared memory intra-node, DPU-orchestrated RDMA inter-node — cutting end-to-end latency from 50 ms to 12 ms, control-plane CPU by 80 to 95%, and raising throughput one to two orders of magnitude.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

Read the scope precisely. FUYAO’s direct channel costs a 6-to-9-millisecond handshake to open, so it pays off only for frequently-invoked function pairs; on sparse, cold-start-heavy workflows the benefit shrinks (as low as 6.6% on one production trace), and the decision rule must predict reuse correctly or the handshake is wasted. It requires DPU hardware (BlueField-class SmartNICs with RDMA) in every node — a real cost, with a limited number of RDMA connections per DPU, though the connection scaling impact is small at the tested scale. Because the DPU’s cores are weak, FUYAO offloads only the low-compute inter-node engine and keeps the compute-heavy gateway on the host CPU (offloading it can raise latency up to 780%), so it frees the data plane rather than the entire control plane. It also adds machinery on the critical path — a per-function sidecar state machine, memory-block management, per-pair RDMA registration, and auto-selection logic — and isolation rests on that per-pair registration and validation being correct. And the large end-to-end wins come on communication-heavy workflows where transfer dominated latency; a compute-bound workflow would gain far less. Within that scope the lessons are sharp and general: decouple control flow from data flow so the data leaves the orchestrator’s path; offload the data movement to a processor near the network (a DPU with one-sided RDMA) to bypass the host CPU, but offload only the low-compute part that fits its weak cores; and amortize a one-time setup over reuse, gated by a rule that only builds the fast path when the workload will use it enough to pay for it.

Background

Decouple control from data, offload to the smart NIC, and amortize the fast path

This paper sits in the site’s datacenter and networking clusters and rhymes with several ideas covered elsewhere on this site. Its founding move is to decouple control flow from data flow so the bytes stop passing through the orchestrator — the same untie-two-conflated-concerns discipline the FHE-accelerator page uses when it separates residue size from precision and the coherent-NIC page uses when it separates signaling from data placement, and here it is the whole unlock, since the coupling is what put the controller on the data path. Its mechanism is to offload the data movement to a DPU near the network and use one-sided RDMA to write straight into the destination’s memory without waking the host CPU — the same bypass-the-CPU-and-use-the-fabric idea the coherent-NIC page builds on when it makes the NIC a memory participant, the microsecond-BFT page uses when one-sided RDMA reaches remote memory without the remote CPU, and the in-memory-computing pages share when they move computation to the data. It is selective about the offload, putting only the low-compute inter-node engine on the DPU and keeping the heavy gateway on the host because the DPU cores are weak — the same match-the-work-to-the-processor, measure-the-true- cost lesson the heterogeneity-aware-scheduling and datacenter-power pages teach. It eliminates the intermediary and its redundant copies and round-trips, the same cut-out-the-middleman, reduce-the-round-trip move the coherent-NIC and attention-overlap pages make when they fuse a step away. And it amortizes the one-time channel setup over reuse, gated by a rule that predicts payoff, the same pay-once-and-reuse-when-it-pays discipline the virtual-domain page uses when it builds a mapping only when it will be reused. Together they make FUYAO a case study in speeding up a system by taking its data off the control path and onto the fabric.

If you remember only three things

1. Serverless workflows chain tiny functions where each one's output feeds the next, but because functions are isolated, stateless, and location-unaware, today's platforms pass the data through a THIRD PARTY — storage or the orchestration controller. That path copies the data twice, makes the controller a bottleneck (over 6 cores and 86% CPU at 500 requests per second), and burns host CPU per byte, so data transfer becomes over 70% of end-to-end latency — crippling for short functions. The deep flaw is coupling control flow (orchestration) with data flow (the bytes), forcing data through the orchestrator.
2. FUYAO decouples control from data and opens persistent DIRECT TRANSFER CHANNELS between function pairs, established once by a handshake and reused. On the same machine the channel is a zero-copy shared-memory FIFO (under 72 microseconds, any size); across machines a DPU (SmartNIC) runs the inter-node engine on its own cores and uses one-sided RDMA to write data straight into the destination's memory, never touching the host CPU (512 MB in 263 ms versus 2078 without the DPU). The runtime auto-selects among four transfer methods, offloads only the low-compute inter-node engine to the weak DPU cores (keeping the heavy gateway on the host), and builds a channel only when a rule predicts enough reuse to amortize the handshake.
3. FUYAO cuts end-to-end latency from 50 milliseconds to under 13 (meeting a 100-millisecond objective), reduces communication from over 70% of latency to under 28.2%, keeps control-plane CPU at most 0.4 of a core (80 to 95% less than baselines), and raises throughput one to two orders of magnitude. The transferable lessons: decouple control flow from data flow so data leaves the orchestrator's path; offload data movement to a processor near the network (a DPU with one-sided RDMA) to bypass the host CPU, but offload only the low-compute part that fits its weak cores; eliminate the intermediary and its redundant copies; and amortize a one-time fast-path setup over reuse, gated by whether the workload will use it enough to pay.

Check yourself

Check your understanding

1. FUYAO argues the deep flaw in serverless data transfer is that control flow and data flow are COUPLED. Predict what that means and why decoupling them helps.
2. FUYAO uses a DPU to move data across machines. Predict what the DPU does that the host CPU was doing, and why that helps.
3. FUYAO offloads only the inter-node engine to the DPU and deliberately keeps the gateway on the host CPU. Predict why it does not offload everything.
4. Opening a Direct Transfer Channel costs a 6-to-9-millisecond handshake, yet FUYAO still comes out far ahead. Predict how, and when it chooses NOT to build one.

Sources

FUYAO: DPU-enabled Direct Data Transfer for Serverless Computing (ACM DL, DOI 10.1145/3620666.3651327) — ASPLOS 2024 (full text read via the ACM PDF).
ACM Digital Library entry — published version and metadata.