Serverless computing on heterogeneous computers

A modern server is secretly several computers — CPU, DPUs with their own Linux, FPGAs — and Molecule is the first serverless runtime that treats them as one machine, packing 50% more function instances onto the DPUs and running matrix-heavy functions up to 34.6× faster on FPGAs.

Why this problem is worth solving

Serverless computing runs single-purpose functions in sandboxes, auto-scales them, and bills by the millisecond. Its economics reward two things a platform can never have enough of: function density — how many instances fit on one machine, which also keeps chained functions close enough for fast calls — and workload reach — which kinds of computation can be sold as functions at all.

Homogeneous CPU servers cap both. Dark silicon limits general-purpose core counts, so density plateaus. And the era’s defining workloads — ML inference, video processing, genome analysis (over 10× faster on an AWS F1 FPGA) — belong on accelerators that serverless platforms cannot offer. Meanwhile the hardware in real datacenters has already moved: servers ship with DPUs (data processing units — system-on-chips like NVIDIA’s BlueField with 16 ARM cores, their own memory, line-rate network engines, and their own Linux), plus FPGAs, GPUs, and smart storage. The paper names this assembly a heterogeneous computer and asks the natural question: why can’t functions run on all of it?

The obstruction is systems software, not silicon. The authors’ experimental server — one Xeon host plus two BlueField DPUs — runs three separate Linux systems. Every mechanism modern serverless performance depends on is single-OS to its bones: fast startup forks instances from templates (one kernel’s fork), function chains communicate via IPC (one kernel’s pipes), and permissions live in one kernel’s process table. Accelerators break a different assumption: a sandbox abstraction (the OCI interface — create, start, kill, delete, state) presumes a device that can hold many concurrent, independently-managed instances, while an FPGA holds one programmed image and takes seconds to swap it.

What people did before, and where it broke

Serverless optimization is a rich field — Catalyzer-style fork-based startup, Nightcore-style IPC-based function calls, snapshot and caching schemes — but all of it targets a single OS on a single kind of processing unit. Offload research put infrastructure (network stacks, storage) onto DPUs, but not tenant functions with full serverless semantics. FPGA-cloud work (AWS F1, FPGA OSes like Coyote) provides the device plumbing but no serverless model: no auto-scaling, no sub-second startup, no function chaining. The gap is precise: nobody had made one serverless runtime manage function lifecycles, permissions, and communication across OS boundaries and hardware species — while keeping the developer experience (write a handler, upload, get scaling) intact.

The idea

XPU-Shim: the missing OS layer for a multi-OS machine

XPU-Shim is a distributed indirection layer running on every general-purpose PU, exposing XPUcalls — syscall-style primitives that give one serverless runtime a unified view of a multi-OS computer. Three pieces do the work:

• Global processes and distributed capabilities. A process gets a globally unique xpu_pid (PU-ID plus local ID — statically partitioned, so no synchronization hotspot), and a per-process capability group records what it may touch across the machine. Owners grant_cap and revoke_cap permissions on distributed objects; every XPUcall checks them. Permission control, which used to live in one kernel, becomes a small distributed protocol.
• Neighbor IPC (nIPC). Between “local IPC” and “network to a remote node,” heterogeneous computers have a third regime the paper names and exploits: PUs in one chassis linked by reliable interconnects (DMA, RDMA, CXL). XPU-FIFOs give functions on different PUs a pipe-like channel over that fabric — no API gateway, no TCP stack. The tuned polling variant reaches ~25 microseconds, faster than Linux IPC on the DPU itself (it bypasses the slow 800 MHz device kernel), within 1.5–3.1× of CPU-local IPC.
• xSpawn. Start a program on another PU, with explicitly granted capabilities — the primitive that lets startup mechanics reach across OS boundaries.

Vectorized sandboxes: making an FPGA speak serverless

The OCI sandbox interface fails on FPGAs for two structural reasons: a device holds one image at a time (so one-sandbox-at-a-time create/start means seconds of reprogramming per request), and explicit delete pays to erase hardware that the next create would overwrite anyway. Molecule’s vectorized sandbox extends each interface to operate on vectors: create packs many function sandboxes into one FPGA image and flashes it once (so requests usually hit an already-resident function); start runs several sandboxes concurrently in partitioned FPGA regions; delete becomes lazy — a state update, with physical replacement deferred to the next create. CPU and DPU containers implement the same interface trivially (vectors of size one, via runc), so the runtime manages every species of function identically. Isolation inside the FPGA is delegated to the wrapper/FPGA-OS layer (e.g., Coyote’s MMU/TLB-backed virtual FPGAs). Developers, meanwhile, just write handlers — Python for CPU/DPU, HLS/OpenCL for FPGA — and upload.

Rebuilding the classic optimizations on the new base

Startup: cfork. Fork-based startup is the fastest known cold-start cure, but container runtimes are multi-threaded and fork is a single-threaded syscall. Molecule’s container fork (cfork) makes the template container forkable — threads temporarily merge into one, the fork happens, and the child restores multi-threading — yielding sub-10-millisecond container startup (a measured pipeline of optimizations takes the breakdown from 85.6 ms naive to 8.4 ms). Language templates are pre-booted per PU, and crucially, cfork composes with xSpawn: forking an instance onto a DPU costs almost nothing extra. On FPGAs, vectorized images plus data retention give 53-millisecond starts versus the seconds of erase-and-reprogram.

Communication: direct DAG calls. Function chains bypass the gateway: same-PU hops use local IPC, cross-PU hops use nIPC, and FPGA chains keep intermediate vectors resident in device memory (1.95× on a five-function chain). Measured on an Alexa-style skill: 15–18× lower call latency than the Node.js-Express baseline within a PU, 10–13× across PUs.

placement: CPU {f1,f3,f5} | DPU {f2,f4} | FPGA {matmul}
OSes involved: host Linux, DPU Linux, FPGA shell

Why it works

Because the density ceiling was a general-purpose silicon ceiling, not a chassis ceiling, enlisting the DPUs’ idle ARM cores adds instances without adding servers — 50% more with two devices, at the cheapest price tier. → Because a single machine’s PU-to-PU links are reliable, ordered, and DMA-capable, cross-PU function calls can shed the network stack and gateway that “distributed” normally implies — which is why nIPC lands within a small factor of local IPC and 10–13× ahead of the baseline. → Because the classic optimizations (fork-based startup, IPC-based DAGs) were rebuilt on the shim’s primitives rather than on one kernel’s syscalls, they survive the multi-OS reality — a cfork onto another OS costs almost nothing extra. → Because the sandbox interface was vectorized rather than replaced, accelerators with one-image-at-a-time semantics amortize reprogramming across many functions and drop deletion costs to zero — while CPUs and DPUs implement the same interface with vectors of one. → Therefore one runtime schedules ordinary containers and FPGA bitstreams with the same verbs, and the platform’s two scarce goods — density and reach — both grow, without surrendering serverless’s promises of auto-scaling and millisecond startup.

What it costs

• Users now choose hardware. Molecule’s pricing model asks developers to pick PU tiers per function — honest about heterogeneity, but a step away from serverless’s hardware-blindness; the platform schedules within the user’s choices.
• A new trusted layer. XPU-Shim is distributed systems software on every PU, maintaining consistent global state (capabilities, FIFO registries) — more machinery to secure and operate than one kernel.
• FPGA functions are FPGA work. Handlers come from HLS/OpenCL/Verilog with offline image compilation; the vectorized image model also means co-resident functions share a flash cycle — density there is bounded by device regions (twelve functions used 10% of an F1’s LUTs).
• Polling buys latency with cycles. The fastest nIPC variant spins on the device; and cfork’s copy-on-write can add page-fault costs versus plainly cached warm instances when cache hit rates are high — the paper says so.
• Isolation on accelerators is delegated. FPGA multi-tenancy rests on the wrapper/FPGA-OS (partial reconfiguration regions, Coyote-style vFPGAs), a younger security foundation than containers on Linux.

The evaluation

Two real heterogeneous computers: a dual-socket Xeon 8160 server (96 cores) with two 100 Gbps BlueField-1 DPUs (16 ARM cores each), and an AWS F1.16xlarge instance (64 vCPUs, eight UltraScale+ FPGAs). Baselines: Molecule-homo (the same runtime without XPU-Shim or heterogeneity — a deliberately strong baseline that already beats OpenWhisk and AWS Lambda), plus those commercial systems; benchmarks from ServerlessBench and FunctionBench plus FPGA applications (gzip, matrix compute).