In-Storage Domain-Specific Acceleration for Serverless Computing

Rohan Mahapatra, Soroush Ghodrati, Byung Hoon Ahn, Sean Kinzer, Shu-Ting Wang, Hanyang Xu and 5 others

Serverless cloud functions run for a moment and vanish, so every time one fires it must haul its inputs and outputs across the network from disaggregated storage — and that data movement is more than half the time, so bolting on a fast accelerator barely helps, capped at about 1.5 times; this work from UC San Diego and Google builds DSCS-Serverless, which instead drops a small domain-specific accelerator inside the storage drive itself, running the function next to its data — so a 4-watt chip in the SSD beats a 250-watt GPU with remote storage by 2.7 times while using a fraction of the energy.

The argument in one glance

Problem

Three good datacenter trends fight each other when combined

Three shifts are reshaping the cloud, each beneficial on its own. Serverless computing lets developers run functions without managing servers. Storage disaggregation separates storage from compute onto its own remote nodes, reached over a fast network, for better utilization and elasticity. And domain-specific accelerators bring specialized hardware for compute efficiency. But put them together and the benefits cancel: a serverless function on disaggregated storage spends most of its time moving data to and from remote storage, so a fast accelerator has little compute left to speed up.
What people did

Add a powerful accelerator at the compute node

The natural response is to accelerate the compute: attach a GPU or ASIC to the compute node so the machine-learning inference or data processing at the heart of the function runs fast. This is how accelerators are used in the cloud today — the function runs on a compute server, pulls its data from remote storage, and hands the heavy computation to the attached accelerator over the local bus.
Why it fell short

When data movement dominates, faster compute barely helps

Profiling real serverless applications shows that reading inputs and writing outputs to remote storage is more than half the end-to-end latency, and over 70 percent for some. Every invocation means an RPC across the network, a deserialization step, a read system call, and a bus transfer — and then, for a GPU, another copy into the accelerator's memory. By Amdahl's law, accelerating the shrinking compute fraction caps the whole-application speedup at about 1.5 times, so even a 250-watt GPU leaves most of the latency untouched.
This paper's idea

Put a small accelerator inside the storage drive, next to the data

DSCS-Serverless attacks the movement instead of the compute. It embeds a small, programmable domain-specific accelerator inside some storage drives themselves — a computational storage drive — sized to fit the drive's tight power budget, and activated only for functions in its domain. A function amenable to acceleration is scheduled directly onto the drive holding its data and runs right there, so the network hop, the RPC, the deserialization, and the host software stack all disappear. A tiny in-storage chip thereby beats a 250-watt GPU with remote storage.

Why this problem is worth solving

The heart of the matter is what a serverless function must do with its data every single time it runs, so it is worth being precise.

Serverless computing runs a developer’s function on demand, billing only for the run, and its functions are stateless and ephemeral — they hold no memory between invocations. So on every invocation, a function must read its inputs from, and write its outputs to, persistent storage. In a modern disaggregated datacenter, that storage is not local: it lives on separate storage nodes, reached over the network, a design that improves utilization and elasticity. The consequence is that each function call pays a heavy data-movement tax: a remote-procedure call across the network, a deserialization of the data format, a read system call to pull the bytes off the physical device over the internal bus, and the reverse to write results back. Profiling eight real serverless applications shows this movement is, on average, more than half of the end-to-end latency, exceeding seventy percent for several, and the storage read latency has a long tail — the ninety-ninth percentile is more than double the median. The compute a function performs is often the smaller part of its life; getting its data to and from remote storage is the larger.

The reason this makes accelerators disappointing is a piece of arithmetic every systems designer knows.

Amdahl’s law says that the speedup of a whole task is limited by the fraction of it you do not speed up: if half the runtime is untouched, no matter how fast you make the other half, you cannot beat a two-times overall speedup. Applied here, if data movement is more than half of a serverless function’s latency and an accelerator only speeds the compute, then the whole-application speedup is capped low — the paper measures the ceiling at about one and a half times, even with a high-end GPU. Worse, conventional accelerators are designed myopically for compute: a GPU attached to the compute node still needs the data delivered to it over the bus, adding yet another copy on top of the remote fetch, and at the batch size of one typical of serverless it is badly underutilized anyway. So the very accelerators meant to make serverless faster are throttled by a bottleneck they do not address. Breaking this ceiling requires attacking the data movement itself, not just the arithmetic — which is what makes rethinking where the computation happens worthwhile.

What people did before, and where it broke

Attach a high-end accelerator to the compute node. Run the function on a compute server and hand its compute to a GPU or ASIC over the local bus. Where it broke. The function still fetches data from remote storage first, so the accelerator speeds only the minority compute fraction and the whole-application gain is capped near 1.5 times.
Accelerate the remote-storage path itself. Build hardware to speed up the RPC, the deserialization, or the network protocol. Where it broke. These shave pieces off the data-movement cost but the data still crosses the network on every invocation, so the fundamental movement remains.
Put a general-purpose processor in the storage (ARM cores). Add a small CPU inside the drive to run functions near the data. Where it broke. A general in-storage CPU removes the network movement but is too weak to run the compute efficiently, so it barely matches, or slightly underperforms, the remote-CPU baseline.
Put an FPGA in the storage. Use a reconfigurable fabric inside the drive for in-storage compute. Where it broke. The FPGA is bounded by the drive’s limited resources and low clock, and carries driver overhead, so it captures the locality benefit but leaves compute performance on the table.
The gap. Every approach either accelerated compute while the data movement dominated, or moved compute in-storage without a design suited to the drive’s tight power budget and the serverless access pattern. What was missing was a small, domain-specific accelerator designed for the storage constraints, running the function next to its data — which is exactly what accelerated serverless on disaggregated storage needs.

How DSCS-Serverless works

Running the function inside the storage drive, next to its data

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Compute where the data already is, instead of hauling the data to the compute. The whole inefficiency is that a serverless function’s data lives on a remote storage node while its computation happens on a distant compute node, so every invocation drags the data across the network and through the software stack. DSCS-Serverless inverts this: it places a small accelerator inside the storage drive, so the function that would have pulled the data out instead runs in place, and the movement simply does not happen. It deliberately does not shove heavy compute back into storage — the accelerator is small and fits the drive’s power budget — but it captures the locality that matters. Bringing the computation to where the data physically resides, so the data never has to make the expensive round trip, is the same principle the compute-in-memory compiler work covered elsewhere on this site is built on, computing inside the memory itself precisely because moving data is the dominant cost.

Attack the bottleneck that actually governs the runtime, not the one that is easy to accelerate. The trap the field fell into was treating a serverless function as a compute problem and reaching for a bigger accelerator, when measurement shows the compute is the minority and data movement is the majority — so a faster accelerator hits Amdahl’s ceiling almost immediately. DSCS-Serverless is grounded in profiling first: it identifies that over half the latency is movement and targets that, which is why a four-watt in-storage chip beats a two-hundred-fifty-watt GPU whose speed is squandered waiting on the network. Measuring where the time actually goes and discovering that data movement, not arithmetic, is the real limit is exactly the conclusion the deep-learning-on-GPUs retrospective covered elsewhere on this site reaches, that accelerating the compute alone plateaus because the bottleneck has moved to moving the data.

Right-size the accelerator to the constraints, because bigger is not better here. A storage drive shares a tight power budget with its flash, so a huge accelerator will not fit, and the serverless access pattern — one request at a time, small data — punishes an oversized design. DSCS-Serverless runs a design-space exploration over hundreds of configurations and finds that a moderate matrix array, not the largest, is optimal: a big array processing a single small request spends more cycles moving each tile than computing it, so a right-sized array that overlaps a tile’s transfer with the previous tile’s compute wins. Sweeping a design space to find the accelerator that best fits the real hardware constraints and workload, rather than assuming the biggest configuration is best, is the same discipline the content-addressable-memory compiler work covered elsewhere on this site applies, exploring configurations against a hardware model to land on the one that actually fits.

Following a wildfire-detection function through DSCS-Serverless

Running an object-detection serverless function

Tracing one function shows how moving compute to the data changes everything.

Step 1 of 5

A drone image arrives to be analyzed

A drone uploads a forest image, and a serverless application must pre-process it and run a neural-network detector to spot fire hazards. In the traditional flow, the function launches on a compute node and pulls the image from remote storage — an RPC, a deserialization, a read system call, a network hop — and if it uses a GPU, another copy into GPU memory.

Serverless applications are chains of such stateless functions exchanging data through storage.

Why it works

Running the function in the storage removes the movement that dominated the latency. A serverless function spends over half its time hauling data to and from remote storage → but placing the accelerator inside the drive lets the function compute right where its data already sits → so the network hop, RPC, deserialization, and host software stack all vanish for that function.
Attacking data movement lifts the Amdahl ceiling that trapped compute accelerators. A high-end accelerator at the compute node can only speed the compute fraction, capping whole-application gains near 1.5 times → but shrinking the dominant data-movement fraction attacks the part that actually governs the runtime → so a tiny in-storage chip beats a 250-watt GPU whose speed is wasted waiting on the network.
A small, right-sized accelerator fits the storage power budget and still wins. A storage drive shares a tight power budget with its flash, far too little for a big accelerator → but a design-space search finds a moderate configuration that fits, and locality does the rest → so a four-watt chip outperforms a 250-watt GPU on these workloads.
Matching the accelerator to a real serverless domain covers most of the work. Serverless is dominated by machine-learning inference, whose functions are mostly matrix math plus a fixed set of vector operations → but a matrix unit paired with a vector unit for activations, reductions, and pre-processing handles that whole domain → so one small design accelerates a large share of real serverless traffic.
Falling back gracefully keeps compatibility with everything else. A domain accelerator cannot run every function, and forcing it would break general workloads → but a drive with an accelerator still behaves as an ordinary storage drive, and non-domain functions simply run on a compute node as before → so the design adds capability without disrupting the datacenter.

What it costs (trade-offs and scope)

The gains are still Amdahl-bounded by whatever stays outside the drive. Moving the acceleratable functions in-storage shifts the bottleneck to the system stack and to functions that still run on a compute node — a mostly-communication benchmark that should get three times sees one-point-eight, because its final function and driver overhead remain on the old path.
It helps only the machine-learning domain, and only functions the developer marks. The accelerator is domain-specific, and the developer must partition the application and configure which functions to offload; anything outside the domain falls back to conventional execution with no benefit.
Most results are modeled or simulated, not deployed at datacenter scale. The real accelerator runs on an SSD’s FPGA; the ASIC numbers come from Verilog synthesis and a cycle-accurate simulator with process scaling, the near-storage CPU and GPU baselines from analytical models, and the at-scale study from simulation.
It requires a new storage class and data co-location. A replica of each acceleratable function’s data must sit on a drive with the accelerator, the scheduler must be extended to route functions there, and requests are assumed small enough to fit one drive; distributed data falls back.
The win is locality, not raw compute. The in-storage chip is deliberately small, so a function that is compute-heavy but communication-light gains little; the advantage comes from eliminating movement, which a general in-storage processor can partly capture too.
Scheduling is basic and cold starts are unchanged. It uses first-come-first-served, run-to-completion scheduling with optimized policies left to future work, and functions incur the same cold-start cost as traditional serverless, mitigated only by keeping them warm in the drive.
It adds real hardware and system complexity. New computational storage drives, accelerator silicon, an OpenCL driver, and a compiler are a genuine deployment cost, and the cost-efficiency case rests on energy savings accruing over years.

The evaluation

Accelerating serverless functions on disaggregated storage

Property	GPU + remote storage	In-storage ARM	In-storage FPGA	DSCS-Serverless
Where the function computes	Compute node, data over network	In storage, general CPU	In storage, FPGA
Data-movement overhead		Low	Low
Power	250 watts	Low	Low
Speedup versus CPU baseline			About 2.2 times	About 3.6 times
DSCS-Serverless advantage over it	2.7 times speed, 4.2 times energy	3.7 times speed, 4.3 times energy	1.7 times speed, 1.9 times energy

Ways of accelerating serverless functions on disaggregated storage, evaluated on eight real-world machine-learning serverless applications (a Samsung SmartSSD FPGA prototype plus a cycle-accurate ASIC simulator and analytical baselines). DSCS-Serverless's roughly 4-watt in-storage accelerator beats a 250-watt GPU with remote storage by 2.7 times in speed, 4.2 times in energy, and 3.0 times in cost efficiency, and beats conventional in-storage ARM and FPGA designs by 3.7 and 1.7 times, because it attacks the dominant data-movement cost rather than the compute.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

DSCS-Serverless is a sharp systems insight backed by a careful design: it observes that a serverless function on disaggregated storage spends most of its time moving data, not computing, so a compute accelerator hits Amdahl’s ceiling — and it responds by embedding a small, right-sized, domain-specific accelerator inside the storage drive, so the function runs next to its data and the movement vanishes. The payoff is striking: a 4-watt in-storage chip beats a 250-watt GPU with remote storage by 2.7 times in speed and 4.2 times in energy, and beats conventional in-storage CPU and FPGA designs too. Read it with its scope. The gains are still Amdahl-bounded by whatever stays outside the drive — the system stack and functions that remain on the CPU — so a benchmark that should get three times gets under two. It helps only the machine-learning domain and only functions the developer marks; most numbers come from an SSD-FPGA prototype plus simulation and analytical models rather than a datacenter deployment; and it needs a new storage class with co-located data. The durable idea is the reframing: when the data movement dominates, the right place for the accelerator is not next to the processor but next to the data.

Background

Why serverless keeps bumping into the same overheads

It helps to see this work as one more skirmish in serverless computing’s long fight with its own overheads. The promise of serverless is that you write a small function and the cloud runs it on demand, billing only for the moment it runs — but that very statelessness and ephemerality mean each invocation has to reconstitute its world from scratch, and the cost of doing so keeps swamping the actual work. On CPUs it shows up as the cold-start tax, the delay of building an execution environment before a function can begin. Here it shows up as the data-movement tax, the delay of fetching a function’s inputs from remote storage every time. Both are the same phenomenon: for a computation that lasts milliseconds, the setup around it dominates. The serverless function-warming work covered elsewhere on this site fights the cold-start side of this, deciding when and where to keep functions ready; DSCS-Serverless fights the data side, moving the computation to where the data already lives. The broad lesson is that as units of work shrink toward the ephemeral, efficiency is decided less by how fast you compute than by how little you have to move and rebuild around each tiny task.

If you remember only three things

1. Three datacenter trends — serverless computing, storage disaggregation, and domain-specific accelerators — each help individually but cancel when combined: serverless functions are stateless, so every invocation reads inputs from and writes outputs to persistent storage, which in a disaggregated datacenter is remote and reached over the network; profiling eight real applications shows this data movement is more than half of end-to-end latency (over 70 percent for some), so by Amdahl's law a compute accelerator at the compute node caps the whole-application speedup at about 1.5 times, even with a 250-watt GPU that also needs the data copied to it and is underutilized at a batch size of one.
2. DSCS-Serverless attacks the data movement instead of the compute by embedding a small, programmable domain-specific accelerator inside some storage drives (a computational storage drive): an acceleratable function (marked by the developer and with a replica of its data co-located) is scheduled directly onto the drive and runs in place, a single peer-to-peer flash-to-DRAM transfer replacing the network hop, RPC, deserialization, and host software stack; the accelerator pairs a matrix unit (systolic array for tensor math) with a vector unit (activations, reductions, casts, pre/post-processing) to cover the machine-learning domain that dominates serverless, and a design-space search over 650-plus configurations finds that a moderate 128-by-128 array (not the largest) is optimal under the drive's tight power budget because a big array stalls moving each tile of a single small request.
3. On the eight benchmarks, DSCS-Serverless's roughly 4-watt in-storage accelerator beats a 250-watt GPU with remote storage by 2.7 times in speed, 4.2 times in energy, and 3.0 times in cost efficiency, and beats conventional in-storage ARM and FPGA designs by 3.7 and 1.7 times in speed — but the gains remain Amdahl-bounded by the system stack and functions that stay on the CPU (a mostly-communication benchmark gets 1.8 rather than 3 times), it helps only the marked machine-learning functions, most numbers come from an SSD-FPGA prototype plus a cycle-accurate simulator and analytical models rather than a real datacenter, it needs a new storage class with co-located data, and its scheduling is basic with cold starts unchanged.

Check yourself

Check your understanding

1. Why do domain-specific accelerators disappoint for serverless functions on disaggregated storage?
2. What is DSCS-Serverless's core idea?
3. Why did the design-space exploration find a moderate accelerator optimal rather than the largest one?
4. Which statement best reflects the limits of DSCS-Serverless?

Sources

The paper is “In-Storage Domain-Specific Acceleration for Serverless Computing” by Rohan Mahapatra, Soroush Ghodrati, Byung Hoon Ahn, Sean Kinzer, Shu-Ting Wang, Hanyang Xu, Lavanya Karthikeyan, Hardik Sharma, Amir Yazdanbakhsh, Mohammad Alian, and Hadi Esmaeilzadeh, published at ASPLOS 2024, DOI 10.1145/3620665.3640413. The authors are at the Alternative Computing Technologies Lab at the University of California, San Diego, with Google, Google DeepMind, and the University of Kansas.

This explainer is written from the full 19-page text. Everything above comes from the paper’s own sections: the introduction framing the confluence of serverless computing, storage disaggregation, and domain-specific acceleration, and the observation that data movement overshadows acceleration gains; the background and characterization profiling eight applications to show that remote-storage communication is on average more than 55 percent of end-to-end latency (over 70 percent for several) with a long tail, capping compute acceleration at about 1.52 times by Amdahl’s law; the DSCS-Serverless model with its DSCS-Drive housing a domain-specific accelerator next to the flash and its peer-to-peer, host-bypassing life-of-a-function flow; the accelerator microarchitecture (a matrix processing unit systolic array plus a coupled vector processing unit for activations, reductions, casts, and pre/post-processing) targeting the machine-learning domain that is over 40 percent of serverless services, and the design-space exploration over 650-plus configurations under the drive’s roughly 25-watt budget that finds a 128-by-128 array with 4-megabyte scratchpad and DDR5 optimal; the software-stack integration over OpenFaaS and Kubernetes (YAML marking, OpenCL driver, ONNX compiler with operator fusion and tiling, data placement, scheduling, fail-over, chaining, and cold starts); and the evaluation reporting a roughly 4-watt in-storage accelerator beating a 250-watt GPU with remote storage by 2.7 times speed, 4.2 times energy, and 3.0 times cost efficiency, and beating in-storage ARM and FPGA designs by 3.7 and 1.7 times speed, with the runtime-breakdown and at-scale cost-efficiency analyses.