CodeCrunch: Improving Serverless Performance via Function Compression and Cost-Aware Warmup Location Optimization

Keeping a serverless function warm costs memory, and memory runs out exactly when the load spikes — CodeCrunch compresses idle functions so more of them fit, and decides which ones to squeeze and which processor to park them on with a randomised search over a space with millions of options.

The argument in one glance

Problem

Starting a function from scratch dominates its runtime

When a serverless function is invoked and no instance exists, the platform must create a container, download the code and runtime, and install every dependency before a single line runs. That initialisation accounts for 40% to 75% of the total latency of the function.
What people did

Keep functions resident and predict when they will be needed

Providers hold functions in memory after execution so the next call is a warm start. Better systems try to forecast invocations and pre-warm accordingly, using regression, greedy caching or Fourier analysis of past call patterns.
Why it fell short

The constraint is memory, and it binds hardest exactly when you need it

A warm function occupies memory, and memory costs money. Under bursty load the keep-alive budget runs out precisely when the most functions want warming. Prediction does not help here — the memory is simply not there, and the more accurate forecasters spend up to 30% of the service time of a function computing their forecasts.
This paper's idea

Make warm functions smaller, and put them where they are cheapest

Compress the filesystem of an idle function in memory so more of them fit, accepting decompression on the critical path where that is cheaper than a cold start. Park functions on ARM or x86 depending on which runs them faster and which is cheaper to hold. Then search that space with a randomised low-dimensional embedding rather than trying to solve it directly.

Why this problem is worth solving

Serverless computing sells the promise of not thinking about servers, and the bill for that promise is the

A cold start is what happens when a function is invoked and no ready instance exists — the platform has to create a container, fetch the code and runtime, and install dependencies before the function can run at all.

. The numbers make it the dominant cost: initialisation accounts for 40% to 75% of a function’s total latency, which is a strange proportion for a model designed around jobs that run for seconds. Providers respond by leaving functions resident after they finish, in the hope of a second call, and the resulting

Keep-alive cost is what a provider pays to hold an idle function in memory waiting for its next invocation — memory occupied, and therefore money spent, on something that is not currently doing any work.

is what the whole field is really optimising against.

Most recent work attacks this by forecasting. If you can predict when a function will next be called, you can warm it just in time and hold it no longer than necessary. That is a reasonable framing and it has produced sophisticated systems using regression, greedy caching and frequency analysis of invocation histories. But it quietly assumes that memory is available once you know what to put in it, and under bursty load that assumption fails. The paper’s measurements show the moment of highest invocation load is also the moment when the keep-alive budget is most exhausted — so the better your prediction, the more functions you correctly identify as worth warming, and the less room you have for any of them. The binding constraint is not knowledge. It is space.

What people did before, and where it broke

Keep everything warm forever. The ideal for latency and impossible for cost. Every idle function occupies memory indefinitely.
A fixed keep-alive window. Amazon Lambda and Microsoft Azure hold functions for around ten minutes after execution. Simple, production-proven, and blind to whether the function will actually be called again.
Predict invocations with regression. Serverless in the Wild profiles function invocation patterns and derives a per-function keep-alive policy. This is the paper’s main baseline and the most practical of the alternatives.
Cache functions greedily. FaasCache treats warm instances as a cache and applies eviction policies with warm-up costs as weights. Effective, but it must track invocation probabilities for every function in the system.
Analyse invocation frequency spectrally. IceBreaker uses Fourier analysis to identify periodic invocation patterns and pre-warms on high-end or low-end servers depending on re-invocation probability. Accurate, and expensive: for ten million unique functions the paper measures its decision overhead at 30% of a function’s service time.
The gap. Every one of these decides when to warm a function and whether to keep it, and none of them changes how much space a warm function takes or how much holding it costs. Under memory pressure, that is the only lever that matters.

How CodeCrunch changes what warming costs

Three new decisions per function, and a way to make them fast

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The last stage is what makes the first four usable at production scale.

The idea

A warm function does not have to be full size. The central move is almost embarrassingly direct: if the problem is that warm functions occupy too much memory, make them occupy less. CodeCrunch compresses the filesystem of an idle instance in place, so more instances fit within the same keep-alive budget. In the motivating experiment, with a fixed ten-minute keep-alive policy and 10% of system memory reserved for warming, this raises the mean warm start rate from 51% to 61% — entirely from squeezing more functions in during the periods when the warm start rate was lowest. What makes this more than a trick is the accompanying observation that the memory budget, not forecast accuracy, is what actually binds during a burst. Attacking the memory footprint of resident serverless state rather than the scheduling around it is the same reframing pursued by work on improving serverless memory efficiency directly, covered elsewhere on this site.

Compression is a genuine trade, not a free win. Decompression sits on the critical path of the next invocation, so compressing a function only helps if decompressing it is cheaper than cold starting it. That holds for 42% of the evaluated functions on x86 and 46% on ARM, and for the rest it is actively harmful — some functions take up to 75% longer to decompress than to cold start. So the decision has to be per function and per moment, since it also depends on what else is competing for memory in the same interval. The paper’s honesty about this is what makes the design interesting rather than naive; the measured decompression cost is 0.37 seconds on average against an Oracle cold start of 10.2 seconds, but the tail matters and the fraction that loses is large. A second lever compounds it: about 38% of functions run faster on ARM than x86, while ARM is cheaper to keep any function alive on. Crucially there is no dominance ordering, which is exactly what distinguishes this from earlier heterogeneity work where high-end servers were faster for everything by construction and placement reduced to a fast-or-slow choice, covered elsewhere on this site.

When the search space is hopeless, project it. Put those two levers together with the keep-alive duration and each function contributes three dimensions to the decision — compress or not, ARM or x86, and how long to hold it, anywhere from zero to sixty minutes. With thousands of functions invoked per minute in a production system, that is a discrete space with millions of points that has to be re-solved every minute. The paper measures gradient descent, Newton’s method and a genetic algorithm all performing poorly on it. The answer is sequential random embedding: rather than optimising over the full space, randomly project it into a low-dimensional subspace, optimise there, and repeat with a fresh projection. The randomness is a feature rather than a compromise, because serverless invocation patterns shift unpredictably and a solution that does not depend heavily on any particular structure degrades gracefully when the structure changes. The willingness to give up on predicting invocations and instead make the reactive path cheap is the same stance taken by systems that attack what a cold start costs rather than when it happens, covered elsewhere on this site.

Following one function through a burst

A function finishes executing during a period of rising load

The platform must decide whether to keep this instance, in what form, on which processor, and for how long — while thousands of other functions make the same claim on the same budget.

Step 1 of 5

The optimisation interval closes

Every minute, CodeCrunch gathers all functions invoked during the interval and generates the space of possible decisions for them.

One minute matches the interval used by Amazon Lambda and Google Cloud Functions. Each candidate combination is generated so that its total keep-alive cost stays within the budget for the interval.

Why it works

Compression converts memory into warm starts. Smaller resident instances fit the same budget → warm start rate rises from 51% to 61% under a fixed policy, and by more than 10 percentage points under the full system → and the gain concentrates in exactly the high-load periods where the budget was previously exhausted.
The compression choice is made where it pays. Only functions whose decompression beats their cold start are compressed → 42% on x86 and 46% on ARM qualify → so the functions that would lose up to 75% by compressing are left alone.
Heterogeneity gives two independent levers. ARM is cheaper to hold for every function and faster for 38% of them → a function can be placed for speed, for holding cost, or both → and 60% of the ARM-faster functions are also compression-favourable on ARM, so the levers compound.
Banking budget across time matches spending to demand. Underspend during quiet intervals, overspend during bursts → 18 percentage points more warm starts than the baseline at identical total budget → because keep-alive demand is bursty while a flat budget is not.
Randomised projection makes the decision affordable. Optimise random low-dimensional subspaces instead of the full space → decision overhead falls to 4.52% of service time → which is what lets the system consider three choices per function for thousands of functions every minute.

What it costs (trade-offs and scope)

The headline is against the simplest baseline. The 32% improvement is over Serverless in the Wild. Against IceBreaker the mean service time gain is 17% and against FaasCache 34%. The authors are explicit that CodeCrunch is complementary rather than strictly competitive with prediction-based systems, and they demonstrate this by adding compression and heterogeneity to all three baselines, which improves all of them.
Some functions get worse. For 6% of invocations CodeCrunch is slower than FaasCache and IceBreaker. These are infrequently invoked functions whose re-invocation period exceeds sixty minutes, which CodeCrunch deliberately declines to keep alive.
Compression costs real CPU time. Compressing a function takes 1.57 seconds on average and up to 2.01 seconds. It runs off the critical path, but it is work the provider pays for on every function it decides to squeeze.
The budget comparison gives CodeCrunch flexibility the baseline lacks. CodeCrunch is granted the same total keep-alive spend as the baseline but is allowed to redistribute it across time. That redistribution is a genuine contribution — but part of the improvement comes from having permission to shift spending, not only from compression and placement.
One statistic is reported inconsistently. The body text states compression beats cold start for 58% of benchmark functions, while the figure caption and the later platform breakdown give 42% on x86 and 46% on ARM.
It is not measured on a commercial serverless platform. AWS Lambda does not expose keep-alive control, so the system runs on a self-built OpenWhisk cluster of EC2 instances — 18 ARM and 13 x86 nodes chosen for equal capital cost — replaying a two-week Azure trace of over 200,000 unique functions.

The evaluation

Property	Metric	Prior systems	CodeCrunch
Mean service time	SitW 8.9s, FaasCache 9.1s, IceBreaker 7.9s
Distance from the ideal	—
Decision-making overhead at 10 million functions
Service-time SLA violations at a 20% allowance	more than 19% of functions
Performance at a reduced budget	SitW at full spend: 8.9s

Measured on an OpenWhisk cluster of 18 ARM and 13 x86 EC2 nodes replaying a two-week Microsoft Azure trace with over 200,000 unique functions.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

The ablation is the most informative table in the paper and deserves more attention than the headline. Full CodeCrunch reaches 6.75 seconds; without compression it is 8.15; on x86 only 7.87; on ARM only 8.4; with a fixed ten-minute keep-alive instead of the optimised one, 7.38. So compression is the single largest contributor, and no one component carries the result alone. It is also worth noticing where the comparison is generous and where it is not. CodeCrunch is given the same total keep-alive spend as the baseline but permitted to move it between time periods, which is a real advantage the baseline does not have — though the authors argue reasonably that letting a provider set a budget rate is itself more practical than the alternative. And they report their losses: 6% of invocations are slower than under the prediction-based systems, and compression actively hurts the majority of functions, which is why the per-function decision matters so much. Finally, the fraction of functions for which compression beats a cold start is stated as 58% in one sentence and as 42% and 46% by platform elsewhere in the same paper.

Background

Where this sits

Almost all cold-start research has taken the same shape: cold starts are expensive, so predict invocations and warm functions just before they are needed. That produced increasingly sophisticated forecasters, and it also produced systems whose decision-making consumes a substantial fraction of the latency they are trying to save. CodeCrunch belongs to a smaller counter-current that asks what a cold start and a warm instance actually cost, and tries to change those quantities rather than the timing around them. Its two levers — reduce the memory a warm instance occupies, and place it where holding it is cheapest — are both about the economics of residency rather than the statistics of invocation. That framing also explains the paper’s own claim that its ideas are complementary: adding compression and heterogeneous placement improves the prediction-based systems too. The broader family of serverless work on cold starts, container reuse and memory efficiency is covered elsewhere on this site.

If you remember only three things

1. Cold starts account for 40% to 75% of the latency of a serverless function, and the usual fix — predicting invocations and pre-warming — fails under bursty load because the binding constraint is memory, not forecast accuracy.
2. Compressing the filesystem of an idle function in memory lets more functions stay warm within the same budget, raising warm start rates from 51% to 61%, but it only pays for the 42% to 46% of functions whose decompression is cheaper than their cold start.
3. Choosing compression, processor and keep-alive duration per function is a three-dimensional decision across thousands of functions per minute; a randomised low-dimensional projection makes it tractable at 4.52% of service time, against 30% for Fourier-based prediction.

Check yourself

Check your understanding

1. Why does predicting invocations more accurately fail to solve the cold start problem under bursty load?
2. When does compressing a warm function actually help?
3. What makes the x86 versus ARM choice harder than earlier heterogeneous serverless work?
4. Why does CodeCrunch use random low-dimensional projections instead of optimising directly?

Sources

The paper is “CodeCrunch: Improving Serverless Performance via Function Compression and Cost-Aware Warmup Location Optimization” by Rohan Basu Roy, Tirthak Patel, Rohan Garg and Devesh Tiwari, published at ASPLOS 2024 Volume 1, pages 85-101, DOI 10.1145/3617232.3624866. The work is from the Goodwill Computing Lab at Northeastern University, and the implementation is archived on Zenodo.

The ACM copy is nominally open access but sits behind an automated access check. This explainer is written from the full text, read from the author-hosted copy linked from the research group’s publication page. All figures above come from the paper’s own sections 2, 4 and 5: the 40% to 75% cold start share, the rise in warm starts from 51% to 61% under compression, the 42% and 46% compression-favourable fractions on x86 and ARM and the 75% worst-case penalty, the 38% of functions faster on ARM, the mean service times of 6.75 seconds for CodeCrunch against 8.9, 9.1 and 7.9 for the three baselines, the Oracle figures of 6.3, 6.99 and 10.2 seconds, the compression and decompression timings, the ablation values of 8.15, 7.87, 8.4 and 7.38 seconds, the decision overheads of 4.52%, 30% and 21%, and the cluster configuration of 18 ARM and 13 x86 nodes replaying a two-week Azure trace. Two internal inconsistencies are worth flagging for readers checking against the original: the compression-favourable fraction appears as 58% in the section 2 text but as 42% and 46% in the figure caption and the platform breakdown, and the introduction quotes improvements of 15% and 25% over IceBreaker and FaasCache in total service time where section 5 gives 17% and 34% in mean service time.