RainbowCake: Mitigating Cold-starts in Serverless with Layer-wise Container Caching and Sharing

Serverless platforms either cache whole containers (fast but wasteful) or share over-packed ones (frugal but slow) — RainbowCake peels containers into three layers, letting an idle container step down from function-specific to language-shared to universal instead of dying, so the next invocation from anyone starts partway warm.

Why this problem is worth solving

Serverless computing’s promise is that users upload a function and the platform handles everything else — provisioning, scaling, billing per invocation. The catch is hiding in “everything else”: each invocation needs a

If arrivals were predictable, platforms could simply warm containers just in time. They are not. The Azure Functions production study the paper builds on found that over 50% of functions have significantly varying invocation patterns; large fractions are invoked rarely (once an hour) yet still expect fast starts. So every caching policy lives with mispredictions — and a mispredicted full container is a double loss: the invocation cold-starts anyway, and hundreds of megabytes sat idle for nothing. At platform scale, idle cached memory is a direct cost; the paper measures baselines wasting several times more memory-seconds than necessary. This site’s serverless explainers (IceBreaker, AQUATOPE, INFless) all orbit this same prediction problem; RainbowCake’s angle is to make the cache entries themselves degrade gracefully when predictions fail.

What people did before, and where it broke

Full-container caching. The industry default: keep a finished container alive for a fixed window (OpenWhisk and commercial platforms use ~10 minutes), or predict per-function inter-arrival times and schedule pre-warming and keep-alive around them (the Azure Histogram policy), or manage the pool greedily like an object cache (FaaSCache). All variants share a weakness: a finished container is specialized — it serves exactly one function. Wrong function, wrong time → cold start plus wasted memory.

Partial-container caching. SEUSS and similar systems observed that initialization has stages, and cached snapshots partway through — much lighter in memory. But an invocation hitting a partial container still pays the remaining initialization: halfway cold-start mitigation.

Container sharing. Pagurus and relatives recycle idle containers across functions by packing multiple functions’ dependencies into one monolithic image. Sharing tolerates mispredictions (someone else’s container can serve you), but the over-packed containers are heavy, so the same memory budget holds fewer of them — excessive memory waste in exchange for generality.

The paper’s motivating experiment runs all three philosophies on the same 8-hour production trace: partial caching wins on memory but loses on latency; sharing helps latency but bloats memory; full caching sits in the uncomfortable middle. The two fine-grained ideas were never combined — that combination is RainbowCake.

The idea

Initialization has natural seams — cut there. A cold start runs three stages: environment setup (network, logging, health checks), language runtime initialization (pull and start Python/Node.js/Java), and user package loading (code, libraries, environment variables). RainbowCake freezes containers at these boundaries as Bare, Lang, and User containers. The seams are real system boundaries — nothing is broken by stopping between them — and they produce a clean gradient: lower layers are lighter and more shareable; higher layers are heavier and save more latency. A Bare container serves any function but leaves the most work to do; a User container starts instantly but serves exactly one function.

Downgrade, don’t kill. Traditional keep-alive is binary: a container idles at full readiness, then dies. RainbowCake’s containers descend a ladder instead. When a User container’s time-to-live (TTL) expires, the platform wipes the user package (and only that) — leaving a Lang container that any same-language function can claim. When that TTL expires, the runtime is removed, leaving a universal Bare container. Only after the third timeout does the container terminate. Each downgrade releases most of the memory-liability while preserving the layers most likely to be reused — the misprediction penalty becomes graduated rather than total.

Model the sharing, not just the function. Here is the quantitative heart. For each function, the History Recorder fits a Poisson arrival model over its recent invocations. But a Lang(Python) container is not waiting for one function — it can be hit by any Python function, so its relevant arrival process is the compound of all Python functions’ models (rates add). A Bare container compounds the entire workload. This is why lower layers earn their keep: their aggregated hit rates are far higher than any single function’s, which a function-specific predictor would never see.

Cap every TTL by an economic argument. Keep-alive length per layer is the predicted time-to-next-hit (at quantile p = 0.8), but never more than a bound β = α·t̄ / ((1−α)·m̄), where t̄ is the layer’s startup saving, m̄ its memory footprint, and α the platform’s latency-vs-memory knob (0.996 in the evaluation). The bound enforces a simple invariant: a container may never burn more idle-memory cost than the startup cost it could possibly save. Past that point, holding it is a guaranteed loss regardless of predictions.

Wipe before sharing. Sharing raises the obvious security question. Downgrades are engineered to be forgetful: functions run as non-root inside a jailed Action folder; the User→Lang transition deletes that folder and unsets user environment variables via a dedicated Clean handler, and user-spawned processes are tracked by PID. Lang and Bare containers are guaranteed free of user remnants before anyone else touches them.

Bare → Lang → User → run
cold start: hundreds of ms, comparable to execution

Why it works

• Initialization stages are real system seams → freezing at Bare/Lang/ User breaks no dependencies → partial containers are always safely resumable, and upgrades install only what is missing.
• Shareability and cost are inversely stacked → the cheap-to-keep layers are exactly the widely-reusable ones → memory spent on idle containers concentrates where hit probability is highest.
• Arrival rates add across functions → a Lang container’s compound hit rate dwarfs any single function’s → lower layers are statistically worth keeping even when each individual function is unpredictable — bursts from any same-language function can land on them.
• The β cap bounds every container’s idle cost by its potential saving → even a fully wrong prediction loses at most what a warm start would have won → memory waste is structurally bounded, not merely hoped away.
• Downgrades convert mispredictions from total losses into partial ones → the system degrades gracefully exactly when predictors fail (bursty traces) → which is where the baselines fall apart.

What it costs

• A small latency concession to the greediest baseline. FaaSCache — which never terminates containers until eviction — beats RainbowCake by about 50 ms average (20 ms startup); RainbowCake repays that with 75% less memory waste.
• Residual partial cold starts. Lang and Bare hits still pay the missing layers’ initialization; only User hits are instant.
• Modeling assumptions. Poisson arrivals over a 6-invocation sliding window, quantile and knob parameters (p = 0.8, α = 0.996) tuned by sensitivity study — different platforms may need retuning.
• Trust in the wipe. Cross-function sharing of Lang/Bare containers rests on the Clean handler, non-root execution, and jailing removing every user remnant; the isolation argument is engineering discipline, not a new formal boundary.
• Transition machinery. Downgrade/upgrade HTTP handlers and FSM states add moving parts, though measured at under 3% of startup latency.

The evaluation

RainbowCake is implemented in Apache OpenWhisk (8K lines of Scala, open source) and evaluated on a three-node AWS EC2 cluster whose 96-core, 240 GB worker runs the containers. The workload: 20 real functions across Node.js, Python, and Java from three benchmark suites (web, data analytics, ML inference, scientific, video), driven by real Azure Functions production traces — one 8-hour set for overall results plus seven 1-hour sets with burstiness (inter-arrival coefficient of variation) swept from 0.2 to 4.0.