IceBreaker: warming serverless functions better with heterogeneity

Serverless platforms pay for expensive servers that sit ready in case a function is called, so IceBreaker keeps each waiting function on a cheap or a costly server depending on how likely the function is to be called soon.

Why this problem is worth solving

Serverless computing is a cloud model where the customer uploads a function — a small program that handles one kind of request — and the cloud provider runs it on demand. The customer does not rent a server. The provider finds a machine, runs the function, and charges only for the run.

The weak point of this model is the start. When a request arrives for a function that is not loaded anywhere, the provider must create an execution environment first: reserve a slice of a machine, start the language runtime, and load the function’s code and libraries. Only then does the actual work begin. This delay is called a cold start, and it can take far longer than the function’s own work. A person clicking a button waits through all of it.

Providers avoid cold starts by warming up functions: loading a function into memory before its anticipated execution, so the next request finds it ready. A warm function skips the preparation entirely.

But warmth is not free. A warm function occupies memory on a real server the whole time it waits. The cost of holding it there, second after second, is the keep-alive cost. The paper’s abstract states the dilemma plainly: warming up can be inaccurate, and the keep-alive cost during the warm-up period is prohibitively expensive. The platform pays for readiness that may never be used, on hardware chosen for speed rather than for waiting.

What people did before, and where it broke

The standard industrial answer is a fixed timer. After a function runs, the provider keeps it warm for a fixed number of minutes, then evicts it. This is simple and fair, and it works well for functions that are called steadily. It breaks in both directions for everything else. A function called every hour is evicted long before its next call, so every call is cold. A function that was called once and never again is held warm for the full window, and that memory is wasted.

Research systems improved the guess. Instead of one fixed window for every function, they studied each function’s invocation history and adapted the policy per function — keeping frequently-called functions warm longer and evicting rare ones sooner.

Both approaches share one silent assumption: all the servers doing the waiting are the same kind of machine. The decision is only whether and when to keep a function warm. Where to keep it warm — on what class of hardware — is not part of the question. Since the machines are the expensive kind that providers buy for running work fast, every warm slot is an expensive slot, and the budget runs out after a limited number of warm functions.

That unasked question is the opening IceBreaker walks through. The abstract states that IceBreaker is the first technique to mix expensive and cheaper nodes to improve both service time and keep-alive cost for serverless functions.

The idea

IceBreaker changes the shape of the warming pool and the rule that fills it.

Heterogeneous nodes. A node is one server in the pool. IceBreaker composes the system from two node types: costly nodes and cheaper nodes. The names are the abstract’s own. A costly node is the class of machine platforms already use. A cheaper node costs less to keep running — and because it costs less, the same total budget buys more of them.

A time-varying probability per function. Real functions are not called at random. Many follow patterns — busy during working hours, quiet at night, or called on a schedule. So the chance that a given function will be invoked in the next moment is not one fixed number; it rises and falls over time. IceBreaker estimates this time-varying probability of the next invocation for each function. The abstract does not describe the estimator’s internals, so this page does not either.

The matching rule. With two node types and a probability per function, IceBreaker dynamically picks the cost-effective node type for each function’s warm copy. The logic the abstract describes is a matching between how much readiness is worth right now and how much that readiness costs. When a call is unlikely, an expensive warm slot is mostly waste. When a call is imminent, readiness is about to pay off. The decision is re-made as the probability changes, which is what “dynamically” means here.

Why cheap nodes help everyone, not just the budget. This is the part of the abstract worth reading twice. Under a fixed total cost, replacing some costly nodes with several cheaper ones increases the number of nodes. More nodes means more functions can be held warm at the same time. More warm functions means fewer requests land on a cold function — and the abstract adds that this reduces the wait time during high load, exactly when many functions compete for warm slots.

14:00 | p = low
warm copy: none
cost now: 0
risk: a surprise call would start cold

Why it works

The causal chain is short.

Because cheaper nodes cost less per unit of time, a fixed budget buys more nodes when some of them are cheap. → Because there are more nodes, more functions can be held warm at once. → Because more functions are warm, fewer incoming requests hit a cold start, and the wait during high load shrinks. → Because the probability of a call rises and falls per function, spending on the costly readiness only in the high-probability windows removes the most expensive waste — an expensive machine holding a function nobody is about to call. → Therefore both numbers can fall at once: the keep-alive cost, because readiness is bought at the right price, and the service time, because more requests find a warm copy waiting.

The last line is what makes the paper interesting. Cost and speed usually pull in opposite directions; buying less readiness normally means more cold starts. IceBreaker’s claim is that pricing readiness correctly — not buying less of it — improves both at the same time.

What it costs

The abstract does not list the design’s prices, so the following are the natural costs of this kind of design, stated as expectations rather than reported facts.

• It depends on prediction. The whole mechanism keys off the estimated probability of the next invocation. A function with an irregular, patternless call history gives the estimator little to work with. Wrong estimates buy readiness that is wasted, or skip readiness that was needed.
• A cheaper node is presumably a weaker node. A function served from a cheap node is warm, but the machine it runs on costs less for a reason. The full paper’s handling of this trade — whether the run stays on the cheap node or moves — is not described in the abstract.
• Two node types mean more moving parts. The platform must operate two hardware classes and continuously re-place warm copies between them, instead of managing one uniform pool.

The evaluation, as far as the abstract reports it

The authors evaluated IceBreaker on a real system — not only in simulation — using representative serverless applications and an industry-grade workload trace. The abstract reports two headline results, and both numbers below come from the abstract alone:

• The overall keep-alive cost fell by 45%. A readiness bill of 1,000 dollars becomes about 550 dollars for the same workload.
• Execution time fell by 27%. Work that took 10 seconds now finishes in about 7.3 seconds. The abstract attributes this to more functions being warm and to less waiting under high load.

The abstract does not describe the baseline configuration, the hardware used for the costly and cheaper nodes, or the per-application results, so this page cannot report them.