Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs

Cloud providers sell their spare capacity at 60 to 90 percent off, with one condition — they can take it back at any moment, killing your machines without warning, which is why most customers pay full price rather than risk it; this paper (Microsoft Azure) starts from a measurement that changes the calculation, namely that those seizures are not random at all, since more than 90 percent of them happen because a full-paying customer wanted that hardware, so they cluster on weekday mornings and can be forecast hours ahead; on top of that forecast it runs a learning agent that continuously adjusts the mix of cheap-and-revocable against expensive-and-safe machines, and crucially it is told to minimize cost SUBJECT TO an availability floor rather than to balance the two, because an agent merely rewarded for saving money learns to dump every machine at the end of the month once its average is safe; the agent also invents a tactic nobody programmed — grab extra cheap machines, see which ones land on hardware likely to be reclaimed, and quietly release those — and ends up 44 percent cheaper than paying full price while staying available 99.96 percent of the time.

The argument in one glance

Problem

The cheapest cloud capacity can be taken away without warning

A cloud provider always has hardware nobody is currently renting, and it sells that spare capacity at a steep discount — commonly 60 to 90 percent below the normal price. The condition is that the capacity is revocable. The instant a full-paying customer wants that hardware, the discounted machines running on it are terminated with almost no notice, an event called EVICTION. That makes the discount genuinely hard to accept, because the trade-off between saving money and staying available is opaque and the failure mode is severe rather than gradual. The paper documents a real incident in which the fraction of discounted machines being reclaimed jumped to roughly 60 percent, degrading a service during its busiest traffic. So customers face a choice between paying full price for machines they know will survive, or accepting a discount whose risk they cannot quantify.
What people did

Pick one, or pad with a fixed amount of extra cheap capacity

The practical responses are simple. Run entirely on full-price machines and accept the bill. Run entirely on discounted machines and accept the interruptions, which suits work that can be restarted. Or compromise — hold a fixed ratio of the two, or request more discounted machines than you actually need so that some survive any given reclamation. There are also tools that automatically swap in discounted capacity where they can.
Why it fell short

A fixed choice cannot follow a risk that moves

Every one of those is a decision made once, while the risk changes continuously. The paper's measurements show reclamations cluster sharply — heavier on weekdays and during working hours, peaking around seven in the morning — because they are not random events at all. More than 90 percent of them occur because a full-paying customer deployed onto that hardware, so the risk you face is driven by other customers' behaviour and moves with their daily rhythm. Against a moving risk a fixed ratio is wrong in both directions: it wastes money during the many calm hours, and it fails during the few bad ones. Padding with extra discounted machines has the same defect, since the amount of padding needed varies by hour, and it also ignores that identical machines are not equally safe, because survival depends on the specific hardware each one landed on.
This paper's idea

Forecast the reclamations, then let a constrained agent adjust the mix hourly

This paper (Microsoft Azure — Yang, Wang, Xu and colleagues) builds Snape on a finding from three months of production traces: because reclamations are driven by full-paying deployments rather than chance, they are partly PREDICTABLE. A forecasting model estimates the reclamation rate for a whole region hours ahead with 11.18 percent error, against 42.68 percent for a linear baseline. On top of that, a reinforcement learning agent adjusts the mixture of cheap and safe machines every hour. The critical design choice is how the objective is stated — not as a balance between cost and availability, but as MINIMIZE COST SUBJECT TO an availability floor, because an agent simply rewarded for spending less learns to release every machine near the end of the period once its running average already satisfies the target. A safety layer therefore projects every proposed action to the nearest one that keeps enough machines in service right now. The agent also discovers a tactic nobody programmed — take extra discounted machines, observe which landed on hardware likely to be reclaimed, and release those — reaching 44 percent below full price at 99.96 percent availability.

Why this problem is worth solving

The discount on offer here is enormous, and the reason it goes unclaimed is a risk nobody could quantify — so it is worth understanding what is actually being sold.

SPOT capacity is a cloud provider’s spare hardware, sold at a steep discount — commonly 60 to 90 percent below the standard ON-DEMAND price — on the condition that it can be taken back. When a customer paying full price deploys onto that hardware, the discounted machines running there are terminated, which is called EVICTION, and it happens with almost no notice. This is not a fault or a rare failure; it is the arrangement working as designed, since the provider is renting out capacity it has already promised to someone else if they ask for it. The paper’s measurement of why evictions happen is the crucial one: more than 90 percent are caused by on-demand deployments. That single fact reframes the whole problem, because it means eviction is not a random hazard but a consequence of other customers’ behaviour — which has structure, follows a daily and weekly rhythm, and can therefore be anticipated rather than merely endured. Machines are managed in groups of identical instances behind a load balancer, and it is that group whose composition can be adjusted

. The prize is large — the discount is most of the bill — and the obstacle is purely the unquantified risk.

Turning that into a decision requires being careful about what is being optimized.

A SERVICE LEVEL OBJECTIVE is a commitment about how often a system must be working — for instance that the required number of machines will be in service 99.9 percent of the time over a month. The natural way to bring this into an optimizer is to treat cost and availability as two terms to be balanced, but that is the wrong shape, because availability below the promised level is not merely undesirable, it is a broken commitment. The right formulation is CONSTRAINED optimization: minimize cost SUBJECT TO availability staying above the floor. In reinforcement learning, where an agent learns a policy by receiving rewards for its actions, this is expressed as a constrained decision process with two kinds of limit. A CUMULATIVE constraint bounds the average shortfall over the whole period, and an INSTANTANEOUS constraint requires that enough machines are in service at every individual step. Both are needed, and the reason is instructive — an agent facing only the cumulative constraint can satisfy it early and then release everything at the end, technically meeting its average while leaving the service with nothing

What people did before, and where it broke

Pay full price for everything. Perfectly reliable and the default for anything that matters, but it forgoes a discount worth most of the bill, and the provider’s spare capacity sits unused.
Run entirely on discounted capacity. Cheapest possible, and genuinely right for restartable batch work. But availability lands around 97 percent in the paper’s measurements, far below what a service needs, and it collapses during reclamation surges.
Request more discounted machines than needed. Padding survives ordinary reclamations, but the amount of padding required varies hour by hour, so a fixed amount over-pays most of the time and still fails when a surge exceeds it.
Hold a fixed ratio of cheap to safe machines. A sensible compromise that is nonetheless a single decision made against a risk that moves, so it is simultaneously too cautious during calm hours and too aggressive during bad ones.
Automatic substitution tools. Existing tooling swaps in discounted capacity opportunistically, which helps, but it reacts to what has already happened rather than anticipating what is about to.
The gap. Every approach treats eviction risk as a fixed property to be hedged against, when the production data shows it is a moving quantity driven by other customers’ deployments — predictable in aggregate, and varying between individual machines depending on which hardware they landed on. That is the gap Snape fills.

How the mixture is chosen and continuously adjusted

Forecast the risk, decide under a hard constraint, and never let the agent cheat

A fixed choice faces a moving risk

Reclamations cluster on weekday working hours and are driven by full-paying deployments rather than chance, so a fixed ratio of cheap to safe machines wastes money during calm hours and fails during surges. Padding with extra discounted capacity has the same defect.
↓ →
Measure why evictions actually happen new

Three months of production traces from one region show that more than 90 percent of evictions are caused by on-demand deployments needing the hardware, that reclamations peak around seven in the morning on weekdays, and that survival varies between machines depending on the node each landed on.
↓ →
Forecast the regional reclamation rate new

A transformer-based forecasting model takes a month of hourly regional history — machine counts, utilization, capacity, past evictions — and predicts the eviction rate from one hour to seven days ahead, with 11.18 percent error over short horizons against 42.68 percent for a linear baseline.
↓ →
Describe each machine by how safe its hardware looks new

Two node-level measures are introduced. Spottiness is the ratio of discounted to full-price cores on a node, and higher spottiness means longer survival. Stableness captures how much a node's free capacity fluctuates, and more fluctuation means earlier eviction.
↓ →
Let an agent adjust the mixture under a hard floor new

Every hour a reinforcement learning agent chooses how many discounted machines to add and which specific instances to release, rewarded only for lowering cost. Availability is imposed as a constraint rather than a competing reward, with the regional forecast reshaping rewards so the agent is not shortsighted.
↓ →
Project every action into the feasible set new

A safety layer replaces any proposed action with the closest one that still leaves enough machines in service right now. Without it, an agent whose average availability is already satisfied would release everything at the end of the period to minimize cost — meeting the letter of the constraint while destroying the service.

Evaluated on Azure production traces from the eastern United States over three months, managing groups of identical machines with targets of 10 to 20 in service, against six baselines including pure on-demand, pure spot, fixed ratios and existing substitution tooling.

The idea

Check whether the thing everyone treats as random actually has structure. The entire design rests on one measurement. Eviction is normally regarded as an unpredictable hazard — the price of the discount, to be hedged against rather than anticipated. The traces say otherwise: more than 90 percent of evictions happen because a full-paying customer deployed onto that hardware. Once that is established the conclusion follows immediately, because other customers’ deployments follow a daily and weekly rhythm, so reclamations cluster on weekday mornings and can be forecast hours ahead. What looked like noise is someone else’s signal. Discovering that an apparently random process is driven by an observable cause, and therefore predictable, is the same look-for-structure-in-the-noise move covered elsewhere on this site, where a system stops hedging against an unknown and starts forecasting it — and here it converts an unquantifiable risk into a managed one.

State the objective as a constraint, because an optimizer exploits whatever you did not constrain. The most transferable lesson is about problem formulation. Availability could have been folded into the reward as a term to be balanced against cost, and that would have been wrong, because falling below the promised level is a broken commitment rather than a slightly worse outcome. So cost is minimized subject to an availability floor. But even that is not enough, and the reason is the sharpest detail in the paper: an agent bound only by an AVERAGE availability constraint learns that if it has been reliable early, it can release every machine at the end of the period and still satisfy the average — minimizing cost exactly as instructed while leaving the service with nothing. The fix is a safety layer that replaces each proposed action with the nearest one keeping enough machines running right now. That an optimizer will find and exploit precisely the gap you left in the specification is the same you-get-what-you-measure lesson covered elsewhere on this site, where a compiler super-optimizer degrades the one dimension absent from its cost model.

Two levels of information, because the aggregate does not describe the individual. The forecast predicts eviction for a whole region, which is what makes anticipation possible, but a particular group of machines does not experience the regional average — it experiences whatever happens on the specific hardware its instances landed on. So the system uses both, with the regional forecast reshaping the agent’s rewards to keep it from being shortsighted, while the agent learns from local observations how its own group deviates. The paper shows this beats using the regional prediction alone. The node-level measures it learns from are themselves discovered rather than assumed — the ratio of discounted to full-price cores on a node, and how much that node’s spare capacity fluctuates — which is the same find-the-metric-that-actually-predicts discipline covered elsewhere on this site, where authors test the obvious quality indicator, find it weaker than two computed ones, and adopt those instead.

Deciding this hour’s mixture

From an unquantified risk to a managed one

One hourly decision in Snape. Numbers from the paper's production traces and evaluation.

Step 1 of 5

What the customer has asked for

A group of identical machines sits behind a load balancer, and the customer states two things — how many must be in service, here between 10 and 20, and how often that must hold, for instance 99.9 percent of the time over a month. Discounted capacity costs about 70 percent less but can be reclaimed at any moment.

adding a machine takes time before it is usable, while removing one takes effect immediately, so the two actions are not symmetric
the system adjusts the group composition but does not control which hardware the provider places machines on

Why it works

More than 90 percent of evictions are caused by full-paying deployments → the risk follows other customers’ daily and weekly rhythm rather than chance → so it can be forecast hours ahead instead of merely hedged against.
The forecast covers a whole region while a customer’s group experiences only its own nodes → combining the regional prediction with local node measures captures both the trend and the deviation → which beats using the regional forecast alone.
Availability is a commitment, not a preference → expressing it as a constraint rather than a reward term means cost is minimized only among policies that keep the promise → so savings never come from quietly breaking it.
An average constraint can be satisfied early and then abandoned → a safety layer projects each action to the nearest one keeping enough machines running now → closing the gap an optimizer would otherwise exploit.
Identical machines differ in survival depending on the node they landed on → taking extra machines and releasing those on unfavourable nodes converts a placement lottery into a selection → which is the tactic the agent discovers by itself.

What it costs (trade-offs and scope)

It buys less discount than going all-in. Running purely on discounted capacity saves around 60 percent against Snape’s 44 percent. The difference is what reliability costs, and for restartable batch work that does not need an availability commitment, the simpler and cheaper option remains correct.
It depends on forecasts that decay quickly. Prediction error grows sharply with horizon, so long-range forecasts guide rather than determine decisions, and the paper notes eviction-rate correlations weaken after about four weeks, requiring retraining every two weeks.
It is trained and evaluated narrowly. All results come from one region over one three-month window, with groups of identical machine types. The authors state that other regions may behave quite differently and that mixed machine types are future work.
It carries real operational machinery. A forecasting model per region retrained fortnightly at about 16 hours each, a learning agent needing roughly 300 episodes to converge, feature extraction per region and node, and drift monitoring — substantially more to run than a fixed ratio.
Some conditions are assumed stable and some questions are left open. The discount ratio held steady at about 70 percent throughout; the system does not control machine placement; and the paper does not work through what happens when the forecast itself fails.

The evaluation

Snape was assessed on Azure production traces from the eastern United States covering three months, driving an emulator that replays real placement data. Twenty machine groups were run per method with targets of 10 to 20 machines in service and discounted capacity priced about 70 percent below full price, against six baselines — paying full price throughout, using only discounted capacity, over-requesting discounted capacity, holding a fixed mixture ratio, an existing automatic substitution tool, and an optimizer driven by the regional forecast alone.

Buying cloud capacity under revocation risk

Property	Pay full price	Use only discounted capacity	Snape
Cost against full price
Availability achieved
Responds to changing risk
Distinguishes individual machines
Operational burden

Snape reaches 44 percent below full price at 99.96 percent availability, 2.77 percentage points above using only discounted capacity and comfortably inside a 99.9 percent commitment. Pure discounted capacity saves more but lands near 97 percent availability, which is the gap reliability costs. The regional forecast achieves 11.18 percent error over short horizons against 42.68 percent for linear regression.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

Read the trade honestly — this buys reliability with discount. Running entirely on discounted capacity saves about 60 percent against Snape’s 44, so for restartable batch work carrying no availability commitment the simpler option remains correct, and Snape is aimed at services that genuinely cannot tolerate interruption. Its foundation is a forecast that decays quickly, with error growing sharply as the horizon lengthens and eviction-rate correlations weakening after roughly four weeks, which forces retraining every two weeks at about 16 hours per region. The evidence base is narrow: one region, one three-month window, groups of identical machine types, with the authors noting that other regions may behave quite differently and that mixed types are future work. The machinery is substantial next to a fixed ratio — a forecasting model per region, a learning agent converging over roughly 300 episodes, per-node feature extraction and drift monitoring. Several conditions are assumed rather than tested, including a discount ratio that held near 70 percent throughout, and the system does not control where the provider places machines. The paper also does not work through what happens when the forecast itself fails, which matters for a design whose safety rests on a constraint layer rather than on the prediction being right. Within that scope the lessons travel well: check whether a risk treated as random is actually driven by something observable; state a commitment as a constraint rather than a term to be balanced; and expect an optimizer to exploit exactly the gap your specification leaves open.

Background

Find the structure in the noise, then constrain what you refuse to trade away

This paper sits in the site’s cloud and datacenter cluster and rhymes with several ideas covered elsewhere on this site. Its foundation is the discovery that a hazard everyone treats as random is in fact driven by an observable cause — more than 90 percent of evictions come from full-paying deployments, so they inherit those customers’ daily rhythm and become forecastable. That is the same look-for-structure-before-hedging move the site rewards, and it converts an unquantifiable risk into a managed one. Its refusal to pick a side, adjusting the mixture continuously instead of committing to cheap or safe, is the reject-the-dichotomy-and-decide-at-runtime instinct the non-uniform-bandwidth page shows when it switches page placement policy according to a measured balance score, and the kernel-machine-learning page shows when it measures the batch size at which acceleration begins to pay. Sharpest of all is the safety layer, which exists because an agent constrained only on AVERAGE availability learns it can be reliable early and then release everything at the end — a textbook case of an optimizer exploiting the gap left in its specification, and the same you-get-exactly-what-you-measure lesson the compiler super-optimizer page teaches when the one dimension missing from its cost model quietly degrades. Its node-level measures, discovered from traces rather than assumed, echo the quantum-chiplet page testing the obvious quality metric and adopting two computed indicators that predict better. And the tactic the agent invents unprompted — take extra machines, then release the ones on unfavourable hardware — is the same detect-and-discard strategy the quantum-noise and quantum-chiplet pages apply when they reject a contaminated sample rather than trying to repair it.

If you remember only three things

1. Cloud providers sell spare hardware at 60 to 90 percent below the normal price, on the condition that it can be reclaimed without warning the moment a full-paying customer wants it. That discount mostly goes unclaimed because the risk is unquantifiable and the failure is severe — the paper documents an incident where roughly 60 percent of discounted machines were reclaimed at once, degrading a service at peak. The simple responses all fail in the same way: paying full price forgoes most of the bill, running purely on discounted capacity lands near 97 percent availability, and fixed ratios or extra padding are single decisions made against a risk that moves hour by hour.
2. The finding that changes the calculation is that eviction is not random. More than 90 percent of evictions happen because a full-paying customer deployed onto that hardware, so reclamations inherit other customers' daily and weekly rhythm, cluster on weekday mornings, and can be forecast — a transformer model predicts regional eviction rates with 11.18 percent error against 42.68 for a linear baseline. Snape pairs that regional forecast with local node measures discovered in the traces, the ratio of discounted to full-price cores on a node and how much its spare capacity fluctuates, then lets a reinforcement learning agent adjust the mixture every hour.
3. The formulation matters more than the algorithm. Availability is imposed as a CONSTRAINT rather than balanced against cost, because falling below a commitment is not merely a worse outcome — and even that is insufficient, since an agent bound only by an average can be reliable early and then release everything at the end, minimizing cost exactly as instructed. A safety layer therefore projects every action to the nearest one keeping enough machines running now. The result is 44 percent below full price at 99.96 percent availability, with the agent inventing an unprogrammed tactic of overprovisioning then discarding machines that landed on unfavourable hardware. The transferable lessons: check whether a risk treated as random is driven by something observable; state commitments as constraints; and expect an optimizer to exploit whatever your specification leaves open.

Check yourself

Check your understanding

1. The whole design rests on one measurement about why evictions happen. Predict what it was and why it changes the problem.
2. Availability is imposed as a constraint rather than balanced against cost in the reward. Predict why that distinction matters.
3. A safety layer replaces each proposed action with the nearest feasible one. Predict what failure it exists to prevent.
4. The agent discovers a tactic nobody programmed — take extra discounted machines, then release some. Predict what it is exploiting.

Sources

Snape: Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs (ACM DL, DOI 10.1145/3582016.3582028) — ASPLOS 2023; full text read for this explainer.
Microsoft Research publication page — abstract and citation details.
ASPLOS 2023 conference talk — the authors’ presentation of the design and results.