Expanding Datacenter Capacity with DVFS Boosting: A safe and scalable deployment experience

Leonardo Piga, Iyswarya Narayanan, Aditya Sundarrajan, Matt Skach, Qingyuan Deng, Biswadip Maity and 4 others

A datacenter can only run as much compute as its power budget allows — the breakers and cooling, not the floor space, are the limit — and building a new one takes years; but the power that's already provisioned is usually underused, because servers rarely run their CPUs at full speed, so this paper (Meta) reclaims that stranded headroom with DVFS BOOSTING — turning up CPU frequency to squeeze more throughput out of existing machines. The hard part is doing it safely across a giant heterogeneous fleet without tripping a power breaker (which would throttle hosts and cause outages), so Meta uses an ML model to pick the services that actually benefit, a cheap "useful instructions" proxy to validate gains, a power-risk simulator that keeps every panel under its limit within a tiny risk budget, and one crucial rule — only ever raise frequency, never lower it — to shrink coordination from thousands of teams to under ten; rolled out from 20% to 99.8% of hosts over a year, it added 25% throughput per host for 23% more power, effectively building half a datacenter's worth of capacity (12.85 MW) in weeks instead of years.

The argument in one glance

Problem

A datacenter is limited by power, and its power is only partly used

The thing that limits how much compute a datacenter can run is not floor space or the number of servers — it is POWER. The electrical infrastructure (the breakers, panels, and cooling) sets a hard ceiling, and once you have provisioned that power, you can only house so many servers under it. Building a new datacenter to add capacity takes months to years and costs a fortune. Here is the opening: that provisioned power is usually UNDERUSED. Servers rarely run their CPUs at maximum frequency — in fact, to fit MORE hosts per facility within the power budget, operators often deliberately run CPUs slower. So there is headroom, stranded power the facility paid for but does not fully consume, sitting between what the servers actually draw and the budget.
What people did

Build more, or just turn the CPU frequencies up

The default way to get more capacity is to build or buy more — more servers, more datacenters — which is slow and, during a supply crunch, sometimes impossible. The tempting alternative is DVFS: a CPU can run at a higher frequency and voltage to do more work per second. So you could simply raise the clock speeds across the fleet to convert the stranded power headroom into extra throughput, adding capacity from hardware you already own, in days instead of years.
Why it fell short

Turning frequencies up blindly trips breakers and causes outages

Boosting naively is dangerous at scale. Power scales SUPERLINEARLY with frequency (roughly cubic with voltage), so a modest speed increase costs disproportionately more power — a median 20% more power for about 10% more throughput. If the combined draw on any electrical panel exceeds its breaker, the datacenter's protection kicks in and CAPS (throttles) hosts to prevent a trip, causing sudden, unexpected capacity LOSS for the services on that panel — the exact opposite of the goal, and a reliability hazard. And the fleet is not uniform: power is unevenly distributed across panels (up to 12% variance), hardware spans multiple generations, and services have wildly different bottlenecks, so a blanket boost helps some hosts, wastes power on others, and endangers still others. Knowing WHICH hosts to boost, HOW MUCH, and how to guarantee no breaker ever trips, across an entire fleet, is the real problem.
This paper's idea

Boost the right hosts, keep every panel safe, and only ever raise frequency

This paper is Meta's deployment experience of DVFS BOOSTING at fleet scale, done safely. Several pieces make it work. An ML classifier predicts which services actually benefit (from micro-architectural counters, at 95% accuracy) — CPU-bound, high-instructions-per-cycle services with little memory waiting — while a cheap proxy called useful_MIPS (instructions minus overhead) validates the throughput gain at 85% accuracy where A/B testing is unavailable. A power-risk SIMULATOR estimates each panel and region's draw and boosts only within a tiny risk budget (at most 0.5% of regional capacity at capping risk), mitigating the rest by rack moves, rebalancing, or withholding at-risk capacity for interruptible workloads. And the central operational discipline — "optimize the baseline to consolidate risk" — only ever transitions frequency LOW to HIGH (never the reverse, which would surprise services with sudden capacity loss) and pre-allocates recovery buffers, shrinking the coordination problem from thousands of service teams to under ten infrastructure teams. Rolled out from 20% to 99.8% of hosts over a year during the COVID supply crisis, it added 25% throughput per host for 23% more power — 12.85 megawatts, "half a datacenter," in weeks instead of years, with no reliability impact.

Why this problem is worth solving

Datacenters are among the most capital-intensive infrastructure a company builds, and the binding constraint on them is not the one most people imagine. Understanding that constraint comes first. A

a datacenter’s capacity is capped by POWER, not by physical space or server count. The electrical system — main switches, switchboards, and leaf-level panels, each with a breaker rating — can only deliver so many watts, so the number of servers you can run at once is limited by their combined power draw, not by how many racks fit on the floor. Because building new power infrastructure takes years, this ceiling is the scarce resource. DVFS (Dynamic Voltage and Frequency Scaling) is the CPU’s ability to change its clock frequency and voltage: raising them makes the processor do more work per second but draws more power, and crucially power scales SUPERLINEARLY with frequency — roughly cubically with voltage — so a small speed gain costs a disproportionately large amount of power. That superlinearity is exactly why operators often run CPUs BELOW maximum: slower cores fit more servers under a fixed power budget, trading per-server speed for server count

is the setup. The key realization is that the power a facility has already paid for is usually not fully used, so there is stranded capacity to harvest — if you can do it without breaking anything.

The thing that makes “harvest it” dangerous, and that the whole design guards against, is the automatic protection that keeps the lights on. A

POWER CAPPING is the datacenter’s safety mechanism: if the power drawn on an electrical panel approaches its breaker limit, the system automatically THROTTLES (caps) the hosts on that panel — forcing their frequency down — to prevent the breaker from tripping and cutting power entirely. Capping is protective, but from a service’s point of view it is an unplanned, sudden loss of capacity, which can violate service-level agreements and destabilize systems that were counting on those hosts. This is the trap of boosting: raising frequencies pushes panels toward their limits, and if the combined draw of many boosted hosts on one panel — worsened by uneven power distribution, correlated load spikes, and hardware heterogeneity — crosses the breaker rating, capping fires and you LOSE capacity instead of gaining it. So the safety requirement is absolute: boost aggressively enough to gain real throughput, but never let any panel exceed its breaker, across a fleet where the margins vary host to host

is the constraint every mechanism serves. The problem is worth solving because the prize is enormous — capacity in weeks, not years — but only if the safety is airtight.

What people did before, and where it broke

Build or buy more capacity. The standard answer: procure more servers and datacenters. Correct but slow (months to years) and capital-intensive, and during a supply crisis simply unavailable.
Disable CPU boost to maximize density. Run CPUs slower so more servers fit under the power budget, optimizing performance-per-watt. This is what Meta did before — it maximizes hosts per facility but leaves per-host throughput, and the stranded power headroom, on the table.
Naive fleet-wide boosting. Just raise all frequencies to use the headroom. Fast, but power scales superlinearly and the fleet is uneven, so it pushes panels past their breakers, triggers power capping, and causes the very outages it was meant to avoid.
Per-service manual tuning. Have each service team decide its own boost settings. Does not scale — it would require coordinating with thousands of teams, and services misjudge whether they benefit or can even consume the extra capacity.
The gap. No approach safely, automatically boosted a heterogeneous fleet to reclaim stranded power without risking capping, while selecting only the hosts that benefit. That is the deployment problem this paper solves.

How Meta boosts safely at fleet scale

Pick beneficiaries, validate cheaply, keep every panel safe, and consolidate the risk

The power-limited datacenter and its stranded headroom

Capacity is capped by power, not space, and building more takes years — but servers rarely run at max frequency, so provisioned power is underused. Boosting frequency could harvest that headroom for more throughput, in days, from hardware already owned.
↓ →
Pick the services that actually benefit new

Not all services gain from faster CPUs. An ML classifier predicts performance-per-watt improvement from micro-architectural counters (IPC, memory-bandwidth use, cache misses, stalls) at 95% accuracy, selecting the CPU-bound, high-IPC, low-memory-wait services (about 65% of the fleet) that benefit directly.
↓ →
Validate the gain with a cheap proxy new

Most services (about 80%) lack A/B-testing infrastructure, so Meta validates throughput with useful_MIPS — total CPU instructions minus overhead from context switches, page faults, and migrations — which correlates with real throughput at 85% accuracy, confirming a boost actually helped without per-service experiments.
↓ →
Keep every panel under its breaker with a risk model new

A power simulator estimates each panel and region's draw (assuming boosted services pull about 1.2x baseline power, with a plus-or-minus 5% tolerance) and boosts only within a tiny risk budget — at most 0.5% of regional capacity at capping risk — classifying residual risk as localized or regional, transient or persistent, with a matching mitigation for each.
↓ →
Withhold at-risk capacity to interruptible workloads new

Where a region's capping risk is transient, Meta holds the at-risk capacity back from high-priority services and hands it to OPPORTUNISTIC (interruptible) workloads, which can be throttled harmlessly if capping ever fires. These jobs used 85 to 97% of the withheld capacity, so little is wasted.
↓ →
Consolidate the risk by only ever raising frequency new

The central rule: transition only LOW to HIGH frequency, never the reverse (a drop would surprise services with about 10% capacity loss and require coordinating with thousands of owners). Pre-allocated recovery buffers absorb any capping, and rollout is paced to 5% of a service's hosts at a time — shrinking coordination from thousands of teams to under ten.

Meta's fleet-wide deployment over about a year during the COVID-19 supply crisis, across a heterogeneous fleet of multiple hardware generations and 100 services, read from the ACM proceedings version. An operational experience paper rather than a single new algorithm.

The idea

Harvest the resource you already paid for instead of buying more. The founding move is to notice that the datacenter’s power budget — the expensive, slow-to-grow constraint — is not fully used, so before building anything new you should extract more from what exists. DVFS boosting turns stranded power headroom into throughput on hardware already owned, adding capacity in days-to-weeks where procurement takes months-to-years. This reclaim-the-underutilized-resource discipline recurs across the site: the attention-overlap page fills idle GPU multiprocessors with complementary work, and the value-profiler runs its analysis on multiprocessors the application leaves idle — the same instinct to use the slack you have rather than provision more. The twist here is that the slack is electrical, and using it converts a capital-and-time problem (build a datacenter) into an operational-and-safety problem (boost without tripping a breaker), which is a far better problem to have when you need capacity now.

Operate close to a hard limit with a controlled risk budget and a safe fallback. The danger is that boosting pushes panels toward their breakers, and a trip is catastrophic, so the design never pretends it can predict power perfectly. Instead it accepts a tiny, explicit risk budget (at most 0.5% of regional capacity at capping risk), estimates power with a tolerant simulator, and backs every risk with a fallback — pre-allocated recovery buffers to absorb capping, and interruptible workloads placed on the at-risk capacity so a cap does no real harm. Running near a hard limit by bounding the risk and guarding it with a correct fallback is the same optimize-aggressively-but-keep-safety-in-the-fallback discipline the site’s microsecond-BFT page uses with its fast/slow path and the approximate-query page uses when a cheap path is backstopped by an exact one: the fast, aggressive behavior is allowed precisely because the consequences of being wrong are bounded and recoverable.

The hard part is deployment, not the idea — so consolidate the coordination surface. DVFS is decades old; the contribution is making it safe and scalable across a live, heterogeneous fleet, and the paper is candid that the obstacles were operational — services’ own monitors misreading the boost, auto-scalers failing to consume the headroom, hardware generations needing different settings, thousands of teams to coordinate. Its sharpest operational insight is to CONSOLIDATE the risk so that coordination becomes tractable: by only ever raising frequency and pre-allocating buffers, Meta reduced a thousands-of-teams problem to under ten infrastructure teams. Shrinking the coordination surface to make a system manageable is the same reduce-the-blast-surface instinct the site’s isolation pages apply when they shrink what must be trusted, and the insistence on learning from real production behavior — measuring in the live fleet, not a model — is the same measure-in-the-real-environment discipline the site’s contention and side-channel pages share. The lesson is that at scale, the engineering is in the rollout.

Turning stranded power into capacity

From stranded power to half a datacenter

Meta's DVFS boosting deployment. Numbers from the paper's fleet-wide rollout.

Step 1 of 5

The stranded resource

A datacenter is limited by power, not space, and building more takes years — but servers underrun their CPUs, so provisioned power sits partly idle. Boosting frequency could harvest it, but power scales superlinearly, and if a panel exceeds its breaker the datacenter caps (throttles) hosts, losing capacity instead of gaining it.

a median 20% more power buys about 10% more throughput, because power grows roughly cubically with voltage
capping is protective but looks to a service like a sudden, unplanned loss of hosts

Why it works

A datacenter’s capacity is capped by power, which is expensive and slow to grow, yet the provisioned power is underused → boosting CPU frequency harvests that stranded headroom → capacity is added in days-to-weeks from hardware already owned, not months-to-years of construction.
Power scales superlinearly and tripping a breaker causes capping, which loses capacity → a power-risk simulator boosts only within a 0.5% risk budget and backs the rest with buffers and interruptible workloads → the fleet operates close to its limit without ever endangering a panel.
Only some services benefit and most cannot A/B test → an ML classifier (95%) selects beneficiaries and useful_MIPS (85%) validates gains → power is spent only where it converts to throughput, without per-service experiments.
Lowering a service’s frequency later would cause a surprise capacity loss and force coordination with thousands of teams → only ever transitioning low-to-high consolidates the risk → coordination shrinks to under ten infrastructure teams, making fleet-wide deployment feasible.
Fast rollouts destabilize and slow ones confuse services’ auto-scalers → pacing to 5% of hosts with days to steady state → the rollout reached 99.8% coverage safely, adding 25% throughput for 23% power.

What it costs (trade-offs and scope)

It buys capacity with power, not efficiency. Boosting adds 25% throughput for 23% more power — only about a 1.08x performance-per-watt gain. The win is avoiding hardware capital expense and build time, not running more efficiently; on a pure energy basis it is close to power-neutral.
The benefit is workload-dependent, and services must harvest it. About 65% of services (CPU-bound) gain directly, but memory-bound and I/O-bound ones gain little to nothing, and network-bound services on older hardware gained under 1%. One service showed no gain for 120 days because it lacked auto-scaling to consume the extra headroom.
Safety is statistical, not absolute. The design accepts up to 0.5% of regional capacity at capping risk and relies on a power simulation with a plus-or-minus 5% tolerance, so a mis-estimate or a correlated load spike could still cause capping — the safety is a bounded, mitigated risk, not a guarantee.
Boosting confuses services’ own control loops. One service’s regression detector misread the 8% CPU-utilization difference as a software regression and blocked binary pushes, violating an SLA; and auto-scalers do not automatically consume the gain. Deployment requires coordinating with services’ monitoring and scaling, not just flipping a switch.
It is one company’s fleet under one crisis, and assumes headroom exists. The numbers and “half a datacenter” reflect Meta’s fleet, power prices, and hardware that ran below its thermal design power; a fleet already run at maximum frequency would have no headroom to reclaim.

The evaluation

Meta’s production deployment over about a year during the COVID-19 supply crisis, across a heterogeneous fleet of multiple hardware generations and 100 boosted services, with an ML candidate classifier, a useful_MIPS validation proxy, a power-risk simulator, and staged rollout, measuring throughput, power, capacity created, power-capping events, and server reliability.

Adding datacenter capacity

Property	Build a new datacenter	Naive fleet-wide boost	DVFS boosting (this deployment)
Lead time
Capacity source
Safety
Which hosts
Result

Building capacity takes years; boosting naively trips breakers and causes capping. Meta's deployment reclaims stranded power safely — ML-selecting beneficiaries, keeping every panel under its breaker within a 0.5% risk budget, and only ever raising frequency — adding 25% throughput per host for 23% power, a net 12.85 MW (half a datacenter) in weeks, with no reliability impact.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

Read this as a deployment-experience report, with its scope. The capacity is bought with POWER, not efficiency: 25% throughput for 23% power is only about a 1.08x performance-per-watt gain, so the value is avoided hardware capital expense and build time, not running leaner — on energy alone it is roughly power-neutral. The benefit is workload-dependent (about 65% of services, the CPU-bound ones, gain directly; memory- and I/O-bound gain little; some network-bound services on old hardware gained under 1%), and a service must have responsive auto-scaling to actually consume the headroom — one service showed no gain for 120 days for lack of it. Safety is a bounded, mitigated RISK, not a guarantee: the design accepts up to 0.5% of regional capacity at capping risk and leans on a power simulation with a plus-or-minus 5% tolerance, so a bad estimate or correlated spike could still cap. And boosting perturbs services’ own control loops (a regression detector misread it as a software bug and violated an SLA), so it takes real coordination, not a switch. The numbers are Meta’s fleet, prices, and below-TDP hardware. Within that scope the lessons are sharp and general: harvest the expensive resource you already provisioned before buying more; operate close to a hard limit by bounding the risk and backing it with a recoverable fallback; consolidate the coordination surface (only ever raise frequency) so a fleet-wide change becomes manageable; and expect the hard part to be the deployment reality — heterogeneity, monitoring, and control-loop interactions — not the underlying mechanism.

Background

Harvest stranded slack, bound the risk near a hard limit, and win on the deployment

This paper sits in the site’s datacenter and energy clusters and rhymes with several ideas covered elsewhere on this site. Its founding move is to harvest a resource already paid for rather than provision more: the datacenter’s power budget is underused, so boosting reclaims the stranded headroom — the same use-the-slack-you-have instinct the attention-overlap page applies when it fills idle GPU multiprocessors with complementary work and the GPU value-profiler applies when it runs its analysis on idle multiprocessors. Its safety design operates close to a hard limit by bounding the risk (at most 0.5% of regional capacity) and backing it with a recoverable fallback (buffers and interruptible workloads), the same optimize-aggressively-but-keep-safety-in-the-fallback discipline the microsecond-BFT page uses with its fast and slow paths and the approximate-query page uses when a cheap path is backstopped by an exact one. It selects beneficiaries and validates gains with cheap predictors — a 95%-accurate ML classifier and an 85%-accurate useful_MIPS proxy — rather than measuring every service directly, the same predict-and-proxy-instead- of-measure discipline the quantum-circuit-search, heterogeneity-aware- scheduling, and autotuning pages rely on. Its central operational insight, consolidating risk by only ever raising frequency so coordination shrinks from thousands of teams to under ten, is the same shrink-the-blast-and- coordination-surface move the memory-isolation pages make when they minimize what must be trusted. And its candor that boosting confuses services’ own monitors and auto-scalers is the same intervention-perturbs- the-observers, observer-effect caution the coherent-NIC and cheap- monitoring pages raise. Together they make this a case study in extracting capacity from a hard-limited system by treating the safe, scalable rollout as the real engineering problem.

If you remember only three things

1. A datacenter's capacity is limited by POWER, not space — the breakers and cooling cap how much compute runs — and building more takes months to years, yet the provisioned power is usually underused because servers rarely run CPUs at max frequency. DVFS boosting reclaims that stranded headroom by raising frequency for more throughput, adding capacity in days-to-weeks from hardware already owned. The danger is that power scales SUPERLINEARLY, so pushing a panel past its breaker triggers power capping (throttling) that loses capacity and causes outages — and the fleet is heterogeneous, so a blanket boost is unsafe.
2. Meta made it safe and scalable with several pieces: an ML classifier (95% accurate) picks the CPU-bound, high-IPC services that actually benefit; a cheap useful_MIPS proxy (85%) validates gains where A/B testing is missing; a power-risk simulator boosts only within a tiny budget (at most 0.5% of regional capacity at capping risk), mitigating the rest by rack moves, rebalancing, or handing at-risk capacity to interruptible workloads; and the central rule "optimize the baseline to consolidate risk" — only ever raise frequency, never lower it, and pre-allocate recovery buffers — shrinking coordination from thousands of service teams to under ten infrastructure teams.
3. Rolled out from 20% to 99.8% of hosts over a year during the COVID supply crisis, it added 25% throughput per host for 23% more power — a net 12.85 megawatts, half a datacenter's worth of capacity, in weeks instead of years, with no reliability impact (boosting stayed within thermal design power and did not correlate with failures). The transferable lessons: harvest the expensive resource you already provisioned before buying more; operate near a hard limit by bounding the risk and backing it with a recoverable fallback; use cheap predictors and proxies to select and validate at scale; consolidate the coordination surface so a fleet-wide change is manageable; and expect the deployment reality — heterogeneity, monitoring, control-loop interactions — to be the hard part.

Check yourself

Check your understanding

1. Meta added the equivalent of half a datacenter without building anything. Predict where that capacity came from and why it was available.
2. Boosting frequencies fleet-wide could trip electrical breakers and cause outages. Predict how Meta boosts aggressively yet safely.
3. Meta could not A/B-test most services to check whether boosting helped, yet it validated gains fleet-wide. Predict how.
4. Meta enforced a rule that a host's frequency may only be raised, never lowered. Predict why this one-directional constraint was so important operationally.

Sources

Expanding Datacenter Capacity with DVFS Boosting: A safe and scalable deployment experience (ACM DL, DOI 10.1145/3617232.3624853) — ASPLOS 2024 (full text read via the ACM PDF).
ACM Digital Library entry — published version and metadata.