Has Machine Learning for Systems Reached an Inflection Point?

Fields like computer vision and language each hit an "AlexNet moment" where machine learning went from one tool among many to the dominant way the field works — but computer systems (schedulers, allocators, compilers) never did; this keynote (Google DeepMind — Martin Maas) argues the ingredients for that moment in systems are now here, and lays out three catalysts that could trigger it — reusable design recipes for learning-based systems, infrastructure that makes ML easy to deploy inside systems, and the new capabilities of generative AI.

The argument in one glance

Problem

Systems never had the "AlexNet moment" that transformed other fields

Over the last decade, one field after another has been transformed by machine learning. Computer vision had its turning point with the AlexNet paper in 2012; natural language processing followed; software engineering and programming tools are being reshaped now by large language models. Each of these fields went through an inflection point — a moment when machine learning stopped being one of many approaches and became the predominant approach, the default way the field works. Computer systems are a striking exception. The core decision-making inside systems — how a scheduler places virtual machines, how an allocator lays out memory, how a compiler optimizes code, how a cache decides what to keep — is still built mostly from hand-crafted heuristics and algorithms, with machine learning appearing only occasionally and at the margins. There is clear promise: learning-based systems keep showing they can do better than hand-tuned rules. Yet the field has not tipped. This keynote asks the direct question: has machine learning for systems reached its inflection point, and if not, what would be the catalyst that finally drives widespread adoption? The answer, it argues, may not be a single event but a combination of forces now coming together.
What people did

Apply machine learning to individual systems problems, one at a time

The way the field has proceeded so far is problem by problem: a researcher takes one systems decision — prefetching, branch prediction, query optimization, cluster scheduling, memory allocation, chip placement — and builds a bespoke machine-learning solution for it. Each such project gathers its own data, picks its own model, designs its own way to feed predictions into the system, and handles its own deployment concerns. Many of these one-off efforts genuinely work and beat the previous heuristics. The field is a large and growing collection of these individual successes.
Why it fell short

One-off efforts do not compound: no shared abstractions, hard deployment, false barriers

The problem is that these successes do not build on each other. Machine learning for systems refers to a huge range of different things — the keynote organizes them along two axes, the application area (databases, compilers, scheduling, memory, networking, and more) and what the ML actually does (anomaly detection, forecasting, extrapolation, discovery, optimization) — and this creates a roughly quadratic space of area-times-use combinations, each seemingly requiring its own fresh datasets, libraries, and interfaces. Without shared abstractions, every project is ad-hoc, so the field cannot accumulate the way distributed systems did once consensus became a clean, reusable abstraction. Deployment is a second obstacle: real systems are not built to host a model, so integrating one raises hard questions about latency, safety and correctness, rollout, and interpretability that each team must solve from scratch. A third obstacle is perception: a belief that ML for systems needs enormous data, enormous compute, and deep ML expertise, which discourages much of the systems and academic community from even trying. As long as the work stays fragmented, awkward to deploy, and seen as out of reach, the field cannot tip toward ML as its predominant approach.
This paper's idea

Three catalysts that together could trigger the inflection point

The keynote argues the ingredients for an inflection point are now in place, and frames them as three catalysts. The first is a critical mass of generalizable design recipes: reusable patterns that apply across many domains, so work stops being ad-hoc. The keynote presents a taxonomy to organize the field and then five concrete recipes drawn from real systems — for example, split a system policy into a learned predictor plus an algorithm that tolerates prediction errors (used in a learned memory allocator that predicts object lifetimes and packs them, correcting for mistakes); predict distributions rather than single values when data is noisy; move the model into the workload when workloads drift; combine ML with an exact solver when correctness must be guaranteed (used in an on-chip memory allocator shipping in production); and learn a heuristic by imitation learning when a problem can be solved offline. The second catalyst is systems and infrastructure that make ML easy to deploy — addressing latency, safety, rollout, and interpretability as reusable engineering (a production VM allocator runs a gradient-boosted-tree model inside the scheduler at a median of 9 microseconds), and dispelling the myths that ML for systems needs huge data, huge compute, or ML expertise. The third catalyst is the rapidly growing capability of generative AI, which enables approaches that were previously impossible: not just improving a system, but accelerating its evolution — for instance an LLM-driven code optimizer that mined performance fixes across Google's monorepo and, in one year, submitted tens of thousands of lines of optimizations saving the equivalent of about two million CPU cores, with under half a percent rolled back. The conclusion: all the ingredients are there; whether and when the inflection point manifests depends on the community.

Why this problem is worth solving

Systems make countless performance-critical decisions with hand-tuned heuristics, and turning those into learned policies could improve efficiency across the entire computing stack.

Every layer of computing rests on components that constantly make decisions — which page to evict, which core to schedule on, how to lay out memory, which query plan to run, what to prefetch. These decisions are usually governed by heuristics that engineers tuned by hand, and they are everywhere, so even modest improvements compound across enormous scale. The fields that have already had their machine-learning inflection point show what is possible: once ML became the predominant approach in vision or language, progress accelerated dramatically because the whole community could build on a shared foundation. Systems have the same latent opportunity — learning-based components repeatedly beat hand-tuned ones — but the gains have stayed local and slow because the field has not tipped. If it did, the benefit would not be a single better allocator or scheduler but a shift in how systems are built, unlocking improvements across databases, compilers, operating systems, and datacenters at once. That is why the question of the inflection point is not academic: it is about whether a large, foundational part of computing gets to ride the same wave that transformed everything else.

What has held systems back is not that ML does not work there, but that the field lacks the connective tissue that let other fields tip.

Systems differ from vision or language in ways that have slowed the transition. The problems are enormously varied, so a breakthrough in one corner does not obviously transfer to another, and there is no single benchmark like ImageNet whose conquest announces a turning point. Systems also impose hard constraints that pure ML fields do not: predictions often sit on a latency-critical path, mistakes can violate correctness, and components must be deployed and rolled out safely inside production infrastructure. And the community has carried a perception that this work requires resources and expertise most systems researchers lack. None of these is a reason ML cannot dominate systems — they are reasons the field has not yet developed the shared abstractions, deployment infrastructure, and cultural confidence that let other fields tip. The keynote’s argument is precisely that these missing pieces are now appearing: reusable recipes are reaching critical mass, deployment infrastructure is maturing, and generative AI is adding capabilities that did not exist before. The transition, it notes, resembles ones the systems community has navigated before — from single-node to distributed, from single-core to multi-core — each of which also demanded new abstractions before it became routine.

What people did before, and where it broke

Build a bespoke ML solution for each systems problem. Solve prefetching, scheduling, allocation, and the rest one at a time. Where it broke. Each project is ad-hoc, with its own data, model, and interfaces, so successes do not compound across the roughly quadratic space of application-times-use-case combinations.
Treat every deployment as a one-off engineering effort. Solve latency, safety, rollout, and interpretability afresh for each learned component. Where it broke. Systems are not designed to host models, so integration is repeatedly hard and there is no shared infrastructure to build on.
Assume ML for systems needs huge data, huge compute, and ML expertise. Leave the work to well-resourced ML specialists. Where it broke. The keynote argues these are myths — system deployments produce ample data, cheap models and in-context learning often suffice, and many problems are really systems problems where ML is an opaque building block.
Use ML only to improve an existing system component. Apply learning to make one policy better. Where it broke. It misses the larger opportunity that generative AI now opens: using AI to accelerate the very loop by which systems are optimized and evolved.
The gap. The field lacked shared recipes, deployment infrastructure, and the new capabilities of GenAI all at once — so despite many individual wins, ML never became the predominant way systems are built.

How the argument works

From fragmented experiments to a field that could tip

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Turn learned prediction under uncertainty into a reusable pattern, not a one-off. The keynote’s central recipe is to split a system policy into a model that predicts some property and an algorithm robust to the prediction being wrong — and to predict distributions, not point values, when the data is noisy. Managing a system by learning to predict a property while explicitly accounting for the uncertainty of that prediction is the same approach the work on uncertainty-aware resource management for multi-stage services (Aquatope) covered elsewhere on this site takes, treating the forecast’s error as a first-class part of the design. The keynote elevates this from a single system into a general recipe meant to apply across memory allocation, storage placement, and VM scheduling alike.

Keep hard guarantees by pairing ML with an exact solver. For problems where correctness or precision cannot be compromised, the keynote’s recipe is to let ML guide search while an exact solver enforces validity — used in an on-chip memory allocator that ships in production. Combining a learned or heuristic component with a constraint solver so the solver guarantees the result while the learning speeds it up is the same principle the work on a solver-aided compiler for packet-processing pipelines (CAT) covered elsewhere on this site is built on, using a solver as the backstop for correctness. The keynote generalizes it into a recipe for any systems problem that is NP-hard yet must never produce an invalid answer.

Use generative AI to accelerate the loop by which systems are optimized. The keynote’s most forward-looking claim is that AI’s value is not only to improve a system but to speed up its evolution — for instance an LLM that retrieves known performance anti-patterns and generates code optimizations at fleet scale. Applying retrieval plus large language models to automatically find and apply code optimizations that a human loop would be too slow or too narrow to cover is the same idea the work on retrieval-augmented loop-transformation optimization (LoopRAG) covered elsewhere on this site develops, letting an LLM optimize code guided by retrieved knowledge. The keynote frames this as a distinct new capability — the “capability delta” of GenAI — that did not exist for systems before.

Following a recipe through a real system

The predictor-plus-error-tolerant-algorithm recipe in a learned memory allocator

A trace, drawn from the keynote's examples, of how one reusable recipe turns a systems problem into a learning-based solution.

Step 1 of 5

Identify a decision that needs information you lack

A memory allocator wants to pack objects to avoid fragmenting huge pages, but doing so well needs each object's lifetime — how long it will live — which the allocator does not know at allocation time.

The missing information is exactly what a model could predict.

Why it works

Recipes let successes compound instead of staying local. A quadratic space of area-and-use-case problems seems to need bespoke solutions each → but a handful of recipes apply across many of them → so new learning-based systems reuse proven patterns rather than starting from scratch.
Predicting distributions handles the noise systems data actually has. Real system signals are noisy and not perfectly predictable → but forecasting a distribution rather than a single value → so the algorithm can reason about uncertainty instead of trusting a brittle point estimate.
A solver backstop makes ML safe where correctness is required. Learning alone can produce invalid or unsafe results → but letting ML guide search while an exact solver enforces the constraints → so the system gets ML’s speed with a hard guarantee of correctness.
Reusable deployment infrastructure removes the recurring integration tax. Latency, safety, rollout, and interpretability are hard for every team → but solving them once as shared infrastructure and cheap in-scheduler models → so integrating a model stops being a from-scratch effort each time.
GenAI adds capability, not just efficiency, to the optimization loop. Human-driven optimization is slow and covers only high-value code → but an LLM mining and generating fixes across a whole codebase → so systems get optimized at a velocity and coverage a human loop could never reach.

What it costs (open questions and caveats)

This is an argument, not a settled result. The keynote claims the ingredients for an inflection point are present, but explicitly leaves open whether and when it will actually manifest — that, it says, depends on the community, so the thesis is a well-supported prediction rather than a demonstrated fact.
The five recipes are only a small subset. The keynote presents a handful of design recipes as illustrations and notes many more exist and remain to be discovered; a truly reusable methodology for ML for systems is still being built, not finished.
Deployment challenges are real and recurring. Even with infrastructure, learning-based systems must meet latency budgets, guarantee safety, integrate into rollout processes, and remain interpretable; these are genuine engineering constraints that each setting revisits, not fully solved problems.
The strongest evidence comes from one organization’s scale. Several headline results (the production VM allocator, the fleet-wide LLM code optimizer) come from Google’s planet-scale infrastructure; the keynote argues academia can contribute meaningfully and that data and compute needs are often modest, but the most dramatic numbers reflect an industrial setting.
Keeping humans in the loop remains necessary. The GenAI code optimizer still routes changes through automated testing and human code review; the win is speeding up and scaling the optimization loop, not removing human judgment, so full autonomy is not claimed.

The framework, at a glance

Three ways to bring machine learning into systems

Property	One-off ML per problem	Recipe- and infrastructure-based	GenAI-accelerated
Reusability across domains
Deployment effort
What it mainly enables
Handling uncertainty and correctness
Evidence in the keynote

The keynote's framing of how machine learning enters systems. Moving from bespoke, one-off ML per problem to reusable design recipes plus shared deployment infrastructure, and then adding the new capabilities of generative AI, is what the keynote argues could trigger an inflection point for ML in systems. The concrete systems and figures cited (a production VM allocator at a median of 9 microseconds, an LLM code optimizer saving about 2 million CPU cores in a year with under half a percent rolled back) come from the underlying papers; details should be read from those and the keynote.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer is based on the keynote’s slide deck (the author’s copy hosted by the ASPLOS 2025 conference), which accompanies the invited-talk paper; it is a vision and framework talk, not a systems paper with a single artifact and evaluation. The described content is Martin Maas’s: the argument that machine learning for systems has not yet reached the inflection point that transformed vision, language, and code, and that three catalysts — a critical mass of generalizable design recipes, systems and infrastructure that facilitate deploying ML, and the growing capabilities of generative AI — could now trigger it. The taxonomy (application area by how ML is used: anomaly detection, forecasting, extrapolation, discovery, optimization) and the five recipes (predictor plus error-tolerant algorithm; predict distributions for noisy data; move the model into the workload under drift; combine ML with a solver for correctness; learn heuristics by imitation learning) are the keynote’s framing, illustrated with the presenter’s own and collaborators’ works (LLAMA learned memory allocation, learning on distributed storage traces, TelaMalloc on-chip allocation shipping on Pixel 6 and TPUv4, LAVA VM allocation, ECO code optimization). The concrete figures — a median 9-microsecond in-scheduler prediction latency, and the ECO optimizer submitting over 25,000 lines of change in a year, saving the equivalent of about 2 million normalized CPU cores with under 0.5 percent rolled back — are reported in those underlying papers. The step-by-step walkthrough above is an illustration of one recipe. Exact claims, the full taxonomy, and all results should be read from the keynote and its cited papers.

Background

Why fields tip to machine learning, and why systems has been slow to follow

It helps to see this keynote within a broad pattern: the transitions where a field tips to machine learning as its predominant method tend to require more than a single clever model — they require shared abstractions, benchmarks, tooling, and a community confident enough to build on them. Vision had ImageNet and a decade of infrastructure around it; language had shared architectures and datasets; each tipped when the connective tissue was in place, not merely when the first good result appeared. Systems has produced many strong individual results but lacked that connective tissue, which is why it has lagged. The keynote’s three catalysts are essentially a plan for growing that tissue — recipes as shared abstractions, deployment infrastructure as tooling, and generative AI as a new source of capability — plus the observation that the systems community has repeatedly built exactly this kind of foundation before, in the moves from single-node to distributed and single-core to multi-core computing. Interpretability and deployability recur throughout, because a learned systems policy has to be trusted and integrated, not just accurate. This concern for making learned decision-making practical, interpretable, and deployable inside real systems is the same terrain the work on a non-intrusive, interpretable learned scheduler for deep learning (Lucid) covered elsewhere on this site occupies, treating a model’s fit into an operational system as central rather than incidental. The keynote’s contribution is to argue that, with recipes, infrastructure, and GenAI now converging, the systems field finally has the ingredients for its own inflection point — and that whether it happens is a choice the community gets to make.

If you remember only three things

1. Field after field — computer vision (AlexNet, 2012), natural language processing, software engineering — has passed an inflection point where machine learning went from one approach among many to the predominant way the field works. Computer systems are a notable exception: their core decisions (scheduling, memory allocation, compilation, caching) are still mostly hand-tuned heuristics, with ML at the margins, even though learning-based systems keep showing they can do better. The keynote diagnoses why: ML-for-systems work has been fragmented (a roughly quadratic space of application-times-use-case problems, each solved ad-hoc), deployment is hard (systems are not built to host models, raising latency, safety, rollout, and interpretability issues each time), and false perceptions (that it needs huge data, huge compute, and ML expertise) keep much of the community away.
2. The keynote argues the ingredients for an inflection point are now present, framed as three catalysts. First, a critical mass of generalizable design recipes — organized by a taxonomy (application area by how ML is used: anomaly detection, forecasting, extrapolation, discovery, optimization) — such as splitting a policy into a predictor plus an error-tolerant algorithm, predicting distributions for noisy data, moving the model into the workload under drift, combining ML with an exact solver for correctness, and learning heuristics by imitation. Second, infrastructure that makes ML deployable (a production VM allocator runs a model inside the scheduler at a median of 9 microseconds) while dispelling the data, compute, and expertise myths. Third, the new capabilities of generative AI, which can accelerate the very loop by which systems evolve.
3. The recipes and infrastructure are illustrated with shipping systems (a learned memory allocator, an on-chip allocator on Pixel 6 and TPUv4, a Google VM allocator), and the GenAI catalyst with an LLM-driven code optimizer that mined performance fixes across Google's monorepo and in one year submitted over 25,000 lines of change, saving the equivalent of about 2 million CPU cores with under 0.5 percent rolled back — beating the human optimization loop on quality, velocity, and coverage. The keynote frames the transition as resembling past shifts (single-node to distributed, single-core to multi-core) that also needed new abstractions. Its conclusion: all the ingredients are there; whether and when ML for systems reaches its inflection point depends on the community. It is a vision argument, not a settled result.

Check yourself

Check your understanding

1. What does the keynote mean by an "inflection point" or "AlexNet moment" for a field, and why does it say systems has not had one?
2. According to the keynote, why have the many individual successes in ML for systems not added up to an inflection point?
3. The keynote presents reusable "recipes." What is the predictor-plus-error-tolerant-algorithm recipe, illustrated by a learned memory allocator?
4. What is the keynote's third catalyst — the role of generative AI — and what evidence does it give?

Sources

The paper is “Has Machine Learning for Systems Reached an Inflection Point?” by Martin Maas (Google DeepMind), an invited keynote at ASPLOS 2025 (the ASPLOS and EuroSys plenary session, April 2025), DOI 10.1145/3676642.3729206.

This explainer is written from the keynote’s slide deck, hosted by the ASPLOS 2025 conference. It is a vision and framework talk rather than a systems paper with a single artifact. Its argument is that machine learning for systems has not yet reached the inflection point that transformed computer vision, natural language processing, and software engineering, and that three catalysts could now trigger it: a critical mass of generalizable design recipes (organized by a taxonomy of application area by how ML is used, and illustrated by recipes such as predictor-plus-error-tolerant-algorithm, predicting distributions, moving the model into the workload, combining ML with a solver, and imitation learning); systems and infrastructure that make ML deployable (addressing latency, safety, rollout, and interpretability, and dispelling myths about data, compute, and expertise requirements); and the growing capabilities of generative AI (which can accelerate the system optimization loop). The concrete systems and figures cited — a learned memory allocator, an on-chip allocator shipping on Pixel 6 and TPUv4, a production VM allocator at a median 9-microsecond in-scheduler latency, and the ECO code optimizer saving about 2 million CPU cores in a year with under 0.5 percent rolled back — come from the underlying papers referenced in the talk. Claims and figures are the keynote’s and its cited works’; details should be confirmed there.