The ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning

To run a giant model across many accelerators, a compiler must decide, for every one of its thousands of operations, how to split that operation's tensors across the devices — and the choices interact, because a tensor split one way must be reshuffled if the next operation wants it another way; this paper (Google — Michael D. Moffitt, Pratik Fegade) is a retrospective on the ASPLOS/EuroSys 2025 contest that distilled this auto-sharding problem into a clean combinatorial optimization over a graph, released 25 real industrial benchmarks, and offered cash prizes to crowdsource better model-partitioning solvers.

The argument in one glance

Problem

Splitting a giant model across accelerators is a huge, interacting choice problem

Modern deep-learning models are too large to run on one accelerator, so their computation must be distributed across many devices — slices or pods of accelerators wired together. A key determinant of how fast that runs is how the computation is sharded: for each operation in the model, you must decide how to split its tensors across the devices. This is intra-operator parallelism — sometimes called tensor or operator parallelism — and it includes familiar forms like data parallelism (split the batch) and other ways of partitioning an operation across the mesh of devices. The catch is that these choices are not independent. If one operation shards its output tensor along one dimension, and the next operation that consumes that tensor wants it sharded along a different dimension, the data must be reshuffled between them, which costs communication time. So the right sharding for each operation depends on the sharding of its neighbors, and a model regularly contains hundreds or thousands of operations. Choosing a good strategy for every one of them, accounting for how they interact and staying within each device's memory, is a large combinatorial optimization problem. Doing it by hand is infeasible, so a compiler must solve it automatically — and how well it does directly sets the performance of training and serving.
What people did

Let each production compiler solve auto-sharding with its own in-house solver

Production ML compilers already tackle this. Systems such as GShard, Alpa, and related frameworks automatically choose sharding strategies for a model's operations, each using its own internal solver — integer programs, heuristics, dynamic programming, or search — tied to its particular framework, cost model, and hardware assumptions. The auto-sharding problem is thus solved, but privately: buried inside each compiler, in a form specific to that system, with its own encoding of costs and constraints.
Why it fell short

With no shared formulation or benchmark, progress on the algorithm is hard to compare

Because the sharding problem lives inside each compiler in its own idiosyncratic form, the underlying algorithmic question — given the operations, their possible strategies, the interaction costs, and the memory limits, find the assignment that minimizes total cost — is hard to study on its own. There is no clean, self-contained statement of the problem that a researcher outside a given compiler team can pick up, no common set of realistic benchmarks to test a new solver against, and no standard way to measure whether one partitioning algorithm beats another. That makes progress fragmented: advances are entangled with the compilers they live in, the state of the art is unclear, and talented people in adjacent fields — combinatorial optimization, operations research, competitive programming — cannot easily contribute, because the problem is not exposed to them in a form they can attack. What was missing was a crisp, standardized formulation of the auto-sharding problem, backed by real benchmarks, that would let anyone build and fairly compare a solver.
This paper's idea

Distill it into a clean graph optimization, release real benchmarks, and hold a contest

The ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism distilled the auto-sharding problem into a clean, self-contained combinatorial optimization and invited the world to solve it. In the contest's formulation, the model becomes a graph: each node is an operation with a set of candidate strategies, and each strategy carries a node cost. Certain pairs of nodes are joined by edges, and each combination of the two endpoints' strategies carries an edge cost — this captures the reshuffling cost incurred when connected operations shard their shared tensor incompatibly. A solution selects exactly one strategy per node, and the objective is to minimize the total cost (node costs plus edge costs). On top of that, each chosen strategy consumes some memory over a fixed time interval, and the total memory in use at any instant must not exceed the benchmark's usage limit — a peak-memory constraint. The organizers released a suite of 25 industrial benchmarks drawn from real models (5 shared with participants, 20 withheld for evaluation), ranging from hundreds to tens of thousands of nodes, and asked teams to submit a solver binary that, given a problem and a time limit, outputs the chosen strategy per node. Solvers were scored by how close their total cost came to the best submission on each withheld benchmark, and cash prizes (up to 25,000 US dollars for first place) went to top teams committing to open-source their code. This paper is the organizers' retrospective: the problem formulation, the challenges and opportunities, the new benchmark suite, and the quality of the submissions received.

Why this problem is worth solving

Auto-sharding is one of the highest-leverage decisions in the entire large-model stack, because the same model can run dramatically faster or slower depending only on how its operations are partitioned.

When a model is spread across many accelerators, the arithmetic to be done is fixed, but the time it takes is not — it depends heavily on how much data has to move between devices and how well memory is used. A good sharding keeps most operations working on their local shards with little communication, so the expensive interconnect is rarely exposed on the critical path; a poor sharding forces constant reshuffling of tensors between devices, and that communication can dominate, leaving the accelerators idle while data crosses the network. Because a large model has thousands of operations, and each one’s best sharding depends on its neighbors’, the space of possible whole-model shardings is astronomically large, and the difference between a mediocre and an excellent choice can be a large multiple in training or serving speed. That is why the problem is worth solving well: it converts directly into how many accelerator-hours a training run costs and how fast a serving system responds, on hardware that is scarce and expensive. And because it must be solved for every new model and hardware configuration, a better general-purpose partitioning algorithm pays off across the whole fleet, repeatedly. The contest’s note that lower solution cost roughly translates to higher accelerator performance — reduced step times — makes this connection concrete.

The deeper contribution of a contest like this is methodological: it turns a capability locked inside proprietary compilers into a shared, well-defined problem that a whole community can advance.

Hard systems problems often make slow progress not because they are unsolvable but because they are not stated in a way that lets many people work on them and compare results. The auto-sharding problem had exactly this shape: every major ML compiler solved a version of it, but each in its own private encoding, so there was no way to ask cleanly whether a new algorithm was actually better, and no way for experts outside those teams to try. By abstracting the problem to its mathematical core — select one strategy per node in a graph to minimize node-plus-edge costs subject to a peak-memory limit — the contest made it a self-contained combinatorial optimization that anyone can attack with any technique, whether integer programming, local search, or clever heuristics. By pairing that with a suite of real industrial benchmarks and an objective, normalized scoring rule, it created a level field where solvers can be measured against each other and against the best known. This is the same move that has repeatedly accelerated fields: distill the essential problem, publish shared benchmarks, and let open competition surface the best ideas. The retrospective’s value is in documenting that formulation and what the competition revealed, so the improved solvers — and the benchmark itself — become a durable resource rather than a one-off event.

What people did before, and where it broke

In-house auto-sharding solvers (GShard, Alpa, and similar). Each compiler chooses sharding strategies with its own internal integer program, heuristic, or search. Where it broke. The solver is entangled with one framework’s cost model and hardware assumptions, so the algorithm is hard to isolate, reuse, or compare across systems.
Framework-specific problem encodings. State the sharding problem in terms internal to a given compiler. Where it broke. There is no common, self-contained statement outsiders can pick up, so the algorithmic core is inaccessible to the broader optimization community.
No shared realistic benchmarks. Evaluate a partitioning algorithm on whatever models a team has on hand. Where it broke. Without a common benchmark suite, results are not comparable and the true state of the art is unclear.
No standard scoring. Judge sharding quality by end-to-end speed in one system. Where it broke. End-to-end numbers mix the solver with everything else in the compiler, so they cannot cleanly rank partitioning algorithms against one another.
The gap. There was no clean, standardized formulation of intra-operator parallelism, backed by real benchmarks and objective scoring, that would let anyone build, test, and fairly compare a model-partitioning solver — which is exactly what the contest provided.

How the contest problem is structured

From a model's operations to a scored partitioning solution

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Partition a model’s operations across devices to minimize communication. The contest’s subject is choosing how to shard each operation of a model over a device mesh, the same auto-parallelization task real ML compilers perform. Automatically deciding how to split each operation and tensor of a model across many devices, so that the layout minimizes exposed communication, is the same SPMD-partitioning task the work on composing partitioning strategies for machine learning (PartIR) covered elsewhere on this site is built on, assigning shardings across a device mesh. The contest abstracts exactly this task into a portable optimization problem so anyone can build a solver for it.

Cast a compiler decision as a formal constraint problem for a solver. The contest reframes auto-sharding as a crisp combinatorial optimization — select one labeled strategy per node to minimize node-plus-edge cost — that a general solver can attack. Turning a messy compiler decision into a clean, formal optimization or constraint problem that a solver can reason about, rather than a bespoke heuristic, is the same solver-aided approach the work on a solver-aided compiler for packet-processing pipelines (CAT) covered elsewhere on this site takes, handing the hard combinatorial choice to a solver. The contest’s node-and-edge-cost formulation is precisely such a solver-friendly encoding of the sharding problem.

Solve resource allocation under a peak-usage constraint as an optimization. Beyond minimizing cost, the contest imposes a memory-usage-over-time limit, making it a constrained allocation problem where choices compete for a bounded resource across a schedule. Allocating resources to satisfy a peak-usage limit over time while optimizing an objective, cast as a formal combinatorial problem, is the same constrained-allocation-as-optimization approach the work on efficient on-chip memory allocation for production machine learning (TelaMalloc) covered elsewhere on this site takes, packing demands under a memory ceiling. The contest’s usage-interval-and-limit constraint gives its partitioning problem the same peak-memory character.

Following a solver through the contest

From a benchmark problem file to a scored partitioning

A walkthrough of the contest's evaluation flow, drawn from the official problem statement and the paper's abstract. The retrospective paper's full text was not available for this explainer.

Step 1 of 5

Read the problem graph

A solver is given a benchmark as a graph: nodes are operations, each with candidate strategies and node costs; edges connect interacting operations with a cost for each pair of endpoint strategies; and each strategy has a memory usage over a time interval, plus an overall usage limit.

Benchmarks range from hundreds to tens of thousands of nodes and edges.

Why it works

A graph formulation exposes the algorithmic core. The real problem is buried in each compiler’s internals → but stating it as nodes with strategy costs and edges with pairwise costs → so the essential optimization is portable and anyone can attack it.
Edge costs capture the interaction that makes it hard. Choosing each operation’s sharding in isolation misses the reshuffling penalty → but charging an edge cost for incompatible neighbor strategies → so the formulation faithfully models the communication that dominates performance.
A memory limit keeps solutions realistic. Minimizing cost alone could pick shardings that overflow device memory → but a peak-usage constraint over time intervals → so only assignments that actually fit the hardware are valid.
Real industrial benchmarks make results meaningful. Toy graphs would not reflect production difficulty → but a suite of 25 real models spanning tasks and modalities → so solvers are tested on the structure and scale that matter.
Normalized scoring fairly ranks different solvers. Raw costs vary wildly across benchmarks → but scoring each team relative to the best submission → so approaches as different as integer programming and local search compare on equal footing.

What it costs (trade-offs and scope)

The formulation abstracts away compiler detail. Reducing sharding to node and edge costs plus a memory limit is a clean model, but a real compiler’s cost model, hardware topology, and additional constraints are richer; the contest’s scores are directionally, not exactly, proportional to end-to-end speedup.
It covers intra-operator, not inter-operator, parallelism. The strategies combine data and operator parallelism (SPMD partitioning) but exclude pipelining, which assigns different operations to different devices — so this is one important slice of the full parallelization problem.
Benchmarks are partly withheld. Twenty of the 25 benchmarks are held back for fair evaluation, so participants tune against only five; the full set is released after the contest, but during it teams optimize with limited visibility.
A retrospective reports, it does not itself solve. This paper documents the contest — formulation, benchmarks, and submissions — rather than presenting a new algorithm; the winning solvers themselves are described in the participants’ own open-source releases.
This explainer is built from the contest’s public materials, not the paper text. The retrospective paper’s full text was not reachable; the problem formulation, benchmark details, scoring, and winners here come from the organizers’ official contest statement and repository (by the same authors) plus the paper’s abstract, so the paper’s specific narrative analysis of the submissions is not reproduced.

The comparison

How the auto-sharding problem was approached

Property	In-house compiler solvers	Ad hoc evaluation	The contest formulation
Where the problem lives
Who can build a solver
Benchmarks
Comparability of solvers
Outcome

How intra-operator parallelism was approached before and during the contest. Production compilers solve auto-sharding with private, framework-tied solvers, and evaluations were ad hoc, so the algorithmic core was hard to compare or contribute to. The ASPLOS/EuroSys 2025 contest distilled it into a portable graph optimization — pick one strategy per node to minimize node-plus-edge cost under a peak-memory limit — released 25 real industrial benchmarks, scored solvers by normalized cost, and awarded prizes (up to 25,000 US dollars) to top teams who open-sourced their code. Details should be read in the paper and the official contest materials.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer covers a contest retrospective paper, and it is assembled from the paper’s published abstract and TLDR together with the contest’s official problem statement and repositories (github.com/asplos-contest/2025 and github.com/google/iopddl), which were authored by the same organizers, Michael D. Moffitt and Pratik Fegade. The retrospective paper’s own full text was not reachable at the time of writing (it is behind the ACM Digital Library with no open preprint), so this page is marked a draft. The problem formulation reported here is the contest’s: the model is a graph where each node (operation) has candidate strategies with node costs, edges carry a cost for each pair of endpoint strategies (capturing resharding/communication), a solution picks one strategy per node to minimize total node-plus-edge cost, and each strategy’s memory usage over a time interval must keep total usage under a benchmark-specific limit at all times. The scope is intra-operator parallelism combining data and operator (SPMD) partitioning, excluding inter-operator pipelining. The benchmark suite is 25 industrial benchmarks (5 released, 20 withheld), ranging from hundreds to tens of thousands of nodes and edges, with 1-to-5-minute time limits; scoring normalizes each team’s cost against the best submission, and prizes up to 25,000 US dollars went to top teams committing to open-source. What the retrospective paper specifically adds beyond these public materials — its narrative on key challenges and opportunities and its detailed analysis of the submissions received — is not reproduced here and should be read in the paper. Background on distributed deep learning, sharding, and combinatorial optimization is standard context, and the walkthrough describes the contest’s evaluation flow, not a solver’s internal algorithm.

Background

Why hard systems problems benefit from being turned into shared benchmarks

It helps to see this contest within the broader history of how fields make progress on hard, practically-important problems. Time and again, a capability that lives inside a few organizations advances slowly until someone distills its essential problem into a clean, shared statement with public benchmarks and an objective way to keep score — at which point a much larger community, often including experts from adjacent disciplines, can pile in and rapidly push the state of the art. Auto-parallelization of large models is a natural candidate: it is economically enormous, genuinely hard as a combinatorial optimization, and until now was solved privately inside each ML compiler. Turning it into a portable graph problem invites the operations-research, integer-programming, and competitive-programming communities to contribute solvers, and the real industrial benchmarks ensure those solvers are tested on the structure that actually matters. This effort to advance distributed model execution by making the partitioning problem a first-class, shareable object is the same terrain the work on efficient spatial-temporal tensor partitioning for large transformers (PrimePar) covered elsewhere on this site occupies, improving how large models are split across devices. The contest’s distinctive move is not a new solver but a new commons: a crisp formulation, a benchmark suite, and an open competition, so that better model-partitioning algorithms — and the shared benchmark itself — outlast the event and become a foundation others build on.

If you remember only three things

1. Running a large deep-learning model across many accelerators requires deciding, for each of its hundreds or thousands of operations, how to shard that operation's tensors over the devices — intra-operator (tensor) parallelism. The choices interact: if one operation shards a tensor one way and the next operation wants it another way, the data must be reshuffled between them at a communication cost, so each operation's best sharding depends on its neighbors'. Choosing well for the whole model, within each device's memory, is a large combinatorial optimization that compilers must solve automatically and that directly sets training and serving performance. But every major ML compiler solved this privately, in its own framework-specific encoding, so the algorithmic core was hard to isolate, compare, or contribute to, and there was no common benchmark or scoring.
2. The ASPLOS 2025 / EuroSys 2025 contest distilled the problem into a clean, portable combinatorial optimization: the model is a graph where each node (operation) has candidate strategies with node costs, edges carry a cost for each pair of neighboring strategies (capturing resharding), a solution selects one strategy per node to minimize total node-plus-edge cost, and each strategy's memory usage over a time interval must keep total usage under a benchmark-specific limit at every instant. The organizers released 25 real industrial benchmarks (5 shared, 20 withheld), spanning hundreds to tens of thousands of nodes across training and serving, language and vision, and asked teams to submit a solver binary that outputs one strategy per node within a time limit. Solvers were scored by normalizing each team's cost against the best submission on each benchmark, with cash prizes (up to 25,000 US dollars) for top teams who open-sourced their code.
3. The contest's real contribution is methodological: it turned a capability locked inside proprietary compilers into a shared, well-defined problem with public benchmarks and objective scoring, so a whole community — including combinatorial-optimization and competitive-programming experts — could build and fairly compare model-partitioning solvers. This paper is the organizers' retrospective, documenting the formulation, the challenges and opportunities, the benchmark suite, and the quality of submissions, so the improved solvers and the benchmark become durable resources. Scope and caveats: it covers intra-operator (SPMD) parallelism but not inter-operator pipelining, its cost model is a clean abstraction of real compiler behavior (lower cost roughly, not exactly, means faster step times), and this explainer draws on the contest's official public materials plus the paper's abstract because the retrospective's full text was not reachable.

Check yourself

Check your understanding

1. What makes intra-operator parallelism (auto-sharding) a hard combinatorial problem?
2. How does the contest formulate the auto-sharding problem?
3. Why hold a contest with shared benchmarks rather than just improving one compiler's solver?
4. What is the scope of the contest problem, and how should this page be read?

Sources

The paper is “The ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning” by Michael D. Moffitt and Pratik Fegade (Google), a contest retrospective published at ASPLOS 2025, DOI 10.1145/3676642.3736399 (proceedings pages 5-17). The official contest materials are at github.com/asplos-contest/2025 (problem statement) and github.com/google/iopddl (benchmarks and utilities).

This explainer is assembled from the paper’s published abstract and TLDR together with the contest’s official problem statement and repositories, which were authored by the same organizers; the retrospective paper’s own full text was not reachable (behind the ACM Digital Library with no open preprint), so this page is marked a draft. The contest formulates intra-operator parallelism as a combinatorial optimization over a graph: nodes (operations) have candidate strategies with node costs, edges carry a cost for each pair of endpoint strategies (capturing resharding), a solution selects one strategy per node to minimize total node-plus-edge cost, and each strategy’s memory usage over a time interval must keep total usage under a benchmark-specific limit. The scope is data-plus-operator (SPMD) partitioning, excluding inter-operator pipelining. The benchmark suite has 25 industrial benchmarks (5 released, 20 withheld), from hundreds to tens of thousands of nodes and edges, with 1-to-5-minute time limits; scoring normalizes each team’s cost against the best submission, and prizes up to 25,000 US dollars went to top teams committing to open-source their solvers. Background on distributed deep learning, sharding, and combinatorial optimization is standard context. The paper’s specific analysis of challenges, opportunities, and submissions should be read in the full paper.