Explainable-DSE: An Agile and Explainable Exploration of Efficient HW/SW Codesigns of Deep Learning Accelerators Using Bottleneck Analysis

Designing a deep-learning accelerator means picking one configuration from a space of about 10 to the 29th options under strict power and area limits, and today's search tools treat each candidate as an opaque score, so they blindly sample thousands of designs and still often fail to find even a workable one; this paper from Arizona State, UCLA, and ASU replaces the opaque score with a bottleneck model — a little tree that says which factor is dominating the cost and by how much — so the search can reason about why a design is slow and fix exactly that, reaching designs 6 times faster in 47 times fewer trials, in minutes instead of days.

The argument in one glance

Problem

Choosing an accelerator design is a needle in a haystack of 10 to the 29th

A deep-learning accelerator is defined by a hardware configuration — how many processing elements, how big the buffers, how wide the on-chip network, how much off-chip bandwidth — and by the software that maps a neural network onto it. Their joint space is astronomically large, on the order of 10 to the 29th combinations, and each candidate must meet hard limits on area and power while minimizing latency. Evaluating one candidate takes milliseconds to hours, so the design must be searched, not enumerated.
What people did

Search it with a black-box optimizer

The standard tools are black-box searches: grid and random search, or feedback-driven methods like simulated annealing, genetic algorithms, Bayesian optimization, and reinforcement learning. They evaluate a candidate, read off its single cost number, and use that to decide where to sample next.
Why it fell short

A single cost number cannot tell you why a design is slow

Seeing only the objective, these searches learn which past trials were better, but never why — which factor dominated the cost, or which of the many parameters to change and by how much. So they sample enormously: thousands of trials, often taking days to weeks, and under strict constraints they mostly land on infeasible or inefficient designs. In one study only around a tenth of a percent of explored solutions were feasible, and the best were still far from good.
This paper's idea

Replace the opaque score with a model that explains the cost

Explainable-DSE feeds the search a bottleneck model instead of a bare number: a small tree showing how the design parameters and intermediate factors — compute time, on-chip communication, off-chip memory access — build up to the total cost. Before each step the search reads this tree, finds the dominant bottleneck and the exact factor by which to shrink it, changes only the parameters that address it, and keeps the candidate feasible. Reasoning about why, it converges in tens of trials rather than thousands.

Why this problem is worth solving

The task is a search, and the size of what is being searched is what makes it hard.

Design space exploration, or DSE, is the process of automatically searching a large space of possible designs to find one that best meets a goal under constraints. For a deep-learning accelerator the space is the product of every hardware choice — the count of processing elements, the sizes of the on-chip buffers, the width and number of the on-chip network links, the off-chip bandwidth — with every way the software can schedule a neural network onto that hardware, which for a single layer alone can run to 10 to the 15th mappings. The two multiply into something on the order of 10 to the 29th, far too many to try, and each trial is expensive to evaluate. The whole difficulty is spending a tiny, fixed budget of trials wisely enough to land on a design that is both feasible, meaning it satisfies the area, power, and throughput limits, and efficient, meaning it minimizes latency. Doing that well is what decides whether an accelerator project ships a good chip or an also-ran.

The tools that do this searching share a blind spot that the paper names precisely.

A black-box optimizer treats the thing it is tuning as a sealed box: it hands in a candidate, reads out a single score, and uses only the history of scores to decide what to try next. Grid and random search ignore the score entirely; smarter methods like Bayesian optimization or reinforcement learning use it to guess where better designs might lie. What none of them can do is look inside the box and reason about why a particular design scored as it did — which cost factor dominated, which parameter is responsible, and how much to change it. With only a number to go on, the search can tell that one past design beat another but not the cause, so it cannot estimate the quality of the next candidate before spending a trial on it. Over an astronomically large space explored with a fraction of the needed trials, that blindness means most of the budget is burned on candidates that were never going to help.

What people did before, and where it broke

Grid or random search over the space. Sweep the parameters on a grid, or sample designs at random, for a fixed number of trials. Where it broke. These ignore the results entirely, so they cannot steer toward good regions; over a space of 10 to the 29th, a budget of thousands of samples is a rounding error, and almost none land on feasible, efficient designs.
Feedback-driven black-box optimizers. Simulated annealing, genetic algorithms, Bayesian optimization, and reinforcement learning use each candidate’s score to bias where they sample next. Where it broke. They know which past trials were better but not why, so they still need thousands of trials to converge and, under tight constraints, mostly explore infeasible or low-quality designs; some runs even got worse over their iterations.
Fix the software, search only the hardware. Many accelerator searches lock in one dataflow mapping and explore only hardware parameters. Where it broke. The chosen hardware is then oblivious to the mapping, so it can be incompatible with it or fail to exploit it, yielding designs that are infeasible or leave performance on the table across the varied layer shapes of a real network.
Bottleneck-driven search in other domains. In multimedia and FPGA high-level synthesis, prior work did reason about bottlenecks to guide design. Where it broke. Their search logic was fused to their specific domain and its cost model, so it could not be lifted out and reused for deep-learning accelerators; and each optimized only a single task at a time, with no way to reconcile the different bottlenecks of a workload made of many diverse layers.
The gap. No accelerator search combined the ability to reason about why a design is costly, a reusable interface separating that reasoning from the domain-specific model, and a way to reconcile the conflicting bottlenecks of a multi-layer workload under hard constraints. Blindness on one side, non-reusable one-task reasoning on the other.

How Explainable-DSE works

The search loop, with bottleneck reasoning inserted before each step

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Stop scoring designs and start explaining them. The move at the heart of the paper is to change what the search gets back from an evaluation. A conventional cost model returns one number — this design has this latency — which tells the optimizer nothing about the cause. A bottleneck model returns a small tree: the total latency at the root, and beneath it the factors that produced it, such as the time spent computing, the time moving data on-chip, and the time fetching data from off-chip memory, each in turn traced down to the specific hardware and software parameters that set it. Now the search can look at a slow design and see that, say, off-chip memory access is consuming almost all the time while compute sits idle — a diagnosis, not just a verdict. That is what the authors mean by explainability: at every step the search can say why this design costs what it does and what would improve it. Turning a bare symptom into a traced-back cause, so that the fix targets the actual culprit instead of guessing, is the same shift the trace-based root-cause work covered elsewhere on this site makes, replacing rules of thumb about what looks anomalous with a method that follows the trouble to its source.

Attack the factor that dominates, by the amount it dominates by, and watch the next one surface. Once the cost is broken into factors, the search knows not just which one is the bottleneck but by how much. If computation takes four times as long as data movement, the analyzer concludes the compute resources should grow by about that factor, and it changes the number of processing elements accordingly — a large, purposeful step, not a timid nudge to a neighboring value. The moment that bottleneck is relieved, re-evaluating reveals the next one, perhaps now the off-chip transfers, and the search pivots to widen the buffers or the bandwidth. This is optimization as a sequence of targeted strikes against whatever is currently binding, each informed by the magnitude of the imbalance, so the objective falls at nearly every step — measured here at about 1.3 times per feasible acquisition, where blind methods sometimes got worse over their runs. Letting the dominant resource bottleneck point directly at what to change, rather than sweeping parameters hoping to stumble on the limiter, is the same diagnosis-guided discipline the GPU-memory-optimization work covered elsewhere on this site applies, pinpointing the object actually responsible for the pressure so the fix lands where it matters.

Encode the domain knowledge once, behind an interface a generic engine can reuse. A bottleneck model is domain-specific — the factors and their mitigations for a deep-learning accelerator are not those of a database or a video codec — and earlier bottleneck-driven searches paid for this by fusing their search logic to their one domain, so none could be reused. Explainable-DSE separates the two with an API: the expert supplies three things — the tree of cost factors, a map from each factor to the parameters that affect it, and subroutines that compute new parameter values — while the search engine that navigates the tree, aggregates across layers, and honors constraints stays entirely domain-independent. Plug in a different model and the same engine explores a different domain. And where the expert leaves a gap — no subroutine for some parameter — the engine falls back gracefully to sampling neighbors like an ordinary black-box search, so partial knowledge still helps. Decoupling the reusable optimization machinery from the domain-specific description it consumes, so one engine serves many targets through a clean interface, is the same architecture the portable compiler-autotuning work covered elsewhere on this site is built on, keeping the search framework general while the specifics of each target are supplied as pluggable input.

Following one search through Explainable-DSE

Tuning processing elements and buffer size for a ResNet layer

A small slice of the real search: starting from a modest design and letting the bottleneck model steer each move. The same reasoning runs across all the network's layers at once.

Step 1 of 5

Evaluate the starting design and read its bottleneck model

The design begins with 64 processing elements and a 64-kilobyte buffer. Rather than just a latency, the bottleneck model reports that computation time is about 4.1 times the time spent moving data — the design is badly compute-bound.

The analyzer computes each factor's share of total latency and names compute as the dominant one, with a scaling factor of roughly 4.1 needed to bring it into balance with the others.

Why it works

A model of the cost lets the search change the right thing. A single latency number hides the cause, so a black-box optimizer must guess which parameter matters → but a bottleneck tree names the dominant factor and the parameter behind it → so nearly every acquisition reduces the objective, about 1.3 times each, where blind methods sometimes drifted worse.
Scaling by the size of the imbalance skips the local neighborhood. Local search nudges parameters to adjacent values and crawls → but knowing a factor overshoots by, say, four times lets the search jump a parameter far in one move → so it converges in roughly 54 trials against thousands, about 47 times fewer.
Changing one critical parameter per candidate keeps the search tiny. Sweeping all parameters at every step is what makes black-box search expensive → but restricting each candidate to a single bottleneck-relevant parameter yields a handful of trials per round → so the whole exploration finishes in minutes, 53 to 103 times faster than black-box runs that take days.
Watching the constraint budget keeps candidates feasible. Blind search under tight limits wanders into infeasible regions and wastes trials → but preferring candidates that use the least constraint budget steers toward the feasible region → so 87 percent of explored designs met area and power here, against a fraction of a percent for black-box search under throughput limits.
A shared model couples hardware and software tightly. Fixing the mapping and tuning hardware alone leaves the two mismatched → but a bottleneck model containing both hardware and mapping parameters lets each new hardware design target the bottlenecks of the freshly optimized mapping, which then re-optimizes → so joint codesigns come out about 4.24 times better than using one fixed mapping.

What it costs (trade-offs and scope)

Everything rests on analytical models, not real silicon. The bottlenecks, costs, and mappings all come from analytical performance models rather than measured hardware or RTL, so the quality of the explanations is only as good as those models; the paper itself notes they do not accurately capture features of real edge accelerators like mixed-precision or sparsity-exploiting hardware.
The explainability is hand-authored domain knowledge. Someone must build the bottleneck tree, the parameter map, and the mitigation subroutines for the domain — the method does not discover them. It is only as good as the expert model behind it, and automatic construction of these models is left to future work.
The comparison rewards knowledge the baselines lack. The 6-times and 47-times gains are against black-box optimizers that have no domain knowledge at all, so the honest reading is that encoded expertise beats blind search, not that this is a universally better optimizer; a black-box method given the same insight might close much of the gap.
The search is greedy and can miss global optima. Attacking one bottleneck at a time, even with large steps and multi-bottleneck aggregation, is a greedy strategy that can settle into a locally good region; the paper discusses workarounds but does not guarantee global optimality.
The aggregation rules are tuned choices. Taking the minimum predicted value, considering only the top five layers, and the specific contribution threshold are heuristic settings that happen to work here, not derived optima, and different choices could change the outcome.
The scope is a fixed template and a single objective. The search explores only within the spatial-architecture template its cost models assume, not arbitrary architectures, and it minimizes latency under constraints rather than exploring a full multi-objective trade-off surface.
The baselines are configured by the authors. Competing methods, including a generalized reinforcement-learning tool, were set up and tuned by the authors on the same cost models; a fair effort, but a self-run comparison rather than an independent one.
It is demonstrated at edge-inference scale. The evaluation covers eleven vision and language inference models on edge accelerators with 16-bit data, not large-scale training accelerators or datacenter deployments, so generality beyond that setting is argued rather than shown.

The evaluation

Four ways to search an accelerator design space

Property	Grid / random search	Black-box feedback (Bayesian, RL)	Prior bottleneck-driven search
What guides the next candidate
Reasons about why a design is costly	No
Handles multi-layer workloads and constraints
Reusable across domains	Generic but blind	Generic but blind	No — search baked into domain
Result	Thousands of trials, mostly wasted		Single-domain, single-task

Comparison on eleven edge vision and language inference models. Explainable-DSE explored about 54 designs versus around 2500 for black-box methods, cutting latency 6 times on average (up to 49 times versus Bayesian optimization) and search time 53 to 103 times.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

The core idea is genuine and clean: give the search a model that explains where the cost comes from, and it can attack the actual bottleneck with a right-sized step instead of blindly sampling — converging in about 54 trials and minutes rather than thousands of trials and days, with 6-times lower latency and far higher feasibility. Read the gains with their framing, though. They are measured against black-box optimizers that carry no domain knowledge at all, so the result really shows that encoded expertise beats blind search; a black-box method handed the same insight would narrow the gap. That expertise is entirely hand-authored — the bottleneck tree and its mitigation subroutines must be built by an expert, and the whole method runs on analytical performance models rather than real hardware, models the authors acknowledge miss features of actual edge accelerators. The search is greedy, attacking one bottleneck at a time with tuned aggregation rules, so it can miss globally better designs; and it works within a fixed architecture template, one objective, and edge-inference scale, with author-configured baselines. What is durable is the reframing: an optimizer that can see why a candidate is bad, exposed through a reusable interface with a graceful black-box fallback, is a better fit for an expensive, constrained, astronomically large search than one that sees only a score.

Background

Why accelerator codesign became a search problem at all

It helps to see why a whole subfield now pours effort into searching accelerator designs automatically. As deep learning has specialized, the hardware running it has too, and a modern accelerator is no longer a fixed design but a template with dozens of knobs — how much compute, how much on-chip memory, how wide the interconnect — each interacting with how the network is scheduled onto it. The number of combinations exploded past anything a human can reason through by hand, and the stakes are high because a poorly matched design can leave most of the silicon idle: a well-known study found a production accelerator running some models at fifteen percent utilization, with one kind of layer taking a twentieth of the arithmetic but two-thirds of the time. That mismatch is precisely a bottleneck, and closing it is why codesign — tuning the hardware and the software together rather than in isolation — became essential, and why doing it automatically, across the joint space, became a defining problem. The same pressure drives the full-stack accelerator-search work covered elsewhere on this site, which searches the datapath, the scheduling, and the compiler together because optimizing any one alone leaves the others mismatched. Explainable-DSE is a bet that the way through such a space is not more brute-force sampling but giving the search the ability to understand what it is looking at.

If you remember only three things

1. Designing a deep-learning accelerator means searching a joint hardware-software space of about 10 to the 29th options under hard area, power, and throughput limits, and the standard black-box optimizers see only a single cost number per candidate — so they can tell which past trials were better but never why, forcing thousands of trials that mostly land on infeasible or inefficient designs over days to weeks.
2. Explainable-DSE replaces the opaque score with a bottleneck model — a tree showing how compute time, on-chip communication, and off-chip memory access build up to the total cost — so before each step the search finds the dominant bottleneck, computes exactly how much to shrink it, changes only the responsible parameter by that amount, reconciles the differing bottlenecks of each network layer by taking the least aggressive value over the most critical layers, and keeps candidates inside the constraint budget.
3. The result is 6-times lower latency in 47-times fewer trials — about 54 designs and minutes instead of thousands and days — with 87 percent of designs feasible on area and power and 4.24-times better hardware-software codesigns, though the gains are against knowledge-free baselines, the bottleneck models are hand-authored and run on analytical rather than real hardware, and the greedy, single-objective search on a fixed template can miss globally better designs.

Check yourself

Check your understanding

1. Why do black-box optimizers struggle to search an accelerator design space well?
2. What is a bottleneck model, and how does it change the search?
3. How does knowing the size of a bottleneck help the search converge so much faster?
4. Which limitation most affects how the results should be read?

Sources

The paper is “Explainable-DSE: An Agile and Explainable Exploration of Efficient HW/SW Codesigns of Deep Learning Accelerators Using Bottleneck Analysis” by Shail Dave, Tony Nowatzki, and Aviral Shrivastava, published at ASPLOS 2023, pages 87-107, DOI 10.1145/3623278.3624772. The authors are at Arizona State University and the University of California, Los Angeles.

This explainer is written from the full 21-page text. Everything above comes from the paper’s own sections: the introduction framing the vast codesign space and the lack of explainability in prior DSE; the limitations section quantifying the inefficiency, infeasibility (around 0.1 percent feasible under throughput constraints), and slowness of black-box search; the motivation and challenges section introducing bottleneck models and the three shortcomings to overcome; the framework section detailing the workflow, the bottleneck analyzer and its computation of factor contributions and scaling, the three-part API for specifying bottleneck models, the aggregation of predictions across multi-layer workloads by taking the minimum value over the top-K critical layers, the bottlenecks-guided acquisition of single-parameter candidates with large step sizes, the constraints-budget-aware update, the latency bottleneck model for DNN accelerators, and the tightly coupled hardware/mapping codesign built on an extended dMazeRunner mapper; and the experimental methodology and results reporting the 6-times lower latency in 47-times fewer iterations, the 53-to-103-times search-time reduction (minutes versus days to weeks), the 1.30-to-1.32-times per-attempt objective reduction, the 4.24-times codesign improvement, and the feasibility and 100-iteration dynamic-exploration comparisons against grid, random, simulated annealing, genetic, Bayesian, HyperMapper 2.0, and reinforcement-learning baselines.