HuffDuff: Stealing Pruned DNNs from Sparse Accelerators

A neural network running on a chip in a device you can hold is protected only by encryption of what crosses the memory bus — and encryption hides the values, not how many bytes there are; earlier work showed that byte counts alone reveal a dense model's architecture, and the field assumed compressed sparse models were safe because the counts no longer match the layer dimensions, so HuffDuff (University of British Columbia) shows the opposite by feeding the chip crafted images that make convolution edge effects visible in the compressed sizes, and by timing the compression engine itself, cutting the space of candidate architectures from ten to the ninety-sixth down to fewer than a hundred.

Why this problem is worth solving

The thing being stolen here is smaller than people expect, and more valuable than they expect.

What made pruned models look safe is precisely what makes them vulnerable.

What people did before, and where it broke

• Assume physical access is out of scope. Datacentre threat models focus on a co-tenant exploiting shared resources, since an attacker rarely gets to hold the machine. Where it broke. Edge devices are single-user, lack virtualisation, and sit in the field where anyone can buy one. The paper argues that attacks designed for shared datacentre hardware mostly do not apply, and the physical ones do.
• Encrypt the memory traffic. Treat the chip as trusted, encrypt everything crossing to external memory, and add enclave-style integrity guarantees. Where it broke. Confidentiality of contents is not confidentiality of metadata. Addresses, read-versus-write, and transfer sizes all remain visible, and the earlier attack showed that for a dense model those are sufficient.
• Recover dense architectures from transfer volumes. Prior work exploits the fact that a layer’s output is later read as another layer’s input, which reveals both the graph and each tensor’s footprint, then solves the resulting equations. On a dense ResNet-18 it yields eight candidates. Where it broke. It depends on transfer volume equalling tensor size, which compression destroys. Applied naively to a sparse accelerator it gives four times ten to the ninety-sixth solutions.
• Bound the sparsity and hope that is enough. If the equations become inequalities, add an upper bound on how sparse a layer could plausibly be. Where it broke. Sparsity varies wildly between layers — the paper reports one layer of a pruned network retaining 3,627 weights out of 2,359,296, which is 99.85% pruned with no accuracy loss — so any bound covering that is far too loose to constrain anything.
• Attack from the software side instead. Query a model repeatedly and infer hyperparameters from its decision boundary. Where it broke. These approaches have no access to hardware behaviour, and the paper argues they are correspondingly weaker than one that exploits how the accelerator actually executes.
• The gap. Everyone treated compression as an obstacle that scrambled the signal. Nobody asked what compression itself reveals — that a compressed size is a count of non-zeros, and a count of non-zeros is something an attacker holding the input can manipulate.

How HuffDuff narrows the space

The idea

The mechanism that broke the old attack is the one that enables the new one. This is the shape of the paper and it is worth stating plainly. Compression made transfer size depend on the data rather than the dimensions, which defeated an attack built on the dimensions. But data-dependence means input-dependence, and the attacker controls the input. So the very property that scrambled the static signal creates a dynamic one that is strictly more informative, because it can be interrogated rather than merely observed. The defensive lesson is uncomfortable: an optimisation adopted purely to save bandwidth turned a fixed leak into a queryable one. That an efficiency mechanism silently becomes a channel, and that hiding contents does nothing about timing and volume, is exactly the concern the memory timing side-channel defence work covered elsewhere on this site was built around.

The physics of the leak is a filter hanging off the edge of an image. A convolution slid across an image is nearly translation-invariant, but not at the borders: when part of the filter extends past the edge, that part contributes nothing, so the output has fewer non-zero values than it would have had in the middle. Ordinarily this is a nuisance that padding papers over. Here it is a measuring instrument. Place a probe impulse at the very first position and some of the filter is wasted; move it one step in and slightly less is; keep going and eventually none is — and the position at which the compressed transfer size stops changing tells you how far the filter extends. The complication is that a bias term makes everything non-zero and erases the distinction, which the attack solves by making the probe negative so the filter’s response is clamped away while the bias survives. Crafting inputs specifically to force a measurable difference, rather than passively observing what a victim happens to do, is the same active posture as the high-resolution control-flow attack work covered elsewhere on this site.

Timing the compressor reveals the size of what it read, not what it wrote. Edge effects are blind to channel count, so something else was needed, and the paper finds it in an architectural detail. Partial sums — the running accumulations before a layer finishes — are essentially never zero, since that would require every product to vanish or to cancel exactly, and they cannot be clamped early because a negative running total may still turn positive. So they sit uncompressed in on-chip memory. When the layer completes, an encoding unit reads them, clamps, quantises and compresses, and writes the compressed result out. The attacker sees only the compressed writes, but the duration between the first and last of them is governed by the reads, which are the bottleneck — the paper measures the uncompressed data as five to six times larger, with wider accumulators than activations. So the clock reveals a quantity the transfer size hides. Exploiting a boundary inside the chip that the designers never considered an information boundary is the same manoeuvre as the on-chip memory extraction work covered elsewhere on this site.

Walking one probe through the attack

This is the boundary effect, normally treated as an artefact to be managed with padding. The attacker cannot see the values because they are encrypted, but the compression elides zeros, so the transfer size is a count of the non-zeros — which is exactly the quantity that changed.

Why it works

• Compression converts a static leak into an interrogatable one. A compressed transfer size counts non-zeros, and non-zero counts depend on the input → the attacker controls the input → so instead of reading one fixed number per layer, the attacker can ask a different question with every probe.
• The boundary effect is a property of convolution itself. A filter overhanging the image edge contributes fewer non-zeros than one fully inside → so the position at which output counts stop changing localises the filter extent → and this holds for every padding mode and strategy, since all of them produce some boundary behaviour.
• The activation function separates what the bias merges. An additive bias makes every output non-zero and erases the count difference → a negative probe makes the filter response negative, which the rectifier clamps to zero, while bias-only positions stay positive → the discriminating structure the model needs for accuracy is what restores the signal.
• Random probing beats the unobservability problem statistically. Any single probe may fail to produce a visible difference at some layer → but failures are independent across random probes → so repeating until the inferred geometry converges recovers everything, which two thousand-odd trials achieved on both networks.
• The compressor is bounded by what it reads, not what it writes. Accumulators are wider than activations and are essentially never zero, making the uncompressed data five to six times larger than the compressed output → so the read side is the bottleneck and the duration tracks the hidden quantity → giving exactly the channel information the probing cannot reach.

What it costs (trade-offs and scope)

• None of it was run against real hardware. The probe responses come from instrumented model code, and the timing channel from a custom analytic simulator built around published accelerator parameters. The threat model assumes bus probing tools that exist, but no device was opened and no bus was tapped in this work.
• The models are small and the dataset is small. Two networks, both on a ten-class image dataset with tiny images, both pruned tenfold by one particular method. Whether the probe patterns survive the depth of a modern production network is not shown.
• The last step rests on an assumption, not a measurement. The timing channel yields only ratios of channel counts between layers, so an anchor is needed. The paper argues first layers are rarely pruned beyond 60% and uses that as a bound. The reasoning is plausible and it is an empirical prior rather than something the side channel established.
• The timing measurement is admittedly approximate. The attacker sees the first and last memory writes but not when the compressor began reading, so the interval is an estimate. The paper reports the inaccuracy as small enough not to need correction.
• A specific bottleneck must hold. The whole channel depends on the compressor being limited by reads rather than writes. The authors verify this across several memory generations and note that more on-chip bandwidth would eventually break it — while arguing the bottleneck would then move to the encoder rather than disappear.
• Several accelerator styles are immune. Designs with no external memory, designs that fuse several layers on-chip, and designs with structured rather than irregular sparsity all fall outside. Structured sparsity is immune to this attack because it is already vulnerable to the older one.
• Some layer types leave no trace. Transposed convolutions, used in segmentation networks and generative models, exhibit no boundary effect and are invisible to the prober. The prober also cannot distinguish among padding modes, so the implementation assumes the most common one.
• Defences are sketched, not evaluated. Two directions are proposed — leaving sensitive positions uncompressed, or randomly leaving some zeros uncompressed — and both are noted as either requiring dynamic hardware support or being defeatable by repeated trials. The paper says a serious defence study would be its own paper.
• The accuracy comparison uses a chosen baseline. Candidates are compared against a prior-generation model pruned to the same footprint, on the reasoning that an attacker would not bother stealing something worse. Reasonable, and a constructed comparison.

The evaluation

Check yourself

Sources

The paper is “HuffDuff: Stealing Pruned DNNs from Sparse Accelerators” by Dingqing Yang, Prashant J. Nair and Mieszko Lis, published at ASPLOS 2023 Volume 2, pages 385-399, DOI 10.1145/3575693.3575738. All authors are at the University of British Columbia.

This explainer is written from the full text, read through a proxy of the ACM Digital Library copy. Everything above comes from the paper’s own sections 2 through 9: the threat model requiring physical access to observe memory addresses, directions and volumes but not contents, with encrypted transfers and no on-chip visibility; the review of the prior dense attack based on read-after-write dependencies between layers, yielding eight candidates for a residual network; the demonstration that compression turns those equations into inequalities and produces four times ten to the ninety-sixth candidates, with the observation that one pruned layer retains 3,627 of 2,359,296 weights; the boundary effect derivation on a one-dimensional example and its use to bound filter extent from both edges; the negative-probe technique for defeating bias and normalisation terms via the rectifying activation; the propagation of probes to downstream layers, the notion of unobservable errors, and the symbolic convolution engine used to test geometry hypotheses; the finding that 2,048 random trials suffice for every layer of both networks in under ten minutes on one graphics card, and that probing is blind to channel counts; the partial-sum argument that accumulators are dense and wider than activations, making the encoder read-bound and its duration a proxy for uncompressed size, validated across several memory generations; the empirical first-layer sparsity bound below 60% used to anchor the ratios, giving 44 and 66 candidates; the accuracy and targeted adversarial transfer evaluation against prior-generation and randomly chosen pruned surrogates; and section 9, listing the accelerator styles and layer types outside scope and sketching two undeveloped defence directions.