Hardwired-Neuron Language Processing Units as General-Purpose Cognitive Substrates

Yang Liu, Yi Chen, Yongwei Zhao, Yifan Hao, Zifu Zheng, Weihao Kong and 21 others

Running a large language model burns most of its energy just fetching billions of weights from memory, so this paper (Chinese Academy of Sciences and Cambricon — Yang Liu, Yi Chen, Qi Guo, Yunji Chen, and many colleagues) proposes baking the weights physically into the chip's wiring, turning every multiply into a cheap fixed-constant operation with no memory fetch at all; the obstacle is that a naive version would need over six billion dollars of fabrication masks per model, which their Metal-Embedding trick — hiding the weights in the metal wires instead of the transistors — cuts by 112 times, making a hardwired LLM chip economically real.

The argument in one glance

Problem

LLM inference burns most of its energy just moving weights around

A large language model has tens or hundreds of billions of weight parameters, and generating each token requires multiplying the input against all of them. On a GPU, those weights live in memory, so the machine spends enormous energy fetching billions of numbers from memory to the compute units for every single token. This weight movement — not the arithmetic itself — is the dominant cost of LLM inference, in both energy and time, and as models grow it only gets worse. Even specialized approaches that bring memory closer to compute reduce the traffic but cannot eliminate the fundamental need to load the weights. At the same time, something has changed about what LLMs are: a single pre-trained model is now general enough to handle a huge range of tasks — reasoning, planning, coding, conversation — that used to need separate systems. Language has become a general-purpose "cognitive substrate," and the generality that hardware and software used to provide has migrated into the model itself. That shift matters for hardware design, because it means one dominant model is now valuable enough, and stable enough, to be worth building a chip around. The question this paper asks is: if you were willing to specialize completely to one model, how efficient could inference get — and what would stop you?
What people did

Hardwire the weights directly into the silicon

The most extreme form of specialization is to physically hardwire the model's weights into the chip. If the weights are literally part of the circuit rather than data in memory, then there is zero parameter-fetching overhead — nothing to load — and the architecture matches the model perfectly. Better still, because each weight is now a fixed constant, every multiplier can be simplified from a general multiply (which must handle any two numbers) into a much cheaper multiply-by-constant circuit, which takes several times less logic. Hardwiring neural networks into hardware is an old idea, dating to the 1980s, and in principle it delivers orders-of-magnitude gains in computational efficiency. The obvious way to do it is to bake each weight into a 2D grid of silicon device cells — transistors — one region of the chip per weight.
Why it fell short

Baking weights into transistors makes fabrication cost astronomical

The obvious approach collides with how chips are manufactured. A chip is built by patterning many layers with photomasks — expensive templates, one set per unique design, that at a 5-nanometer process cost about 30 million dollars for a full set of roughly 70 mask layers. Normally that cost is amortized across the hundreds of thousands of identical chips stamped from the same masks, so per chip it is tiny. But if you bake a specific model's weights into the transistors, then every chip's pattern is different everywhere — the weights are unique to that model — so the mask set cannot be reused across a production run, and its cost lands on essentially one design. For a 120-billion-parameter model, hardwiring the weights into device cells would require photomask sets valued at over six billion dollars, versus about 780 dollars per chip for a normal amortized design. That is economically impossible. Worse, LLMs need at least annual updates to stay competitive, and each new set of weights would demand a whole new multi-billion-dollar mask set. So the very thing that makes hardwiring efficient — encoding the specific weights into the chip — is what makes it unaffordable. What was missing was a way to hardwire the weights without making the whole, expensive mask set unique to each model.
This paper's idea

Hide the weights in the wiring, so most of the masks stay reusable

HNLPU (Hardwired-Neurons Language Processing Unit) is the first hardwired LPU design, and its key enabler is a methodology called Metal-Embedding. The insight is where you put the weights. A chip is built in two stages: the front-end-of-line (FEOL) forms the transistors (about 30 mask layers, using the finest and most expensive lithography, including all the extreme-ultraviolet masks), and the back-end-of-line (BEOL) forms the stack of metal wires that connect them (about 40 more layers, with the upper layers using coarser, cheaper masks). Instead of embedding weights in the 2D grid of transistors, Metal-Embedding embeds them into the 3D topology of the metal wires — encoding each weight in how the wires are routed. A companion Hardwired-Neuron architecture rearranges the usual multiply-accumulate arithmetic into an accumulate-multiply-accumulate form that lets the weights live up in the metal. This brings two payoffs. First, packing weights into wire topology is about 15 times denser than into device cells. Second, and decisively, the weights now only affect a few upper metal layers, so 60 of the 70 mask layers — including every one of the expensive transistor and extreme-ultraviolet masks — become homogeneous: identical across all chips and all models, and therefore reusable. Only a handful of cheap metal masks are customized per model. This cuts photomask cost by 112 times, and makes an annual weight update a cheap re-spin of just those few layers rather than a whole new mask set. That brings the one-time engineering cost into a viable range. Implementing a 120-billion-parameter model in 4-bit precision, HNLPU reports 249,960 tokens per second (5,555 times a GPU, 85 times a wafer-scale engine), 36 tokens per joule (1,047 times a GPU), a one-time engineering cost of about 59 to 124 million dollars at 5 nanometers, and — under an annual weight-update assumption — 41.7 to 80.4 times better cost-effectiveness and 357 times less carbon than an equivalent cluster of H100 GPUs.

Why this problem is worth solving

LLM inference is becoming a dominant, energy-hungry workload, and specializing hardware to it could cut its cost and carbon by orders of magnitude — if the economics can be made to work.

The energy consumption of serving large language models is growing into one of the defining costs of computing, and most of that energy goes not to useful arithmetic but to shuttling billions of weights between memory and compute. If a single model is going to be run at enormous scale for a long time — which is now the case, since one general-purpose LLM serves countless tasks — then the payoff from specializing hardware perfectly to that model is huge: eliminate the weight movement, simplify the arithmetic, and the efficiency gains are measured in thousands of times, not percentages. The paper’s own numbers illustrate the stakes: over a thousand times the energy efficiency of a GPU, and a claimed 357 times reduction in carbon footprint versus an equivalent GPU cluster. When a workload is this large and this stable, even an extreme, inflexible design becomes worth considering, because the efficiency it buys is applied across an immense volume of inference. The obstacle has never been whether hardwiring is efficient — it obviously is — but whether it can be made affordable to build. That is the real problem worth solving.

The whole feasibility hinges on the economics of chip fabrication, where one number — the reusability of the masks — decides everything.

Chip manufacturing has a peculiar cost structure: the masks used to pattern each layer are enormously expensive to make, but that cost is normally spread across every chip produced from them, so it disappears per unit. This only works because the chips are identical. The instant you make each chip’s pattern unique — which is exactly what hardwiring a specific model’s weights does — you lose the amortization, and the mask cost, undivided, becomes the dominant term, ballooning to billions. So the feasibility of a hardwired LPU is not really about transistors or speed; it is about whether you can keep the expensive masks reusable while still encoding model-specific information somewhere. Metal-Embedding is the answer precisely because it isolates all the model-specific customization into a few cheap layers and keeps the costly ones — the transistor masks and the extreme-ultraviolet masks — identical for every model. That single move, keeping 60 of 70 layers homogeneous, is what converts an impossible six-billion-dollar bill into a viable one, and it is also what makes updating the model cheap. The entire design is organized around this economic lever, which is why understanding the mask structure is the key to understanding the paper.

What people did before, and where it broke

Run the LLM on a GPU. Store weights in memory and fetch them for every token. Where it broke. Weight movement dominates energy and time, and grows with model size, so inference is enormously power-hungry.
Bring memory closer to compute (processing-in-memory, wafer-scale). Reduce the distance weights travel. Where it broke. It lessens the traffic but cannot eliminate the fundamental need to load the weights, so the memory cost remains.
Hardwire the weights into a 2D grid of transistors. Encode each weight in silicon device cells for zero fetching and constant-multiply circuits. Where it broke. Every chip’s pattern becomes unique, so the photomask set cannot be amortized — over six billion dollars for one large model.
Re-fabricate for each model update. Make a fresh full mask set whenever the weights change. Where it broke. LLMs need at least annual updates, and a new multi-billion-dollar mask set per update is unthinkable.
The gap. No design hardwired the weights for efficiency while keeping the expensive, model-independent masks reusable, so hardwiring stayed a theoretical ideal rather than an affordable chip.

How HNLPU works

Hardwiring an LLM efficiently by embedding weights in the wiring

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Put the computation where the weights are, so they never have to move. HNLPU’s deepest premise is that the crippling cost of LLM inference is moving weights, so the answer is to fuse storage and compute completely — the weights become the circuit. Attacking the weight-movement bottleneck by bringing computation into the memory that holds the weights, rather than streaming weights to a separate compute unit, is the same direction the work on processing-in-memory for batched transformer generation (AttAcc) covered elsewhere on this site pushes, collapsing the distance between weights and arithmetic. HNLPU takes it to the limit: there is no distance at all, because the weights are hardwired into the fabric and every multiply is a fixed-constant operation.

Rebuild the arithmetic circuits themselves for efficiency, not just the schedule around them. HNLPU does not merely rearrange data flow; it changes what the multiplier is, turning general multiplies into constant multiplies and reorganizing multiply-accumulate into an accumulate-multiply-accumulate form that lets weights live in the wires. Redesigning the low-level arithmetic circuitry to slash the energy of the core operation, rather than treating the arithmetic unit as fixed, is the same approach the work on energy-efficient convolutions with temporal arithmetic covered elsewhere on this site takes, reworking how the numbers are actually computed to save power. HNLPU’s constant-arithmetic circuits are where a large part of its orders-of-magnitude efficiency comes from.

Judge the design by its full lifetime cost of building and running, not just its speed. HNLPU’s entire argument is economic and environmental: it only makes sense once you account for the one-time engineering cost, the update cadence, and the carbon of both fabrication and operation. Assessing a processor by a first-order model of its manufacturing and operational cost and carbon, rather than performance alone, is the same lens the work on a first-order carbon model to assess processor sustainability (FOCAL) covered elsewhere on this site applies, weighing what a chip costs to make against what it saves to run. HNLPU lives or dies by exactly this accounting, which is why Metal-Embedding’s effect on the one-time mask cost is the heart of the paper.

Following a weight through HNLPU

From a model weight to a hardwired, low-cost computation

A trace, drawn from the paper, of how a weight becomes part of the chip and participates in inference.

Step 1 of 5

A weight is assigned to metal wiring, not a transistor

Rather than baking each weight into a region of silicon transistors, Metal-Embedding encodes the weight in the 3D routing topology of the metal interconnect layers, using the Hardwired-Neuron architecture that places weights in the metal above the transistors.

Wire topology stores weights about 15 times denser than device cells.

Why it works

Hardwiring removes the dominant cost of inference. Fetching billions of weights per token dominates energy → but making the weights part of the circuit means nothing is fetched → so the biggest energy cost of LLM inference simply disappears.
Constant weights make the arithmetic far cheaper. A general multiplier must handle any two operands → but a fixed weight lets each multiplier become a multiply-by-constant with several times less logic → so the core operation itself gets much more efficient.
Putting weights in the wires keeps the costly masks reusable. Baking weights into transistors makes every mask unique and un-amortizable → but embedding them in a few upper metal layers → so 60 of 70 masks, including all extreme-ultraviolet ones, stay identical and shared across models.
Reusable masks convert an impossible bill into a viable one. Unique full mask sets cost over six billion dollars per model → but isolating customization to a few cheap metal layers → so photomask cost drops 112 times and the one-time engineering cost becomes affordable.
Cheap layer re-spins make updates sustainable. Annual model updates would each need a new full mask set → but only the few customized metal masks change per update → so keeping the hardwired chip current is a cheap re-spin, not a rebuild.

What it costs (trade-offs and scope)

The chip is locked to one model. Hardwiring means the design serves exactly one set of weights; any change beyond the anticipated weight update, or a different architecture, requires new customized masks, so it trades all flexibility for efficiency and is only worthwhile for a dominant, stable model run at huge scale.
There is still a large up-front engineering cost. Even reduced 112 times, the one-time engineering cost is tens to over a hundred million dollars at 5 nanometers, and the die area is very large (over 13,000 square millimeters), so the approach only pays off across an enormous volume of inference.
The economics depend on the update-cadence assumption. The headline cost-effectiveness and carbon advantages are stated under an annual weight-update assumption; more frequent updates, or a shorter useful model lifetime, would erode the amortization that makes it favorable.
Metal-Embedding constrains the arithmetic and layout. Encoding weights in wire topology requires the specific Hardwired-Neuron accumulate-multiply-accumulate reorganization and imposes real limits on how weights map to metal, so the approach is a co-design of arithmetic and fabrication rather than a drop-in.
The results are from modeling and estimation, not a fabricated chip. The reported throughput, efficiency, cost, and carbon are analytical and design estimates at a 5-nanometer node for a specific 4-bit model; real fabrication would surface effects the estimates cannot fully capture, so the numbers should be read as a well-argued projection.

The evaluation

Serving a large LLM: GPU cluster versus hardwired approaches

Property	GPU cluster (H100)	Naive hardwiring (transistors)	HNLPU (Metal-Embedding)
Weight handling
Photomask cost per model
Model update
Energy efficiency
Cost-effectiveness and carbon

A comparison of ways to serve a large LLM. By hardwiring weights into the metal interconnect rather than the transistors, HNLPU keeps 60 of 70 photomask layers homogeneous and reusable, cutting photomask cost 112 times and making a hardwired LLM chip economically viable. Implementing a 120-billion-parameter model in 4-bit, the paper reports 249,960 tokens per second, 36 tokens per joule (1,047 times a GPU), a one-time engineering cost of about 59 to 124 million dollars at 5 nanometers, and 41.7 to 80.4 times better cost-effectiveness with 357 times less carbon than an H100 cluster, under an annual weight-update assumption. Figures are analytical estimates; exact numbers should be read from the paper.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This explainer is based on the paper’s full text (the authors’ arXiv copy, 2508.16151, of the ASPLOS 2026 paper). The described system is the paper’s: HNLPU is a Hardwired-Neurons Language Processing Unit that physically hardwires an LLM’s weight parameters into the computational fabric for orders-of-magnitude efficiency through extreme specialization, with zero parameter-fetching overhead and constant-arithmetic (multiply-by-constant) circuits. Its enabling methodology, Metal-Embedding, embeds weights into the 3D topology of metal wires rather than a 2D grid of silicon device cells, via a Hardwired-Neuron architecture that rearranges multiply-accumulate into accumulate-multiply-accumulate; this yields about a 15 times density increase and keeps 60 of 70 photomask layers homogeneous across chips (including all transistor front-end and extreme-ultraviolet masks), reducing photomask cost by 112 times (the paper cites minus 86.5 percent for initial tapeout and minus 92.3 percent for weight-update re-spins) and bringing the non-recurring engineering (NRE) cost into a viable range. Implementing a 120-billion-parameter model in 4-bit precision, the paper reports 249,960 tokens per second (5,555 times a GPU, 85 times a wafer-scale engine), 36 tokens per joule (1,047 times a GPU, 283 times a wafer-scale engine), 13,232 square millimeters total die area, an estimated 59.46 to 123.5 million dollars NRE at 5 nanometers, and 41.7 to 80.4 times better cost-effectiveness with a 357 times carbon-footprint reduction versus OpenAI-scale H100 clusters under an annual weight-update assumption. These are analytical and design estimates, not measurements from a fabricated chip. The step-by-step walkthrough above is an illustration of the described mechanism. Exact architecture, dataflow, the Metal-Embedding details, and the full cost and carbon models should be read from the paper.

Background

Why stable, dominant workloads pull hardware toward extreme specialization

It helps to see HNLPU within a broad pattern: the history of computing hardware is a pendulum between generality and specialization, and it swings toward specialization whenever a workload becomes both dominant and stable enough to justify building silicon around it. General-purpose hardware wins when workloads are diverse and changing, because flexibility is worth its efficiency cost; specialized hardware wins when one workload is so prevalent and so long-lived that the efficiency gains, applied at scale, dwarf the loss of flexibility. What is striking about LLMs is that a single pre-trained model has become general enough to be the workload — one network valuable enough, for the first time, to be worth hardwiring. That inverts the usual worry about specialization being too inflexible: the flexibility now lives inside the model, so the hardware underneath can be utterly rigid and still serve countless tasks. The recurring engineering challenge in these swings is always the up-front cost of specialization, and the recurring answer is to make as much of the specialized artifact as possible reusable, so the customization is confined to a small, cheap part. This drive to reap the efficiency and sustainability of specialization once a workload is stable enough to commit to is the same terrain the work on harnessing the power of specialization for sustainable computing covered elsewhere on this site occupies, arguing that tailoring hardware to enduring workloads is how computing gets more efficient and greener. HNLPU’s contribution is to show that, with weights hidden in the wiring and the expensive masks kept reusable, even the most extreme specialization — baking an entire LLM into a chip — can be made economically and environmentally worthwhile.

If you remember only three things

1. LLM inference spends most of its energy fetching billions of weight parameters from memory for every token, not on the arithmetic itself, and this weight movement grows with model size. Meanwhile a single pre-trained LLM has become general enough to serve countless tasks, so one dominant, stable model is now worth building dedicated hardware around. The most extreme specialization is to hardwire the weights physically into the chip: this eliminates all parameter fetching and lets every multiplier become a cheap multiply-by-constant circuit, giving orders-of-magnitude efficiency. But baking a specific model's weights into a 2D grid of transistors makes every chip's fabrication masks unique and un-amortizable — over six billion dollars of photomasks for one 120-billion-parameter model, plus a fresh set for each annual update — which is economically impossible.
2. HNLPU is the first hardwired LPU, enabled by the Metal-Embedding methodology. Instead of the transistors, it encodes each weight into the 3D topology of the metal interconnect wires, using a Hardwired-Neuron architecture that rearranges multiply-accumulate into accumulate-multiply-accumulate so the weights live in the metal. This packs weights about 15 times denser than device cells and, crucially, keeps 60 of the 70 photomask layers — including all the costly transistor and extreme-ultraviolet masks — identical across every chip and model, and therefore reusable. Only a few cheap upper metal masks are customized per model, cutting photomask cost 112 times and turning an annual model update into a cheap re-spin of a few layers rather than a whole new mask set.
3. Implementing a 120-billion-parameter model in 4-bit precision, the paper reports HNLPU achieves 249,960 tokens per second (5,555 times a GPU, 85 times a wafer-scale engine), 36 tokens per joule (1,047 times a GPU), 13,232 square millimeters die area, an estimated 59 to 124 million dollars one-time engineering cost at 5 nanometers, and 41.7 to 80.4 times better cost-effectiveness with 357 times less carbon than an equivalent H100 GPU cluster, under an annual weight-update assumption. The trade-offs: the chip is locked to one model, the up-front cost and die area are still large, the economics depend on the update-cadence assumption, and the results are analytical estimates rather than measurements from a fabricated chip.

Check yourself

Check your understanding

1. What is the core idea of a Hardwired-Neurons LPU, and why does it make LLM inference so much more efficient?
2. Why is the straightforward way of hardwiring an LLM — baking the weights into a 2D grid of transistors — economically impossible?
3. What is the key insight of the Metal-Embedding methodology that makes HNLPU affordable?
4. What does the paper report for HNLPU, and what are the main caveats to keep in mind?

Sources

The paper is “Hardwired-Neurons Language Processing Units as General-Purpose Cognitive Substrates” by Yang Liu, Yi Chen, Yongwei Zhao, Yifan Hao, Qi Guo, Tianshi Chen, Yunji Chen, and colleagues (State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Science and Technology of China; Institute of Software, CAS; and Cambricon Technologies), published at ASPLOS 2026, DOI 10.1145/3779212.3790169. It is available on arXiv as 2508.16151.

This explainer is written from the paper’s full text (the authors’ arXiv copy). HNLPU is a Hardwired-Neurons Language Processing Unit that physically hardwires an LLM’s weights into the computational fabric for orders-of-magnitude efficiency, achieving zero parameter fetching and constant-arithmetic circuits. Its enabling Metal-Embedding methodology embeds weights into the 3D topology of metal wires (via a Hardwired-Neuron accumulate-multiply-accumulate architecture) rather than a 2D grid of transistors, yielding about a 15 times density increase and keeping 60 of 70 photomask layers homogeneous across chips (including all transistor and extreme-ultraviolet masks), which reduces photomask cost by 112 times and brings the non-recurring engineering cost into a viable range. Implementing a 120-billion-parameter model in 4-bit, the paper reports 249,960 tokens per second, 36 tokens per joule (1,047 times a GPU), 13,232 square millimeters die area, an estimated 59.46 to 123.5 million dollars NRE at 5 nanometers, and 41.7 to 80.4 times better cost-effectiveness with 357 times less carbon than OpenAI-scale H100 clusters under an annual weight-update assumption. These are analytical estimates, not measurements from a fabricated chip. Numbers are the paper’s own; exact mechanisms and full results should be confirmed in the paper.