PIM Design Challenges I — Storage & Array Geometry

The last two pages sold the dream: an A×B array computing A×B MACs in one cycle. Section 10.2.4 is the invoice. Because PIM pulls computation into the storage array (or its periphery), it must compute in analog — and analog computation is sensitive to circuit and device non-idealities: nonlinearity, and process/voltage/temperature variations. Every fix trades among energy efficiency, throughput, area density, and accuracy, eroding the headline gains over conventional designs. (Architecture-level estimation tools like Accelergy help price these trade-offs; and the book’s footnote 13 laments that literature rarely separates accuracy lost to reduced precision — reproducible on any processor — from accuracy lost to circuit non-idealities like ADC noise, making designs hard to compare.)

This page covers the first three challenges: how many cells a weight needs, how big the array can be, and how many rows can fire at once. Part II covers columns, input delivery, and multi-bit MACs — and totals the bill.

The toy example (Fig. 10.7) — our recurring measuring stick

An FC layer with N = 1, C·H·W = 4, M = 4: input vector [1, 2, 0, 3] times a 4×4 filter matrix gives psums [7, 13, 10, 5]. The loop nest (Design 22) is all parallel-for — in theory one cycle, all 16 weights accessed, all 16 MACs done:

i = Array(CHW)          # input activations   = [1, 2, 0, 3]
f = Array(CHW, M)       # filter weights      (4 × 4)
o = Array(M)            # output partial sums = [7, 13, 10, 5]

parallel-for m in [0, M):
    parallel-for chw in [0, CHW):
        o[m] += i[chw] * f[chw, m]

Each challenge below will take a bite out of that one cycle. Keep the binary view in mind too (Fig. 10.7b): input [01, 10, 00, 11], weights in 2-bit binary, psums [00111, 01101, 01010, 00101] — 5-bit values, because accumulating four 2×2-bit products needs headroom.

Challenge 1 — storage elements per weight (Fig. 10.8)

Ideal: one element per weight (maximum density). Reality: an element holds 1–4 bits, so a higher-precision weight is split across several low-precision elements. Our toy’s 2-bit weights at one bit per element:

Figure 10.8, recreated: 2-bit weights at one bit per storage element turn the 4×4 array into 4×8.

And it compounds. NVM designs may use multiple devices per weight just to average out device variation (e.g., 3×3 devices per weight), and two elements (often two arrays) per weight for sign (differential coding, since conductance can’t be negative — from 10.2.1). SRAM cells grow extra transistors to compute or isolate (10.2.2). Every factor divides the effective density — the metric PIM exists to win.

Challenge 2 — array size

Ideal: a huge A×B (more parallel weights, more throughput, and the expensive periphery — ADC s/ DACs can be over 50% of NVM-design energy — amortized over more cells). Reality: the wires say no, in both circuit domains.

Charge domain: the capacitor-divider limit (Eq. 10.1)

Sensing a local capacitor’s charge through the bit line is a capacitance divider (Fig. 10.9: precharge BL to V_DD at t=0, open the access switch at t=1):

Eq. 10.1 lab — how bit-line capacitance swallows the signal

\Delta V_{BL} = (V_{DD} - V_{local}) \cdot \frac{C_{local}}{C_{local} + C_{BL}}

term	meaning	units	dictated by
$\Delta V_{BL}$	bit-line voltage change — what the read-out must detect	V	result
$V_{local}$	voltage on the local capacitor (V_DD or 0 for binary values)	V	stored data
$C_{local}$	local capacitor — its size is capped by cell density (1.2 fF in [351])	fF	cell design
$C_{BL}$	bit-line capacitance — grows with wire length, i.e., with array size	fF	array geometry

C_local1.2 fFC_BL (≈ array height)50 fFV_DD1 V

ΔV_BL (V_local = 0 case)23.4 mV

Worked example: C_local = 1.2 fF (the density-limited value from [351]), C_BL = 50 fF, V_DD = 1 V → ΔV_BL ≈ 23 mV. Stretch the bit line to 500 fF and the signal drops to ~2.4 mV — and the more psum bits you want to distinguish on that line, the LARGER the required ΔV_BL. Minimum resolvable ΔV_BL ⇒ maximum C_BL ⇒ maximum bit-line length. That is an array-size limit written in farads.

What Eq. 10.1 really says

The cell’s tiny capacitor must move the whole bit line. Its charge is fixed (density says so); the wire’s appetite grows with every added row. Past some length, the cell’s “voice” falls below the noise — so charge-domain arrays are short, or their read-out is exquisitely (and expensively) sensitive.

Current domain: the resistance-divider limit (Eq. 10.2)

Current-summing designs hit the mirror-image wall (Fig. 10.10): sensing whether a device is in R_ON or R_OFF through a resistive bit line:

Eq. 10.2 lab — how bit-line resistance blurs the weight states

\Delta V_{BL} = V_{HIGH} - V_{LOW} = V_{in} \, R_{BL} \, \frac{R_{OFF} - R_{ON}}{(R_{ON} + R_{BL})(R_{OFF} + R_{BL})}

term	meaning	units	dictated by
$V_{in}$	input voltage on the word line (∝ input activation)	V	input
$R_{ON}, R_{OFF}$	device low/high resistance states (LRS/HRS) — their gap is device-limited	kΩ	NVM device
$R_{BL}$	bit-line wire resistance — grows with length; worsens below 7 nm	kΩ	array geometry
$\Delta V_{BL}$	voltage difference distinguishing the two weight states	V	result

V_in0.5 VR_ON10 kΩR_OFF200 kΩR_BL (≈ array height)5 kΩ

ΔV_BL154 mV

Worked example: V_in = 0.5 V, R_ON = 10 kΩ, R_OFF = 200 kΩ, R_BL = 5 kΩ → ΔV_BL ≈ 154 mV — comfortably sensable. But the R_OFF − R_ON gap is fixed by the device, so as R_BL climbs (longer line, thinner wires at advanced nodes) the distinguishable window collapses. Again: minimum ΔV_BL ⇒ maximum R_BL ⇒ maximum bit-line length. Note the interesting non-monotonicity — at very small R_BL the divider also yields little signal; real designs live on a ridge.

The utilization ceiling (Fig. 10.11)

Even when physics allows a big array, the workload may not fill it — the PIM mapping constraints in reverse. Rows idle when the filter is small (C·H·W < B); columns idle when output channels are few (M < A):

Figure 10.11, recreated: a workload smaller than the array strands storage elements — the flip side of the “bigger array” instinct. (If instead C·H·W > B or M > A, temporal tiling and weight re-writing are needed — with a stricter ADC-precision requirement, since partial sums from multiple passes must add exactly.)

Where the dust settles — fabricated arrays and why multi-array scaling doesn’t rescue the economics:

Typical fabricated array sizes, and the residual costs of scaling out
	Fabricated sizes	Binding constraints
SRAM PIM	16b×64 up to 512×512	C_BL, disturb, cell area
NVM PIM	128×128 to 256×256	R_BL, R_OFF−R_ON, variation
Scaling out (many arrays)	fits big models, adds throughput…	…but periphery amortizes barely, + a new NoC

Dotted-underlined cells have explanations — click one.

Challenge 3 — rows activated in parallel (Fig. 10.12)

Ideal: all B rows at once — maximum parallelism. Reality: typically ≤ 64 rows (often far fewer on silicon — 10.2.1’s fabricated chips managed ~32), because:

ADC bits — more simultaneous rows = larger accumulated psums = more bits to resolve. Under-provisioning the ADC saves power but costs accuracy.
Cumulative device variation — each additional row stacks another device’s error onto the sum.
Bit-line tolerance — the total voltage drop / current has a ceiling (electromigration gets worse at 7 nm and below; and in 6T SRAM, a big drop can flip cells — the read-disturb tax again).

Run the toy example under a 2-rows-per-cycle limit — the psums arrive in installments, exactly as in Figure 10.12 ([1, 4, 7, 2] after cycle one, [7, 13, 10, 5] after cycle two):

Figure 10.12, live — only 2 of 4 rows may activate per cycle (3-bit ADC)

cycle 1 / 2limits: 2/4 rows · all cols per cycle

Cycle 1: rows 0, 1 drive their inputs on the word lines; columns 0, 1, 2, 3 accumulate products on their bit lines. Click any cell to inspect its MAC.

The one-cycle dream is now two cycles, and a digital accumulator after each ADC must add the installments ([1,4,7,2] + [6,9,3,3]). Cost: halved throughput and a second ADC conversion per column (recent work accumulates in the analog domain to avoid the extra conversions). Design 23’s loop nest says the same thing: the row loop split into a parallel-for over CHW/2 and a temporal for over 2.

# Design 23 — row-limited loop nest (2 parallel rows)
parallel-for m in [0, M):
    parallel-for chw1 in [0, CHW/2):
        for chw0 in [0, 2):              # temporal — the new cycles
            chw = chw1 * 2 + chw0
            o[m] += i[chw] * f[chw, m]

Running total on the toy example

Theoretical: 1 cycle. After the row limit: 2 cycles. Part II adds the column, input-delivery, and bit-serial-MAC limits — and the same 16-MAC computation will reach 12 cycles.

Check yourself

Check yourself — PIM Design Challenges I

1.Why does one weight usually need MULTIPLE storage elements in a real PIM array?

2.In Eq. 10.1, ΔV_BL = (V_DD − V_local) · C_local/(C_local + C_BL). Predict what happens as the bit line gets longer.

3.The 4×4 toy workload runs on an 8×8 array (Fig. 10.11b). What utilization results, and why does the trend toward efficient DNN models make this worse?

4.Why is the number of rows activated in parallel typically limited to ~64 or fewer, when B might be 256?

5.With only 2 of 4 rows active per cycle (Fig. 10.12), the toy example takes 2 cycles: psums [1,4,7,2] then final [7,13,10,5]. What extra hardware and cost does this incur?

5 questions