C4CAM: A Compiler for CAM-based In-memory Accelerators

Hamid Farzaneh, João Paulo Cardoso de Lima, Mengyuan Li, Asif Ali Khan, Xiaobo Sharon Hu and Jerónimo Castrillón

Content-addressable memories are a kind of chip that searches all of its stored data in a single parallel step — ideal for nearest-neighbor, recommendation, and genome-matching workloads — but until now they could only be programmed by hardware experts writing low-level, device-specific code; this work from TU Dresden and Notre Dame builds C4CAM, the first compiler that takes ordinary high-level PyTorch code and, through a stack of MLIR abstractions that lower it step by step, generates optimized code for these search chips, matching hand-tuned expert designs while letting anyone retarget across CAM types and explore latency, power, and area trade-offs by editing a spec file instead of rewriting the program.

The argument in one glance

Problem

Search chips are fast and efficient, but only experts can program them

A content-addressable memory, or CAM, is a chip that turns memory inside out: instead of giving it an address to fetch data, you give it data and it tells you, in a single parallel step, every location that matches. That makes it wonderful for search-heavy work — nearest-neighbor classification, recommendation, genome matching — at a fraction of the time and energy a normal processor would spend. The catch is that these chips are built by hand and driven by low-level, device-specific commands that only a hardware expert can write.
What people did

Program each CAM by hand, or build a one-off tool for one kernel

In practice, a CAM accelerator is programmed directly in its own low-level code, with the application painstakingly mapped onto the array by someone who understands both deeply. A few point tools exist that map one specific kernel — a decision tree, say — onto one specific CAM type. And the general compilers that do exist for in-memory computing target other styles of the technology: matrix-multiply crossbars or bitwise logic engines.
Why it fell short

No compiler speaks CAM search, so applications and design choices stay stuck

None of the existing in-memory-computing compilers support search operations, the one thing CAMs are built for. The point tools do not generalize beyond their single kernel and still demand deep expertise. So a machine-learning engineer with a high-level PyTorch model cannot target a CAM at all, and a hardware designer cannot quickly ask what a different CAM type, array size, or power budget would cost, because every such question means rewriting low-level code by hand. The application layer and the hardware layer are separated by a gap nobody had bridged.
This paper's idea

Compile high-level code down to CAMs through a stack of abstractions

C4CAM is the first compiler that takes ordinary high-level PyTorch and generates optimized code for CAM accelerators. It is built on MLIR, so the program descends through a hierarchy of abstraction levels — from tensor operations, to a device-agnostic in-memory-computing layer, to a new CAM-specific layer — with each transformation applied where it is most natural. A separate architecture-specification file describes the target hardware and the goal (latency, power, or area), so the same application retargets across CAM designs and explores the trade-offs without a single line of it being rewritten.

Why this problem is worth solving

The whole appeal of these chips rests on one unusual capability, so it is worth being precise about it.

Ordinary memory is addressed by location: you supply an address and it returns the data stored there. A content-addressable memory works the other way around — you supply a data pattern (a query) and it returns which of its stored entries match, comparing the query against every row at once in a single, constant-time operation. Physically, each memory cell compares its stored bit against the corresponding query bit, and a shared match line for each row combines those comparisons into a verdict for the whole row. Different sensing circuits give different flavors of match: exact match, best match (the closest row, used for nearest-neighbor search), or threshold match (all rows within some distance). Because the comparison happens in the memory itself and in parallel across the entire array, a CAM can do in one step what a conventional processor does by reading and comparing entries one after another — which is why search-heavy workloads run so much faster and cooler on it.

The reason a compiler matters here, rather than just a faster chip, is a structural gap between how people write programs and how these chips are driven.

Compute-in-memory (CIM) is the broad idea of doing computation inside the memory array instead of shuttling data back and forth to a separate processor, which is what makes it energy-efficient — data movement, not arithmetic, is where conventional machines burn most of their time and power. CIM comes in a few styles: crossbars that do matrix-vector multiplication, logic fabrics that do bitwise operations, and CAMs that do search. The difficulty is that each of these is a very different piece of hardware with its own low-level command set, and a high-level program — a PyTorch model, say — expresses none of that. Someone or something has to bridge the enormous distance between a line of tensor code and the electrical operations of a memory array. For crossbars and logic there were compilers to do it; for CAM search there was nothing, so the bridge was built by hand, once per application and once per chip.

What people did before, and where it broke

Program the CAM by hand in low-level code. An expert who understands both the application and the chip maps the kernel directly onto the array using device-specific commands. Where it broke. It works but does not scale: every application and every chip variant is a fresh manual effort, closed to anyone who is not a device expert.
Build a point tool for one kernel and one CAM type. Tools like DT2CAM and X-TIME map, say, a decision tree onto a specific CAM. Where it broke. They do not generalize to other comparison-intensive kernels, and they still require the programmer to deeply understand both the algorithm and the accelerator.
Reuse in-memory-computing compilers for crossbars. Frameworks such as OCC and CIM-MLC recognize matrix-multiply patterns and offload them to CIM crossbars. Where it broke. They target multiplication, not search — they have no notion of a CAM primitive, so they simply cannot express what a CAM does.
Reuse compilers for CIM logic. Other tools compile bitwise logic-in-memory using majority-inverter graphs and similar techniques. Where it broke. Again the wrong operation: logic-in-memory is not search, and these flows offer no path from a high-level search kernel to a CAM.
Lean on a generic CIM abstraction. The prior CINM framework offers a retargetable programming model for in-memory computing. Where it broke. It focuses on arithmetic and logic and does not support the search-based operations — distances, similarities, comparisons — that are the entire point of a CAM.
The gap. Every path either demanded hand-written low-level code, solved only one narrow case, or targeted a different kind of in-memory hardware entirely. What was missing was a general, retargetable compiler that starts from high-level code, understands search, and lowers it to real CAM designs — which is exactly what programming and exploring CAM accelerators needs.

How C4CAM works

From PyTorch to CAM hardware through a stack of MLIR dialects

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Descend through a hierarchy of abstractions, and fix each thing at the level where it is clear. A CAM program spans an enormous range, from tensor arithmetic at the top to the electrical timing of a match line at the bottom, and trying to handle all of it at once is what forced everything into hand-written low-level code. C4CAM’s foundation, borrowed from the MLIR compiler framework, is to make the program pass through a stack of intermediate representations — a high-level tensor dialect, a device-agnostic in-memory-computing dialect, and a new CAM-specific dialect — lowering step by step. Recognizing that a computation is really a search is done on the high-level graph where the shape is visible; laying work out across banks and subarrays is done on the low-level dialect where the hardware is visible; and nothing tangles application logic with device detail. Separating a hard problem into distinct levels so that each concern can be addressed on its own terms, instead of fighting them all together, is the same discipline the memory-resizing work covered elsewhere on this site follows, splitting apart what to resize from when to resize it so neither decision contaminates the other.

Partition the computation to fit fixed hardware, then merge the pieces. A CAM subarray is small — tens of rows and columns — while a real query might be thousands of dimensions wide, so no interesting kernel fits in one array. C4CAM handles this by tiling: it partitions the search into subarray-sized slices at the device-agnostic level, and inserts explicit merge operations that recombine the partial matches, aware of both the kind of operation and the direction in which results accumulate. The programmer never sees the split; the compiler sizes it to whatever the target’s subarrays are. Deciding how to carve a computation too large for the available hardware units into pieces that fit, and stitching the partial answers back together, is the same partitioning problem the parallelization work covered elsewhere on this site solves for large models, choosing how to divide a computation across limited devices so the whole thing can run.

Put the hardware in a spec, so one program explores a whole design space. The final move is what makes C4CAM more than a code generator: the target CAM is described in a separate architecture-specification file — its type, its hierarchy, its access modes, and the optimization goal — and that file, not the program, drives the lowering. Change the spec and the same PyTorch application regenerates for a different CAM design, letting a designer sweep array sizes, cell precisions, and power budgets and read off the latency and energy of each, without recoding or re-verifying anything. Turning a costly, expertise-bound design space into something that can be swept quickly by varying a description of the target is the same enabling move the virtual-memory research work covered elsewhere on this site makes, standing up a faithful, retargetable model so that questions which used to require building the real thing can be answered fast.

Following an HDC classifier through C4CAM

Compiling a hyperdimensional-computing similarity search

Tracing one kernel from PyTorch to CAM hardware shows the stack at work.

Step 1 of 5

Start from plain PyTorch

The input is a hyperdimensional-computing classifier in TorchScript: it multiplies the query hypervector against the stored class vectors and takes the top match. The front end imports this into the high-level Torch dialect of MLIR, using C4CAM's added support for the top-k primitive.

The same handful of tensor operations describe most CAM-suited kernels.

Why it works

Progressive lowering puts each transformation at the level where it is natural. A CAM program stretches from tensor math down to match-line timing → but layering the compiler into MLIR dialects lets search recognition happen on the high-level graph and hardware mapping happen on the low-level one → so each optimization is expressed cleanly instead of tangling application logic with device detail.
Pattern-matching recovers the search intent that high-level code hides. A dot product followed by a top-k looks like ordinary tensor arithmetic, not a search → but recognizing that idiom in the data-flow graph and fusing it into one similarity operation exposes exactly what the CAM is built for → so the compiler can offload the whole kernel to the search hardware rather than emulating it piece by piece.
Partitioning to the subarray granularity makes any kernel fit fixed hardware. Real queries dwarf a single CAM subarray of a few dozen rows → but tiling the search into subarray-sized pieces and merging their partial matches respects the hardware limit → so arbitrarily large problems run without the programmer hand-managing the split.
A hardware spec decouples the program from the device. CAMs vary widely in type, size, hierarchy, and power profile → but describing the target in a separate spec file that drives the lowering lets the same application retarget to any of them → so exploring the design space becomes editing a spec rather than rewriting and re-verifying code.
Matching expert-written code unlocks the accelerator for everyone. These chips were reachable only by device experts writing low-level commands → but C4CAM matches hand-tuned latency and energy to within a few percent starting from plain PyTorch → so a non-expert gets expert-quality results while an expert gains fast design-space exploration.

What it costs (trade-offs and scope)

Everything is simulated, not run on fabricated silicon. The results come from the CAMASim simulator with Eva-CAM circuit models; the technology parameters trace to a manufactured FeFET CAM, but no physical chip executed the generated code end to end.
The dramatic GPU wins are kernel-only and workload-favorable. The thousands-fold energy and large speed advantages are for the similarity-search kernel in isolation, often on small, low-dimensional datasets where the GPU is under-utilized. By Amdahl’s law the end-to-end gain is capped by the fraction of time actually spent in search, which the paper puts at roughly half to nearly all depending on the application.
The optimizations are simple heuristics, not a search for the best design. C4CAM optimizes for latency, power, or utilization with hand-written rules and generates configurations quickly, but it does not automatically find the optimal one — it is framed as an extensible base for future auto-tuning.
The power and density knobs trade latency steeply. Cutting power to a few percent of baseline can stretch execution more than a hundredfold; the framework exposes the trade-off honestly, but the extreme settings are impractical.
Front-end coverage is partial. The team had to hand-extend the PyTorch front end for the norm and top-k primitives, some CAM-suited applications like DNA mapping do not express cleanly in TorchScript, and the suggested alternative front ends are not demonstrated.
Preprocessing is excluded from the measurements. Reported latency and energy leave out hashing, quantization, and training-data transfer on the assumption they amortize, which may not hold for every deployment.
Heterogeneous mapping is left open. The architecture spec can describe heterogeneous systems, but choosing the optimal way to map an application across them is stated as future work.

The evaluation

Programming and exploring CAM accelerators

Property	GPU (RTX 3090)	Hand-crafted CAM	Point tool (DT2CAM)
Who can program it
Targets CAM search	No, does matrix multiply	Yes
Retarget across CAM designs	Not applicable		No
Matches hand-tuned quality	Not applicable	Is the baseline	Not reported
Search energy versus GPU	Baseline	Orders lower	Not reported

Programming and design-space exploration for CAM accelerators, evaluated in simulation (CAMASim with FeFET-CAM parameters) across K-nearest-neighbors, hyperdimensional computing, and DNA read mapping. C4CAM compiles high-level PyTorch to CAM code that matches hand-tuned expert designs within about 1 percent on latency and 5.5 percent on energy, beats an RTX 3090 GPU by roughly 5 to 14 times in speed and thousands of times in search energy, and retargets across CAM designs by editing a spec file.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

C4CAM is a genuine first: the first general, retargetable compiler that takes high-level PyTorch and generates optimized code for content-addressable-memory search accelerators, built cleanly on MLIR’s progressive lowering, matching hand-tuned expert designs to within a few percent while opening these chips to non-experts and turning design-space exploration into a spec edit. Read the performance numbers with their frame. Everything is measured in simulation, not on fabricated silicon. The eye-popping advantages over a GPU — thousands of times less energy — are for the search kernel alone, often on small datasets that under-utilize the GPU, so Amdahl’s law bounds the real end-to-end gain by how much of the application is search. The built-in optimizations are hand-written heuristics rather than an optimal search, the power-saving knobs cost enormous latency at their extremes, the front end covers only part of what users might write, and preprocessing costs are left out of the totals. The durable contribution is the architecture: a layered compiler that recognizes search, partitions to fit the array, and takes its hardware from a spec, so the CAM programming problem finally has a real compiler under it.

Background

Why compute-in-memory is worth a compiler at all

It helps to step back and ask why anyone builds a chip that searches in memory in the first place, because the answer explains the whole field this paper sits in. For decades, making programs faster meant making the processor’s arithmetic faster. But on modern workloads the arithmetic is rarely the bottleneck; the cost is in moving data between memory and the processor, over and over, across a bus that has not kept pace. Compute-in-memory attacks that directly by doing the work where the data already lives — a crossbar multiplies, a logic fabric computes bitwise, and a CAM searches — so the data never has to make the round trip. That is why a CAM can be thousands of times more energy-efficient than a GPU on a search: not because it computes faster, but because it barely moves anything. The same realization drives the deep-learning-on-GPUs retrospective covered elsewhere on this site, which measures whole real applications and concludes that data movement, not raw compute, is now the frontier. C4CAM matters because a hardware idea this powerful is useless if only a handful of experts can program it — the compiler is what turns a clever memory into something a machine-learning engineer can actually use.

If you remember only three things

1. Content-addressable memories (CAMs) invert normal memory: you give them a query and they return, in one parallel constant-time step, every stored row that matches — making them fast and extremely energy-efficient for search-heavy workloads like nearest-neighbor classification, hyperdimensional computing, recommendation, and DNA read mapping — but they have been designed and programmed by hand in low-level device code, and the existing compute-in-memory compilers target multiply-crossbars or bitwise-logic engines, not search, so no compiler could map a high-level program onto a CAM.
2. C4CAM is the first general, retargetable compiler from high-level PyTorch to CAM accelerators, built on MLIR so the program lowers through a hierarchy of dialects: a high-level tensor layer, a device-agnostic in-memory-computing layer that pattern-matches similarity searches and fuses them plus partitions kernels to subarray size, and a new CAM-specific layer that maps onto the bank/mat/array/subarray hierarchy using a separate architecture-specification file — so recognizing search, tiling to fit hardware, and choosing a device all happen at the level where each is natural, and the same application retargets across CAM designs and optimization goals by editing the spec.
3. In simulation across KNN, HDC, and genome mapping, C4CAM matches hand-tuned expert designs within about 1 percent on latency and 5.5 percent on energy, and beats an RTX 3090 GPU by roughly 5 to 26 times in speed and thousands of times in search energy — but all results are simulated, the GPU comparison is kernel-only on GPU-unfriendly datasets (so Amdahl's law bounds the real gain), the optimizations are simple heuristics rather than an optimal search, the power knobs trade latency more than a hundredfold at the extremes, and preprocessing costs are excluded.

Check yourself

Check your understanding

1. What is a content-addressable memory, and why is it useful?
2. How does C4CAM bridge high-level code and CAM hardware?
3. What makes C4CAM able to explore different CAM designs without rewriting the program?
4. Which statement best reflects the limits of the C4CAM evaluation?

Sources

The paper is “C4CAM: A Compiler for CAM-based In-memory Accelerators” by Hamid Farzaneh, Joao Paulo C. de Lima, Mengyuan Li, Asif Ali Khan, Xiaobo Sharon Hu, and Jeronimo Castrillon, published at ASPLOS 2024 (Volume 3), DOI 10.1145/3620666.3651386. The authors are at TU Dresden (with ScaDS.AI and the Barkhausen Institut) and the University of Notre Dame.

This explainer is written from the full 14-page text. Everything above comes from the paper’s own sections: the introduction and background motivating CAM-based search accelerators and the compute-in-memory landscape (crossbars, logic, and CAMs), and describing CAM types (binary, ternary, multi-state, analog), match schemes (exact, best, threshold), and the bank/mat/array/subarray hierarchy; the related-work section establishing that existing CIM compilers target crossbars or logic and that only narrow point tools existed for CAMs; the C4CAM framework section detailing the MLIR-based progressive lowering through the Torch, extended cim, and new cam dialects, the similarity-search pattern matching and fusion, the compulsory partitioning with merge_partial, the architecture-specification-driven cam-map pass, and the latency/power/utilization optimization heuristics; and the evaluation section reporting validation against hand-crafted HDC and DNA-read-mapping designs (about 0.9 percent latency and 5.5 percent energy deviation for HDC, 7.8 percent and 6 percent for DNA), the GPU comparisons (roughly 5 times faster and about 14,900 times less energy on HDC, about 14 times faster and 13,700 times less energy on KNN cosine similarity, up to 26 times higher throughput and about 9 times less energy on DNA read mapping against BWA-MEM), and the design-space-exploration study across subarray sizes and the base, power, density, and power-plus-density configurations.