UPMEM shipped the first commercial processing-in-memory hardware — thousands of tiny cores living inside DRAM, each running up to 24 hardware threads that share memory — but the vendor library hands those threads almost nothing to synchronise with, not even a read-write lock; PIM-STM (INESC-ID and Instituto Superior Tecnico, Lisbon) ports software transactional memory onto that hardware for the first time, building seven STM designs that span the classic taxonomy, letting a programmer pick one with a compile-time macro, and measuring which design the hardware actually rewards — the winner on average is the coarse-metadata NOrec design, because with only 24 threads per core the extra concurrency that fine-grained ownership records buy costs more in bookkeeping than it ever returns.
The argument in one glance
-
Problem
The hardware arrived; the programming model did not
Processing-in-memory is the idea of putting small processors inside the memory chips so that data-hungry work never has to travel to the CPU. UPMEM turned that idea into a product you can buy: a module that looks like an ordinary DIMM but carries dozens of DRAM Processing Units, and a full server holds 2560 of them across 160 GB of memory. Each unit has its own 64 MB bank of slow DRAM, a 64 KB scratchpad of fast memory, and up to 24 hardware threads. Those 24 threads share the same memory. The moment two of them touch the same word, the programmer is on their own — because the vendor library ships only the crudest synchronisation primitives, and does not even include a read-write lock.
-
What people did
Do what everyone did before transactions: hand-roll locks
The obvious answer is the one every systems programmer already knows. Give each shared structure a mutex, take it before touching the structure, release it afterwards. UPMEM does provide a basic mutual-exclusion primitive and a pair of atomic acquire and release instructions, so a determined programmer can build whatever they need on top. This is exactly how concurrent programming worked on ordinary multicore CPUs for decades, and it does work here too.
-
Why it fell short
Locks do not compose, and nobody knows which transactional design fits this machine
Locks are famously error-prone. Forget one and you get a silent data race; take two in the wrong order and you deadlock; and two lock-based components that are each correct in isolation cannot simply be composed into a correct larger one. Transactional memory was invented precisely to fix this, and after twenty years there is a rich literature of software transactional memory designs for CPUs. But every one of those designs was tuned for a machine with dozens of powerful cores, a hardware compare-and-swap instruction, and megabytes of cache. A DRAM Processing Unit has 24 slow threads, no compare-and-swap at all, and 64 KB of fast memory that the application also wants. Which of the classic designs still makes sense there was simply unknown, because nobody had ever built one.
-
This paper's idea
Build the whole design space, expose it as a compile-time dial, and let the hardware vote
PIM-STM is the first software transactional memory for processing-in-memory. Rather than pick one design and defend it, the authors lay out the classic taxonomy along four axes — how coarse the conflict-tracking metadata is, whether reads are visible to other transactions, when locks are taken, and when writes become visible — prune the combinations that are incorrect or pointless, and implement all seven that survive. A programmer selects one with a compile-time macro, and separately chooses whether the transactional bookkeeping lives in the fast 64 KB scratchpad or the slow 64 MB bank. Then the paper measures. The headline finding is that the design which wins on this hardware is the one that tracks the least: a NOrec-style single sequence lock, because with only 24 threads per unit the extra parallelism that fine-grained ownership records unlock is worth less than the memory traffic they cost.
Why this problem is worth solving
Almost every heavy workload today is limited not by arithmetic but by moving data. Putting small, weak processors physically inside the memory chips, so that data-intensive work is done where the data already lives instead of being shipped across the memory bus to a CPU. The processors are weak because they are built in the DRAM manufacturing process, but there are thousands of them and each sits next to its own private slice of memory, so the aggregate bandwidth is enormous.
That is where the second half of the problem comes in. A way of writing concurrent code where the programmer marks a region as a transaction instead of naming which locks to take. The runtime records what the transaction read and wrote, checks at commit time whether any other transaction interfered, and either makes all the transaction’s effects visible at once or throws them away and retries. The strong correctness property the good implementations provide is called opacity, which means that even a transaction destined to abort never observes a state that could not have arisen in some sequential execution — so a doomed transaction cannot be tricked into dividing by zero or following a corrupt pointer.
What people did before, and where it broke
- Hand-rolled locks on the in-memory cores. This is what the UPMEM toolchain supports today, and it is what every published UPMEM application uses. It works, and for a single well-understood data structure it is fast. Where it broke. It puts the entire correctness burden on the application programmer, on a machine where debugging is much harder than on a CPU, and it does not compose: you cannot atomically move an element from one lock-protected structure to another without inventing a global locking order by hand.
- Distributed transactional memory, the natural fit for thousands of separate memories. There is a whole literature on transactions spanning nodes that do not share memory, which is structurally what 2560 DRAM Processing Units look like. Where it broke. The units cannot talk to each other at all. Every message must be copied by the host CPU from source to destination, and the authors measured what that costs: reading one 64-bit word from another unit takes 331 microseconds, against 231 nanoseconds for a read of the local DRAM bank. That is three orders of magnitude. Worse, the current system cannot overlap communication with computation, which kills the speculative techniques distributed transactional memory normally uses to hide exactly that latency.
- Just port a well-known CPU implementation and call it done. TinySTM and NOrec are mature, well-studied, and open. Where it broke. Both lean heavily on compare-and-swap, which the UPMEM instruction set does not have, so the primitive has to be emulated. And their design assumptions — that tracking conflicts at word granularity is worth the metadata, that cache makes metadata access nearly free — were calibrated on hardware that shares almost nothing with this one.
- Reason about it from first principles instead of measuring. The CPU literature has clear folklore: fine-grained ownership records beat a single global lock, encounter-time locking and commit-time locking each have their workloads, write-back and write-through are a wash. Where it broke. Folklore calibrated on a 32-thread Xeon does not obviously transfer to a core with 24 slow threads where a single DRAM access dwarfs everything else in the instrumentation path.
- The gap. Nobody had built transactional memory for processing-in-memory at all, so there was no way to know which of the classic design choices the hardware rewards, how much it matters that the fast memory is only 64 KB, or whether the whole abstraction pays for itself once you are running on thousands of units instead of one CPU.
How PIM-STM turns a research taxonomy into a compile-time dial
From an unsynchronised DPU kernel to a measured transactional one
-
-
new
-
new
-
new
-
new
-
new
The idea
Enumerate the design space, prune it honestly, and expose what is left as a knob. The transactional-memory literature is not a single algorithm but a family, and PIM-STM starts by drawing the family tree. The first axis is how conflicts are detected: either metadata is attached to individual memory words through structures called ownership records, or there is a single sequence lock that every update transaction bumps at commit. The second is whether a read leaves a trace other transactions can see, which costs a write to shared state but spares you from re-validating. The third is whether locks are taken as you touch data or deferred to commit. The fourth is whether writes go straight to memory with an undo log or are buffered and applied at commit. Two combinations are struck out for principled reasons rather than measured ones, leaving seven implementations that ship together and are chosen by macro. Turning a research taxonomy into a dial the application author can turn is the same move as the adaptive sparse-matrix accelerator covered elsewhere on this site, which stopped arguing about which dataflow is best and made the choice reconfigurable instead.
On this hardware the least sophisticated design usually wins, and the reason is arithmetic about thread counts. Fine-grained ownership records exist to let non-conflicting transactions proceed in parallel. That is worth a great deal when you have 32 or 64 threads competing. It is worth much less when you have 24 slow ones, and the price is paid on every single instrumented read and write in the form of extra accesses to the DRAM bank — which dominate the cost of everything else the instrumentation does. So the NOrec-inspired design, which keeps one sequence lock and validates by re-reading values, delivers the best average and median throughput across the whole benchmark suite. It also aborts less under contention, partly because its transactions are shorter and partly because waiting for the sequence lock to go free before starting acts as an accidental contention manager. The lesson that the constraints of the memory system, rather than the elegance of the algorithm, decide the winner is the same one drawn by the near-data processing work covered elsewhere on this site, where how the data is spread across banks mattered more than how cleverly it was traversed.
Where the bookkeeping lives is a first-class design decision, not an implementation detail. A DRAM Processing Unit has two memories with a thousand-fold difference in character: 64 KB that is fast and 64 MB that is not. PIM-STM lets the transactional metadata sit in either. Moving it into the fast tier speeds up the transaction-heavy workloads by between 2.46x and 5.1x, a geometric mean of 2.86x, and shortens transactions enough that conflicts also become rarer. But it barely helps a workload that spends little time inside transactions, it consumes the very capacity the application wanted, and for the largest Labyrinth grids the read and write sets simply do not fit. It also reshuffles the ranking: with the metadata in the fast tier, ownership records become cheap enough that the two encounter-time variants take the best average. Treating placement across a fast small tier and a slow large tier as a tunable rather than a given is the same instinct as the memory-tiering architecture work covered elsewhere on this site, where the question was never which tier is better but which bytes deserve the good one.
Walking one transaction through the winning design
A NOrec-style transaction on a single DRAM Processing Unit
One tasklet out of 24, with the sequence lock and the read set both in the fast scratchpad.
Step 1 of 5
Start, but only when the coast is clear
The tasklet begins a transaction by reading the single global sequence lock. If some other transaction is currently committing, the lock reads as busy and this one waits rather than charging ahead.
This back-off is free — the lock has to be read anyway to snapshot the version — and it doubles as a contention manager. Under high contention it is a large part of why this design aborts less than the alternatives, because a transaction that waits a moment for a committer to finish does not have to throw away the work it would otherwise have done in that window.
Why it works
- The scarce resource on this machine is memory accesses, not parallelism. Every design choice that trades metadata for concurrency is betting that unlocked parallelism repays the bookkeeping → with 24 threads per unit that bet is small, while each extra access to the slow bank is expensive → the coarse design wins the average because it simply does fewer of them.
- Deferred writes make aborts nearly free. With a buffered write set there is nothing to roll back → an aborted transaction discards a buffer instead of replaying an undo log → the wasted-time slice of the breakdown shrinks, which is why the coarse design leads on the high-contention synthetic workload and the linked list rather than merely tying.
- Waiting on a busy global lock is accidental contention management. A transaction that starts while a committer holds the lock is nearly certain to abort → having it pause instead costs nothing, since the lock must be read anyway → the abort rate falls without any explicit back-off machinery, tuning parameter, or extra state.
- Emulated compare-and-swap is cheap because it never touches memory. The acquire and release instructions operate on a hardware bit register → the emulation costs no access to either the scratchpad or the bank → the missing instruction turns out not to be the obstacle it appears to be, and even the false serialisation from bit aliasing is short enough to disappear into the noise.
- Confining transactions to one unit is what makes them affordable at all. Cross-unit atomicity would have to be brokered by the host CPU → at 331 microseconds per remote word against 231 nanoseconds locally, and with no way to overlap the transfer with computation, every distributed technique for hiding that latency is unavailable → restricting scope keeps the abstraction inside the regime where processing-in-memory is actually winning.
What it costs (trade-offs and scope)
- The programmer inherits the partitioning problem. Because transactions never cross a unit, the application must be split so that atomic updates stay local. Cross-unit atomicity is still possible, but only sequentially through the host CPU. The paper is explicit that how hard this is depends entirely on the application, and that finer partitioning means both more algorithmic surgery and more overhead to support it.
- No design wins everywhere, and the average winner has a bad case. The paper states its own limit plainly: the most competitive design overall can be up to 2x slower on unfavourable workloads. On the low-contention synthetic benchmark, where transactions read up to 120 locations and rarely conflict, it is around 2.5x slower than the best option at 11 tasklets, because it validates constantly for nothing. The visible-reads design is roughly 2x faster than the ownership-record designs there — and clearly the worst of the three on the linked list, where upgrading a read lock to a write lock triggers spurious aborts.
- The fast tier is 64 KB, and the transaction metadata is competing for it. The 2.46x to 5.1x speedup from putting metadata in the scratchpad is real, but so is the capacity it takes from the application. For the largest Labyrinth grid the read and write sets do not fit at all, so that configuration has to keep its metadata in the slow bank.
- A single unit is dramatically slower than a CPU. For KMeans, one unit is roughly 100x slower than the CPU baseline on the low-contention workload and 300x on the high-contention one. Gains only begin to appear at around 300 to 400 units, and the reported speedups all depend on scaling out to thousands.
- Energy is the weak result, and the paper says so. At 2500 units the speedups are 6x and 14x for the two KMeans workloads and 8.48x down to 2.22x for Labyrinth as the grid grows, but the energy gains are consistently smaller than the speedups, and the largest Labyrinth grid actually consumes 31.5% more energy than the CPU despite running 2.22x faster. The energy figure for the in-memory system is also an estimate rather than a measurement, since the platform has no energy counters — the authors multiply the 370 W thermal design power by the runtime, while measuring the CPU side properly.
- Hardware limits cap scaling before the software does. On Labyrinth all designs peak at around 5 tasklets, not 24, because the workload is memory-bound enough to under-utilise the unit’s pipeline. In that regime the choice of transactional design barely matters, and the honest reading is that the study is measuring the machine as much as the algorithms.
- The lock table size is a compile-time constant. Ownership records are reached by hashing an address into a fixed table, so the table size trades aliasing against memory footprint — a trade that is much sharper here than on a CPU precisely because the fast tier is so small.
The evaluation
Three ways to synchronise 24 tasklets inside one DRAM Processing Unit
| Property | Hand-rolled locks (what the platform ships) | PIM-STM with word-level ownership records | PIM-STM with a single sequence lock |
|---|---|---|---|
| How conflicts are detected | |||
| Standing across the whole benchmark suite | |||
| Worst case reported | |||
| Margin over the runner-up where it wins | Not applicable | ||
| What the application programmer must reason about |
Check yourself
Check your understanding
1. Why does PIM-STM confine every transaction to a single DRAM Processing Unit?
2. Why does the coarse-metadata design beat the word-level ownership-record designs on this hardware, when the opposite is usually true on CPUs?
3. What happens when the transactional metadata is moved from the 64 MB bank into the 64 KB scratchpad?
4. What do the multi-unit energy results show?
Sources
The paper is “PIM-STM: Software Transactional Memory for Processing-In-Memory Systems” by Andre Lopes, Daniel Castro and Paolo Romano, published at ASPLOS 2024 Volume 2, pages 897-911, DOI 10.1145/3620665.3640428. The authors are at INESC-ID and Instituto Superior Tecnico, Universidade de Lisboa, and the library and benchmarks are publicly released.
This explainer is written from the full text, read from the authors’ preprint on arXiv (2401.09281), which carries an open licence. Everything above comes from the paper’s own sections 2 through 5: the description of the UPMEM system with its 2560 units, 24 tasklets, 64 KB scratchpad and 64 MB bank; the measured 331 microseconds versus 231 nanoseconds for a remote versus local word read; the four-axis taxonomy and the pruning that leaves seven viable designs; the emulation of compare-and-swap over a 256-bit atomic bit register and the discussion of bit aliasing; the read-write lock bit layout that packs a reader count, 24 reader identities and an owner read-set address into a 32-bit word; the single-unit study across the synthetic benchmarks, the linked list, KMeans and Labyrinth with its throughput, abort-rate and phase-breakdown plots; the summary conclusions on metadata granularity, read visibility, lock timing and write policy including the explicit statement that no design is best everywhere and that the average winner can be up to 2x slower; the scratchpad-versus-bank study with its 2.46x to 5.1x range and 2.86x geometric mean; and the multi-unit results at 2500 units with their speedups, their break-even point around 300 to 400 units, and the energy comparison including the 31.5% regression on the largest Labyrinth grid.