An Encoding Scheme to Enlarge Practical DNA Storage Capacity by Reducing Primer-Payload Collisions

DNA is an incredibly dense archival medium, and to grab specific files back you tag strands with short primer sequences and copy only the ones you want — but as the data grows, those primers accidentally match the encoded data itself, misfiring the copy step and destroying capacity; this work from Minnesota and UT Dallas shows the real limit on DNA storage is these primer-payload collisions, not encoding density, and builds Collision Aware Code, which trades a little density to shape the data away from the primers, freeing 120 percent more usable primers and lifting a tube from 212 to 295 gigabytes.

The argument in one glance

Problem

DNA is a dense archival medium, but you must be able to grab one file back

DNA can store digital data at extraordinary density and last for centuries, making it a promising medium for long-term archives as the world generates more data than tape and disk can hold. Files are encoded into sequences of the four DNA bases, synthesized into physical strands, and stored in tubes. Reading a specific file back without sequencing the entire tube relies on random access: each strand is tagged with a pair of short primer sequences, and a chemical copying reaction amplifies only the strands carrying the primer pair you name, so just the file you want gets read. A tube's capacity therefore depends on how many distinct primer pairs you can use.
What people did

Pack more bits per base by maximizing encoding density

Prior work enlarges a tube's capacity by increasing encoding density — the number of bits packed into each DNA base — while obeying the constraints biology imposes, like avoiding long runs of identical bases and keeping a balanced mix of the four bases. A whole line of encoding schemes competes to approach the theoretical density limit of about two bits per base, and the assumption is that denser encoding means more data per tube.
Why it fell short

Density is not the real limit — the primers collide with the data

The overlooked catch is that a primer can accidentally resemble a stretch of the encoded data. During the copying reaction, the primer meant to grab your target strands also latches onto these look-alike payloads, amplifying the wrong strands and burning through the reaction's limited reagents. Because copying is exponential, even a few collisions snowball until the tube fills with junk and your file cannot be read — and this loss of millions of strands is beyond what error-correcting codes can fix. So every colliding primer must be retired, and at real scale four of five leading schemes lose over 99 percent of their capacity.
This paper's idea

Shape the data to look nothing like the primers

Since collisions come from the data looking like a primer, the fix is to encode data that is deliberately dissimilar to every primer. Collision Aware Code trades a little encoding density for the freedom to do this: instead of mapping each chunk of bits to one fixed DNA sequence, it offers four candidate sequences and, looking back at what it just wrote, picks the one least likely to resemble any primer — avoiding the base patterns primers have. It frees far more primers from collisions, so the practical capacity of a tube grows even though each base carries fewer bits.

Why this problem is worth solving

The heart of the matter is that a tube’s real capacity is set by two multiplied factors, and the field had been optimizing only one.

Storing data in DNA means encoding bits into the four bases and synthesizing strands, but retrieving one file cheaply needs random access, provided by the polymerase chain reaction — a copying process that exponentially amplifies only the strands whose primer pair you specify, until they dominate the sample and can be sequenced alone. So a tube holds, roughly, the number of usable primer pairs times how much data each pair can tag. Encoding density improves the second factor: pack more bits per base, more data per strand. But density is capped near two bits per base by biology anyway, and it does nothing for the first factor, the number of usable primers. And that first factor turns out to collapse as data grows, because of primer-payload collisions. A collision is a stretch of a dozen or more bases, allowing a couple of mismatches, that a primer and some encoded payload share almost identically. When the copying reaction runs, the primer meant to bind your target latches onto every colliding payload too, amplifying irrelevant strands and consuming the reaction’s finite reagents. Since the process is exponential, a slight misfire compounds cycle after cycle until the sample is saturated with the wrong sequences and the target file cannot be sequenced. A single primer pair tags millions of strands, so this failure loses far too much to recover with error-correcting codes, which are built for scattered base errors, not the loss of a whole file. The only safe response is to retire any primer that collides — which is why a tube’s usable primers, and thus its true capacity, hinge on collisions, not density.

The reason this went unnoticed, and why measuring it took real work, is that proof-of-concept demonstrations were far too small and too uniform to trigger it.

Early DNA storage systems stored kilobytes to a few megabytes of homogeneous data — often a handful of synthetic files padded with copies of the same background strands, because copying is far cheaper than synthesizing fresh diverse data. Homogeneous data barely collides, and tiny data uses few primers, so collisions simply did not surface; the one prior study that pushed past a hundred megabytes did hit a copying failure from collisions but the problem remained a footnote. This work measures it properly. The authors built the largest primer library reported, twenty-eight thousand primers, each a twenty-base sequence satisfying the biological rules for a reliable tag. Then they encoded 2.5 terabytes of real, diverse data across five types with five leading encoding schemes, and — since synthesizing that much DNA to test in a lab would cost on the order of tens of thousands of dollars per megabyte — checked collisions computationally with a standard genomic alignment tool. The result is stark: without collisions the schemes would reach 461 to 840 gigabytes per tube, but with collisions four of the five lose more than 99 percent of capacity, down to hundreds of megabytes, and even the most collision-resistant loses about 71 percent. The near-direct binary-to-DNA schemes produce payloads that resemble almost every primer; at just 500 megabytes of audio, one such scheme already collides with 99 percent of the library. Making the invisible cost visible, at realistic scale and diversity, is what turns density-chasing into the wrong objective and makes collision reduction the thing worth solving.

What people did before, and where it broke

Maximize encoding density under biological constraints. Pack as many bits per base as possible while avoiding long identical runs and unbalanced base composition. Where it broke. Density is capped near two bits per base and does nothing about usable primers; the densest, near-direct binary-to-DNA schemes produce payloads that resemble almost every primer, so they lose over 99 percent of capacity to collisions at scale.
Add error-correcting codes to survive DNA errors. Protect the data with Reed-Solomon and similar codes for the base insertions, deletions, and substitutions that synthesis and sequencing introduce. Where it broke. These codes handle scattered base errors, but a collision fails a whole primer pair tagging millions of strands, a loss far beyond what error correction can recover.
Test on small, homogeneous datasets. Demonstrate feasibility on kilobytes to megabytes, often padded with copies of the same background strands. Where it broke. Homogeneous data barely collides and small data uses few primers, so the collision problem never appeared — it only emerges at the large, diverse scale that practical archives demand.
Lean on the one scheme that happens to resist collisions. Rotation code forbids consecutive identical bases, so it accidentally collides less and keeps the most capacity. Where it broke. It was never designed for collision avoidance, so it still loses about 71 percent of capacity, leaving most primers unusable.
The gap. Every prior approach optimized encoding density or generic error tolerance while ignoring the interaction between primers and payloads that actually caps a tube. What was missing was an encoding scheme that deliberately shapes payloads to be dissimilar from primers, trading a little density for far more usable primers — which is exactly what practical, large-scale DNA storage needs.

How Collision Aware Code works

Encoding data that avoids resembling any primer

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Find the real limiter instead of pushing the metric everyone chases. The field had raced to maximize encoding density, but this work measures the practical capacity at realistic scale and discovers that density was never the binding constraint — usable primers were, and they collapse to collisions. Only by encoding terabytes of diverse data and counting the collisions does the true ceiling become visible, redirecting the whole effort. Diagnosing that the community optimized the wrong figure of merit, and that the actual ceiling sits somewhere it had overlooked, is exactly the reckoning the deep-learning-on-GPUs retrospective covered elsewhere on this site delivers, arguing that chasing peak arithmetic missed the real bottleneck that had quietly become the thing that governs performance.

Trade a little of the headline metric for what actually binds. Because collisions, not density, cap the tube, Collision Aware Code deliberately gives up encoding density — down to one bit per base — to gain the freedom to shape payloads away from the primers, and the huge jump in usable primers more than repays the loss. Sacrificing some of the obvious, maximized quantity to buy robustness against the effect that really governs the outcome is the same bargain the noise-resilient quantum-optimization work covered elsewhere on this site strikes, giving up some circuit fidelity or breadth to keep the computation working under the noise that actually limits it in practice.

Design the code so its output obeys the medium’s constraints by construction. Rather than encoding freely and hoping the result behaves, Collision Aware Code builds collision avoidance into the encoding itself: every three-bit chunk carries multiple candidate sequences chosen so the encoder can always pick one that satisfies the no-identical-run, few-complements, balanced-composition patterns the physical medium demands. Constructing a code whose every codeword respects the target medium’s physical and error constraints, rather than correcting afterward, is the same discipline the quantum-error-correcting-code synthesis work covered elsewhere on this site follows, generating codes tailored to what the noisy physical substrate can actually tolerate.

Following a chunk of bits through Collision Aware Code

Encoding three bits into collision-avoiding DNA

Tracing one three-bit chunk shows how the encoder dodges primer patterns.

Step 1 of 5

The next three bits arrive to be encoded

The encoder takes the next three-bit chunk of the data. A density-first scheme would map it to one fixed DNA sequence, whatever the surrounding bases, leaving no room to avoid resembling a primer.

Encoding three bits as three DNA bases gives a density of one bit per base.

Why it works

Shaping payloads to be dissimilar from primers frees the primers that collisions retired. Density-first codes produce payloads whose subsequences resemble almost every primer → but Collision Aware Code encodes payloads with the opposite patterns — no identical runs, few complements, balanced composition → so far fewer primers collide, and the count of usable primers rises 120 percent over the best prior scheme.
Removing consecutive-base similarity defeats the exponential copying failure. A primer with even a few collisions fails, because the copying reaction compounds each misfire cycle after cycle → but avoiding identical runs and complements creates differences at consecutive bases, which no primer subsequence can match → so the target strands amplify cleanly and the primer stays usable.
Multiple candidates per chunk give the encoder room to dodge collisions. Mapping each chunk to one fixed sequence leaves no way to steer away from a primer pattern → but four candidate triplets per three-bit chunk, chosen to be diverse, always include a safe one → so whatever the surrounding bases, the encoder can pick a collision-avoiding output.
The gain in usable primers outweighs the deliberate loss of density. Dropping to one bit per base seems to shrink capacity → but freeing more than twice as many primers overwhelms the density loss → so the practical tube grows from about 212 to 295 gigabytes, a 39 percent gain, despite the lower density.
Caching the complement checks keeps encoding negligible next to synthesis. Rechecking complements against the previous bases for every candidate is costly → but a look-back window caches the complementary stretches already seen, so each check becomes a lookup → so encoding stays linear-time and thousands of times faster than the DNA synthesis that actually bottlenecks the system.

What it costs (trade-offs and scope)

The density sacrifice is large, and only pays off at scale. Collision Aware Code drops to one bit per base, well below the 1.58 to 1.82 of the schemes it beats; its advantage depends on collisions dominating, which they do for large diverse archives but not for the tiny datasets earlier work used.
Everything is digital analysis, not wet-lab chemistry. Collisions are estimated with a genomic alignment tool, errors with a simulator, and hairpin propensity with a sequence analyzer, because synthesizing terabytes to test in a lab is cost-prohibitive; the collision model is an approximation of real biochemistry, not a measurement of it.
Collision probability is not fully captured by sequence similarity. Detecting collisions is essentially approximate string matching, but the real chance of a misfire also depends on the experimental conditions, so the true capacity bound remains unknown and the estimates are indicative.
Error recoverability is comparable, not superior. Because it decodes three bases at a time, an insertion or deletion can misalign the reading, so it needs a few more sequencing reads to recover than a scheme that decodes base by base — solid and on par with a well-verified code, but not the most error-robust.
The three-bits-per-triplet choice is one point on a tradeoff. Encoding two bits per triplet frees even more primers but loses too much density, and four bits per triplet is denser but leaves fewer candidates; three is the sweet spot for this primer library and today’s technology, and could shift as the technology changes.
It is a proof of a principle, not a final code. The authors present it as one instance showing that trading density for collision avoidance pays, and explicitly invite the community to design codes that achieve both high density and strong collision avoidance; the results are tied to the specific 28,000-primer library and current biological constraints.

The evaluation

Three ways to encode data for DNA storage

Property	Density-first codes	Rotation code
What it optimizes
Encoding density		1.58 bits per base
Usable primers (of 28,000)		About 8,100 (29 percent)
Practical tube capacity	Hundreds of megabytes to a few gigabytes
Error recoverability	Varies	Good

Three ways to encode data for DNA storage, evaluated by digital analysis: a 28,000-primer library, 2.5 terabytes of real data across five types, collisions checked with genomic alignment and errors with a DNA storage simulator. Density-first codes lose over 99 percent of capacity to primer-payload collisions and the best prior code (Rotation) loses about 71 percent, while Collision Aware Code trades density down to 1 bit per base to free 120 percent more usable primers, lifting a tube from about 212 to 295 gigabytes with comparable error recoverability — because it shapes payloads to be unlike any primer rather than maximizing bits per base.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

This paper does two valuable things: it exposes an overlooked ceiling on DNA storage, and it offers a first fix. By encoding terabytes of diverse real data and counting primer-payload collisions with genomic alignment, it shows that the true limit on a tube is not encoding density — the metric the field had been racing to maximize — but the number of usable primers, which collapses because primers accidentally resemble the encoded data and misfire the exponential copying reaction, losing over 99 percent of capacity for the densest schemes. Its Collision Aware Code trades density down to one bit per base for the freedom to shape payloads away from every primer pattern, freeing 120 percent more usable primers and lifting a tube from about 212 to 295 gigabytes, with error recoverability on par with proven codes and encoding cost negligible beside DNA synthesis. Read it with its scope. The density sacrifice is real and only pays off at large, diverse scale; all results are digital analysis rather than wet-lab chemistry, so the collision model approximates biology; the true capacity bound is still unknown, since collision probability depends on conditions beyond sequence similarity; and the code is explicitly a proof of principle, tied to one primer library and today’s technology, inviting better codes that get both density and collision avoidance. The durable lesson is that when a field has been optimizing one number for years, the highest-value move can be to measure the system at realistic scale and discover the limit was somewhere else entirely.

Background

Why the constraint that binds keeps moving as a technology scales

It helps to see this work as a case of a recurring pattern: the metric that matters early in a technology is rarely the one that matters once it scales. DNA storage began as a feasibility question — can we write bits into molecules and read them back at all — and at that stage encoding density is the natural figure of merit, because every base is precious and the datasets are tiny. But scaling reveals second-order effects that were negligible at small size and become dominant at large size, and here the primer-payload collision is exactly such an effect: harmless when a tube holds a few megabytes of similar data, ruinous when it holds hundreds of gigabytes of diverse data. The real bottleneck of the whole system, meanwhile, is not the encoding at all but the physical process of synthesizing DNA base by base, which is orders of magnitude slower than any computation around it — so spending extra computation to encode more carefully is essentially free. The in-storage serverless-acceleration work covered elsewhere on this site makes the kindred point that the dominant cost in a data system is often the physical movement and access, not the compute, so the smart move is to spend cheap computation to reduce the expensive physical work. The broad lesson is that as a storage or computing technology matures from demonstration to deployment, the engineering that pays off shifts from maximizing the obvious first-order metric to taming the second-order effect that scale has quietly turned into the true limit.

If you remember only three things

1. DNA stores data at extraordinary density and lasts for centuries; files are encoded into the four bases, synthesized into strands, and stored in tubes, and one file is retrieved without sequencing everything via random access — tagging each strand with a primer pair and using an exponential chemical copying reaction to amplify only the strands whose primer you name. A tube's capacity is thus usable primers times data per primer, and while the field raced to maximize encoding density (bits per base, capped near two), the actual ceiling is usable primers, which collapse because of primer-payload collisions: a primer that accidentally resembles a stretch of the encoded data misfires the copying reaction, amplifying junk and burning reagents until the target file cannot be read, a loss of millions of strands beyond error correction, so every colliding primer must be retired.
2. Measuring the practical capacity at realistic scale (a 28,000-primer library, 2.5 terabytes of diverse real data, collisions checked with genomic alignment since wet-lab synthesis is cost-prohibitive) reveals that four of five leading schemes lose over 99 percent of capacity and the best (Rotation) loses about 71 percent. Collision Aware Code responds by trading encoding density (down to 1 bit per base) for the freedom to shape payloads unlike any primer: it maps each three-bit chunk to four candidate DNA triplets (chosen for no internal runs, diverse starting bases, complementary pairs, and balanced composition) and picks the candidate that, given the previous seventeen bases, best avoids identical runs, complementary stretches, and unbalanced composition — the patterns primers have — with a look-back window caching the checks to keep encoding linear-time.
3. By digital analysis on ImageNet and audio, Collision Aware Code raises usable primers from about 8,100 (29 percent) to about 17,900 (64 percent) of the library — a 120 percent increase, near the 71-percent upper bound of an undecodable exhaustive search — lifting practical tube capacity from about 212 to 295 gigabytes (39 percent) despite lower density, with error recoverability comparable to proven codes and encoding thousands of times faster than DNA synthesis. Its limits: the density sacrifice only pays off at large diverse scale, all results are simulated rather than wet-lab, the true capacity bound is unknown, it decodes triplets so needs a few more reads under insertion/deletion errors, three bits per triplet is one point on a tradeoff, and it is a proof of principle inviting codes that achieve both high density and collision avoidance.

Check yourself

Check your understanding

1. Why is encoding density not the real limit on how much data a DNA tube can hold?
2. What is the core idea of Collision Aware Code?
3. How did the authors show that primer-payload collisions are a severe, real-scale problem?
4. Which statement best reflects the scope and limits of Collision Aware Code?

Sources

The paper is “An Encoding Scheme to Enlarge Practical DNA Storage Capacity by Reducing Primer-Payload Collisions” by Yixun Wei, Bingzhe Li, and David H.C. Du, published at ASPLOS 2024, DOI 10.1145/3620665.3640417. The authors are at the University of Minnesota and the University of Texas at Dallas.

This explainer is written from the full 14-page text. Everything above comes from the paper’s own sections: the introduction and background on DNA storage (encoding to bases, synthesis into tubes, and polymerase-chain-reaction random access via primer pairs) and the factors affecting tube capacity (parallel factor, encoding density capped near two bits per base, payload length, and usable primers); the analysis of primer-payload collisions (almost-identical subsequences over twelve bases with at most two mismatches, the exponential amplification failure, and why error correction cannot recover the loss of millions of strands); the investigation building a 28,000-primer library under the primer design rules (GC content, melting temperature, homopolymer, Hamming distance, and complementarity constraints) and encoding 2.5 terabytes across five data types with five schemes (Church, Rotation, Blawat, Grass, Fountain), checking collisions with the BLAST alignment tool and reporting that four of five lose over 99 percent of capacity while Rotation loses about 71 percent to reach roughly 212 gigabytes; the collision-reduction patterns derived from characterizing primers; the Collision Aware Code design with its four-candidate bit-triplet-to-DNA-triplet mapping table (four principles), the look-back candidate selection policy prioritizing homopolymers then complementary sequences then GC balance, and the look-back window caching for linear-time encoding; and the evaluation reporting usable primers rising from 8,136 to 17,897 (against an exhaustive-search upper bound of 19,992), tube capacity from 211.96 to 295.11 gigabytes, error recoverability comparable to Grass and other codes across three error models with Reed-Solomon protection, hairpin free-energy analysis via OligoAnalyzer, coding speed far below the DNA-synthesis bottleneck, and the density-versus-collision tradeoff showing three bits per triplet as the sweet spot.