Getting a Handle on Unmanaged Memory

Programs written in C and C++ can never move an object once it is allocated, because raw pointers nail it to a fixed address — and over time this leaves the memory heap riddled with gaps, so a long-running database or web server hogs far more memory than its live data needs; this work from Northwestern revives an old idea, the handle, letting the compiler transparently route every pointer through one movable reference so objects can be relocated, and builds a defragmenter on top that shrinks Redis's memory use by up to 40 percent with no changes to the application's code.

The argument in one glance

Problem

C and C++ cannot move heap objects, so memory fragments

In unmanaged languages, a heap object is stuck at its address because raw pointers hold that address directly and can be copied anywhere. Over time the heap fills with gaps between live objects, so a long-running program's memory balloons far beyond its live data. Any allocator that cannot relocate objects provably suffers this fragmentation.
What people did

Ignore it, or hand-build a per-application defragmenter

Either let fragmentation fester and pay in crashes, out-of-memory kills, or cloud bills, or write an ad-hoc defragmenter for one application, as Redis did with activedefrag.
Why it fell short

Ignoring it hurts; custom defrag does not transfer

Ignoring fragmentation forces restarts and wastes memory. A hand-built defragmenter works but is enormous and brittle — Redis's activedefrag took thousands of lines to handle its specific data structures and pointer semantics, and none of it transfers to another application without a great deal of reengineering.
This paper's idea

Let the compiler add transparent handles (Alaska)

Alaska revives handles — a single level of indirection through a movable reference — so any heap object can be relocated by updating just one pointer. Its compiler automatically and transparently rewrites unmodified C/C++ code to use handles, so programmers never see them and cannot get them wrong, and compiler-runtime co-design holds the overhead to about 10 percent. On top, Anchorage defragments any unmodified application, cutting Redis memory usage by up to 40 percent.

Why this problem is worth solving

A limitation baked into C and C++ quietly wastes memory in some of the most important software we run.

In unmanaged languages like C and C++, a heap object lives at a fixed address for its entire life, and it cannot be moved. The reason is how pointers work: a raw pointer simply holds the object’s address, and the program can copy that address, store it in other structures, do arithmetic on it, and scatter references to it anywhere. There is no way to find and update all of those references, so once an object is placed, it is stuck. The costly consequence is memory fragmentation. As a long-running program allocates and frees objects, the heap develops gaps — freed slots wedged between live objects — and because those gaps cannot be closed by moving anything, the program’s memory footprint balloons far beyond the size of its live data. This has long plagued databases and web servers, and it is not a matter of a smarter allocator: it has been proven that any allocation strategy that cannot relocate objects will suffer fragmentation. Managed languages like Java and C# do not have this problem, because their strict pointer semantics let the runtime find and update every reference, so a compacting garbage collector can freely move objects and squeeze out the gaps. Unmanaged languages simply lack that capability.

Faced with fragmentation, unmanaged programmers have only bad choices.

One option is to ignore fragmentation and let it fester, which leads to application restarts, the operating system’s out-of-memory killer stepping in, or paying for extra memory on a cloud service just to run a bloated program. The other is to attack it head-on with a custom, per-application defragmenter. Redis, the popular in-memory database, did exactly this: its activedefrag feature took thousands of lines of intricate code to handle Redis’s specific data structures and pointer semantics, and it works well — but none of that effort transfers to another application. The idea that could solve this generally is old. A handle is a single level of indirection: instead of a raw pointer, the allocator hands out an opaque handle that refers to the object through one movable master pointer. To use the object, the program pins the handle to get its current address, and unpins it afterward; while a handle is unpinned, the runtime is free to move the object, updating only that single master pointer. Early personal-computer operating systems, before hardware virtual memory existed, used handles exactly this way to defragment a shared heap. But classic handles were manual and directly visible to programmers, so they were easy to misuse and became a nightmare to program with — which is why they were abandoned.

What people did before, and where it broke

Ignore fragmentation. Let the heap fragment and hope for the best. Where it broke. The program’s memory balloons, leading to restarts, out-of-memory kills, or paying for more cloud memory than the live data needs.
Write a per-application defragmenter. Hand-build a defragmentation system for one program, like Redis’s activedefrag. Where it broke. It takes thousands of lines to handle that application’s specific data structures and pointer semantics, and none of it transfers to another application without major reengineering.
Use classic manual handles. Reintroduce handles so objects can move, as early operating systems did. Where it broke. Manual, visible handles are cumbersome and easy to misuse — a single pin-or-unpin mistake can corrupt memory — so they were abandoned.
The gap. No approach gave unmanaged languages general, transparent heap-object movement without programmer effort or per-application code. Providing exactly that, by having the compiler automate handles, is what Alaska does.

How Alaska works

From immovable raw pointers to compiler-managed transparent handles

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Add one movable reference so objects can be relocated. Alaska’s foundation is the handle: routing every reference to a heap object through a single master pointer, so that moving the object requires updating just that one pointer rather than the impossible task of finding and fixing every scattered raw pointer — pinning a handle yields the object’s current address, and while unpinned the object is free to move. Introducing a controlled level of indirection so that an object can be pinned in place when in use and safely relocated when it is not, without breaking the references to it, is the same mechanism the copy-on-write pinning work covered elsewhere on this site is built around, carefully governing when a memory object may move or must stay put so that relocation never corrupts the references that depend on it.

Have the compiler do it, transparently, on unmodified code. The reason classic handles failed was that they were manual and visible; Alaska’s answer is to make the compiler insert all the pinning and unpinning automatically and hide handles entirely, so a programmer writes ordinary pointer code with any custom semantics they like, cannot possibly misuse handles because they never see them, and existing programs get the capability with zero changes. Transparently rewriting all of a program’s pointer usage at compile time to gain a new capability, on unmodified C and C++ and with no programmer effort, is the same approach the pointer-integrity work covered elsewhere on this site takes, having the compiler instrument every pointer in existing code so a property the source never asked for is enforced without the developer lifting a finger.

Turn movement into a reusable service that reclaims wasted memory. Because Alaska exposes object movement through an extensible interface, a defragmenter (Anchorage) can be built once and applied to any program, compacting live objects to squeeze out the gaps and return wasted memory to the system — a capability that previously cost thousands of lines per application. Automatically reclaiming memory that a program is needlessly holding, through a runtime mechanism that works on unmodified applications, is the same service the goroutine-leak work covered elsewhere on this site provides, extending the runtime so that memory which is being wasted is recognized and given back without the developer having to intervene.

Following an object through Anchorage defragmentation

Compacting a fragmented heap in an unmodified program

A trace of Alaska and Anchorage defragmenting a long-running application, following the paper's design.

Step 1 of 5

A long-running program fragments its heap

A program like Redis allocates and frees many objects over time, so its heap fills with gaps between live objects and it holds far more memory than its live data needs.

Fragmentation grows in phases and cannot be undone without moving objects.

Why it works

Fragmentation is unavoidable without movement. Because any allocator that cannot relocate objects provably suffers fragmentation, which grows in phases → a long-running program’s memory balloons past its live data → so the only general cure is the ability to move objects.
One indirection makes movement cheap. Because a handle routes every reference through a single master pointer → moving an object updates only that one pointer instead of hunting down every raw pointer → so relocation becomes a one-reference operation the runtime can do safely.
The compiler removes the human from the loop. Because classic handles failed by being manual and visible → having the compiler insert pinning and unpinning automatically and hide handles entirely → so programmers cannot misuse them and existing code needs no changes.
Co-design tames the indirection cost. Because a naive extra layer of indirection would be slow → optimizing the pinning in the compiler and managing handles carefully in the runtime → so the overhead stays around 10 percent despite the added lookup.
A generic mechanism unlocks whole services. Because object movement is exposed through an extensible interface → an application-independent defragmenter can be built once and applied to any program → so a capability that took thousands of lines per application becomes automatic and reusable.

What it costs (trade-offs and scope)

Indirection is not free. Every access to a handled object involves pinning to get its current address, an extra layer over a raw pointer. Careful compiler optimization and runtime design hold the geometric-mean overhead to about 10 percent, but it is a real cost, and some workloads will see more.
It requires recompiling with the Alaska compiler. The transparency comes from compiler transformation, so a program must be built with Alaska to gain handles. It is a drop-in replacement for malloc at the toolchain level, not a library that any binary can load.
Pinned objects cannot move. An object is movable only while unpinned, so code that holds objects pinned for long stretches limits how much defragmentation can do. The compiler aims to keep pin windows short, but the pattern of pinning affects the benefit.
The benefit depends on how fragmented a workload is. Defragmentation helps most for long-running, fragmentation-prone applications like databases and servers. A program that is not fragmented has little to reclaim, so the memory savings vary widely by workload.
Interactions with low-level pointer tricks. Unmanaged code can do arbitrary things with pointers; making handles fully transparent under every custom pointer manipulation is the hard part the compiler analysis must handle, and unusual patterns can stress it.
Anchorage is one demonstration. Defragmentation is the showcase service, but the broader promise — using handles to bring other managed-runtime features to unmanaged code — is opened rather than fully explored here.

The evaluation

Three ways to deal with heap fragmentation in C and C++

Property	Ignore it	Per-application defrag	Alaska and Anchorage
Can move heap objects
Works on unmodified applications	Not applicable
Reusable across applications	Not applicable
Memory reduction
Cost	Restarts, out-of-memory, cloud bills	Large engineering effort	About 10 percent runtime overhead

Three ways to deal with heap fragmentation in unmanaged C and C++, as framed by the paper. Ignoring fragmentation lets memory balloon, forcing restarts, out-of-memory kills, or larger cloud bills. A per-application defragmenter (like Redis's activedefrag) works but takes thousands of lines tailored to one program's data structures and pointer semantics, and does not transfer. Alaska brings general, transparent object movement to unmanaged languages by having its compiler automatically rewrite unmodified C/C++ code to use handles — a single indirection through a master pointer — so an object can be relocated by updating one reference, with careful compiler-runtime co-design holding the overhead to a geometric mean of about 10 percent. On top, Anchorage defragments any unmodified application, reducing Redis memory usage by up to 40 percent, on par with the bespoke activedefrag but reusable everywhere. Alaska is a drop-in replacement for malloc and requires no code changes; the overhead is measured across well-established C and C++ benchmarks and applications.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

Alaska is a clever revival of an old idea, made practical by modern compiler technology. The problem it solves is real and stubborn: unmanaged languages cannot move heap objects, so fragmentation is unavoidable, and the only fixes were to tolerate the waste or hand-build a defragmenter for each application — Redis’s took thousands of lines and does not transfer. Managed languages have enjoyed transparent object relocation and compacting collection for decades; Alaska brings that capability to C and C++ through handles, a single indirection that lets an object move by updating one master pointer. The two things that make it work are what earlier handle systems lacked. First, the compiler inserts and optimizes all the pinning automatically and hides handles completely, so programmers write ordinary pointer code, cannot misuse handles because they never see them, and existing programs need no changes. Second, careful compiler-runtime co-design keeps the overhead to a geometric mean of about 10 percent — genuinely surprising for a scheme that adds a layer of indirection to every object access. The payoff is demonstrated concretely: Anchorage, a defragmenter built on Alaska, cuts Redis’s memory use by up to 40 percent, matching the hand-tuned activedefrag while being generic. Read the scope precisely. The indirection is not free, and some workloads will pay more than the average; a program must be recompiled with Alaska to benefit; objects can only move while unpinned, so pinning patterns bound the gains; and the benefit is largest for fragmentation-prone, long-running applications. Defragmentation is one showcase of a more general capability the paper opens rather than exhausts. The durable idea is that a single well-placed indirection, made invisible and safe by the compiler, can retrofit a whole class of managed-runtime abilities onto the unmanaged code that still runs much of the world.

Background

Why a well-placed indirection keeps unlocking capabilities

It helps to see Alaska within a long-standing truth of computer systems, sometimes stated as the aphorism that every problem can be solved by another level of indirection. A raw pointer is the most direct possible reference, and its directness is exactly what forbids moving the thing it points to; interposing one movable indirection between the reference and the object restores the freedom to relocate, and with it a cascade of abilities — here, defragmentation, and the paper hints at more. What makes this instance work is that the indirection is made transparent and safe by the compiler, so the classic downsides of handles vanish. The binary-lifting work covered elsewhere on this site reflects a kindred ambition in the same unmanaged territory, recovering the higher-level structure that raw machine code discarded so that capabilities taken for granted in more structured settings can be brought to bear on low-level code. The common thread is that the rawest, most direct representations — bare pointers, bare machine instructions — are fast and flexible but strip away the structure that useful runtime services depend on, and that carefully reintroducing just enough structure, without burdening the programmer, can hand unmanaged code the conveniences of managed environments. As memory costs and long-running services make fragmentation and its cousins ever more expensive, mechanisms that retrofit managed-style capabilities onto C and C++ — transparently, at acceptable cost — are likely to matter more, because so much critical software will remain unmanaged for the foreseeable future.

If you remember only three things

1. In unmanaged languages like C and C++, a heap object cannot be moved after it is allocated, because raw pointers hold its address directly and can be copied and scattered anywhere, so there is no way to find and update every reference. The costly consequence is memory fragmentation: as a long-running program allocates and frees, the heap fills with gaps between live objects and its memory footprint balloons far beyond its live data — proven unavoidable for any allocator that cannot relocate objects. Programmers can only ignore it (facing restarts, out-of-memory kills, and cloud bills) or hand-build a per-application defragmenter, as Redis did with activedefrag, which took thousands of lines specific to Redis and does not transfer. Managed languages like Java and C# avoid all this with compacting garbage collection that freely moves objects.
2. Alaska brings general heap-object movement to unmanaged languages using handles — a single level of indirection where each object is referenced through one movable master pointer, so relocating it means updating just that one pointer; a handle is pinned to get the object's current address and unpinned to let it move. Classic handles failed because they were manual and easily misused, so Alaska has its compiler automatically and transparently rewrite unmodified C/C++ code to insert pinning and unpinning, hiding handles entirely — programmers write ordinary pointer code and cannot get handles wrong because they never see them, and existing programs need no changes. Compiler-runtime co-design, including optimizing the pinning and managing handles carefully, holds the overhead to a geometric mean of about 10 percent despite the added indirection.
3. Alaska is a drop-in replacement for malloc, and its runtime exposes object movement through an extensible interface. On top of it, Anchorage is a defragmenter that dynamically compacts the heap of any unmodified application, reducing Redis memory usage by up to 40 percent — on par with the bespoke activedefrag but reusable everywhere. Scope: the indirection costs about 10 percent on average (more for some workloads), programs must be recompiled with Alaska, objects move only while unpinned so pinning patterns bound the benefit, and gains are largest for fragmentation-prone long-running applications; defragmentation is one showcase of a broader capability. The durable lesson is that a single well-placed indirection, made invisible and safe by the compiler, can retrofit managed-runtime abilities onto the vast body of unmanaged code that still runs much of the world.

Check yourself

Check your understanding

1. Why can C and C++ programs not move heap objects, and what problem does that cause?
2. What is a handle, and how does it enable object movement?
3. How does Alaska avoid the pitfalls that made classic handles unusable?
4. What does Anchorage demonstrate, and what does its result show?

Sources

The paper is “Getting a Handle on Unmanaged Memory” by Nick Wanninger, Tommy McMichen, Simone Campanoni, and Peter A. Dinda, published at ASPLOS 2024, DOI 10.1145/3620666.3651326. The authors are at Northwestern University.

This explainer is based on the full text of the paper, obtained from the authors’ arXiv version (arXiv:2405.00038). The problem framing (the inability of unmanaged languages to move heap objects because raw pointers hold addresses directly and are scattered, the resulting memory fragmentation and the proof that any non-relocating allocator suffers it, the poor options of ignoring fragmentation or building a per-application defragmenter such as Redis’s activedefrag, and the contrast with compacting garbage collection in managed languages like Java and C#), the handle concept (the single level of indirection through a master pointer, pinning and unpinning, its origin in early personal-computer operating systems without hardware virtual memory, and the reasons classic manual handles failed), the Alaska design (a compiler and runtime acting as a drop-in replacement for malloc, transforming unmodified pointer-based code to use handles transparently with optimizations, co-designed with a runtime exposing an extensible service interface, at a geometric-mean overhead of about 10 percent), and the results (the 10 percent overhead across well-established C and C++ benchmarks and applications with zero code modifications, and Anchorage reducing Redis memory usage by up to 40 percent, on par with activedefrag) are all drawn directly from the paper. Alaska and its test suite are publicly available. Background explanations of pointers, the heap, fragmentation, indirection, and compacting garbage collection are standard and provided to make the paper’s contribution understandable.