Where Did My Variable Go? Poking Holes in Incomplete Debug Information

Cristian Assaiante, Daniele Cono D'Elia, Giuseppe Antonio Di Luna and Leonardo Querzoni

When you debug optimized code and the debugger says a variable is optimized out, you have no way of knowing whether that is honest — the optimization genuinely destroyed it — or whether the compiler simply forgot to record where it went; and unlike wrong debug information, there is no reference build you can compare against, because the unoptimized program shows everything. This paper (Sapienza University of Rome) works around the missing oracle by writing down rules a compiler has no excuse to break, and finds 24 confirmed bugs across two compilers and their debuggers.

The argument in one glance

Problem

The program you have to debug is the optimized one

Optimizations preserve what a program computes while rearranging almost everything about how, which makes reporting its state in terms of the original source genuinely hard — a conflict identified four decades ago and still unresolved. Debugging the unoptimized build instead is often not an option: a crash dump comes from the deployed binary, some bugs only appear once optimizations are on, and some programs cannot afford to run unoptimized at all. So compilers carry debug information through every pass, and every pass is a chance to lose or corrupt it.
What people did

Test it by comparing against the unoptimized build

Recent work found a clean way to catch debug information that is wrong. Compile the same program without optimization, where the mapping from source to machine code is straightforward and well tested, then run both builds and check that the optimized one reports the same variable values at the same points. Any disagreement is a defect, and this has worked well, exposing real bugs in production compilers. It is the natural differential test, and it needs no model of what the compiler should have done.
Why it fell short

For missing information the reference tells you nothing

A variable that the debugger reports as optimized out, or does not report at all, might be a defect — or it might be the honest consequence of a transformation that genuinely destroyed the value. The unoptimized build cannot distinguish these, because by construction it shows everything. Compensating for optimization well enough to know the truth would need logging machinery at runtime or reversal machinery in the compiler, both of which are prohibitive. So there is no reference, and identifying these defects has depended on a compiler engineer happening to think of them.
This paper's idea

Write down rules the compiler has no excuse to break

This paper (Sapienza University of Rome — Assaiante, D Elia, Di Luna and Querzoni) accepts that no oracle exists and substitutes something weaker on purpose: empirically derived conjectures about when debug information ought to be available, each carved so that a legitimate optimization could not produce a violation. If a variable is passed to a function the compiler knows nothing about, its value must exist at that call. A variable visibility should only degrade over its lifetime, never improve. Violations are then candidate bugs, and 24 were confirmed.

Why this problem is worth solving

The distinction the paper introduces is between two failures that look identical to a user and are completely different to a compiler.

Debug information tells a debugger how machine code relates to source: which instruction belongs to which line, and where the current value of each variable lives. It can fail in two ways. It can be INCORRECT — reporting a value that is not what the source-level program would have — and that is testable, because compiling the same program without optimization produces a trustworthy reference to compare against. Or it can be INCOMPLETE — a variable the debugger cannot show at all, either absent from the frame or displayed as optimized out. That second failure has no reference. The unoptimized build shows every variable by construction, so a discrepancy proves nothing: the optimization may genuinely have destroyed the value, which is an unavoidable and accepted cost, or the compiler may have failed to record something it still knows, which is a bug. Nothing in the artefacts distinguishes them

Rather than find an oracle, the paper substitutes a weaker thing and is explicit about the trade.

A CONJECTURE is a claim about when debug information ought to be available, derived from reasoning about what compilers can and cannot legitimately do, in the spirit of likely invariants inferred from observed behaviour. It is not sound and the authors do not claim it is — its justification is theoretical argument from compiler construction, statistical observation, and ultimately whether compiler developers agree that violations are bugs. What makes one usable for systematic testing is three properties. It must be checkable in terms of SOURCE-LANGUAGE constructs, so an ordinary debugger can verify it. It must rest on general compiler-construction ideas rather than on one compiler internals, so it transfers. And it must not assume a particular sequence of optimizations, so it does not overfit to a pipeline. Those constraints are what let three conjectures find bugs in two independent compilers and two debuggers

What people did before, and where it broke

Compile without optimization and compare values. The established technique for finding wrong debug information, and it works.
But it cannot see missing information. The unoptimized build shows every variable, so its disagreement with an optimized build carries no signal about whether a defect occurred.
Make the compiler compensate for its own optimizations. In principle a compiler could record enough to reconstruct the original program state.
But it would cost runtime logging or reversal machinery. Both were judged prohibitive decades ago and modern compilers do not attempt them, which is why some loss is genuinely unavoidable.
Rely on compiler engineers noticing. Someone implementing an optimization thinks about where debug information might be lost and handles it.
But that is proactive awareness, not testing. It scales with attention rather than with compute, and the paper documents defects that survived for years.
Measure how much information each optimization level retains. Count steppable lines and visible variables against the unoptimized build.
But the metric conflates the two failures. It shows gaps between levels and progress across releases without ever saying which losses were avoidable.
The gap. Every method either required a reference that does not exist for this problem, or produced numbers that could not be turned into a bug report.

How you test a property with no reference answer

Substituting conjectures for an oracle, and turning violations into filed bugs

Differentially test against an unoptimized build

Compile the same program with and without optimization, run both under a debugger, and flag any variable whose reported value differs. This is the established approach for finding incorrect debug information and it is effective, but it is the method whose reference this problem does not have.
↓ →
Establish that completeness has no reference at all new

Show that an absent variable can equally be an unavoidable consequence of optimization or a compiler defect, and that nothing in the two builds separates them. Quantify the situation across a decade of compiler releases with two metrics, and note explicitly that those metrics cannot attribute a loss to either cause.
↓ →
State conditions under which availability is predictable new

Find situations where the presence of debug information for one thing can be inferred from the visibility of something else that depends on it. Three are proposed, concerning the arguments of a call, the constituents of an assignment, and how a variable visibility evolves through its own lifetime.
↓ →
Carve each conjecture so no legitimate optimization violates it new

Restrict each claim until the compiler has no defence. Only calls to functions the optimizer cannot see inside. Only assignments to global storage through expressions that cannot be simplified. Only constituents that are constants or that the optimizer provably must leave alone. Each restriction closes off a false positive.
↓ →
Generate programs, check in a real debugger, blame an optimization new

Fuzz C programs, run them under the debugger that ships with each compiler, and record which variables are visible at each line. When a conjecture is violated, find the responsible transformation — by bisecting the optimizer where that is supported, and otherwise by disabling each optimization flag in turn.
↓ →
Shrink the program while keeping the same culprit new

Reduce a 400-line generated program to something a developer will read, but check at every step that the violation is still caused by the same optimization rather than a different one that the reduction happened to expose. Otherwise the grouping breaks and one dominant bug masks the others.

Applied to trunk versions of two major C compilers across six optimization levels, using 1000 fuzzer-generated programs and the debugger native to each compiler.

The idea

Give up on the oracle, on purpose, and say so. The paper is unusually clear that it is not solving the problem it identifies. For incorrect debug information there is a reference build and the test is a comparison; for incomplete debug information there cannot be one, because the honest answer to “should this variable be visible?” is sometimes no, and nothing in the compiler’s output records which kind of no it is. Rather than construct an approximate oracle and defend its accuracy, the authors substitute a different kind of claim: empirically derived conjectures about when availability ought to hold, explicitly analogous to likely invariants inferred from observed program behaviour. These are not sound, and the paper says their justification rests on three legs — theoretical argument from compiler construction, statistical observation of debugger traces, and the judgement of compiler developers on the reports. That last leg is the one that actually settles it, and 24 confirmations across two compilers and two debuggers is the evidence offered. Accepting a weaker, unsound signal because the sound one provably does not exist, and being explicit that developer agreement is what validates it, is the same honest-about-the-oracle stance covered elsewhere on this site where a testing method for a property with no natural failure mode has to manufacture its own notion of evidence.

Every restriction in a conjecture is a false positive being closed off. The craft is in how narrowly each claim is drawn, and reading the qualifications backwards shows exactly which excuse the compiler is being denied. The first conjecture concerns variables passed as arguments to functions the optimizer cannot see inside — a function in another compilation unit, or an indirect call with varying targets — because for such a call the compiler must materialise the actual value, so the value provably exists at that point and there is no legitimate reason not to record where. The second concerns assignments, but only to global storage, because that is where the write must actually become visible; only through expressions that cannot be simplified, so every constituent is genuinely needed; and only for constituents that are either constants or provably untouchable by the optimizer and used later, so the compiler cannot have reclaimed their storage. The third is the most elegant: a variable’s visibility should only ever degrade through its lifetime, never improve, because the only mechanism that removes a variable is the optimizer claiming its storage, and that is one-way. Narrowing a claim until the system under test has no defence, so that any violation is evidence rather than a candidate for argument, is the same eliminate-the-excuses discipline covered elsewhere on this site where a test is designed so that a failure cannot be explained away as intended behaviour.

The debugger-friendly optimization level is sometimes the worst one. Several results invert what a user would assume, and the paper reports them without smoothing. The third conjecture — visibility only degrades — is violated 115 times at the level explicitly designed to keep debugging usable, against 28, 2, 1, 1 and 1 at the more aggressive levels in the same compiler. One documented bug appears only at that level, with the variable’s value showing correctly at every more aggressive setting; the analysis found that the aggressive levels hoist a load out of a loop, which incidentally makes the value available early, while the friendly level does not and its recorded ranges leave the value unavailable until much later. Elsewhere, a size-reducing level indirectly preserves more steppable lines, because refusing to inline or unroll denies later optimizations the opportunity to destroy information. There is a similar shape to the fix side too: one patch to a helper shared across many transformations cut violations of one conjecture by 63.5 percent and closed almost half the gap between two optimization levels on a whole-program metric. Discovering that a setting chosen to serve a goal can serve it worse than settings that ignore it, because the mechanism is indirect, is the same the-intuitive-ordering-does-not-hold result covered elsewhere on this site where a knob turned toward an objective moves it the wrong way.

Following one missing variable to a filed bug

From a fuzzed program to a confirmed compiler defect

Following the paper pipeline, with figures from its evaluation on trunk versions of two major C compilers.

Step 1 of 5

Generate programs that will stress the optimizer

Use an established C program fuzzer, drawing each time from a different assortment of twenty configuration options so the programs exercise varied optimization behaviour. The output runs to four or five hundred lines. For the call-argument conjecture the programs are modified afterwards, linking a stub the optimizer cannot see inside and calling it with many local variables.

a general-purpose generator is chosen deliberately over conjecture-specific ones, to avoid narrowing which optimizations get exercised
programs are also checked for undefined behaviour, since a violation in an ill-defined program proves nothing

Why it works

A conjecture is checkable through an ordinary source-level debugger → no compiler internals are needed to verify one → so the same three conjectures find defects in two independent compilers and their debuggers.
Calls to functions the optimizer cannot see inside force the value to be materialised → the compiler provably still has the value at that point → so failing to record where it lives cannot be defended as an unavoidable loss.
The only mechanism that removes a variable from view is the optimizer reclaiming its storage, which is one-way → visibility that improves later must be a bookkeeping failure → giving a conjecture that needs no reference build at all.
Test programs come from a general-purpose fuzzer rather than conjecture-specific generators → the optimizer draws from its full repertoire → so violations implicate a wide range of transformations rather than a rehearsed few.
Reduction can silently substitute one cause for another → checking at each step that disabling the original culprit still suppresses the violation → keeping reports grouped correctly so a dominant bug does not mask the rest.

What it costs (trade-offs and scope)

The conjectures are not sound and cannot be. Their justification is compiler reasoning plus developer agreement, so a violation is a candidate rather than a proof, and 14 reports remained unconfirmed at the time of writing.
Blaming an optimization is expensive and sometimes fails. Twenty minutes per program on one compiler, and the flag-based method breaks down when a behaviour has no controlling flag or several must be disabled together.
Everything runs through debuggers. A line that does not execute, or a debugger that mishandles a case, distorts the result, which the authors mitigate by re-checking in a second debugger.
Volume vastly exceeds what anyone can triage. Hundreds to nearly a thousand violations per conjecture, which is why grouping by culprit optimization is load-bearing rather than a convenience.
Three conjectures on one language. All the claims concern variable values in C, and the authors present them as examples of a method rather than a complete set.

The evaluation

Two studies. A retrospective one, generating 5000 programs and measuring across a decade of releases of both compilers how many source lines remain steppable and how many variables remain viewable relative to the unoptimized build — which shows the gaps and the trends while being explicit that it cannot attribute any single loss. Then the testing itself, on 1000 programs against trunk versions at six optimization levels, with violations traced to a culprit transformation, reduced, and filed upstream. A regression study afterwards measures what a single accepted fix does to the violation counts.

Property	Differential testing against unoptimized	Retention metrics per level	Conjecture violation testing
What it can detect
Basis for calling it a bug
What settles a disputed case
Output per run
Reaches the debuggers too

Testing trunk versions of two compilers across six optimization levels on 1000 generated programs exposes about 38 implementation defects, of which 24 were confirmed by developers — 11 in one compiler, 10 in the other, and 3 in the debuggers used to check the results. A single accepted fix to a widely shared helper cut violations of one conjecture by 63.5 percent, spanning 68 distinct combinations of transformations, and closed almost half the gap between two optimization levels on a whole-program availability metric.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

The method is unsound and the paper says so plainly, which is the right posture but does shape how the results should be read. A violation is a candidate defect, not a proof of one, and what converts it is a compiler developer agreeing — so the honest score is 24 confirmed against 14 still unconfirmed at the time of writing, with response times ranging from same-day to more than two weeks. The pipeline around the conjectures is also heavy. Attributing a violation to a specific optimization takes twenty minutes per program on the compiler that cannot bisect its own optimizer, requires trying 81 to 151 disabling flags in turn, and fails outright when a behaviour has no controlling flag or when several optimizations must be off together. Volume is another real constraint: hundreds to nearly a thousand violations per conjecture per compiler, far beyond what anyone can triage, which is why grouping by culprit and preserving that culprit through reduction are load-bearing parts of the design rather than conveniences. Everything is observed through debuggers, so a line that never executes or a debugger that mishandles a case distorts the picture, mitigated but not eliminated by re-checking elsewhere. And the scope is three conjectures about variable values in C, presented as examples of an approach rather than a complete account of what could be checked.

Background

When the reference answer does not exist

This paper sits in the site’s compilers and correctness clusters and rhymes with several ideas covered elsewhere on this site. Its closest relative is the compilation consistency page, which also tests debug information and also finds a property nobody had checked — but the contrast is the interesting part, since that work has a clean differential oracle available and this one establishes that none can exist for its problem, so the two together map both halves of the same territory. Its response, manufacturing evidence for a property that produces no natural failure signal, is the same move the container isolation page makes when it runs a workload twice to create a difference where none was observable. Its technique of narrowing a claim until the system under test has no legitimate defence rhymes with the pages where a test is constructed so a failure cannot be argued away as intended. Its finding that the setting designed to help debugging is sometimes the worst for it echoes the pages where a knob turned toward a goal moves it the wrong way through an indirect mechanism. And its dependence on a fuzzer for input, with the consequent question of what the corpus can reach, sits alongside the compiler fuzzing pages.

If you remember only three things

1. Debugging optimized code depends on the compiler recording how transformed machine code maps back to source, and that bookkeeping can fail two ways. Wrong information is testable, because compiling without optimization gives a trustworthy reference to compare against, and recent work exploits exactly that. Missing information has no reference at all: the unoptimized build shows every variable by construction, so when a variable is absent from an optimized build, nothing distinguishes an unavoidable consequence of a transformation from a compiler that failed to record something it still knows. Compensating well enough to know the truth would need runtime logging or reversal machinery, both judged prohibitive decades ago.
2. The response is to give up on an oracle deliberately and substitute empirically derived conjectures — claims about when debug information ought to be available, explicitly analogous to likely invariants and explicitly unsound. What makes them work is how narrowly each is carved, since every qualification closes off an excuse. A variable passed to a function the optimizer cannot see inside must have its value materialised there, so failing to record it cannot be defended. An assignment conjecture is restricted to global storage, to unsimplifiable expressions, and to constituents that are constants or provably untouchable. And visibility should only ever degrade through a variable lifetime, because the only mechanism that removes one is the optimizer reclaiming storage, which is one-way.
3. Running this over 1000 fuzzed programs against trunk versions of two compilers at six optimization levels exposed about 38 defects, 24 confirmed — including three in the debuggers, which were under test because each compiler was checked with its own. Several results invert intuition: the debugger-friendly optimization level violates the visibility-only-degrades conjecture 115 times against 1 or 2 at aggressive levels, and one bug appears only there. The supporting machinery matters as much as the conjectures, since violations number in the hundreds and must be grouped by culprit optimization, with test-case reduction constrained to preserve that culprit rather than merely the symptom.

Check yourself

Check your understanding

1. Wrong debug information can be found by compiling the same program without optimization and comparing. Predict why that method cannot find missing debug information.
2. Each conjecture is stated with several restrictions attached. Predict what those restrictions are for.
3. One conjecture needs no reference build and no reasoning about what the compiler could have done. Predict what it says.
4. Automatic test-case reduction is standard practice for compiler bug reports, but the paper adds a constraint to it. Predict what and why.

Sources

Cristian Assaiante, Daniele Cono D’Elia, Giuseppe Antonio Di Luna and Leonardo Querzoni. “Where Did My Variable Go? Poking Holes in Incomplete Debug Information.” ASPLOS 2023, Vancouver. Sapienza University of Rome. Preprint at arXiv 2211.09568.
Full text of the paper was used for this explainer, including the completeness problem and the retrospective study of retention across compiler releases in Section 2, the rationale for conjecture-based testing and the three conjectures with their confirmed-bug examples in Section 3, the generation, checking, culprit identification and reduction pipeline in Section 4, and the violation counts, bug taxonomy, developer response and regression study in Section 5.
The authors released their pipeline publicly.