VClinic: A Portable and Efficient Framework for Fine-Grained Value Profilers

Some program waste is invisible to ordinary profilers because the hardware did everything correctly and quickly — loading a value that was already loaded, storing one nothing will read, computing over arrays that are mostly zeros — and finding it means watching the actual values a compiled binary moves around; the tools that do this are individually hand-written in architecture-specific assembly, which makes them fast and unportable, so VClinic (Beihang University) factors out the machinery underneath them and rebuilds four such tools on top, in roughly a tenth of the value-handling code and running on two processor families instead of one.

Why this problem is worth solving

The technique is powerful precisely because it operates on the shipped artefact.

The thing being hunted does not look like a bug or a slow loop.

What people did before, and where it broke

• Use a conventional sampling profiler. Read the hardware counters, find the hot functions, optimise those. Where it broke. Prior work has established that these inefficiencies cannot be identified this way. A redundant load is a cache hit; a dead store retires normally. The counters report healthy execution.
• Instrument the source with a compiler pass. Add instrumentation during compilation, which is straightforward and fast. Where it broke. The analysis scope is limited, often to within a function; static analysis of pointers and aliases is imprecise; external libraries loaded at runtime cannot be seen at all; and there is no guarantee the instrumented code matches the generated binary, so some paths get missed. It also cannot find inefficiencies the compiler introduced, which other work has shown to exist.
• Write a binary instrumentation tool using safe framework calls. Portable across whatever the framework supports, and correct by construction. Where it broke. Every observation requires saving and restoring general registers, vector registers, control state and framework structures. Existing tools compound this by doing the detection inside the same call, so the expensive machinery runs on every value.
• Write it with hand-inserted assembly instead. Much faster, and what the serious tools do. Where it broke. It requires expert knowledge of each target. The paper’s example is precise: on one architecture an add instruction disturbs the arithmetic flags so they must be saved and restored, unless you use an address-computation instruction instead; on the other, add leaves the flags alone and no saving is needed at all. Getting this wrong is silent and easy.
• Build higher-level abstractions for instrumentation tools. Several frameworks and domain-specific languages exist for writing binary analysis tools portably. Where it broke. They abstract at the level of instructions and basic blocks. A value profiler wants a particular operand’s runtime value, its address, its element size and its calling context — so the developer still writes the extraction code, which is the architecture-specific part.
• The gap. Every tool in this family reimplements the same value-capturing machinery, in assembly, for one architecture, tangled together with its own analysis. Nobody had asked what that machinery has in common, or whether the coupling between capturing and analysing was necessary in the first place.

How a value profiler gets built

The idea

Relaxing a requirement nobody actually needed is what unlocks the performance. The existing tools do their detection inside the same safe call that captures the value, and that call is expensive because it must leave the program’s state exactly as it found it. The observation here is that the coupling is gratuitous: whether a load was redundant, or a region of data is mostly zeros, can be decided long after the value was seen — and for several of these analyses the order values arrive in does not matter either, because the finding depends only on the value. Dropping the ordering guarantee is what lets the framework keep a separate buffer per data type, which in turn is why two of the ten recorded attributes never have to be written at runtime. A strict-ordering flag remains for the analyses that genuinely need it, and the paper measures the cost of turning it on. Identifying a fidelity requirement that everyone assumed was part of the problem and discovering the workload never needed it is the same move the graphics stack recording work covered elsewhere on this site makes when it stops trying to reproduce nondeterminism and eliminates it instead.

The abstraction was at the wrong level, and that is why the same code kept being written. Existing frameworks let you instrument an instruction. But nobody analysing value inefficiencies wants an instruction; they want a particular operand’s runtime value, its address, its element size, its calling context. Between those two things sits extraction code that is architecture-specific, and so every tool writes it again. Shifting the interface to operands — describe which ones you care about with a mask, receive a uniform record for each — moves that code into the framework once. The measured effect is that the value-handling portion of four rebuilt tools shrinks by between 67 and 87 percent, and the same source now runs on both architectures. Several groups independently rebuilding identical low-level machinery because no shared layer existed at the right altitude is exactly the situation the shared stencil compilation stack covered elsewhere on this site was built to end.

Almost everything expensive can be moved out of the running program. The efficiency comes from a consistent discipline of deciding things during rewriting rather than during execution. Four of the ten attributes are statically determinable and get packed into one integer that is written as a constant. The number of records a basic block will emit is countable in advance, so one bounds check at the block entry replaces a check per value. Operands the tool filtered out are never instrumented, so they cost nothing rather than being captured and discarded. And the per-data-type buffers mean the size attributes are encoded in the destination rather than the payload. What remains inside the program is a handful of stores and a pointer increment. Doing the analysis offline and leaving the runtime with the smallest possible marker is the same division of labour that critical-slice prefetching covered elsewhere on this site arranges by reducing its runtime cost to a single instruction prefix.

Following one operand through the machinery

Operands on the legacy floating-point stack are not supported, on the grounds that modern compilers do not generate those instructions. That is a real coverage gap, stated plainly.

Why it works

• Batching converts a per-value cost into a per-buffer cost. The state-preserving call is what makes portable instrumentation slow → but it is only strictly needed when handing control to arbitrary tool code → so writing values into a buffer and calling the tool once per full buffer amortises that cost across thousands of observations.
• Relaxing ordering removes two attributes from the runtime path entirely. Access size and element size have to be recorded if all values share one buffer → but if each data type gets its own buffer, the destination already encodes them → which is available only because most of these analyses do not depend on the order values arrived in.
• A filter evaluated during rewriting is free at runtime. Deciding whether an operand is interesting requires attributes that are all statically known → so the decision can be made while the code is being rewritten rather than while it runs → and uninteresting operands are never instrumented, rather than being captured and thrown away.
• Counting records per block replaces a bounds check per value. How many entries a basic block will produce is fixed once the block is instrumented → so one check at the entry guarantees safety for every trace inside it → letting the per-operand code be a store and an increment with no branch.
• Hiding the architecture removes a class of silent bugs, not merely effort. Knowing which instructions disturb which processor state is expert knowledge that differs per target, and the value-handling code was exactly the architecture-specific part → so hand-written instrumentation gets it wrong in ways that fail quietly, and every tool has to be rewritten to move → moving that code into a framework with two backends leaves tool code that is analysis only, which is how four single-architecture tools came to run on both, and how the rebuilt zero-detector found a redundancy the published hand-written tool had been silently missing.

What it costs (trade-offs and scope)

• The overheads are very large in absolute terms. Simple configurations run 12 to 15 times slower than native; capturing both memory and register operands runs 60 to 112 times slower; the four real tools average between 40 and 191 times. The paper’s defence is that scientific applications run for weeks so this is tolerable, which is fair, but it makes this firmly an offline analysis tool.
• The comparison baseline is a non-default branch. The published tools being compared against are built on an instrumentation framework version that does not support the evaluation machine’s kernel, so a different branch on a newer version was used instead.
• Portability means two processor families. Both are built on one existing instrumentation framework. Extending to graphics processors is described as needing non-trivial engineering and left as future work, so the generality of the design is argued rather than demonstrated.
• Some instructions are skipped, silently from the tool’s perspective. Prefetch instructions are ignored when values are requested because their address may be invalid. Exclusive load and store instructions are skipped entirely on one architecture, because instrumenting them can trigger a state transition that livelocks a lock-free atomic operation. Legacy floating-point stack operands are unsupported.
• Data type detection is a heuristic and is labelled as such. An operand is assumed to hold a floating-point value if a floating-point instruction handles it; on one architecture memory operand types are inferred from the register at the other end. The attribute table calls it the estimated data type.
• Relaxing ordering is not always faster. For programs whose hot basic blocks touch many data types, having a separate buffer per type means checking several buffers for overflow, and the paper reports this making the relaxed configuration slower than the strict one.
• Productivity is measured in lines of code, by the framework’s authors. The reductions are large and the argument is plausible, but lines of code is a weak proxy and the comparison tools were written by other people to different conventions.
• The framework provides no thread safety to its clients. Per-thread buffers mean the framework itself needs no locks, but tool callbacks can fire on many threads at once and handling that is explicitly the tool author’s problem.
• Results do not port even though tools do. On the dead-store case, a compression benchmark shows far less redundancy on the second architecture and the corresponding optimisation makes it slower, reported as not applicable. A zero-redundancy finding in one numerical benchmark similarly fails to reproduce. A portable tool does not produce portable conclusions.
• Representative inputs are required, as for every tool of this kind, and the guidance is only as good as the input chosen.

The evaluation

Check yourself

Sources

The paper is “VClinic: A Portable and Efficient Framework for Fine-Grained Value Profilers” by Xin You, Hailong Yang, Kelun Lei, Zhongzhi Luan and Depei Qian, published at ASPLOS 2023, pages 892-904, DOI 10.1145/3575693.3576934. The framework is open source and the paper includes a data availability statement with container images for artifact evaluation on both architectures.

This explainer is written from the full 13-page text. Everything above comes from the paper’s own sections: the survey of compiler-based and binary-instrumentation approaches to value profiling and the abstractions built for tool development; the three observations and corresponding design principles covering the tight coupling of collection and detection, the engineering burden of architecture-specific instrumentation with its worked example of arithmetic flag handling across the two targets, and the mismatch between instruction-centric frameworks and operand-centric analysis; the two-level design of the low-level tracing component and the high-level profiling component, with the table of ten operand attributes and the note that four of them are resolved during instrumentation and packed into a single integer; the implementation across initialisation, instrumentation, runtime execution and finalisation, including per-data-type trace allocation, the default buffer size and page alignment, static operand attribute detection with its per-architecture data type inference and element size lookup, trace buffer management with the block-entry overflow check and its differing instruction sequences on the two architectures, lightweight operand tracing, the handling of gather and scatter, prefetch and exclusive load and store instructions, per-thread buffers for parallel programs, and the programming interface; the evaluation covering the hardware and software configuration, the productivity comparison with line counts and code efficiency ratios, the runtime and memory overhead tables across eight configurations on both architectures, the scalability study, and the four case studies with their configuration listings, the discovered detection gap in a published tool, the table of real-world speedups, and the two cases that did not transfer between architectures; and the discussion of extending to other platforms and the requirement for representative inputs.