ValueExpert: exploring value patterns in GPU-accelerated applications

GPU programs waste huge amounts of memory bandwidth and compute not because of clumsy code but because of the DATA itself — arrays full of zeros that get copied around, duplicate values, numbers stored in a type twice as wide as they need, results computed over and over — and the standard profilers cannot see any of it, because they read hardware counters and look at code, not the actual values flowing through memory; this paper (Rice and NC State) builds ValueExpert, a value-centric GPU profiler that instruments execution to capture the real values, recognizes eight named "value patterns" of waste, and draws a VALUE FLOW GRAPH showing how data propagates across kernels and API calls (the cross-kernel view old tools miss) — mapping each pattern to a source line and a fix; it keeps the notoriously huge instrumentation cost tolerable (about 4x instead of 1000x) by doing the heavy reduction on the GPU itself using idle multiprocessors, so a student unfamiliar with the code, following its guidance, sped up real applications by a geometric-mean 1.58x (up to 8.28x) with under 5 lines of change each.

The argument in one glance

Problem

GPU programs waste resources through their DATA, not just their code

GPUs power deep learning and scientific computing, and making them efficient is worth a lot. But a large class of waste hides not in the shape of the code — the loops and kernels — but in the DATA VALUES that flow through it. Memory can be full of zeros that get dutifully copied back and forth. Two different arrays can hold identical values, computed twice. A number that only ever ranges over small integers can be stored in a 32-bit type, doubling or quadrupling the bandwidth it needs. A result can be recomputed when it never changed, or a value multiplied by 1.0. Each of these value-related inefficiencies quietly burns memory bandwidth and compute cycles — often the scarcest resources on a GPU — without any obvious flaw in the code itself.
What people did

Profile the GPU with the standard performance tools

The usual way to find GPU inefficiency is to run a profiler like nvprof or Nsight Compute. These read the GPU's hardware performance counters — cache miss rates, memory throughput, occupancy, instruction mixes — and attribute them to kernels and lines of code, pointing you at the hotspots that consume the most time. This is the standard, well-supported workflow, and it is good at finding code-structural problems.
Why it fell short

Counters and code metrics are blind to value waste

Hardware counters give a MACROSCOPIC, code-centric view, and value waste is invisible in it. A cache-miss counter cannot tell you that an array is entirely zeros, that a data type is twice as wide as the values it holds, or that a result is being recomputed identically. The counters describe HOW MUCH memory traffic happened, not WHAT values moved, so the specific inefficiency — and its fix — never surfaces. Worse, these tools examine each kernel or API call in ISOLATION, with no holistic view of how data flows across them, so waste that spans multiple operations (a zeroed array copied to the GPU and then never used) is missed entirely. And you cannot simply log every value: instrumenting every memory access across thousands of parallel threads can slow a program by 1000x, which is why nobody does it by default.
This paper's idea

A value-centric profiler that names patterns and stays affordable

This paper (Rice University and NC State — Zhou, Hao, Mellor-Crummey, Meng, Liu) builds ValueExpert, a profiler that looks at the VALUES rather than the counters. It instruments GPU execution (by overloading GPU APIs and using NVIDIA's Sanitizer API to hook every load and store) to capture the actual values flowing through memory, and it automatically recognizes EIGHT named value-inefficiency patterns — redundant values, duplicate values, single value or zero, frequent values, heavy (over-wide) type, structured values, and approximate values — each tied to a known optimization. It assembles a VALUE FLOW GRAPH whose vertices are GPU operations and whose edges are data dependencies, making visible how values propagate ACROSS kernels and API calls — the cross-kernel view the old tools lacked — and it maps each detected pattern back to a source line with a fix suggestion. Crucially, it makes value profiling affordable: instead of shipping every memory access to the CPU, it performs the heavy reduction ON THE GPU in parallel on otherwise-idle multiprocessors, adaptively copies only what is needed, and samples — bringing the overhead from 1000x down to about 4x. Acting on its guidance, a student unfamiliar with the applications sped them up by a geometric-mean 1.58x (up to 8.28x) with under 5 lines of change each.

Why this problem is worth solving

GPUs are the most expensive and most contended hardware in a modern datacenter, so waste on a GPU is waste at the highest price — and a whole category of it was going unmeasured. Understanding that category comes first. A

a property of the DATA a program moves and computes that indicates wasted work, independent of how the code is written. ValueExpert catalogs eight. REDUNDANT values: an operation leaves most of a data object unchanged, so the work (or the transfer) was largely wasted. DUPLICATE values: two data objects hold the same contents, so one was computed or stored needlessly. SINGLE value or SINGLE ZERO: every accessed element is identical (or all zero), so a full-size array is representing one number. FREQUENT values: a few values dominate the accesses, hinting at compressibility or poor locality. HEAVY type: the data type is wider than the values need (a 32-bit type holding only small integers), wasting bandwidth. STRUCTURED values: the value stored at an address is a simple linear function of the address, so it could be computed instead of stored. APPROXIMATE values: floating-point numbers carry more mantissa precision than the result needs. Each pattern maps to a concrete fix — skip the copy, deduplicate, narrow the type, compute instead of load — and each is invisible to a hardware-counter view because it is about WHAT the values are, not how many bytes moved

is the lens the paper introduces. The waste is real and common; the reason it stayed hidden is that nobody was looking at the values.

The abstraction that makes value waste visible ACROSS a program, not just within one kernel, is the second key idea. A

a graph that captures how values propagate through a GPU program’s memory and operations, giving the holistic, cross-kernel view that per-kernel profilers lack. Its VERTICES are GPU API invocations — allocations, memory copies, memsets, kernel launches — and its EDGES connect one operation to another when the first writes a data object that the second reads, so an edge is a data dependency. The graph is annotated: edge thickness shows how many bytes flowed, vertex size shows how often an operation ran, and colors flag patterns (green for benign, red for a redundant-values pattern). By tracing the edges you can see a value’s whole life — for instance, an array allocated, zeroed, copied to the GPU, and then read by a kernel that never needed the zeros — which appears as a thick red edge screaming that a transfer was wasted. This makes inter-kernel waste, which prior tools could not represent at all, something you can literally point at

is what turns a pile of per-instruction value data into an actionable map. The graph reifies the data flow that was previously implicit, so the tool (and the developer) can reason about it.

What people did before, and where it broke

Hardware-counter profilers (nvprof, Nsight Compute, VTune). Read cache, occupancy, and throughput counters and attribute them to code. Great for code-structural hotspots, but they see aggregate traffic, not values, so every value-level inefficiency is invisible to them.
Per-kernel analysis. These tools examine one kernel or API call at a time. That misses any waste that spans operations — a zeroed buffer copied across the bus and never used shows up nowhere, because no single kernel is at fault.
GVProf and prior value-redundancy tools. Detect some value redundancy, a step toward value-centric analysis, but only within individual kernels, with a single redundancy category, and by copying all the profiling data to the CPU — so slowly that it could not finish profiling some real applications within a day.
Naive full instrumentation. Log every value at every memory access for a complete picture. Correct but unusable: it can slow the program by 1000x, and the flood of data overwhelms the limited GPU profiling buffers.
The gap. No tool gave a value-centric, cross-kernel view, with named patterns and source-level fixes, at an overhead low enough to actually run on real applications. That is the gap ValueExpert closes.

How ValueExpert finds value waste

Capture the values, name the patterns, trace them across kernels, and stay affordable

The blind spot: counters miss value waste

Standard profilers read hardware counters and attribute time to code, so they cannot see that an array is all zeros, a type is over-wide, or a value is recomputed, and they examine kernels in isolation. The waste lives in the values and often spans multiple operations, exactly where the counter view is blind.
↓ →
Capture the actual values new

ValueExpert instruments execution to record values, not just counts. For coarse-grained patterns it overloads GPU APIs (memory copy, memset, kernel launch) to snapshot each data object's bit-wise values before and after; for fine-grained patterns it uses NVIDIA's Sanitizer API to hook every load and store, logging each instruction's program counter, address, size, and raw value.
↓ →
Recognize eight named patterns new

It classifies the captured values into eight patterns: redundant (an operation left over 33% of a snapshot unchanged), duplicate (data objects with the same hash), single value or zero, frequent values, heavy type (wider than the values need), structured (value linear in address), and approximate (excess floating-point precision). Raw bits are interpreted using type information recovered offline.
↓ →
Build the value flow graph new

It assembles a value flow graph whose vertices are GPU operations and whose edges are write-then-read data dependencies, annotated with byte volume, invocation count, and pattern color. This shows how values propagate across kernels and API calls — surfacing inter-operation waste (like a zeroed array copied and never used) that per-kernel tools cannot represent.
↓ →
Reduce on the GPU to survive the overhead new

Instead of shipping every memory access to the CPU, ValueExpert merges the accessed memory intervals in parallel ON the GPU with an order-log-N algorithm (radix sort plus prefix scan), launched on idle multiprocessors, and adaptively copies only the needed regions. With kernel filtering and block sampling, this cuts overhead from 1000x to about 4x.
↓ →
Map to source and guide the fix new

The offline analyzer recovers source lines and call contexts from the binary's debug information and attaches each pattern to its origin. A GUI lets the developer explore how a data object's values change along a path and points them at the thick red edges first — turning raw value data into a named diagnosis with a concrete optimization.

Built on binary instrumentation (NVIDIA Sanitizer API) with an offline analyzer, evaluated on 10 Rodinia benchmarks and 9 real applications (PyTorch, Darknet, LAMMPS, Castro, NAMD, QMCPACK) on NVIDIA RTX 2080 Ti and A100 GPUs, against the prior value-redundancy profiler GVProf.

The idea

Look at the values, not the counters — a new lens reveals waste the old one cannot. The founding move is to change WHAT is measured. Hardware counters answer “how much memory traffic, how many misses,” which is a proxy for efficiency that is blind to the values; ValueExpert measures the values themselves, so it can say “this array is all zeros” or “this type is twice as wide as it needs to be” — statements a counter can never make. This measure-the-thing-that-actually-reveals-the-waste discipline is one the site keeps rewarding: the I/O-scheduler page insists on true device occupancy rather than a queue-length proxy, and the Go-concurrency fuzzer guides its search by channel state rather than generic branch coverage, because the right signal is the one that lights up when the real problem is near. And by building the value flow graph, ValueExpert reifies the data-propagation that was implicit — the same make-the-hidden-flow-explicit move the memory-timing-defense page makes with its request graph and the in-memory-fusion page makes with its tensor dataflow graph — so a previously invisible, cross-kernel inefficiency becomes something you can point at.

Reduce the data where it is produced — on the GPU, using idle multiprocessors. Value profiling drowns in its own data: thousands of threads each generating memory accesses, far too much to ship to the CPU. The naive approach copies it all and merges on the host, which is exactly why the prior tool could not finish real applications in a day. ValueExpert instead performs the heavy reduction — merging tens of millions of memory-access intervals — ON the GPU in parallel, and schedules that work onto multiprocessors the application leaves idle, so only the compact summary crosses the bus. This compute-near-the-data, reduce-before-you-move discipline is the same one the in-memory-computing and coherent-NIC pages build on to avoid moving data needlessly, and filling the idle multiprocessors echoes the attention-overlap page’s use of spare SM capacity. Moving the reduction to the data, not the data to the reduction, is what turns an unusable 1000x slowdown into a tolerable 4x.

Make the measurement cheap enough to actually use. An observability tool that costs 1000x is a research curiosity; one that costs 4x is a tool a developer will run. Beyond the on-GPU reduction, ValueExpert adaptively chooses how to copy value snapshots (only the touched addresses when accesses are sparse, a min-to-max range when dense), lets the user profile only hot kernels, and samples blocks — each a deliberate trade of a little completeness for a large drop in overhead. This make-monitoring-cheap-enough-to-be-worth-it discipline is the same one the site’s adaptive-instrumentation and cheap-simulation pages follow, and the same observer-effect awareness that a measurement must not cost more than what it saves. The whole design is organized around the truth that a profiler is only useful if people can afford to run it.

Seeing the waste in the data

From invisible value waste to a five-line fix

Finding value-related inefficiency in GPU applications. Numbers from the paper's evaluation on RTX 2080 Ti and A100.

Step 1 of 5

Why counters miss it

A cache-miss count or occupancy number cannot reveal that a copied array is all zeros, that a 32-bit type holds only small integers, or that a result never changed. Counters measure how much traffic happened, not what the values were, and they look at each kernel alone — so cross-kernel value waste is invisible.

the standard profilers are code-centric and macroscopic; the waste is value-centric and often spans operations
logging every value naively would slow the program by 1000x, so it is never done by default

Why it works

Value waste (zeros, duplicates, over-wide types, recomputation) is about what the values ARE → hardware counters measure only aggregate traffic → a value-centric lens surfaces inefficiencies the counter view is blind to.
Much value waste spans multiple kernels and API calls → a value flow graph of operations and data dependencies makes cross-operation propagation explicit → inter-kernel waste (a zeroed array copied and never used) becomes visible as a thick red edge.
Naive value instrumentation ships too much data to the CPU and costs 1000x → merging the memory-access intervals in parallel on idle GPU multiprocessors sends only the summary → the overhead drops to about 4x, low enough to run on real applications.
Each detected pattern corresponds to a known optimization → mapping patterns to source lines with fix suggestions → a developer unfamiliar with the code can act, getting 1.58x geometric-mean speedup with under 5 lines each.
The prior tool copied everything to the host and analyzed per-kernel → it could not finish Castro or NAMD in a day → ValueExpert’s on-GPU reduction finishes them in about 5 minutes.

What it costs (trade-offs and scope)

It is a dynamic analysis. ValueExpert only sees the value patterns that the given INPUT triggers; a different input could expose or hide different patterns, and it cannot reason about all possible executions the way a static analysis might.
It is a profiler, not an auto-fixer. It names the pattern and points to the source, but a human must judge whether the pattern is genuinely wasteful (some zeros or duplicates are semantically required) and apply the fix; the reported speedups reflect a trained user’s judgment.
About 4x overhead is affordable but not free. The profiled run is several times slower, so this is an offline analysis tool, not always-on production monitoring, and worst cases (millions of distinct addresses) cost more.
It is NVIDIA-specific. The implementation relies on NVIDIA’s Sanitizer API and binary instrumentation on Maxwell and later; the methodology generalizes to other vendors, but the tool as built does not.
Fixed catalog and weaker mapping for framework code. The eight patterns are a fixed set, so a novel form of value waste outside the catalog is missed, and source mapping is coarser for interpreted or framework code (Python/PyTorch), landing on the operator rather than the precise line.

The evaluation

ValueExpert applied to 10 Rodinia benchmarks and 9 real applications (PyTorch, Darknet, LAMMPS, Castro, NAMD, QMCPACK, BarraCUDA) on NVIDIA RTX 2080 Ti and A100 GPUs, with a student unfamiliar with the applications applying its suggested optimizations, measuring speedup, profiling overhead, and runtime against the prior value profiler GVProf.

Finding inefficiency in a GPU program

Property	Counter profilers (nvprof, Nsight)	Prior value profiler (GVProf)	ValueExpert (this work)
What it sees
Cross-kernel view
Where analysis runs
Overhead and runtime
Output

Counter profilers are cheap but blind to values; the prior value profiler saw one redundancy category per kernel and was too slow to finish real applications. ValueExpert gives a value-centric, cross-kernel view with eight named patterns and source-level fixes, at about 4x overhead — finishing in 5 minutes what took the prior tool over a day, and yielding a 1.58x geometric-mean speedup with under 5 lines of change each.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

Read the scope precisely. ValueExpert is a DYNAMIC analysis, so it reports only the value patterns the given input actually triggers — a different input may reveal or hide others — and it is a PROFILER, not an auto-fixer: it names patterns and points to source, but a human must decide whether a pattern is genuinely wasteful (some zeros and duplicates are semantically necessary) and apply the fix, so the headline speedups reflect a trained user’s judgment. Its roughly 4x overhead is a huge improvement over the 1000x of naive instrumentation and the day-long runs of the prior tool, but the profiled run is still several times slower, making this an offline tool rather than always-on monitoring, with worst cases (millions of distinct addresses) higher. The implementation is NVIDIA-specific (built on the Sanitizer API for Maxwell and later), though the methodology generalizes, and its eight patterns are a fixed catalog, so a novel value inefficiency outside it is missed, with coarser source mapping for framework code like PyTorch. Within that scope the contribution is sharp and general: efficiency problems that are invisible to the usual metric become obvious once you measure the right thing (the values, not the counters) and make that measurement affordable — by reducing the data where it is produced (on the GPU, using idle capacity) rather than shipping it all away — and by turning the raw signal into named diagnoses tied to source and a fix.

Background

Measure the values, reduce near the data, and make the tool cheap enough to run

This paper sits in the site’s GPU and program-analysis clusters and rhymes with several ideas covered elsewhere on this site. Its founding move is to change what is measured: hardware counters are a proxy blind to values, so ValueExpert measures the values themselves and surfaces waste the counter view cannot — the same measure-the-thing-that-actually-reveals-the-problem discipline the I/O-scheduler page applies with true occupancy over a queue-length proxy, and the Go-concurrency fuzzer applies with channel state over branch coverage. It reifies the implicit data flow into a value flow graph, the same make-the-hidden-structure-explicit move the memory-timing-defense page makes with a request graph and the in-memory- fusion page makes with a tensor dataflow graph, turning invisible cross-kernel waste into something you can point at. Its systems core is to reduce the profiling data where it is produced — merging tens of millions of intervals on the GPU, on idle multiprocessors, so only a summary crosses the bus — the same compute-near-the-data, reduce-before-you-move idea the in-memory-computing and coherent-NIC pages build on, and the same use-the-idle-capacity move the attention-overlap page makes with spare SMs. Around that it makes the measurement cheap enough to run (adaptive copy, kernel filtering, sampling), the same make-monitoring-cheap, respect-the- observer-effect discipline the adaptive-instrumentation and cheap-simulation pages follow. And the waste it hunts is exploitable structure in the data — zeros, duplicates, narrow ranges, linear patterns — the same there-is- structure-in-real-data-to-exploit theme behind the sparsity and value- packing pages. Together they make ValueExpert a case study in seeing what the standard tools cannot by measuring the right thing affordably.

If you remember only three things

1. GPU programs waste memory bandwidth and compute through their DATA, not just their code: memory full of zeros that gets copied, arrays holding duplicate values, numbers stored in a type far wider than their range, and results recomputed identically. Standard profilers (nvprof, Nsight) read hardware counters and code metrics, so they measure how much traffic happened, not what the values were, and examine kernels in isolation — missing value waste entirely, especially waste that spans multiple operations. And naive value instrumentation is unusable, slowing programs by 1000x.
2. ValueExpert is a value-centric profiler that captures the actual values (overloading GPU APIs and hooking every load/store), recognizes eight named value patterns (redundant, duplicate, single-zero, frequent, heavy-type, structured, approximate), and builds a VALUE FLOW GRAPH whose edges show how data propagates across kernels and API calls — surfacing cross-kernel waste as thick red edges that per-kernel tools cannot represent. It maps each pattern to a source line and a fix, and it stays affordable by reducing the profiling data on the GPU itself (parallel interval merge on idle multiprocessors), adaptively copying only what is needed, and sampling — cutting overhead from 1000x to about 4x.
3. Following its guidance, a student unfamiliar with the applications achieved a geometric-mean 1.58x speedup (up to 8.28x) with under 5 lines of change each, cut Darknet CPU-GPU traffic by 84.2%, and upstreamed verified patches, while finishing in 5 minutes what the prior tool could not finish in a day. The transferable lessons: efficiency problems invisible to the usual metric become obvious once you measure the right thing (the values, not the counters); reify the hidden data flow so cross-boundary waste is visible; reduce profiling data where it is produced rather than shipping it away; and make the measurement cheap enough that people will actually run it.

Check yourself

Check your understanding

1. Standard GPU profilers report cache misses, occupancy, and throughput, yet ValueExpert argues they miss a whole class of inefficiency. Predict what they miss and why.
2. Naive value instrumentation slows a GPU program by 1000x, yet ValueExpert profiles real applications at about 4x. Predict the key technique that makes this possible.
3. Much of the waste ValueExpert finds spans several kernels and API calls, which per-kernel profilers cannot represent. Predict how ValueExpert makes this cross-operation waste visible.
4. ValueExpert does not just report raw value statistics; it classifies findings into eight named patterns like "heavy type" and "single zero." Predict why the named catalog matters.

Sources

ValueExpert: exploring value patterns in GPU-accelerated applications (ACM DL, DOI 10.1145/3503222.3507708) — ASPLOS 2022 (full text read via the ACM PDF).
ACM Digital Library entry — published version and metadata.
GVProf / HPCToolkit project (related value profiler) — the prior tool ValueExpert is compared against.