Flexible Non-intrusive Dynamic Instrumentation for WebAssembly

WebAssembly is a fast, sandboxed bytecode now running everywhere from browsers to servers to tiny devices, but its engines let you see almost nothing of a program as it runs — no way to trace, profile, or debug without rewriting the program itself and risking a change to its behavior; this paper from Carnegie Mellon builds the first system, in the open-source Wizard engine, that lets an analysis attach programmable probes to any bytecode, running the analysis code inside the engine so it cannot perturb the program, with zero overhead when switched off and only a few hundred lines added to the engine by reusing machinery it already had.

Why this problem is worth solving

The thing being instrumented is fast becoming a universal substrate, which is why the gap in tooling matters.

The precise quality that makes an analysis trustworthy is subtle, and naming it clarifies the whole design.

What people did before, and where it broke

• Rewrite the bytecode before it runs (static rewriting). Inject instrumentation into the program ahead of time, as the Wasm tool Wasabi does. Where it broke. It mixes analysis into the program’s state, risking intrusiveness; the injected code is low-level and tedious; it shifts every instruction offset so source locations no longer map; it bloats the binary; and being fixed before runtime it cannot adapt or reach code discovered only at runtime.
• Recompile the machine code at runtime (dynamic binary instrumentation). Tools like Pin and DynamoRIO rebuild a native binary as it runs, splicing in analysis code. Where it broke. They require building a whole purpose-made just-in-time compiler — tens of thousands of lines — run in the program’s own process where supplying low-level analysis code can be intrusive, and offer only coarse, region-level control over turning instrumentation on and off, with no strong guarantees when adding and removing it dynamically.
• Emulate the whole machine. Valgrind and QEMU virtualize execution and add analysis in their interpreters, which is naturally side-effect free. Where it broke. Their compilers are built for cross-emulation, not instrumentation, and full emulation imposes heavy overhead; they are a poor fit for a bytecode that already has fast native engines.
• Use a managed runtime’s built-in profiling interface. The Java Virtual Machine offers JVMTI and .NET offers a profiling API, giving side-effect-free callbacks on events. Where it broke. These expose only coarse events like method entry and exit, nothing as fine-grained as reaching an individual bytecode, and even that coarse handling is slow — a method-entry monitor on a call-heavy Java benchmark ran 50 to 100 times slower.
• Sample or trace with hardware and the operating system. Linux Perf samples, debug registers and processor trace watch execution directly on the CPU. Where it broke. Sampling is imprecise and hardware tracing is limited; accurately capturing high-frequency events — every branch, call, and memory access, millions per second — needs a mechanism inside the execution engine, not beside it.
• The gap. No approach for Wasm was at once non-intrusive, fine-grained down to any bytecode, dynamic enough to insert and remove instrumentation at runtime with guarantees, and cheap enough to leave in a production engine. Static rewriting gave up non-intrusiveness and flexibility; machine-code tools and emulators cost enormous engineering and overhead; managed-runtime APIs were too coarse. What was missing was instrumentation built into the engine at the bytecode level.

How Wizard is put together

The idea

Run the analysis inside the engine, at the bytecode level, so it cannot touch the program. The root cause of intrusiveness is that injected code lives in the program’s own space, where it can collide with the program. Wizard removes that possibility by construction: an analysis is written as a monitor whose callbacks are compiled into the engine itself, and when a probe fires the engine invokes that callback from the interpreter or compiled code directly. Because the callback executes as part of the engine — which already virtualizes and owns all of the Wasm program’s state — it simply has no way to perturb program state except through a deliberately controlled reading interface. Instrumenting at the level of bytecode locations, rather than machine code or a full emulator, is what makes this both clean and cheap: a bytecode offset is a stable, meaningful reference point, and the engine’s existing interpreter and compiler already know how to execute around it. The analysis becomes a first-class, side-effect-free observer rather than a graft onto the program. Building a fine-grained observation framework as a layer that watches execution from outside without the watching becoming part of what is watched is the same posture the fine-grained value-profiling work covered elsewhere on this site takes, instrumenting a program densely enough to see individual value-level waste while keeping the profiler itself from distorting the picture it reports.

Make instrumentation invisible when off and trivial to toggle when on. For instrumentation to be usable in a production engine, it must cost nothing when nobody is looking — otherwise engines split into slow debug builds and fast production builds. Wizard achieves this with two mechanisms, one per kind of probe. A global probe, which fires on every instruction, is turned on by flipping a single register to point at an alternate dispatch table whose entries all route through the probe before continuing; when no global probe is active the register points at the normal table, so there is exactly zero overhead, and the engine can even switch into this heavyweight mode and back without discarding any compiled code. A local probe, which fires at one spot, is installed by overwriting that one bytecode with an illegal probe opcode and saving the original aside; uninstrumented instructions are untouched, insertion and removal are constant-time single-byte swaps, and the original instruction offsets are preserved so locations still map cleanly. In both cases the feature is free until used and cheap to flip. Designing a capability so that it imposes no cost until the moment it is actually needed, letting it live permanently in the fast path, is the same principle the hardware-safepoint work covered elsewhere on this site relies on, where a marker sits inert as an ordinary instruction and only becomes a delivery point when an interrupt is genuinely pending.

Do not build a new instrumentation engine — repurpose the hard machinery the engine already has. Machine-code instrumentation tools are enormous because they must build a purpose-made just-in-time compiler from scratch, tens of thousands of lines, to rewrite and manage code. Wizard’s central engineering insight is that a modern multi-tier engine already contains the one genuinely hard mechanism instrumentation needs: deoptimization, the machinery that correctly transfers a running function between optimization levels — from compiled code back to the interpreter — reconstructing its exact state. That is precisely the problem that must be solved to make a probe’s change to program state take effect consistently, or to switch a function into an instrumented mode mid-execution. So rather than reinvent it, Wizard reuses the deoptimization path, adding only a few hundred lines across the existing interpreter and compiler. The consistency guarantees that let independent analyses compose without interfering come almost for free from this reuse. Noticing that the difficult sub-problem your new feature needs has already been solved by an existing mechanism in the system, and repurposing that mechanism instead of rebuilding it, is the same move the paging-replacement work covered elsewhere on this site makes when it has the compiler and the privileged layer cooperate to retire a whole mechanism rather than each constructing its own.

Following a probe through Wizard

This is far simpler than machine-code approaches: no hardware traps, no rearranging code to make room — just one byte swapped, which is why installing and removing a probe is a constant-time operation.

Why it works

• Running the analysis in the engine makes it non-intrusive by construction. Injected code lives in the program’s state and can collide with it → but a callback compiled into the engine runs in the engine’s own state space → so it cannot alter program state except through a controlled interface, guaranteeing side-effect freedom rather than merely hoping for it.
• A dispatch-table swap turns global instrumentation on for free. Checking for a global probe at every instruction would tax even uninstrumented runs → but pointing one register at an alternate dispatch table switches all handlers at once, and the normal table is used otherwise → so global probes impose exactly zero overhead when off and need no separate debug build.
• Bytecode overwriting gives per-probe control at constant cost. Machine-code overwriting needs hardware traps and code rearrangement → but swapping a single Wasm opcode for a probe opcode installs or removes a probe in constant time with offsets preserved → so control is per-probe, not the region-level all-or-nothing of prior tools, and consistency is trivial.
• Reusing deoptimization avoids building a new engine. Machine-code tools construct a purpose-made compiler of tens of thousands of lines → but the deoptimization mechanism already solves the hard state-transfer problem instrumentation needs → so Wizard adds only a few hundred lines and gets the consistency guarantees that let analyses compose almost for free.
• Intrinsifying common probes collapses their overhead. A counter probe that saves all state and calls a generic function to do one increment is wildly wasteful → but the compiler can inline the increment directly into the code → so the overhead of counting every instruction drops from up to 134 times to a few times, even undercutting intrusive bytecode rewriting.

What it costs (trade-offs and scope)

• It is built into one specific engine. Everything lives in Wizard, a research engine, and the cheapness of the approach leans on Wizard’s existing multi-tier design and deoptimization mechanism. The claim that production engines could adopt it as easily is argued from the small code footprint, not demonstrated in a shipping engine like V8 or Wasmtime.
• It is a mechanism, not a standard. Monitors are written in the Virgil language against Wizard’s internal API; there is no standard Wasm instrumentation interface, so analyses are not portable across engines, and adopting the idea elsewhere means each engine building the same hooks.
• “Zero overhead” means when disabled, not when active. Dense instrumentation still costs real time — counting every instruction runs a few times slower even with intrinsification, and global probes effectively drop the engine back to interpreter speed while active. The achievement is a production-safe off switch, not free analysis.
• The comparisons are not head-to-head. DynamoRIO, Wasabi, and JVMTI operate at different levels and languages, and the DynamoRIO and bytecode-rewriting baselines are the authors’ own analogous reimplementations; the striking numbers, like beating intrusive rewriting or a 50-to-100-times JVMTI overhead, are informative but come from comparisons the authors constructed, some against tools used at their least favorable granularity.
• Reading full frame state has a cost. Every frame carries a one-word accessor slot, and a callback that needs program state forces the compiler to checkpoint and materialize a FrameAccessor, with extra guards to keep a buggy monitor from touching a frame that has already unwound — so the cheapest path is a probe that needs little or no state.
• It is single-threaded today. Wizard is not yet multi-threaded, and while WebAssembly is gaining threads, the consistency story for concurrent instrumentation is sketched with locks rather than built and evaluated.
• Intrinsification covers only common patterns. Counters and probes that read a little frame state are specialized into fast code, but an arbitrary analysis callback still pays the general cost of checkpointing state and calling out, so the best speedups apply to the simplest, most common building blocks.
• The evaluated analyses are deliberately simple. The hotness and branch monitors were chosen partly because they could be reimplemented in the competing systems for comparison; the overhead of richer analyses, like a full debugger or taint tracker under load, is not the focus.

The evaluation

Check yourself

Sources

The paper is “Flexible Non-intrusive Dynamic Instrumentation for WebAssembly” by Ben L. Titzer, Elizabeth Gilbert, Bradley Wei Jie Teo, Yash Anand, Kazuyuki Takayama, and Heather Miller, published at ASPLOS 2024, pages 398-415, DOI 10.1145/3620666.3651338. The authors are at Carnegie Mellon University.

This explainer is written from the full 18-page text. Everything above comes from the paper’s own sections: the introduction and its taxonomy of instrumentation (intrusive versus non-intrusive, static versus dynamic rewriting, emulation, and direct engine support) and the definitions of monitors and M-code; the design of non-intrusive instrumentation in Wizard, including global probes, local probes, the FrameAccessor API and its dangling-accessor safety, the consistency guarantees (deterministic firing order, deferred inserts and removals, frame-modification consistency), and the observation that function entry/exit and after-instruction hooks are libraries built atop probes; the Monitor Zoo of built analyses (Trace, Coverage, Loop, Hotness, Branch, Memory, the Debugger REPL, Calls, and the Call-tree profiler); the optimization section covering dispatch-table switching for global probes, bytecode overwriting for local probes, local probes in the JIT, and JIT intrinsification of counters and operand probes; and the evaluation on PolyBench, Ostrich, and Libsodium reporting the global-versus-local comparison (branch 1.0 to 2.2 times local versus 7.7 to 16.4 times global), the intrinsification gains (hotness 7 to 134 times down to 2.2 to 7.7 times; branch 1.0 to 16.6 times down to 1.0 to 2.8 times), the interpreter-versus-JIT overheads, and the comparisons against bytecode rewriting, Wasabi (29.9 to 4721.5 times), DynamoRIO (4.4 to 153 times), and a JVMTI method-entry monitor (50 to 100 times against Wizard’s 2.5 to 3 times).