FPGA prototyping is how chip designers test hardware before manufacturing and how early software is written, but developing on an FPGA is painful — a single edit takes hours to recompile into a bitstream, and there are no software-style debuggers, so bugs are found only after a crash and localized by guessing and recompiling for hours; this paper (UC Berkeley — Tianrui Wei, Kevin Laeufer, Katie Lim, Jerry Zhao, Koushik Sen, Jonathan Balkind, Krste Asanovic) presents Zoomie, an open-source platform that brings a software-like debugging experience to FPGAs — breakpoints, single-stepping, snapshots, and forcing values in a running design — together with fast incremental compilation that recompiles only the changed part, reaching an 18 times speedup over the vendor toolchain on million-gate designs.
The argument in one glance
-
Problem
Developing on an FPGA has none of the comforts of software development
A field-programmable gate array, or FPGA, is a chip you configure into custom hardware by loading a design called a bitstream. Chip designers use FPGAs to test a hardware design before committing it to a manufactured chip, and to write software early against that hardware; FPGAs are also used directly as datacenter accelerators. But developing on an FPGA is slow and painful. Turning a change in the hardware description into a running bitstream requires synthesis and place-and-route, which can take hours, and there is almost no support for recompiling only the part you changed. Worse, there are no software-style debugging tools: no breakpoints, no stepping through execution, no way to inspect or change the running state. Bugs are found only after the design crashes or returns a wrong answer, and finding where the bug is means guessing, adding a monitor, and recompiling for hours, over and over. This is a high barrier to the fast, iterative development that software takes for granted.
-
What people did
Use vendor logic analyzers, full recompiles, and expensive emulation systems
Developers work around this with the tools they have. To see inside a running design they insert a vendor instrument such as an Integrated Logic Analyzer, which records a few pre-chosen signals over time. To find a bug they recompile the whole design with different signals exposed and try again. For a richer experience there are commercial emulation systems, but they are costly and closed.
-
Why it fell short
These tools are slow, narrow, closed, and cannot pause or change the design
Each workaround falls short. An Integrated Logic Analyzer only lets you observe specific signals you chose before compiling, and it adds substantial hardware to the design, so if you guessed the wrong signals you must recompile — hours each time — to look somewhere else. It can never pause the design, step it, or change its state; it only watches. There are no off-the-shelf tools that offer breakpoints. Because commercial toolchains barely support incremental compilation, even a one-line change re-runs almost the entire multi-hour flow. In one real case the authors describe, localizing a single bug took several two-hour recompiles, and testing the fix took another two hours. Commercial emulation systems have some of the missing features, but they are expensive and proprietary, out of reach for most. What is missing is an open, fast, software-like way to debug an FPGA design running on real hardware.
-
This paper's idea
Bring the software debugger to the FPGA, and recompile only what changed
Zoomie is an accessible, open-source platform that gives FPGA prototyping a software-like debugging experience, implementing features previously found only in expensive commercial systems. It rests on three ideas. First, fast incremental compilation: a component the authors call the Vendor Tool Incrementalizer partitions the design and drives the vendor tools to re-synthesize and re-place only the region that changed, so an edit reaches the running FPGA in minutes instead of hours — an 18 times speedup over the vendor toolchain on million-gate designs. Second, reuse of existing verification infrastructure: the assertions engineers already write to check their design (SystemVerilog Assertions) are synthesized into hardware and turned into assertion breakpoints that fire on the FPGA. Third, a software-inspired approach to open-source emulation: by deeply investigating the Xilinx FPGA architecture and uncovering undocumented mechanisms, Zoomie can pause and resume a running design by gating its clock, single-step it a precise number of cycles with a hardware counter, take a snapshot of all the design's state using the FPGA's readback feature (and map that raw state back to the register and memory names in the hardware description), replay a snapshot, and force new values into a running design. Zoomie is also the first to support modern multi-chiplet FPGAs, handling designs more than 10 times larger than prior work.
Why this problem is worth solving
FPGA prototyping sits on the critical path of building almost every modern chip, so its slowness taxes the whole hardware industry. Before a chip is manufactured — an expensive, months-long, irreversible step — designers must be confident the hardware is correct. Software simulation of the design is far too slow to run real workloads or boot an operating system, so teams map the design onto an FPGA, which runs the actual hardware logic fast enough to test it against real inputs and outputs like memory and networking, and to let software developers start writing code before the chip exists. This makes FPGA prototyping indispensable for pre-silicon verification and early software development, and FPGAs are increasingly deployed as accelerators in their own right. The catch is the development loop. Every time a designer changes the hardware description, the change must be compiled — synthesized and placed and routed into a bitstream — and this routinely takes hours for a large design. Commercial toolchains offer little real incremental compilation, so even a tiny edit pays close to the full cost. When something goes wrong, the pain compounds, because a slow compile is now inside a debugging loop that may repeat many times. The result is that hardware teams spend enormous amounts of time waiting, which slows every project that depends on an FPGA.
The deeper problem is that hardware debugging lacks the tools that make software debugging tractable. In software, when a program misbehaves, a developer has a rich toolbox: set a breakpoint to stop at a suspect line, step through execution one operation at a time, inspect any variable, change a value and continue, or even record and replay a run. These tools let you localize a bug — pin down where and why it happens — quickly and interactively. On an FPGA almost none of this exists. The main instrument, an Integrated Logic Analyzer, is a block you add to your design that captures the waveforms of a handful of signals you must choose before you compile; it only observes, it cannot stop or alter the design, and it consumes chip resources. If the bug is not visible in the signals you picked, you change your selection and recompile — hours gone — and guess again. There is no breakpoint, no stepping, no way to read out the full state or to force a value. So bugs are typically discovered post-mortem, after a crash or a wrong result, with little clue as to their origin. Bug localization becomes a slow game of guess-compile-look, and because each compile is measured in hours, a single bug can consume days. Closing this gap — giving FPGA developers the interactive, software-like debugging they are used to — is what would finally make agile hardware development possible.
What people did before, and where it broke
- Integrated Logic Analyzers (vendor waveform capture). Insert a block that records a few pre-selected signals over time so you can inspect them after a run. Where it broke. You must choose the signals before compiling; it only observes and cannot pause, step, or change the design; it adds real hardware overhead; and if the bug is elsewhere you must recompile for hours to watch different signals.
- Full recompilation for bug localization. Change which signals are exposed, or add instrumentation, and rebuild the whole design to look again. Where it broke. Commercial toolchains barely support incremental compilation, so each rebuild costs close to the full multi-hour flow; localizing one bug can take several two-hour recompiles plus more to test a fix.
- Software simulation of the design. Run the hardware description in a simulator, where full visibility and debugging are available. Where it broke. Simulation is orders of magnitude too slow to run real workloads, boot software, or interface with real inputs and outputs, which is exactly why FPGA prototyping is used instead.
- Commercial emulation systems. Buy a proprietary emulation platform that offers some interactive debugging features. Where it broke. These systems are expensive and closed, so they are out of reach for most academic and smaller industrial teams, and their internals cannot be studied or extended.
- The gap. No open, accessible tool offered fast incremental compilation together with true software-like debugging — breakpoints, stepping, state snapshots, and forcing values — on a design running on real FPGA hardware.
How Zoomie works
Turning an FPGA into an interactively debuggable target
-
-
new
-
new
-
new
-
new
-
new
The idea
Give hardware the interactive debugging that software has, on the real device. The guiding goal of Zoomie is to let an FPGA developer do what a software developer does: stop the design at a chosen point, look at its state, step it forward, change a value, and continue — all while it runs on real hardware against real inputs, not in a slow simulator. Bringing the debugging conveniences of software to hardware running on an FPGA, so developers can introspect a real execution rather than only inspect it after a crash, is the same aspiration the work on record and replay for reconfigurable hardware covered elsewhere on this site pursues, capturing a real FPGA run so it can be examined repeatedly. Zoomie approaches the same aspiration from the live-debugging side, pausing and inspecting the design as it executes instead of replaying it afterward.
Shrink the edit-run loop by recompiling only what changed. The single biggest tax on FPGA development is compilation time, and Zoomie attacks it directly with incremental compilation: partition the design, and when an edit lands, rebuild only the partition it touched, leaving the rest untouched. This turns a multi-hour rebuild into minutes, which is what makes an interactive loop possible at all. Cutting the long compile that stands between a hardware change and a running design, so the development cycle becomes fast and iterative, is the same bottleneck the work on fast FPGA compilation to make reconfigurable acceleration practical covered elsewhere on this site sets out to remove. Zoomie’s contribution is to drive the existing vendor tools to do this incrementally and to pair it with debugging, so speed and introspection arrive together.
Reuse the verification work developers have already done. Hardware teams write large suites of assertions to check their designs during verification, and Zoomie turns those same assertions into live breakpoints on the FPGA rather than asking developers to build debugging support from scratch. Treating the assertions and checks written for verification as a resource that can be reused to reason about a design, instead of a one-time cost, is the same idea the work on scaling symbolic execution of hardware designs covered elsewhere on this site builds on, leveraging the formal description of a design to explore its behavior. Zoomie takes those verification assertions out of the simulator and compiles them into hardware that watches the running design and stops it when something goes wrong.
Following a bug hunt through Zoomie
Localizing a bug that used to take a full day of recompiles
A trace, drawn from the paper's case study, of how Zoomie turns a slow guess-compile-look loop into an interactive one.
Step 1 of 5
The design crashes, but where is the bug?
A large design (in one case study, a complete processor tile) running on the FPGA produces a wrong result. In the old flow the developer would guess a location, add a logic analyzer on some signals, and start a multi-hour recompile — repeating this several times at about two hours each.
Post-mortem debugging gives no direct clue to the bug's location, so localization is guesswork.
Why it works
- Incremental compilation removes the loop’s dominant cost. A full FPGA rebuild takes hours and sits inside every debugging iteration → but rebuilding only the changed partition takes minutes → so the edit-run loop becomes fast enough to be interactive, an 18 times speedup on million-gate designs.
- Clock gating gives true breakpoints and stepping. An FPGA normally runs freely with no way to stop it → but gating the clock-enable freezes every register in place and releasing it resumes → so the design can be halted at a condition and advanced a precise number of cycles, just like software.
- Readback plus name mapping gives full, meaningful visibility. The FPGA can dump all its state as raw bits, which are meaningless alone → but Zoomie matches those bits to the register and memory names in the hardware description → so the developer sees named values instead of a handful of pre-chosen waveforms.
- Reusing assertions turns verification into debugging. Engineers already write many assertions to check their designs → but those are normally used only in simulation → so synthesizing them into hardware breakpoints reuses that effort to catch and localize bugs on the real device.
- Being open and exploiting the real FPGA makes it accessible and complete. Commercial emulators have some features but are closed and costly → but Zoomie uncovers the undocumented FPGA mechanisms itself → so an open-source tool can offer pausing, snapshots, and forcing on real hardware, including multi-chiplet FPGAs 10 times larger than prior work.
What it costs (trade-offs and scope)
- Incremental compilation trades chip area for speed. To rebuild only a changed partition, Zoomie over-provisions each partition’s area (its resource budget is scaled up by an over-provision coefficient) so there is room for the region to change; this leaves some FPGA capacity unused in exchange for fast incremental rebuilds and stable timing.
- It depends on undocumented, vendor-specific mechanisms. Many of Zoomie’s abilities — pausing via clock control, readback, selecting logic regions on multi-chiplet parts — come from reverse-engineering opaque Xilinx FPGA details; this ties the implementation to that vendor’s architecture and to behavior that is not officially supported and could change.
- The 18 times figure is incremental compilation, not everything. The headline 18 times speedup is specifically the incremental compilation time versus the vendor toolchain on million-gate designs; it is a large and meaningful win, but it measures the rebuild step, and overall debugging productivity also depends on how a given bug is approached.
- Debugging support has hardware overhead and limits. Assertion breakpoints and the debug machinery run on the FPGA alongside the design, consuming resources, and the granularity of what can be paused, inspected, and forced is set by what the FPGA’s state elements and readback expose, not by an unlimited software-style view.
- It targets FPGA prototyping of digital designs. Zoomie is built for debugging hardware designs on FPGAs; its techniques are shaped by the FPGA configuration and readback flow, so they do not transfer directly to debugging a manufactured chip or to non-FPGA targets.
The evaluation
Debugging a hardware design on an FPGA
| Property | Integrated Logic Analyzer + full recompile | Commercial emulation system | Zoomie |
|---|---|---|---|
| Incremental compile after an edit | |||
| Breakpoints and stepping | |||
| Full, named state visibility | |||
| Force values into a running design | |||
| Open-source and scalable |
Check yourself
Check your understanding
1. Why is debugging a hardware design on an FPGA so much slower and harder than debugging software?
2. What are Zoomie's three main ideas?
3. How does Zoomie actually implement a breakpoint and stepping on an FPGA?
4. Predict: a developer using Zoomie fixes one line in a million-gate design and wants to test it. Compared with the vendor toolchain, what happens, and why?
Sources
The paper is “Zoomie: A Software-like Debugging Tool for FPGAs” by Tianrui Wei, Kevin Laeufer, Katie Lim, Jerry Zhao, Koushik Sen, Jonathan Balkind, and Krste Asanovic (University of California, Berkeley), published at ASPLOS 2024, DOI 10.1145/3620666.3651356. It is licensed CC BY-SA and won a Distinguished Artifact Award.
This explainer is written from the paper’s full text (the authors’ PDF). FPGA prototyping is indispensable for pre-silicon verification and early-stage software development, and FPGAs are also popular datacenter accelerators, but FPGA development suffers from ultra-long compilation times, little incremental compilation, and a lack of software-style debugging tools, so bugs are found post-mortem and localized by repeated multi-hour recompiles. Zoomie addresses this with fast incremental compilation (a Vendor Tool Incrementalizer that partitions the design and re-synthesizes and re-places only changed regions, over-provisioning partition area to trade area for timing), reuse of verification infrastructure (SystemVerilog Assertions synthesized into hardware assertion breakpoints running alongside the module under test), and a software-inspired approach to open-source emulation that exploits undocumented Xilinx FPGA mechanisms to pause and resume the design via clock gating, single-step it with a 64-bit hardware cycle counter, snapshot all state through the FPGA readback feature and map it to the register and memory names in the design, replay snapshots, and force values into a running design. Zoomie is the first work to support modern multi-chiplet FPGAs (multiple Super Logic Regions), with supported design sizes exceeding 10 times prior work, and it achieves an 18 times speedup over the vendor toolchain in incremental compilation time for million-gate designs. No numbers beyond the paper’s own have been introduced; exact partitioning heuristics, the reverse-engineered FPGA details, and the full evaluation should be confirmed in the paper.