SMAPPIC: Scalable Multi-FPGA Architecture Prototype Platform in the Cloud

Studying hardware that does not exist yet means building a prototype on reconfigurable chips, which costs thousands of dollars per board, demands expertise most systems researchers do not have, and has stopped working now that interesting designs are too large for one chip; this paper (Princeton) rents the chips by the hour instead, and observes that the round-trip delay between two rented boards is about 125 cycles — the same order as the delay between two sockets in a real server — so the awkward artifact of splitting a design across boards becomes an accurate model of the thing being studied, yielding a 48-core coherent RISC-V machine booting Linux in a genuine NUMA configuration.

Why this problem is worth solving

The choice of modelling tool decides which questions can be asked at all.

The second thing to understand is what a multi-die machine is and why it now dominates.

What people did before, and where it broke

• Software simulators. Flexible, familiar, instrumentable anywhere, and the default for most architecture research. Some model multiple dies with shared memory.

• The gap. Speed, and its consequence for cost. A detailed simulator runs a design thousands of times slower than hardware and demands enormous host memory — one benchmark here needed a 350 GB machine — which the paper turns into a cost comparison where simulation is four to five orders of magnitude more expensive per experiment.

• Locally owned prototype boards. The established fast option, and genuinely fast. Groups that can afford them get a credible machine to run real software on.

• The gap. The board is expensive, the expertise is specialised, and physical access is assumed. Scaling out to run many experiments at once means buying many boards, and the designs that most need prototyping no longer fit in one.

• Commercial emulation machines. Purpose-built hardware that solves the partitioning problem properly, splitting a huge design across a rack.

• The gap. Prices in the millions of dollars, and proprietary. This is a solution for large companies, not for a research group or a class.

• Existing cloud FPGA simulators. Prior work already put architecture models on rented chips and made them scale, aimed at simulating a datacenter with many machines and a network between them.

• The gap. That goal shapes the design. Emphasis on modelling the network between independent machines constrains how large a single machine can be, whereas the question here is what happens inside one large machine made of several dies sharing memory.

How a rented chip becomes a multi-die research machine

The idea

The awkward constraint turns out to be the right model. Splitting a design across several chips is normally a compromise you accept because the design does not fit, and it introduces a delay between the pieces that the real machine would not have. The observation that reorganises this paper is a measurement — a round trip between two rented chips over their shared bus takes about 1,250 nanoseconds, which at a typical prototype clock of 100 megahertz is about 125 cycles, the same order of magnitude as the delay between two sockets in a commercial server. So the artifact of the modelling technique lands where the thing being modelled already is. A prototype split across four chips is not an approximation of a single-die machine; it is a good model of a four-die machine, and the paper confirms this by measuring core-to-core latency and finding four clearly separated regions at about 100 cycles within a die and 250 across, which matches published figures for multi-socket servers. Discovering that a platform’s constraint coincides with a property of the target is a happier version of what the research-computer work covered elsewhere on this site had to do the hard way, by building a machine from scratch because no commercial platform exposed the interface it needed.

Everything else is the plumbing that makes a rented chip usable. The rented platform gives you a fixed vendor-supplied shell, four memory controllers, and a bus to the host, and it does not give you the things an operating system expects. There is no serial port, so one is emulated and tunnelled to the host as a virtual device — and then a second one is run at roughly ten times the standard rate and used with a dial-up utility to put the prototype on the Internet. There is no storage device, so writes are injected into the memory the prototype believes its card occupies. The memory protocol does not match, so a translating controller reassembles requests onto 64-byte boundaries and tracks outstanding misses. Interrupts, which the instruction set defines as a wire, become packets because a wire cannot cross a chip. None of this is deep, and all of it is the reason a researcher can use the platform in an afternoon rather than a semester. That gap between an idea being possible and a tool being usable is what the software RTL simulation work covered elsewhere on this site attacks from the other direction, by making the flexible option fast instead of making the fast option easy.

The case studies are chosen to be things other tools cannot do. Booting Linux on 48 coherent cores across four dies and measuring what the kernel’s memory-placement mode is worth — 1.6 to 2.8 times on a parallel sort, largest at high thread counts — is not available on any simulator at usable speed. The most instructive study is smaller. An accelerator from prior work was re-evaluated here, and the system kept hanging; hours of debugging traced it to Verilog that recorded which core it started on and relied on that later, which breaks when the operating system migrates a thread. The original authors had never seen it, because their board held only two cores. That is a bug which exists only at the intersection of a real operating system, enough cores for migration to happen, and speeds fast enough to reach the failure — precisely the region that the reconfigurable hardware debugging work covered elsewhere on this site identifies as poorly served, where the failure is in front of you and no tool helps you ask why.

Finding a bug that only appears on a big enough machine

The performance results reproduced the original paper closely, which is the expected and slightly boring outcome — the modelled hardware is nearly the same. The interesting result was the failure that came with it.

Why it works

• The link delay matches what it is standing in for. A round trip between chips of about 1,250 nanoseconds, which is roughly 125 cycles at the prototype’s clock → the same order as a real inter-socket delay → the measured core-to-core map shows four clean regions at 100 and 250 cycles, close to published figures for commercial multi-socket servers.
• One encapsulation serves both distances. Wrapping network-on-chip traffic in ordinary bus writes → the identical mechanism carries a packet to a node on the same chip through a crossbar and to a node on another chip through the host link → nodes can be placed on one chip or spread across four without changing the design.
• Renting converts a purchase into an hourly rate. Paying per hour instead of buying an eight-thousand-dollar machine → experiments cost what they use and many can run at once → the crossover where owning becomes cheaper is over 200 days of continuous use.
• Small targets share a chip instead of wasting one. Up to four nodes fit on one chip because the platform provides four independent memory interfaces → four separate experiments run in parallel on hardware that would otherwise host one → about four times the cost-efficiency of a comparable cloud platform running one model per chip.
• Mapping the design straight to gates keeps the clock high. Not using the tricks that pack more modelled logic into a chip at reduced speed → the prototype runs at 75 to 100 megahertz → a full operating system and a real benchmark complete in a usable amount of time, which is the whole reason to prototype rather than simulate.

What it costs (trade-offs and scope)

• The available cores are not the cores people want to study. Every case study uses an in-order, single-issue core, which does not represent a large out-of-order design. A user can supply their own core through the framework’s interface, but then they are back to writing hardware.
• Every limit of the rented platform becomes a limit of the tool. Four memory interfaces per chip means at most four nodes per chip; only four chips in an instance share the fast link, so a prototype is at most four chips; the chip fits at most 12 cores of this kind, and only at the lower clock. The authors note these are the cloud provider’s limits rather than fundamental ones.
• The link delay is a floor, not a dial. It can be lengthened with a configurable traffic shaper to model a slower interconnect, but it cannot be shortened below the 1,250 nanoseconds the physical bus takes.
• This does not replace simulation. Changing any parameter means regenerating the configuration, which is about four hours before anything runs. A study that sweeps a parameter is far better served by a simulator, and the paper says so.
• Input and output are modelled functionally only. Storage and other peripheral interfaces reproduce the behaviour of the devices but not their timing, so their performance is not being modelled.

The evaluation

Check yourself

Sources

SMAPPIC: Scalable Multi-FPGA Architecture Prototype Platform in the Cloud. Grigory Chirkov, David Wentzlaff. ASPLOS 2023, Volume 2, pages 733 to 746. DOI 10.1145/3575693.3575753. Full text read from the authors’ copy hosted at Princeton University.