Virtuoso: Enabling Fast and Accurate Virtual Memory Research via an Imitation-based Operating System Simulation Methodology

Konstantinos Kanellopoulos, Konstantinos Sgouras, F. Nisa Bostanci, Andreas Kosmas Kakolyris, Berkin Kerim Konar, Rahul Bera and 4 others

Studying how to speed up virtual memory means simulating both the operating system and the hardware together, but researchers are stuck between fast simulators that fake the OS with a fixed made-up latency and accurate ones that boot a whole real kernel yet are slow and brutal to modify; this work from ETH Zurich and partners threads the needle with imitation — a lightweight userspace kernel that mimics only the memory-management code you care about and feeds its real instruction stream into an architectural simulator, hitting most of a full-system simulator's accuracy at a fraction of its cost and effort, and it ships integrated with five simulators.

The argument in one glance

Problem

Virtual memory is a bottleneck, but evaluating fixes is hard

Virtual memory — the machinery that maps a program's addresses to physical memory through page tables and translation buffers — has become a major performance drain, eating up to 40 percent of runtime in long-running workloads on address translation and up to 95 percent in short ones on memory allocation. So researchers keep proposing hardware and OS co-designs to fix it. But evaluating those designs is genuinely hard: virtual memory straddles the hardware/software boundary, its pieces are deeply interdependent, and its overheads vary enormously — a page fault might take two microseconds or fifty — so no fixed number captures them.
What people did

Pick a simulator, fast or accurate

Computer architects evaluate ideas in simulators, and for virtual memory there are two kinds. Emulation-based simulators like Sniper are fast, and fake the operating system with simple approximations. Full-system simulators like gem5 boot a real, complete kernel on top of the simulated hardware, so the OS behaviour is authentic. In principle one of these should let you measure a new virtual-memory design.
Why it fell short

The fast ones lie, and the accurate ones are unbearable

Neither works. The fast simulators model the OS with fixed made-up latencies — a constant page-walk time, a constant page-fault time — which is wildly wrong for a subsystem whose overheads swing by orders of magnitude, giving a 35 percent error in performance estimates. The accurate full-system simulators boot a real kernel, which makes them slow, memory-hungry, and worst of all, to test a new design you must modify a huge, intricate production kernel and verify it on simulated hardware — a task few researchers outside kernel experts can stomach. So people reinvent a custom tool for every proposal.
This paper's idea

Imitate only the kernel code you care about

Virtuoso takes a middle path it calls imitation. Instead of a fixed latency or a whole real kernel, it runs a lightweight userspace kernel, MimicOS, written in ordinary C++, that imitates just the memory-management routines a researcher cares about — the page fault handler, the allocator, huge-page policy. MimicOS runs as a normal program beside the architectural simulator, connected by two channels: one carries functional events like a page fault and its outcome, the other injects MimicOS's real instruction stream into the simulator so it models the actual overhead of the OS code. Fast, accurate, and easy to hack.

Why this problem is worth solving

The subsystem this whole framework is built to study is one of the oldest and most load-bearing parts of a computer.

Virtual memory is the abstraction that lets every program act as if it owns a large, private, contiguous block of memory, while the operating system and hardware quietly map those virtual addresses onto the real physical memory underneath. That mapping is done through page tables — data structures the hardware walks to translate a virtual address — accelerated by a small cache of recent translations called a translation lookaside buffer, and maintained by the OS through operations like the page fault handler, which allocates physical memory the first time a page is touched. It is one of the most fundamental parts of a computer, and as data footprints explode it has become a serious bottleneck: walking page tables and handling faults can consume a large fraction of a program’s runtime. Crucially, that cost is not constant — it depends on the workload, the memory layout, the page sizes the OS chose, and the state of the system — which is exactly what makes it hard to model and hard to optimize.

The reason this is a research-tooling problem, and not just an engineering one, is how architects actually test ideas.

Computer architects rarely build a chip to test an idea; they simulate it. An architectural simulator is a program that models a processor and memory system in enough detail to estimate how fast a real machine would run a given workload, letting researchers try out a new cache, translation buffer, or memory design before committing it to silicon. There is an unavoidable tension between speed and accuracy. A simulator that models every microarchitectural detail is faithful but slow, so exploring many designs takes forever; one that abstracts away detail is fast but approximate. The operating system is the sharpest version of this tension: modeling it faithfully means running a real kernel inside the simulator, which is accurate but enormously slow and hard to modify, while ignoring it means faking its behaviour with rough numbers. For anything that depends on the OS, like virtual memory, neither extreme serves research well, which is the gap this work targets.

What people did before, and where it broke

Emulation-based simulators (Sniper, ChampSim, ZSim). Fast simulators that focus on the microarchitecture and fake the OS with simple, fixed-latency approximations. Where it broke. A constant page-walk latency or page-fault latency is wildly wrong for virtual memory, whose overheads swing by orders of magnitude across workloads, giving a 35 percent error in performance estimates.
Full-system simulators (gem5-FS, QFlex). Boot a real, complete operating system on top of the simulated hardware, so the OS behaviour is authentic. Where it broke. Running a real kernel makes them slow (a 77 percent time increase) and memory-hungry, and to test a new design you must modify an intricate production kernel and verify it on simulated hardware — an effort few non-experts can manage.
Custom per-proposal tools. Build a bespoke evaluation harness for each new virtual-memory scheme. Where it broke. The work is reinvented from scratch every time, there is no common ground to compare designs on, and the tools rarely outlive the paper they were built for.
The gap. The field was caught between fast-but-fake and accurate-but-unbearable, with no way to model the OS side of virtual memory both faithfully and cheaply, and no shared infrastructure to develop and compare schemes. What was missing was a methodology that kept the speed and hackability of the fast simulators while recovering most of the accuracy of the full-system ones — and that could be ported across the many simulators the field already uses.

How Virtuoso works

Modeling only the OS code that matters, at real fidelity

↓ →
new
↓ →
new
↓ →
new
↓ →
new
↓ →
new

The idea

Imitate only the part of the kernel that matters, and skip the rest. A full-system simulator’s fatal cost is that it runs the entire operating system — the scheduler, the file system, every syscall path — even when a study only cares about how physical memory gets allocated. Virtuoso’s core move is imitation: MimicOS is a lightweight userspace kernel that reproduces just the memory-management routines the researcher is investigating and leaves everything else out, so it behaves like the real kernel where it counts and costs nothing where it does not. Reproducing only a compact, faithful stand-in for the piece of a heavy system you actually need, rather than dragging the whole thing along, is the same economy the out-of-core random-walk work covered elsewhere on this site relies on, keeping only a succinct representation of what the computation requires in the fast path while the rest is left aside.

Give one clean interface and let it plug into every simulator. The waste in the old world was that each new virtual-memory study spawned a one-off tool welded to a single simulator, so nothing could be reused or compared. Virtuoso instead defines a narrow contract between the OS model and the simulator — a functional channel for events and an instruction channel for timing — and holds MimicOS behind it, so the same kernel drops into five very different simulators with minimal changes and brings a whole library of virtual-memory schemes with it. Decoupling a capability from the many backends that consume it through one small, well-defined interface, so that implementing the interface once unlocks everything, is exactly the architecture the simulator-independent-coverage work covered elsewhere on this site is built on, where a single primitive lets one implementation serve many otherwise-incompatible tools.

Model the real overhead by running the real code, not by guessing a number. The reason fast simulators are inaccurate is that they replace the operating system’s work with a fixed latency, which cannot possibly track a page fault that ranges from two microseconds to fifty. Virtuoso refuses the guess: it dynamically instruments MimicOS and injects its actual instruction stream into the simulator’s core model, so the simulator measures the true cost of the OS routine — the cycles it takes, the caches it pollutes, the memory contention it causes — from the code itself. Rejecting a black-box approximation in favor of reasoning from the real implementation is the same instinct the silent-data-corruption work covered elsewhere on this site follows, insisting that the honest way to model a system’s behaviour is to work from its actual internals rather than a convenient abstraction of them.

Following the reasoning through a page fault

Handling a page fault the imitation way

The paper's worked example traces one page fault to show how the two channels combine fidelity with speed.

Step 1 of 5

The application touches a page that is not mapped

The simulated program issues a memory access, the core forwards it to the memory management unit, and the translation is not in the translation buffer, so the hardware walks the page table — and finds no entry. That triggers a page fault, an event only the operating system can resolve.

In a fast simulator this is where a fixed, made-up fault latency would simply be added, wrong or right.

Why it works

Imitating only the routines that matter skips the dead weight of a whole kernel. A full-system simulator pays to run every OS routine, most irrelevant to a virtual-memory study → but MimicOS models just the memory-management code the researcher cares about → so mimicking only the Linux memory subsystem runs 49 percent faster than simulating the entire kernel, at a total overhead near 20 percent.
Injecting the real instruction stream measures the true overhead instead of guessing it. A fast simulator charges a fixed made-up latency for a page fault, wrong by orders of magnitude → but feeding the handler’s actual instructions into the simulator’s core model makes it time the real work, including memory contention and cache pollution → so page-walk and fault latencies come out close to a real machine rather than a constant.
Writing the kernel in userspace C++ turns OS hacking into ordinary programming. Modifying a production kernel and verifying it on simulated hardware is a task for kernel experts → but MimicOS is a standalone C++ program a researcher can edit like any other → so a new allocator or even a machine-learning page-replacement policy can be dropped in with a high-level library, no kernel expertise required.
A clean two-channel interface makes the same kernel portable across simulators. Every new virtual-memory study used to spawn its own bespoke tool tied to one simulator → but MimicOS talks to any simulator through a functional channel for events and an instruction channel for timing → so it plugs into five very different simulators and packages the field’s schemes into one common evaluation ground.
Choosing which routines to imitate dials the accuracy-speed trade to the question. Some studies need faithful fault handling, others barely touch it → but a researcher composes MimicOS from only the modules the study requires → so simulation time tracks the amount of OS code actually exercised, and the tool is as detailed as it needs to be and no more.

What it costs (trade-offs and scope)

Accuracy is high but not full-system-exact. Against a real server it reaches 82 percent on translation-buffer behaviour and 85 percent on page-walk latency, but only 66 percent on average (up to 79) for page-fault latency, and 80 percent on instructions-per-cycle — a big jump over the emulation baseline’s 66 percent, yet still around a fifth off the real machine, and conclusions drawn with it inherit that error.
It imitates a subset, not the whole kernel. MimicOS faithfully mimics Linux memory management, but behaviour outside that subsystem — scheduler interactions, full syscall paths, subtle concurrency — is not captured unless the researcher adds modules, and accuracy depends on how completely the mimicry is built out.
It is still a simulator, validated against one machine. The numbers come from a single Intel server and one Linux version; generality to other processors and operating systems is assumed rather than demonstrated.
The speed win carries a memory cost. Simulation time rises only about 20 percent, but memory rises 1.45 times on average and up to 2.1 times when online binary instrumentation is used, so on memory-limited machines the number of parallel simulation jobs drops by up to half.
It reimplements kernel logic, which can drift from reality. Imitation means re-coding the OS’s behaviour in C++, and a mimic can diverge from what the real kernel actually does; the tool is only as accurate as the fidelity of the imitation the researcher writes.
The use-case insights are demonstrations, not settled findings. The five case studies show the tool works and yield plausible new observations, but their conclusions rest on its roughly 80 percent accuracy, so they are directional evidence rather than definitive measurements.
It depends on external instrumentation and simulator hooks. Integration requires a binary instrumenter such as Pin or DynamoRIO and magic-operation support in the target simulator, so porting to a new simulator is easy but not free.

The evaluation

Ways to simulate the OS side of virtual memory

Property	Emulation-based (Sniper)	Full-system (gem5-FS)	Custom per-proposal tool
How the OS is modeled
Accuracy versus a real CPU			Varies
Simulation overhead			Varies
Development effort
Portability	Not applicable	Tied to one simulator	One-off

Approaches to simulating virtual memory, with Virtuoso validated against a real Intel Xeon server running Linux. Its imitation methodology — the MimicOS userspace kernel plus a two-channel interface — models the translation unit within 82 to 85 percent and IPC within 80 percent of the real machine (up from the emulation baseline's 66 percent), at only a 20 percent average simulation-time overhead versus full-system gem5's 77 percent, and ships integrated with five simulators and a library of state-of-the-art schemes.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

Virtuoso is a well-conceived piece of research infrastructure: by imitating only the memory-management routines that matter in a hackable userspace kernel and injecting its real instruction stream into an architectural simulator, it recovers most of a full-system simulator’s accuracy at close to emulation speed and effort, and ships integrated with five simulators plus a library of virtual-memory schemes as a common ground for the field. Read it as tooling, with its accuracy in mind. It reaches 82 to 85 percent on translation behaviour and 80 percent on IPC, a real improvement over the fixed-latency baseline’s 66 percent, but page-fault latency is only about 66 percent on average, so it is roughly a fifth off the real machine and any insight it produces carries that error. It mimics a subset of Linux, not the whole kernel, and is validated on one processor and one OS. Its speed gain comes with up to a 2.1-times memory cost, its imitation can drift from the real kernel, and its five case studies are demonstrations resting on that accuracy rather than settled results. The durable contribution is the methodology: imitate the piece that matters, run its real code, and make it portable.

Background

Why simulation methodology decides what research happens

It is worth stepping back to see why a paper about a simulator matters as much as a paper about a new design. In computer architecture, almost every idea is first evaluated in a simulator, because fabricating a chip to test a hunch is impossibly slow and expensive. That makes the simulator not a neutral instrument but a lens that shapes which ideas can be explored at all: if the available tools cannot model some part of a system accurately or cheaply, then designs that depend on that part go unstudied, not because they are uninteresting but because no one can measure them. Virtual memory sat in exactly this blind spot, its OS half either faked or unbearably expensive to model, so the community kept building throwaway tools and comparing results that were hard to trust. A shared, accurate-enough, easy-to-extend framework changes what the field can do, lowering the cost of asking a question until far more questions get asked. The same conviction that better infrastructure unlocks a whole line of research drives the FPGA-middlebox work covered elsewhere on this site, which set out to make an awkward, expert-only kind of development pleasant enough that more people could do it. The broader pattern is that the tools a research community shares quietly determine the shape of what that community discovers.

If you remember only three things

1. Virtual memory is a growing performance bottleneck, so researchers keep proposing hardware and OS co-designs, but evaluating them requires modeling the OS and hardware together — and the overheads vary so much (a page fault can take two microseconds or fifty) that no fixed latency captures them; existing simulators force a bad choice between fast emulation-based ones that fake the OS with fixed latencies and get address translation badly wrong (35 percent IPC error), and full-system ones that run a real kernel accurately but are slow, memory-hungry, and require rewriting complex kernel code.
2. Virtuoso introduces an imitation methodology: a lightweight userspace kernel, MimicOS, written in ordinary C++, imitates only the memory-management routines a study needs, runs beside an architectural simulator as a standalone program, and connects through two channels — a functional channel that carries events like a page fault and its outcome, and an instruction-stream channel that injects MimicOS's real instructions into the simulator's core model so it times the true OS overhead rather than a fixed guess — combining full-system fidelity with emulation speed and userspace-level ease of development.
3. Validated against a real Intel Xeon server, Virtuoso models the translation unit within 82 to 85 percent and IPC within 80 percent (up from a fixed-latency baseline's 66 percent) at only a 20 percent average simulation-time overhead versus full-system gem5's 77 percent, and ships integrated with five simulators and a library of state-of-the-art schemes — but it is still a simulator roughly a fifth off the real machine on IPC and 66 percent on page-fault latency, mimics only a subset of Linux, is validated on one machine, costs up to 2.1 times more memory, and its use-case insights inherit its accuracy.

Check yourself

Check your understanding

1. Why can't existing simulators evaluate virtual-memory designs well?
2. What is Virtuoso's core idea?
3. How does Virtuoso get accuracy while staying fast?
4. Which statement best reflects the limits of Virtuoso?

Sources

The paper is “Virtuoso: Enabling Fast and Accurate Virtual Memory Research via an Imitation-based Operating System Simulation Methodology” by Konstantinos Kanellopoulos, Konstantinos Sgouras, F. Nisa Bostanci, Andreas Kosmas Kakolyris, Berkin Kerim Konar, Rahul Bera, Mohammad Sadrosadati, Rakesh Kumar, Nandita Vijaykumar, and Onur Mutlu, published at ASPLOS 2025, pages 1399 to 1420, DOI 10.1145/3676641.3716027. The authors are at ETH Zurich, NTNU in Norway, and the University of Toronto, and the source code is openly available.

This explainer is written from the full 22-page text. Everything above comes from the paper’s own sections: the introduction and background quantifying virtual-memory overheads (up to 40 percent of runtime for address translation in long-running workloads and up to 95 percent for allocation in short ones) and the high variability of page-fault and page-walk latencies that defeats fixed-latency models; the classification of existing simulators into emulation-based and full-system, and the limits of each (Sniper’s 35 percent IPC error from a fixed page-walk latency; gem5’s 77 percent time and 1.69-times memory overheads in full-system mode); the Virtuoso overview and the imitation-based methodology with its lightweight userspace kernel, the functional and instruction-stream channels, POSIX shared-memory and magic-operation synchronization, and the step-by-step page-fault workflow; the MimicOS design that mimics Linux memory management (page fault handling, slab and buddy allocation, 1GB/2MB/4KB page decisions, hugetlbfs, swap and page cache, disk via an SSD simulator); the integration with five simulators (Sniper, ChampSim, Ramulator2, gem5-SE, MQSim) and the VirTool library of state-of-the-art schemes; and the validation against a real Intel Xeon Gold 6226R server (82 percent L2 TLB MPKI, 85 percent page-walk latency, 66-to-79 percent page-fault latency, 80 percent versus 66 percent IPC accuracy, 20 percent versus 77 percent simulation-time overhead, 1.45-to-2.1-times memory overhead, and 49 percent faster than simulating the full kernel) plus five case studies including alternative page-table designs where hash-based tables cut page-walk latency but elastic cuckoo hashing raises DRAM row-buffer conflicts by 52 percent.