SEVeriFast: Minimizing the root of trust for fast startup of SEV microVMs

Serverless platforms boot a fresh virtual machine per request in about 40 milliseconds by never compressing the kernel and stripping out every bootstrap stage, and encrypting that machine so the cloud provider cannot read it turns 40 milliseconds into over three seconds; this paper (Virginia Tech) finds the cost is not encryption but measurement, and that both fast-boot rules invert once you account for it — so it adds back a bootstrap stage and re-compresses the kernel, and boots 86 to 94 percent faster than the current approach.

The argument in one glance

Problem

You want the provider locked out, and you want to boot in milliseconds

Serverless platforms run each function in its own tiny virtual machine, created on demand and thrown away afterwards. That only works because those machines start in tens of milliseconds — the startup is not amortised over a long lifetime the way it is for a conventional server, and for short functions it can dominate the whole runtime. Separately, tenants increasingly want the cloud provider itself outside the trust boundary, which hardware can now provide by encrypting the memory a machine runs in. Both are desirable. Together they have not worked.
What people did

Turn on memory encryption and keep the fast boot path

The hardware feature exists and is deployed. Memory backing a guest is encrypted with a key the host cannot obtain, and a small dedicated security coprocessor on the chip hashes the machine initial contents to build a launch measurement, which is later signed and shown to the tenant as proof that what booted is what they asked for. The obvious plan is to enable that on the same lightweight virtual machine monitors that already achieve fast boot, and pay a modest constant for the privilege.
Why it fell short

The measurement is per byte, and the kernel is enormous

The initial contents have to be hashed and encrypted before anything runs, and that work is done by a small low-power coprocessor at a cost proportional to size. A minimal Linux kernel of 23 megabytes takes 5.65 seconds to process, against roughly 40 milliseconds for the whole boot without encryption. Compressing it to a few megabytes still costs 840 milliseconds, and the standard confidential boot path drags in a full firmware implementation whose own runtime exceeds three seconds. Nothing about this is a modest constant.
This paper's idea

Measure something tiny, and check the rest more cheaply

This paper (Virginia Tech — Benjamin Holmes, Jason Waterman and Dan Williams) observes that the expensive step is measurement, not encryption, and that measurement cost is what every design decision should be minimising. So put a 13-kilobyte verifier into the hardware root of trust instead of the kernel, and have it check the kernel on the main processor by comparing hashes. Two consequences follow that contradict established fast-boot practice — you should add a bootstrap stage rather than remove one, and you should compress the kernel rather than not.

Why this problem is worth solving

The two established rules for booting a microVM quickly are both consequences of a cost model, which is what makes them fragile.

A conventional machine boots through firmware and a bootloader, which decompress a kernel image and hand control over. A microVM skips all of that, on the reasoning that it is starting on a system already bootstrapped, so the monitor does the work itself: it loads an UNCOMPRESSED kernel directly into the memory where it will run, builds the data structures Linux expects, and enters at the 64-bit entry point without any of the legacy mode transitions. Both rules exist for measurable reasons. Compression was historically worth it because reading a large kernel from disk cost more than decompressing a small one, but a serverless host runs thousands of machines from the same kernel image, so it is already in memory and decompression is pure added latency. And the bootstrap stages are redundant work on a machine that has already been bootstrapped. The result is a boot of roughly 40 milliseconds

Adding hardware-enforced confidentiality introduces a third cost that neither rule accounts for.

Encrypting a guest memory is not the expensive part — a dedicated engine in the memory controller does that transparently. The expensive part is proving to the tenant what was loaded. Before the guest runs, the hypervisor asks a small on-chip security coprocessor to hash regions of guest memory into a LAUNCH MEASUREMENT and encrypt them with the new guest key, a step this paper calls PRE-ENCRYPTION. That measurement is later signed with a chip-unique key and sent to the tenant as proof that the machine started from exactly the expected contents. The problem is arithmetic. The coprocessor is a small low-power core, the cost scales linearly with the number of bytes measured, and the thing being measured is the initial code — which in a microVM means the kernel. Twenty-three megabytes takes 5.65 seconds, which is more than a hundred times the entire unprotected boot

What people did before, and where it broke

Boot a confidential VM the way a normal one boots. Use the standard guest firmware, which supports the full firmware specification and its six initialisation phases.
But almost all of that is redundant here. The firmware alone runs for over three seconds, and its smallest build is a megabyte, which itself costs 256 milliseconds just to measure.
Skip the firmware and load the kernel directly, as microVMs do. This is the established fast path and it works well without encryption.
But the kernel becomes the thing that must be measured. A 23-megabyte kernel costs 5.65 seconds to pre-encrypt, and its uncompressed initial ramdisk another 2.85.
Compress everything to reduce the bytes measured. Fewer bytes, less measurement, and the compression cost is paid on the fast main processor.
But it is still two orders of magnitude too slow. A compressed kernel takes 840 milliseconds to pre-encrypt, against a 40-millisecond target.
Measure hashes instead of contents. A published technique pre-encrypts a hash of each boot component, and the guest re-hashes and compares once running, which moves the work to the main processor.
But something still has to be in the root of trust to do the checking, and if that something is the kernel you are back where you started.
The gap. Every approach kept the kernel as the initial trusted code, because that is what fast microVM boot means. The cost of trusting it had grown by two orders of magnitude, and nobody had reconsidered the arrangement.

How the root of trust gets small enough

Rebuilding the boot path around measurement cost instead of decompression cost

Load an uncompressed kernel directly and skip bootstrap stages

The standard microVM recipe: no firmware, no bootloader, the monitor places an uncompressed kernel where it will run, builds the data structures Linux needs, and jumps straight to the 64-bit entry point. About 40 milliseconds, and the baseline everything here is measured against.
↓ →
Measure where confidential boot time actually goes new

Break the encrypted boot into its parts and find that the dominant term is the hashing and encrypting of initial contents by a small on-chip coprocessor, at a cost linear in size. That reframes every subsequent decision: the objective is not fewer instructions but fewer measured bytes.
↓ →
Replace the kernel in the root of trust with a tiny verifier new

Write a standalone 13-kilobyte binary whose only jobs are to mark guest pages as encrypted and validated, and then check the kernel. That, rather than the kernel, is what the coprocessor measures — so the expensive per-byte step now applies to kilobytes rather than tens of megabytes.
↓ →
Have the main processor check the kernel by hash new

Pre-encrypt only the hashes of the kernel and ramdisk, which puts them in the launch measurement and beyond the host reach. The verifier copies the components from shared memory into encrypted memory, re-hashes them, and compares — verification on a fast core instead of a slow coprocessor.
↓ →
Compress the kernel again, because measurement is per byte new

That checking still costs twice per byte, once to copy into encrypted memory and once to hash. Halving the bytes by compressing therefore beats the decompression it adds back, which reverses the microVM rule. Compressing the ramdisk does not pay, because its archive already has to be unpacked.
↓ →
Take redundant work off the critical path and pre-encrypt selectively new

The kernel hash is a property of the kernel, so compute it once offline and hand it to the monitor rather than recomputing per boot. And for each small data structure Linux needs, pre-encrypt it only when the code that would generate it in the guest is larger than the structure itself.

Implemented as about 1,100 lines of Rust inside a production microVM monitor, the first such monitor to support confidential guests, evaluated against the standard confidential boot path on three kernel configurations.

The idea

Two rules everyone follows turn out to be consequences of a cost model that just changed. Fast microVM boot rests on removing bootstrap stages and refusing to compress the kernel, and both rules are well justified: the stages are redundant on an already-bootstrapped host, and decompression is pure added latency when the kernel image is already in memory because thousands of machines share it. Introduce hardware attestation and a third cost appears that neither rule was weighed against, and it dominates both — every byte of initial content must be hashed and encrypted by a small on-chip coprocessor before anything runs. Measured against that, the rules invert. Adding a bootstrap stage is now correct, because a 13-kilobyte verifier can stand in the root of trust where a 23-megabyte kernel otherwise would. Compressing the kernel is now correct, because the checking that replaces measurement still costs twice per byte and halving the bytes beats the decompression it reintroduces. The paper is explicit that this is counter-intuitive, and it is right to be, because the underlying rules were never wrong — they were answers to a question that had one term added to it. Re-deriving a received best practice from scratch when the cost structure it optimised against has changed, rather than porting the practice forward, is the same the-optimisation-assumed-a-different-machine correction covered elsewhere on this site where a technique tuned for one environment becomes actively harmful in another.

Make the expensive verifier tiny and let a cheap one do the bulk. The structure that falls out is a two-tier trust chain, and it works because the two tiers have wildly different unit costs. The hardware coprocessor is authoritative but slow, so give it as little as possible: a 13-kilobyte verifier, a handful of small data structures, and the hashes of the large components. The main processor is fast but must be told what to trust, so let it do the volume work — copy the kernel into encrypted memory, re-hash it, compare against the hash the coprocessor vouched for. Neither tier could do the job alone, and the split is what makes the arithmetic work. The same instinct shows up in a smaller decision the paper states as a rule: for each little structure Linux needs at boot, pre-encrypt it only when the code that would generate it inside the guest is larger than the structure itself. That sends a 155-byte command line and a 4-kilobyte parameter block to the coprocessor, and keeps page-table construction in the verifier, because generating page tables costs 2.4 kilobytes of code to avoid measuring 4 kilobytes of data. Shrinking what an expensive authority must attest to, and delegating the volume to something cheaper that the authority has vouched for, is the same minimise-the-trusted-core structure covered elsewhere on this site where a small checker replaces a large trusted producer.

The honest finding is that this is not enough, and why. The evaluation includes a result the paper did not need to publish. Launching confidential guests concurrently makes each one slower, linearly: at 50 concurrent launches the average boot time reaches about 1800 milliseconds, where the same experiment without encryption stays essentially flat. The authors check whether this is their fault, find the same behaviour with the standard firmware-based path, and conclude it is a property of the hardware — the security coprocessor is a single core, and every launch command for every guest must pass through it, so guest creation is effectively serialised. Their design still wins by a wide margin, since 50 concurrent launches remain below the time for a single guest under the old approach, but they state plainly that this bottleneck must be solved before confidential serverless is viable, and that the obvious workaround — letting multiple machines share an encryption key — weakens the trust model. Reporting a limit that the paper’s own contribution cannot reach past, and locating it precisely enough that somebody else can attack it, is the same name-the-wall-you-hit discipline covered elsewhere on this site where an evaluation identifies the next bottleneck rather than stopping at the one it removed.

Booting one confidential function

From a launch request to an attested guest, and where the time goes

Measurements on AMD hardware with the latest generation of its confidential VM extensions, across small, medium and large kernel configurations.

Step 1 of 5

Start from the fast path and watch it collapse

Without encryption, a microVM monitor loads an uncompressed kernel straight into place and boots in about 40 milliseconds. Turn on confidential mode with the standard tooling and the guest firmware alone runs for over three seconds, with only one small part of it — checking the boot components — actually required for the security property.

that firmware exists to provide full standards compliance, including device emulation and a shell, none of which a microVM needs
even its smallest build is a megabyte, which costs 256 milliseconds just to hash and encrypt before anything runs

Why it works

Establishing the root of trust costs time proportional to the bytes measured → the thing measured should be as small as possible → so a 13-kilobyte verifier replaces a 23-megabyte kernel and pre-encryption becomes a constant under 9 milliseconds.
The on-chip coprocessor is authoritative but slow while the main processor is fast → pre-encrypt only hashes and let the verifier check contents against them → moving the volume work onto hardware that is orders of magnitude quicker.
Checking a component costs twice per byte, once to copy and once to hash → compressing the kernel halves that at the price of decompression on a fast core → which is why the microVM rule against compression reverses here.
A kernel hash depends only on the kernel, not on this particular boot → compute it offline and pass it in → removing up to 23 milliseconds of redundant hashing from every launch.
Small boot structures can either be measured or generated in the guest → measure one only when the code to generate it exceeds its own size → which keeps page-table construction in the verifier and sends the tiny command line to the coprocessor.

What it costs (trade-offs and scope)

It is still four times slower than an unprotected microVM. Most of that is inherent to the hardware rather than to this design, since Linux itself boots 2.3 times slower under these extensions.
Concurrency does not scale. Average boot time grows linearly with concurrent launches to about 1800 milliseconds at 50 guests, because every launch command goes through a single-core coprocessor.
Cold start only. Snapshot-based warm start is hard here, since identical contents encrypt differently at different physical addresses and lazy page mapping would require the guest to verify pages the host supplies.
The boot components are handed to the host in plain text. That is argued to be safe because they contain no secrets and attestation proves they were not altered, but it is a real assumption about how images are constructed.
A narrow platform. One vendor’s extensions, one monitor, and a verifier built by stripping an existing loader down to two functions.

The evaluation

Three kernel configurations standing for small, medium and large, booted a hundred times each with the image already warm in the page cache, which is the realistic serverless case since thousands of machines share one kernel. The comparison points are the standard confidential boot path built on full guest firmware, and the same monitor with confidentiality switched off, which brackets the result between what exists today and what it costs to give up. Boot time is broken into monitor time, pre-encryption, verification, decompression and Linux boot, using an in-guest timing channel that had to be built specially because the usual one raises an exception under these extensions.

Property	Unprotected microVM	Confidential boot via full firmware	Minimal root of trust
What sits in the root of trust
Cost of establishing trust
Kernel format
Bootstrap stages
Behaviour at 50 concurrent launches

Against the standard confidential boot path, total boot time including attestation falls by 93.8 percent for a small kernel, 88.5 percent for the medium one and 86.1 percent for the large one. Against the same monitor without confidentiality, boot is about four times slower, of which Linux itself accounts for a 2.3 times increase from exception handling and memory ownership checks. Enabling huge pages cuts guest page validation from over 60 milliseconds to under 1.Underlined values explain themselves when clicked.

Where this stops working

What to keep in mind

The headline improvement is against the existing confidential boot path, not against unprotected boot, and the paper is careful to present both. Measured against a normal microVM the result is still about four times slower, and most of that gap is not something this design can address — Linux itself takes 2.3 times longer to boot under these extensions because of exception handling on world switches and an ownership check on every write to guest memory, and the monitor pays extra to register pages with the kernel since identical plain text encrypts differently at different physical addresses. The more serious limit is concurrency. Average boot time rises linearly with simultaneous launches, reaching roughly 1800 milliseconds at 50, because every launch command for every guest passes through a single-core security coprocessor. The authors verify this is not their doing by reproducing it on the standard path, and say directly that it must be solved for confidential serverless to be viable, noting that the near-term workaround of sharing encryption keys between machines weakens the trust model. Scope is narrow in the usual ways: one vendor’s extensions, one monitor, a verifier assembled by stripping an existing loader to two functions. And it addresses cold start only — warm start from snapshots is genuinely hard here, because encrypted pages cannot simply be mapped in and the guest would have to verify state it may not know.

Background

When the received wisdom was an answer to a different question

This paper sits in the site’s datacenter and security clusters and rhymes with several ideas covered elsewhere on this site. Its central move — noticing that two well-justified optimisations invert once a new dominant cost enters the accounting, so the right response is to re-derive them rather than carry them forward — is the same reasoning the serverless graph processing page applies when structures built to accelerate computation on a permanent server become the dominant cost in a short-lived container. Its architecture, giving an expensive authority as little as possible to vouch for and delegating the bulk to something faster that the authority has blessed, is the same minimise-the-trusted-core bargain the software sandboxing page strikes when it replaces a two-million-line trusted compiler with a small verifier, and the same two-tier arrangement the error-correction predecoding page uses to fit an exact algorithm inside a deadline. Its subject sits directly alongside the serverless cold-start pages, which attack the same latency from the scheduling and caching side rather than the trust side. And its closing negative result, tracing a linear slowdown to a single shared coprocessor that nothing in the software can route around, echoes the pages where the binding constraint turns out to be a piece of hardware nobody was counting.

If you remember only three things

1. Serverless microVMs boot in about 40 milliseconds by following two rules — load an uncompressed kernel directly, and remove every bootstrap stage — both well justified, since the host is already bootstrapped and the kernel image is already in memory because thousands of machines share it. Adding hardware confidentiality introduces a cost neither rule was weighed against: before the guest runs, its initial contents must be hashed and encrypted by a small on-chip security coprocessor at a price linear in size. A 23-megabyte kernel costs 5.65 seconds, compressing it still costs 840 milliseconds, and the standard confidential path drags in firmware whose own runtime exceeds three seconds.
2. Once measurement is recognised as the dominant term, both rules invert. Adding a bootstrap stage becomes correct, because a purpose-built 13-kilobyte verifier can occupy the root of trust where the kernel otherwise would, cutting the expensive per-byte step from tens of megabytes to kilobytes. And compressing the kernel becomes correct, because the verifier checks it on the main processor at a cost of twice per byte — once to copy into encrypted memory, once to re-hash — so halving the bytes beats the decompression it adds back. The ramdisk stays uncompressed, since its archive must be unpacked regardless.
3. The result is a two-tier trust chain matched to two very different unit costs: the slow authoritative coprocessor measures only the verifier, a few small structures and the hashes of the large components, while the fast main processor does the volume checking against hashes it has vouched for. Boot time falls 86 to 94 percent against the existing confidential path, with pre-encryption becoming a constant under 9 milliseconds. It remains about four times slower than unprotected boot, mostly for reasons inherent to the hardware — and concurrent launches degrade linearly to 1800 milliseconds at 50 guests, which the authors trace to the coprocessor being a single core and name as the thing that must be fixed next.

Check yourself

Check your understanding

1. MicroVMs boot fast by loading an uncompressed kernel and stripping out bootstrap stages. Predict why both rules reverse once the machine must be confidential.
2. The kernel is no longer part of what the hardware measures, yet the tenant still needs assurance it was not swapped. Predict how that works.
3. For each small data structure Linux needs at boot, the design must choose between measuring it and generating it inside the guest. Predict the rule.
4. The design cuts boot time by 86 to 94 percent, yet the paper argues confidential serverless is still not viable. Predict the reason.

Sources

Benjamin Holmes, Jason Waterman and Dan Williams. “SEVeriFast: Minimizing the root of trust for fast startup of SEV microVMs.” ASPLOS 2024, Volume 2. Virginia Tech, with work done while one author was at Vassar College. DOI 10.1145/3620665.3640424.
Full text of the paper was used for this explainer, including the microVM and confidential VM boot background and the measured direct boot mechanism in Section 2, the incompatibility analysis and its cost measurements in Section 3, the four design components covering the minimal boot verifier, selective pre-encryption, out-of-band hashing and kernel compression in Section 4, the implementation inside a production microVM monitor in Section 5, and the boot time comparisons, breakdown against unprotected boot and concurrent launch study in Section 6.
The authors stated they planned to release source for all components.