15.5"H" IV: Two-Stage Address Translation

Part II Linux boot: optional Vol. II (Privileged) pp. 173–176 · ~4 min read

  • two-stage address translation
  • guest physical address

Whenever V=1, every virtual memory access runs two stages : VS-stage (vsatp, the guest’s own page tables) maps guest virtual → guest physical ; G-stage (hgatp, the hypervisor’s tables) maps guest physical → supervisor physical. Either stage can be Bare. There is no third option: even accesses that “bypass translation” — including the walker’s own reads of VS-level page tables — still pass through G-stage. PMP always checks the final supervisor physical address.

MXR splits by level: vsstatus.MXR opens execute-only pages at VS-stage only; the HS-level sstatus.MXR overrides execute-only at both stages.

The Sv*x4 schemes

VPN[2]4030VPN[1]2921VPN[0]2012page offset110Sv39x4 guest physical address
Click a field for its role.

Sv32x4/Sv39x4/Sv48x4/Sv57x4 widen the incoming address by 2 bits (34/41/50/59), absorbed entirely by a 16 KiB, 16 KiB-aligned root table — non-root tables and all PTE formats are exactly the Sv* originals. The translation algorithm is the Sv* algorithm with four substitutions:

  1. hgatp replaces satp;
  2. the effective privilege must be VS or VU for translation to begin;
  3. the U-bit check always assumes U-mode — every G-stage access is a user access, so G-stage leaves need U=1 (the bit survives in the format only to allow sharing tables with HS-level translation);
  4. guest-page faults (codes 20/21/23) replace page faults.

Accesses made in support of VS-stage translation are checked at G-stage as implicit loads (or stores, when hardware updates VS-level A/D bits) — but any exception reports as the original access type. The G bit in G-stage PTEs is unused: software zeros it, hardware ignores it.

Guest-page faults

Delegable M→HS via medeleg (never further — hedeleg bits 20/21/23 are RO-0). The handler receives: stval/mtval = the guest virtual address (page-boundary rules as usual for straddles); htval/mtval2 = the guest physical address >>2, or zero; htinst/mtinst = transform or pseudoinstruction. When the fault isn’t from an implicit VS-stage access, the GPA in htval corresponds exactly to the VA in stval.

The fence matrix

Which fence synchronizes what
Required fence
SFENCE.VMA at V=0HS-level structures (satp world) only.
SFENCE.VMA at V=1 (guest executes it)VS-stage structures of the CURRENT VMID only — the effective tag is VMID+ASID. Guests fence themselves without exits.
PMP settings changed (M-mode)SFENCE.VMA x0,x0 for HS translations, plus HFENCE.GVMA x0,x0 for G/VS-stage entries holding cached PMP results. No HFENCE.VVMA needed.
menvcfg.PBMTE/ADUE changedHFENCE.GVMA x0,x0 — reinterprets G-stage (and VS-stage) PBMT/AD fields.
henvcfg.PBMTE/ADUE changedHFENCE.VVMA x0,x0 for the current VMID (or fold into the world-switch sequence below).
Dotted-underlined cells have explanations — click one.

World switch (no atomic vsatp+hgatp pair exists): write vsatp = 0 → swap hgatp (+ henvcfg bits) → write the new vsatp. Zeroing first prevents speculative walks from caching one guest’s VS translations under the next guest’s VMID.

Pointer masking: with vsatp=Bare, guest “virtual” addresses are GPAs — up to 2 bits wider than the VA the masking hardware assumed. Implementations may mask on the TLB refill path (accepting up to 4 duplicate entries per page) rather than the timing-critical lookup path; hypervisors then issue HFENCE.GVMA when henvcfg.PMM crosses the GPA width (PMLEN 7/16 vs Sv57x4, PMLEN 16 vs Sv48x4).

Hardware Designer Notes

The nested-walk state machine is the real cost of H: your Sv39 walker becomes re-entrant (each VS-stage step calls the G-stage walker), or you flatten it into a 5×4-step sequencer. Verify the fault-attribution matrix carefully — which stage faulted, original-access-type reporting, GPA-vs-VA in the right tval registers. This is the most-bug-prone corner of the extension.

Minimal Linux-boot hart MUST

  • Route EVERY V=1 access through G-stage — including VS-stage page-table reads and hardware A/D updates of VS-level PTEs (checked as implicit load/store at G-stage)
  • Enforce the zero-check on GPA high bits (63:41/50/59) raising guest-page faults, and the 16 KiB root indexing (11-bit top VPN)
  • Tag TLB entries with VMID (+ ASID for VS-stage/combined entries) so SFENCE.VMA-at-V=1 and the HFENCEs can scope correctly

MAY simplify / trap-and-emulate

  • Build a combined GVA→SPA TLB (the common choice) — HFENCE.GVMA then flushes by VMID unless you pay for GPA tags
  • Add G-stage walk caches: the nested walk is (LEVELS+1)² memory reads worst-case (16 for Sv39x4) — intermediate caching is close to mandatory for performance
  • Support only Sv39x4 alongside Sv39

Check yourself — two-stage translation

1.During a VS-stage page-table walk, the walker reads a VS-level PTE. How is THAT read addressed?

2.Why is the G-stage root page table 16 KiB instead of 4 KiB?

3.In G-stage translation, how is the PTE's U bit checked?

4.Why does world-switch code zero vsatp BEFORE swapping hgatp?

4 questions