The memory arc ends with assembly: how the pieces — tables, TLBs, policies, swapping — combine into complete systems, with the features (performance, functionality, security) a real OS needs.
The Crux: How To Build A Complete VM System
What features are needed to realize a complete virtual memory system? How do they improve performance, increase security, or otherwise improve the system?First case study: VAX/VMS (DEC, late 1970s; principal architect Dave Cutler, who later led Windows NT). A surprising number of its techniques survive today — some ideas, even 50-year-old ones, are still worth knowing.
Memory Management Hardware
The VAX-11: 32-bit virtual addresses, 512-byte pages (23-bit VPN, 9-bit offset), with the top two VPN bits selecting a segment — chapter 20’s paging-segmentation hybrid, in production silicon:
Those tiny pages (a historical accident) made linear page tables ruinously large, and VMS fought the bloat twice over: P0 and P1 get separate page tables with base/bounds registers (bounds = table size, so the unused middle costs nothing), and user page tables live in kernel virtual memory — segment S — where they can themselves be swapped to disk under pressure. Translation grows baroque (find the user PT’s page via the system PT, then translate for real), but hardware TLBs usually skip the whole excursion.
Aside: The Curse Of Generality
VMS had to run on cheap VAXen (yes, that’s the plural) and monsters alike — the OS curse of supporting everything means excelling at nothing in particular. No less real today: the same Linux runs your phone, your laptop, and a datacenter node with thousands of processes.A Real Address Space
click a region marked ⓘ for details
Two deliberate oddities repay study. Page 0 is invalid — so dereferencing NULL traps instantly:
int *p = NULL; // set p = 0
*p = 10; // try to store 10 to virtual addr 0TLB miss on VPN 0 → page table → invalid → trap → (on UNIX, a signal; uncaught, death). Debugging support, encoded in the address space. And the kernel is mapped into every process — the S registers simply don’t change on a context switch. The kernel can copy to and from user pointers naturally, page tables can live (and swap) in kernel virtual memory, and the kernel appears to applications as something like a protected library. Protection-level bits in each PTE keep user code out; a construction now used nearly everywhere.
Page Replacement
The VAX PTE: valid, 4 protection bits, a modify bit, 5 OS-reserved bits, PFN — and no reference bit. The policy must fly blind. Worse, the designers feared memory hogs monopolizing frames under global policies like LRU. Their answer, segmented FIFO segmented fifo VMS's replacement scheme: per-process FIFO bounded by the RSS, backed by global second-chance lists (a clean-page free list and a dirty-page list). Fault on a listed page before it's recycled and you reclaim it without disk I/O; the bigger the lists, the closer to LRU. defined in ch. 23 — open in glossary : a per-process page cap — the resident set size resident set size VMS's per-process cap on resident pages (RSS): each process keeps at most this many pages in memory, evicting FIFO-style from its own set — a fairness guard against memory hogs that global policies like LRU can't provide. defined in ch. 23 — open in glossary — with FIFO eviction inside it, rescued by global second-chance lists:
1Process P sits at its resident set size
P may keep at most RSS = 4 pages in memory, managed as a plain FIFO — no reference bit required, which is convenient, because the VAX has none. P now touches a new page, e.
Aside: Emulating Reference Bits
Babaoglu and Joy later showed the missing bit was no obstacle: mark pages inaccessible (stashing true protections in the OS-reserved PTE bits); a touch traps, the OS notes “referenced” and restores real protections. Come eviction, the still-inaccessible pages are the cold ones. The art is calibration — mark too eagerly and traps swamp you; too lazily and everything looks referenced.Other Neat Tricks
Two lazy inventions round out VMS — both now universal: demand zeroing demand zeroing Lazy page allocation: map the new heap page as inaccessible and only find, zero (a security must), and map a physical page when it's actually touched — never touched, never paid for. defined in ch. 23 — open in glossary and copy-on-write copy-on-write COW (from TENEX): to 'copy' a page, map it read-only into both address spaces; only when someone writes does a trap allocate a real private copy. Makes fork()+exec() affordable and shared libraries cheap. defined in ch. 23 — open in glossary :
1Demand zeroing: a heap page is "added" — with no work at all
The process asks for a heap page. Naively the OS would find a frame, ZERO it (a security must — otherwise you'd read another process's leftovers), and map it. Instead, VMS just writes an inaccessible PTE and stops. No frame. No zeroing.
Tip: Be Lazy
Deferring work can reduce latency (report the write done; flush later) and sometimes eliminates the work entirely (the file gets deleted; the page never gets touched; exec() discards the copy). Laziness works in life too — delay the OS project and classmates may debug the spec for you. But the project won’t be canceled. Don’t make professors sad.Check yourself
1.The VAX's 512-byte pages threatened to drown memory in page tables. How did VMS fight back — twice?
2.Two deliberate oddities of the VMS address space: page 0 is invalid, and the kernel appears inside EVERY process's space. Why each?
3.The VAX PTE has no reference bit, and VMS feared memory hogs. Describe the replacement design that answered both.
4.Babaoglu and Joy showed the VAX could fake the missing reference bit. How?
5.VMS's two lazy tricks — demand zeroing and copy-on-write — share a design skeleton. What is it, and why must fresh heap pages be zeroed at all?