§6.1–6.2Basic Technique: LDE … Problem #1: Restricted Operations

Chapter 4 promised mechanisms; here is the big one. To time-share the CPU the OS must run one process a while, then another — while solving two problems at once: performance (don’t make everything slow) and control (don’t lose the machine to a runaway or malicious process). Without control, a process could run forever or read anything it likes; the OS would be, as the book puts it, “just a library.”

6.1 Basic Technique: Limited Direct Execution

The “direct execution” half of limited direct execution limited direct execution Run programs directly on the CPU for speed, but with hardware-enforced limits (modes, trap table, timer) so the OS keeps control. defined in ch. 6 — open in glossary is simple: just run the program on the CPU. Create a process-list entry, allocate memory, load the code, set up the stack, call main(). Step through the unlimited version and watch for the moment things go wrong:

Fast? Absolutely — the program runs natively. But two questions have no answer in that protocol: (1) how does the OS make sure the program doesn’t do anything it shouldn’t, while still running efficiently, and (2) how does the OS ever stop it and switch to another process? The “limited” half of the name is the answer to both.

6.2 Problem #1: Restricted Operations

Letting any process issue raw I/O would destroy protection: a file system that checks permissions is meaningless if any process can read or write the whole disk directly. So the hardware provides modes: in user mode, a process attempting a restricted operation (like an I/O request) causes the processor to raise an exception exception The processor's reaction to an illegal act (raw I/O in user mode, divide by zero, bad memory access) — traps into the OS, which likely kills the process. defined in ch. 6 — open in glossary — and the OS likely kills it (“one strike and you’re out”). The kernel, in kernel mode, may do anything.

For legitimate privileged needs, there’s the controlled doorway you met in chapter 2: the system call (pioneered on the Atlas machine; POSIX systems export a few hundred, early UNIX about twenty). The trap instruction jumps into the kernel and raises privilege in one step; the OS finishes with return-from-trap, which does the reverse. Along the way, the hardware saves the caller’s registers — on x86, PC, flags and more are pushed onto the process’s per-process kernel stack kernel stack The per-process stack in the kernel where hardware saves user registers (PC, flags…) on a trap/interrupt, popped by return-from-trap. defined in ch. 6 — open in glossary — so return-from-trap can pop them and resume exactly where the program left off.

One crucial detail remains: where does the trap jump? The calling process can’t be allowed to name a kernel address — jumping just past a permission check would be, in the book’s words, a Very Bad Idea. Instead:

The kernel checks that the number is valid and runs the corresponding code — indirection as protection. And because installing the trap table is itself privileged, a user-mode attempt earns the usual reward (adios, offending program). Point to ponder, from the book: what horrible things could you do if you could install your own trap table?

Now the whole protocol, both phases, with every privileged step badged. This is the book’s most important table — worth stepping slowly:

Two phases: at boot, the kernel initializes the trap table and the CPU remembers where the handlers live. At run, the kernel sets up the process, launches it with a return-from-trap (a lovely trick: it fills the kernel stack so that “returning” lands at main() in user mode), the process traps in for services and gets return-from-trapped back out, and finally exits via a last trap. Efficient — the program mostly runs directly — yet the OS never loses control of what the program may do.

Except… one thing is still missing. What if the program simply never traps — an infinite loop, no system calls? Nothing in this protocol lets the OS take the CPU back. That’s Problem #2.