ยง33.5โ€“33.9A Problem: Blocking System Calls โ€ฆ Summary

Part II OSTEP pp. 405โ€“409 ยท ~7 min read

  • asynchronous i/o
  • manual stack management
  • continuation

The event loop looked wonderful in the last section โ€” no locks, full control. But it hides one sharp constraint.

33.5 A Problem: Blocking System Calls

What if a handler must issue a call that blocks? Say a request needs open() then read() on a file whose data isnโ€™t in memory โ€” those calls issue disk I/O and can take a long time. In a thread-based server this is fine: the blocked thread suspends and others run. But an event server has no other threads โ€” just the one loop:

The event model's Achilles' heel: one blocking read() freezes the single-threaded loop โ€” every other client waits.
t =
Event loop
Client B
tick 1 / 4 ยท one steprunning a handlerBLOCKED on disk I/Oready, but unservedfinally served

The loop starts handling fd_A, whose handler issues a blocking read() to disk.

When the loop blocks, the whole server sits idle. Hence the iron rule: no blocking calls allowed.

33.6 A Solution: Asynchronous I/O

The fix is asynchronous I/O (AIO): interfaces that issue an I/O and return control immediately, plus a way to check for completion. On a Mac you fill a struct aiocb, then issue the read:

struct aiocb {
    int           aio_fildes;  // file descriptor
    off_t         aio_offset;  // file offset
    volatile void *aio_buf;    // destination buffer
    size_t        aio_nbytes;  // length
};

int aio_read(struct aiocb *aiocbp);   // issue; returns immediately
int aio_error(struct aiocb *aiocbp);  // 0 = done, EINPROGRESS = not yet

aio_read() returns right away, so the loop keeps serving events; later, aio_error() tells you when the data has arrived. Trace the whole non-blocking flow:

Asynchronous I/O: issue the read and return at once, keep serving other events, and finish when the I/O completes.
Server(event loop)
OS
Disk
Issue the I/O โ€” and return immediately
Keep the loop alive
Detect completion
step 1 / 9 ยท time flows downward

Describe the read: which file, where, how much, and where to put the data.

Polling dozens of outstanding I/Os with aio_error() is tedious, so many systems instead deliver a UNIX signal (an interrupt) when an async I/O completes โ€” the same polling-vs-interrupts tension youโ€™ll meet again with I/O devices.

Aside: UNIX Signals

Signals let the OS (or another process) interrupt a process to run a signal handler, then resume. Each has a name โ€” HUP, INT, SEGVโ€ฆ Sometimes the kernel signals you: a bad memory access sends SIGSEGV. A tiny example that catches SIGHUP:

void handle(int arg) { printf("stop wakin' me up...\n"); }

int main() {
    signal(SIGHUP, handle);
    while (1) ; // catch some sigs
}
// prompt> kill -HUP <pid>   โ†’ runs handle()

33.7 Another Problem: State Management

Event code is simply harder to write than threaded code. In a thread, reading a file then writing the result to a socket is trivial:

int rc = read(fd, buffer, size);   // when this returns...
rc = write(sd, buffer, size);      // ...sd is right here on the stack

The socket sd sits on the threadโ€™s stack across the blocking read(). But in an event server, the handler issues the async read and returns โ€” its stack vanishes. Adya et al. call the fix manual stack management : save the state youโ€™ll need in a continuation :

Manual stack management: since the handler's stack vanishes when it returns, save what you need in a continuation.
aio_read(fd = 3)issue async readcontinuation table{ fd 3 โ†’ sd 7 }write(sd = 7)finish

1Issue the async read

The task: read from fd, then write the data to socket sd. The handler issues aio_read(fd) and returns to the event loop โ€” but its local variable sd is lost when the stack unwinds. How will the completion handler know which socket to write to?

step 1 / 3

33.8 What Is Still Difficult With Events

Async I/O rescues the model, but events remain awkward in several ways:

Even with async I/O, events stay tricky โ€” four lingering difficulties.
why it hurts
Multiple CPUsto use >1 CPU, handlers must run in parallel โ€” and then critical sections and LOCKS return
Paging (implicit blocking)a handler that page-faults BLOCKS โ€” even though you avoided explicit blocking calls
Changing API semanticsif a routine goes non-blocking โ†’ blocking, its calling handler must be split in two
Async I/O integrationdisk AIO never unifies cleanly with async network I/O
Dotted-underlined cells have explanations โ€” click one.

33.9 Summary

Event-based concurrency hands scheduling control to the application, at a real cost in complexity and integration (paging, multicore, API churn). No single approach has won โ€” threads and events both persist as two answers to the same concurrency problem. Learn both; reach for whichever fits.

Check yourself: blocking, async I/O, and continuations

1.Why is a single blocking system call disastrous in an event-based server, but fine in a thread-based one?

2.How does asynchronous I/O (aio_read) let an event loop avoid blocking?

3.After issuing an aio_read, how does the server learn the I/O has finished?

4.A handler issues an async read on fd and must later write the result to socket sd. Why does it need a continuation, and what is one?

5.Why does moving an event-based server to multiple CPUs erode its simplicity?

6.Even after avoiding explicit blocking calls, how can an event handler still block the whole server?

6 questions