§49.7–49.9Idempotent Operations, Client Caching, and the Cache Consistency Problem

A stateless server plus self-contained requests set up NFS’s most elegant trick: one uniform way to survive every kind of failure.

49.7 Handling Server Failure with Idempotent Operations

When a client sends a request and no reply comes back, it can’t tell why — the request was lost, the server was down, or the reply was lost. NFSv2 doesn’t try to tell them apart. It just retries: set a timer on send; if it fires with no reply, resend.

Retrying blindly is only safe because most NFS operations are idempotent idempotency The property that performing an operation multiple times has exactly the same effect as performing it once — 'store value X to memory' is idempotent, while 'increment the counter' is not. It is the heart of NFS crash recovery: because most requests are idempotent (READ and LOOKUP trivially, and WRITE because it carries the exact offset), a client can safely retry after any failure — lost request, crashed server, or lost reply — in one uniform way. defined in ch. 49 — open in glossary — performing them many times has the same effect as performing them once:

Not everything is naturally idempotent, though. MKDIR isn’t: if the directory is created but the reply is lost, the retried MKDIR finds it already exists and reports failure — even though the operation really did succeed.

49.8 Improving Performance: Client-Side Caching

Sending every read and write across the (slow) network would be painful, so the NFS client caches file data and metadata in client memory: the first access pays the network cost, later ones are served locally. The cache also does write buffering write buffering Delaying writes in memory (typically 5–30 s) before committing them to disk. It lets the file system batch updates, schedule them efficiently, and even cancel them (create-then-delete). The cost is the durability/performance trade-off: buffered writes are lost if the system crashes first. defined in ch. 40 — open in glossary — an application’s write() returns as soon as the data lands in the client cache, and the data is sent to the server later. Great for performance — but multiple client caches create a big problem.

49.9 The Cache Consistency Problem

The cache consistency problem cache consistency problem The challenge that arises once multiple clients cache copies of the same file: keeping their views consistent as clients read and write. Illustrated with clients caching different versions of a file F, it splits into two subproblems — update visibility (when do one client's writes become visible elsewhere?) and the stale-cache problem (a client still holding an old cached copy after another has updated the server). defined in ch. 49 — open in glossary is clearest with three clients and a server:

There are two subproblems. Update visibility update visibility One half of the cache consistency problem: the question of when updates made by one client become visible to other clients. If a client buffers its writes in its local cache before propagating them to the server, other clients that read the file in the meantime will see an old version — until the buffered writes reach the server. defined in ch. 49 — open in glossary : while C2’s new version sits buffered in its cache, another client (C3) reading F gets the old F[v1] from the server. And the stale cache stale cache The other half of the cache consistency problem: after one client (C2) has flushed its update F[v2] to the server, another client (C1) may still hold the old version F[v1] in its cache. Unless C1 re-validates before reading, it returns outdated data instead of the latest copy. defined in ch. 49 — open in glossary : even after C2 flushes F[v2] to the server, C1 still holds F[v1] in its cache and would read it. NFSv2 patches both:

The flush-on-close flush-on-close NFS's fix for update visibility, also called close-to-open consistency: when a client finishes writing a file and closes it, the client-side file system flushes all of that file's dirty cached pages to the server. This guarantees that a subsequent open of the file on another client will see the writer's latest version. defined in ch. 49 — open in glossary rule (a.k.a. close-to-open) handles visibility; a GETATTR last-modified check handles staleness — softened by an attribute cache attribute cache A client-side cache of file attributes, with a short timeout (e.g. 3 seconds), added to NFS so that validating a cached file usually needs no network round-trip. It exists because the naive stale-cache fix — issuing a GETATTR to check the last-modified time before every cached access — flooded the server; the attribute cache lets a client trust its cached copy for the timeout window, at the cost of occasionally using a slightly stale file. defined in ch. 49 — open in glossary so clients don’t bombard the server with “has this changed?” on every access. It works, mostly — but as we’ll see next, these are engineering patches, and the attribute-cache timeout means the implementation ends up defining the semantics.