§39.7–39.10Durability, Atomic Rename, Metadata, and Removing Files

Reading and writing is the core, but a real interface needs more: durability guarantees, safe updates, a way to inspect files, and a way to remove them.

39.7 Writing Immediately With fsync()

When a program calls write(), the file system usually doesn’t write to disk right away. For performance it buffers the data in memory (the page cache) and flushes it later — seconds later. So write() returns fast, and only a crash in that window loses the data. Most of the time that’s fine; but a database building a correct recovery protocol needs to force data to disk on demand. That’s fsync fsync A system call that forces all buffered ('dirty') data for a file descriptor to persistent storage and returns only once those writes complete. Needed by apps (e.g. databases) that require durability, since write() normally just buffers in memory for a few seconds. To durably create a file you often must fsync() its parent directory too. defined in ch. 39 — open in glossary (int fd): it forces all dirty (not-yet-written) data for a file to persistent storage and returns only once those writes complete.

int fd = open("foo", O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR);
write(fd, buffer, size);
fsync(fd);   // returns only once the data is durably on disk

The subtle catch: for a newly created file you often must also fsync() the parent directory, so the directory entry that names the file is on disk too. Overlooking this causes real application-level bugs.

39.8 Renaming Files, Atomically

mv foo bar calls rename(char *old, char *new). Its key guarantee: rename() is (usually) atomic with respect to crashes — if the machine dies mid-rename, the file ends up with either the old name or the new one, never a broken in-between. That makes it the backbone of the safe-update idiom an editor uses to change a file without risking corruption:

int fd = open("foo.txt.tmp", O_WRONLY | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR);
write(fd, buffer, size);   // write the NEW version to a temp file
fsync(fd);                 // force it to disk
close(fd);
rename("foo.txt.tmp", "foo.txt");  // atomically swap it into place

This is exactly the crash-consistency concern we met with RAID’s consistent-update problem: an update that must appear all-or- nothing. Journaling file systems (ch42) generalize it.

39.9 Getting Information About Files

The file system keeps a fair amount of data about each file — its metadata metadata Data about a file rather than its contents — size, inode number, owner/group, permissions, link count, and access/modify/change times. Retrieved with stat()/fstat() (the stat structure); stored in the file's inode. defined in ch. 39 — open in glossary . stat()/fstat() (given a pathname or descriptor) fill in a stat structure:

All of this lives in a per-file on-disk structure called an inode inode The persistent per-file data structure the file system keeps to hold a file's metadata — size, owner, permissions, timestamps, link count, and pointers to its data blocks. Named by its inode number. Inodes live on disk; active ones are cached in memory. (Its internals are the subject of the file-system-implementation chapters.) defined in ch. 39 — open in glossary — think of it, for now, as the persistent record holding everything above. Inodes live on disk; copies of active ones are cached in memory for speed. (We’ll dissect the inode itself when we build the file system.)

39.10 Removing Files

So how do you delete a file? rm traces to a single, oddly-named system call:

prompt> strace rm foo
...
unlink("foo")   = 0
...

unlink() unlink The system call that removes a name from a directory (what rm uses). It is called unlink, not delete, because it removes one link to an inode and decrements the inode's link count; the file's inode and data blocks are freed only when the count reaches zero. defined in ch. 39 — open in glossary takes just the file’s name and returns 0 on success. But why unlink, not remove or delete? That name is a genuine clue about how the file system works underneath — and to unravel it, we first need to understand hard links, which is exactly where the next page begins.