We know how to use a file system (ch39). Now we build one: vsfs, the Very Simple File System β a stripped-down UNIX-style FS. A file system is pure software, so the design space is enormous (AFS to ZFS); we learn by case study, starting simple.
The Crux: How To Implement A Simple File System
What on-disk structures does a file system need? What do they track? And how are they accessed during opens, reads, and writes?40.1 The Way To Think
Understand two things and you understand a file system:
- Data structures β what on-disk structures organize its data and metadata (vsfs uses simple arrays of blocks; fancier systems use trees).
- Access methods β how it maps
open(),read(),write()onto reads and writes of those structures.
The rest of this chapter develops both. Build your mental model: whatβs on disk, and what gets touched on each call.
40.2 Overall Organization
Start by dividing the disk into fixed-size blocks block The fixed-size unit a file system divides its disk region into and allocates in (commonly 4 KB) β distinct from a hardware disk sector (usually 512 bytes). vsfs addresses its partition as blocks 0β¦Nβ1; the on-disk layout is a sequence of these blocks. defined in ch. 40 β open in glossary β 4 KB each (distinct from the driveβs 512-byte sectors). On a tiny 64-block partition, we assemble the layout region by region:
1Divide the disk into blocks
A simple file system uses one block size β here 4 KB. Our tiny partition is 64 blocks, addressed 0β¦63. Everything the FS stores must fit in these blocks.
The final layout: block 0 is the superblock superblock A block (usually the first) holding file-system-wide info: how many inodes and data blocks there are, where the inode table starts, and a magic number identifying the file-system type. On mount, the OS reads the superblock first to initialize the volume.
defined in ch. 40 β open in glossary
(S),
blocks 1β2 are the inode and data bitmaps bitmap An allocation structure: one bit per object marking it free (0) or in-use (1). vsfs keeps an inode bitmap (for the inode table) and a data bitmap (for the data region). Bitmaps are consulted and updated only when allocating or freeing β never during a plain read.
defined in ch. 40 β open in glossary
(i, d),
blocks 3β7 are the inode table inode table The on-disk region holding an array of inodes. Because inodes are fixed-size (e.g. 256 bytes, 16 per 4 KB block), the file system computes an inode's disk location directly from its i-number: blk = (inumber Β· sizeof(inode)) / blockSize, offset from the table's start.
defined in ch. 40 β open in glossary
(I), and blocks
8β63 are the data blocks data block A block in the file system's data region that holds actual user file contents (or a directory's entries, or an indirect block of pointers). Most of a disk is data blocks; the inode records which ones belong to a file.
defined in ch. 40 β open in glossary
(D). A bitmap is dead
simple β one bit per object, 0 = free, 1 = in-use β one bitmap for inodes,
one for data.
40.3 File Organization: The Inode
The inode (βindex nodeβ) is the key structure. Each is named by its i-number, and β because inodes are fixed-size and packed in an array β the FS computes an inodeβs disk location directly from that number:
For inode 32 (256 B inodes, inode table starting at 12 KB): the byte offset is 32 Γ 256 = 8192, so the block sits at 12 KB + 8 KB = 20 KB, i.e. disk sector 20 KB / 512 = 40. Inside the inode is the fileβs metadata plus its block pointers:
| size | field | purpose | |
|---|---|---|---|
| mode | 2 | mode | read/write/execute permissions |
| uid/gid | 2+2 | uid / gid | owner user and group |
| size | 4 | size | file length in bytes |
| times | 4Γ4 | time / ctime / mtime / dtime | access / create / modify / delete times |
| links_count | 2 | links_count | number of hard links |
| blocks | 4 | blocks | number of blocks allocated |
| block | 60 | block | 15 disk pointers (12 direct + single/double/triple indirect) |
The multi-level index
How does the inode record where the data blocks are? A few direct pointers direct pointer A disk address stored directly in the inode that points at one of the file's data blocks. An inode holds a small fixed number of them (e.g. 12); together they cover small files entirely β which is most files. defined in ch. 40 β open in glossary (disk addresses) live right in the inode β enough for small files. For bigger files, an indirect pointer indirect pointer An inode pointer to an indirect block β a data block full of MORE pointers to data blocks (not user data). With 4 KB blocks and 4-byte addresses, one indirect block adds 1024 pointers. Double- and triple-indirect pointers add further levels for very large files. defined in ch. 40 β open in glossary points to an indirect block filled with 1024 more pointers; a double indirect pointer adds a level (1024 Γ 1024 blocks), and a triple adds another. This imbalanced tree is the multi-level index multi-level index The imbalanced-tree scheme for locating a file's blocks: a few direct pointers, then single, double, and triple indirect pointers. It optimizes for the common case (most files are small β direct pointers suffice) while still supporting huge files. Used by ext2/ext3, the original UNIX FS, and WAFL. defined in ch. 40 β open in glossary :
With 12 direct + single + double indirect pointers, 4 KB blocks and 4-byte addresses, a file can reach:
Why this lopsided design? Because of a truth that holds across the decades β most files are small β so itβs worth making the common case (a handful of direct pointers) cheap, and paying the indirect-block cost only for the rare big file:
| finding | detail | |
|---|---|---|
| small | Most files are small | ~2 KB is the most common size |
| growing | Average file size is growing | almost 200 KB on average |
| bytes | Most bytes are in large files | a few big files use most of the space |
| many | File systems hold lots of files | ~100K on average |
| half | File systems are ~half full | stays ~50% full even as disks grow |
| dirs | Directories are small | most have 20 or fewer entries |
Tip: Consider Extent-Based Approaches
Instead of a pointer per block, an extent extent An alternative to per-block pointers: a disk pointer plus a length (in blocks), describing a contiguous run of a file. Extent-based file systems (XFS, ext4) allow several extents per file. More compact than pointers but less flexible; works best when files can be laid out contiguously. defined in ch. 40 β open in glossary is a disk pointer plus a length (in blocks) β one extent describes a whole contiguous run. Extent-based file systems (XFS, ext4) allow several extents per file. Theyβre less flexible than pointers but far more compact, and work well when files can be laid out contiguously β which allocation policy aims for anyway.Check yourself: vsfs layout and the inode
1.What two aspects should you understand to grasp any file system?
2.In the vsfs layout (S i d I I I I I Dβ¦D), what is the superblock (S) for?
3.Why can the file system compute an inode's disk location directly from its i-number?
4.How does a bitmap track allocation, and when is it consulted?
5.How do direct and indirect pointers differ in the multi-level index?
6.Why is the multi-level index an imbalanced tree (a few direct pointers, then indirect levels)?