COMP 3150Operating Systems

01 Introduction

Unit 9 closed with thrashing, the failure mode of virtual memory. This unit turns from memory to a different resource: persistent storage. A computer's memory is volatile — its contents are lost when the machine is powered off — but the data that users care about must survive across sessions, across reboots, and across the years. That data lives on secondary storage: magnetic disks, solid-state drives, and the network and cloud storage that sits behind them. And the way it is organised on that storage is the file system.

The file system is the operating system's most visible abstraction to the user. Every program reads and writes files; every command-line session manipulates them; every graphical user interface presents them as icons and folders. The file system is what allows a user to think of their data as a set of named documents and photographs rather than as a collection of bytes on a disk, and it is what allows a program to open a file by name without knowing where on the disk its data is stored.

The file system rests on two central abstractions. The first is the file: a named, persistent sequence of bytes. A file hides the details of the storage device — the disk's sectors, the SSD's flash pages, the network's packet boundaries — behind a uniform interface: a file is a stream of bytes that can be read from the beginning, written to the end, or, with some file systems, modified at an arbitrary position. The second abstraction is the directory: a mapping from names to files. A directory organises files into a hierarchy, gives them human-readable names, and allows the file system to scale from a handful of files to billions.

This lesson presents both abstractions in detail. It defines the file and its attributes, presents the standard file operations, and surveys the types of files that a modern Unix-like system provides. It defines the directory and its operations, presents the classical directory structures from the single-level directory of early systems to the tree and graph structures of modern systems, and explains how path resolution works. It presents the distinction between hard links and symbolic links, the mechanisms for file sharing and protection, and the concept of mounting — the mechanism by which multiple file systems are combined into a single hierarchy. It closes with a tour of the Linux file-system hierarchy, which is the standard organisation for Unix-like systems and the basis for the specific file systems examined in the rest of the unit.

The lesson is the first of seven in Unit 10. The lessons that follow present file metadata (Lesson 10.2), inodes (Lesson 10.3), directory organization in detail (Lesson 10.4), file allocation (Lesson 10.5), journaling (Lesson 10.6), and modern file systems (Lesson 10.7). This lesson provides the vocabulary and the conceptual framework for those lessons; it is the foundation on which the rest of the unit builds.

Prerequisites

This lesson opens Module 4, so it has no prerequisites within the file-systems unit itself. It builds on the storage-hierarchy material from Lesson 8.1 (the distinction between volatile memory and persistent storage, and the enormous latency gap between them) and on the process model from Unit 3 (a process's open-file table, file descriptors, and the fork and exec semantics that inherit them). The discussion of allocation and free-space management in section 4.10 assumes the fragmentation material from Lesson 8.5, but no new fragmentation concepts are introduced. Familiarity with the Linux command line from Lesson 2.2 is assumed, since the interactive activity uses standard Unix tools.

02 Learning objectives

By the end of this lesson you will be able to

  • Define a file and describe its attributes, operations, and types.
  • Define a directory and describe its operations and its role in the file-system hierarchy.
  • Compare the classical directory structures — single-level, two-level, tree, acyclic graph, and general graph.
  • Explain path resolution and distinguish absolute from relative paths, and hard links from symbolic links.
  • Describe file sharing, protection, and mounting, and explain the layout of the Linux file-system hierarchy.

This lesson contributes to U10.1 and U10.2, and to course learning outcomes CLO1, CLO5, and CLO6.

03 Required reading

  • Operating System Concepts — Chapter 13, Sections 13.1–13.3, File Concept, Access Methods, and Directory and Disk Structure. The standard treatment of file attributes, operations, types, access methods, and directory structures.
  • OSTEP — Chapter 39, Files and Directories. The clearest available treatment of the file and directory abstractions from the programmer's perspective, with the POSIX interface presented through the system calls that implement it. OSTEP's treatment is the basis for the worked example in this lesson.
  • Tanenbaum & Bos — Modern Operating Systems, Chapter 4, Sections 4.1–4.2, Files and Directories. A complementary treatment that develops the file abstraction from first principles, with the historical context of the single-level and two-level directory structures.
  • Silberschatz et al. — Chapter 13, Sections 13.1–13.3, and Chapter 14, Sections 14.1–14.3, File-System Structure, File-System Implementation, and Directory Implementation. Read the discussion of directory implementation and the mounted file-system structure.
  • Bryant, R. E., & O'Hallaron, D. R. — Computer Systems: A Programmer's Perspective, 3rd ed., Chapter 10, System-Level I/O. The programmer's view of files, file descriptors, and the Unix I/O interface.
  • Kerrisk, M. — The Linux Programming Interface, Chapters 1–5 and 14–18. The definitive reference for the Linux file interface, including file descriptors, files and directories, file systems, and extended attributes.
  • Filesystem Hierarchy Standard (FHS) — version 3.0. The specification of the standard Linux directory layout, available online.
  • Linux man pages: man 7 path_resolution, man 7 symlink, man 2 open, man 2 read, man 2 write, man 2 close, man 2 lseek, man 2 stat, man 2 link, man 2 symlink, man 2 mkdir, man 2 rmdir, man 2 unlink, man 2 rename, man 2 chdir, man 2 getcwd, man 2 mount, man 3 readdir, man 7 hier, man 8 mount.

Reading strategy

Read the Silberschatz section first for the formal definitions and the classification of file attributes, operations, and types. Then read the OSTEP chapter for the programmer's view: how the abstractions are realised by the POSIX system calls, what the system calls guarantee and what they leave undefined, and why the interface is designed the way it is. The two views complement each other: the textbook's treatment is concerned with the file system's design, and OSTEP's is concerned with the programmer's use of it. Keep the man 7 path_resolution page open as a reference for the path-resolution discussion; it is the authoritative specification and it is more precise than any textbook treatment.

04 Core content

4.1 The file abstraction

A file is a named, persistent sequence of bytes managed by the operating system. The definition has three important parts.

  • Named. A file has a human-readable name — report.txt, photo.jpg, main.c — which distinguishes it from every other file in its directory. The name is what the user types to refer to the file; the file system maintains a mapping from names to the file's storage.
  • Persistent. A file's contents survive across the termination of the process that created them, across the closing of the file, across reboots, and across the lifetime of the operating system. This distinguishes files from the memory that a process uses: memory is volatile, files are not.
  • Sequence of bytes. A file is a stream: a sequence of bytes with a defined order. Most files do not have an internal structure that the operating system knows about — a text file, an image, an executable, and a database table are all just sequences of bytes as far as the file system is concerned. The meaning of the bytes is provided by the programs that interpret them, not by the file system itself.

The abstraction is deliberately minimal. It says nothing about what the bytes mean, how they are stored, or how they are accessed. Those decisions are left to the applications, which allows the file system to be general and the storage device to be changed without requiring any changes to the applications.

4.2 File attributes

A file's attributes are the metadata associated with it: information about the file rather than the file's contents. The attributes vary between file systems, but the following are universal.

Standard file attributes
Attribute Description
Name The human-readable name by which the file is known. Stored in the directory, not in the file's metadata.
Identifier A unique number that identifies the file within the file system. On Unix, this is the inode number, examined in detail in Lesson 10.3.
Type The kind of file: regular, directory, device, socket, and so on.
Location A pointer to the storage that holds the file's contents. The details depend on the file system's allocation strategy (Lesson 10.5).
Size The current size of the file, in bytes. Some file systems also track the space the file occupies on disk, which may be larger because of block alignment.
Protection The access permissions: who may read, write, or execute the file. On Unix, this is a combination of owner, group, and other permission bits.
Owner The user (and group) that owns the file. Used for permission checks and for accounting.
Timestamps The times of the file's creation, last modification, and last access. Most modern systems track modification and access times; some also track a change time and a creation time.

The attributes are stored in the file system's metadata structures, not in the file itself. On Unix, they live in the file's inode (Lesson 10.3), which is a separate data structure from the file's data blocks. This separation is fundamental: it allows the file system to look up a file's attributes without reading the file's contents, and it allows the file's contents to be stored anywhere on the disk without affecting how the file is located by name.

4.3 File operations

The standard operations that a file system provides are the same across almost every system. On POSIX, they are exposed through system calls.

open
Open an existing file (or create a new one) and return a file descriptor — a small integer that identifies the open file within the process. The file descriptor is the process's handle for subsequent operations. The open call takes a set of flags (read-only, write-only, read-write, create, truncate, append) that determine how the file may be used.
read
Read a specified number of bytes from the file into a buffer, starting at the current position and advancing the position by the number of bytes read. The call returns the number of bytes actually read, which may be less than requested (at end of file, or if the read was interrupted).
write
Write a specified number of bytes from a buffer to the file, starting at the current position and advancing the position. The call returns the number of bytes actually written.
lseek
Move the file's current position to a specified offset. Used for random access: reading or writing at an arbitrary position within the file, rather than sequentially.
close
Close the file and release its file descriptor. When a process terminates, the operating system closes all of its open files automatically.
unlink
Remove the file's directory entry. If no other hard links to the file exist and no process has it open, the file's data blocks are freed and the file ceases to exist. If other links exist or the file is open, only the name is removed; the file's contents persist until the last reference is gone.
rename
Change a file's name, or move it to a different directory. On POSIX, the operation is atomic: a reader either sees the old name or the new one, never an intermediate state.
truncate
Reduce a file's size to zero (or to a specified length), discarding the contents beyond the new length. The file's attributes are preserved; only its contents are removed.

The operations are collected in the open-file table, which is the operating system's record of the files that are currently open. On Unix, the table has two parts: a per-process table that maps file descriptors to entries in a system-wide table, and the system-wide table that holds the file's open state (its current position, its access mode, and a reference to the file's inode). The two-level structure allows multiple processes to share the same open file while maintaining independent file descriptors, which is what makes the fork semantics work: a child inherits its parent's file descriptors, and the two processes share the same open-file entries, including the current position.

The open and close operations exist for efficiency. If the file system had to look up the file's attributes on every read or write, the overhead would be prohibitive. By opening the file once and holding a file descriptor, the program amortises the lookup over many operations. The open call also performs the access-permission check once, rather than on every read or write, which simplifies the code and improves performance.

4.4 File types

Unix-like systems distinguish several kinds of file. Every file has a type, recorded in its metadata, that determines how it behaves.

File types on a Unix-like system
Type Description Example
Regular file An ordinary file: a sequence of bytes with no special meaning to the file system. Text files, images, executables, and databases are all regular files. report.txt, /bin/ls
Directory A file that contains a mapping from names to files. Directories are themselves files, and can be read (with restrictions) as a sequence of directory entries. /home, /etc
Symbolic link A file whose contents are the pathname of another file. When the operating system encounters a symbolic link, it substitutes the link's contents and continues from there. /usr/bin/cc → gcc
Character device A file that represents a device accessed as a stream of bytes: a terminal, a serial port, a pseudo-terminal. /dev/tty, /dev/null
Block device A file that represents a device accessed in fixed-size blocks: a disk, a partition, an SSD. Reading from a block device reads a block; writing to it writes a block. /dev/sda, /dev/nvme0n1
FIFO (named pipe) A file that provides inter-process communication through a named channel. Two processes can open the same FIFO; one writes, the other reads. /run/systemd/initctl/fifo
Socket A file that represents a communication endpoint. Unix-domain sockets are bound to a name in the file system and allow processes to communicate through the socket interface. /run/docker.sock

The variety of file types reflects Unix's design philosophy: everything is a file. Devices, communication endpoints, and ordinary data are all presented through the same interface — the file descriptor — so that a program can read from a disk, a terminal, and a network connection using the same system calls. This uniformity simplifies programs enormously and is one of the reasons Unix-like systems have been so influential.

4.5 File access methods

The way a program accesses a file's contents is determined by the access method. The standard methods are:

  • Sequential access. The file is read or written from beginning to end, in order. Each read or write advances the file's position by the number of bytes transferred. The classic example is a text file: a program reads it line by line, processing each line in order. Sequential access is the default on Unix-like systems and is what a program gets when it uses read and write without seeking.
  • Direct (or random) access. The file is read or written at an arbitrary position, selected by the program. Each access specifies the position explicitly, and the file's internal order does not constrain the access order. The classic example is a database file: a program seeks to the record it wants, reads it, and seeks to the next record, which may be anywhere in the file. On Unix, direct access is provided through the lseek system call and through the pread and pwrite calls, which combine the seek and the read or write into a single operation.
  • Indexed access. The file has an index that maps a key to a position, and the program accesses the file through the index. This is not a file-system feature in the classical Unix model — the file system does not know about the index — but it is provided by the program or by the file system on some systems (mainframe systems had native indexed files). On Unix, the index is typically maintained by the application, using a separate index file and the file system's direct access.

The distinction between sequential and direct access is important for the file system's implementation. A sequential access is the file system's ideal: it can read ahead (as in Lesson 9.4's demand paging) and optimise for the streaming access pattern. A direct access is more expensive, because the file system cannot predict where the next access will be and cannot prefetch. Most modern file systems therefore perform much better on sequential workloads than on random ones, and programs that can be structured to access files sequentially are usually much faster than programs that seek randomly.

4.6 The directory abstraction

A directory is a file that contains a mapping from names to files. It is the mechanism by which the file system provides human-readable names and organises files into a hierarchy.

The mapping is from a name (a string) to a file's identity (its metadata, typically an inode number). When a program asks the file system to open report.txt, the file system looks up the name in the current directory, obtains the inode number, and uses the inode number to find the file's metadata and data. The name is only an entry in the directory; the file itself is identified by its inode, which is independent of the name.

This separation between names and identity is central to the design of Unix file systems. It allows a single file to have multiple names (through hard links, as we will see in section 4.10), it allows a file to be renamed without affecting its identity, and it makes the file system's directory operations simple: adding a name is adding an entry, removing a name is removing an entry, and neither affects the file's contents.

4.7 Directory operations

The standard operations on a directory are the same across systems and mirror the operations on files, with the addition of operations specific to the name-to-file mapping.

mkdir
Create a new, empty directory. The new directory is added as an entry to an existing directory (its parent).
rmdir
Remove an empty directory. If the directory contains any files, the operation fails; the caller must remove the entries first or use a recursive delete. This restriction prevents accidental deletion of files by a single rmdir call.
opendir and readdir
Open a directory and read its entries one at a time. On Unix, the directory is read as a sequence of entries, each consisting of a name and an inode number. The order of the entries is unspecified; it is typically the order in which they are stored in the directory, which depends on the file system's internal structure (Lesson 10.4).
link and unlink
Add or remove a name-to-file mapping. The link call creates a new name for an existing file (a hard link); the unlink call removes a name. These are the operations that change the directory's contents without changing the files themselves.
rename
Change a file's name, or move it between directories. The rename operation is atomic on POSIX systems: a reader either sees the old state or the new one, never an intermediate state. This makes rename useful for atomic file updates: a program writes to a temporary file, then renames it over the original, so readers never see a half-written file.
chdir and getcwd
Change and query the process's current working directory (section 4.9). The working directory is the starting point for relative path resolution.

4.8 Directory structures

The file system's directory structure determines how files are organised and how the file system scales. The classical structures, in order of increasing complexity, are the following.

Single-level directory
All files are in a single directory. The structure is simple and easy to understand, but it has serious limitations. All file names must be unique, so a user cannot have two files with the same name (a common problem when two users submit files with the same name). As the number of files grows, the directory becomes unwieldy, and finding a file by name becomes slow. The single-level directory was used in early systems with a small number of files, but it does not scale.
Two-level directory
Each user has a separate directory, and the file system has a master directory containing one entry per user. The user's directory is a single-level directory containing the user's files. The structure solves the name-collision problem (two users can have files with the same name) and simplifies the file system's access control (a user's files are in one place). But it does not allow users to organise their own files into subdirectories, so a user with many files faces the same problem as the single-level directory.
Tree-structured directory
The directory hierarchy is a tree: each directory may contain files and subdirectories, and every file has a unique path from the root. The tree structure is the standard for modern file systems. It allows users to organise their files as they see fit, with as many levels of nesting as they need. It provides natural grouping (related files in the same directory) and natural paths (the sequence of directory names from the root). The tree has one important property: each file has a unique path, and the directory structure is acyclic. This simplifies the file system's algorithms but restricts the sharing of files between users (a file can only appear in one place).
Acyclic graph directory
The directory structure is a directed acyclic graph: a file or directory may have multiple parents, but the structure has no cycles. The structure is created by hard links (which add a new name for an existing file) and by symbolic links (which add a file whose contents are a path to another file). It allows sharing (a file can appear in multiple directories) without the complications of cycles. The acyclic graph is what most Unix-like file systems provide: the tree structure is the base, and hard and symbolic links extend it into a graph.
General graph directory
The directory structure may contain cycles: a directory may contain a reference to one of its ancestors, or to itself. The general graph allows arbitrary sharing and is what is used in some distributed file systems and in network file systems that mount remote directories into the local hierarchy. It has a significant drawback: operations that traverse the directory structure — a recursive delete, a recursive search, a disk-space accounting — must detect and handle cycles, which complicates the file system's implementation. Cycle detection requires either a visited-set or a count of visits, both of which are expensive. For this reason, cycles are avoided in most Unix-like file systems, and the acyclic graph is the structure that is actually provided.
Directory structures: single-level, two-level, tree, and acyclic graph Four panels comparing directory structures. The first panel shows a single-level directory with all files in one directory. The second panel shows a two-level directory with one subdirectory per user. The third panel shows a tree structure with a root directory and nested subdirectories. The fourth panel shows an acyclic graph structure in which a file can have multiple parents through hard links and symbolic links. DIRECTORY STRUCTURES Single-level / (root) file1 file2 file3 all files in one directory name collisions; does not scale Two-level / (root) alice bob carol one directory per user solves name collisions; simple access control Tree / (root) home etc var alice bob file file file each file has a unique path from the root Acyclic graph (with links) / (root) dir1 dir2 dir3 shared symlink a file may have multiple parents hard and symbolic links create the graph no cycles: every traversal terminates root directory file symbolic link
Figure 10.1 — Directory structures. The single-level directory has all files in one directory, with name collisions and poor scalability. The two-level directory gives each user a separate directory, solving collisions but not allowing organisation within a user's space. The tree structure allows arbitrary nesting with a unique path for each file. The acyclic graph extends the tree with hard and symbolic links, allowing a file to have multiple parents while keeping traversals terminating.

4.9 Paths and path resolution

A file's location in the directory structure is given by its path: the sequence of directory names from the root to the file, with the names separated by a slash character (/). The path /home/alice/report.txt means: start at the root directory, find the entry for home, find the entry for alice in that directory, and find the entry for report.txt in that directory. The file is identified by that entry.

A path that starts with a slash is an absolute path: it begins at the root and its meaning does not depend on the process's context. A path that does not start with a slash is a relative path: it begins at the process's current working directory, a per-process attribute that can be changed with the chdir system call. The working directory is the process's context for path resolution; it is what allows a program to refer to files by short names (report.txt) without specifying the full path from the root.

The special entries . and .. are present in every directory. . refers to the directory itself, and .. refers to its parent. They allow relative paths to move up and down the hierarchy: ../sibling/file.txt refers to a file in the parent's sibling directory, and ./file.txt refers to a file in the current directory. These entries are created automatically when a directory is created, and they are maintained by the kernel.

Path resolution is the process by which the kernel converts a path into a reference to a file. The kernel starts at the root (for an absolute path) or the working directory (for a relative path), and it processes each component in turn:

  1. The kernel looks up the current component's name in the current directory, obtaining the entry's inode number.
  2. It reads the inode for that entry, checking that the entry is a directory (if there are further components to process) and that the process has execute permission on the directory.
  3. If there are further components, the kernel descends into the directory: the current directory becomes the directory the inode refers to, and the kernel continues with the next component.
  4. When all components have been processed, the kernel has a reference to the final file. It checks the process's permissions on the final file (read, write, or execute, according to the operation being performed) and either proceeds with the operation or returns an error.

The resolution can fail at any step: a name may not exist in the directory, a component may not be a directory, or the process may lack permission to search the directory. In each case the kernel returns an appropriate error (ENOENT for a missing file, ENOTDIR for a non-directory in the middle of a path, EACCES for a permission problem) and the operation fails. The path-resolution process is one of the most frequently executed in a Unix-like system: every file open, every stat, every directory change goes through it.

Absolute paths and security

The choice between absolute and relative paths has a security dimension that is easy to overlook. A privileged program that opens a file using a relative path is exposed to the possibility that the file's meaning has been changed by an attacker who has modified the process's working directory or its environment. A privileged program that opens a file using an absolute path avoids that risk, because the path's meaning does not depend on the process's context. This is one reason the Linux kernel's own internal file opens use absolute paths.

4.10 Hard links and symbolic links

Unix-like file systems provide two mechanisms for giving a file more than one name. They differ in how they relate to the underlying file.

Hard link
A hard link is a directory entry that refers to an existing file's inode. Creating a hard link adds a new name to the directory, pointing to the same inode as an existing file. The two names are equivalent: they refer to the same file, and the file's contents are shared. The file system tracks the number of hard links to a file (the link count), and the file is deleted only when the last link is removed. Hard links have two important restrictions. First, they cannot cross file-system boundaries, because an inode number is meaningful only within its own file system. Second, they cannot refer to a directory (except for the special . and .. entries), because allowing directory hard links would introduce cycles into the directory structure. Within these restrictions, hard links are the classical mechanism for sharing a file among multiple names, and the same file can appear in multiple directories on the same file system.
Symbolic link (soft link)
A symbolic link is a file whose contents are a pathname. It is an indirect reference: the link's pathname is resolved independently of the link itself, and the resolution may produce a different file each time. A symbolic link can cross file-system boundaries, because it stores a pathname rather than an inode number. It can refer to a directory, because the kernel does not create a cycle at the inode level. And it can be dangling: a symbolic link whose target does not exist is a valid file; the error occurs only when the link is dereferenced. The cost is that dereferencing a symbolic link requires extra work: the kernel must read the link's contents and resolve the path, which may itself involve more symbolic links (a chain) or even cycles (which the kernel detects by limiting the resolution depth).

The choice between hard and symbolic links depends on the requirements. A hard link is efficient (no additional resolution), it is indistinguishable from the original file (both names refer to the same inode), and it is the right choice when a file should have two equivalent names within the same file system. A symbolic link is more flexible (it can cross file-system boundaries, refer to directories, and dangle), it is visible (the link's target is stored as a pathname, and commands like ls -l show it explicitly), and it is the right choice when the reference should be indirect: for example, /usr/bin/cc is usually a symbolic link to the current default C compiler, and it is updated when the default changes.

The distinction between hard links and symbolic links also affects how the file system handles the deletion of a file. Deleting a hard link removes only that name; the file's contents are freed only when the link count reaches zero and no process has the file open. Deleting a symbolic link removes the link itself; the target file is unaffected. And creating a hard link is not possible if the target does not exist (the inode must exist); a symbolic link to a nonexistent file is perfectly legal.

4.11 File sharing and protection

On a multi-user system, files are shared among users and among processes. The file system must provide mechanisms for sharing while protecting files from unauthorised access.

The classical Unix protection mechanism is the permission bits. Each file has three sets of three bits:

  • Owner permissions — the rights of the file's owner.
  • Group permissions — the rights of the members of the file's group.
  • Other permissions — the rights of everyone else.

Each set contains three bits: read, write, and execute. A file's permissions are written as a nine-bit string, or as three octal digits. For example, rwxr-xr-x (or 755) means the owner may read, write, and execute the file; the group may read and execute but not write; and others may read and execute but not write. The permission bits are checked by the kernel on every access, at the file's inode, and the check is what enforces the system's access-control policy.

Modern Unix-like systems also support access control lists (ACLs), which allow more fine-grained permissions: a file may be readable by one specific user, writable by another, and completely inaccessible to everyone else. ACLs are an extension of the classical permission model; a file with an ACL still has the classical permission bits, and the ACL is consulted only if the classical bits do not already permit the access. ACLs are used in systems that require finer-grained control than the classical model provides, and they are supported by most modern Unix file systems.

File sharing also has a per-process dimension. When a process opens a file, the kernel creates an entry in the process's file descriptor table that refers to an entry in the system-wide open-file table. The open-file table entry records the file's current position, access mode, and a reference to the file's inode. When a process forks, the child inherits the parent's file descriptor table, and the two processes share the same open-file table entries. This means that a read by the parent advances the file position for the child as well, and vice versa. The behaviour is what allows a shell pipeline to work: the shell forks a child for each command in the pipeline, and the children share their standard input and output, which are the connections between the commands.

4.12 Mounting and the file-system hierarchy

A modern system usually has several file systems: the root file system, a separate file system for /home, a separate file system for /var, a temporary file system for /tmp, and possibly several remote file systems mounted from other machines. The operating system combines them into a single, unified hierarchy through the process of mounting.

Mounting attaches a file system to a directory in an existing file system. The directory is called the mount point, and after the mount, the contents of the mounted file system appear at the mount point. A process that accesses /home after /home has been mounted sees the contents of the mounted file system, not the contents of the underlying directory on the root file system. The underlying directory is hidden — inaccessible — until the file system is unmounted.

The mount operation is implemented by the kernel, and its state is recorded in a mount table. On Linux, the mount table is visible in /proc/self/mounts (or /proc/mounts), and the mounts can be manipulated with the mount and umount commands (with root privileges). The mount table records, for each mount, the file system's type, the device that holds it, the mount point, and the mount options (read-only, no-execute, and so on).

Mounting is what allows Unix-like systems to present a single, uniform hierarchy even when the underlying storage is heterogeneous. The user sees one tree, rooted at /, and does not need to know that /home is on one disk and /var is on another, or that /mnt/backup is on a network server. This transparency is one of the reasons Unix-like systems are so convenient to use, and it is one of the reasons the mount mechanism has been copied by other operating systems.

The Linux file-system hierarchy is standardised by the Filesystem Hierarchy Standard (FHS), which specifies the purpose of each top-level directory. The standard layout includes:

The Linux file-system hierarchy
Directory Purpose
/ The root of the hierarchy. All other directories are reached through it.
/bin Essential user binaries. Commands that are needed to bring the system up and to repair it.
/boot The files needed to boot the system: the kernel image, the initial RAM disk, and the boot loader configuration.
/dev Device files. A file for each device on the system, including character and block devices.
/etc System-wide configuration files. Almost every system service reads its configuration from /etc.
/home User home directories. Each user typically has a subdirectory containing their personal files.
/lib Shared libraries needed by the binaries in /bin and /sbin. The location of the C library, the dynamic linker, and the kernel modules.
/media Removable media. Mount points for USB drives, optical discs, and other removable storage.
/mnt Temporary mount points. Used by administrators to mount file systems manually.
/opt Optional application software. Used for self-contained software packages that do not fit the standard layout.
/proc A virtual file system that exposes information about the kernel and the running processes. Not a real file system: its contents are generated on demand.
/root The home directory of the root user. By convention, separate from /home.
/sbin System binaries. Commands that are intended for administrative use and are usually run by root.
/sys A virtual file system that exposes the kernel's view of devices and their attributes.
/tmp Temporary files. Usually on a memory-backed file system or a small partition, and cleared on reboot.
/usr User system resources. A large hierarchy containing most of the system's installed software, with subdirectories for binaries (/usr/bin), libraries (/usr/lib), documentation (/usr/share), and local installations (/usr/local).
/var Variable data. Files that change as the system runs: log files, mail spools, printer queues, and the temporary state of running services.

The layout is not arbitrary. It separates files by their role: files that are essential for booting the system (in /bin, /lib, /etc), files that are installed by the system's package manager (in /usr), files that change as the system runs (in /var), and files that belong to users (in /home). The separation is designed to allow different directories to be stored on different file systems with different properties: /usr might be read-only (it changes only when software is installed), /var is writable and may be on a different disk, and /home may be on a network file server. The layout is one of the reasons a Unix-like system can be administered effectively even when its storage is distributed across many devices.

4.13 Common misconceptions

"A file's name is stored in the file."

The name is stored in the directory, not in the file's metadata. The file is identified by its inode number, and the same file can have multiple names (hard links) within the same file system.

"A file is a sequence of records."

In the Unix model, a file is a sequence of bytes. The file system does not impose any record structure; records are a convention of the applications that use the file. This is a change from early mainframe systems, which often had record-oriented file systems.

"Directories are not files."

On Unix, directories are files. They are files whose contents are a sequence of directory entries, and they are subject to the same permission checks (though with restrictions on what operations are permitted). This uniformity is one of the central ideas of the Unix file model.

"Deleting a file always frees its space."

Deleting a file removes its directory entry. The file's data blocks are freed only when the link count reaches zero and no process has the file open. A process that is still holding a file open after unlink continues to access the file's contents, and the space is reclaimed only when the process closes the file.

"A symbolic link is a special kind of file."

A symbolic link is a file, but its contents are a pathname. It is not the target file; it is a reference to the target file. The distinction matters for operations that act on the link itself (lstat, unlink) versus operations that follow it (stat, open).

"Hard links and symbolic links are interchangeable."

They are not. Hard links cannot cross file-system boundaries and cannot refer to directories. Symbolic links can do both and can dangle. Hard links are equivalent references; symbolic links are indirect references. The choice depends on the requirements.

"Mounting copies the file system into the mount point."

Mounting does not copy anything. It makes the mounted file system's contents visible at the mount point, hiding whatever was there before. The mount point itself is a directory in the parent file system, and the mounted file system is a separate structure that is attached to it.

"Every file has a single owner and a single group."

Every file has one owner and one group in the classical model, but modern systems extend this with ACLs that grant permissions to additional users and groups. The classical owner and group are still present, but they are not the whole story.

05 Worked example

The following example traces the resolution of an absolute path, compares the behaviour of hard and symbolic links, and computes the effect of the Unix permission bits on a file access.

Path resolution of /home/alice/report.txt

A program calls open("/home/alice/report.txt", O_RDONLY). The kernel resolves the path one component at a time, starting at the root. The state after each step is:

Step Component Current directory Inode Action
0 — / 2 Start at the root. The process has execute permission on the root (every process does).
1 home / — Look up home in the root directory. The entry exists and its inode number is 131073.
2 — /home 131073 The inode for home is a directory. Check execute permission on /home: the process has it. Descend.
3 alice /home — Look up alice in /home. The entry exists and its inode number is 262145.
4 — /home/alice 262145 The inode for alice is a directory. Check execute permission on it: the process is running as alice, so it has it. Descend.
5 report.txt /home/alice — Look up report.txt in /home/alice. The entry exists and its inode number is 262147.
6 — /home/alice/report.txt 262147 The inode for report.txt is a regular file. Check read permission: the process is the owner, and the owner has read permission. Open the file.

The resolution required five directory lookups: one for each component of the path except the last, plus one for the last. Each lookup is a search in the directory's data structure (Lesson 10.4), and each is a potential source of a fault or a permission error. The kernel performs the lookups in sequence, and the whole resolution is cached: the first lookup of a path is the expensive one, and subsequent lookups of the same path (or of paths that share a prefix) benefit from the directory entry cache (the dcache on Linux), which holds the results of recent lookups.

If the process lacked execute permission on /home/alice (for example, because the directory's mode was 700 and the process were running as a user other than alice), the resolution would fail at step 4 with EACCES. If report.txt did not exist in /home/alice, the resolution would fail at step 5 with ENOENT. And if alice were not a directory (which cannot happen here, but could in a corrupted file system), the resolution would fail at step 4 with ENOTDIR.

Comparing hard and symbolic links

Consider a file /home/alice/report.txt with inode number 262147, owner alice, mode 644 (owner read-write, group read, other read). Alice creates a hard link to the file and a symbolic link to it:

$ ln /home/alice/report.txt /home/alice/report-copy.txt
$ ln -s /home/alice/report.txt /home/alice/report-link.txt

$ ls -li /home/alice/report*.txt
262147 -rw-r--r-- 2 alice alice 4096 ... report.txt
262147 -rw-r--r-- 2 alice alice 4096 ... report-copy.txt
262148 lrwxrwxrwx 1 alice alice   24 ... report-link.txt -> /home/alice/report.txt

Several things are visible in the output. The hard link report-copy.txt has the same inode number (262147) as report.txt and the same link count (2, because there are now two names for the inode). The two names are equivalent: modifying the file through one name modifies it through the other.

The symbolic link report-link.txt has its own inode number (262148), a link count of 1, and a mode of lrwxrwxrwx — the l indicates that it is a symbolic link. Its contents are the pathname /home/alice/report.txt, shown by ls after the arrow. Deleting the symbolic link has no effect on the target file.

Now consider what happens if report.txt is deleted:

$ rm /home/alice/report.txt
$ ls -li /home/alice/report*.txt
262147 -rw-r--r-- 1 alice alice 4096 ... report-copy.txt
262148 lrwxrwxrwx 1 alice alice   24 ... report-link.txt -> /home/alice/report.txt
$ cat /home/alice/report-copy.txt   # succeeds — the inode still has one link
$ cat /home/alice/report-link.txt   # fails — the target path no longer exists
cat: /home/alice/report-link.txt: No such file or directory

The hard link report-copy.txt still refers to the file's inode (262147), and the file's contents are still accessible. The link count dropped from 2 to 1, but the inode is not freed because the link count is still positive. The symbolic link report-link.txt still exists as a file, but its contents refer to a path that no longer resolves to any file. The link is dangling; the error appears only when it is dereferenced.

This is the key behavioural difference between the two kinds of link. A hard link keeps the inode alive; a symbolic link does not. A program that relies on a symbolic link continuing to work after the target is deleted will be disappointed; a program that relies on a hard link will not.

Permission checks

A file with mode 640 is readable and writable by its owner, readable by its group, and inaccessible to everyone else. The permission check performed by the kernel when a process tries to open the file depends on the process's relationship to the file:

Process runs as Which bits apply Read allowed? Write allowed?
alice (the owner) Owner bits (rw-) yes yes
bob (a member of the file's group) Group bits (r--) yes no
carol (neither owner nor group member) Other bits (---) no no

The kernel applies one set of bits, not all three. If the process is the owner, the owner bits apply, and the group and other bits are ignored — even if they would grant more permission. This is why a file with mode 640 is not readable by group members if the owner bits are ---: the group bits never apply to the owner, and the owner has no read permission.

The check is performed at the file's inode on every access. For a program that opens many files or performs many accesses, the check is fast — it is a comparison of a small number of bits — but it is performed on every access, and it is one of the reasons file operations have a cost even when the data is already in the page cache.

Mounting example

Suppose a system has a root file system on one device and a separate file system for user home directories on another. The administrator mounts the home file system at /home:

$ mount /dev/sdb1 /home
$ cat /proc/self/mounts | grep home
/dev/sdb1 /home ext4 rw,relatime 0 0

After the mount, any access to a path under /home reaches the contents of the /dev/sdb1 file system, not the contents of the /home directory in the root file system. The root file system's /home directory is hidden — it still exists, but it is inaccessible — until the file system is unmounted. Programs that operate on paths under /home are unaware of the distinction; from their point of view, there is one hierarchy, and the file system boundaries are invisible. This transparency is what makes the mount mechanism useful, and it is what allows a system to distribute its storage across devices without changing the way its users think about the hierarchy.

06 Interactive activity

Activity · explore files and directories on a live system

The goal of this activity is to use the Linux command line to explore the file and directory abstractions, to observe the difference between hard and symbolic links, and to see the file-system hierarchy in action. Work through the steps in your course VM.

  1. Inspect file attributes. Use stat to see a file's attributes, including its inode number, its permission bits, its owner and group, its size, and its three timestamps.
    stat /etc/passwd
    stat /bin/ls
    stat .
    Note the differences: /etc/passwd is a regular file with mode 644, /bin/ls is a regular file with mode 755, and . is a directory. The inode number is the file's identity within its file system; the permission bits are the access-control information.
  2. List directory entries with inode numbers. Use ls -li to list a directory's entries with their inode numbers.
    ls -li /etc | head -20
    ls -li /home
    Each line begins with the entry's inode number, then its permission bits, its link count, its owner, its group, its size, and its name. The link count for a regular file is the number of hard links to it; the link count for a directory includes the . and .. entries and any subdirectories.
  3. Create hard and symbolic links. Create a file, then create a hard link and a symbolic link to it. Compare their inode numbers and their behaviour when the original is deleted.
    mkdir /tmp/links-demo
    cd /tmp/links-demo
    echo "hello" > original.txt
    
    ln original.txt hard.txt
    ln -s original.txt sym.txt
    
    ls -li
    # original.txt and hard.txt have the same inode; sym.txt has a different one
    
    cat original.txt
    cat hard.txt
    cat sym.txt
    
    rm original.txt
    cat hard.txt   # succeeds
    cat sym.txt    # fails: dangling symlink
    ls -li
    Observe that the hard link survives the deletion of the original name, because it refers to the same inode; the symbolic link fails, because its contents refer to a path that no longer exists.
  4. Read the symbolic link's contents. Use readlink to see the target of a symbolic link, and ls -l to see the arrow notation.
    ln -s /etc/passwd /tmp/links-demo/passwd-link
    ls -l /tmp/links-demo/
    readlink /tmp/links-demo/passwd-link
    Note that readlink returns the link's contents — the pathname — while cat would follow the link and read the target file. This is the distinction between operating on the link and operating on its target.
  5. Resolve paths with strace and realpath. Use strace to see the system calls a program makes when it opens a file, and use realpath to see a path's canonical form.
    strace -e trace=openat cat /etc/passwd 2>&1 | head -10
    realpath /etc/passwd
    realpath ../links-demo/original.txt 2>/dev/null
    The openat call shows the path the kernel resolves and the file descriptor it returns. The realpath call shows the canonical form of the path, with symbolic links resolved and redundant components removed.
  6. Explore the file-system hierarchy. Examine the top-level directories and their purposes.
    ls -l /
    ls -ld /bin /lib /usr /var /tmp /etc /home /dev /proc /sys
    man 7 hier | head -60
    For each directory, read the description in man 7 hier and identify its role. Note that /proc and /sys are virtual file systems — they appear as directories, but their contents are generated by the kernel.
    ls /proc | head -20
    cat /proc/version
    cat /proc/self/status | head
  7. Observe the mount table. Examine the current mount points and the file systems they refer to.
    cat /proc/self/mounts | head -20
    df -h
    findmnt | head -20
    The output shows, for each mount, the device, the mount point, the file-system type, and the mount options. Look for the root file system, the /proc and /sys virtual file systems, and any separate file systems for /home, /tmp, or /var.
  8. Inspect file types. Use file and ls -l to identify the type of various files in the system.
    file /bin/ls
    file /etc/passwd
    file /dev/sda 2>/dev/null
    file /dev/tty 2>/dev/null
    file /dev/null
    file /run/systemd/initctl/fifo 2>/dev/null
    ls -l /dev | head -20
    ls -l /dev/null /dev/zero /dev/tty /dev/random
    The ls -l output begins each line with a character that indicates the file's type: - for a regular file, d for a directory, l for a symbolic link, c for a character device, b for a block device, p for a FIFO, and s for a socket.
  9. Explore permission checks. Create a file with restrictive permissions and attempt to access it as a different user (if you have access to sudo or a second user account).
    # As the current user
    echo "secret" > /tmp/restricted.txt
    chmod 600 /tmp/restricted.txt
    ls -l /tmp/restricted.txt
    
    # Attempt to read as another user
    sudo -u nobody cat /tmp/restricted.txt 2>&1
    
    # Change the permission and try again
    chmod 644 /tmp/restricted.txt
    sudo -u nobody cat /tmp/restricted.txt
    Observe how the permission bits determine whether the second user can read the file. The mode 600 allows only the owner to read and write; the mode 644 allows everyone to read.
  10. Read the man pages. Read the man pages that document the file and directory abstractions.
    man 7 path_resolution
    man 7 symlink
    man 7 hier
    man 2 open
    man 2 stat
    man 3 readdir
    For each, write a sentence describing its main contribution to your understanding of the abstractions. The man 7 path_resolution page is particularly valuable: it documents the exact rules the kernel follows when resolving a path, including the treatment of symbolic links, the special entries . and .., and the permission checks at each step.

Reflect: the file and directory abstractions are so familiar that it is easy to forget how much design went into them. A file is a sequence of bytes with no structure; a directory is a mapping from names to inode numbers; the hierarchy is built from these two primitives plus the mount mechanism that attaches separate file systems to the tree. Every program you have ever written that reads or writes a file is using these abstractions, and every command-line session you have ever had is manipulating them. What would it take to design a different set of abstractions? What would be gained, and what would be lost? Consider the file-oriented model of Unix, the object-oriented model of some modern systems, and the key-value model of cloud storage services. Which is the right abstraction for the workloads of the next decade?

07 Knowledge check

Check your understanding

Question 1

Define a file and list its standard attributes. What is the distinction between a file's name and its identity, and why is the distinction important?

Show answer
A file is a named, persistent sequence of bytes managed by the operating system. The name is a human-readable string that identifies the file within its directory. Persistence means the file's contents survive across the termination of the process that created them, across the closing of the file, and across reboots. The sequence of bytes is a stream with a defined order but no internal structure that the file system knows about. The standard attributes are: name (the human-readable string), identifier (a unique number that identifies the file within the file system, called the inode number on Unix), type (regular, directory, device, socket, and so on), location (a pointer to the storage that holds the contents), size (the current size in bytes), protection (the access permissions), owner (the user and group that own the file), and timestamps (the times of creation, last modification, and last access). The distinction between a file's name and its identity is central to the Unix file model. The name is an entry in a directory; the identity is the inode number, which is stored in the file's metadata and is independent of any name. The distinction matters for several reasons. First, it allows a single file to have multiple names — hard links — which is the mechanism by which the same file can appear in multiple directories. Second, it allows a file to be renamed without affecting its identity, so programs that hold the file open are not affected by a rename. Third, it makes the directory operations simple: adding a name is adding an entry, removing a name is removing an entry, and neither affects the file's contents. Fourth, it allows the file system to distinguish between deleting a name (which may leave the file alive through other names or open file descriptors) and deleting the file itself (which happens only when the last reference is gone).

Question 2

List the standard file operations and describe what each does. What is a file descriptor, and why does the open-file table have a two-level structure?

Show answer
The standard file operations are: open (open a file and return a file descriptor, checking permissions), read (read bytes from the current position, advancing the position), write (write bytes at the current position, advancing the position), lseek (move the file's current position to a specified offset, for random access), close (close the file and release its descriptor), unlink (remove the file's directory entry, freeing the file's data when the last reference is gone), rename (change a file's name or move it to a different directory, atomically), and truncate (reduce a file's size, discarding the contents beyond the new length). A file descriptor is a small integer that a process receives from open and uses in subsequent operations to refer to the open file. It is the process's handle for the file. The open-file table has a two-level structure to support sharing between processes. The per-process table maps file descriptors to entries in the system-wide table. The per-process table is what the process sees: its descriptors are private, and the same number in two different processes refers to different open files. The system-wide table holds the file's open state — the current position, the access mode, and a reference to the file's inode — and it is shared among processes. When a process forks, the child inherits the parent's per-process table, and the two processes share the same system-wide entries. This is what allows a shell pipeline to work: the shell forks a child for each command, and the children share their standard input and output entries, so a read by one advances the position for the other. The two-level structure also allows a single process to have multiple file descriptors pointing to the same open file, which is what makes dup and its variants work.

Question 3

Describe the classical directory structures: single-level, two-level, tree, acyclic graph, and general graph. What are the advantages and disadvantages of each, and which is used by modern Unix-like systems?

Show answer
The single-level directory has all files in one directory. It is simple, but all file names must be unique (which causes collisions when two users have files with the same name), and it does not scale: as the number of files grows, the directory becomes unwieldy and lookups become slow. The two-level directory gives each user a separate directory, with a master directory containing one entry per user. It solves the name-collision problem and simplifies access control, but it does not allow users to organise their own files into subdirectories. The tree-structured directory allows arbitrary nesting: each directory may contain files and subdirectories, and every file has a unique path from the root. It is the standard for modern file systems. Its advantage is that it allows users to organise their files as they see fit, with as many levels of nesting as they need; its disadvantage is that it does not support sharing a file between multiple directories. The acyclic graph directory extends the tree by allowing a file or directory to have multiple parents, through hard links and symbolic links. It supports sharing while keeping traversals terminating (no cycles). This is the structure used by modern Unix-like systems: the tree is the base, and hard and symbolic links extend it into an acyclic graph. The general graph directory allows cycles. It is the most general structure and supports arbitrary sharing, but operations that traverse the structure must detect and handle cycles, which complicates the file system's implementation. For this reason, cycles are avoided in most Unix-like file systems. Modern Unix-like systems use the acyclic graph structure: the tree is the base, hard and symbolic links provide sharing, and the kernel prevents cycles from forming (hard links to directories are forbidden, symbolic link resolution is bounded in depth).

Question 4

What is the difference between a hard link and a symbolic link? Give an example of when each is the right choice, and explain how the two behave when the target file is deleted.

Show answer
A hard link is a directory entry that refers to an existing file's inode. It is a direct reference: the link and the original are equivalent names for the same file, and modifying the file through one name modifies it through the other. Hard links cannot cross file-system boundaries (an inode number is meaningful only within its own file system) and cannot refer to directories (except . and ..). A symbolic link is a file whose contents are a pathname. It is an indirect reference: the link's contents are resolved independently of the link itself, and the resolution may produce a different file each time. Symbolic links can cross file-system boundaries and can refer to directories; they can also dangle (refer to a path that does not exist). An example where a hard link is the right choice: a user wants the same file to appear under two names within the same directory, and wants both names to be genuinely equivalent (so that modifying through one modifies through the other, and deleting one does not delete the file). An example where a symbolic link is the right choice: /usr/bin/cc refers to the current default C compiler, and the link is updated when the default changes; the link is a reference to a target that may be replaced, and its purpose is to provide a stable name for a changing target. When the target file is deleted, the two links behave differently. A hard link continues to refer to the file's inode, and the file's contents remain accessible (the link count drops, but the inode is freed only when the count reaches zero and no process has the file open). A symbolic link continues to exist as a file, but its contents refer to a path that no longer resolves to any file; the link is dangling, and any attempt to dereference it fails with ENOENT.

Question 5

What is mounting? How does it allow a system with multiple file systems to present a single unified hierarchy, and what happens to the contents of the mount point directory?

Show answer
Mounting is the process of attaching a file system to a directory in an existing file system. The directory is called the mount point, and after the mount, the contents of the mounted file system appear at the mount point. The kernel records the mount in a mount table, which on Linux is visible in /proc/self/mounts. Mounting is what allows a system with multiple file systems to present a single unified hierarchy. A modern system usually has several file systems: the root file system, a separate file system for /home, a separate file system for /var, a temporary file system for /tmp, and possibly several remote file systems. The mount mechanism combines them into a single tree rooted at /, and the user sees one hierarchy without needing to know which file system a given path belongs to. When a file system is mounted at a mount point, the contents of the mount point directory in the parent file system are hidden: the directory itself still exists, but its entries are inaccessible until the file system is unmounted. An access to a path under the mount point reaches the mounted file system's contents, not the underlying directory's contents. This is why the mount point directory is often empty: its contents are not used while the mount is active. The mount operation is reversible: the file system can be unmounted with umount, and after the unmount, the contents of the mount point directory in the parent file system become visible again. Mounting requires root privileges (or, on modern systems, specific capabilities), because it changes the system's view of its storage.

08 Unit discussion

Discussion prompt

The file and directory abstractions have been remarkably stable. The Unix file model — a byte stream, a hierarchical namespace, hard and symbolic links, permission bits — was designed in the 1970s and remains the dominant model today. It has been extended (with ACLs, extended attributes, journaling, and copy-on-write snapshots) but not replaced, and every general-purpose operating system today provides an interface that is recognisably the same.

Other abstractions have been proposed and have largely failed to displace it. Object-oriented file systems (such as those proposed for persistent programming languages) did not achieve widespread adoption. Content-addressed storage (used by Git and by some distributed systems) is powerful but is not exposed as a general-purpose file system. Key-value stores (used by cloud services) are extremely popular but are typically accessed through a network API rather than through the file system interface.

Why has the file model been so durable? Is it because it is the right abstraction, or because it is familiar, or because it is simple enough to be implemented efficiently on any storage device? What would a genuinely new abstraction need to provide to replace it? Consider the workloads of the next decade: machine-learning training on enormous datasets, distributed applications running on many machines, persistent memory that sits between DRAM and disk, and edge devices with limited storage. Does the file model serve these workloads well, or would a different abstraction be better?

Consider the systems you use and the programs you write. When did you last encounter the limits of the file abstraction? What were you trying to do, and what would you have done differently with a different abstraction? Post your response to the Unit 10 discussion forum and reply to at least two classmates. Supports CLO1, CLO5, and CLO6.

09 Practice exercise

This ungraded exercise consolidates the lesson and prepares you for Lesson 10.2. Aim to spend 60–90 minutes on it.

  1. Classify the file types. For each of the following, state which file type it is (regular, directory, symbolic link, character device, block device, FIFO, or socket) and explain your reasoning:
    • /bin/ls
    • /etc
    • /dev/null
    • /dev/sda
    • /usr/bin/cc
    • /run/systemd/initctl/fifo
    • /var/run/docker.sock
    • /proc/version
    Use ls -l and file to verify your answers.
  2. Trace a path resolution. For the path /usr/share/doc/README, identify each component and describe what the kernel does at each step of the resolution. What permission checks are performed? What errors can occur at each step, and what error codes do they produce? If you have strace available, use it to observe the resolution of an open call on this path.
    strace -e trace=openat,newfstatat cat /usr/share/doc/README 2>&1 | head -20
  3. Compare hard and symbolic links. Create a file, a hard link, and a symbolic link. Then:
    • Modify the file through the hard link and confirm that the modification is visible through the original name.
    • Delete the original name and confirm that the hard link still works.
    • Confirm that the symbolic link dangles after the deletion and produces an error when dereferenced.
    • Recreate the original name and confirm that the symbolic link works again.
    Write down the commands you used and the results of each.
  4. Compute permission bits. For each of the following symbolic permissions, write the octal representation and describe who can read, write, and execute:
    • rwx------
    • rw-r--r--
    • rwxr-xr-x
    • rw-rw----
    • r-x--x--x
    • rwxrwxrwx
    • ---------
    Then, for a file with mode 640 owned by alice and grouped to staff, state whether each of the following processes can read the file: alice, a member of staff, a user who is neither alice nor in staff, and the root user.
  5. Design a directory structure. A small company has a file server with the following requirements: each employee has a private directory; there are shared directories for each project, with different team members having different levels of access; there is a public directory that everyone can read but only administrators can write; and there is an archival directory that is read-only and holds old data. Design a directory structure that satisfies these requirements. Specify the paths, the permission bits for each directory, and the group memberships needed. Justify your design.
  6. Explore the Linux file-system hierarchy. Write a short paragraph (200–250 words) describing the Linux file-system hierarchy and the purpose of each top-level directory. Your paragraph should cover: /bin, /etc, /home, /lib, /proc, /sys, /tmp, /usr, and /var. Then identify which of these directories are commonly mounted as separate file systems in a production Linux system, and explain why.
  7. Investigate the "everything is a file" principle. Write a short paragraph (150–200 words) explaining what the Unix "everything is a file" principle means, what it buys, and what it does not cover. Give at least three concrete examples of non-file objects that are exposed as files (/dev, /proc, /sys) and one example of a system feature that is not exposed as a file. What does the principle fail to capture?
  8. Reflect. In your own words, explain why the distinction between a file's name and its identity is important in the Unix file model. Then describe one situation in your own experience — a program you have written or a system you have used — where the distinction mattered. What was the situation, and what would have gone wrong if names and identities were conflated?

10 Summary

  • A file is a named, persistent sequence of bytes managed by the operating system. The abstraction hides the details of the storage device and presents a uniform interface for reading and writing.
  • A file's attributes — name, identifier, type, location, size, protection, owner, and timestamps — are stored in the file system's metadata structures, not in the file itself. On Unix, the metadata lives in the file's inode.
  • The standard file operations are open, read, write, lseek, close, unlink, rename, and truncate. The file descriptor is the process's handle for an open file, and the open-file table has a two-level structure that allows sharing among processes.
  • Unix-like systems distinguish several file types: regular files, directories, symbolic links, character devices, block devices, FIFOs, and sockets. The variety reflects the principle that everything is a file: devices and communication endpoints are exposed through the same interface as ordinary data.
  • The standard access methods are sequential (read or write from beginning to end), direct (read or write at an arbitrary position), and indexed (access through a key-based index). Sequential access is the default on Unix and is what the file system optimises for.
  • A directory is a file that maps names to files. The separation of names from identity — names are directory entries, identity is the inode number — is central to the Unix file model. It allows multiple names for the same file, renames without affecting open file descriptors, and simple directory operations.
  • The classical directory structures are single-level, two-level, tree, acyclic graph, and general graph. Modern Unix-like systems use the acyclic graph: the tree is the base, and hard and symbolic links extend it with sharing while the kernel prevents cycles.
  • Path resolution converts a path into a reference to a file by processing each component in turn, checking permissions at each step. Absolute paths begin at the root; relative paths begin at the process's current working directory. The special entries . and .. refer to the current and parent directories.
  • A hard link is a directory entry that refers to an existing file's inode. A symbolic link is a file whose contents are a pathname. Hard links cannot cross file-system boundaries or refer to directories; symbolic links can do both and can dangle. When the target is deleted, a hard link keeps the file alive, while a symbolic link becomes a dangling reference.
  • File protection on Unix is provided by the permission bits — owner, group, and other, each with read, write, and execute — checked by the kernel at the file's inode. Modern systems also support access-control lists (ACLs) for finer-grained permissions.
  • Mounting attaches a file system to a directory in an existing hierarchy, allowing a system with multiple file systems to present a single unified tree. The contents of the mount point directory are hidden while the file system is mounted.
  • The Linux file-system hierarchy is standardised by the Filesystem Hierarchy Standard: the root, /bin, /boot, /dev, /etc, /home, /lib, /media, /mnt, /opt, /proc, /root, /sbin, /sys, /tmp, /usr, and /var. The layout separates files by role and allows different directories to be stored on different file systems.

Looking ahead

This lesson presented the file and directory abstractions: their attributes, operations, types, structures, and their place in the Linux hierarchy. It introduced the inode — the metadata structure that holds a file's attributes and identity — but did not present it in detail. Lesson 10.2 presents file metadata: the information that describes a file, including the inode's fields, the timestamps, the ownership and permission information, and the extended attributes that modern file systems provide. Lesson 10.3 then presents the inode itself: its structure, its role in path resolution, its lifetime, and the operations that manipulate it. The rest of the unit builds on these foundations.

11 References

  • Silberschatz, A., Galvin, P. B., & Gagne, G. Operating System Concepts, 10th ed., Chapter 13, Sections 13.1–13.3, and Chapter 14, Sections 14.1–14.3.
  • Arpaci-Dusseau, R. H., & Arpaci-Dusseau, A. C. Operating Systems: Three Easy Pieces, Chapter 39. Available online at ostep.org.
  • Tanenbaum, A. S., & Bos, H. Modern Operating Systems, 4th ed., Chapter 4, Sections 4.1–4.2.
  • Bryant, R. E., & O'Hallaron, D. R. Computer Systems: A Programmer's Perspective, 3rd ed., Chapter 10, System-Level I/O.
  • Kerrisk, M. The Linux Programming Interface. No Starch Press, 2010. Chapters 1–5 and 14–18. The definitive reference for the Linux file interface.
  • Ritchie, D. M., & Thompson, K. The UNIX Time-Sharing System. Communications of the ACM, 1974. The paper that introduced the Unix file model, including the hierarchical directory structure and the "everything is a file" principle.
  • McKusick, M. K., Joy, W. N., Leffler, S. J., & Fabry, R. S. A Fast File System for UNIX. ACM Transactions on Computer Systems, 1984. The paper that introduced the Berkeley Fast File System, which established many of the design principles still in use today.
  • Filesystem Hierarchy Standard Group. Filesystem Hierarchy Standard (FHS) version 3.0. Available online. The specification of the standard Linux directory layout.
  • Linux man-pages: man 7 path_resolution, man 7 symlink, man 7 hier, man 2 open, man 2 read, man 2 write, man 2 close, man 2