Working with Files

Two ways into the same idea

Before You Start

The Core Idea: Four Calls That Build Any File Operation

Unlike fopen/fread (standard-library functions that add a buffering layer), the following four system calls interact directly with the OS kernel:

Figure 1, every file operation goes through this triple loop: open, then read/write (possibly several times), then close.

1File Descriptor

A small integer returned by open() that represents a "handle" to the open file, every subsequent read, write, and close call uses this number to refer to the same file. In practice, every process owns its own file descriptor table, a small array of integers, with 0, 1, and 2 already reserved for stdin, stdout, and stderr. Each slot in that table points to an entry in the kernel's open-file table, and that entry in turn holds the current read/write offset and a pointer to the file's actual data on disk. That's why opening the same file twice with two separate open() calls gives you two different descriptors with completely independent offsets, even though both ultimately point to the same file.

2Open flags

O_RDONLY for read-only, O_WRONLY for write-only, O_CREAT to create the file if it doesn't exist, O_TRUNC to truncate its contents, these flags are combined with a bitwise OR. That works because each flag is defined as a distinct bit within the same integer (powers of two like 1, 2, 4...), so combining them with | turns on every requested bit at once without any of them overwriting another, and the kernel checks each bit separately to decide how to open the file. Note that O_RDONLY is usually just 0, so it isn't really a bit at all and can't be combined with O_WRONLY (the access mode itself can't be both read and write at once without O_RDWR), unlike O_CREAT and O_TRUNC, which behave like add-on modifiers layered on top of either access mode.

3The return value

read() and write() return the number of bytes actually read/written, it can be less than requested, and -1 means the operation failed. The reason is that neither call guarantees transferring the full requested count in one go: a read can stop early at end-of-file and return fewer bytes than the buffer size asked for, both calls can be interrupted by a signal before finishing, and pipes or sockets may simply have less data available at that exact moment. That's why any code that needs a specific number of bytes transferred reliably has to call read()/write() inside a loop that keeps going until the full count is reached or the end of the file is hit.

Comparison: System Calls vs. Standard Library Functions

How It Works, Step by Step: Copying a File's Contents

As a direct application of this four-call pattern, here is a function that copies a file's entire contents from a source to a destination using the four calls directly instead of fopen. The function follows exactly this sequence: open the source for reading only, open the destination (creating it if it doesn't exist and truncating it if it does) for writing only, then a repeated read/write loop, and finally close both descriptors. Notice that the two open() calls here are completely independent of each other: each one creates a brand-new entry in the kernel's open file table, with its own offset starting at zero, so even if the source and destination happened to be the same file on disk (a rare but possible case), fd_src and fd_dst would still advance through two completely separate offsets with no effect on each other, exactly as explained in the File Descriptor card above.

#include <fcntl.h>
#include <unistd.h>

int copy_file(const char *src, const char *dst)
{
    int fd_src, fd_dst;
    char buf[1024];
    ssize_t n;

    fd_src = open(src, O_RDONLY);
    if (fd_src == -1)
        return (-1);

    fd_dst = open(dst, O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (fd_dst == -1)
    {
        close(fd_src);
        return (-1);
    }

    while ((n = read(fd_src, buf, sizeof(buf))) > 0)
        write(fd_dst, buf, n);

    close(fd_src);
    close(fd_dst);
    return (0);
}

Figure 2, Notice the while loop: reading happens in chunks the size of the buffer instead of reading the whole file at once, because the file's size might exceed the memory available.

Figure 3, the copy loop repeats one buffer-sized chunk at a time; it stops automatically once read() returns 0, meaning it has reached the end of the source file.

The value 0644 passed as open()'s third argument only matters at all when O_CREAT is actually among the flags, it's the permission mode for the new file in case open() has to create one, expressed in octal: the first digit (6) for the owner (read and write), the second and third (4 and 4) for the group and everyone else (read only), and even this value isn't applied literally as written; the kernel first ANDs it with the complement of the calling process's umask before it becomes the actual permission that lands on disk. The while loop, meanwhile, depends entirely on what read() returns: a value greater than zero is the number of bytes actually read in that particular chunk (which can be less than 1024 once the source gets close to its end), zero specifically, not any negative value, is the official end-of-file (EOF) signal, and -1 means the read genuinely failed. Note too that, for simplicity, the example above doesn't check write()'s return value the same way; production-ready code actually needs to check both calls inside a complete loop, exactly as flagged in the "Return Value" card above.

Figure 4, every open() call, even from the same process and on the same file, creates an independent entry with its own offset in the kernel's open file table; only that entry holds a pointer to the actual inode that carries the file's real data on disk.

Worked Example: Reading a Text File Line by Line

Another practical example of the same pattern: a function that reads a text file and prints each line with its number. The challenge here is that read() has no concept of "line", it just transfers bytes, so we have to search manually for the newline character '\n' inside the buffer:

#include <fcntl.h>
#include <unistd.h>

void print_lines(const char *path)
{
    int fd = open(path, O_RDONLY);
    if (fd == -1)
        return;

    char buf[4096];
    ssize_t n;
    int line_num = 1;
    int col = 0;

    /* print first line number */
    char header[16];
    int hlen = 0;
    int tmp = line_num;
    header[hlen++] = '0' + tmp;
    header[hlen++] = ':';
    header[hlen++] = ' ';
    write(1, header, hlen);

    while ((n = read(fd, buf, sizeof(buf))) > 0)
    {
        ssize_t i;
        for (i = 0; i < n; i++)
        {
            write(1, &buf[i], 1);
            if (buf[i] == '\n')
            {
                line_num++;
                hlen = 0;
                tmp = line_num;
                if (tmp >= 10) header[hlen++] = '0' + tmp / 10;
                header[hlen++] = '0' + tmp % 10;
                header[hlen++] = ':';
                header[hlen++] = ' ';
                write(1, header, hlen);
            }
        }
    }
    write(1, "\n", 1);
    close(fd);
}

Notice that the function reads the file in large chunks (4096 bytes) to minimize system calls, but processes each byte individually to search for '\n'. This pattern (read in chunks + process byte by byte) is extremely common in system tools, cat -n, grep, and wc all work on roughly the same principle.

Myth vs. Reality

Practical Applications

Deep Dive: I/O Layers, Safe Checking and Byte Order

Beyond what was covered about descriptors and flags, we add three things: the I/O layers, the safe pattern of checking every call, and how byte order matters when writing binary data (marshaling).

The I/O layers

An ordinary function runs entirely inside your program. A system call is a request to the kernel, because only the kernel may touch the disk. read, write, open and close are system calls. printf is a library function: it gathers text in a buffer and calls write behind the scenes. That is why the project bans library functions: to show you what lies underneath.

Figure 5, I/O layers: library functions on top of system calls on top of the kernel.

Reading octal permissions

Permissions have three groups: owner, group, others. Each group has read 4, write 2, execute 1 summed into one digit. So 0644 means: owner 6 (read and write), group 4, others 4. 0600 is for the owner alone.

The safe-checking pattern

Figure 6, The safe pattern for a file: check every call and always close at the end.

Byte order and marshaling

A number wider than one byte is stored as several bytes, and there are two orderings. In little endian (common x86 processors) the least significant byte is stored first at the lowest address. In big endian the most significant byte comes first. The number 0x12345678 appears in memory as follows:

If you write a raw int to a file or send it over the network to a machine with a different order, it will be read wrongly. The fix is to agree on one order: network byte order, which is big endian. Functions such as htonl and htons convert from host order to network order before sending, and ntohl and ntohs do the reverse on receipt. This process is called marshaling. Text files avoid the problem because each character is a single byte.

Figure 7, Marshaling: both sides agree on one order in the middle whatever each one’s own order is.

Try it yourself

Quick Review Flashcards

Check Your Understanding