Computer Systems, the Shell, and Docker

Two ways into the same idea

Before You Start

The Core Idea: System Layers

Figure 1, the shell isn't the operating system itself, but a text interface that translates your commands into requests the kernel understands.

1The Shell

A program that reads your text commands and executes them, like bash. Every command you type is actually the name of a small program the shell finds and runs.

2Redirection & Pipes

> redirects output to a file instead of the screen. | (the pipe) passes one command's output directly as another's input, letting you build an "assembly line" of simple commands. Technically, a pipe is a small memory buffer the kernel itself manages between the two processes; both commands actually run at the same time instead of the first finishing before the second starts, and the second reads each chunk of output as soon as it's available, which is what keeps pipes fast even with huge amounts of data.

3Permissions

Every file has three permission classes (owner, group, others) and three types (read r, write w, execute x), represented numerically (like 755) or symbolically (rwxr-xr-x). The number itself is just a sum of fixed values: read = 4, write = 2, execute = 1, adding up the permissions wanted for each class produces its digit (e.g. rwx = 4+2+1 = 7, r-x = 4+1 = 5), and this is exactly how the system checks any access request: it compares the requester's class (owner/group/other) against the permissions encoded in its matching digit.

4Init Files

Files like .bashrc run automatically when a new shell session opens, an ideal place to define environment variables and permanent shortcuts.

How It Works, Step by Step: Building a Pipeline of Commands

# Count lines containing "error" in a log file, after sorting alphabetically
cat server.log | grep "error" | sort | wc -l
#  ^ prints the      ^ filters      ^ sorts   ^ counts
#    whole file        lines only              final lines

# Check a script file's permissions and make it executable for the owner only
ls -l deploy.sh          # shows e.g.: -rw-r--r--
chmod 744 deploy.sh      # 7=rwx for owner, 4=r-- for group, 4=r-- for others
./deploy.sh               # now executable by its owner

Figure 2, Every command in the chain above is very simple on its own, the shell's real power is composing simple commands with pipes to solve complex problems.

From Shell to Docker: Why Isolate Runtime Environments?

A classic problem: "it works on my machine but not on the production server", usually because library versions or system settings differ between the two. Docker solves this by packaging the application with everything it needs (libraries, settings, even a minimal filesystem) into one unit that runs the same way everywhere.

# A simple example Dockerfile for a Python app
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python3", "app.py"]

Deep Dive: What Happens Inside the Computer

Before the shell and before C, you need to know what really goes on inside a computer. These ideas later explain why memory and pointers are central.

1. The Von Neumann architecture

The modern computer follows a model called the Von Neumann architecture: memory stores the program and the data together, and the central processing unit (CPU) fetches from it and executes. The CPU has an arithmetic logic unit (ALU) that performs operations, a control unit that coordinates execution, and tiny very fast registers holding the current values. It connects to memory and devices through paths called buses. From this comes the idea that a program itself is just data in memory.

Figure 3, The computer’s components and the data flow between them.

2. The fetch–decode–execute cycle

The CPU runs without stopping on a loop of three steps. It fetches the next instruction from an address held in a special register called the program counter, decodes it to know what it means, executes it, and the counter advances to the next instruction, except in a jump, where a new address is written into the counter. From this you understand that conditions and loops in any language are only jumps that change the program counter depending on a comparison result.

Figure 4, The basic execution cycle, with a jump as a change to the program counter.

3. How data is represented

Everything in a computer is binary numbers (0 and 1). A bit is a single unit, and every eight bits form a byte which can represent 256 values (0 to 255). Because binary is long to read, values are often written in hexadecimal (Hex), where each hex symbol stands for exactly four bits, so one byte is two symbols.

To convert binary to decimal you add the powers of 2 for the positions holding 1. So 0100 0001 equals 64 + 1 = 65. Letters are represented with an encoding table such as ASCII that gives each letter a number: A is 65, a is 97, and the written digit "0" is 48, not zero. That is why 65, 0x41 and A can be the same byte interpreted three ways: the reading depends on context, not on the bits.

4. What an operating system is responsible for

5. Absolute and relative paths

An absolute path starts from the root / and describes the whole location regardless of where you are, such as /home/user/docs/a.txt. A relative path starts from the current directory: docs/a.txt, where .. means the parent directory and . the current one. For example, if you are in /home/user/docs then ../music/b.mp3 means /home/user/music/b.mp3.

6. The journey of launching a program

Figure 5, From typing a command to the prompt returning.

Deep Dive: The Shell, Files and Permissions

The shell is a language for talking to the operating system. Here we detail the ideas behind most commands: the folder structure, links, wildcards, quoting, expansions, streams and permissions.

1. The structure of the Linux file tree

Everything in Linux is a single tree rooted at /. The main folders have conventional roles: /home holds user folders, /etc configuration files, /bin and /usr/bin commands, /tmp temporary files, /var changing data such as logs, and /dev devices as files. Each user has a home folder that cd returns to without arguments. Note the difference between / (the system root), /root (the home folder of the user root) and ~ (a shortcut for your own home).

Figure 6, The basic tree: every file or device is a node under the root.

2. Hard links and symbolic links

A file name in Linux is not the file itself. The real file is represented by a disk structure called an inode that carries its data and permissions, and names are just entries in folders pointing to an inode number. A hard link is another name for the same inode: deleting one name does not erase the data while another name exists, and it does not work across file systems or on folders. A symbolic link is a small file containing a text path to a target, like a shortcut: if the target is deleted the link dangles and points to nothing.

Figure 7, Two hard names pointing to one inode, and a symbolic link pointing to a name rather than an inode.

3. Wildcards and who expands them

A point that surprises beginners: when you type ls *.txt it is the shell that turns *.txt into the list of matching files, then passes the list to ls, which never sees the star. The basic wildcards are: * any number of characters, ? exactly one character, and [abc] one character from the set. If nothing matches, the word stays as it is by default.

4. Quoting and the order of expansions

Before running any command, the shell passes the line through stages that expand special things. Then quoting decides which stages you want: single quotes ' ' disable everything so characters stay literal. Double quotes " " disable wildcards but keep variable expansion $ and command substitution $( ). The backslash \ disables only the next character.

Figure 8, A simplified order of the stages the shell applies to a line.

5. Streams and pipes

Every program has three ready streams: standard input stdin (descriptor 0), standard output stdout (1), and errors stderr (2). The symbol > redirects output to a file (erasing its content), >> appends to its end, < makes a file the input, 2> redirects errors alone, and the pipe | connects one program’s output to the next one’s input. These small pieces combine into powerful tools: each command does one thing, and the pipe composes them.

Figure 9, A pipeline: each program passes its output on, and errors travel a separate path.

6. Users and permissions as numbers

Every file has three classes: owner (u), group (g) and others (o), and each class has three permissions: read (r) worth 4, write (w) worth 2 and execute (x) worth 1. Add the values per class to get one digit, so rwxr-xr-- becomes 754. Folders mean something different: x allows entering it, r listing its contents, and w creating and deleting files in it. A normal user cannot give a file away (chown) so nobody can dodge disk quotas or dump files on others, which needs root privileges through sudo.

7. Initialization files and where they load

When a shell starts it reads files that define its environment. A login shell reads /etc/profile then the first of the personal profile files it finds in your home, while an interactive non-login shell (like a terminal window opened inside the desktop) reads ~/.bashrc. That is why many put a line in the login file that calls .bashrc. Any change in these files does not show in an open shell until you reload them with the source command (or the dot).

Deep Dive: Containers versus Virtual Machines

"It works on my machine" is an old problem: versions and libraries differ between machines. Containers solve it by packaging the program with its environment. Here we detail how a container differs from a virtual machine, what an image is made of, and how a container lives and dies.

1. Virtual machine or container?

A virtual machine emulates a whole computer with a complete operating system on top of a hypervisor, so it is heavy and slow to boot but strongly isolated. A container shares the host system’s kernel and isolates processes and files with namespaces and limits resources with control groups (cgroups), so it is light and starts in seconds. The price is that every container on a machine depends on the same kernel.

Figure 10, A virtual machine carries a full system for each copy, while containers share one kernel.

2. Image versus container

An image is a read-only template containing a file system, a program and its settings. A container is a running instance of an image, much like the relation between a class and an object in programming. An image is built of immutable layers stacked on one another, so images share common layers and save space. When a container runs, a temporary writable layer is added on top, and it is erased when the container is removed.

Figure 11, The image layers are read-only, with a temporary write layer on top that disappears with the container.

3. The container lifecycle and keeping data

A container lives as long as the main process inside it: when the process ends, the container ends. It passes through states: created, running, stopped, then removed. Because the write layer is erased with the container, important data must be kept outside it deliberately with a volume managed by Docker, or by binding a host folder (a bind mount). Whoever does not do so loses their data when the container is removed.

Figure 12, The container states and the commands moving between them.

Try it yourself

Quick Review Flashcards

Check Your Understanding