Redirection and pipes
Lesson 4 of 11
4Redirection and pipes
What the picture shows: Lines pass from tool to tool through the pipe, some drop out on the way, and one number comes out at the end.
If you type sort | uniq -c without thinking, jump to the exercises. Everyone else starts at the picture and ends up typing it without thinking.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
Each command is like a small machine in a factory with an input slot, an output slot and a slot for errors. A pipe connects one machine's output to the next one's input, so simple small machines become a whole production line. An arrow sends the output into a box (a file) instead of the screen.
Every process has three streams: stdin (0), stdout (1) and stderr (2). > writes output to a file afresh, >> appends, and 2> redirects errors. | joins one command's stdout to the next one's stdin, as in grep error app.log | sort | uniq -c.
Before You Start
📟The shell lesson
You know ls and cat, and that a command prints text.
🗂️The files lesson
You know that text is a file and a file is text.
The Core Idea
1Three doors on every command
Every program is born with three doors: an input it reads from, an output it writes its results to, and a second output for errors. By default the input is the keyboard and both outputs are the screen.
printf printed three lines, and > sent them to a file instead of the screen. cat shows the file, and wc -l counts its lines.
You opened no editor and clicked nothing. You wrote a file with one arrow. That is the output door with its direction changed.
printf "apple\nbanana\ncherry\n" > fruits.txt cat fruits.txt wc -l fruits.txt
Output:
apple banana cherry 3 fruits.txt
2> rewrites, >> appends, 2> catches errors
> wipes the file and writes from the start. >> adds at the end. Most lost files come from a > where a >> was meant.
Errors leave through the second door, so > does not catch them. 2> does. And the exit code $? says whether the command succeeded: zero is success, anything else is failure.
echo first > log.txt echo second >> log.txt cat log.txt ls /nonexistent 2> errors.txt echo "exit code: $?" cat errors.txt
Output:
first second exit code: 2 ls: cannot access '/nonexistent': No such file or directory
3The pipe: one command's output is the next one's input
| takes what the left command writes and hands it to the right command as input, with no file in between. Follow the picture: cat emits the lines, grep keeps what matches, wc counts.
grep an kept the line containing an. And grep -c a counted lines containing the letter a straight from the file with no pipe. Both ways are right; the pipe pays off as the chain grows.
printf "apple\nbanana\ncherry\n" > fruits.txt cat fruits.txt | grep an grep -c a fruits.txt
Output:
banana 2
4Small tools make a report
Four tools in one line: sort orders, uniq -c merges adjacent repeats and counts them, sort -rn orders by the count, descending. The result: the most frequent at the top.
This exact sentence analyses real server logs every day: replace the letters with visitor addresses and you know who visits most. You wrote no program; you composed four.
printf "b\na\nb\nc\na\nb\n" | sort | uniq -c | sort -rn
Output:
3 b
2 a
1 cHow It Works, Step by Step: How the shell builds a pipeline
The shell reads the line, sees |, and splits it into commands.
It creates a process for each and connects each one's output to the next one's input.
The processes run together: as soon as a line leaves the first, the second reads it.
Only the final result reaches the screen, or a file if you redirect it.
Worked example
ls prints names one per line, grep '\.txt$' keeps those ending in .txt, and wc -l counts. The whole line: ls | grep '\.txt$' | wc -l. Three tools, none of which knows anything about the others.Myth vs Reality
| Myth | Reality |
|---|---|
| A pipe copies files between commands. | There is no file on the way at all; the text passes in memory from process to process directly. |
| > and >> are alike and either will do. | > wipes what was there, >> appends. Mixing them up is the most common way to lose a log file. |
| Errors are part of the normal output. | They have their own door, number 2, which is why you can save the output to a file and keep errors on screen, or the reverse. |
Try it yourself
You typed ls > list.txt 2> errors.txt and found list.txt empty and errors.txt with text in it. What happened?
Normal output went to list.txt and errors to errors.txt as asked. An empty file and one with text means the command failed, such as on a missing folder, so it wrote only to the error door.
Quick review cards
$? equal to 0 mean?Check Your Understanding
Why does > not catch error messages?
Because they leave through door 2, not standard output, and > redirects standard output only.
What happens if you write sort | uniq -c without the sort?
uniq merges only adjacent repeats, so separated repeats stay apart and the counts come out wrong.
When do you prefer an intermediate file to a pipe?
When you need the middle result later or want to inspect it; a pipe is for when only the final result matters.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Done with this lesson?
Sign in to save your progress.