Terminal & Scripting

Foundations (terminal navigation, text processing, process control, and bash scripting) · prereq 09 (Linux (RHEL)) — you need a real shell to practice these commands in; this module gives the general-purpose command-line skills that every later module (and every real engagement) assumes you already have. · ~1 day (4 reference submodules — navigation & permissions, text processing & pipes, process monitoring, and bash scripting). Reality check: this is the 80/20 on-ramp; real fluency (writing scripts you'd trust unattended, debugging a stuck process under pressure) is built through daily use.

Every module before this one assumed you can already move around a Linux box, read what a command printed, and tell whether it worked — this is the module that actually builds those skills. A junior who can only run commands one at a time, by hand, from memory, hits a wall the moment a task needs to be repeated, chained, or done unattended. The shell itself — navigation, text tools, process control, and turning a sequence of commands into a script — is the tool you reach for dozens of times a day, underneath every other module in this course.

TL;DR — in 30 seconds:

• Everything in the shell is text flowing through small programs that each do one job well — the skill is picking the right tool and chaining them, not memorizing one giant command.
• grep finds lines, sed transforms them, awk computes over columns — three different jobs; reaching for the wrong one is the most common beginner mistake.
• A script is just the commands you'd type, saved to a file — set -euo pipefail turns silent partial failures into a script that stops and reports the moment something actually goes wrong.

Learning objectives

By the end, the junior can:

• Navigate the filesystem with absolute and relative paths, and read/set Unix file permissions (rwx, chmod symbolic and octal notation, ownership).
• Chain commands with pipes and redirection, and pick the right tool — grep, sed, awk, cut, sort, uniq — for a given text-processing task.
• Inspect running processes (ps, top) and control them (kill with the right signal, jobs/fg/bg for background work).
• Write a bash script with variables, conditionals, loops, functions, and correct exit codes, guarded by set -euo pipefail.
• Explain why an unguarded script can "succeed" while doing the wrong thing, and how each part of set -euo pipefail closes one specific gap.

---

Do it with Claude

Pair, Do and Prove run in Claude Code, with the onboarding-tutor skill. Click a button to copy the exact prompt, then paste it into Claude Code in this repo.

Open the tutor ⧉ Start the gate ⧉

1. Learn · acquire the concept

No external course — read this brief, then go deeper in the four reference pages below.

Mental model (20 minutes):

Four skills stack on top of each other, and each one assumes the one before it:

flowchart LR
    A["Navigate & permissions<br/>move around, read/set access"] --> B["Combine tools<br/>pipes & redirection"]
    B --> C["Text tools<br/>grep · sed · awk · cut · sort · uniq"]
    C --> D["Process control<br/>ps · top · kill · jobs"]
    D --> E["Bash scripting<br/>variables · loops · functions · exit codes"]

• Navigate & permissions — you can't run or read anything until you can find it (pwd/cd/ls) and confirm you're allowed to touch it (rwx, chmod, ownership).
• Combine tools — the shell's actual power: a pipe (|) sends one command's output straight into the next command's input; redirection (>, >>, <, 2>) moves a stream to or from a file instead.
• Text tools — grep finds lines, sed transforms them, awk computes over columns, and cut/ sort/uniq round out the common "find → extract → count" pipeline shape.
• Process control — ps/top show what's running; kill sends a signal (not always a hard termination); jobs/fg/bg manage what's running in your own shell.
• Bash scripting — once you can do a task by hand, a script is the same commands, made repeatable, with set -euo pipefail turning a silent partial failure into a loud, immediate stop.

Go deeper: Navigation & permissions · Text processing & pipes · Process monitoring · Bash scripting

Check yourself — what's the difference between a pipe and redirection, in one sentence?

A pipe (|) sends one command's output into another command's input; redirection (>, >>, <) moves a stream to or from a file instead — same underlying idea (connect a stream somewhere), two different destinations.

Check yourself — you need the 5 most common values in a log file's 4th column. Which tools do you chain?

awk '{print $4}' | sort | uniq -c | sort -rn | head -5 — extract the column, sort so duplicates sit next to each other, count adjacent duplicates, sort the counts descending, then take the top 5.

Done when: you can name, in order, the four skills this module builds on each other (navigate/permit → combine → text tools → process control → scripting), before moving to Pair.

2. Pair · passive → active

Drive the onboarding-tutor:

• "Give me a file's ls -l permission string and quiz me on what each of the nine bits means, then make me convert it to octal."
• "Give me a messy log file scenario and make me pick between grep, sed, and awk for it — then explain why the other two would have been the wrong choice."
• "Quiz me on the difference between SIGTERM and SIGKILL, and when I'd actually want each one."
• "Walk me through what set -euo pipefail protects against, one flag at a time, with a concrete failure example for each."
• "Give me a small text-processing task and make me write the full pipeline unprompted — don't let me skip straight to awk if a simpler tool would do."

Common gotchas

• Reaching for chmod 777 or kill -9 as a first move. Fix: both are the maximally destructive option — 777 grants write to everyone, SIGKILL skips all cleanup. Diagnose the actual missing bit or try SIGTERM first; reach for the heavy option only once the precise one has failed.
• Writing a pipeline with the wrong tool for the job. Fix: "which lines" is grep, "transform each line" is sed, "compute/extract from columns" is awk — picking the heaviest tool out of habit (awk for everything) works, but picking the right one is the skill being tested.
• Trusting a script's "it finished" as "it worked." Fix: without set -euo pipefail and explicit exit codes, a script can fail halfway through and still report success to whatever called it — always check what a script actually returns, not just whether it stopped.
• Treating uniq as "count all duplicates in the file." Fix: uniq only collapses adjacent identical lines — always sort first, or it silently undercounts.

Done when: you can pick the right text-processing tool for a given task unprompted, and explain what each flag in set -euo pipefail individually protects against, without notes.

3. Do · produce an artifact

Exercise — a defensive log-summary script:

1. Pick (or generate) a plain-text log file with at least a timestamp-like first field and a status word somewhere in each line (e.g. INFO/WARN/ERROR).
2. Write log-summary.sh: a bash script, starting with #!/usr/bin/env bash and set -euo pipefail, that takes the log file's path as its one argument ($1).
3. Inside the script, validate the argument: if no path was given, or the file doesn't exist, print a clear error to stderr and exit 1 (do not let the script fall through to processing a file that isn't there).
4. Have it print a count of lines matching each status word (ERROR, WARN, INFO, or whatever your file uses), highest count first — build this with a real pipeline (grep/awk/sort/uniq -c), not a hand-rolled loop reinventing what those tools already do.
5. Make the script executable (chmod +x log-summary.sh) and run it two ways: once against your real log file, once with no argument at all, to confirm the error path actually fires and exits non-zero.

Done when: ./log-summary.sh <file> prints an accurate, sorted count per status word; ./log-summary.sh (no argument) prints a clear error to stderr and exits with a non-zero code; and echo $? after each run shows the exit code you expect.

4. Prove · understanding gate

Mandatory. Pass bar in the Socratic gate.

1. Pipelines: Walk through your script's counting pipeline stage by stage — what does each command contribute, and what would break (or silently misbehave) if you dropped the sort before uniq -c?
2. Tool choice: Why awk/grep for the parts you used them for, and not the others? Where would sed have been the wrong tool for this exercise?
3. Defensive scripting: What does set -euo pipefail actually do to your script if the log file path is wrong? Walk through what would happen without it.
4. Process control: If this script were long-running and stuck, how would you find its PID and stop it — and which signal would you reach for first, and why?
5. Provenance: Which specific line of your script (or explanation of a flag/command) did the tutor suggest, and how did you verify it was correct — by testing the actual behavior, not by trusting the explanation?

Pass bar: you can defend every command in your pipeline and every flag in set -euo pipefail by what it specifically prevents — not just "it's good practice" — and show how you verified a tutor-suggested detail against real command output. "It works but I can't say why" = back to Pair.

5. Retain · fight forgetting

This module has no generated Anki deck yet. Recall by re-reading the four reference pages and re-running the Do exercise's script against a different log file (or a different status vocabulary) until you can write the pipeline and the set -euo pipefail guards from memory, unprompted. Done when you can write both, cold, for any log file someone hands you.

Key takeaways

• The shell's power comes from chaining small, single-purpose tools — grep finds, sed transforms, awk computes over columns — not from memorizing one command that does everything.
• A pipe (|) connects one command's output to another command's input; redirection (>, >>, <, 2>) connects a stream to a file instead — same idea, different destination.
• kill sends a signal, it doesn't inherently terminate — SIGTERM (default) asks a process to clean up and exit; SIGKILL (-9) ends it immediately with no chance to clean up, and should be the second move, not the first.
• A script's exit code (0 = success, non-zero = failure) is how it reports outcome to whatever calls it — a script with no exit-code discipline can "finish" while lying about whether it actually worked.
• set -euo pipefail closes three specific gaps: -e stops on the first failing command, -u errors on any undefined variable, and pipefail makes a pipeline's exit code reflect its first failure, not just its last command's.

Go deeper — curated resources

Hand-picked external material to take this topic further — the best books, courses, talks, and writing. Cost is tagged Free, Free online (full text/course free on the web), or Paid.

• Book · The Linux Command Line — William Shotts. The standard from-scratch introduction to the shell and bash scripting; the full text is free to download as a PDF. Free online.
• Docs · Greg's Bash Guide — the Wooledge wiki. A thorough, practically-minded reference for writing correct bash, from basics through arrays and job control. Free.
• Docs · Google Shell Style Guide — Google. The conventions real teams hold shell scripts to (quoting, structure, when not to use bash). Free.
• Tool · ShellCheck — a static analyzer that catches quoting bugs, portability issues, and common scripting mistakes before they bite you in production. Free.
• Video · The Missing Semester of Your CS Education — MIT. A from-first-principles course on the shell, command-line environment, and scripting that most CS curricula skip. Free.

---

Gate log (mentor fills on pass)

• Date passed:
• Passed by: / checked by:
• Weak spots noticed (feeds system improvement):