The command line & automation

The terminal always has a current folder (your "you are here"). These four commands are how you find out where you are, see what's around you, and move somewhere else — the absolute basics of getting around a shell.

1. pwd means print working directory — it prints the full path of the folder you're currently sitting in, like /home/you.
2. ls -la lists what's in the current folder. The -l flag makes it a detailed "long" listing (sizes, dates, permissions) and -a shows all files, including hidden ones whose names start with a dot.
3. cd projects changes directory into a folder called projects. The comment lists the handy shortcuts: cd .. goes up one level, cd ~ jumps to your home folder, and cd - hops back to the previous folder.
4. tree -L 1 draws a little picture of the folder one level deep — useful, but it's an extra tool you may need to install first.

What the output means: Nothing is changed on disk — pwd and ls just report, and cd only moves you. You end up standing inside projects.

Try this: Open a terminal and type pwd, then ls -la. Do you see files whose names begin with a dot (like .git)? Those were hidden until you added -a.

This block creates a folder, then writes, shows, copies, moves, and deletes a file — the everyday file chores you'd otherwise do by clicking around, done with typed commands.

1. mkdir -p demo/sub makes a folder demo and a folder sub inside it in one go; -p means "create parents too, and don't complain if they already exist". && means "only run the next command if this one succeeded", then cd demo moves inside.
2. touch a.txt creates an empty file. The single > then writes hello into it, replacing whatever was there; the double >> appends a new line instead of overwriting. That difference matters a lot.
3. cat a.txt prints the file's contents to the screen; wc -l a.txt counts the lines in it.
4. cp a.txt b.txt copies the file, mv b.txt sub/c.txt moves (and renames) it into the sub folder, and rm a.txt deletes the original.

What the output means: After this runs you have demo/sub/c.txt containing the two lines, and the original a.txt is gone.

Try this: Run just the two echo lines, then cat a.txt. Now change the second one from >> to > and re-run — you'll see the file drop to a single line because > overwrites.

A pipe (the | symbol) feeds one command's output straight into the next as its input. This is the Unix superpower: chain tiny single-purpose tools into an answer. Read each line left-to-right as a small assembly line.

1. grep "ERROR" app.log finds every line containing ERROR; piping | wc -l counts those lines — so the whole line answers "how many errors?".
2. grep -rn "TODO" src/ searches recursively (-r, through every file under src/) and prints line numbers (-n) — great for finding leftover to-dos.
3. The access.log line is a four-stage pipe: pull the first column (the IP) with awk, sort them, uniq -c counts duplicates, sort -rn orders by count high-to-low, and head shows the top few — your busiest visitors.
4. The last two lines show redirection: > py_files.txt saves output into a file instead of the screen, and 2> errors.txt sends only the error stream (stderr) to its own file.

What the output means: Each line prints its result to the screen, except the last two which quietly write their output into py_files.txt and errors.txt.

Try this: Build a pipe one stage at a time: run grep "ERROR" app.log alone, then add | wc -l. Watching the output change as you add each stage is the fastest way to understand pipes.

Once automation gets serious you do file work in Python instead of shell, because it's portable and easy to guard. pathlib is Python's modern way to handle files and folders: a Path object represents a location, and you build sub-paths with the / operator.

1. Path(tempfile.mkdtemp()) makes a fresh temporary folder just for this demo, so it can't touch your real files. (base / "logs").mkdir() creates a logs folder inside it — notice / joins path pieces here, it is not division.
2. f.write_text(...) writes four lines into a new app.log. The \n markers are newlines, so it's really four separate log lines.
3. f.read_text().splitlines() reads the file back and splits it into a list of lines. The next line keeps only the lines that startswith("ERROR") — the Python equivalent of grep.
4. The glob("*.log") loop finds every file ending in .log (just like ls *.log) and prints each one's name and size in bytes from child.stat().

What the output means: It prints total lines: 4 | errors: 2, then one found: line for app.log with its size — matching the console block shown below.

Try this: Add another ERROR line to the write_text string and re-run; the error count and total both go up. This is the same idea as grep ERROR | wc -l, but in Python.

Build tools, deploy scripts and CI runners all need to run shell commands from code and react to whether they worked. subprocess.run is Python's way to do that: it runs a command, captures its output, and reports an exit code (0 means success, anything else means failure).

1. subprocess.run(cmd, capture_output=True, text=True) runs the command. capture_output=True grabs what it printed, and text=True hands it back as normal strings instead of raw bytes.
2. if r.returncode != 0: checks the exit code. A non-zero code means the command failed, so the function raises an error with the details instead of silently continuing with bad data — this is "fail loudly".
3. On success it returns r.stdout.strip() — the command's normal output with surrounding whitespace removed.
4. The two print calls pass commands as a list (["python3", "--version"], ["ls"]). Each list item is one argument — no quoting or shell parsing needed.

What the output means: You see python: Python 3.x.x and the first three filenames from ls in the current folder.

Try this: Change a command to something that fails, like run(["ls", "/no/such/dir"]), and watch the function raise a clear RuntimeError with the exit code — exactly what you want a script to do when a step breaks.

A professional command-line tool reads flags, prints --help, and returns proper exit codes. argparse is Python's standard library for that. The key design idea shown here: keep the real work in a plain function, and let a thin main() handle the command-line wiring.

1. count_lines(paths) is the pure logic: given a list of file paths, it returns a dictionary of {path: line count}. It knows nothing about the command line, which makes it easy to test on its own.
2. ArgumentParser describes the tool. add_argument("paths", nargs="+") says "accept one or more file paths", and the --total option with action="store_true" is an on/off flag — present means true.
3. args = ap.parse_args(argv) reads the command line into args. Then the code either prints the grand total (if --total was given) or one line per file. return 0 signals success to the shell.
4. The last lines create a test file and call main([...]) directly so the demo runs here; normally you'd run python linecount.py somefile from the terminal instead.

What the output means: It prints 3 /tmp/df_cli.txt then the same counts as a dictionary — see the console block below.

Try this: Add -t to the call: main(["-t", "/tmp/df_cli.txt"]). Now it prints just the total. That flag flowed from add_argument("--total") straight into your if args.total: branch.

When the job really is shell (gluing tools together for ops), write it defensively so it stops the moment something goes wrong instead of continuing with a half-broken state. This little script makes a timestamped backup of a folder.

1. set -euo pipefail is the safety belt: -e exits on any failed command, -u errors on an unset variable (a common typo bug), and pipefail makes a pipe fail if any stage in it fails.
2. SRC="${1:?usage...}" reads the first argument the user passed. The :? part means "if it's missing, print this usage message and stop" — so the script refuses to run without a folder.
3. [ -d "$SRC" ] tests that $SRC is really a directory; if not, it prints an error to stderr (>&2) and exits with code 1 (failure).
4. date +%Y%m%d-%H%M%S builds a timestamp like 20260906-141530 for a unique filename, then tar -czf creates a compressed .tar.gz archive of the folder and the final echo reports the result and its size.

What the output means: Run as ./backup.sh myfolder it prints something like backed up myfolder -> backup-20260906-141530.tar.gz (2.1M).

Try this: Run it with no argument at all. Thanks to ${1:?...} you'll get the usage message and the script stops immediately — that's the defensive style protecting you from a silent mistake.

Real tools log instead of using print, so operators can trace what happened: each message gets a level (INFO, ERROR) and a timestamp, and logs go to stderr so the tool's actual output on stdout stays clean and machine-readable.

1. logging.basicConfig(...) sets up logging once: level=logging.INFO shows INFO and above, stream=sys.stderr sends logs to the error stream, and format=... puts a timestamp and level in front of every line.
2. log = logging.getLogger("mytool") gets a named logger so messages are tagged with your tool's name.
3. Inside process, log.info("starting, %d items", len(items)) records progress. The %d is filled in with the count — logging does this substitution for you, which is cheaper than building the string yourself.
4. The try/except guards each item; log.exception(...) logs the full error traceback if something throws, without crashing the whole run.

What the output means: The result: 3 line prints on stdout; the timestamped INFO lines print on stderr (shown mixed together on your screen, but they're separate streams).

Try this: Redirect stdout to a file when you run it (python logging_tool.py > out.txt) — the INFO log lines still appear on screen because they go to stderr, while out.txt holds only the clean result. That separation is the whole point.

The final step a team lead takes: make the tool installable so everyone can just type its name. pyproject.toml is the standard config file that describes a Python package and how to build it.

1. The [project] section is the package's identity card: its name, version, and the minimum Python it needs (requires-python).
2. The [project.scripts] section is the magic: linecount = "team_tools.cli:main" says "create a command called linecount that runs the main function in team_tools/cli.py". After pip install ., typing linecount in any terminal just works.
3. The [build-system] section names the tool that packages it up (hatchling) — standard boilerplate you rarely need to change.

Try this: This is why installed tools like pip or black are commands you can type anywhere: each one declared an entry point exactly like this [project.scripts] line.