Multi-service apps with Compose

1 · Real apps are several services essential

Picture the demo that goes wrong. You've dockerized your API — one clean image, one docker run, it works. Then you add a database. Now the ritual is: start Postgres in one terminal, wait, copy its address, start Redis in another, remember the three environment variables that glue them together, then launch the API and hope you did it in the right order. Do that by hand every morning, teach it to a new teammate, and reproduce it identically on a CI runner — and you'll feel exactly why single-container knowledge stops being enough the moment your app grows a second moving part.

Here's the mental shift. In CD1 and CD2 you learned to think of one container as a sealed, reproducible box. A real application is not one box — it's a small team of boxes that have to be hired together, introduced to each other, and started in a sane order. Think of a restaurant: the kitchen, the fridge, the till, and the delivery driver are separate stations, but they only produce a meal when they're in the same building and can shout across the pass. Launching them one at a time from separate street corners is not a restaurant; it's four lonely people. Your API, database, cache, and worker are those stations.

The word that captures each box is service — a single job the app performs (serve requests, store state, cache hot data, grind through background work), packaged as its own container. The common beginner mistake is to cram all of that into one giant container "to keep it simple." It feels simpler for about a day, and then you can't restart the database without killing the API, can't scale the worker independently, and can't swap Postgres 16 for 17 without rebuilding everything. Splitting responsibilities into separate services is what lets each piece fail, restart, scale, and upgrade on its own — which is the entire reason the industry containerizes in the first place.

A real app is your API plus a database plus often a cache and a worker. Running and wiring those by hand (right order, networking, env) is error-prone. docker-compose declares the whole stack in one file and runs it with one command.

2 · Your first compose file essential

Stop imagining a script and start imagining a recipe. The natural first instinct is to write a shell script — docker run this, then docker run that, with these flags — a list of commands to execute. Compose asks you to do the opposite: describe the end state you want and let it figure out the commands. That difference — telling the tool what instead of how — is called declarative configuration, and it's why a compose file reads like a description of your app rather than a set of instructions.

The mental model is an architect's blueprint versus a builder's to-do list. A to-do list says "pour foundation, then raise walls, then add roof" and breaks the instant one step is out of order. A blueprint just says "here is the finished house" — and anyone can build it, rebuild it, or check that reality matches. Your docker-compose.yml is the blueprint: it names every service, the image or build each one uses, the ports it opens, the environment it needs, and how they connect. Run docker compose up and Compose reads the blueprint and makes reality match it.

The single most important thing this file quietly gives you is networking by name. When Compose starts your services it drops them all onto one private virtual network and runs a tiny internal DNS, so the API can reach the database at the hostname db — literally the service's name — instead of a brittle IP address that changes every restart. This is the mistake that eats hours: newcomers write localhost:5432 in their connection string and get "connection refused," because inside the API's container localhost means the API itself, not the database. Each container has its own private localhost. The address of another service is always its service name. Watch for the @db:5432 in the file below — that one detail is the whole trick, and getting it wrong is the number-one Compose beginner bug.

One more note on the format: compose files are YAML, where indentation is structure. The lines nested under a service name belong to that service; a stray space or a tab instead of spaces will change the meaning or break the parse entirely. Read it like an outline, not like code.

Each service is a container. Compose builds/pulls each, puts them on a shared network, and lets them reach each other by service name.

docker-compose.ymlservices:
  api:
    build: .                       # build from local Dockerfile
    ports: ["8000:8000"]
    environment:
      DATABASE_URL: postgresql://app:secret@db:5432/app   # 'db' = the service name
      REDIS_URL: redis://cache:6379
    depends_on: [db, cache]
  db:
    image: postgres:16
    environment: { POSTGRES_USER: app, POSTGRES_PASSWORD: secret, POSTGRES_DB: app }
    volumes: ["dbdata:/var/lib/postgresql/data"]   # persist across restarts
  cache:
    image: redis:7
volumes:
  dbdata:

compose.shdocker compose up -d            # build + start everything, detached
docker compose ps               # what's running
docker compose logs -f api      # follow one service's logs
docker compose exec db psql -U app   # shell into a service
docker compose down             # stop + remove (add -v to wipe volumes/data)

3 · The startup-ordering trap intermediate

This is the bug that works on your machine and dies in CI. You add depends_on: [db], run up on your laptop, and it's fine — because your laptop's Postgres happens to boot in a fraction of a second. Then a colleague runs it on a slower machine, or CI runs it in a fresh container with cold disk, and the API face-plants on startup with "connection refused." Nothing changed in your code. What changed is timing, and timing bugs are the cruellest kind: invisible until the one environment where they aren't.

The root cause is a subtle but crucial distinction: started is not the same as ready. When Compose says the db container has "started," it means the process is running — Postgres has been launched. But a database that's just launched is still opening its data files, replaying its write-ahead log, and setting up its listener. For a second or two it's alive but not answering. Think of it like a shop: the owner has arrived and unlocked the door (started), but the till isn't counted and the lights aren't on yet (not ready). A customer who walks in during that gap gets turned away. Your API is that customer, and depends_on in its plain form only waits for the door to unlock — not for the shop to actually open.

The mistake almost everyone makes first is to "fix" this with a sleep 10 in the API's startup — just wait a bit and hope the database is up by then. It's fragile in both directions: too short and it still fails on a slow day; too long and you've wasted ten seconds on every single boot forever. Guessing at time is never the answer. The real fix is to ask the database whether it's ready and only proceed when it says yes. That's what a healthcheck is: a small command Compose runs on the container over and over until it passes, at which point the service is marked healthy. Pair it with the longer depends_on form and condition: service_healthy, and Compose will hold the API back until Postgres genuinely answers — not merely until its container exists.

depends_on controls start order but NOT readiness — Postgres's container can be "started" while the database is still initializing, so your API crashes connecting on boot. The fix is a healthcheck + condition: service_healthy.

health.yml  db:
    image: postgres:16
    environment: { POSTGRES_USER: app, POSTGRES_PASSWORD: secret, POSTGRES_DB: app }
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]   # is the DB accepting connections?
      interval: 5s
      timeout: 3s
      retries: 5
  api:
    depends_on:
      db:
        condition: service_healthy   # start api only AFTER db passes its healthcheck

start_order.pydef start_order(deps):
    """deps: service -> [needs]. Topological sort; raises on a cycle."""
    order, done = [], set()
    def visit(s, stack):
        if s in stack: raise ValueError(f"dependency cycle at {s}")
        if s in done: return
        for d in deps.get(s, []): visit(d, stack | {s})
        done.add(s); order.append(s)
    for s in deps: visit(s, set())
    return order

print(start_order({"api": ["db", "cache"], "worker": ["db"], "db": [], "cache": []}))

['db', 'cache', 'api', 'worker']

4 · Volumes — persist and share data intermediate

Here's the horror story that teaches this lesson once and forever. You've been developing all week against your local Postgres container — real users, real orders, real test data you carefully built up. You run docker compose down to stop for the night, up again in the morning, and your database is empty. Every row gone. Nothing was broken; the container did exactly what containers are designed to do — and if you don't understand why, you will lose data that matters someday.

The reason is that a container's own filesystem is disposable by design. A container is like a hotel room: you can rearrange the furniture and drop things on the floor while you're there, but housekeeping wipes it clean the moment you check out. Everything a program writes inside its container lives in a thin, throwaway layer that is destroyed when the container is removed. That's a feature — it's what makes containers reproducible and cheap to recreate — but it means anything you actually want to keep has to live somewhere outside that disposable layer. That somewhere is a volume: storage that Docker manages separately from any single container, so it survives even when the container is deleted and recreated.

There are two flavours, and confusing them is the classic mistake. A named volume (like dbdata) is storage Docker owns and keeps in its own area — you name it, mount it into the container, and Docker takes care of where it physically lives. That's what you use for a database: you don't care where the bytes sit, you just want them to persist. A bind mount is different — it maps a specific folder from your host machine straight into the container, so the two see the same files live. That's perfect for development (edit code on your laptop, see it change inside the container instantly), but it's the wrong tool for a database, because you'd be tying your data's fate to a fragile host path. Rule of thumb: named volume for data you must not lose, bind mount for code you're actively editing.

Which brings back the horror story: it happens when you never declared a volume at all, so the database wrote into that disposable layer and lost it on removal. Note the sharp edge — docker compose down keeps your named volumes, but down -v deletes them. That -v is a loaded gun: safe when you truly want a clean slate, catastrophic when you fat-finger it on real data.

Containers are ephemeral; their writable layer vanishes on removal. Named volumes persist data (your database) beyond the container. Bind mounts map a host folder in (great for live-reloading code in dev).

5 · Advanced — a full realistic stack advanced

Everything so far has been one idea at a time; now watch them click together. Up to this point you've met services, name-based networking, healthchecks, and volumes as separate lessons. But you never ship "a healthcheck" — you ship a system, and the skill that actually matters is seeing how these pieces compose into one coherent stack. This section is where the course stops teaching parts and starts teaching the machine.

The stack below is deliberately production-shaped: an API to serve requests, a background worker to grind through slow jobs, Postgres for durable state, and Redis for hot data. Think of it as a small factory. The API is the front counter taking orders; the worker is the back room doing the heavy lifting so the counter never gets clogged; Postgres is the filing cabinet that must never lose a record; Redis is the sticky-note board for things you look up constantly. Each has its own reason to exist, and the compose file is the factory floor plan that puts them all under one roof and wires them together.

Two patterns in it are worth naming because they're where beginners either level up or trip. The first is one image, two roles: the API and the worker are built from the same image and differ only by the command that launches them. That's not a shortcut — it's how real teams keep a worker and its API perfectly in sync, since they literally share the same code and dependencies. The mistake it prevents is maintaining two near-identical images that silently drift apart. The second is config in one place: instead of scattering passwords and URLs across the YAML, every service reads from a shared .env file via env_file. That keeps secrets out of the committed compose file and means changing a setting is a one-line edit, not a hunt-and-replace. Read the file below not as five unrelated blocks but as one wired system, and notice how every earlier lesson shows up in it.

Put it together: API + Postgres + Redis + a background worker, with health gating, volumes, env files, and a network. This is a production-shaped local stack.

docker-compose.full.ymlservices:
  api:
    build: .
    ports: ["8000:8000"]
    env_file: .env
    depends_on:
      db: { condition: service_healthy }
      cache: { condition: service_started }
    restart: unless-stopped
  worker:
    build: .
    command: python -m worker           # same image, different entrypoint
    env_file: .env
    depends_on:
      db: { condition: service_healthy }
    restart: unless-stopped
  db:
    image: postgres:16
    env_file: .env
    volumes: ["dbdata:/var/lib/postgresql/data"]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 5s
      retries: 5
  cache:
    image: redis:7
    volumes: ["cachedata:/data"]
volumes: { dbdata: {}, cachedata: {} }

6 · Professional — dev vs prod overrides professional

The temptation here is obvious, and it's a trap. Dev and production genuinely need different settings — in dev you want your code changes to appear instantly, in prod you want the frozen, built image and a restart policy. So the lazy fix is to keep two whole compose files, docker-compose.dev.yml and docker-compose.prod.yml, each a full copy. It works for a week. Then you add a service to one and forget the other, or change a port in dev that never makes it to prod, and now your two environments have quietly diverged — which is exactly the "works on my machine" disease containers were supposed to cure.

The professional pattern is to have one base file that is the single source of truth, plus a small override that lists only what's different for dev. Think of it like a document and its track-changes: the base is the agreed text, and the override is a short list of edits layered on top. You never restate the whole config — you only say "in dev, also mount the code and turn on auto-reload." Because production is the base file untouched, prod can never accidentally inherit a dev-only convenience, and the two environments stay honest with each other.

Compose makes this almost invisible through a naming convention that surprises people: a file literally named docker-compose.override.yml is loaded automatically and merged on top of the base whenever you run docker compose up. You don't pass it on the command line — Compose finds it by name. That's the magic and also the gotcha: the override is on by default, so dev is what you get for free, and reaching production is the deliberate act of pointing at only the base file with -f docker-compose.yml. Get that mental model right — default is dev, prod is the explicit choice — and the whole pattern stops feeling like magic.

Dev and prod differ: dev mounts code for live reload; prod uses the built image, no mount, restart policies. Compose overrides keep both clean from one base file.

override.yml# docker-compose.override.yml  (loaded automatically in dev)
services:
  api:
    volumes: [".:/app"]              # live-mount code -> edits reflect without rebuild
    environment: { ENV: dev }
    command: uvicorn main:app --host 0.0.0.0 --port 8000 --reload

# prod uses ONLY the base file (built image, no mount, restart policy):
#   docker compose -f docker-compose.yml up -d

7 · Tech-lead — one command onboards the team tech-lead

Remember your own worst first day. A new engineer joins, clones the repo, opens a README titled "Local Setup," and finds thirty steps: install this version of Postgres, no wait, that one; create a database with this exact name; install Redis; export nine environment variables; oh, and Mike knows the last two, ask him. A day and a half later they've got it half-working. Multiply that by every hire, every laptop reformat, every "I deleted node_modules and now nothing works," and onboarding friction becomes one of the quiet, expensive taxes on a whole team's time.

This is where Compose stops being a convenience and becomes a leadership tool. The win a tech lead delivers is collapsing that thirty-step README into two commands: git clone and docker compose up. The compose file is the setup instructions — except it's executable, version-controlled, and reviewed like any other code, so it can't rot the way a prose README silently does. When the stack changes, the file changes in the same commit; there's no separate document to forget. New hires are writing real code in minutes instead of days, and "it works on my machine" stops being a sentence anyone says.

But a promise you don't verify is just a hope, and here's the mistake leads make: they assume the compose file is complete because it was complete last month. Services get added, someone comments one out to debug and forgets to restore it, a merge drops a block. The professional move is to make completeness checkable — a tiny automated test that reads the compose file and confirms every service the app needs is actually declared. That's exactly what the runnable check below does, and it's the seed of the kind of guardrail real teams wire into CI so a broken stack fails the build instead of failing a new teammate's first morning.

A lead's win: git clone && docker compose up spins up the entire app — no "install Postgres, then Redis, then set these 9 env vars" README. New hires are productive in minutes. Verify the stack is complete with a check like this.

check_stack.pyimport re
def check_stack(compose_yaml, required):
    present = set(re.findall(r"^  (\w+):", compose_yaml, re.M))
    missing = [s for s in required if s not in present]
    return (not missing), missing

compose = """services:
  api:
    build: .
  db:
    image: postgres:16
  cache:
    image: redis:7"""
print("has api+db+cache:", check_stack(compose, ["api","db","cache"]))
print("needs a worker too:", check_stack(compose, ["api","db","cache","worker"]))

has api+db+cache: (True, [])
needs a worker too: (False, ['worker'])

services:
  api:
    build: .
    ports: ["8000:8000"]
    environment:
      DATABASE_URL: postgresql://app:secret@db:5432/app   # 'db' = the service name
      REDIS_URL: redis://cache:6379
    depends_on: [db, cache]
  db:
    image: postgres:16
    environment: { POSTGRES_USER: app, POSTGRES_PASSWORD: secret, POSTGRES_DB: app }
    volumes: ["dbdata:/var/lib/postgresql/data"]   # persist across restarts
  cache:
    image: redis:7
volumes:
  dbdata:

  db:
    image: postgres:16
    environment: { POSTGRES_USER: app, POSTGRES_PASSWORD: secret, POSTGRES_DB: app }
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]   # is the DB accepting connections?
      interval: 5s
      timeout: 3s
      retries: 5
  api:
    depends_on:
      db:
        condition: service_healthy   # start api only AFTER db passes its healthcheck

def start_order(deps):
    """deps: service -> [needs]. Topological sort; raises on a cycle."""
    order, done = [], set()
    def visit(s, stack):
        if s in stack:
            raise ValueError(f"dependency cycle at {s}")
        if s in done:
            return
        for d in deps.get(s, []):
            visit(d, stack | {s})
        done.add(s)
        order.append(s)
    for s in deps:
        visit(s, set())
    return order

print(start_order({"api": ["db", "cache"], "worker": ["db"], "db": [], "cache": []}))
# -> ['db', 'cache', 'api', 'worker']

services:
  api:
    build: .
    ports: ["8000:8000"]
    env_file: .env
    depends_on:
      db: { condition: service_healthy }
      cache: { condition: service_started }
    restart: unless-stopped
  worker:
    build: .
    command: python -m worker           # same image, different entrypoint
    env_file: .env
    depends_on:
      db: { condition: service_healthy }
    restart: unless-stopped
  db:
    image: postgres:16
    env_file: .env
    volumes: ["dbdata:/var/lib/postgresql/data"]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 5s
      retries: 5
  cache:
    image: redis:7
    volumes: ["cachedata:/data"]
volumes: { dbdata: {}, cachedata: {} }

# docker-compose.override.yml  (loaded automatically in dev)
services:
  api:
    volumes: [".:/app"]              # live-mount code -> edits reflect without rebuild
    environment: { ENV: dev }
    command: uvicorn main:app --host 0.0.0.0 --port 8000 --reload

docker compose -f docker-compose.yml up -d

import re
def check_stack(compose_yaml, required):
    present = set(re.findall(r"^  (\w+):", compose_yaml, re.M))
    missing = [s for s in required if s not in present]
    return (not missing), missing

compose = """services:
  api:
    build: .
  db:
    image: postgres:16
  cache:
    image: redis:7"""
print("has api+db+cache:", check_stack(compose, ["api","db","cache"]))
print("needs worker too:", check_stack(compose, ["api","db","cache","worker"]))
# -> (True, [])  then  (False, ['worker'])

Knowledge check check yourself