CI/CD with Docker

Docker is the dominant container toolchain — the CLI, the image format, the registry protocol, and the build system used by the vast majority of CI/CD pipelines that produce container artifacts. Lesson 27 introduced containers conceptually and showed where they fit in the pipeline. This lesson goes deeper into Docker specifically: how to write Dockerfiles that build fast, stay lean, and hold no security surprises; how Docker's layer caching works and how to exploit it deliberately in a pipeline; how BuildKit unlocks build features that the legacy builder cannot provide; and how the full build-tag-push workflow is structured in a production GitHub Actions pipeline.

Dockerfile Best Practices for CI/CD

A Dockerfile that was written quickly to get something working often becomes a source of slow pipelines, oversized images, and security findings once it reaches CI. The most impactful improvements come from understanding how Docker layers work and ordering instructions to take advantage of caching.

Layer Caching — The Biggest Pipeline Speed Lever

Docker's layer cache is the single most impactful performance optimisation available in a container build pipeline. When Docker builds an image, it checks whether each instruction has been executed before with the same inputs. If the layer is cached and its inputs have not changed, Docker reuses it instantly rather than re-executing the instruction. A cache hit on the dependency installation layer can save two to five minutes per pipeline run.

Cache invalidation happens when any instruction or its inputs change — and it cascades: once a layer is invalidated, every subsequent layer must also be rebuilt regardless of whether their own inputs changed. This is why instruction ordering matters so much. Copying all source files before installing dependencies means every source change invalidates the dependency layer, forcing a full npm ci on every build. Copying only the lock files first means the dependency layer is only invalidated when dependencies actually change.

GitHub Actions runners do not share Docker layer caches between jobs by default — each job starts with an empty cache. BuildKit's cache export feature solves this by exporting the layer cache to the container registry after each build and importing it at the start of the next. The GitHub Actions docker/build-push-action supports this natively with a single configuration option.

BuildKit — The Modern Docker Build Engine

BuildKit is the next-generation Docker build engine, enabled by default in Docker 23+ and available as an opt-in in earlier versions. It provides several capabilities that the legacy builder cannot: parallel stage execution in multi-stage builds, cache mounts that persist across builds without exporting to the registry, secret mounts that inject credentials into a build step without writing them to any layer, and significantly faster builds through improved scheduling.

In GitHub Actions, BuildKit is used through the docker/setup-buildx-action action, which configures a BuildKit builder instance. The docker/build-push-action then uses that builder for all subsequent builds, unlocking registry-backed caching, multi-platform builds, and secret mount support. These three actions together — setup-buildx, login, and build-push — are the standard pattern for Docker builds in production GitHub Actions pipelines.

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write

    steps:
      - uses: actions/checkout@v4

      - name: Set up BuildKit
        uses: docker/setup-buildx-action@v3      # Enable BuildKit builder

      - name: Log in to GHCR
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata for tags
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=sha,prefix=                    # Tag with commit SHA: abc1234
            type=semver,pattern={{version}}     # Tag with semver if triggered by a tag
            type=raw,value=latest,enable={{is_default_branch}}  # latest on main only

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          cache-from: type=registry,ref=ghcr.io/${{ github.repository }}:cache
          cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:cache,mode=max
          # Exports layer cache to registry after build — imported on next run
          # mode=max exports all intermediate layers, not just the final stage

Registry Strategy — Tagging, Retention, and Multi-Platform

A thoughtful registry strategy determines how images are named, tagged, retained, and distributed. The tagging strategy from Lesson 14 applies directly here: every image gets a unique, immutable SHA tag for traceability, and human-readable tags like latest or semantic version tags are applied as additional aliases — never as the sole reference for a deployment.

Multi-platform builds are increasingly relevant as teams run workloads on ARM infrastructure — Apple Silicon developer machines, AWS Graviton instances, and ARM-based Kubernetes nodes. BuildKit supports building for multiple CPU architectures in a single pipeline step using platforms: linux/amd64,linux/arm64 in the build-push-action. The result is a multi-platform manifest: a single image tag that serves the correct architecture to any pulling host automatically. The build takes longer, but every downstream consumer — developer laptop, CI runner, production cluster — pulls the right binary without any configuration change.

Practice Questions

Answer in your own words — then check against the expected answer.

Lesson Quiz