Docker Lesson 33 – Secrets Management | Dataplexa
Section III · Lesson 33

Secrets Management

In 2022, a researcher scanned public GitHub repositories and found over 6 million exposed secrets — database passwords, API keys, AWS credentials — sitting in plain text, many of them inside Dockerfiles or docker-compose files committed to version control. Most had been there for years. The applications were running in production the whole time. Secrets management isn't about complex tooling — it's about understanding why a 12-character password in a Dockerfile is a career-ending mistake, and what to do instead.

This lesson covers every common pattern for getting secrets into containers — from the wrong ways developers reach for first, to the right patterns used in production environments at scale. Each approach is shown with concrete code, real implications, and the exact failure mode it prevents.

The Wrong Way vs The Right Way

What most teams start with

  • Passwords hardcoded in the Dockerfile
  • Secrets in ENV instructions — baked into image layers forever
  • .env files committed to version control
  • Secrets passed as plain-text build args
  • Database credentials visible in docker inspect
  • API keys printed in container logs
  • Same credentials across dev, staging, and production

Production-grade secrets handling

  • Zero secrets in Dockerfiles or image layers
  • Runtime injection via environment variables or mounted files
  • .env in .gitignore — never touches version control
  • Build args used only for non-sensitive config
  • Docker Swarm secrets or external vault for production
  • Secrets never logged — scrubbed at the application layer
  • Separate credentials per environment, rotated regularly

Why Secrets in Image Layers Are a Permanent Problem

Docker images are built in layers. Every ENV instruction, every RUN echo, every COPY of a file containing a secret — these are frozen into the image history permanently. Even if you delete the secret in a later layer, it remains readable in the earlier one. Anyone who pulls the image can extract it. Anyone who has ever pulled the image already has it.

# WRONG — the secret is now permanently baked into this image layer
FROM node:18-alpine

ENV DB_PASSWORD=superSecretPassword123
# This ENV instruction creates a layer containing the password in plain text.
# docker history myapp:latest will show it.
# docker inspect myapp:latest will show it.
# Anyone with docker pull access has the password. Forever.
# Removing this line in a later commit does NOT remove it from the layer.

RUN npm install
COPY . .
CMD ["node", "server.js"]
# Anyone can run this to extract the secret from the image:
docker inspect myapp:latest | grep DB_PASSWORD
"DB_PASSWORD=superSecretPassword123"

# Or via image history:
docker history --no-trunc myapp:latest
IMAGE    CREATED BY
sha256   /bin/sh -c #(nop) ENV DB_PASSWORD=superSecretPassword123

# The password is visible in plain text in the image metadata.
# It cannot be removed without rebuilding the image from scratch.
# If the image was ever pushed to a registry, treat the secret as compromised.

This Cannot Be Undone

Once a secret is baked into an image layer and that image has been pushed to a registry, the only correct response is to treat the secret as compromised — rotate it immediately. Deleting the image tag from the registry is not enough. Registries retain layers. Developers who pulled the image still have it locally. The secret is out.

Approach 1 — Environment Variables at Runtime

The simplest correct pattern: keep the Dockerfile entirely free of secrets, and inject them at container startup using -e flags or an --env-file. The image contains no sensitive data. The same image is deployed to every environment — only the runtime values differ.

# CORRECT Dockerfile — no secrets anywhere
FROM node:18-alpine

WORKDIR /app
COPY package*.json ./
RUN npm install --omit=dev
COPY . .

RUN addgroup -S appgroup && \
    adduser -S appuser -G appgroup && \
    chown -R appuser:appgroup /app
USER appuser

EXPOSE 3000
CMD ["node", "server.js"]
# The application reads process.env.DB_PASSWORD at runtime.
# The Dockerfile has zero knowledge of what that value is.
# Option A — pass secrets individually at runtime
docker run -d \
  --name payment-api \
  -e DB_PASSWORD=superSecretPassword123 \
  -e STRIPE_API_KEY=sk_live_abc123xyz \
  -p 3000:3000 \
  payment-api:v1.2.0
# Secrets are injected into the container's environment at start.
# They never touch the image. They live only in the running container.

# Option B — load from a local .env file (NEVER commit this file)
docker run -d \
  --name payment-api \
  --env-file .env.production \
  -p 3000:3000 \
  payment-api:v1.2.0
# .env.production lives only on the deployment server.
# It must be in .gitignore — no exceptions.

# Verify the .gitignore entry exists:
echo ".env*" >> .gitignore
# Confirm the secret is present in the running container:
docker exec payment-api printenv DB_PASSWORD
superSecretPassword123

# Confirm it is NOT in the image:
docker inspect payment-api:v1.2.0 | grep DB_PASSWORD
(no output — the image has no knowledge of this value)

# Confirm the image history is clean:
docker history payment-api:v1.2.0
IMAGE    CREATED BY
sha256   CMD ["node" "server.js"]
sha256   EXPOSE 3000
sha256   USER appuser
# No secrets visible anywhere in the image metadata.

What just happened?

The image was built once, pushed to the registry, and deployed with different credentials for each environment — staging uses test credentials, production uses production credentials. Rotating a secret means restarting the container with a new -e value, not rebuilding and redeploying the image. An attacker who steals the image gets nothing. The secrets exist only on the machines running the containers, injected at startup.

Approach 2 — Docker Compose with Env Files

Docker Compose supports env files natively. The correct pattern: define variable names in the Compose file, populate values in a .env file that lives on the server and never in version control. The docker-compose.yml itself is safe to commit — it contains only variable references, never values.

# .env file — lives on the server, NEVER in the git repository
DB_PASSWORD=superSecretPassword123
STRIPE_API_KEY=sk_live_abc123xyz
REDIS_PASSWORD=redisSecret456
JWT_SECRET=jwt-signing-key-goes-here

# Confirm this pattern is in .gitignore before anything else:
# .env
# .env.*
# docker-compose.yml — safe to commit, contains no secret values
version: "3.8"

services:
  api:
    image: payment-api:v1.2.0
    environment:
      - DB_PASSWORD=${DB_PASSWORD}
      # Compose reads the value from .env at runtime.
      # The compose file only holds the variable name — never the value.
      - STRIPE_API_KEY=${STRIPE_API_KEY}
      - JWT_SECRET=${JWT_SECRET}
    ports:
      - "3000:3000"
    depends_on:
      - db

  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      # Same secret, injected into both services at runtime.
      # One .env file — two consumers — zero hardcoded values.

  redis:
    image: redis:7-alpine
    command: redis-server --requirepass ${REDIS_PASSWORD}
docker compose up -d

[+] Running 3/3
 ✔ Container postgres-db    Started
 ✔ Container redis          Started
 ✔ Container payment-api    Started

# Confirm secret was injected into the API container:
docker exec payment-api printenv DB_PASSWORD
superSecretPassword123

# Confirm docker-compose.yml has no hardcoded values:
grep -i "password" docker-compose.yml
      - DB_PASSWORD=${DB_PASSWORD}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
# Variable references only — no actual secret values in the committed file.

Approach 3 — Docker Swarm Secrets

For production deployments on Docker Swarm, the platform has a built-in secrets system. Secrets are encrypted at rest on the manager nodes, encrypted in transit between nodes, and mounted as files inside containers at a predictable path. They are never visible as environment variables — docker inspect cannot reveal them. Applications read them as files.

# Create a secret — value stored encrypted on the Swarm manager
echo "superSecretPassword123" | docker secret create db_password -
# The `-` reads from stdin — no secret ever touches a file on disk.
# The secret is now encrypted and stored in the Swarm's Raft log.

echo "sk_live_abc123xyz" | docker secret create stripe_api_key -
echo "jwt-signing-key-goes-here" | docker secret create jwt_secret -

# List secrets — names are visible, values are not
docker secret ls
ID                          NAME             CREATED
xk2p9abcd1ef2gh3ij4kl5mn6   db_password      2 minutes ago
op7qr8st9uv0wx1yz2ab3cd4e   stripe_api_key   2 minutes ago

# Deploy a service with access to specific secrets
docker service create \
  --name payment-api \
  --secret db_password \
  --secret stripe_api_key \
  --secret jwt_secret \
  -p 3000:3000 \
  payment-api:v1.2.0
# Each secret is mounted as a file at /run/secrets/<secret_name>
# inside the container — readable only by the process user.
// Application code — reads secrets from files, not environment variables
const fs = require('fs');

function readSecret(name) {
  const secretPath = `/run/secrets/${name}`;
  try {
    return fs.readFileSync(secretPath, 'utf8').trim();
    // .trim() removes the trailing newline Docker appends
  } catch (err) {
    throw new Error(`Secret '${name}' not found at ${secretPath}`);
  }
}

const dbPassword   = readSecret('db_password');
const stripeApiKey = readSecret('stripe_api_key');
const jwtSecret    = readSecret('jwt_secret');

// Values are now in memory only.
// Never in environment variables.
// Never printed by docker inspect.
// Never in the image.
# Inside the running container — secret is a file, not an env var:
docker exec payment-api ls /run/secrets/
db_password   stripe_api_key   jwt_secret

docker exec payment-api cat /run/secrets/db_password
superSecretPassword123

# Attempt to extract via inspect — nothing returned:
docker inspect payment-api | grep -i password
(no output)

# Attempt to see the value via docker secret inspect:
docker secret inspect db_password
[{ "ID": "xk2p9abcd1ef2gh3ij4kl5mn6", "Spec": { "Name": "db_password" } }]
# Name only. The value is never returned by the API — not even to admins.

What just happened?

The secret was created once and stored encrypted inside the Swarm cluster. It was delivered to the container as an in-memory file — not an environment variable, not a volume mount, not anything that persists on disk. When the container stops, the file disappears. When the secret is rotated, the service is updated and restarted — the application reads the new file on startup. No rebuild, no redeployment, no config file changes, and the old secret is immediately invalidated across every node in the cluster.

Build-Time Secrets — The Safe Way with BuildKit

Sometimes the build process itself needs a secret — an SSH key to pull from a private repo, or an npm token to install private packages. --build-arg is the wrong answer: build args are stored in the image history in plain text. Docker BuildKit's --secret flag solves this — the secret is available only during the RUN step that needs it, then discarded. It never appears in any layer.

# syntax=docker/dockerfile:1
FROM node:18-alpine

WORKDIR /app
COPY package*.json ./

# The secret is mounted at /run/secrets/npm_token for this RUN step only.
# After the step completes, the mount is gone — it exists in no image layer.
RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) \
    npm config set //registry.npmjs.org/:_authToken=${NPM_TOKEN} && \
    npm install --omit=dev && \
    npm config delete //registry.npmjs.org/:_authToken
# The token is used, then immediately deleted from .npmrc.
# BuildKit discards the secret mount after the RUN — it leaves no trace.

COPY . .
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
USER appuser
EXPOSE 3000
CMD ["node", "server.js"]
# Build with BuildKit secret — token read from env, never a build arg
DOCKER_BUILDKIT=1 docker build \
  --secret id=npm_token,env=NPM_TOKEN \
  -t payment-api:v1.2.0 .
# id=npm_token   → matches --mount=type=secret,id= in the Dockerfile
# env=NPM_TOKEN  → reads value from the local environment variable NPM_TOKEN
# The token never appears in --build-arg.
# The token never appears in docker history.

# Verify it left no trace:
docker history --no-trunc payment-api:v1.2.0 | grep -i token
(no output — the token left no trace in any layer)

Build-time secrets — wrong vs right

--build-arg Stored in image history in plain text. Visible to anyone with docker history access. Never use for secrets.
ENV in Dockerfile Permanently baked into the image layer. Visible via docker inspect and docker history. Never use for secrets.
--mount=type=secret Available only during the RUN step. Leaves no trace in any layer. The correct solution for build-time secrets.
--mount=type=ssh Forwards the host SSH agent into the build — lets the build clone private repos without copying keys into the image.

The Secrets Checklist

Before any image is built or container is deployed

No secrets in Dockerfiles — no ENV, no hardcoded values, no COPY of credential files
No secrets in build args — use BuildKit --mount=type=secret for build-time credential needs
.env in .gitignore — every .env* file pattern excluded before the first commit
Runtime injection only — secrets passed via -e, --env-file, or Swarm secrets at container start
Swarm secrets for production — encrypted at rest and in transit, never visible via docker inspect
Separate credentials per environment — dev, staging, and production never share secrets
Secrets never logged — application code must never print environment variables or secret file contents
Rotate on exposure — if a secret was ever in version control, even briefly, rotate it immediately

Teacher's Note

Start with runtime environment variables and a properly gitignored .env file. That alone removes 90% of the risk most teams carry. Add Swarm secrets when you move to production. Use BuildKit mounts when your build process needs a token. These are not complex tools — they're habits. The habit of never putting a secret in a Dockerfile is the most valuable one to build.

Practice Questions

1. You have a file containing database credentials for local development that Docker Compose reads automatically. To ensure it never reaches version control, which filename must be added to .gitignore?



2. When Docker Swarm secrets are mounted into a container, at which directory path do the secret files appear?



3. To pass a build-time secret into a Dockerfile RUN step without it appearing in any image layer, which BuildKit instruction is used inside the RUN command?



Quiz

1. A developer adds ENV DB_PASSWORD=mypassword to a Dockerfile, builds the image, pushes it to a private registry, then removes the line in a later commit. What is the security status of that secret?


2. A cluster administrator runs docker secret inspect db_password on a live Swarm node. What does the command return?


3. A team wants to use the same docker-compose.yml across environments but with different database passwords. The compose file is committed to version control. What is the correct approach?


Up Next · Lesson 34

Resource Limits

Secrets secured — now the runtime risk: a single misbehaving container that consumes all available CPU and memory, taking down every other service on the host. Resource limits are the circuit breaker that keeps one bad container from becoming a platform-wide outage.