Jenkins Lesson 22 – Pipeline with Docker | Dataplexa
Section II · Lesson 22

Pipeline with Docker

Docker and Jenkins are one of the most common combinations in modern CI/CD. This lesson covers the two directions of that relationship — using Docker as a build environment inside Jenkins, and using Jenkins to build and ship Docker images.

This lesson covers

Docker as a build agent → Building Docker images in a pipeline → Tagging strategies → Pushing to a registry → Multi-stage builds → Docker layer caching → The pipeline patterns every containerised team uses

There are two completely different ways Jenkins interacts with Docker. The first is Docker as the build environment — Jenkins runs your pipeline steps inside a container, giving you a clean, reproducible toolchain without installing anything on the agent itself. The second is Docker as the output — Jenkins builds a Docker image, tags it, and pushes it to a registry as part of the release process.

Most production pipelines use both at the same time. This lesson covers each in depth.

Direction 1 — Docker as the Build Environment

Instead of installing Java, Node.js, Python, or Maven on every agent machine, you pull a Docker image that already has everything installed. Jenkins runs your build steps inside that container. When the build finishes, the container is deleted — the agent's disk is clean, the environment is reproducible, and every build starts from the exact same state.

Without Docker agents

  • Install Java 17 on every agent
  • Install Node 18 on every agent
  • Manage version conflicts between services
  • "Works on my agent, fails on yours"
  • Dependency updates require agent downtime

With Docker agents

  • Agent only needs Docker installed
  • Each pipeline declares its own image
  • Zero version conflicts — isolated containers
  • Identical environment on every run
  • Upgrade the image tag to update dependencies
// Using Docker as the build environment
// The pipeline runs inside a Maven container — no Maven needed on the agent
pipeline {

    agent {
        docker {
            // Use the official Maven image with Java 21
            // This image has Maven and Java pre-installed
            image 'maven:3.9-eclipse-temurin-21'

            // Mount the Maven cache from the host into the container
            // Without this, Maven re-downloads all dependencies on every build
            // $HOME/.m2 is the Maven local repository — expensive to rebuild
            args '-v $HOME/.m2:/root/.m2'
        }
    }

    stages {

        stage('Build') {
            steps {
                // mvn is available because we're running inside the Maven container
                // No 'mvn not found' errors — it's baked into the image
                sh 'mvn --version'   // confirm the correct Maven version
                sh 'mvn clean package -DskipTests'
            }
        }

        stage('Test') {
            steps {
                sh 'mvn test'
            }
            post {
                always {
                    junit 'target/surefire-reports/**/*.xml'
                }
            }
        }

    }

}

// -------------------------------------------------------
// ADVANCED: Different stages in different containers
// Use 'agent none' at the top and declare agents per-stage
// -------------------------------------------------------

pipeline {
    agent none  // no global agent — each stage picks its own

    stages {

        stage('Test — Java') {
            agent {
                docker { image 'eclipse-temurin:21-jdk' }
            }
            steps {
                sh './gradlew test'
            }
        }

        stage('Test — Node') {
            agent {
                docker { image 'node:20-alpine' }
            }
            steps {
                sh 'npm ci && npm test'
            }
        }

        stage('Lint Python') {
            agent {
                docker { image 'python:3.12-slim' }
            }
            steps {
                sh 'pip install flake8 && flake8 scripts/'
            }
        }

    }
}

Key concept — the Maven cache mount

The args '-v $HOME/.m2:/root/.m2' flag mounts the host machine's Maven cache into the container. Without it, Maven downloads every dependency from the internet on every build — a 200MB download that takes 3 minutes. With it, the first build downloads everything and every subsequent build uses the cache. Always mount caches when using Docker agents.

Direction 2 — Building and Pushing Docker Images

The second role Docker plays in Jenkins pipelines is as the output. Jenkins compiles the code, runs the tests, then builds a Docker image, tags it with the build number and commit hash, and pushes it to a registry. That image is what gets deployed to servers or Kubernetes.

The scenario:

You're a DevOps engineer at a logistics company. The shipment-tracker service is being containerised. Your job is to write the pipeline that builds a production-ready Docker image on every push to main, tags it with both the build number and the Git commit hash for traceability, and pushes it to the company's private registry — ready for the Kubernetes deployment pipeline to pick up.

New terms in this code:

  • docker.build() — a Groovy step provided by the Docker Pipeline plugin. Builds a Docker image and returns an image object you can tag and push programmatically.
  • docker.withRegistry() — a block that authenticates with a Docker registry using stored Jenkins credentials. Everything inside the block runs with that registry context — pushes and pulls are authenticated automatically.
  • image.push() — pushes the image to the registry with a specific tag. Call it multiple times with different tags to push the same image under multiple names.
  • image.push('latest') — pushes the same image with the latest tag. latest always points to the most recent successful build on the main branch.
  • Multi-stage Dockerfile — a Dockerfile with multiple FROM stages. The build stage compiles the code; the final stage copies only the compiled artifact into a minimal runtime image. The final image is much smaller because it doesn't contain build tools.
  • IMAGE_TAG — a composite tag combining build number and short commit hash, e.g. 58-a3f2b1c. Using both gives you traceability: the build number links to Jenkins, the commit hash links to GitHub.
pipeline {
    // The agent needs Docker installed to build images
    agent { label 'linux && docker' }

    environment {
        APP_NAME    = 'shipment-tracker'
        REGISTRY    = 'registry.acmecorp.com'
        // Composite tag: build number + short git commit hash
        // e.g. 58-a3f2b1c — links to both Jenkins build and Git commit
        IMAGE_TAG   = "${BUILD_NUMBER}-${env.GIT_COMMIT?.take(7) ?: 'unknown'}"
        DOCKER_CREDS = credentials('docker-registry-credentials')
    }

    stages {

        stage('Checkout') {
            steps {
                checkout scm
                echo "Building image: ${REGISTRY}/${APP_NAME}:${IMAGE_TAG}"
            }
        }

        stage('Test') {
            steps {
                // Run tests first — don't build the image if tests fail
                sh './gradlew test'
            }
            post {
                always { junit 'build/test-results/**/*.xml' }
            }
        }

        stage('Build Docker Image') {
            when { branch 'main' }
            steps {
                script {
                    // docker.build() builds the image from the Dockerfile in the current directory
                    // Returns an image object we can use to tag and push
                    def image = docker.build("${REGISTRY}/${APP_NAME}:${IMAGE_TAG}")

                    // docker.withRegistry() authenticates with the registry
                    // Uses credentials from the Jenkins credential store — never plain text
                    docker.withRegistry("https://${REGISTRY}", 'docker-registry-credentials') {

                        // Push with the composite tag (build number + commit hash)
                        // This is the immutable, traceable tag — never changes after push
                        image.push(IMAGE_TAG)

                        // Also push as 'latest' — always points to the newest main build
                        // Kubernetes deployments can reference 'latest' for auto-update
                        image.push('latest')

                    }
                    echo "Pushed: ${REGISTRY}/${APP_NAME}:${IMAGE_TAG}"
                    echo "Pushed: ${REGISTRY}/${APP_NAME}:latest"
                }
            }
        }

        stage('Security Scan') {
            when { branch 'main' }
            steps {
                // Trivy is an open-source container vulnerability scanner
                // It scans the image for known CVEs before it goes anywhere near production
                sh """
                    docker run --rm \
                      -v /var/run/docker.sock:/var/run/docker.sock \
                      aquasec/trivy image \
                      --exit-code 1 \
                      --severity HIGH,CRITICAL \
                      ${REGISTRY}/${APP_NAME}:${IMAGE_TAG}
                """
            }
        }

        stage('Update Deployment Manifest') {
            when { branch 'main' }
            steps {
                // Update the Kubernetes deployment YAML with the new image tag
                // The GitOps pipeline (ArgoCD/Flux) picks this up and deploys automatically
                sh """
                    sed -i 's|image: ${REGISTRY}/${APP_NAME}:.*|image: ${REGISTRY}/${APP_NAME}:${IMAGE_TAG}|g' \
                        k8s/deployment.yaml
                    git config user.email 'jenkins@acmecorp.com'
                    git config user.name 'Jenkins'
                    git add k8s/deployment.yaml
                    git commit -m 'ci: update ${APP_NAME} image to ${IMAGE_TAG} [skip ci]'
                    git push origin main
                """
            }
        }

    }

    post {
        success {
            slackSend(
                channel: '#deployments',
                color: 'good',
                message: "🐳 *${APP_NAME}:${IMAGE_TAG}* pushed to registry — <${env.BUILD_URL}|Build #${BUILD_NUMBER}>"
            )
        }
        failure {
            slackSend(
                channel: '#deployments',
                color: 'danger',
                message: "❌ *${APP_NAME}* Docker build failed — <${env.BUILD_URL}|Build #${BUILD_NUMBER}>"
            )
        }
        always {
            // Clean up local Docker images after pushing
            // Prevents the agent disk from filling up with old images
            sh "docker rmi ${REGISTRY}/${APP_NAME}:${IMAGE_TAG} || true"
            sh "docker rmi ${REGISTRY}/${APP_NAME}:latest || true"
            cleanWs()
        }
    }

}

Where to practice: Install the Docker Pipeline plugin from Manage Jenkins → Plugin Manager to get the docker.build() and docker.withRegistry() steps. You'll also need Docker installed and running on your Jenkins agent. For a free registry to push to, use Docker Hub (free for public images) or GitHub Container Registry. Full Docker Pipeline docs at jenkins.io — Using Docker with Pipeline.

Started by GitHub push by dev-lena (branch: main)
[Pipeline] Start of Pipeline
[Pipeline] node (agent-linux-01)
[Pipeline] { (Checkout) }
[Pipeline] checkout
> git checkout main — HEAD: a3f2b1c
[Pipeline] echo
Building image: registry.acmecorp.com/shipment-tracker:58-a3f2b1c
[Pipeline] { (Test) }
+ ./gradlew test
BUILD SUCCESSFUL — 63 tests completed, 0 failed
[Pipeline] { (Build Docker Image) }
[Pipeline] script
[Pipeline] docker.build
Building image: registry.acmecorp.com/shipment-tracker:58-a3f2b1c
Step 1/8 : FROM eclipse-temurin:21-jre-alpine AS runtime
Step 2/8 : COPY --from=build /app/build/libs/shipment-tracker.jar /app/
...
Successfully built 7b4c2d1e9f3a
Successfully tagged registry.acmecorp.com/shipment-tracker:58-a3f2b1c
[Pipeline] docker.withRegistry
[Pipeline] image.push (58-a3f2b1c)
Pushed: registry.acmecorp.com/shipment-tracker:58-a3f2b1c
[Pipeline] image.push (latest)
Pushed: registry.acmecorp.com/shipment-tracker:latest
[Pipeline] echo
Pushed: registry.acmecorp.com/shipment-tracker:58-a3f2b1c
Pushed: registry.acmecorp.com/shipment-tracker:latest
[Pipeline] { (Security Scan) }
registry.acmecorp.com/shipment-tracker:58-a3f2b1c (trivy)
Total: 0 (HIGH: 0, CRITICAL: 0)
[Pipeline] { (Update Deployment Manifest) }
[Pipeline] sh
+ sed -i 's|image: registry.acmecorp.com/shipment-tracker:.*|image: registry.acmecorp.com/shipment-tracker:58-a3f2b1c|g' k8s/deployment.yaml
+ git commit -m 'ci: update shipment-tracker image to 58-a3f2b1c [skip ci]'
[main a4b5c6d] ci: update shipment-tracker image to 58-a3f2b1c [skip ci]
[Pipeline] post (success)
Slack: 🐳 shipment-tracker:58-a3f2b1c pushed to registry
[Pipeline] docker rmi (cleanup)
[Pipeline] cleanWs
Finished: SUCCESS

What just happened?

  • 58-a3f2b1c — the IMAGE_TAG was constructed from BUILD_NUMBER (58) and the first 7 characters of GIT_COMMIT (a3f2b1c). This tag is immutable — it will always point to this exact build of this exact code. Six months from now, you can look at a running container and trace it back to both the Jenkins build and the Git commit instantly.
  • docker.build() returned an image object — the Docker Pipeline plugin wraps the docker CLI and gives you a Groovy object. Calling image.push() on it is cleaner than writing sh 'docker push ...' manually and handles authentication correctly.
  • Two tags pushed58-a3f2b1c is the immutable version tag. latest is the mutable convenience tag that always points to the newest build. Use the version tag in deployment manifests for reproducibility. Use latest only when you genuinely want auto-update behaviour.
  • Trivy found 0 HIGH or CRITICAL vulnerabilities — the security scan passed. The --exit-code 1 flag means if Trivy had found any HIGH or CRITICAL CVEs, it would have exited with code 1, failing the Jenkins stage and stopping the pipeline before the image was ever deployed.
  • Deployment manifest updated with [skip ci] — the pipeline committed a change to k8s/deployment.yaml with the new image tag. The [skip ci] in the commit message tells Jenkins not to trigger another build on this commit — preventing an infinite loop.
  • Docker images cleaned up in post — the || true at the end of the rmi commands means the cleanup never fails the build even if the image was already removed. Always clean up local Docker images — agent disks fill up fast otherwise.

Docker Tagging Strategy

How you tag Docker images directly affects how traceable and deployable your releases are. Here are the three most common strategies:

Strategy Example tag Pros / Cons
Build number :58 Simple and sequential. Doesn't link to Git without checking Jenkins.
Git commit hash :a3f2b1c Directly traceable to code. Hard to tell which is newer without checking Git.
Build + commit (recommended) :58-a3f2b1c Sequential ordering + direct Git traceability. Best of both. Use this.
Semantic version :2.4.1 Human readable. Requires version management discipline. Good for public images.

Teacher's Note

Never deploy latest to production. Always use the immutable version tag. latest is a convenience label — it changes every build. Pin your production deployments to a specific tag you can roll back to.

Practice Questions

1. Which Docker Pipeline plugin block authenticates with a Docker registry using stored Jenkins credentials before pushing an image?



2. When using a Maven Docker agent, what args value mounts the Maven dependency cache from the host into the container to avoid re-downloading dependencies on every build?



3. What string do you add to a Git commit message to prevent Jenkins from triggering a new build on that commit?



Quiz

1. What does using agent { docker { image '...' } } in a Jenkinsfile do?


2. Why should you never deploy the latest Docker tag to production?


3. What does the Trivy security scan step in the pipeline do when it finds a CRITICAL vulnerability?


Up Next · Lesson 23

Pipeline with Kubernetes

Running Jenkins agents as Kubernetes pods, deploying to clusters from pipelines, and the patterns teams use to ship to Kubernetes without a dedicated ops team.