Jenkins Lesson 21 – Pipeline with Git | Dataplexa
Section II · Lesson 21

Pipeline with Git

Jenkins and Git are inseparable in real CI/CD. This lesson covers how to connect them properly — branch strategies, pull request builds, Multibranch pipelines, and structuring your repo so both Jenkins and your team stay sane.

This lesson covers

checkout scm in depth → Branch-specific pipeline behaviour → Multibranch Pipeline setup → Pull request builds → Git credentials → Useful Git environment variables → Common branch strategy patterns

In every lesson so far, checkout scm has appeared as a single line that magically pulls code. That line hides a lot. This lesson pulls back the curtain — what SCM checkout actually does, how to configure it for different branch strategies, and how Multibranch Pipelines turn a repo with many branches into a fleet of pipelines that manage themselves.

What checkout scm Actually Does

checkout scm is shorthand. When Jenkins sees it, it reads the source control configuration from the Pipeline job itself — the repo URL, credentials, and branch — and runs a Git clone or pull into the workspace. It also sets several environment variables automatically that you can use throughout the pipeline.

Environment variables set automatically after checkout scm

GIT_COMMIT

The full SHA of the commit being built. Use .take(7) to get a short hash.

GIT_BRANCH

The branch name including remote prefix — e.g. origin/main. Use BRANCH_NAME for just the branch name.

BRANCH_NAME

The short branch name — e.g. main or feature/add-retry. Set automatically in Multibranch Pipelines.

GIT_URL

The repository URL Jenkins checked out from. Useful for logging and audit trails.

GIT_PREVIOUS_COMMIT

The SHA of the previous successful build's commit. Useful for generating changelogs between builds.

CHANGE_ID

The pull request number — only set when building a PR in a Multibranch Pipeline. Use to post PR status checks back to GitHub.

Multibranch Pipeline — One Setup, Infinite Branches

A regular Pipeline job builds one branch. A Multibranch Pipeline job scans your entire repository and automatically creates a separate pipeline for every branch it finds. Push a new branch — Jenkins creates a pipeline for it. Delete the branch — Jenkins removes the pipeline. Zero manual job management.

What Multibranch Pipeline looks like in the Jenkins UI

Jenkins — checkout-service (Multibranch Pipeline)
S Branch Last Build Status
main 3 min ago SUCCESS
feature/add-retry-logic 12 min ago SUCCESS
feature/payment-v2 1 hr ago FAILED
release/v2.1 2 hrs ago SUCCESS

Each branch is a separate pipeline. All managed automatically. Create or delete a branch in Git — Jenkins keeps up.

Setting Up a Multibranch Pipeline

Creating a Multibranch Pipeline in Jenkins is a one-time setup — after that, Git drives everything automatically:

1

New Item → Multibranch Pipeline

Give it a name matching the service — e.g. checkout-service. Select Multibranch Pipeline from the job type list.

2

Branch Sources → Add source → GitHub (or Git)

Paste the repository URL. Add credentials for private repos. Jenkins will discover all branches and PRs automatically.

3

Set Scan Interval or configure a webhook

Under Scan Multibranch Pipeline Triggers, set a periodic scan (e.g. every 1 minute) or configure GitHub webhooks (preferred — faster, less polling overhead).

Save — Jenkins scans and creates branch pipelines automatically

Jenkins immediately scans the repository, finds all branches with a Jenkinsfile, and creates a pipeline for each. From now on, push a branch → pipeline appears. Delete a branch → pipeline disappears.

A Branch-Aware Jenkinsfile

The scenario:

You're a senior engineer at an e-commerce platform. The team uses a GitFlow-style branching strategy: feature branches for development, a main branch for staging deploys, and a release/* naming convention for production deployments. The same Jenkinsfile needs to handle all three differently — feature branches run tests only, main deploys to staging, release/* deploys to production with an approval gate.

New terms in this code:

  • branch pattern matching — the when { branch 'release/*' } syntax uses glob-style matching. The * wildcard matches any characters, so release/v2.1, release/v3.0, and release/hotfix-99 all match.
  • CHANGE_ID — a Jenkins environment variable set only during pull request builds. Its presence tells you the pipeline is building a PR, not a regular branch push. Used to skip deploy stages on PR builds.
  • sh returnStatus: true — runs a shell command and returns the exit code as an integer instead of failing the pipeline. Use when you want to check whether a command succeeded without failing the whole build if it didn't.
  • git log — a standard Git command. Used here inside the pipeline to generate a changelog between the previous successful build and the current commit.
pipeline {
    agent { label 'linux' }

    options {
        timeout(time: 30, unit: 'MINUTES')
        buildDiscarder(logRotator(numToKeepStr: '20'))
        timestamps()
    }

    environment {
        APP_NAME     = 'checkout-service'
        DOCKER_CREDS = credentials('docker-registry-credentials')
        REGISTRY     = 'registry.acmecorp.com'
    }

    stages {

        stage('Checkout') {
            steps {
                checkout scm
                script {
                    // Log key Git context for this build
                    echo "Branch: ${env.BRANCH_NAME}"
                    echo "Commit: ${env.GIT_COMMIT?.take(7)}"
                    echo "Repo:   ${env.GIT_URL}"

                    // Generate a changelog from the previous build's commit to now
                    // GIT_PREVIOUS_SUCCESSFUL_COMMIT is the last commit that passed
                    if (env.GIT_PREVIOUS_SUCCESSFUL_COMMIT) {
                        def changelog = sh(
                            script: "git log --oneline ${env.GIT_PREVIOUS_SUCCESSFUL_COMMIT}..HEAD",
                            returnStdout: true
                        ).trim()
                        echo "Changes since last successful build:\n${changelog}"
                    }
                }
            }
        }

        stage('Test') {
            steps {
                sh './gradlew test'
            }
            post {
                always { junit 'build/test-results/**/*.xml' }
            }
        }

        stage('Build Docker Image') {
            // Skip image build on pure PR builds — test is enough for feedback
            // CHANGE_ID is only set when Jenkins is building a pull request
            when {
                not { expression { return env.CHANGE_ID != null } }
            }
            steps {
                sh """
                    docker login -u ${DOCKER_CREDS_USR} -p ${DOCKER_CREDS_PSW} ${REGISTRY}
                    docker build -t ${REGISTRY}/${APP_NAME}:${BUILD_NUMBER} .
                    docker push ${REGISTRY}/${APP_NAME}:${BUILD_NUMBER}
                """
            }
        }

        // Deploy to staging — only on main branch, not PRs, not feature branches
        stage('Deploy — Staging') {
            when {
                allOf {
                    branch 'main'
                    // Ensure this is not a PR build
                    not { expression { return env.CHANGE_ID != null } }
                }
            }
            steps {
                echo "Deploying ${APP_NAME} build ${BUILD_NUMBER} to staging"
                sh "./deploy.sh staging ${BUILD_NUMBER}"
            }
        }

        // Deploy to production — only on release/* branches
        // Requires manual approval from the release manager
        stage('Deploy — Production') {
            when {
                // branch pattern — matches release/v2.1, release/v3.0, etc.
                branch 'release/*'
            }
            steps {
                // Pause and wait for approval before any production change
                input(
                    message: "Deploy ${APP_NAME} ${BUILD_NUMBER} to PRODUCTION?",
                    ok: 'Deploy',
                    submitter: 'release-managers'
                )
                echo "Deploying ${APP_NAME} build ${BUILD_NUMBER} to production"
                sh "./deploy.sh production ${BUILD_NUMBER}"
            }
        }

    }

    post {
        failure {
            slackSend(
                channel: '#deployments',
                color: 'danger',
                message: "❌ *${APP_NAME}* failed on `${env.BRANCH_NAME}` — <${env.BUILD_URL}|Build #${BUILD_NUMBER}>"
            )
        }
        success {
            script {
                // Only notify on staging and production deploys — not every feature branch build
                if (env.BRANCH_NAME == 'main' || env.BRANCH_NAME?.startsWith('release/')) {
                    slackSend(
                        channel: '#deployments',
                        color: 'good',
                        message: "✅ *${APP_NAME}* deployed from `${env.BRANCH_NAME}` — <${env.BUILD_URL}|Build #${BUILD_NUMBER}>"
                    )
                }
            }
        }
        always { cleanWs() }
    }

}

Where to practice: Create a Multibranch Pipeline job in Jenkins pointing at your test repository. Create three branches — main, feature/test-branch, and release/v1.0. Push the Jenkinsfile to each. Watch Jenkins create three separate pipelines automatically and notice how each one behaves differently based on BRANCH_NAME. Official Multibranch Pipeline guide at jenkins.io — Multibranch Pipelines.

Started by GitHub push by dev-priya (branch: feature/add-retry-logic)
[Pipeline] Start of Pipeline
[Pipeline] node (agent-linux-01)
[Pipeline] { (Checkout) }
[Pipeline] checkout
> git checkout feature/add-retry-logic — HEAD: c8d9e0f
[Pipeline] script
[Pipeline] echo
Branch: feature/add-retry-logic
[Pipeline] echo
Commit: c8d9e0f
[Pipeline] echo
Repo:   https://github.com/acmecorp/checkout-service.git
[Pipeline] echo
Changes since last successful build:
c8d9e0f Add exponential backoff to payment retry logic
b4c5d6e Fix null check in cart total calculation
[Pipeline] { (Test) }
+ ./gradlew test
BUILD SUCCESSFUL — 52 tests completed, 0 failed
[Pipeline] { (Build Docker Image) }
Stage "Build Docker Image" skipped due to when condition
[Pipeline] { (Deploy — Staging) }
Stage "Deploy — Staging" skipped due to when condition
[Pipeline] { (Deploy — Production) }
Stage "Deploy — Production" skipped due to when condition
[Pipeline] post (success)
[Pipeline] script
Branch is feature/add-retry-logic — skipping Slack notification
[Pipeline] cleanWs
Finished: SUCCESS

What just happened?

  • Branch: feature/add-retry-logic — the BRANCH_NAME variable was set automatically by the Multibranch Pipeline. No manual configuration — Jenkins knew which branch it was building because Multibranch creates a separate pipeline context per branch.
  • Changelog printed automatically — the git log --oneline command used GIT_PREVIOUS_SUCCESSFUL_COMMIT to show exactly which commits are in this build. This information is in the build log for free without any extra tools.
  • Three stages skipped — Build Docker Image, Deploy Staging, and Deploy Production were all skipped because this is a feature branch. The when conditions evaluated correctly and Jenkins printed the reason for each skip. The developer gets fast test feedback without triggering any deployments.
  • No Slack notification sent — the success post block checked BRANCH_NAME and found it was neither main nor a release/* branch. The notification was intentionally suppressed. The #deployments channel stays quiet for feature branch builds.
  • If this had been main — the Deploy Staging stage would have run, and a success Slack notification would have been sent. If it were release/v2.1, the production deploy stage would have appeared and waited for a human approval.

Common Branch Strategies and How Jenkins Handles Them

Trunk-based development

All developers commit directly to main (or via very short-lived branches). Jenkins runs tests on every push. Simple — one pipeline, one branch. Works well with high code review discipline and feature flags. The Jenkinsfile only needs staging and production deploy stages on main.

GitFlow

Feature branches → develop → release/* → main. Multibranch Pipeline handles this naturally. Feature branches run tests only. Develop deploys to a dev environment. Release branches trigger production deploys with approval. Main is the production state of record.

GitHub Flow (simplified GitFlow)

Feature branches → pull requests → main. Main is always deployable. Multibranch Pipeline builds every branch and PR. The when { branch 'main' } condition on the deploy stage means merging a PR automatically triggers a staging or production deploy.

Teacher's Note

One Jenkinsfile per repo, Multibranch Pipeline per service. That combination handles 90% of team workflows without extra tooling.

Practice Questions

1. Which Jenkins environment variable holds the short branch name (e.g. main or feature/add-retry) in a Multibranch Pipeline?



2. Which environment variable is set only when Jenkins is building a pull request — and can be checked to skip deploy stages on PR builds?



3. What when { branch } pattern matches branches named release/v2.1, release/v3.0, and release/hotfix-99?



Quiz

1. What is the main advantage of a Multibranch Pipeline job over a regular Pipeline job?


2. Which Git environment variable holds the commit SHA from the last successful build — useful for generating a changelog?


3. How do you run a shell command and capture its output as a string variable inside a pipeline script{} block?


Up Next · Lesson 22

Pipeline with Docker

Build, tag, push, and deploy Docker images from Jenkins — the pipeline patterns every containerised team needs.