Jenkins Course
Stages and Steps
Stages and steps are the two things every Jenkinsfile is built from. Understanding how they nest, how they fail, and which built-in steps do what — that's the skill that separates a pipeline that barely works from one that's actually maintainable.
This lesson covers
Stage structure and nesting → Sequential stages inside parallel → The most useful built-in steps → Error handling inside steps → Structuring a pipeline that's easy to debug
In Lesson 14 you saw the full range of pipeline-level directives. Now let's go inside the stages themselves — how they're shaped, how they interact, what happens when one fails, and which steps you'll reach for in almost every pipeline you ever write.
Inside a Stage — Everything It Can Contain
A stage is not just a wrapper for shell commands. It can carry its own agent, its own environment variables, its own options, a when condition, and a post block — making each stage a self-contained, independently configurable unit.
pipeline {
agent { label 'linux' }
stages {
stage('Full Example Stage') {
// Stage-level agent — overrides the pipeline-level agent for this stage only
// Remove this if you want the stage to use the pipeline agent
agent { label 'linux && docker' }
// Stage-level environment — only available inside this stage
environment {
STAGE_ENV = 'integration'
}
// Stage-level options — timeout and retry apply to this stage only
options {
timeout(time: 10, unit: 'MINUTES')
retry(2)
}
// when — this stage only runs on main or release branches
when {
anyOf {
branch 'main'
branch 'release/*'
}
}
// steps — the actual work happens here
steps {
echo "Running in environment: ${STAGE_ENV}"
sh './run-integration-tests.sh'
}
// Stage-level post — runs when this stage completes
// Useful for publishing results immediately after the stage
post {
always {
junit 'test-results/**/*.xml'
}
failure {
echo 'Integration tests failed — check the test report above'
}
}
}
}
}Important — Stage-level post vs Pipeline-level post
Stage-level post fires immediately when that stage finishes — before the next stage starts. Pipeline-level post fires after all stages are done. Use stage-level post for publishing test results (you want them available as soon as tests run). Use pipeline-level post for final notifications and cleanup.
Sequential Stages Inside Parallel
In Lesson 14 you saw parallel stages running at the same time. What if you need a sequence of steps inside one branch of a parallel block? You can nest sequential stages inside a parallel branch using stages inside the branch.
How sequential + parallel nesting looks in the stage view
Branch A — sequential
Branch B — single stage
pipeline {
agent { label 'linux' }
stages {
stage('Checkout') {
steps { checkout scm }
}
stage('Build and Analyse in Parallel') {
parallel {
// Branch A: two sequential stages inside a parallel branch
stage('Build and Test') {
// 'stages' inside a parallel branch = sequential stages
stages {
stage('Compile') {
steps {
sh './gradlew compileJava'
echo 'Compilation complete'
}
}
// This only runs after Compile finishes successfully
stage('Unit Test') {
steps {
sh './gradlew test'
}
post {
always { junit 'build/test-results/**/*.xml' }
}
}
}
}
// Branch B: runs simultaneously with Branch A
stage('Static Analysis') {
steps {
sh './gradlew checkstyle'
sh './gradlew pmd'
}
}
}
}
stage('Deploy') {
when { branch 'main' }
steps { sh './deploy.sh staging' }
}
}
}The Steps You'll Use in Almost Every Pipeline
Jenkins has hundreds of pipeline steps. But there's a core set of maybe fifteen that appear in nearly every real pipeline. Here they are — what each one does and when to use it.
The scenario:
You're a DevOps engineer at a logistics company building a pipeline for the shipment-tracker service. This pipeline demonstrates all the most common built-in steps in a realistic order — exactly the kind of pipeline you'd build on your first real project.
New steps introduced in this code:
- checkout scm — pulls code from the source control repo configured in the Jenkins job. The most common first step in any pipeline.
- sh — runs a shell command (Linux/Mac). The workhorse of pipeline steps — tests, builds, deployments all use this.
- bat — same as
shbut for Windows batch commands. Use on Windows agents. - echo — prints a message to the console log. Use it to narrate what your pipeline is doing — makes logs much easier to read.
- dir — changes the working directory for the steps inside it. Useful in monorepos where different services live in subdirectories.
- withEnv — temporarily sets environment variables for the steps inside the block. Unlike the
environment { }block, these only apply within thewithEnvscope. - withCredentials — injects Jenkins credentials into the block's scope. More granular than the
environment { credentials() }approach — use when you only need a credential in one specific place. - archiveArtifacts — saves files from the build workspace to Jenkins so they can be downloaded later. Use for build outputs like JARs, WARs, or test reports.
- stash / unstash — temporarily saves files between stages that run on different agents. Like a relay baton — one stage stashes the compiled code, another stage unstashes it to run tests.
- sleep — pauses the pipeline for a given duration. Useful for waiting for a service to start before running smoke tests.
- error — immediately fails the current stage with a custom message. Use inside a
script { }block when you need to fail the pipeline based on a condition.
pipeline {
agent { label 'linux' }
environment {
APP_NAME = 'shipment-tracker'
VERSION = "1.${BUILD_NUMBER}"
}
stages {
stage('Checkout') {
steps {
// Pull the latest code from the configured SCM
checkout scm
echo "Building ${APP_NAME} version ${VERSION}"
}
}
stage('Build') {
steps {
// dir changes the working directory for steps inside it
// Useful when your build file isn't in the repo root
dir('backend') {
sh './gradlew clean build -x test'
}
}
}
stage('Test') {
steps {
// withEnv sets temporary environment variables for this block only
withEnv(['TEST_MODE=unit', 'LOG_LEVEL=DEBUG']) {
sh './gradlew test'
}
}
post {
always {
// Publish JUnit XML results to Jenkins test trend tracking
junit 'backend/build/test-results/**/*.xml'
// Save the test report HTML so it's downloadable from the build page
archiveArtifacts artifacts: 'backend/build/reports/tests/**/*',
allowEmptyArchive: true
}
}
}
stage('Package') {
steps {
sh './gradlew bootJar'
// stash saves the JAR so it can be used in a later stage
// even if that stage runs on a different agent
stash includes: 'backend/build/libs/*.jar', name: 'app-jar'
// Save the JAR as a downloadable artifact in the Jenkins build
archiveArtifacts artifacts: 'backend/build/libs/*.jar'
}
}
stage('Deploy to Staging') {
when { branch 'main' }
steps {
// unstash retrieves the files saved by 'stash' above
unstash 'app-jar'
// withCredentials injects a secret for use inside this block only
// The secret is masked in all console output automatically
withCredentials([string(credentialsId: 'staging-deploy-key',
variable: 'DEPLOY_KEY')]) {
sh "./deploy.sh staging ${VERSION} ${DEPLOY_KEY}"
}
// Wait 15 seconds for the service to start before smoke testing
sleep time: 15, unit: 'SECONDS'
// Quick smoke test to confirm the service is responding
sh "curl --fail https://staging.acmecorp.com/health"
}
}
stage('Validate Deploy') {
when { branch 'main' }
steps {
script {
// Read the health check response and fail if it's not 'ok'
def response = sh(
script: "curl -s https://staging.acmecorp.com/health",
returnStdout: true
).trim()
if (response != 'ok') {
// error() immediately fails this stage with a clear message
error("Health check failed — staging is not responding correctly. Got: ${response}")
}
echo "Health check passed — response: ${response}"
}
}
}
}
post {
success {
echo "✅ ${APP_NAME} ${VERSION} deployed to staging successfully"
}
failure {
echo "❌ ${APP_NAME} ${VERSION} pipeline failed — staging deploy was not completed"
}
always {
cleanWs()
}
}
}Where to practice: Use the Jenkinsfile from Lesson 13 as your base and try adding one new step at a time — start with archiveArtifacts, then stash/unstash, then withCredentials. Watch the Jenkins build page after each run to see the Artifacts section appear. Full step reference at jenkins.io — Pipeline Steps Reference.
Started by SCM change
[Pipeline] Start of Pipeline
[Pipeline] node (agent-linux-01)
[Pipeline] { (Checkout)
[Pipeline] checkout
Cloning repository https://github.com/acmecorp/shipment-tracker.git
> git checkout main
[Pipeline] echo
Building shipment-tracker version 1.87
[Pipeline] { (Build) }
[Pipeline] dir (backend)
[Pipeline] sh
+ ./gradlew clean build -x test
BUILD SUCCESSFUL in 38s
[Pipeline] { (Test) }
[Pipeline] withEnv
[Pipeline] sh
+ ./gradlew test
BUILD SUCCESSFUL in 1m 12s
52 tests completed, 0 failed
[Pipeline] junit — recording test results
[Pipeline] archiveArtifacts — archiving artifacts
Archiving: backend/build/reports/tests/index.html
[Pipeline] { (Package) }
[Pipeline] sh
+ ./gradlew bootJar
BUILD SUCCESSFUL in 14s
[Pipeline] stash
Stashed 1 file(s)
[Pipeline] archiveArtifacts
Archiving: backend/build/libs/shipment-tracker-1.87.jar
[Pipeline] { (Deploy to Staging) }
[Pipeline] unstash
Unstashed 1 file(s)
[Pipeline] withCredentials
Masking supported pattern matches of $DEPLOY_KEY
[Pipeline] sh
+ ./deploy.sh staging 1.87 ****
Deploying shipment-tracker 1.87 to staging...
Deployment complete.
[Pipeline] sleep (15s)
[Pipeline] sh
+ curl --fail https://staging.acmecorp.com/health
ok
[Pipeline] { (Validate Deploy) }
[Pipeline] script
[Pipeline] sh
+ curl -s https://staging.acmecorp.com/health
[Pipeline] echo
Health check passed — response: ok
[Pipeline] stage (post - success)
✅ shipment-tracker 1.87 deployed to staging successfully
[Pipeline] cleanWs — deleting workspace
[Pipeline] End of Pipeline
Finished: SUCCESS
What just happened?
Building shipment-tracker version 1.87— theVERSIONenvironment variable was built usingBUILD_NUMBER, a Jenkins built-in that holds the sequential build number. Each build automatically gets a unique version string without any manual input.[Pipeline] dir (backend)— Jenkins changed into thebackendsubdirectory before running the Gradle command. After thedirblock closes, the working directory returns to the workspace root automatically.[Pipeline] withEnv— the two environment variables were set for the duration of the test command only. They are not visible outside thewithEnvblock.Stashed 1 file(s)— the JAR was saved under the nameapp-jar. It's now available to any stage that callsunstash 'app-jar', even on a different agent. This is how pipelines pass build outputs between stages safely.Masking supported pattern matches of $DEPLOY_KEY— Jenkins detected the credential and confirmed it will be masked. The****in the deploy command is the masked credential value — the actual key was used but never printed.[Pipeline] sleep (15s)— the pipeline paused for 15 seconds to let the service start before the health check ran. The executor was held during this pause — for longer waits, consider freeing the executor with a different approach.Health check passed — response: ok— thesh(returnStdout: true)captured the curl output as a string, which was then compared in theifblock. This is the pattern for making decisions based on command output inside a pipeline.
Structuring a Pipeline You Can Actually Debug
A pipeline that's hard to debug costs your team hours every time something breaks at 2 AM. These four habits make the difference:
One concern per stage
A stage called "Build and Test and Package and Deploy" is a maintenance nightmare. When it fails, you don't know where. Split concerns into separate stages — Compile, Unit Test, Integration Test, Package, Deploy. Each failure is immediately locatable in the stage view.
Echo before every significant action
Add an echo statement before any shell command that talks to an external system — a deploy, a Docker push, a health check. When something hangs, you'll know exactly where it got stuck. Silent pipelines are hard to debug.
Publish test results in stage post, not pipeline post
If you archive test results in the global post block and a deploy stage fails before that, you lose the test report. Publish immediately in a stage-level post { always { } } so the data is captured regardless of what happens later.
Use error() with a clear message
When you detect a problem in a script { } block, don't let the pipeline drift into an ambiguous state. Call error('Descriptive message about what went wrong') — the message appears in the console log and in the build's failure summary, making the root cause obvious without digging.
Teacher's Note
The pipeline that took 10 minutes to write takes 2 hours to debug if the stages are poorly named and the steps are silent. Name everything clearly. Echo liberally. Future-you will be grateful.
Practice Questions
1. Which pipeline step saves files from the current stage so they can be retrieved in a later stage — even on a different agent?
2. Which step saves files from the build workspace to Jenkins so they can be downloaded from the build page later?
3. Which step immediately fails the current stage and prints a custom message to the console log?
Quiz
1. What is the difference between stage-level post and pipeline-level post?
2. How do you run sequential stages inside one branch of a parallel block?
3. What is the correct way to capture the output of a shell command as a variable in a pipeline script{} block?
Up Next · Lesson 16
Environment Variables
Every way to set, read, and pass environment variables in a Jenkins pipeline — built-in variables, custom variables, credential injection, and the ones that trip everyone up.