Jenkins Course
Jenkins for Microservices
One monolith, one pipeline. Thirty microservices, thirty pipelines — each with different build tools, different deploy targets, different team owners, and different release cadences. This lesson covers the patterns that keep that complexity manageable.
This lesson covers
One Jenkinsfile per service → Cross-service pipeline orchestration → Fan-out and fan-in patterns → Service dependency pipelines → Versioning and promotion across environments → The shared library pattern for microservices
The shift from a monolith to microservices doesn't just change how you write code — it changes how you build, test, and deploy it. A monolith pipeline builds one thing. A microservices platform needs a pipeline per service, coordination across services, and a clear answer to: "when service A changes, which dependent services need to be rebuilt and tested?"
The Analogy
Managing CI for microservices is like running an airport with dozens of airlines. Each airline (service) has its own gate, its own schedule, and its own ground crew (team). The control tower (Jenkins) doesn't care about the internal workings of each flight — it just ensures every plane gets a runway slot, departs on time, and the right connections are made. The airline's own crew handles the rest.
The Core Pattern — One Jenkinsfile Per Service
The foundation is simple and non-negotiable: every microservice has its own Jenkinsfile in its own repository. The pipeline only builds, tests, and deploys that one service. It doesn't know about other services. It doesn't trigger other services. It minds its own business.
❌ Anti-pattern: Monolith pipeline for microservices
- One Jenkinsfile builds all 30 services
- Any service failure blocks all deployments
- Full rebuild on every commit to any service
- Teams can't release independently
- One team's slow test suite blocks everyone
✅ Correct pattern: Independent pipelines
- Each service has its own Jenkinsfile and pipeline
- Failures are isolated to the failing service
- Only the changed service rebuilds
- Teams release on their own schedule
- Parallel deploys across services are safe
A Minimal Microservice Jenkinsfile Using Shared Library
With a shared library (Lesson 36), every microservice Jenkinsfile becomes a short declaration of what's unique about that service. The standard CI/CD steps are inherited. Adding a new service takes minutes, not hours.
// Jenkinsfile — payment-api microservice
// This is ALL that's needed. The shared library handles the rest.
@Library('acme-pipelines@v3.0.0') _
// servicePipeline() is a shared library function that encapsulates
// the entire standard build → test → docker → deploy → notify flow
// Teams only override what's different for their service
servicePipeline(
appName: 'payment-api',
language: 'java', // picks the right build container
dockerRegistry: 'registry.acmecorp.com',
deployTargets: [
staging: [namespace: 'staging', autoApprove: true],
production: [namespace: 'production', autoApprove: false]
],
slackChannel: '#payments-team',
// Override only what differs from the default
testCommand: './gradlew test integrationTest',
healthCheckUrl: 'https://payment-api.staging.acmecorp.com/health'
)What servicePipeline() does internally
The servicePipeline() function in the shared library generates a full Declarative pipeline from the config map. It handles: checkout, lint, unit tests, integration tests, Docker build and push, Kubernetes deploy with rollout status, post-deploy health check, rollback on failure, and Slack notification with state-change logic. 12 lines in the Jenkinsfile. 200 lines of tested, reviewed shared logic doing the work.
Cross-Service Orchestration — Fan-Out and Fan-In
Sometimes services aren't completely independent. A change to a shared library or a common API contract might require rebuilding and testing multiple downstream services. Jenkins handles this with a fan-out/fan-in pattern — one trigger kicks off multiple parallel downstream pipelines, and the coordinator waits for all of them to pass.
The scenario:
Your company has a shared acme-commons library that 8 microservices depend on. When acme-commons is updated, all 8 services need to be rebuilt and tested to confirm they haven't broken. You don't want to do this manually — you want the acme-commons pipeline to fan out and trigger all dependent service pipelines automatically.
New terms in this code:
- build() — a Jenkins pipeline step that triggers another Jenkins job by name. Returns a build object you can use to check the result. The step blocks by default — set
wait: falseto trigger and move on without waiting. - parallel() — runs multiple blocks simultaneously in the same stage. Used here to trigger all 8 downstream builds in parallel rather than sequentially — total time equals the slowest service, not the sum of all.
- propagate: false — when triggering a downstream build, this prevents the downstream failure from immediately failing the upstream pipeline. You collect results first, then decide whether to fail.
- build().result — the final result of the triggered build — SUCCESS, FAILURE, ABORTED, or UNSTABLE. Used to aggregate results after all parallel builds finish.
// Jenkinsfile — acme-commons shared library pipeline
// When commons changes, fan out to test all dependent services in parallel
@Library('acme-pipelines@v3.0.0') _
pipeline {
agent { label 'linux' }
stages {
stage('Build and Publish Commons') {
steps {
checkout scm
sh './gradlew clean build publish'
echo "Published acme-commons:${BUILD_NUMBER} to Artifactory"
}
}
stage('Integration Test — Downstream Services') {
steps {
script {
// Define all services that depend on acme-commons
def dependentServices = [
'payment-api',
'checkout-service',
'fraud-detection',
'user-auth',
'notification-service',
'reporting-api',
'audit-service',
'admin-portal'
]
// Build a map of parallel closures — one per service
def parallelBuilds = dependentServices.collectEntries { service ->
["${service}": {
// Trigger each service pipeline with the new commons version
// propagate: false — collect all results before deciding pass/fail
def result = build(
job: "${service}/main",
parameters: [
string(name: 'COMMONS_VERSION', value: "${BUILD_NUMBER}")
],
propagate: false, // don't fail immediately on downstream failure
wait: true // wait for each build to complete
)
// Store result for reporting
return [service: service, result: result.result, url: result.absoluteUrl]
}]
}
// Run all downstream builds in parallel
def results = parallel(parallelBuilds)
// Aggregate results — fail only after ALL builds complete
def failures = results.values().findAll { it.result != 'SUCCESS' }
if (failures) {
def failureList = failures.collect { " ❌ ${it.service} — ${it.result}" }.join('\n')
error("${failures.size()} downstream service(s) failed:\n${failureList}")
}
echo "✅ All ${dependentServices.size()} downstream services passed"
}
}
}
}
post {
failure {
slackSend(
channel: '#platform-alerts',
color: 'danger',
message: "❌ *acme-commons* update broke downstream services — <${env.BUILD_URL}|Build #${BUILD_NUMBER}>"
)
}
success {
slackSend(
channel: '#platform-alerts',
color: 'good',
message: "✅ *acme-commons:${BUILD_NUMBER}* — all 8 downstream services passed"
)
}
always { cleanWs() }
}
}
Where to practice: Create two simple Jenkins jobs — service-a and service-b. Then create a third job that uses build(job: 'service-a') and build(job: 'service-b') in parallel to trigger both. The Jenkins Pipeline Syntax Generator at Pipeline → Pipeline Syntax → Snippet Generator → build generates the correct build() call syntax for your specific setup. Full step reference at jenkins.io — Pipeline Build Step.
[Pipeline] { (Build and Publish Commons) }
BUILD SUCCESSFUL — acme-commons:147 published to Artifactory
[Pipeline] { (Integration Test — Downstream Services) }
[Pipeline] parallel
Starting 8 downstream builds in parallel...
payment-api/main #203 ✅ SUCCESS (1m 42s)
checkout-service/main #188 ✅ SUCCESS (2m 11s)
fraud-detection/main #91 ✅ SUCCESS (3m 04s)
user-auth/main #156 ✅ SUCCESS (1m 28s)
notification-service/main #72 ❌ FAILURE (0m 47s) — 2 tests failed
reporting-api/main #134 ✅ SUCCESS (1m 55s)
audit-service/main #58 ✅ SUCCESS (2m 03s)
admin-portal/main #44 ✅ SUCCESS (4m 12s)
Aggregating results...
1 downstream service(s) failed:
❌ notification-service — FAILURE
[Pipeline] error
hudson.AbortException: 1 downstream service(s) failed
Slack: ❌ acme-commons update broke downstream services
Finished: FAILURE
What just happened?
- 8 builds ran in parallel — total wall time was 4m 12s (the slowest service: admin-portal), not the sum of all durations which would have been ~17 minutes. Parallel fan-out is what makes this pattern practical at scale.
- All 8 builds ran to completion before failure was declared —
propagate: falselet all services run even after notification-service failed at 47 seconds. The commons pipeline collected all results and reported a complete picture. Withoutpropagate: false, the first failure would have aborted the remaining 7 builds mid-run. - notification-service was the only failure — 7 services confirmed compatible with the new commons version. The notification-service team can now investigate their 2 test failures knowing the problem is specific to their service, not a broad API break.
- The acme-commons pipeline itself failed — this is intentional. The commons team cannot publish a version that breaks downstream services. The failure in the commons pipeline is the signal to fix the commons change before it reaches any deployed environment.
Service Versioning and Environment Promotion
In a microservices platform, each service is versioned independently. The challenge is tracking which version of each service is running in which environment. Here's the pattern that keeps this manageable:
Service version matrix — what version is in each environment
Each version tag is build-number + short git commit. Services promote independently — user-auth is on the same version everywhere; payment-api is two builds ahead in dev vs production.
Teacher's Note
The hardest part of Jenkins for microservices isn't the pipeline syntax — it's the discipline of keeping pipelines independent. Resist the temptation to make pipelines aware of each other beyond explicit fan-out orchestration. Every coupling between pipelines is a coupling between teams.
Practice Questions
1. When triggering downstream builds in parallel using build(), which parameter prevents a downstream failure from immediately aborting the remaining parallel builds?
2. Which Jenkins pipeline step triggers another job by name and returns a build object containing the result?
3. What is the correct pipeline structure for a microservices architecture with 30 services?
Quiz
1. Eight downstream service pipelines each take between 1–4 minutes. Why does running them in parallel with parallel() save time?
2. What approach minimises the boilerplate in individual microservice Jenkinsfiles while ensuring consistent standards across all services?
3. In the fan-out pattern, what happens to the acme-commons pipeline when one downstream service fails its integration test?
Up Next · Lesson 42
Migration Strategies
Moving from Freestyle jobs to Pipelines, from one Jenkins version to another, or from Jenkins to a new CI/CD platform — the migration patterns that keep teams building while the infrastructure changes underneath them.