Terraform Lesson 37 – Terraform in CI/CD | Dataplexa

Section IV · Lesson 37

Terraform in CI/CD Pipelines

Running Terraform locally is fine for learning. In production, every infrastructure change must flow through a pipeline — reviewed, approved, audited, and applied consistently. This lesson builds a complete Terraform CI/CD pipeline from scratch using GitHub Actions, covering keyless OIDC authentication, plan-on-PR, apply-on-merge, approval gates, and the patterns that make automated infrastructure deployments safe.

This lesson covers

Why CI/CD for Terraform → OIDC keyless authentication → Full GitHub Actions pipeline anatomy → Plan on PR with comment output → Saved plan files for approval gates → Apply on merge → Handling multiple environments → Pipeline security hardening

Why CI/CD for Terraform

When engineers run Terraform locally, three problems appear at scale. First, no one else can see what is about to be applied — a colleague cannot review the plan before real infrastructure changes. Second, credentials must be distributed to every engineer's laptop — a security risk that grows with team size. Third, there is no audit trail tying a specific Git commit to a specific infrastructure change.

A CI/CD pipeline solves all three simultaneously — the plan is visible to reviewers, credentials live only in the pipeline, and every apply is tied to a specific commit and pull request.

The Analogy

Running Terraform locally in production is like a surgeon operating without a checklist — skilled people can do it, but it relies entirely on individual discipline with no systemic safety net. A CI/CD pipeline is the surgical checklist: every step happens in the same order every time, a second person reviews before the critical step, and there is a written record of exactly what happened and when.

OIDC Keyless Authentication

The traditional approach stores AWS access keys as GitHub secrets. The better approach uses OIDC — the pipeline proves its identity to AWS by presenting a short-lived token issued by GitHub, and AWS issues temporary credentials in return. No long-lived secrets to store, rotate, or accidentally expose.

New terms:

OIDC (OpenID Connect) — an identity protocol that lets GitHub Actions prove to AWS who it is without storing a password. GitHub issues a signed JWT token identifying the workflow, repository, and branch. AWS validates the token signature against GitHub's public keys and issues temporary STS credentials.
IAM Identity Provider — an AWS resource that trusts tokens from an external OIDC provider. Must be created in the AWS account before OIDC authentication can work. Created once per account, not per repository.
Condition in the trust policy — restricts which GitHub repositories and branches can assume the role. Without conditions, any GitHub Actions workflow anywhere could assume your IAM role. The condition is the access control gate.

# Step 1: Create the OIDC Identity Provider in AWS (done once per account)
resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  client_id_list  = ["sts.amazonaws.com"]

  # GitHub's OIDC thumbprint — rotate when GitHub updates their certificate
  thumbprint_list = ["6938fd4d98bab03faadb97b34396831e3780aea1"]
}

# Step 2: Create the IAM role that the pipeline will assume
resource "aws_iam_role" "github_actions_terraform" {
  name = "github-actions-terraform-prod"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Federated = aws_iam_openid_connect_provider.github.arn }
      Action    = "sts:AssumeRoleWithWebIdentity"

      Condition = {
        StringEquals = {
          # Only this specific repository can assume this role
          "token.actions.githubusercontent.com:aud" = "sts.amazonaws.com"
        }
        StringLike = {
          # Only the main branch — prevent feature branches from applying to prod
          "token.actions.githubusercontent.com:sub" = [
            "repo:acme-corp/infrastructure:ref:refs/heads/main",
            "repo:acme-corp/infrastructure:environment:production"
          ]
        }
      }
    }]
  })
}

# Step 3: Attach only the permissions Terraform needs
resource "aws_iam_role_policy_attachment" "terraform_permissions" {
  role       = aws_iam_role.github_actions_terraform.name
  policy_arn = aws_iam_policy.terraform_execution.arn  # Scoped policy from Lesson 29
}

# The GitHub Actions workflow then uses:
# - uses: aws-actions/configure-aws-credentials@v4
#   with:
#     role-to-assume: arn:aws:iam::ACCOUNT:role/github-actions-terraform-prod
#     aws-region: us-east-1
# No secrets stored anywhere — entirely keyless

Full GitHub Actions Pipeline — Plan on PR

The plan-on-PR workflow runs on every pull request targeting main. It runs security scanning, initialises Terraform, generates a plan, and posts the plan output as a PR comment — giving reviewers the exact infrastructure changes alongside the code changes.

# .github/workflows/terraform-plan.yml
name: Terraform Plan

on:
  pull_request:
    branches: [main]
    paths: ['infrastructure/**']  # Only trigger when infrastructure files change

permissions:
  id-token: write    # Required for OIDC — request the JWT token
  contents: read     # Read repository contents
  pull-requests: write  # Post plan output as PR comment

jobs:
  plan:
    name: Terraform Plan
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: infrastructure/services/payments  # Which root module

    steps:
      # 1. Checkout the repository
      - name: Checkout
        uses: actions/checkout@v4

      # 2. Authenticate to AWS via OIDC — no secrets needed
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/github-actions-terraform-prod
          aws-region: us-east-1

      # 3. Security scan BEFORE init — catch misconfigs without any AWS calls
      - name: Run tfsec
        uses: aquasecurity/tfsec-action@v1.0.0
        with:
          working_directory: infrastructure/services/payments
          minimum_severity: HIGH
          soft_fail: false  # Hard fail on HIGH or CRITICAL findings

      # 4. Setup Terraform — pin the version to ensure consistency
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: "1.6.3"  # Always pin — never use latest

      # 5. Init — download providers and modules
      - name: Terraform Init
        id: init
        run: terraform init

      # 6. Validate — syntax and reference checking without API calls
      - name: Terraform Validate
        id: validate
        run: terraform validate -no-color

      # 7. Plan — generate and save the plan file
      - name: Terraform Plan
        id: plan
        run: terraform plan -no-color -out=tfplan
        continue-on-error: true  # Continue to post comment even if plan fails

      # 8. Post plan output as a PR comment
      - name: Post Plan to PR
        uses: actions/github-script@v7
        with:
          script: |
            const output = `#### Terraform Plan — \`${{ steps.plan.outcome }}\`

            Show Plan

            \`\`\`terraform
            ${{ steps.plan.outputs.stdout }}
            \`\`\`

            

            *Pushed by: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })

      # 9. Fail the job if plan failed — after posting the comment
      - name: Plan Status
        if: steps.plan.outcome == 'failure'
        run: exit 1

      # 10. Upload the plan file as an artifact for the apply job
      - name: Upload Plan
        uses: actions/upload-artifact@v4
        with:
          name: tfplan
          path: infrastructure/services/payments/tfplan
          retention-days: 1  # Plan files expire — cannot apply a stale plan

Apply on Merge with Approval Gate

The apply workflow triggers when a PR is merged to main. It downloads the saved plan file from the plan job and applies exactly that plan — ensuring the approved plan is what gets deployed, not a freshly generated plan that might have changed.

# .github/workflows/terraform-apply.yml
name: Terraform Apply

on:
  push:
    branches: [main]                          # Trigger on merge to main
    paths: ['infrastructure/**']

permissions:
  id-token: write
  contents: read

jobs:
  apply:
    name: Terraform Apply
    runs-on: ubuntu-latest
    environment: production  # GitHub Environment — requires manual approval from designated reviewers

    defaults:
      run:
        working-directory: infrastructure/services/payments

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/github-actions-terraform-prod
          aws-region: us-east-1

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: "1.6.3"

      - name: Terraform Init
        run: terraform init

      # Download the exact plan that was reviewed on the PR
      - name: Download Plan
        uses: actions/download-artifact@v4
        with:
          name: tfplan
          path: infrastructure/services/payments/

      # Apply the saved plan file — not a fresh plan
      # This guarantees exactly what was reviewed gets deployed
      - name: Terraform Apply
        run: terraform apply -auto-approve tfplan
        # -auto-approve skips the interactive yes/no — safe because:
        # 1. The plan was already reviewed and approved via PR
        # 2. The environment requires manual approval before this job runs
        # 3. The plan file is immutable — same diff that was reviewed

# PR opened → Plan job triggers automatically
✓ tfsec: 0 HIGH findings
✓ terraform init: Initializing modules... Installing hashicorp/aws v5.31.0...
✓ terraform validate: Success! The configuration is valid.
✓ terraform plan: Plan: 3 to add, 1 to change, 0 to destroy.
✓ PR comment posted with full plan diff
✓ tfplan artifact uploaded (expires in 1 day)

# PR reviewed and approved → merged to main
# Apply job waits for manual approval from production environment reviewers

# After approval:
✓ Configure AWS credentials via OIDC (no secrets)
✓ terraform init
✓ Downloaded tfplan artifact
✓ terraform apply tfplan

aws_security_group_rule.payments_ingress: Creating...
aws_ecs_task_definition.payments: Creating...
aws_ecs_service.payments: Modifying...
aws_ecs_service.payments: Modifications complete

Apply complete! Resources: 2 added, 1 changed, 0 destroyed.

# Full audit trail:
# PR #142 → commit abc1234 → plan artifact → manual approval → apply
# Every change traceable from code to infrastructure

What just happened?

The saved plan file is the bridge between review and apply. The plan was generated, reviewed, and approved during the PR. The apply job downloads that exact artifact and applies it. Even if a developer pushed a commit between plan and apply, the plan file is immutable — the infrastructure change is exactly what the reviewer approved. A fresh plan on apply would re-evaluate the current configuration, potentially including unreviewed changes.
GitHub Environments add a mandatory human gate. The environment: production block pauses the job until designated reviewers click Approve in GitHub. Without this, every merge to main would immediately apply to production with no human checkpoint. The approval is recorded in GitHub's audit log — who approved, when, for which commit.
OIDC means zero stored credentials. The entire pipeline ran without a single AWS access key stored anywhere. GitHub's OIDC provider issued a token, AWS validated it, and temporary STS credentials were returned for the duration of the job. When the job ends, the credentials expire automatically.

Handling Multiple Environments

Most organisations deploy through dev, staging, and production environments in sequence. The pattern: deploy to dev on every PR merge to a feature branch, deploy to staging on every merge to main, deploy to production after explicit promotion approval.

# Multi-environment pipeline strategy
# Uses GitHub Actions matrix to run plan/apply for each environment

name: Terraform Multi-Environment

on:
  push:
    branches: [main]

jobs:
  deploy:
    strategy:
      matrix:
        environment: [dev, staging]  # Deploy dev and staging automatically
      fail-fast: false   # Don't cancel staging if dev fails
    runs-on: ubuntu-latest
    environment: ${{ matrix.environment }}  # Different GitHub Environment per matrix entry

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          # Different role per environment — different AWS accounts
          role-to-assume: ${{ vars.TERRAFORM_ROLE_ARN }}  # Set per GitHub Environment
          aws-region: us-east-1

      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: "1.6.3"

      - name: Terraform Init and Apply
        working-directory: infrastructure/services/payments
        run: |
          terraform init \
            -backend-config="key=services/payments/${{ matrix.environment }}.tfstate"
          terraform apply -auto-approve \
            -var="environment=${{ matrix.environment }}"

  # Production requires explicit promotion — not automatic
  deploy-prod:
    needs: [deploy]          # Wait for dev and staging to succeed first
    runs-on: ubuntu-latest
    environment: production  # Required: human approval in GitHub UI

    steps:
      - uses: actions/checkout@v4
      - name: Configure AWS credentials (prod account)
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ vars.TERRAFORM_ROLE_ARN }}  # Prod account role
          aws-region: us-east-1
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: "1.6.3"
      - name: Terraform Init and Apply — Production
        working-directory: infrastructure/services/payments
        run: |
          terraform init -backend-config="key=services/payments/prod.tfstate"
          terraform apply -auto-approve -var="environment=prod"

Pipeline Security Hardening

# Security best practices for Terraform CI/CD pipelines

# 1. Pin ALL action versions to a full SHA — not a tag
# Tags are mutable — v4 today could be different code tomorrow
# A malicious update to a dependency action could steal your AWS credentials
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683  # v4.2.2 SHA
- uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502

# 2. Scope OIDC role to specific branches only
# The trust policy condition prevents feature branches from assuming prod role:
# "token.actions.githubusercontent.com:sub":
#   "repo:acme/infra:ref:refs/heads/main"  # Only main branch

# 3. Never echo secrets or plan output that contains sensitive values
# Bad:
- run: echo "${{ secrets.TF_VAR_DB_PASSWORD }}"  # Secret in logs

# Good: Pass as environment variables — GitHub masks them in logs automatically
- name: Terraform Apply
  env:
    TF_VAR_db_password: ${{ secrets.TF_VAR_DB_PASSWORD }}
  run: terraform apply -auto-approve tfplan

# 4. Store state in a separate AWS account the pipeline cannot delete
# The pipeline role has s3:GetObject and s3:PutObject on the state bucket
# But NOT s3:DeleteObject or s3:DeleteBucket — cannot destroy the audit trail

# 5. Require plan artifact to be from the same SHA as the apply
# If the PR commit SHA doesn't match the artifact SHA, reject the apply
- name: Verify plan SHA matches current commit
  run: |
    PLAN_SHA=$(cat tfplan.sha256 2>/dev/null || echo "")
    if [ "$PLAN_SHA" != "${{ github.sha }}" ]; then
      echo "Plan was generated for a different commit — refusing to apply"
      exit 1
    fi

# 6. Set minimum permissions at the job level
permissions:
  id-token: write   # Only what this specific job needs
  contents: read
  # Do NOT set permissions: write-all — gives every action full access

Common CI/CD Pipeline Mistakes

Generating a fresh plan at apply time instead of using a saved plan

If the apply job runs terraform apply -auto-approve without a saved plan file, it generates a new plan at apply time. Between the reviewed plan and the apply, a developer may have pushed a commit, a resource may have drifted, or another pipeline may have changed shared state. What gets applied may differ from what was reviewed. Always save the plan with -out=tfplan during planning and apply exactly that file.

Using mutable action tags instead of pinned SHAs

A GitHub Actions workflow that uses uses: some-action@v4 is referencing a mutable tag. The repository owner can push a new commit to the v4 tag at any time — changing what code runs in your pipeline silently. For pipelines that assume AWS IAM roles, a compromised action could exfiltrate credentials. Always pin to a full commit SHA: uses: some-action@FULL_SHA.

Not pinning the Terraform version in the pipeline

Using terraform_version: latest or omitting the version in hashicorp/setup-terraform means the pipeline installs whatever is current at run time. A Terraform patch release with a bug, or a minor release with changed behaviour, breaks the pipeline without any code change. Always pin to an exact version — terraform_version: "1.6.3". Update it deliberately as part of your upgrade strategy (Lesson 41).

The complete CI/CD Terraform pipeline — in sequence

On PR open/update: tfsec scan → terraform init → terraform validate → terraform plan -out=tfplan → post plan as PR comment → upload tfplan artifact. On PR merge to main: download tfplan artifact → (wait for environment approval) → terraform init → terraform apply tfplan. This sequence ensures that security is checked before any plan, the plan is immutable from review to apply, and no credentials are stored anywhere outside the pipeline runtime.

Practice Questions

1. Which command saves the Terraform plan to a file so it can be applied exactly as reviewed?

2. Which GitHub Actions permission must be set to write for OIDC authentication to work?

3. Which GitHub Actions configuration adds a mandatory human approval gate before the apply job runs?

Quiz

Up Next · Lesson 38

Terraform with Jenkins

GitHub Actions pipeline built. Lesson 38 covers the same patterns in Jenkins — Jenkinsfile pipeline anatomy, credential binding for AWS authentication, shared libraries for reusable pipeline stages, and the differences between Jenkins declarative and scripted pipeline approaches for Terraform.

← Previous Course Index Next →

Terraform Course