Terraform Lesson 15 – Local vs Remote State | Dataplexa
Section II · Lesson 15

Local vs Remote State

Local state is fine for learning. The moment a second person touches the same Terraform project, local state becomes a liability. This lesson covers the full spectrum — what local state is, where it breaks down, every major remote backend option, and how to set up the S3 backend that every real AWS team uses in production.

This lesson covers

Local state and exactly where it breaks → Remote backend options compared → Building the S3 + DynamoDB backend from scratch → Migrating existing local state to remote → Workspace-aware state paths

Local State — What It Is and Where It Breaks

When you run terraform init with no backend configuration, Terraform stores state in a file called terraform.tfstate in the working directory. That file lives on your local disk. Only you can see it. Only you can use it.

For a solo project that only you will ever run, this is fine. For anything shared, it creates four distinct problems that compound over time.

Problem What happens Consequence
No sharing State lives on one machine Teammates cannot apply without copying state manually
No locking Two applies run at the same time State file corrupted — resources orphaned
No backup Laptop lost, disk failed, file deleted State gone — Terraform orphans all managed resources
No access control State contains secrets and resource IDs Anyone who can read the file sees all sensitive data

The Analogy

Using local state for a shared project is like keeping the company's only set of keys in your jacket pocket. When you are in the office, everything works. The moment you are on holiday, sick, or have a different jacket on — nobody can get in. Remote state is the key cabinet on the wall that everyone with the right badge can access, with a log of every time someone used it.

Remote Backend Options

Terraform supports multiple remote backends. The right choice depends on your cloud provider, team size, and whether you are using Terraform Community Edition or HCP Terraform.

Backend Locking Best for
S3 + DynamoDB Yes — DynamoDB AWS-based teams — most widely used in production
Azure Blob Storage Yes — blob leases Azure-based teams
GCS Yes — object locks GCP-based teams
HCP Terraform Yes — built-in Teams wanting managed state + run history + RBAC
HTTP Optional Custom state servers, GitLab-managed Terraform

Building the S3 Backend from Scratch

The S3 + DynamoDB backend is the standard for AWS teams. Before any project can use it, the S3 bucket and DynamoDB table must exist. You create them with Terraform — but in a separate bootstrap configuration that uses local state. The bootstrap configuration is the one project in your organisation that legitimately uses local state.

Create the bootstrap project first:

mkdir terraform-state-bootstrap
cd terraform-state-bootstrap
touch versions.tf main.tf outputs.tf

Add this to versions.tf:

terraform {
  required_version = ">= 1.5.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  # No backend block — this bootstrap config intentionally uses local state
  # It creates the S3 bucket and DynamoDB table that all other projects use
}

provider "aws" {
  region = "us-east-1"
}

Add this to main.tf:

New terms:

  • aws_s3_bucket_versioning — enables versioning on the state bucket. Every write to the state file creates a new version rather than overwriting the previous one. This means you can recover any previous state version if something goes wrong — essential for a state bucket.
  • aws_s3_bucket_server_side_encryption_configuration — configures server-side encryption for all objects stored in the bucket. AES256 uses Amazon S3-managed keys — no cost, no key management overhead. aws:kms uses AWS KMS for customer-managed keys — more control at additional cost.
  • aws_s3_bucket_public_access_block — explicitly blocks all public access to the bucket. A state bucket must never be publicly accessible — it contains sensitive resource data. This resource sets all four public access block settings to true, overriding any future misconfiguration.
  • aws_dynamodb_table — creates a DynamoDB table used for state locking. The table needs only one attribute — LockID — which is the partition key Terraform uses when writing lock entries. billing_mode = "PAY_PER_REQUEST" means you only pay when locks are acquired — near zero cost for most teams.
  • prevent_destroy = true — on a state bucket, this is non-negotiable. Losing the state bucket while projects are using it means every Terraform project that depends on it loses state management. Add this lifecycle rule to both the bucket and the DynamoDB table.
# S3 bucket to store all Terraform state files for this organisation
resource "aws_s3_bucket" "terraform_state" {
  bucket = "acme-terraform-state-${data.aws_caller_identity.current.account_id}"  # Account ID makes name unique

  tags = {
    Name      = "Terraform State Bucket"
    ManagedBy = "Terraform"
    Purpose   = "terraform-state"
  }

  lifecycle {
    prevent_destroy = true  # Never allow this bucket to be destroyed — all state lives here
  }
}

# Enable versioning — every state file write creates a new version
# This is the recovery mechanism if state becomes corrupted
resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  versioning_configuration {
    status = "Enabled"  # Enabled = versioning on. Suspended = versioning paused but history kept.
  }
}

# Encrypt all objects at rest using AES-256
# State files contain sensitive data — encryption is mandatory
resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"  # S3-managed keys — no additional cost or key management
    }
  }
}

# Block all public access — a state bucket must never be publicly accessible
resource "aws_s3_bucket_public_access_block" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  block_public_acls       = true  # Block any attempt to set public ACLs
  block_public_policy     = true  # Block any attempt to set a public bucket policy
  ignore_public_acls      = true  # Ignore public ACLs already on objects
  restrict_public_buckets = true  # Restrict access to the bucket owner only
}

# DynamoDB table for state locking
# When any Terraform process runs apply, it writes a lock here
# Any concurrent apply sees the lock and waits
resource "aws_dynamodb_table" "terraform_locks" {
  name         = "terraform-state-lock"
  billing_mode = "PAY_PER_REQUEST"  # No provisioned capacity — pay only for actual lock operations

  # LockID is the partition key Terraform uses — this exact name is required
  hash_key = "LockID"

  attribute {
    name = "LockID"
    type = "S"  # S = String — LockID is always a string
  }

  tags = {
    Name      = "Terraform State Lock Table"
    ManagedBy = "Terraform"
    Purpose   = "terraform-state-lock"
  }

  lifecycle {
    prevent_destroy = true  # Destroying this table while projects are applying corrupts state
  }
}

# Data source to get current account ID for the unique bucket name
data "aws_caller_identity" "current" {}

Add this to outputs.tf:

output "state_bucket_name" {
  description = "S3 bucket name — use this in every project's backend configuration"
  value       = aws_s3_bucket.terraform_state.bucket
}

output "state_bucket_arn" {
  description = "S3 bucket ARN — use this in IAM policies granting state access"
  value       = aws_s3_bucket.terraform_state.arn
}

output "dynamodb_table_name" {
  description = "DynamoDB table name — use this in every project's backend configuration"
  value       = aws_dynamodb_table.terraform_locks.name
}

output "backend_config_snippet" {
  description = "Copy this block into any project's versions.tf to use this remote backend"
  value = <<-EOT
    backend "s3" {
      bucket         = "${aws_s3_bucket.terraform_state.bucket}"
      key            = "PROJECT_NAME/ENV/terraform.tfstate"
      region         = "us-east-1"
      encrypt        = true
      dynamodb_table = "${aws_dynamodb_table.terraform_locks.name}"
    }
  EOT
}

Now run the bootstrap:

terraform init
terraform apply
$ terraform apply

Plan: 5 to add, 0 to change, 0 to destroy.

  Enter a value: yes

aws_s3_bucket.terraform_state: Creating...
aws_s3_bucket.terraform_state: Creation complete [id=acme-terraform-state-123456789012]

aws_s3_bucket_versioning.terraform_state: Creating...
aws_s3_bucket_server_side_encryption_configuration.terraform_state: Creating...
aws_s3_bucket_public_access_block.terraform_state: Creating...
aws_dynamodb_table.terraform_locks: Creating...

aws_s3_bucket_versioning.terraform_state: Creation complete
aws_s3_bucket_server_side_encryption_configuration.terraform_state: Creation complete
aws_s3_bucket_public_access_block.terraform_state: Creation complete
aws_dynamodb_table.terraform_locks: Creation complete [id=terraform-state-lock]

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Outputs:

backend_config_snippet = <

What just happened?

  • Five resources created — bucket, versioning, encryption, public access block, DynamoDB table. Versioning, encryption, and public access blocking are all separate resource types in the AWS provider — not arguments on aws_s3_bucket itself. This is intentional AWS provider design: each concern is managed independently, so teams can adopt them incrementally without rebuilding the bucket.
  • The account ID in the bucket name guarantees global uniqueness. S3 bucket names are globally unique across all AWS accounts worldwide. Using the account ID — 123456789012 — as a suffix ensures the name will never conflict with another organisation's state bucket. No random suffix needed here — the account ID is stable and meaningful.
  • backend_config_snippet output prints a ready-to-use backend block. The <<-EOT heredoc syntax in HCL outputs a multi-line string. The output contains the exact backend block to paste into any project's versions.tf — with the bucket name and DynamoDB table name already filled in. Replace PROJECT_NAME/ENV with the appropriate path for each project.

Migrating Local State to Remote

You have an existing project with local state. The backend infrastructure now exists. Migrating is a three-step process: add the backend block to versions.tf, run terraform init, confirm the migration. Terraform handles the file copy automatically.

# Step 1 — add the backend block to versions.tf in your existing project
# Replace PROJECT_NAME and ENV with the appropriate values for this project

terraform {
  required_version = ">= 1.5.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  # Add this block — run terraform init after saving to trigger migration
  backend "s3" {
    bucket         = "acme-terraform-state-123456789012"  # From bootstrap output
    key            = "lesson-15/dev/terraform.tfstate"     # Unique path for this project
    region         = "us-east-1"
    encrypt        = true                                  # Always true
    dynamodb_table = "terraform-state-lock"               # From bootstrap output
  }
}

provider "aws" {
  region = "us-east-1"
}
# Step 2 — run terraform init to trigger the migration
terraform init
$ terraform init

Initializing the backend...

Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "s3" backend. No existing state was found in the newly configured
  "s3" backend. Do you want to copy this state to the new backend?

  Enter a value: yes

Successfully configured the backend "s3"!

Terraform will automatically use this backend unless the backend configuration
changes.

$ terraform plan

No changes. Your infrastructure matches the configuration.

# The local terraform.tfstate file is now empty — state lives in S3
$ cat terraform.tfstate
{
  "version": 4,
  "terraform_version": "1.6.0",
  "serial": 1,
  "lineage": "abc-def-123",
  "outputs": {},
  "resources": [],
  "check_results": null
}

What just happened?

  • terraform init detected the new backend and offered to migrate state. Terraform found existing local state and asked whether to copy it to S3. After confirming yes, it uploaded the state file to s3://acme-terraform-state-123456789012/lesson-15/dev/terraform.tfstate. The local terraform.tfstate file now contains an empty state — all the real state is in S3.
  • The local file still exists but is now empty. Terraform does not delete the local file — it empties it. This is intentional: the empty local file signals that state has been migrated. If something goes wrong and you need to roll back to local state, the original data is still in the pre-migration backup at terraform.tfstate.backup.
  • A follow-up plan shows no changes. The migrated state is identical to the local state — same serial, same lineage, same resource records. Terraform reads the remote state, queries AWS to confirm resources still exist, and confirms everything matches. Migration was transparent.

State Locking in Practice

With the S3 backend and DynamoDB locking active, every terraform apply acquires a lock before modifying state. Here is what that looks like — including what happens when a lock already exists.

# Normal apply — lock is acquired, apply runs, lock is released automatically
terraform apply

# What happens if two engineers run apply at the same time
# Engineer A runs first — acquires the lock
# Engineer B runs simultaneously — sees the lock and gets this error:
$ terraform apply   # Engineer A — succeeds and acquires lock

Acquiring state lock. This may take a few moments...

aws_s3_bucket.example: Creating...
aws_s3_bucket.example: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Releasing state lock. This may take a few moments...

$ terraform apply   # Engineer B — runs while A's lock is active

Acquiring state lock. This may take a few moments...

╷
│ Error: Error acquiring the state lock
│
│ Error message: ConditionalCheckFailedException: The conditional request failed
│
│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
│
│ Lock Info:
│   ID:        abc-def-123-lock
│   Path:      acme-terraform-state-123456789012/lesson-15/dev/terraform.tfstate
│   Operation: OperationTypeApply
│   Who:       engineer-a@hostname
│   Version:   1.6.0
│   Created:   2024-01-15 14:22:33.456 +0000 UTC
│   Info:
╵

# If Engineer A's process crashes and the lock is not released:
# Inspect the lock — see who owns it and when it was created
terraform force-unlock abc-def-123-lock

What just happened?

  • Engineer B's apply failed immediately with a clear lock error. The error message shows exactly who holds the lock, from which machine, which operation type, and when it was created. This information is written to the DynamoDB table when Engineer A's apply started. Engineer B does not need to guess whether someone else is running — the error tells them everything.
  • The lock is released automatically when the apply finishes. Whether the apply succeeds or fails, Terraform releases the DynamoDB lock entry at the end. The "Releasing state lock" message appears at the end of every apply. If you do not see it — if the process was killed — the lock persists until manually released.
  • terraform force-unlock clears a stuck lock. Only use this when you are certain the process that acquired the lock is no longer running. Force-unlocking a lock that is actively held by another process allows two concurrent applies — which can corrupt state. Verify the lock's Created timestamp and confirm the holding process is dead before force-unlocking.

Key Naming Convention for State Files

When multiple projects share one S3 bucket, the key argument is what separates them. A consistent naming convention prevents collisions and makes the bucket navigable.

# Recommended key naming convention: team/project/environment/terraform.tfstate
# Examples for a multi-team organisation:

# Platform team — networking infrastructure
key = "platform/networking/prod/terraform.tfstate"

# Platform team — shared services
key = "platform/shared-services/prod/terraform.tfstate"

# Application team — payments microservice
key = "payments/application/prod/terraform.tfstate"

# Application team — payments database
key = "payments/database/prod/terraform.tfstate"

# DevOps team — CI/CD infrastructure
key = "devops/cicd/prod/terraform.tfstate"

# Each team's state is isolated — no team can accidentally overwrite another's state
# All states benefit from the same bucket's versioning and encryption

What just happened?

  • One bucket serves the entire organisation — isolation comes from the key path. Every team and project gets a unique key path. The S3 bucket's versioning, encryption, and access controls apply to all state files equally. One bucket to manage, one set of access policies, one cost centre — with complete isolation between teams via key paths.
  • The environment is always part of the key path. prod/terraform.tfstate and dev/terraform.tfstate are completely separate state files. Applying with the wrong environment variable is a common mistake — making the environment explicit in the key path means the wrong state file path would be used if someone forgets to set the right environment, catching the mistake immediately.

Common Mistakes

Using the same key for two different projects

If two projects use backend "s3" { key = "terraform.tfstate" }, the second project to apply overwrites the first project's state. Resources from project one are now orphaned — Terraform no longer knows they exist. State keys must be unique per project per environment — always.

Destroying the state bucket while projects are using it

If the S3 bucket is destroyed — even accidentally — every project using it loses remote state. The next terraform init on any of those projects fails with a bucket-not-found error. That is why prevent_destroy = true is mandatory on the state bucket and DynamoDB table. To legitimately destroy them, you must first migrate all projects to a different backend.

Using -lock=false in production

Terraform allows you to skip locking with -lock=false. This flag exists for specific automation scenarios — never for regular use. Running an apply with locking disabled in a shared environment means a concurrent apply can corrupt state. The only safe uses of -lock=false are in single-operator environments where no concurrent access is possible.

IAM permissions for the state bucket

Every IAM identity that runs Terraform needs specific permissions on both the S3 bucket and the DynamoDB table. For S3: s3:GetObject, s3:PutObject, s3:DeleteObject, and s3:ListBucket on the bucket and its contents. For DynamoDB: dynamodb:GetItem, dynamodb:PutItem, and dynamodb:DeleteItem on the lock table. Granting s3:* on the entire bucket is convenient but overpermissive — use the minimum required permissions, especially in production.

Practice Questions

1. You add a backend block to an existing project that was using local state. Which command triggers the migration of state to the remote backend?



2. Which DynamoDB billing mode should you use for a state lock table to minimise cost when locks are infrequent?



3. A Terraform apply process crashed and left a DynamoDB lock entry behind. Which command removes the stale lock?



Quiz

1. What is the most important recovery mechanism to enable on an S3 state bucket?


2. Why does the bootstrap configuration that creates the S3 bucket and DynamoDB table use local state?


3. Your organisation has 20 Terraform projects all using the same S3 bucket for state. How is each project's state kept separate?


Up Next · Lesson 16

Backend Configuration

You have the backend running. Lesson 16 goes deeper — partial backend configuration for CI/CD, switching between backends, backend-specific arguments, and the pattern that lets one codebase deploy to multiple environments without duplicating backend blocks.