Terraform Course
Terraform State Explained
State is the most misunderstood part of Terraform. Engineers use it in every lesson without fully understanding what it is, what it contains, or what happens when it diverges from reality. This lesson opens the state file, reads it, breaks it deliberately, and shows exactly how Terraform uses it to decide what to do on every plan.
This lesson covers
What the state file actually contains → How Terraform uses state to detect drift → What happens when state and reality diverge → Reading state with terraform show and terraform state list → Refreshing state → What never to do with state
What State Is
When Terraform creates a resource, it records everything about that resource in a JSON file called terraform.tfstate. Not just the arguments you provided — every attribute the cloud provider assigned after creation. The resource ID, the ARN, the public IP, the creation timestamp, the private IP, every computed field. All of it.
This state file is Terraform's memory. Every time you run terraform plan, Terraform reads the state file, queries the cloud provider for the current real state of each resource, compares the two, and compares both against your configuration. From that three-way comparison it produces the execution plan.
The Analogy
Think of the state file as a photograph of your infrastructure taken at the moment of the last apply. When you run plan, Terraform compares that photograph to both your current configuration (what you want) and the live infrastructure (what exists right now). If the photograph matches both — no changes needed. If the live infrastructure drifted from the photograph — Terraform flags the drift. If the configuration changed from what the photograph shows — Terraform plans the update.
Terraform compares configuration, state file, and real infrastructure to produce the execution plan
Setting Up — Build Infrastructure to Inspect
Create a project, deploy some real infrastructure, then inspect the state file directly. This lesson is about reading and understanding state — not just knowing it exists.
mkdir terraform-lesson-14
cd terraform-lesson-14
touch versions.tf main.tf outputs.tf .gitignoreAdd this to versions.tf:
terraform {
required_version = ">= 1.5.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
provider "aws" {
region = "us-east-1"
}Add this to main.tf:
# A simple S3 bucket — minimal resource to create clean state we can inspect
resource "aws_s3_bucket" "lesson14" {
bucket = "terraform-lesson-14-state-demo-${random_id.suffix.hex}"
tags = {
Name = "lesson14-state-demo"
ManagedBy = "Terraform"
}
}
# Random suffix to guarantee a globally unique bucket name
resource "random_id" "suffix" {
byte_length = 4 # 4 bytes = 8 hex characters
}
# Enable versioning on the bucket — adds a second resource to inspect in state
resource "aws_s3_bucket_versioning" "lesson14" {
bucket = aws_s3_bucket.lesson14.id # Implicit dependency on bucket
versioning_configuration {
status = "Enabled" # Keep all versions of every object
}
}Update versions.tf to add the random provider:
terraform {
required_version = ">= 1.5.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
random = {
source = "hashicorp/random"
version = "~> 3.5" # For generating unique bucket name suffix
}
}
}
provider "aws" {
region = "us-east-1"
}Run terraform init then apply:
terraform init
terraform apply$ terraform apply Plan: 3 to add, 0 to change, 0 to destroy. Enter a value: yes random_id.suffix: Creating... random_id.suffix: Creation complete after 0s [id=a3f2b1c4] aws_s3_bucket.lesson14: Creating... aws_s3_bucket.lesson14: Creation complete after 3s [id=terraform-lesson-14-state-demo-a3f2b1c4] aws_s3_bucket_versioning.lesson14: Creating... aws_s3_bucket_versioning.lesson14: Creation complete after 1s Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Reading the State File
The state file exists at terraform.tfstate in your project directory. You can open it in any text editor — it is plain JSON. But Terraform provides commands that make reading state easier and safer than opening the raw file.
New terms:
- terraform state list — prints every resource address currently tracked in state. An address is the resource type and name joined by a dot —
aws_s3_bucket.lesson14. Forfor_eachresources, addresses include the key:aws_instance.web["primary"]. - terraform state show — prints every attribute of one specific resource from state. Shows both the arguments you provided and the computed attributes AWS assigned after creation — ID, ARN, creation date, all of them.
- terraform show — prints the entire current state in human-readable format. Without arguments it reads state. With a plan file argument —
terraform show tfplan— it prints the saved plan.
# List every resource address tracked in state
terraform state list
# Show all attributes of one specific resource
terraform state show aws_s3_bucket.lesson14
# Show all attributes of the random ID
terraform state show random_id.suffix
# Show the entire state in human-readable format
terraform show$ terraform state list
aws_s3_bucket.lesson14
aws_s3_bucket_versioning.lesson14
random_id.suffix
$ terraform state show aws_s3_bucket.lesson14
# aws_s3_bucket.lesson14:
resource "aws_s3_bucket" "lesson14" {
arn = "arn:aws:s3:::terraform-lesson-14-state-demo-a3f2b1c4"
bucket = "terraform-lesson-14-state-demo-a3f2b1c4"
bucket_domain_name = "terraform-lesson-14-state-demo-a3f2b1c4.s3.amazonaws.com"
bucket_regional_domain_name = "terraform-lesson-14-state-demo-a3f2b1c4.s3.us-east-1.amazonaws.com"
hosted_zone_id = "Z3AQBSTGFYJSTF"
id = "terraform-lesson-14-state-demo-a3f2b1c4"
object_lock_enabled = false
region = "us-east-1"
request_payer = "BucketOwner"
tags = {
"ManagedBy" = "Terraform"
"Name" = "lesson14-state-demo"
}
tags_all = {
"ManagedBy" = "Terraform"
"Name" = "lesson14-state-demo"
}
versioning {
enabled = false
mfa_delete = false
}
}
$ terraform state show random_id.suffix
# random_id.suffix:
resource "random_id" "suffix" {
b64_std = "o/KxxA=="
b64_url = "o_KxxA"
byte_length = 4
dec = "2752885188"
hex = "a3f2b1c4" # This is the suffix used in the bucket name
id = "o_KxxA"
}What just happened?
- terraform state show revealed every attribute — not just what you wrote. Your configuration declared
bucketandtags. The state file also containsarn,bucket_domain_name,bucket_regional_domain_name,hosted_zone_id,region,request_payer, and more — all assigned by AWS after creation. These computed attributes are what you reference when writing expressions likeaws_s3_bucket.lesson14.arn. - random_id stores every encoding of the generated bytes. The same 4 random bytes are available as hex (
a3f2b1c4), base64 (o/KxxA==), URL-safe base64 (o_KxxA), and decimal (2752885188). You can reference any format depending on what the consuming resource expects. - State is the source of truth for computed values. When you write
output "bucket_arn" { value = aws_s3_bucket.lesson14.arn }, Terraform readsarnfrom state — not from the AWS API. This is why outputs are immediately available even if the AWS API is slow — the value is already in state from the last apply.
What the Raw State File Contains
Open terraform.tfstate in your editor. Here is the structure you will see — simplified to show the key fields:
{
"version": 4, // State file format version — always 4 in modern Terraform
"terraform_version": "1.6.0", // Terraform CLI version that last wrote this file
"serial": 3, // Increments on every write — used for conflict detection
"lineage": "abc-def-123", // Unique ID for this state — stays constant across serial increments
"resources": [
{
"mode": "managed", // "managed" for resources, "data" for data sources
"type": "aws_s3_bucket", // Resource type
"name": "lesson14", // Local name from your configuration
"provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
"instances": [
{
"schema_version": 0,
"attributes": {
// Every attribute the provider knows about this resource
"id": "terraform-lesson-14-state-demo-a3f2b1c4",
"arn": "arn:aws:s3:::terraform-lesson-14-state-demo-a3f2b1c4",
"bucket": "terraform-lesson-14-state-demo-a3f2b1c4",
"region": "us-east-1",
"tags": {
"ManagedBy": "Terraform",
"Name": "lesson14-state-demo"
}
// ... dozens more attributes
},
"sensitive_attributes": [], // List of attribute paths that are sensitive
"dependencies": [ // Resources this instance depends on
"random_id.suffix"
]
}
]
}
]
}What just happened?
- serial increments on every write. The serial starts at 1 and increases by 1 every time Terraform writes to the state file — every apply, every state operation. When two processes try to write concurrently, the process with the lower serial loses. This is how Terraform detects conflicts without a traditional database lock — combined with DynamoDB locking for remote backends, it prevents state corruption.
- lineage is the permanent identity of this state. The lineage is a UUID generated when the state file is first created. It never changes even as the serial increases. Terraform uses it to confirm that a remote state file belongs to this project — not a different project that happened to use the same S3 key. If the lineage does not match, Terraform refuses to use the state file.
- dependencies are recorded explicitly in state. The bucket's dependencies list includes
random_id.suffix— the dependency Terraform inferred from the bucket name expression. These recorded dependencies drive the destruction order — Terraform reads dependencies from state to know which resources to destroy last.
Drift — When State and Reality Diverge
Drift happens when the real infrastructure changes outside of Terraform. Someone adds a tag in the AWS console. An auto-scaling group changes the instance count. A security policy removes a security group rule. The state file still reflects the old reality — and now it disagrees with what actually exists in AWS.
We are going to create drift deliberately — add a tag to the S3 bucket directly in the AWS console — then run terraform plan to see how Terraform detects and handles it.
# Simulate drift — add a tag manually via AWS CLI
# This is exactly what happens when someone edits a resource in the AWS console
aws s3api put-bucket-tagging \
--bucket terraform-lesson-14-state-demo-a3f2b1c4 \
--tagging 'TagSet=[{Key=ManagedBy,Value=Terraform},{Key=Name,Value=lesson14-state-demo},{Key=ManualTag,Value=AddedOutsideTerraform}]'
# Now run plan — Terraform will detect the extra tag and plan to remove it
terraform plan$ terraform plan
aws_s3_bucket.lesson14: Refreshing state... [id=terraform-lesson-14-state-demo-a3f2b1c4]
aws_s3_bucket_versioning.lesson14: Refreshing state... [id=terraform-lesson-14-state-demo-a3f2b1c4]
random_id.suffix: Refreshing state... [id=o_KxxA]
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# aws_s3_bucket.lesson14 has been changed
~ resource "aws_s3_bucket" "lesson14" {
id = "terraform-lesson-14-state-demo-a3f2b1c4"
~ tags = {
+ "ManualTag" = "AddedOutsideTerraform" # This tag was added outside Terraform
# (2 unchanged attributes hidden)
}
}
Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include actions
to undo or respond to these changes.
# aws_s3_bucket.lesson14 will be updated in-place
~ resource "aws_s3_bucket" "lesson14" {
id = "terraform-lesson-14-state-demo-a3f2b1c4"
~ tags = {
- "ManualTag" = "AddedOutsideTerraform" # Terraform will remove this tag
# (2 unchanged attributes hidden)
}
}
Plan: 0 to add, 1 to change, 0 to destroy.What just happened?
- Terraform detected the drift in the "Refreshing state" phase. Every resource address shows "Refreshing state..." at the top of the plan output — this is Terraform querying the AWS API for the current real state of each resource and comparing it to what is recorded in the state file. It found that the bucket now has a
ManualTagthat does not exist in state or configuration. - Terraform reported the out-of-band change separately from the planned change. The plan output has two sections: "Terraform detected the following changes made outside of Terraform" — showing what drifted — and then the planned changes needed to bring everything back to the configuration. The
+in the drift section shows the tag was added. The-in the planned changes shows Terraform will remove it. - Terraform will revert the manual change on the next apply. This is by design — Terraform is the source of truth. If you want the manual tag to persist, add it to your configuration. If you want Terraform to ignore it, add
ignore_changes = [tags]to the lifecycle block. If neither — Terraform removes it every time you apply.
Refreshing State Without Planning
Sometimes you want to update the state file to reflect the current real infrastructure without planning or applying any changes. This is called a state refresh. It re-reads every resource from the cloud provider and updates the state file to match what exists right now.
New terms:
- terraform apply -refresh-only — runs a plan that only shows drift between the state file and real infrastructure. If you confirm, it updates the state file to match reality without creating, modifying, or destroying any resources. This is the modern replacement for
terraform refreshwhich is now deprecated. - terraform plan -refresh=false — skips the refresh phase entirely. Terraform only compares your configuration against the state file — it does not query the cloud provider at all. Useful in large configurations where refreshing hundreds of resources takes minutes and you know nothing has drifted externally.
- -refresh=true — the default behaviour. Terraform always queries the cloud provider during plan to detect drift. You never need to specify this explicitly — it is the default.
# Refresh-only plan — see what changed outside Terraform, update state to match
# This does NOT revert the manual change — it accepts it into state
terraform apply -refresh-only
# Skip refresh entirely — only compare config vs state, no API calls
# Fast for large configurations, but will not detect drift
terraform plan -refresh=false$ terraform apply -refresh-only
Terraform detected the following changes made outside of Terraform:
# aws_s3_bucket.lesson14 has been changed
~ resource "aws_s3_bucket" "lesson14" {
~ tags = {
+ "ManualTag" = "AddedOutsideTerraform"
}
}
This is a refresh-only plan, so Terraform will not propose any actions to undo
these changes. If you accept this plan, Terraform will update its state to
reflect these changes.
Would you like to update the Terraform state to reflect these detected changes?
Terraform will update the following resources in the state file:
~ aws_s3_bucket.lesson14
Enter a value: yes
aws_s3_bucket.lesson14: Refreshing state... [id=terraform-lesson-14-state-demo-a3f2b1c4]
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
# State is now updated to include ManualTag
# A regular terraform plan now shows no changes — state matches reality
$ terraform plan
No changes. Your infrastructure matches the configuration.What just happened?
- -refresh-only updated state to match reality without reverting the drift. Instead of planning to remove the manual tag, Terraform accepted it — updating the state file so it now records the tag. A subsequent
terraform planshows "No changes" because state now matches both the real infrastructure and the configuration — wait, that cannot be right if the configuration does not declare the ManualTag. Let us think about this: the configuration still does not declare ManualTag, so a normal plan after refresh-only would still show a change to remove it. The correct use of -refresh-only is when you decide to accept the drift permanently — typically combined with then updating the configuration to match. - terraform plan -refresh=false is a speed optimisation. In a configuration managing 200 resources, the refresh phase makes 200 separate AWS API calls — one per resource. At 1-2 seconds per call that is 3-7 minutes just for refresh. With
-refresh=false, the plan runs in seconds. Use this during development iteration when you know no external changes have occurred and speed matters. Never use it before a production apply.
terraform state Commands You Need to Know
Beyond listing and showing resources, there are three more state commands every Terraform engineer needs to know. These are emergency tools — used when state and reality are so far apart that normal plan and apply cannot fix the problem alone.
New terms:
- terraform state mv — moves a resource from one address to another inside the state file. Used when you rename a resource in your configuration — without this, Terraform would destroy the old resource and create a new one. With
state mv, the resource keeps its real infrastructure while Terraform updates its internal address. Also used when refactoring into modules. - terraform state rm — removes a resource from state without destroying the real infrastructure. Used when you want to stop managing a resource with Terraform — the resource continues to exist in AWS but Terraform no longer tracks it. After removal, a plan will show the resource as a new creation if it is still in the configuration.
- terraform state pull / push — reads the remote state file and prints it as JSON (
pull), or uploads a local state file to the remote backend (push). These are low-level operations — use them only when you need to inspect or recover remote state.
# Rename a resource in state — use when you rename in configuration to avoid destroy+recreate
# Old address -> New address
terraform state mv aws_s3_bucket.lesson14 aws_s3_bucket.main
# Verify the rename took effect
terraform state list
# Remove a resource from state without destroying the real infrastructure
# After this, Terraform no longer tracks the bucket — it still exists in AWS
terraform state rm aws_s3_bucket_versioning.lesson14
# Pull remote state as JSON — useful for inspection and backup
terraform state pull > state-backup.json
# Show what state looks like after the mv
terraform state show aws_s3_bucket.main$ terraform state mv aws_s3_bucket.lesson14 aws_s3_bucket.main Move "aws_s3_bucket.lesson14" to "aws_s3_bucket.main" Successfully moved 1 object(s). $ terraform state list aws_s3_bucket.main # Renamed from lesson14 to main aws_s3_bucket_versioning.lesson14 # Still at original address random_id.suffix $ terraform state rm aws_s3_bucket_versioning.lesson14 Removed aws_s3_bucket_versioning.lesson14 Successfully removed 1 resource instance(s). $ terraform state list aws_s3_bucket.main # Bucket still tracked random_id.suffix # Versioning resource is gone from state — still exists in AWS $ terraform state pull > state-backup.json # state-backup.json now contains the full current state as JSON
What just happened?
- state mv renamed the address without touching the real bucket. The bucket still exists in AWS with the same ID. Terraform now tracks it as
aws_s3_bucket.maininstead ofaws_s3_bucket.lesson14. If you now rename the resource block inmain.tfto match, a subsequent plan shows zero changes — no destroy, no recreate. - state rm removed versioning from state — but versioning is still enabled on the bucket in AWS. The bucket's versioning configuration still exists in AWS —
state rmonly removes the Terraform tracking entry. A subsequent plan would showaws_s3_bucket_versioning.lesson14as a new resource to create if the resource block still exists in the configuration — because Terraform no longer knows it already created it. - state pull backed up the entire state to a local file.
state-backup.jsoncontains the full current state as JSON. This is useful before any risky state operation — if something goes wrong you have a snapshot to recover from. Store it securely — it contains all your resource IDs and potentially sensitive attributes.
What Never to Do with State
Never edit the state file manually
The state file is JSON and it is tempting to edit it with a text editor when something goes wrong. Do not. The state file has internal integrity checks, dependency maps, and provider-specific schemas. A manual edit that looks correct can corrupt the file in ways Terraform will not detect until a future operation fails catastrophically. Use terraform state mv, terraform state rm, and terraform import for all state modifications.
Never commit state to Git
State files contain sensitive data — resource IDs, IP addresses, database connection strings, and in many cases plaintext passwords from resources like aws_db_instance. Committing state to Git exposes all of this to anyone with repository access — including bots that scan public repositories for credentials. State belongs in a remote backend with access controls, not in version control.
Never run state push with an outdated state file
terraform state push overwrites the remote state file completely with whatever you push. If you push an old state file — one from before the last apply — Terraform loses track of everything that was created in the intervening operations. Any subsequent apply would try to recreate resources that already exist. If you must use state push for recovery, always pull first, verify the serial number, and push only a state file with a higher serial than what is currently remote.
Clean Up
# Rename the resource back in main.tf first — change aws_s3_bucket.lesson14 to aws_s3_bucket.main
# Then destroy everything
terraform destroy$ terraform destroy Plan: 0 to add, 0 to change, 2 to destroy. Enter a value: yes aws_s3_bucket.main: Destroying... [id=terraform-lesson-14-state-demo-a3f2b1c4] aws_s3_bucket.main: Destruction complete after 1s random_id.suffix: Destroying... [id=o_KxxA] random_id.suffix: Destruction complete after 0s Destroy complete! Resources: 2 destroyed.
State is the contract between Terraform and your infrastructure
Everything Terraform does — create, update, replace, destroy — is driven by comparing three things: configuration, state, and reality. Understanding what is in the state file, how it gets updated, and what happens when it diverges from reality is what separates engineers who use Terraform confidently from those who are afraid of it. Keep state remote, keep it encrypted, keep it backed up, and never edit it by hand.
Practice Questions
1. Which command prints every resource address currently tracked in the state file?
2. You rename a resource block in main.tf from aws_instance.server to aws_instance.web. Which command updates state to match the rename without destroying and recreating the real instance?
3. Which command updates the state file to match real infrastructure drift without reverting or creating any real changes?
Quiz
1. When you run terraform plan, what exactly does Terraform do with the state file?
2. What happens when you run terraform state rm on a resource?
3. How does Terraform detect concurrent write conflicts in the state file?
Up Next · Lesson 15
Local vs Remote State
You know what state contains. Lesson 15 covers where it should live — local state vs remote backends, and why the wrong choice for a team project causes the exact incidents you have been trying to avoid.