Terraform Modules and Remote State

Introduction¶

Two things separate a toy Terraform setup from one a team can operate: modules (so configuration is reusable and not copy-pasted) and remote state with locking (so two people don't corrupt it). See Terraform state explained for what state is; this is about structuring it.

Modules are functions¶

A module is a directory of .tf files with declared inputs (variable) and outputs (output). The root module calls child modules.

modules/
  vpc/
    main.tf
    variables.tf
    outputs.tf
environments/
  prod/
    main.tf          # calls modules/vpc etc.
    backend.tf
  staging/
    main.tf

# modules/vpc/variables.tf
variable "cidr_block"   { type = string }
variable "az_count"     { type = number, default = 2 }
variable "name"         { type = string }

# modules/vpc/outputs.tf
output "vpc_id"          { value = aws_vpc.this.id }
output "private_subnets" { value = aws_subnet.private[*].id }

# environments/prod/main.tf
module "vpc" {
  source     = "../../modules/vpc"
  name       = "prod"
  cidr_block = "10.0.0.0/16"
  az_count   = 3
}

module "app" {
  source          = "../../modules/app"
  subnet_ids      = module.vpc.private_subnets
  vpc_id          = module.vpc.vpc_id
}

Module design rules:

• Inputs are the contract. Type them, document them, give safe defaults where sensible. Validate: validation { condition = ... }.
• Outputs expose only what callers need. Don't output an entire resource.
• No provider blocks inside reusable modules (pass providers explicitly if needed). No backend blocks in modules — only the root.
• Version external modules: source = "terraform-aws-modules/vpc/aws", version = "~> 5.0".

Remote state with locking¶

Local state (terraform.tfstate on disk) doesn't work for teams: no sharing, no locking, secrets on a laptop. Use a backend.

# environments/prod/backend.tf
terraform {
  backend "s3" {
    bucket         = "acme-tfstate"
    key            = "prod/network.tfstate"
    region         = "eu-west-1"
    dynamodb_table = "tf-locks"     # the lock table
    encrypt        = true
  }
}

• Locking — the DynamoDB table (S3 backend) or the backend's native lock (GCS, azurerm, Terraform Cloud) makes concurrent apply wait instead of racing. A crashed run can leave a stale lock: terraform force-unlock <id> after confirming nobody's running.
• key — the path within the bucket. One key = one state file.

Splitting state by blast radius¶

Don't put the whole platform in one state. A terraform apply on the app shouldn't be able to touch the VPC. Common split:

network.tfstate       # VPC, subnets, TGW, DNS zones — changes rarely
data.tfstate          # RDS, ElastiCache, S3 buckets — changes sometimes
app-<service>.tfstate  # ECS/EKS services, ALB rules — changes constantly

Downstream configs read upstream outputs:

data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "acme-tfstate"
    key    = "prod/network.tfstate"
    region = "eu-west-1"
  }
}

resource "aws_lb" "app" {
  subnets = data.terraform_remote_state.network.outputs.public_subnets
}

Trade-off: more states = more applys to wire up a change that spans layers, but far smaller blast radius and faster plans.

Verification and troubleshooting¶

terraform init -reconfigure          # after changing the backend
terraform state list
terraform state show module.vpc.aws_vpc.this
terraform output -json
terraform plan -out plan.tfplan && terraform apply plan.tfplan

• Error acquiring the state lock — someone's running apply, or a stale lock from a crash. Check with them; terraform force-unlock <lock-id> only when certain.
• terraform_remote_state output is null — the upstream config didn't output it, or hasn't been applied yet, or you're reading the wrong key.
• Module change didn't take effect — terraform get -update / terraform init -upgrade to re-pull versioned modules; local path modules update automatically.
• Provider configuration not present after refactoring into a module — the module needs providers passed via providers = { aws = aws.useast1 }.
• State drift — someone changed a resource in the console. terraform plan shows it; terraform apply reverts, or terraform import / -refresh-only to adopt the change.
• Moved a resource into a module, plan wants to destroy+recreate — use a moved {} block (or terraform state mv) so Terraform tracks the rename.

Related tools and reading¶

• On-site: JSON formatter, YAML validator.
• Related posts: Terraform state explained, VPC subnets and route tables.