How to Manage Multiple Environments with Terraform Workspaces and an S3 Remote Backend

ByAkensai

How to Manage Multiple Environments with Terraform Workspaces and an S3 Remote Backend is one of the most practical skills you can add to your DevOps toolkit. If you’ve ever struggled to keep your development, staging, and production infrastructure cleanly separated, Terraform workspaces combined with an S3 remote backend offer a reliable solution. Instead of duplicating entire configuration directories for each environment, you use a single codebase and switch between isolated state files. This tutorial walks you through setting up an S3 bucket as your remote backend, creating workspaces for each environment, and deploying environment-specific infrastructure without stepping on your own toes. By the end, you’ll have a repeatable workflow that scales as your team grows.

Prerequisites and Requirements for Managing Multiple Environments

Before you start, make sure you have the following in place.

Required tools and access:

  • Terraform v1.3 or later installed on your local machine or CI server
  • AWS CLI installed and configured with a profile that has S3 and DynamoDB permissions
  • An AWS account with the ability to create S3 buckets and DynamoDB tables
  • Basic familiarity with HCL (HashiCorp Configuration Language)
  • A Linux or macOS terminal (Windows users should use WSL)

Assumed knowledge: You should already understand what Terraform is and have run terraform init at least once. You don’t need to be an expert.

Estimated time: 30 to 45 minutes.

You’ll also want to check the official Terraform remote state documentation before proceeding. It covers backend configuration options in detail.

Step-by-Step Guide to Terraform Workspaces and an S3 Remote Backend

You might also find this useful: How to Containerize a Node.js Web Application with Docker

Follow these steps in order. Each one builds on the last.

Step 1: Create your S3 bucket and DynamoDB table

You need an S3 bucket to store state files and a DynamoDB table for state locking. State locking prevents two people from running Terraform at the same time.

Run these AWS CLI commands:

aws s3api create-bucket 
  --bucket my-terraform-state-bucket 
  --region us-east-1

aws s3api put-bucket-versioning 
  --bucket my-terraform-state-bucket 
  --versioning-configuration Status=Enabled

aws dynamodb create-table 
  --table-name terraform-lock 
  --attribute-definitions AttributeName=LockID,AttributeType=S 
  --key-schema AttributeName=LockID,KeyType=HASH 
  --billing-mode PAY_PER_REQUEST 
  --region us-east-1

Versioning on the bucket means you can roll back to a previous state file if something goes wrong.

Step 2: Configure the S3 backend in your Terraform project

Create a new directory for your project and add a backend.tf file.

mkdir terraform-multi-env && cd terraform-multi-env
touch backend.tf main.tf variables.tf

Add this content to backend.tf:

terraform {
  backend "s3" {
    bucket         = "my-terraform-state-bucket"
    key            = "global/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-lock"
    encrypt        = true
  }
}

The key value is the path inside the bucket where the state file lives. Terraform workspaces will automatically prefix this path with the workspace name.

Step 3: Initialize the backend

Run the following command to connect your project to the S3 backend:

terraform init

You’ll see a message confirming that Terraform has been successfully initialized. If you see an error about credentials, double-check your AWS CLI profile with aws sts get-caller-identity.

Step 4: Create your workspaces

By default, Terraform uses a workspace called default. Create separate workspaces for each environment:

terraform workspace new dev
terraform workspace new staging
terraform workspace new prod

Switch between them like this:

terraform workspace select dev

List all workspaces at any time:

terraform workspace list

Each workspace stores its state at a unique path in S3, such as env:/dev/global/terraform.tfstate. This keeps your environments completely isolated.

Step 5: Write environment-aware configuration

Add this to your variables.tf file:

variable "instance_type" {
  default = "t3.micro"
}

Now add logic to main.tf that changes behavior based on the active workspace:

locals {
  env = terraform.workspace

  instance_types = {
    dev     = "t3.micro"
    staging = "t3.small"
    prod    = "t3.medium"
  }
}

resource "aws_instance" "app" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = local.instance_types[local.env]

  tags = {
    Name        = "app-${local.env}"
    Environment = local.env
  }
}

The terraform.workspace variable returns the name of the current workspace. This lets you use one configuration file to describe all three environments.

Step 6: Plan and apply per environment

Switch to your dev workspace and run a plan:

terraform workspace select dev
terraform plan

Review the output. When you’re happy, apply:

terraform apply

Repeat the process for staging and prod. Each environment gets its own isolated state file in S3. You won’t accidentally destroy prod resources while working on dev.

Step 7: Verify state isolation in S3

Check that separate state files exist in your bucket:

aws s3 ls s3://my-terraform-state-bucket/env:/ --recursive

You should see paths for each workspace. This confirms the isolation is working correctly.

Troubleshooting Common Issues with Terraform Workspaces

Even with a clean setup, things can go wrong. Here are the most common problems and how to fix them.

Error: Failed to get existing workspaces

This usually means your AWS credentials are missing or expired. Run aws sts get-caller-identity to confirm your session is active. If you’re using assumed roles, refresh your token.

State lock errors

If a previous run crashed, the DynamoDB lock entry may still exist. Find and remove it:

aws dynamodb scan --table-name terraform-lock

Then force-unlock with the lock ID shown in the Terraform error:

terraform force-unlock LOCK_ID

Only do this if you’re certain no other process is running Terraform.

Wrong workspace selected

Always run terraform workspace show before applying. It’s an easy habit to build and it prevents costly mistakes. Many teams add this as the first step in their CI pipeline.

Backend config changes not taking effect

If you change backend.tf, you must run terraform init -reconfigure to apply the new settings. A plain terraform init won’t pick up backend changes automatically.

For more guidance on AWS S3 bucket configuration and permissions, refer to the AWS S3 bucket naming and configuration documentation.

Conclusion

You now know how to manage multiple environments with Terraform Workspaces and an S3 Remote Backend. You created an S3 bucket with versioning, set up a DynamoDB lock table, configured the backend, and used workspaces to keep dev, staging, and production state files completely separate. You also wrote environment-aware Terraform code using terraform.workspace. This approach keeps your infrastructure organized and your team safe from accidental cross-environment changes. From here, you can explore using Terraform modules to share common patterns across environments, or integrate this workflow into a CI/CD pipeline using GitHub Actions or GitLab CI. Both options build naturally on what you’ve set up today.

Similar Posts