How to manage multiple environments with Terraform

You used Terraform to deploy one environment (e.g., development or prod), and things are working great, but now you need to deploy several more environments, and you need to figure out how to organize your code. The question arises as to whether go with workspaces? Or branches? Or Terragrunt?

In the initial stage of a Terraform project many developers make a decision without giving it a second thought. Later it becomes headache to manage huge amounts of code duplication, insufficient isolation between environments, and difficulty in debugging. To help you avoid these problems, I’ve put together a blog post that compares the 4 most popular options for defining and managing environments in Terraform:

• Terraform CLI Workspaces

• Reusable Modules

• Branches

• Terragrunt

What this guide is not about

• An introduction to Terraform

• An introduction to Terragrunt

Instead, this guide will help you understand which path to choose going forward for multiple environments such as production/QA/Staging.

If you're looking for an overview of Terraform and how to use it, the Terraform docs is a great place to start.

If you're looking for an intro to Terragrunt, the Terragrunt docs is a great place to start.

How to Manage Multiple Environments with Terraform?

When Terraform deploys resources, it stores metadata to track the resources in a state file. So if we want to deploy resources for multiple environments such as production/QA/Staging, we need to differentiate deployment actions and state files as well.

1. Terraform CLI Workspaces

Note: Workspaces in the Terraform CLI refer to separate instances of state data inside the same Terraform working directory. They are distinctly different from workspaces in Terraform Cloud, which each have their own Terraform configuration and function as separate working directories.

When workspaces run locally, Terraform manages each collection of infrastructure with a persistent working directory, which contains a configuration, state data, and variables. Since Terraform CLI uses content from the directory it runs in, you can organize infrastructure resources into meaningful groups by keeping their configurations in separate directories.

Workspace does isolate the terraform.tfstate file from other workspaces in the same backend.

Right now, we have a workspace named default, which is default one and can’t be deleted ever

Below are few commands to manage workspaces,

• terraform workspace list --> To list workspaces

• terraform workspace new --> To create a new workspace

• terraform workspace select --> To select appropriate workspace

• terraform workspace delete --> To delete workspace

I created some other workspaces. Dev & prod -

In this demo, we used s3 as the backend to store state files. Configure s3 bucket, region and key in the backend block of terraform -

Based on the key which is defined in the backend block, inside the bucket it will create a defined path for default workspace and env folder for workspaces -

Inside env, it will create other folders for each workspace and store the terraform.tfstate file -

Now let’s launch an ec2 instance and security group from terraform configuration files to understand how to deploy the resources with the same working copy of your configuration.

In the resource block, we provided “${terraform.workspace}” to provide the current workspace name to the specific parameters in the specified block.

This code configures the name tag to include the workspace name (so it’s dev instance in the dev environment, prod instance in the prod environment, etc.)

Below is the configuration of the security group, it will create a security group upon below configuration -

We have configured variables.tf file to expose variables and we will provide values of those variables in tfvars files. We created dev.tfvars for dev and prod.tfvars for prod workspace-

Let’s deploy resources in dev workspace, Let’s switch workspace to dev workspace with command “terraform workspace select dev”, and give terraform plan to view execution plan of defined AWS resources, then apply if everything is as expected -

terraform workspace select dev
terraform apply -auto-approve -var-file dev.tfvars

After apply completes, it will store the metadata in the state file -

Let’s deploy same resource in prod workspace-

terraform workspace select prod
terraform apply -auto-approve -var-file prod.tfvars

I created an output block for instance name and tags output-

Let’s check that output value in both workspaces-

●Dev workspace:

terraform workspace select dev

terraform output instance_value
Output: i-055910feb12098 is dev instance

●Prod workspace:

terraform workspace select dev

terraform output instance_value
Output: i-022410be4e12312 is prod instance

Note that if you use terraform local workspaces to manage environments, navigating your environments and understanding what’s deployed is really hard. That’s because you can’t see the environments in the code itself, but only from the terraform workspace CLI commands, which you have to run on one module at a time. Let say you have your infrastructure defined across many small modules and you’re using workspaces. It becomes hard to answer questions like “What’s deployed in my dev environment?” and “What are the differences between dev and prod?”

So, this is the first way to manage multiple environments deployments with terraform workspaces. Now let’s look at the other method to manage multiple environments.

2. Reusable modules

Reusable modules architecture is based on the module block of terraform which can source directory stored “.tf” configurations and can pass variables.

In this architecture we would have one folder containing all the resource configurations files including variables and output config files. Then we will create separate folders for dev and stage, where we will define module block, provider block, backend configuration etc.

Right now we have below configuration for ec2 and s3 creation in multiple environments from a single source of configuration files-

In order to differentiate state files we need to update the backend manually

Let’s deploy this first in the dev environment:

terraform_folder/dev$ terraform apply -auto-approve

Now we deploy in Prod environment:

terraform_folder/prod$ terraform apply -auto-approve

Note - We saw how workspaces and reusable modules help us to manage separate infrastructure from separate state files. Reusable modules function on separate directory architecture and contain module blocks, variables, provider and backend configuration etc. Where workspaces work on a single source of configuration files and creates workspace specific state files in the backend.

3. Branches

When you use branches (Git in this case), there is no native mechanism built into Terraform to tell you what branch you’re on. That means you have to use a workaround.

Let’s start with the simple EC2 instance example:

provider "aws" {
  region = "us-east-2"
}

resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"

  tags = {
    Name = "HelloWorld"
  }
}

To deploy this code in the dev environment, the first step is to create a dev branch:

$ git checkout -b dev
Switched to a new branch 'dev'

Now you’re ready to deploy by running apply:

$ terraform apply

If everything looks good, enter yes, and Terraform will deploy the server. Commit your changes to the dev branch:

$ git add .
$ git commit -m "Set up dev environment"
$ git push origin dev

OK, now it’s time to work on the preprod environment. First, create a new branch:

$ git checkout -b preprod
Switched to a new branch 'preprod'

Run apply to deploy:

$ terraform apply

And then commit your changes to the preprod branch:

$ git add .
$ git commit -m "Set up preprod environment"
$ git push origin preprod

Finally, repeat the entire process for production.

At this point, you have three environments (three branches), with one EC2 instance deployed in each one. Unfortunately, those three branches mean you now have three full copies of your Terraform code that you have to manage: that’s 100% code duplication for each environment you add, which can be a significant maintenance challenge.

Switching between environments

To switch between environments, you need to switch between branches using git commands.

$ git branch
  main
  dev
  stage
* prod

Note

• With branches, it’s easier to see what’s deployed in each environment than with workspaces, as each environment is in its own branch. That said, browsing or making changes across multiple environments is still a bit tedious and error prone, as you have to repeatedly switch branches.

• Folders are roughly equivalent to branches. In effect, you duplicating 100% of your code for each environment you add: in the case of branches, the duplicate copies live in separate branches; in the case of folders, the duplicate copies live in separate folders. Using modules under the hood reduces the duplication, but not entirely: you still have duplicated provider blocks, backend configurations, output variables, input variables (at least for secrets), and so on.

• You can configure the backend block differently in each branch. For example, in the dev branch, you could configure the the S3 bucket terraform-dev-bucket as a backend as follows:

terraform {
  backend "s3" {
    bucket = "terraform-dev-bucket"
    key    = "terraform/terraform.tfstate"
    region = "us-east-1"
  }
}

Drawbacks of branches

• A huge amount of code duplication makes maintenance a nightmare.

• Working with multiple modules is manual and error-prone.

• Propagating major changes across environments is tricky and error prone, especially if you don’t externalize differences between branches into separate files.

4. Terragrunt

What is Terragrunt??

Terragrunt is a thin wrapper that provides extra tools for keeping your configurations DRY, working with multiple Terraform modules, and managing remote state.

Challenges of Terraform

If your infrastructure is very large and complex and it has multiple environments in it, then we have to duplicate the same code in each environment and it becomes very tedious process to do so. Terragrunt will keep your terraform code DRY so that we can reuse the same code in different environments.

To get more idea about different features , refer its official documentation:

Features of Terragrunt

Terragrunt can help you accomplish the following:

• Keep your backend configuration DRY

  Terragrunt allows you to keep your backend configuration DRY (“Don’t Repeat Yourself”) by defining it once in a root location and inheriting that configuration in all child modules. Let’s say your Terraform code has the following folder layout:

stage
├── frontend-app
│   └── main.tf
└── mysql
    └── main.tf

To use Terragrunt, add a single terragrunt.hcl file to the root of your repo, in the stage folder, and one terragrunt.hcl file in each module folder:

stage
├── terragrunt.hcl
├── frontend-app
│   ├── main.tf
│   └── terragrunt.hcl
└── mysql
    ├── main.tf
    └── terragrunt.hcl

Now you can define your backend configuration just once in the root terragrunt.hcl file:

# stage/terragrunt.hcl
remote_state {
  backend = "s3"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }
  config = {
    bucket = "my-terraform-state"

    key = "${path_relative_to_include()}/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "my-lock-table"
  }
}

The final step is to update each of the child terragrunt.hcl files to tell them to include the configuration from the root terragrunt.hcl:

# stage/mysql/terragrunt.hcl
include "root" {
  path = find_in_parent_folders()
}

The find_in_parent_folders() helper will automatically search up the directory tree to find the root terragrunt.hcl and inherit the remote_state configuration from it.

To deploy the database module, you would run:

$ cd stage/mysql
$ terragrunt apply

• Keep your provider configuration DRY

Unifying provider configurations across all your modules can be a pain, especially when you want to customize authentication credentials. Terragrunt allows you to refactor common Terraform code to keep your Terraform modules DRY

• Keep your Terraform CLI arguments DRY

  CLI flags are another common source of copy/paste in the Terraform world. For example, a typical pattern with Terraform is to define common variables in an ".tfvars" file. You can tell Terraform to use these variables using the -var-file argument:

terraform apply \
    -var-file=../../common.tfvars

Having to remember these -var-file arguments every time can be tedious and error prone. Terragrunt allows you to keep your CLI arguments DRY by defining those arguments as code in your terragrunt.hcl configuration:

# terragrunt.hcl
terraform {
  extra_arguments "common_vars" {
    commands = ["plan", "apply"]

    arguments = [
      "-var-file=../../common.tfvars",
      "-var-file=../region.tfvars"
    ]
  }
}

Now, when you run the plan or apply commands, Terragrunt will automatically add those arguments:

$ terragrunt apply

Running command: terraform with arguments
[apply -var-file=../../common.tfvars -var-file=../region.tfvars]

• Promote immutable, versioned Terraform modules across environments

One of the most important lessons we’ve learned is that it's a Bad Idea to define all of your environments, or even a large amount of infrastructure, in a single Terraform module. Large modules are slow, insecure, hard to update, hard to code review, hard to test, and brittle (i.e., you have all your eggs in one basket).

Terragrunt allows you to define your Terraform code once and to promote a versioned, immutable “artifact” of that exact same code from environment to environment.

Advantages of workspaces vs Terragrunt

1. Workspaces are natively built into Terraform, so you don't need to learn an external tool like Terragrunt.

2. Workspaces are natively used by Terraform Cloud (TFC) and Enterprise (TFE).

3. Workspaces are natively supported in the Terraform code itself: you can get the name of the workspace using terraform.workspace.

4. Workspaces allow you to keep one copy of your code, but to have the state of the infrastructure captured in separate state files. This reduces code duplication. With Terragrunt, you do need to create extra files/folders to define each environment, so you do end up with a bit more duplication.

Note that the Idea behind Terragrunt is to make those environments visible in your repo, but it does mean having more files/folders to manage.

Drawbacks of workspaces vs Terragrunt

1. The most common way people try to use workspaces is to define separate environments. It turns out that even HashiCorp itself does NOT recommend this. Workspaces don't have good enough support for the separation of configuration and backends that you need between different environments.

Terragrunt is specifically designed to help you define and manage multiple environments in a DRY manner, and this is an effective way to keep configurations separate across environments and using separate backends.

1. Terragrunt is natively built to not only support multiple environments, but to use different versions of your Terraform modules in each environment by setting different source URLs. With workspaces, there's no native support for this (the source URL in Terraform modules doesn't allow any variables or interpolation).

2. Just by browsing the repo, you can't tell what workspaces or environments actually exist; you'd have to look at state files or run terraform workspace list to figure that out. Whereas in Terragrunt, we define them in files/folders on disk, so just by browsing a repo, you have a really good idea of what's deployed and the intention behind it.

3. To create or switch to a workspace, you run a command, which is quite error prone. With Terragrunt, since environments are defined in files/folders on disk, it's a lot easier to tell that you're in the dev or prod folder, which minimizes that sort of mistake.

Thanks for reading. If you loved this article, feel free to hit that like & follow button so we can stay in touch.

This article is possible because of these references.

Terragrunt Key features

Terraform

pros/cons of using Terragrunt versus Terraform Workspaces

𝐵𝓊𝓎 𝑀𝑒 𝒶 𝒞𝑜𝒻𝒻𝑒𝑒 ☕️