How To Build a Custom Terraform Module

Introduction

Terraform modules encapsulate distinct logical components of your infrastructure by grouping their resources together. You can reuse them later with possible customizations, without repeating the resource definitions each time you need them, which is beneficial to large and complexly structured projects. You can customize module instances using input variables you define as well as extract information from them using outputs. Aside from creating your own custom modules, you can also use the pre-made modules published publicly at the Terraform Registry. Developers can use and customize them using inputs like the modules you create, but their source code is stored in and pulled from the cloud.

In this tutorial, you’ll create a Terraform module that will set up multiple Droplets behind a Load Balancer for redundancy. You’ll also use the for_each and count looping features of the Hashicorp Configuration Language (HCL) to deploy multiple customized instances of the module at the same time.

Prerequisites

A DigitalOcean Personal Access Token, which you can create via the DigitalOcean control panel. Instructions to do that can be found in this link: How to Generate a Personal Access Token.
Terraform installed on your local machine and a project set up with the DO provider. Complete Step 1 and Step 2 of the How To Use Terraform with DigitalOcean tutorial, and be sure to name the project folder terraform-modules, instead of loadbalance. During Step 2, do not include the pvt_key variable and the SSH key resource.
Familiarity with HCL data types and loops. For more information, see the How To Improve Flexibility Using Terraform Variables, Dependencies, and Conditionals tutorial.
Familiarity with Terraform outputs and their usage. You can follow the How To Manage Infrastructure Data with Terraform Outputs tutorial to learn about them.

Note: This tutorial has specifically been tested with Terraform 0.13.

Module Structure and Benefits

In this section, you’ll learn what benefits modules bring, where they are usually placed in the project, and how they should be structured.

Custom Terraform modules are created to encapsulate connected components that are used and deployed together frequently in bigger projects. They are self contained, bundling only the resources, variables, and providers they need.

Modules are typically stored in a central folder in the root of the project, each in its respective subfolder underneath. In order to retain a clean separation between modules, always architect them to have a single purpose and make sure they never contain submodules.

It is useful to create modules from your resource schemes when you find yourself repeating them with infrequent customizations. Packaging a single resource as a module can be superfluous and gradually removes the simplicity of the overall architecture.

For small development and test projects, incorporating modules is not necessary because they do not bring much improvement in those cases. Modules, with their ability for customization, are the building element of complexly structured projects. Developers use modules for larger projects because of the significant advantages in avoiding code duplication. Modules also offer the benefit that definitions only need modification in one place, which will then be propagated through the rest of the infrastructure.

Next you’ll define, use, and customize modules in your Terraform projects.

Creating a Module

In this section, you’ll define multiple Droplets and a Load Balancer as Terraform resources and package them into a module. You’ll also make the resulting module customizable using module inputs.

You’ll store the module in a directory named droplet-lb, under a directory called modules. Assuming you are in the terraform-modules directory you created as part of the prerequisites, create both at once by running:

mkdir -p modules/droplet-lb

The -p argument instructs mkdir to create all directories in the supplied path.

Navigate to it:

cd modules/droplet-lb

As was noted in the previous section, modules contain the resources and variables they use. Starting from Terraform 0.13, they must also include definitions of the providers they use. Modules do not require any special configuration to note that the code represents a module, as Terraform regards every directory containing HCL code as a module, even the root directory of the project.

Variables defined in a module are exposed as its inputs and can be used in resource definitions to customize them. The module you’ll create will have two inputs: the number of Droplets to create and the name of their group. Create and open for editing a file called variables.tf where you’ll store the variables:

nano variables.tf

Add the following lines:

variables.tf

variable "droplet_count" {}
variable "group_name" {}

Save and close the file.

You’ll store the Droplet definition in a file named droplets.tf. Create and open it for editing:

nano droplets.tf

Add the following lines:

droplets.tf

resource "digitalocean_droplet" "droplets" {
  count  = var.droplet_count
  image  = "ubuntu-18-04-x64"
  name   = "${var.group_name}-${count.index}"
  region = "fra1"
  size   = "s-1vcpu-1gb"
}

For the count parameter, which specifies how many instances of a resource to create, you pass in the droplet_count variable. Its value will be specified when the module is called from the main project code. The name of each of the deployed Droplets will be different, which you achieve by appending the index of the current Droplet to the supplied group name. Deployment of the Droplets will be in the fra1 region and they will run Ubuntu 18.04.

When you are done, save and close the file.

With the Droplets now defined, you can move on to creating the Load Balancer. You’ll store its resource definition in a file named lb.tf. Create and open it for editing by running:

nano lb.tf

Add its resource definition:

lb.tf

resource "digitalocean_loadbalancer" "www-lb" {
  name   = "lb-${var.group_name}"
  region = "fra1"

  forwarding_rule {
    entry_port     = 80
    entry_protocol = "http"

    target_port     = 80
    target_protocol = "http"
  }

  healthcheck {
    port     = 22
    protocol = "tcp"
  }

  droplet_ids = [
    for droplet in digitalocean_droplet.droplets:
      droplet.id
  ]
}

You define the Load Balancer with the group name in its name in order to make it distinguishable. You deploy it in the fra1 region together with the Droplets. The next two sections specify the target and monitoring ports and protocols.

The highlighted droplet_ids block takes in the IDs of the Droplets, which should be managed by the Load Balancer. Since there are multiple Droplets, and their count is not known in advance, you use a for loop to traverse the collection of Droplets (digitalocean_droplet.droplets) and take their IDs. You surround the for loop with brackets ([]) so that the resulting collection will be a list.

You’ve now defined the Droplet, Load Balancer, and variables for your module. You’ll need to define the provider requirements, specifying which providers the module uses, including their version and where they are located. Since Terraform 0.13, modules must explicitly define the sources of non-Hashicorp maintained providers they use; this is because they do not inherit them from the parent project.

You’ll store the provider requirements in a file named provider.tf. Create it for editing by running:

nano provider.tf

Add the following lines to require the digitalocean provider:

provider.tf

terraform {
  required_providers {
    digitalocean = {
      source = "digitalocean/digitalocean"
    }
  }
  required_version = ">= 0.13"
}

Save and close the file when you’re done. The droplet-lb module now requires the digitalocean provider.

Modules also support outputs, which you can use to extract internal information about the state of their resources. You’ll define an output that exposes the IP address of the Load Balancer, and store it in a file named outputs.tf. Create it for editing:

nano outputs.tf

Add the following definition:

outputs.tf

output "lb_ip" {
  value = digitalocean_loadbalancer.www-lb.ip
}

This output retrieves the IP address of the Load Balancer. Save and close the file.

The droplet-lb module is now functionally complete and ready for deployment. You’ll call it from the main code, which you’ll store in the root of the project. First, navigate to it by going upward through your file directory two times:

cd ../..

Then, create and open for editing a file called main.tf, in which you’ll use the module:

nano main.tf

Add the following lines:

main.tf

module "groups" {
  source = "./modules/droplet-lb"

  droplet_count = 3
  group_name    = "group1"
}

output "loadbalancer-ip" {
  value = module.groups.lb_ip
}

In this declaration you invoke the droplet-lb module located in the directory specified as source. You configure the input it provides, droplet_count and group_name, which is set to group1 so you’ll later be able to discern between instances.

Since the Load Balancer IP output is defined in a module, it won’t automatically be shown when you apply the project. The solution to this is to create another output retrieving its value (loadbalancer_ip). Save and close the file when you’re done.

Initialize the module by running:

terraform init

The output will look like this:

OutputInitializing modules...
- droplet-lb in modules/droplet-lb

Initializing the backend...

Initializing provider plugins...
- Using previously-installed digitalocean/digitalocean v1.22.2

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

You can try planning the project to see what actions Terraform would take by running:

terraform plan -var "do_token=${DO_PAT}"

The output will be similar to this:

Output...
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.groups.digitalocean_droplet.droplets[0] will be created
  + resource "digitalocean_droplet" "droplets" {
...
      + name                 = "group1-0"
...
    }

  # module.groups.digitalocean_droplet.droplets[1] will be created
  + resource "digitalocean_droplet" "droplets" {
...
      + name                 = "group1-1"
...
    }

  # module.groups.digitalocean_droplet.droplets[2] will be created
  + resource "digitalocean_droplet" "droplets" {
...
      + name                 = "group1-2"
...
    }

  # module.groups.digitalocean_loadbalancer.www-lb will be created
  + resource "digitalocean_loadbalancer" "www-lb" {
...
      + name                     = "group1-lb"
...
    }

Plan: 4 to add, 0 to change, 0 to destroy.
...

This output details that Terraform would create three Droplets, named group1-0, group1-1, and group1-2, and would also create a Load Balancer called group1-lb, which will manage the traffic to and from the three Droplets.

You can try applying the project to the cloud by running:

terraform apply -var "do_token=${DO_PAT}"

Enter yes when prompted. The output will show all the actions and the IP address of the Load Balancer will also be shown:

Outputmodule.groups.digitalocean_droplet.droplets[1]: Creating...
module.groups.digitalocean_droplet.droplets[0]: Creating...
module.groups.digitalocean_droplet.droplets[2]: Creating...
...
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

loadbalancer-ip = ip_address

Because you’ll modify the configuration significantly in the next step, destroy the deployed resources by running:

terraform destroy -var "do_token=${DO_PAT}"

Enter yes when prompted. The output will end in:

Output..
Destroy complete! Resources: 4 destroyed.

In this step, you’ve created a module containing a customizable number of Droplets and a Load Balancer that will automatically be configured to manage their ingoing and outgoing traffic. You’ll now deploy multiple instances of a module from the same code using for_each and count.

Deploying Multiple Module Instances

In this section, you’ll use count and for_each to deploy the droplet-lb module multiple times, with customizations.

Using `count`

One way to deploy multiple instances of the same module at once is to pass in how many to the count parameter, which is automatically available to every module. Open main.tf for editing:

nano main.tf

Modify it to look like this:

main.tf

module "groups" {
  source = "./modules/droplet-lb"

  count  = 3

  droplet_count = 3
  group_name    = "group1-${count.index}"
}

By setting count to 3, you instruct Terraform to deploy the module three times, each with a different group name. When you’re done, save and close the file.

Plan the deployment by running:

terraform plan -var "do_token=${DO_PAT}"

The output will be long, and will look like this:

Output...
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.groups[0].digitalocean_droplet.droplets[0] will be created
...
  # module.groups[0].digitalocean_droplet.droplets[1] will be created
...
  # module.groups[0].digitalocean_droplet.droplets[2] will be created
...
  # module.groups[0].digitalocean_loadbalancer.www-lb will be created
...
  # module.groups[1].digitalocean_droplet.droplets[0] will be created
...
  # module.groups[1].digitalocean_droplet.droplets[1] will be created
...
  # module.groups[1].digitalocean_droplet.droplets[2] will be created
...
  # module.groups[1].digitalocean_loadbalancer.www-lb will be created
...
  # module.groups[2].digitalocean_droplet.droplets[0] will be created
...
  # module.groups[2].digitalocean_droplet.droplets[1] will be created
...
  # module.groups[2].digitalocean_droplet.droplets[2] will be created
...
  # module.groups[2].digitalocean_loadbalancer.www-lb will be created
...

Plan: 12 to add, 0 to change, 0 to destroy.
...

Terraform details in the output that each of the three module instances would have three Droplets and a Load Balancer associated with them.

Using `for_each`

You can use for_each for modules when you require more complex instance customization, or when the number of instances depends on third-party data (often presented as maps) and is not known while writing the code.

You’ll now define a map that pairs group names to Droplet counts and deploy instances of droplet-lb according to it. Open main.tf for editing by running:

nano main.tf

Modify the file to make it look like this:

main.tf

variable "group_counts" {
  type    = map
  default = {
    "group1" = 1
    "group2" = 3
  }
}

module "groups" {
  source   = "./modules/droplet-lb"
  for_each = var.group_counts

  droplet_count = each.value
  group_name    = each.key
}

You first define a map called group_counts that contains how many Droplets a given group should have. Then, you invoke the module droplet-lb, but specify that the for_each loop should operate on var.group_counts, the map you’ve defined just before. droplet_count takes each.value, the value of the current pair, which is the count of Droplets for the current group. group_name receives the name of the group.

Save and close the file when you’re done.

Try applying the configuration by running:

terraform plan -var "do_token=${DO_PAT}"

The output will detail the actions Terraform would take to create the two groups with their Droplets and Load Balancers:

Output...
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.groups["group1"].digitalocean_droplet.droplets[0] will be created
...
  # module.groups["group1"].digitalocean_loadbalancer.www-lb will be created
...
  # module.groups["group2"].digitalocean_droplet.droplets[0] will be created
...
  # module.groups["group2"].digitalocean_droplet.droplets[1] will be created
...
  # module.groups["group2"].digitalocean_droplet.droplets[2] will be created
...
  # module.groups["group2"].digitalocean_loadbalancer.www-lb will be created
...

In this step, you’ve used count and for_each to deploy multiple customized instances of the same module, from the same code.

Conclusion

In this tutorial you’ve created and deployed Terraform modules. You’ve used modules to group logically linked resources together and customized them in order to deploy multiple different instances from a central code definition. You’ve also used outputs to show attributes of resources contained in the module.

If you would like to learn more about Terraform, check out our How To Manage Infrastructure with Terraform series.

Originally posted on DigitalOcean Community Tutorials
Author: Savic

How To Build a Custom Terraform Module

Introduction

Prerequisites

Module Structure and Benefits

Creating a Module

Deploying Multiple Module Instances

Using `count`

Using `for_each`

Conclusion

Related

Deja una respuesta Cancelar la respuesta

Introduction

Prerequisites

Module Structure and Benefits

Creating a Module

Deploying Multiple Module Instances

Using count

Using for_each

Conclusion

Related

Deja una respuesta Cancelar la respuesta

Using `count`

Using `for_each`