How To Import Existing DigitalOcean Assets into Terraform

The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

Introduction

Terraform is an infrastructure as code tool created by HashiCorp that helps developers with deploying, updating, and removing different assets of their infrastructure in an efficient and more scalable way.

Developers can use Terraform to organize different environments, track changes through version control, and automate repetitive work to limit human error. It also provides a way for teams to collaborate on improving their infrastructure through shared configurations.

In this tutorial you’ll import existing DigitalOcean infrastructure into Terraform. By the end of this tutorial you’ll be able to use Terraform for all of your existing infrastructure in addition to creating new assets.

Prerequisites

• A DigitalOcean Personal Access Token. To create this, you can follow the
  How To Create a Personal Access Token guide.
• A DigitalOcean Droplet with a tag. You can use the following guide on How To Create a Droplet from the DigitalOcean Control Panel.
• A DigitalOcean Cloud Firewall applied to your Droplet. You can use the guide How To Create Firewalls.
• The DigitalOcean Command Line Client installed on your local machine by following the install instructions on the doctl GitHub page. You can read the following tutorial for guidance on How To Use Doctl, the official DigitalOcean Command-Line Client.

Step 1 — Installing Terraform Locally

In this first step you’ll install Terraform on your local machine. This step details the installation of the Linux binary. If you use Windows or Mac, you can check the Download Terraform page on the Terraform website.

Move to the folder you want to download Terraform to on your local machine, then use the wget tool to download the Terraform 0.12.12 binary:

cd /tmp
wget https://releases.hashicorp.com/terraform/0.12.12/terraform_0.12.12_linux_amd64.zip

To check if the sha256 checksum is the same value provided on the Terraform website, you’ll download the checksum file with the following command:

wget -q https://releases.hashicorp.com/terraform/0.12.12/terraform_0.12.12_SHA256SUMS

Then run the following command to verify the checksums:

sha256sum -c --ignore-missing terraform_0.12.12_SHA256SUMS

The SHA256SUMS file you downloaded lists the filenames and their hashes. This command will look for the same file terraform_0.12.12_SHA256SUMS locally and then check that the hashes match by using the -c flag. Since this file has more than one filename and its platform listed, you use the --ignore-missing flag to avoid errors in your output because you don’t have a copy of the other files.

You will see output like the following:

Output
terraform_0.12.12_linux_amd64.zip: OK

Use unzip to extract the binary:

sudo unzip terraform_0.12.12_linux_amd64.zip -d /usr/local/bin/

Now check if Terraform is installed properly by checking the version:

terraform version

You’ll see output similar to the following:

Output
Terraform v0.12.12

You’ve installed Terraform to your local machine, you’ll now prepare the configuration files.

Step 2 — Preparing Terraform Configuration Files

In this step you’ll import your existing assets into Terraform by creating a project directory and writing configuration files. Since Terraform doesn’t support generating configs from the import command at this time, you need to create those configurations manually.

Run the following command to create your project directory:

mkdir -p do_terraform_import

Then move into that directory with:

cd do_terraform_import

Within this step you’ll create three additional files that will contain the required configurations. Your directory structure for this project will look like the following:

├── digitalocean_droplet.tf
├── digitalocean_firewall.tf
└── provider.tf

To begin you’ll create the file provider.tf to define your DigitalOcean Access Token as an environment variable instead of hardcoding it into your configuration.

Warning: Your access token gives access to your complete infrastructure with unrestricted access, so treat it as such. Be sure that you’re the only one who has access to the machine where that token is stored.

Besides your access token, you’ll also specify which provider you want to use. In this tutorial that’s digitalocean. For a full list of available Data Sources and Resources for DigitalOcean with Terraform, visit the Providers page on their website.

Create and edit provider.tf with the following command:

nano provider.tf

Add the following content into the provider.tf file:

variable "do_token" {}

provider "digitalocean" {
    token   = "${var.do_token}"
    version = "1.9.1"
    }

In this file you add your DigitalOcean Access Token as a variable, which Terraform will use as identification for the DigitalOcean API. You also specify the version of the DigitalOcean provider plugin. Terraform recommends that you specify which version of the provider you’re using so that future updates don’t potentially break your current setup.

Now you’ll create the digitalocean_droplet.tf file. Here you’ll specify the resource that you’re going to use, in this case: droplet.

Create the file with the following command:

nano digitalocean_droplet.tf

Add the following configuration:

resource "digitalocean_droplet" "do_droplet" {
    name   = "testing-terraform"
    region = "fra1"
    tags   = ["terraform-testing"]
    count  = "1"
}

Here you specify four parameters:

• name: The Droplet name.

• region: The region that the Droplet is located in.

• tags: A list of the tags that are applied to this Droplet.

• count: The number of resources needed for this configuration.

Next you’ll create a configuration file for your firewall. Create the file digitalocean_firewall.tf with the following command:

nano digitalocean_firewall.tf

Add the following content to the file:

resource "digitalocean_firewall" "do_firewall" {
  name  = "testing-terraform-firewall"
  tags  = ["terraform-testing"]
  count = "1"
}

Here you specify the name of the firewall you wish to import and the tags of the Droplets to which the firewall rules apply. Finally the count value of 1 defines the required number of the particular resource.

Note: You can include firewall resources in the digitalocean_droplet.tf file as well, however if you have multiple environments where multiple Droplets share the same firewall, it’s a good idea to separate it in case you only want to remove a single Droplet. This will then leave the firewall unaffected.

Now it’s time to initialize those changes so Terraform can download the required dependencies. You will use the terraform init command for this, which will allow you to initialize a working directory containing Terraform configuration files.

Run this command from your project directory:

terraform init

You’ll see the following output:

Output
Terraform has been successfully initialized!

Terraform has successfully prepared the working directory by downloading plugins, searching for modules, and so on. Next you’ll begin importing your assets to Terraform.

Step 3 — Importing Your Assets to Terraform

In this step, you’ll import your DigitalOcean assets to Terraform. You’ll use doctl to find the ID numbers of your Droplets before importing your assets. You’ll then check the import configuration with the terraform show and terraform plan commands.

To begin, you’ll export your DigitalOcean Access Token as an environment variable, which you’ll then inject into Terraform during runtime.

Export it as an environment variable into your current shell session with the following command:

export DO_TOKEN="YOUR_TOKEN"

In order to import your existing Droplet and firewall you’ll need their ID numbers. You can use doctl, the command line interface for the DigitalOcean API. Run the following command to list your Droplets and access their IDs:

doctl compute droplet list

You’ll see output similar to the following:

Output
ID           Name                     Public IPv4        Private IPv4     Public IPv6    Memory    VCPUs    Disk    Region    Image                       Status    Tags    Features              Volumes
DROPLET-ID    DROPLET-NAME                          DROPLET-IPv4                 1024      1        25      fra1      Ubuntu 18.04.3 (LTS) x64              active
DROPLET-ID    DROPLET-NAME                          DROPLET-IPv4                 2048      1        50      fra1      Ubuntu 18.04.3 (LTS) x64           active
DROPLET-ID    DROPLET-NAME                          DROPLET-IPv4                 1024      1        25      fra1      Ubuntu 18.04.3 (LTS) x64

Now you’ll import your existing Droplet and firewall into Terraform:

terraform import -var "do_token=${DO_TOKEN}" digitalocean_droplet.do_droplet DROPLET_ID

You use the -var flag to specify your DigitalOcean Access Token value that you previously exported to your shell session. This is needed so the DigitalOcean API can verify who you are and apply changes to your infrastructure.

Now run the same command for your firewall:

terraform import -var "do_token=${DO_TOKEN}" digitalocean_firewall.do_firewall FIREWALL_ID

You’ll check that the import was successful by using the terraform show command. This command provides human-readable output of your infrastructure state. It can be used to inspect a plan to ensure that wanted changes are going to be executed, or to inspect the current state as Terraform sees it.

In this context state refers to the mapping of your DigitalOcean assets to the Terraform configuration that you’ve written and the tracking of metadata. This allows you to confirm that there’s no difference between existing DigitalOcean assets that you want to import and assets that Terraform is keeping track of:

terraform show

You’ll see output similar to this:

Output
. . .
# digitalocean_droplet.do_droplet:
resource "digitalocean_droplet" "do_droplet" {
    backups            = false
    created_at         = "2020-02-03T16:12:02Z"
    disk               = 25
    id                 = "DROPLET-ID"
    image              = "DROPLET-IMAGE"
    ipv4_address       = "DROPLET-IP"
    ipv6               = false
    locked             = false
    memory             = 1024
    monitoring         = false
    name               = "testing-terraform-0"
    price_hourly       = 0.00744
    price_monthly      = 5
    private_networking = false
    region             = "fra1"
    resize_disk        = true
    size               = "s-1vcpu-1gb"
    status             = "active"
    tags               = [
        "terraform-testing",
    ]
    urn                = "DROPLET-URN"
    vcpus              = 1
    volume_ids         = []
. . .
}

You’ll see two resources in the output along with their attributes.

After you import your Droplet and firewall into Terraform state, you need to make sure that configurations represent the current state of the imported assets. To do this, you’ll specify your Droplet’s image and its size. You can find these two values in the output of terraform show for digitalocean_droplet.do_droplet resource.

Open the digitalocean_droplet.tf file:

nano digitalocean_droplet.tf

In this tutorial:

• The operating system image used for our existing Droplet is ubuntu-16-04-x64.
• The region your Droplet is located in is fra1.
• The Droplet tag for your existing Droplet is terraform-testing.

The Droplet you imported using the configuration in digitalocean_droplet.tf will look like this:

resource "digitalocean_droplet" "do_droplet" {
    image   = "ubuntu-16-04-x64"
    name    = "testing-terraform"
    region  = "fra1"
    size    = "s-1vcpu-1gb"
    tags    = ["terraform-testing"]
}

Next you’ll add in the firewall rules. In our example, open ports for inbound traffic are 22, 80, and 443. All ports are opened for outbound traffic. You can adjust this configuration accordingly to your open ports.

Open digitalocean_firewall.tf:

nano digitalocean_firewall.tf

Add the following configuration:

resource "digitalocean_firewall" "do_firewall" {
  name  = "testing-terraform-firewall"
  tags  = ["terraform-testing"]
  count = "1"

  inbound_rule {
      protocol                = "tcp"
      port_range              = "22"
      source_addresses        = ["0.0.0.0/0", "::/0"]
    }
  inbound_rule {
      protocol                = "tcp"
      port_range              = "80"
      source_addresses        = ["0.0.0.0/0", "::/0"]
    }
  inbound_rule {
      protocol                = "tcp"
      port_range              = "443"
      source_addresses        = ["0.0.0.0/0", "::/0"]
    }

  outbound_rule {
      protocol                = "tcp"
      port_range              = "all"
      destination_addresses   = ["0.0.0.0/0", "::/0"]
    }
  outbound_rule {
      protocol                = "udp"
      port_range              = "all"
      destination_addresses   = ["0.0.0.0/0", "::/0"]
    }
  outbound_rule {
      protocol                = "icmp"
      destination_addresses   = ["0.0.0.0/0", "::/0"]
    }
}

These rules replicate the state of the existing example firewall. If you’d like to limit traffic to different IP addresses, different ports, or different protocol, you can adjust the file to replicate your existing firewall.

After you’ve updated your Terraform files, you’ll use the plan command to see if changes you made replicate state of existing assets on DigitalOcean.

The terraform plan command is used as a dry run. With this command you can check if changes Terraform is going to make are the changes you want to make. It is a good idea to always run this command for confirmation before applying changes.

Run terraform plan with the following:

terraform plan -var "do_token=$DO_TOKEN"

You’ll see output similar to the following output:

Output
No changes. Infrastructure is up-to-date.

You’ve successfully imported existing DigitalOcean assets in Terraform, and now you can make changes to your infrastructure through Terraform without the risk of accidentally deleting or modifying existing assets.

Step 4 — Creating New Assets via Terraform

In this step you’ll add two additional Droplets to your existing infrastructure. Adding assets in this way to your existing infrastructure can be useful, for example, if you have a live website and don’t want to make any potentially breaking changes to that website while working on it. Instead you can add one more Droplet to use as a development environment and work on your project in the same environment as the production Droplet, without any of the potential risk.

Now open digitalocean_droplet.tf to add the rules for your new Droplets:

nano digitalocean_droplet.tf

Add the following lines to your file:

resource "digitalocean_droplet" "do_droplet" {
    image   = "ubuntu-16-04-x64"
    name    = "testing-terraform"
    region  = "fra1"
    size    = "s-1vcpu-1gb"
    tags    = ["terraform-testing"]
    count   = "1"
}

resource "digitalocean_droplet" "do_droplet_new" {
    image   = "ubuntu-18-04-x64"
    name    = "testing-terraform-${count.index}"
    region  = "fra1"
    size    = "s-1vcpu-1gb"
    tags    = ["terraform-testing"]
    count   = "2"
}

You use the count meta-argument to tell Terraform how many Droplets with the same specifications you want. These new Droplets will also be added to your existing firewall as you specify the same tag as per your firewall.

Apply these rules to check the changes you’re specifying in digitalocean_droplet.tf:

terraform plan -var "do_token=$DO_TOKEN"

Verify that the changes you want to make are replicated in the output of this command.

You’ll see output similar to the following:

Output
. . .
# digitalocean_droplet.do_droplet_new[1] will be created
  + resource "digitalocean_droplet" "do_droplet_new" {
      + backups              = false
      + created_at           = (known after apply)
      + disk                 = (known after apply)
      + id                   = (known after apply)
      + image                = "ubuntu-18-04-x64"
      + ipv4_address         = (known after apply)
      + ipv4_address_private = (known after apply)
      + ipv6                 = false
      + ipv6_address         = (known after apply)
      + ipv6_address_private = (known after apply)
      + locked               = (known after apply)
      + memory               = (known after apply)
      + monitoring           = false
      + name                 = "testing-terraform-1"
      + price_hourly         = (known after apply)
      + price_monthly        = (known after apply)
      + private_networking   = true
      + region               = "fra1"
      + resize_disk          = true
      + size                 = "s-1vcpu-1gb"
      + status               = (known after apply)
      + tags                 = [
          + "terraform-testing",
        ]
      + urn                  = (known after apply)
      + vcpus                = (known after apply)
      + volume_ids           = (known after apply)
    }

Plan: 2 to add, 1 to change, 0 to destroy.

Once you’re satisfied with the output, use the terraform apply command to apply the changes you’ve specified to the state of the configuration:

terraform apply -var "do_token=$DO_TOKEN"

Confirm the changes by entering yes on the command line. After successful execution, you’ll see output similar to the following:

Output
. . .
digitalocean_droplet.do_droplet_new[1]: Creating...
digitalocean_droplet.do_droplet_new[0]: Creating...
digitalocean_firewall.do_firewall[0]: Modifying... [id=FIREWALL-ID]
digitalocean_firewall.do_firewall[0]: Modifications complete after 1s [id=FIREWALL-ID]
digitalocean_droplet.do_droplet_new[0]: Still creating... [10s elapsed]
digitalocean_droplet.do_droplet_new[1]: Still creating... [10s elapsed]
digitalocean_droplet.do_droplet_new[0]: Creation complete after 16s [id=DROPLET-ID]
digitalocean_droplet.do_droplet_new[1]: Still creating... [20s elapsed]
digitalocean_droplet.do_droplet_new[1]: Creation complete after 22s [id=DROPLET-ID]

Apply complete! Resources: 2 added, 1 changed, 0 destroyed.

You’ll see two new Droplets in your DigitalOcean web panel:

You’ll also see them attached to your existing firewall:

You’ve created new assets with Terraform using your existing assets. To learn how to destroy these assets you can optionally complete the next step.

Step 5 — Destroying Imported and Created Assets (Optional)

In this step, you’ll destroy assets that you’ve imported and created by adjusting the configuration.

Begin by opening digitalocean_droplet.tf:

nano digitalocean_droplet.tf

In the file, set the count to 0 as per the following:

resource "digitalocean_droplet" "do_droplet" {
    image   = "ubuntu-16-04-x64"
    name    = "testing-terraform"
    region  = "fra1"
    size    = "s-1vcpu-1gb"
    tags    = ["terraform-testing"]
    count   = "0"
}

resource "digitalocean_droplet" "do_droplet_new" {
    image   = "ubuntu-18-04-x64"
    name    = "testing-terraform-${count.index}"
    region  = "fra1"
    size    = "s-1vcpu-1gb"
    tags    = ["terraform-testing"]
    count   = "0"
}

Save and exit the file.

Open your firewall configuration file to alter the count as well:

nano digitalocean_firewall.tf

Set the count to 0 like the following highlighted line:

resource "digitalocean_firewall" "do_firewall" {
  name  = "testing-terraform-firewall"
  tags  = ["terraform-testing"]
  count = "0"

  inbound_rule {
      protocol                = "tcp"
      port_range              = "22"
      source_addresses        = ["0.0.0.0/0", "::/0"]
    }
  inbound_rule {
      protocol                = "tcp"
      port_range              = "80"
      source_addresses        = ["0.0.0.0/0", "::/0"]
    }
  inbound_rule {
      protocol                = "tcp"
      port_range              = "443"
      source_addresses        = ["0.0.0.0/0", "::/0"]
    }

  outbound_rule {
      protocol                = "tcp"
      port_range              = "all"
      destination_addresses   = ["0.0.0.0/0", "::/0"]
    }
  outbound_rule {
      protocol                = "udp"
      port_range              = "all"
      destination_addresses   = ["0.0.0.0/0", "::/0"]
    }
  outbound_rule {
      protocol                = "icmp"
      destination_addresses   = ["0.0.0.0/0", "::/0"]
    }
}

Save and exit the file.

Now apply those changes with the following command:

terraform apply -var "do_token=${DO_TOKEN}"

Terraform will ask you to confirm if you wish to destroy the Droplets and firewall. This will destroy all assets you imported and created via Terraform, so ensure you verify that you wish to proceed before typing yes.

You’ll see output similar to:

Output
. . .
digitalocean_droplet.do_droplet[0]: Destroying... [id=YOUR-DROPLET-ID]]
digitalocean_droplet.do_droplet_new[0]: Destroying... [id=YOUR-DROPLET-ID]
digitalocean_droplet.do_droplet_new[1]: Destroying... [id=YOUR-DROPLET-ID]
digitalocean_firewall.do_firewall[0]: Destroying... [id=YOUR-FIREWALL-ID]
digitalocean_firewall.do_firewall[0]: Destruction complete after 1s
digitalocean_droplet.do_droplet_new[1]: Still destroying... [id=YOUR-DROPLET-ID, 10s elapsed]
digitalocean_droplet.do_droplet[0]: Still destroying... [id=YOUR-DROPLET-ID, 10s elapsed]
digitalocean_droplet.do_droplet_new[0]: Still destroying... [id=YOUR-DROPLET-ID, 10s elapsed]
digitalocean_droplet.do_droplet_new[1]: Still destroying... [id=YOUR-DROPLET-ID, 20s elapsed]
digitalocean_droplet.do_droplet_new[0]: Still destroying... [id=YOUR-DROPLET-ID, 20s elapsed]
digitalocean_droplet.do_droplet[0]: Still destroying... [id=YOUR-DROPLET-ID, 20s elapsed]
digitalocean_droplet.do_droplet_new[1]: Destruction complete after 22s
digitalocean_droplet.do_droplet[0]: Destruction complete after 22s
digitalocean_droplet.do_droplet_new[0]: Destruction complete after 22s

Apply complete! Resources: 0 added, 0 changed, 4 destroyed.

You’ve deleted all assets managed by Terraform. This is a useful workflow if you no longer need an asset or are scaling down.

Conclusion

In this tutorial you installed Terraform, imported existing assets, created new assets, and optionally destroyed those assets. You can scale this workflow to a larger project, such as deploying a production-ready Kubernetes cluster. Using Terraform you could manage all of the nodes, DNS entries, firewalls, storage, and other assets, as well as use version control to track changes and collaborate with a team.

To explore further features of Terraform read their documentation. You can also read DigitalOcean’s Terraform content for further tutorials and Q&A.