Today we are announcing the general availability of Clustering for HashiCorp Terraform Enterprise. Increasingly, organizations are adopting Terraform Enterprise as their standard provisioning platform and the new Clustering functionality enables them to easily install and manage a scalable cluster that can meet their performance and availability requirements. Clustering supports installations of Terraform Enterprise in AWS, Azure, and GCP.
This blog will discuss the clustering capability in Terraform Enterprise, including:
- Scaling and availability improvements
- Installation using Terraform itself
- Migrating to a cluster
A Horizontally Scalable, Redundant Cluster
Until today, Terraform Enterprise followed the industry standard of a single instance appliance. This was acceptable at first, but as organizations expand their usage of Terraform Enterprise and it becomes their central provisioning platform across business units, the requirements around performance and availability increase. Organizations expect to be able to:
- Scale Terraform Enterprise horizontally to keep up with ever-growing workloads
- Trust Terraform Enterprise will always be available to provision their most critical infrastructure
The new Clustering functionality in Terraform Enterprise meets both of these needs.
Scale to Meet Demand
As organizations look to spread the benefits of Terraform Enterprise across their businesses, performance becomes a top concern. Even with the largest instances available, end-users can experience sluggishness in the UI and long wait times for runs to complete.
Running Terraform Enterprise as a cluster solves these performance issues by spreading the workload across multiple nodes. All API requests, background processes, and Terraform runs are fanned out across the cluster, allowing any number of concurrent end-users and runs to be supported.
In fact, our default installation pattern now includes configuration options to enable autoscaling groups. Giving organizations the ability to respond to spikes in demand without human intervention.
Better Availability
Terraform Enterprise is responsible for the infrastructure behind some of the world’s most critical applications. Before today, organizations relied on a pair of instances with one acting as a passive standby to make sure Terraform Enterprise was always available.
While sufficient for some, the risk was too high for many organizations. The existing strategy required too much downtime to promote the passive instance, and then there was always the question of what would happen if both instances were lost.
Running Terraform Enterprise as a cluster mitigates these risks. The provided installer provisions nodes across availability zones by default, and running Terraform Enterprise across three or more nodes ensures zero downtime for most common failure events.
Easier to Install
Installation is now as simple as a terraform apply from an operator’s machine. Instead of configuring a single instance from the command line, operators can now use an official module from the Terraform Module Registry to deploy an entire cluster to AWS, Azure, or GCP.
That same Terraform config can then be edited later to reconfigure the cluster. For example, increasing the maximum size of Azure Scaling Set before rolling Terraform Enterprise out to a new business unit.
Please see the documentation or watch how simple it is on Azure in the demo below:
The journey begins and ends with infrastructure as code, just as it should.
Custom Installations
We know our recommended architectures cannot work for everyone; so, customers are welcome to fork our modules to bend them to their needs.
The underlying scripts that power our modules are also available and documented to make manual installations possible. This allows a bespoke cluster to be created specifically to an organization’s internal requirements, including inside private datacenters running VMware.
Migration
While we don’t recommend trying to upgrade an existing installation in place, migrating your existing data to a new cluster is simple when using external services:
- Spin up a new cluster without configuring the data tier
- Shut down your existing instance
- Enter the database and blob storage credentials into your cluster’s dashboard
- Reboot the cluster
Now you can login to your new cluster and should look just like it did before.
Note: As of today, mounted disk is not an option and external services are required to run a production cluster. Removing this requirement is the team’s top priority.
What’s Next?
This GA release is not the last you will hear of Clustering improvements. The Terraform Enterprise team still has many plans for the platform, including:
- HA without External Services for customers currently using a mounted disk
- New snapshot tooling for a simpler backup and restore experience
- Forwarding logs via syslog to a centralized platform such as Splunk
Getting Started
We are excited to release the GA version of clustering for Terraform Enterprise to help organizations with scalable performance, high availability, and simple install and management of the application. To learn more about clustering in Terraform Enterprise refer to the documentation here.
For more information on Terraform Enterprise or to get started with your free trial, visit the Terraform product page.
Originally posted on Hashicorp Blog
Author: Evan Phoenix