Faster Delivery With Infrastructure as Code and Terraform

Nearly everyone in enterprise IT organizations has faced this challenge. It starts when a new business initiative requires new software features, tools, or technology procurement. These requirements accumulate and exceed the organization’s current capacity. Public cloud offerings promise to decrease time to market, but still require expertise and time to manage. In a complex system of manual or poorly documented processes, onboarding new developers or system administrators can take months. When faced with these obstacles, the potentially multimillion-dollar question becomes how to go to market with new features faster?

Infrastructure as Code (IaC) is part of the answer to this question. IaC helps remove the manual processes involved in deploying, configuring, and administering IT systems by enabling teams to automate these processes with code. There are many IaC products and tools available, but here we’ll discuss one popular contender—Terraform.

Causes of Long Lead Times

Unfortunately, it is all too common for development of a feature to be complete months before it is deployed to production. Terraform can address several of the key causes for these long lead times, including:

• Manual provisioning processes.

• Multiple handoffs and communication breakdowns between siloed teams.

• Lack of working non-production environments, either because the environments are in use by other teams or their configurations have drifted from production configurations to the point of being unusable.

• Slow approval processes that disconnect the development cycle from the actual changes being deployed.

How to Decrease Lead Times

Using Terraform can decrease lead times by facilitating fast feedback loops, automated deployments, and cross-functional teams.

1. Fast Feedback Loops

Fast feedback loops ease the development process and are key to iterating quickly. Compared to other IaC tools, Terraform excels at providing fast feedback loops. Terraform executes quickly and provides engineers with immediate feedback on the changes that will occur as a result of modified configuration. Terraform provides this feedback by outputting an execution “plan” that is not only useful for the individual Terraform developer but other teammates as well. System architects can also use the “plan” output to reduce time spent deciphering the impact of a change and spend more time providing feedback on code quality and system design.

2. Automated Deployments

Using IaC tools reduces deployment times by automating creation of environments of any scale, from a single server to multi-tiered suites of services. While fully automated deployments require orchestration across several tools, Terraform fulfills the role of provisioning the infrastructure of on-demand environments for load testing, feature testing, or QA. Long-lived non-production environments that slowly drift out of sync with production can become a problem of the past.

3. Cross-Functional Teams

Cross-functional teams consisting of both developers and operations engineers are critical to implementing successful DevOps practices that reduce time to market for new features. IaC integrates with software development and deployment pipelines and can even co-exist in the same repository as application code. This allows dev and ops engineers to share toolsets and processes, which improves visibility and collaboration on changes. Using common toolsets and processes also reduces delays and miscommunications stemming from multiple handoffs between separate teams.

Simplify Management of Multiple Platforms and Services

Modern enterprise architectures are complex systems composed of moving parts across a variety of platforms and services. For example, you might deploy your applications to servers running in AWS, host DNS in Cloudflare, and control inbound traffic using firewall rules on an F5 appliance. Changes to any of these pieces need to be synchronized for the application to remain functional, and each piece that is changed manually slows the process and increases the risk of costly downtime.

One goal of Terraform is to provide a single interface and deployment process for managing all of these platforms and services. Terraform can manage AWS, Azure, and GCP cloud resources as well as traditional or on-premises systems such as Palo Alto, F5, and VMWare. The advantages of this unified interface include:

• Lowering the chance of errors from manual changes.

• Reducing handoffs to mitigate communication failures.

• Allowing changes across many dependent systems to be rolled out in a coordinated fashion.

• Reducing training time and engineering complexity associated with maintaining multiple provisioning systems for different platforms (i.e., no more switching between Azure ARM Templates, AWS CloudFormation, GCP Deployment Manager templates, Ansible, etc.).

Terraform Workflow

Terraform provides this cross-platform functionality without abstracting the core functionality of each service. Terraform is not cloud or platform agnostic as some cloud management tools try to be. Tools that try to be platform agnostic often only support the “lowest common denominator” of functionality, thereby blocking the use of platform-specific features that can reduce costs, improve security, and increase developer productivity.

All of the above does come with certain restrictions. Each provider is managed separately and support for each platform varies. It is important to research each Terraform provider to determine if it will cover your use case before investing in Terraform. Fortunately, the providers for the main public clouds (Azure, AWS, and GCP) are well-maintained by HashiCorp, which collaborates directly with each cloud organization. Another silver lining is that all public providers are open source, and your team can submit contributions to a provider if it does not support functionality critical to your use case.

Change Management

Another challenge in dealing with infrastructure at scale is ensuring that initial deployment and subsequent changes are predictable and repeatable. Does the resulting state match what was planned? Were mistakes made along the way for some resources but not others? We can manage certain configuration guardrails with provider policy tools like AWS Config or Azure Policy—especially regulatory and compliance requirements—but when it comes to predictable and repeatable changes, IaC has a lot more to offer.

Terraform enables us to separate changes into planning and execution (or “apply”) stages so we know exactly what will happen when a configuration change is deployed. This reduces time spent assessing change impact and remediating configuration errors. Some platform-specific tools like AWS CloudFormation offer these features, but others like Azure Resource Manager (ARM) Templates (at the time of this writing) do not allow us to view the before-and-after state of a configuration change. For example, when enabling versioning on AWS S3 bucket storage, Terraform displays the following output:

With this output we can see exactly what properties will be changed, how many resources will be affected, and whether it causes any resource to be recreated (thereby disrupting service uptime). We can include this plan output in our change record to facilitate discussions and approvals prior to its execution. Ultimately, having an accurate preview of changes helps reduce the risk of deployments.

While we can audit the state of traditionally managed infrastructure without IaC, IaC allows configuration to be stored in version control systems (VCS) like GitHub, Bitbucket, and Azure Repos, where any changes to the infrastructure code can be reviewed and audited using the same processes as application code. When we integrate Terraform with a VCS, we enable the automation of the speculative plan step (shown above) as part of an existing CI/CD pipeline or via Terraform Cloud. Therefore, the infrastructure code is automatically validated during the review process. By integrating infrastructure changes with the normal code review process, we enable faster feedback to our teams on what effects these changes will have. Taking this a step further, we can use code approvals to stage changes for deployment and require additional approvals to release changes into production. These processes further reduce time to market without sacrificing change control.

Terraform Cloud

Terraform Cloud is a service that adds several governance and collaboration features to Terraform open source that can reduce the barriers to adopting Terraform. Terraform Cloud is available as a hosted SaaS product or as a self-hosted installation known as Terraform Enterprise.

Sentinel Policy

Sentinel Policy is a tool within Terraform Cloud that allows organizations to evaluate and enforce policies during the development process—prior to deployment—through policy-as-code. This supports faster feedback to teams when their configuration violates internal standards for tagging, location, cost allocation, etc., and can prevent noncompliant resources from being created. We see this as complementary to other provider policy tools such as Azure Policy or AWS Config.

Self-Service

As you scale your organization’s Infrastructure as Code capabilities, you must also scale the investment in maintaining and enhancing that code. If every deployment requires involvement from the core IaC engineering team, then it becomes difficult to find time to add and improve automation processes. Enabling self-service for consumers of infrastructure alleviates this problem and is one of the final steps in adopting IaC tools. Terraform Cloud provides a few tools to help self-service workflows:

• Private Module Registry hosts re-usable configurations that adhere to organizational standards and deployment patterns.

• Configuration Designer allows developers who are less familiar with Terraform to consume published modules via a visual interface.

• The Service Catalog integration for ServiceNow allows service requests to feed directly into Terraform Cloud provisioning workflows (see Announcing ServiceNow Integration for Terraform Enterprise for more detail).

Pre-Built CI/CD

Terraform Cloud includes out-of-the-box CI/CD capabilities tailored for Terraform deployment workflows either locally or integrated with VCS. It is possible to build out similar capabilities on other CI/CD platforms like Jenkins, Travis CI, or Azure DevOps, but Terraform Cloud can get you up and running with Terraform faster by packaging together the core Terraform workflow and many automation best practices. If customizations are necessary down the road, you can also integrate with the Terraform Cloud API using a more general-purpose CI system while still leveraging Terraform Cloud’s other features.

Go to Market Faster and More Reliably

Infrastructure as Code is as an essential tool in the DevOps toolchain that enables teams to go to market faster and more reliably. We’ve discussed the advantages of Terraform as one of the leading IaC tools and have seen how it can accelerate deployments while reducing risk.

Do you have questions about how DevOps tools and practices can improve technology delivery within your organization? Contact us to find out more.

• Software Development
• Software
• Terraform
• IaC