If you’re deploying public cloud infrastructure at scale in your organization, you’ve likely encountered challenges when addressing issues such as security, policy governance, drift control, and operational efficiency. How do you support infrastructure deployments for numerous decentralized teams while maintaining security and policy governance centrally and managing other risks? That’s where infrastructure-as-code tools and the DevOps framework come into play.
By 2025, Gartner analysts project more than 85% of organizations will embrace a cloud-first principle and will not be able to fully execute on their digital strategies without the use of cloud-native architectures and technologies. We’re at a point where centralized workload operation teams are having to rapidly adapt their security, governance, and operational controls (which took decades to mature) as they move to decentralized workload models leveraging public cloud providers.
This rapid shift can reintroduce business risks without the right toolbox. Deploying infrastructure as code (IaC) at scale using Terraform and the DevOps framework should become the common platform of tools, approaches, and best practices that cloud administrators use to automate and standardize cost, compliance, security, and operations controls across the distributed workload model.
Based on our experience helping organizations deploy IaC at scale, we present an overview of the infrastructure-as-code tools we recommend, and why.
Start with Native Terraform for Infrastructure-as-Code Tools
For many DevOps teams, using Terraform to deploy IaC seems simple and obvious. Native Terraform supports codified interaction with a vast array of hardware platforms, cloud providers, cloud services, and development products. The added plus is that it’s a declarative language and is meant to be easy to use.
Moreover, we see increasing market demand and trust in the tool. Not only are early adopter companies increasingly doubling down with Terraform for their IaC deployments, but public cloud providers and software companies are investing in it, too.
If you haven’t lived through an IaC-at-scale implementation, you may not know of the painful lessons that come with choosing the wrong products to pair with Terraform. You’ll need a suite of tools to enable the at-scale automation needed for operational and security governance. Be aware – choosing the wrong kinds of tools will restrict your ability to implement IaC at scale.
The Limitations of Terraform Accelerators and Wrappers
Increased market demand spurred the introduction of many Terraform accelerators and wrappers. These tools aim to make it easier for developers and cloud administrators to develop and execute Terraform within their continuous integration/continuous delivery (CI/CD) pipeline. You’re likely familiar with some of these tools, like Terragrunt and Pretf, which abstract Terraform code creation, and Terraform execution wrappers like Atlantis and RubyTerraform.
These products—available for free—help mid-market and enterprise customers with the potential for an accelerated IaC adoption ramp-up or lower IT transformation hurdles to climb. They help DRY (Don’t Repeat Yourself) out code, allow organizations to scale beyond Terraform open source, and help with teams larger than one to two people.
However, rarely do these tools used early in an IaC adopter’s journey meet the long-term needs of the organizations that adopt them. Accelerators, wrappers, and shortcuts may introduce concepts and technologies leveraged in IaC—like Terraform—but you may encounter issues when using them to support IaC-at-scale approaches. By abstracting access to native Terraform functions, these tools introduce unnecessary complexity and restrict scale.
Organizations that have adopted these tools have encountered issues such as the following:
- Lack of ability to scale well
- Organizations needing to decentralize workspace deployment and operations
- Siloed talent exposed to the wrapper technology can become a single point of failure or limit growth
- Increased risk to governance controls without centralized policy management
- Inability to manage sprawling security profiles across platforms and roles
- Excessive IaC drift at scale with the ability to manage centrally controlled state files and test configurations through native Terraform application programming interfaces (APIs)
- Lack of access to native Terraform APIs and features
We recommend pursuing native Terraform and tools that support it wherever possible. You’ll also want to invest in training for your teams to understand how to work with Terraform in a structured CI/CD pipeline.
Implement Terraform Automation and Collaboration Software (TACOs)
Unlike the accelerators and wrappers described above, TACOs, such as Terraform Cloud (TFC), Scalr, env0, and Spacelift, enable Infrastructure-as-Code at scale. They leverage Terraform and integrate with version control systems (VCS) and policy-as-code offerings common within an IaC-at-scale deployment.
The Benefits of TACOs
Well-implemented TACOs using Terraform help address the risks of public cloud adoption at scale. Those risks include security compliance, policy governance, drift control, and operational efficacy.
While the specific features of each TACO will differ in implementation and approach, they generally provide great benefits. Examples include:
- Enable a distributed security hierarchy to give a center of excellence (CoE) control over what developers can change within the environment and validate compliance policy enforcement
- Provide and integrate with CI/CD pipeline workflows
- Integrate a policy-as-code function, like OPA (Open Policy Agent or Sentinel), into the Terraform deploy workflow to enforce governance and security policies centrally
- Remediate Terraform state file and policy drift automatically or via a notification process—enabling policy or CoE owners to opportunistically engage the impacted developers
- Require minimal training or shadowing for team members to become effective contributors
- Integrate with and manage application workloads running Kubernetes and other docker-based container architectures
- Allow self-service for development/application teams to deploy resources via Terraform in an environment with proper guardrails
- Provide workspaces for deployments to reduce blast radius
The bottom line: make sure you have the right tool for the problem you need to solve. TACOs enable the automation necessary to achieve IaC-at-scale and provide a low bar for adoption—at least, assuming your organization made the commitment to IaC. TACOs provide the ability to manage hundreds or thousands of cloud services leveraging the same tools, approaches, and best practices achieved for IaC-at-scale.
So, what comes next after implementing TACOs?
Integrate TACOs with Existing Workflows
Embarking on an IaC-at-scale project gives you a chance to reevaluate best practices, approaches, and tools used in your existing infrastructure and application deployment workflows. TACOs increasingly integrate with:
- Messaging collaboration products like Microsoft Teams and Slack
- VCSs like GitHub, Gitlab, and Bitbucket
- Service desk ticket flows from Service Now, Salesforce, and BMC
- CI/CD workflow products like Jenkins, CircleCi, Azure DevOps, and AWS Code Pipeline
The de facto standard CI/CD tools can coexist with a Terraform workflow. However, they are not ideal to use as a Terraform execution or code management platform, so you may need to adapt your processes. In contrast, TACOs add the proper guardrails required for Terraform workflows, including state management, role-based access control (RBAC), policy management, observability, auditing, run locking, cost estimation, and private module registry.
Benefits of Integrating TACOs with Existing Workflows
Unlike in a typical environment, where an application developer must request infrastructure from their cloud or data center administrators to deploy an application, TACOs automate that process.
TACOs allow an organization to delegate the deployment of architectures to the application teams (while maintaining the controls necessary for security and compliance). You can build workflows that integrate directly or coexist with application CI/CD pipeline, which—once established—would automatically deploy the infrastructure and application resources. As actions occur within any CI/CD process, TACOs can send/receive messages and actions from message collaboration platforms and service desk products.
Integration benefits include:
- Minimizes process changes for organizations with mature application CI/CD pipelines
- Enables a single source of truth captured within an existing (or newly implemented) VCS
- Reduces implementation hurdles by supporting existing tools favored by your developer community
- Improves process velocity by enabling interaction with commonly used messaging collaboration products
- Promotes automation through extensible API and CLI connection points, providing options for where and how to best enable priority workflows
Integrating TACOs will provide a full GitOps experience, allowing your team to apply DevOps best practices to infrastructure automation.
Transform Your IT Organization with Infrastructure-as-Code Tools
Adopting a Terraform for IaC DevOps framework is not easy or quick, but it can be transformational. If your IT organization is embracing cloud-native approaches, make sure you have the right tools to scale. If you haven’t already, now is the time to start investigating how tools like TACOs and Open Policy Agent (OPA) can help you on your journey.
Not sure if your team is fully ready for IaC-at-Scale? Contact us to schedule a Terraform Health Check. And watch out for the next blog post in our series, where we’ll dive into the nitty gritty of IT transformation.
About Sierra-Cedar
Sierra-Cedar specializes in helping education, public sector, and enterprise clients operate complex applications and infrastructure in the public cloud, at scale. Sierra-Cedar is an AWS Advanced Consulting Partner, AWS Public Sector Partner, and reseller with AWS competency status in Migration, Education, and Oracle with APN Certification Distinction for achieving 100+ AWS Certifications.
To learn more about Sierra-Cedar’s DevOps Services practice, please visit https://www.sierra-cedar.com/devops-services/