How to Manage Cloud Services with Terraform – The New Stack



















Get, Post or Go Home?

Should GET and POST be the only HTTP request methods used by developers?

Yes, GET and POST are the only verbs needed.

0%

No, DELETE, PATCH and other requests have their place.

0%

GraphQL.

0%

2023-11-20 09:34:30

How to Manage Cloud Services with Terraform

sponsor-gcore,sponsored-post-contributed,

Walk through an example to explore Terraform’s IaC benefits, including multicloud support, state management and deployment previews.


Nov 20th, 2023 9:34am by


Featued image for: How to Manage Cloud Services with Terraform

Featured image by Jerry Zhang on Unsplash.

Terraform is a popular tool for managing cloud infrastructure through code. Since it’s open source and not tied to a particular cloud provider, it has a huge community in the DevOps space. Globally, over 60,000 companies are using Terraform today. This article explores the benefits of using Terraform to manage cloud services, looks at some usage examples and compares Terraform to alternative Infrastructure as Code (IaC) tools.

What Is Terraform?

Terraform is an open source, IaC tool created and maintained by HashiCorp, a software company that creates provider-neutral cloud tools. It provides a consistent command line interface (CLI) and definition language to manage cloud infrastructure. Terraform’s IaC approach means you create and configure cloud resources by writing code and, just like application code, you can version, test and deploy it. To learn more about using Terraform, check out this tutorial.

Terraform is provider-independent, which bolsters its consistency. Whether you’re using AWS, Microsoft Azure, Gcore or on-premises infrastructure, with Terraform you can manage all your infrastructure in a consistent manner. This unified approach reduces complexity, enhances productivity and lowers the risk of human error across your entire operation.

Benefits of Managing Cloud Services with Terraform

Terraform offers a unified interface for managing multicloud and on-premises infrastructure, providing features like state tracking, change previews and idempotent deployments, while also enabling community collaboration, supporting seamless integration with existing development tools and reducing the risk of vendor lock-in.

Deploy Cloud Services across Multiple Providers

Terraform lets you manage cloud services across public cloud providers and on-premises environments with a single tool. The same familiar interface lets you choose the right provider for your workload.

Some public cloud providers, like AWS, have IaC tools that work only with their cloud services. Such tools are optimized for their infrastructure but can increase the risk of vendor lock-in, which could defeat the point of using Terraform.

Manage Cloud Services in One Place

Since Terraform tracks the status of your cloud services, there’s no need to check in multiple places to find out what’s deployed and where. This is helpful for managing services within a single cloud ecosystem, but it really shines when orchestrating services across multiple cloud providers. With one click, you see which cloud services are online and what changes may be necessary.

For example, you might notice that encryption isn’t enabled on some of your databases on Azure while it is on AWS. Consequently, you can change your Azure database security settings to match AWS. Or you might see your cloud storage costs are spiking and decide to migrate some data to another service to reduce costs.

Preview Changes before Deploying

Terraform’s execution plans show what changes will be applied before they are executed. This helps avoid changes that could trigger an accidental removal and redeployment of a cloud service.

Terraform, like many other IaC tools, is a declarative system. You define your desired end state, and Terraform calculates the changes required to reach it. In theory, this process is limited by services that only allow the definition of a config value at deployment time. Terraform solves the issue by removing and redeploying such services with new config values. While this can risk data loss or accidental outages, Terraform’s preview feature warns you if that’s a possible outcome before the deployment, allowing you to reconsider the action.

Repeat Deployments with Identical Outcomes

When you redeploy a definition file, either after deleting the deployment or to run the same cloud service on another account or provider, the resulting infrastructure remains the same. This is because Terraform’s declarative approach allows only idempotent definitions. You define the end state, and Terraform reaches it after each deployment, even if it fails midway and you have to redeploy multiple times.

Leverage Community-Maintained Service Definitions

Terraform’s popularity means there’s a huge community creating and maintaining definition files for common cloud services and workloads. Like reusing application software libraries and frameworks, you can download cloud-service definitions and augment them with your values and code. This allows you to spin up your infrastructure fast without reinventing the wheel.

Popular definitions benefit from reviews by multiple maintainers. You can benefit from others’ work upfront, with best practices already baked in.

Integrate with Existing Code Tools

Another advantage of Terraform’s IaC approach is that all definitions are just code files, meaning you can integrate them with your existing application development tools. You can put the files into your chosen version control system. As Terraform is solely a command-line interface (CLI) tool, you can run it with a CI/CD pipeline to automate deployments.

Example Terraform Deployment

I’ll deploy two cloud services with Terraform on Gcore to illustrate its benefits. I’ll start with a virtual instance and then test it with Terraform’s new built-in test feature.

The following code belongs to one HCL (HashiCorp Configuration Language) file, but I will split it up and provide explanations to add context. The code connects to the Gcore Cloud API, fetches region information and tells the API to deploy the infrastructure required to run a web server in a custom network on Gcore.

Examine the Terraform Definition

Create a new directory with a main.ts inside. Then, add the following code snippets to the file.

First, the terraform block tells Terraform where it can download the gcore provider. As it is a custom provider supported by Gcore and not HashiCorp, you can’t find it in Terraform’s default location; you need to tell Terraform about it explicitly.

This illustrates the first benefits: community-provided definitions and cloud-provider independence.

A variable block saves data you either want to use in multiple locations or to supply from outside the definition, whether through a command-line argument (e.g., --var <VAR_NAME>="<VAR_VALUE>") or a Terraform test. Variables let you change a value in multiple locations by editing it in one place. By contrast, if you use a graphical user interface (GUI) for the deployment, you must set these values manually for every resource at creation time.

Next up is a provider block. Set up the access token to connect to Gcore’s Cloud API. You can have multiple blocks in one file to deploy one application across multiple clouds or accounts. This is where you use the first variable to insert the Gcore API token.

The data block tells Terraform to request information from the cloud API before deploying the cloud services. Here, it has to get the data for Gcore’s Luxembourg region so you can ensure that Terraform deploys every resource in the same region.

Now, let’s turn to the infrastructure. The resource blocks define the cloud services that Terraform will deploy. They start with the resource type; they are always prefixed with the provider’s name, so you can have, for instance, a “gcore_network” and a “vsphere_network” in one file. Notice that it uses the gcore_project_id variable and gcore_region data source.

The first three services (gcore_network, gcore_subnet and demo-sg) define a private network in the Gcore Cloud that allows traffic to get in and out only through TCP port 80.

Finally, the resource blocks define the virtual instance. The gcore_volume defines a persistent block storage. With the image_id, you can select prebuilt or custom hard drive images, so you don’t have to install an operating system from scratch.

Volumes are independent of instances, so you can reuse them when you change the instance type. Sometimes, a change leads the cloud API to destroy the old cloud service and create a new one with the updated definition. Separation of volumes and instances means you can be sure that the volume will retain its data when Terraform replaces its original instance.

In the gcore_instance block, the properties of all resources come together. The volume attribute links the volume defined in the previous block, and the interface attribute links all the network services to the instance.

In the user_data attribute, Terraform allows the addition of scripts that execute when the instance launches. You can leverage generic images from the cloud provider or the community and augment them with custom packages, such as an Nginx web server.

You can version the code in Git and reuse it for multiple deployments without starting from scratch every time.

Initialize a Terraform Project

Run the following CLI command in the same directory as this file, and Terraform will download the G-Core/gcore Terraform provider and set it up for deployment:

Test the Terraform Definition

The new test command in version 1.6 enables you to test the definition before deploying it. Create a file at tests/deployment.tftest.hcl with:

To execute the tests, run:

The output should look like this:

Deploy the Terraform Project

To experience the preview feature, run:

The command’s output will display the changes that Terraform is set to make in the deployment:

This shows all attributes you defined, plus those with default values or supplied by Gcore’s Cloud API after deployment (e.g., the region_id Terraform got from the data source.) Terraform uses this information for state management.

Resource action symbols make the preview feature powerful. Since this is a fresh deployment, there are numerous + symbols indicating the actions Terraform will take when deploying, meaning that Terraform will create something new in that place. The - symbol indicates Terraform will destroy a service, and the -/+ symbol shows that it will replace a service, which means destroying and creating in the same deployment. Thus, you can check whether Terraform will destroy a service with important data or one that takes a long time to deploy. The ~ symbol indicates an in-place update, and the <= symbol means it’s reading data, so they are usually fast and safe actions.

Run the apply command to deploy the definition:

How Terraform Compares to Other IaC Tools

Terraform isn’t the only IaC tool on the market; popular competitors include AWS CloudFormation, Ansible and Chef.

Terraform vs. AWS CloudFormation

CloudFormation is the preferred IaC tool when working with AWS. AWS created it, maintains and uses it to deploy AWS services. Among IaC tools, it has the best AWS integration; however, it works only with AWS.

If you work with services outside of AWS, or with multiple cloud providers at once, Terraform is the better choice. Since Terraform is extensible with custom providers, Gcore and other cloud providers can leverage the complete Terraform ecosystem by creating Terraform provider definitions.

On the technical side, the major difference between Terraform and CloudFormation is the definition language. Terraform uses HCL, a declarative language with static typing. CloudFormation uses a declarative untyped domain-specific language on top of YAML and JSON, as JSON is part of the YAML specification. As such, it comes with common YAML issues like ambiguous syntax.

Terraform vs. Ansible and Chef

While they overlap in some areas, Ansible and Chef have different goals than Terraform. Their focus is application configuration, either by running an agent or by connecting remotely to an instance to install and update software. Terraform’s focus is infrastructure provisioning. It uses cloud APIs to deploy and update cloud services. Terraform is a better fit if you want to manage cloud services distributed over different accounts or cloud providers.

Terraform’s built-in state management and execution plans make it a solid tool to show what’s already deployed. It also lets you check which actions it will take before applying them. Getting this information from Ansible or Chef requires you to install additional tools or extensions.

Like CloudFormation, Ansible and Chef use a domain-specific language on top of YAML, while Terraform uses HCL, which is tailored for IaC.

Conclusion

Terraform is a flexible and powerful tool. Cloud providers and their customers can leverage a huge ecosystem with the support of a vibrant community. With Terraform, HCL files are your consistent interface to cloud providers worldwide. You can reuse your definitions to deploy multiple instances of the same stack while execution plans protect against accidental service downtime or data loss.

The Gcore instance example shows Terraform’s benefits, including multicloud support, state management and deployment previews. The Gcore Terraform provider is maintained by Gcore and verified by HashiCorp, so you can rest assured that the tool is tested to the highest standards.

Group Created with Sketch.