Deploy a Kubernetes cluster and application

Terraform Stacks allow you to automatically manage dependencies within complex infrastructure deployments. In addition to explicit orchestration rules, a Stack recognizes when a component requires attributes that are not yet available, and HCP Terraform defers those changes until it can apply them. For example, when managing Kubernetes workloads that use custom resources, you cannot provision both the custom resource definition (CRD) APIs and resources that use those APIs in a single step, because the Terraform provider relies on the existence of the APIs to create and apply your plan. In these scenarios, you must first deploy your CRD, and then deploy the resources that use it in a second plan and apply, usually in a separate workspace.

When you deploy a Stack that includes resources that depend on the availability of APIs provisioned by other components in your Stack, HCP Terraform recognizes the dependency between components, and automatically defers the plan and apply steps for your components until they can complete successfully.

In this tutorial, you will use an HCP Terraform Stack to deploy a Kubernetes cluster with an example service. When you deploy your Stack, HCP Terraform will automatically detect that the service you are deploying on your Kubernetes cluster needs attributes that aren't available during the initial plan. HCP Terraform will defer those changes until the custom resource definition is created and the API becomes available. Then, HCP Terraform will load the attribute values and proceed with the plan and apply steps for your service.

Prerequisites

This tutorial assumes that you are familiar with the Terraform and HCP Terraform workflows. If you are new to Terraform, complete the Get Started collection first. If you are new to HCP Terraform, complete the HCP Terraform Get Started tutorials first.

In order to complete this tutorial, you will need the following:

• An AWS account.
• An HCP Terraform account.
• An HCP Terraform variable set configured with your AWS credentials.

Create example repository

Navigate to the template repository for this tutorial. Click the Use this template button and select Create a new repository. Choose a GitHub account to create the repository in and name the new repository learn-terraform-stacks-eks-deferred. Leave the rest of the settings at their default values.

Clone your example repository, replacing USER with your own GitHub username.

$ git clone https://github.com/USER/learn-terraform-stacks-eks-deferred.git

Change to the repository directory.

$ cd learn-terraform-stacks-eks-deferred

Review components and deployment

Explore the example configuration to review how this Terraform Stack's configuration is organized.

$ tree
.
├── CODEOWNERS
├── LICENSE
├── README.md
├── cluster
│   ├── iam.tf
│   ├── main.tf
│   ├── outputs.tf
│   ├── variables.tf
│   └── vpc.tf
├── components.tfstack.hcl
├── copywrite.hcl
├── deployments.tfdeploy.hcl
├── kube
│   ├── crd.tf
│   └── kube.tf
├── providers.tfstack.hcl
└── variables.tfstack.hcl

The example repository contains two directories containing Terrraform modules to define the Stack’s components, kube, and cluster. It also contains .tfstack.hcl files to define your Stack, and a deployments.tfdeploy.hcl to define your Stack’s deployment.

Review components

Open the providers.tfstack.hcl file. This file contains the provider configuration for your Stack.

provider "aws" "main" {
  config {
    region = var.region

    assume_role_with_web_identity {
    role_arn        = var.role_arn
    web_identity_token = var.identity_token
    }

    default_tags {
    tags = var.default_tags
    }
  }
}

provider "kubernetes" "main" {
  config {
    host                = component.cluster.cluster_url
    cluster_ca_certificate = component.cluster.cluster_ca
    token               = component.cluster.cluster_token
  }
}

provider "random" "main" {}

This configuration file defines provider blocks for AWS and Kubernetes. The AWS provider block sets the role ARN and web identity token that your AWS provider will use to authenticate with AWS. Each deployment can configure a different role ARN for its instance of the Stack.

The Kubernetes provider configuration uses output values from the cluster component, which provisions your Kubernetes cluster on AWS. Since HCP Terraform will not know about these values until after it plans and applies cluster component’s configuration, HCP Terraform automatically defers initialization of your Kubernetes provider until after those values become available. Then, it plans and applies the configuration that relies on the Kubernetes provider. This process is similar to the way Terraform automatically determines dependencies manages dependencies within a given Stack even across multiple plan/apply steps. between resources and provisions them in the correct order. Stacks, however,

Next, review the components.tfstack.hcl file. This file contains all of the components for your Stack. Like Terraform configuration, you can organize your Stacks configuration into multiple files without affecting the resulting infrastructure.

component "cluster" {
  source = "./cluster"

  providers = {
    aws = provider.aws.main
    random = provider.random.main
  }

  inputs = {
    cluster_name    = var.cluster_name
    kubernetes_version = var.kubernetes_version
    region = var.region
  }
}

component "kube" {
  source = "./kube"

  providers = {
    kubernetes = provider.kubernetes.main
  }
}

This file includes configuration for two components, one for the EKS cluster itself, and another for the example API for this tutorial.

The configuration in the cluster directory defines an EKS cluster

Review deployment

identity_token "aws" {
  audience = ["aws.workload.identity"]
}

deployment "development" {
  inputs = {
    cluster_name       = "stacks-demo"
    kubernetes_version = "1.30"
    region             = "us-east-1"
    role_arn           = "<YOUR_ROLE_ARN>"
    identity_token     = identity_token.aws.jwt
    default_tags       = { stacks-preview-example = "eks-deferred-stack" }
  }
}

This Stack includes a single deployment of your Kubernetes cluster on AWS.

Replace the <YOUR_ROLE_ARN> for both deployments with the role_arn output value from your identity token workspace.

Commit the role ARN to your git repository.

$ git add deployments.tfdeploy.hcl && git commit -m "Configure role ARN"

Push the change to Github.

$ git push

Fork identity token repository

Your Stack uses an identity token to authenticate the AWS provider in each region. To do so, you will establish a trust relationship between your AWS account and HCP Terraform, and an AWS role with the appropriate permissions to create and manage your Stack.

Navigate to the template repository for the identity token. Click the Use this template button and select Create a new repository. Choose a GitHub account to create the repository in and name the new repository learn-terraform-stacks-identity-tokens. Leave the rest of the settings at their default values.

Create a project

Create an HCP Terraform project for your identity token workspace and Kubernetes Stack.

To do so, first log in to HCP Terraform, and select the organization you wish to use for this tutorial.

Next, ensure that Stacks is enabled for your organization by navigating to Settings > General. Ensure that the box next to Stacks is checked, and click the Update organization` button.

Then navigate to Projects, click the + New Project button, name your project Learn Terraform Stacks deferred actions, and click the Create button to create it.

Next, ensure that your AWS credentials variable set is configured for your project. Navigate back to < Projects, then to your organization’s **Settings

Variable sets page, and select your AWS credentials variable set. Under Variable set scope, select Apply to specific projects and workspaces, and add your Learn Terraform Stacks deferred actions project to the list under Apply to projects. Scroll to the bottom of the page and click the Save variable set** button to apply it to your new project.

Provision and set identity token

Next, provision the AWS identity token and role that HCP Terraform will use to authenticate with AWS when it deploys your Stack.

Navigate to Projects and select your Learn Terraform Stacks deployments project. Create a workspace to provision your project's identity tokens:

1. On the project overview page, select New > Workspace.
2. On the next screen, select Version Control Workflow and select your GitHub account.
3. On the Choose a repository page, select the learn-terraform-stacks-identity-tokens repository you created in the previous step.
4. On the next page, select Advanced options and enter aws/ in the Terraform Working Directory field.
5. Scroll to the bottom of the page and click the Create button to create your identity token workspace.

Once you create the workspace, HCP Terraform will load the configuration for your identity token. This process may take a few seconds to complete. Once it does, HCP Terraform will prompt you to enter values for your organization and project. Enter your organization name and the project name for this tutorial, Learn Terraform Stacks deferred actions and click the Save variables button.

Next, click the Start new plan button, and then Sart, to plan your changes. HCP Terraform will plan your changes, and prompt you to apply them. Once the plan is complete, click the Confirm and apply button to create your OpenID provider, policy, and role.

Make a copy of the role_arn output value. You will use it in the next step.

Set role ARN

In your Stack’s configuration on your local machine, replace the <YOUR_ROLE_ARN> your deployment with the role_arn output value from your identity token workspace.

deployment "development" {
  inputs = {
    cluster_name        = "stacks-demo"
    kubernetes_version  = "1.30"
    region              = "us-east-1"
    role_arn            = "<YOUR_ROLE_ARN>"
    identity_token      = identity_token.aws.jwt
    default_tags        = { stacks-preview-example = "eks-deferred-stack" }
  }
}

Commit the role ARN to your git repository.

$ git add deployments.tfdeploy.hcl && git commit -m "Configure role ARN"

Push the change to Github

$ git push

Create Stack

Return to your project by navigating to Projects and selecting your Learn Terraform Stacks deferred actions project. Select Stacks from the left nav and click + New Stack.

On the Connect to VCS page, select your GitHub account. Then, choose the repository you created for this tutorial, learn-terraform-stacks-eks-deferred. On the next page, leave your Stack name the same as your repository name, and click Create Stack to create it.

HCP Terraform will load your Stack’s configuration from your VCS repository.

Provision cluster and example services

Once HCP Terraform loads your configuration, it will plan your changes. Select your development deployment to review the plan status. HCP Terraform will notify you that the plan includes deferred changes. You can review the plan to identify which parts of your Stack are parts of the deferred changes. Click Approve plan to apply your changes. Once the plan is complete, you can review the resources that HCP Terraform will create, and apply the plan.

As it provisions your infrastructure, HCP Terraform will automatically detect that the API you are deploying on your Kubernetes cluster is waiting on the other Kubernetes components to deploy successfully, and will defer those changes until the components are available. Then it will load the attribute values and proceed with the plan.

Destroy infrastructure

Now you have used a Terraform Stack to deploy a Kubernetes cluster with an API. HCP Terraform was able to deploy the entire Stack as a single unit, despite the dependency between the API and the Kubernetes cluster.

Before finishing this tutorial, destroy your infrastructure.

Navigate to your Stack's Deployments page, and select the development deployment. Navigate to the Destruction and Deletion page and click the Create destroy plan button to create a destroy plan. Once the destroy plan is complete, approve it to remove your resources.

Then, remove your Stack by navigating back to your Stack and selecting your Stack's Destruction and Deletion page, and clicking the Force delete Stack learn-terraform-stacks-deploy button. Confirm the action, and click the Delete button. Finally, navigate to your learn-terraform-stacks-identity-token workspace’s Settings > Destruction and Deletion page, and follow the prompts to destroy your identity token infrastructure and delete the workspace. You can also remove your project by navigating to its settings page and following the steps to delete it.

Next steps

In addition to allowing you to manage complex deployments with hidden dependencies, Terraform Stacks include power orchestration and workflow features.

• Read the Terraform Stacks documentation for more details on Stacks features and workflow.
• Read the Terraform stacks, explained blog post.