Enforce OPA policies using HCP Terraform and Styra

As your organization grows and your infrastructure provisioning workflows mature, it gets harder to enforce best practices with socialization and bespoke tooling. Terraform offers multiple ways to validate that your infrastructure satisfies both industry best practices and organization-specific standards. By automating validation and enforcement, you can programmatically ensure compliance and consistency in your infrastructure.

Pre- and postconditions let you to write resource-level checks within your resource definitions. If you include them in your module definitions, you can ensure that downstream module users comply with configuration standards and use modules in the way the author intended.

HCP Terraform run tasks let you integrate third-party tools such as vulnerability scanners, cost calculators, and code scanners into your Terraform workflow. Run tasks send Terraform plan contents to the configured external tools between the plan and apply stages of a run. You can set your run tasks to apply your configuration or not based on the response returned by the tool.

Styra Declarative Authorization Service (DAS) provides agentless Open Policy Agent (OPA) deployments and a centralized control plane for managing your policies. It includes a policy builder and library of public policies to help you get started. As with Terraform and infrastructure-as-code, OPA allows you to use a single language for policy-as-code for different types of configurations and resources.

In this tutorial, you will use both Terraform preconditions and HCP Terraform run tasks to validate configuration and enforce organizational practices. First, you will use Terraform preconditions to prevent users from deploying bastion hosts that are too big, helping reduce your infrastructure costs. Then, you will integrate an HCP Terraform run task with a Styra OPA policy to prevent users from creating AWS security groups that allow public ingress. You will also define a custom OPA policy to only allow specific HCP Terraform users to deploy infrastructure changes on Fridays.

Prerequisites

This tutorial assumes that you are familiar with the Terraform and HCP Terraform workflows. If you are new to Terraform, complete the Get Started tutorials first. If you are new to HCP Terraform, complete the HCP Terraform Get Started tutorials first.

In order to complete this tutorial, you will need the following:

• Terraform v1.2+ installed locally.
• An AWS account.
• An HCP Terraform account and HCP Terraform locally authenticated.
• An HCP Terraform variable set configured with your AWS credentials.
• A Styra account.

Clone and review configuration

Clone the example repository for this tutorial.

$ git clone https://github.com/hashicorp-education/learn-terraform-validation-enforcement

Change to the repository directory.

$ cd learn-terraform-validation-enforcement

This repository contains a local Terraform module that defines a network and bastion host, and a root configuration that uses the module.

$ tree
.
├── README.md
├── main.tf
├── modules
│   └── network
│       ├── LICENSE
│       ├── README.md
│       ├── main.tf
│       └── variables.tf
├── outputs.tf
├── terraform.tf
├── terraform.auto.tfvars
└── variables.tf

Open the modules/network/main.tf file in your code editor. This configuration uses the public vpc module to provision networking resources, including public and private subnets and a NAT gateway. It then launches a bastion host in one of the public subnets.

##...
resource "aws_security_group" "bastion" {
  name   = "bastion_ssh"
  vpc_id = module.vpc_id

  ingress {
    from_port = 22
    to_port   = 22
    protocol  = "tcp"
    # Example CIDR
    cidr_blocks = ["192.168.0.0/16"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

resource "aws_instance" "bastion" {
  instance_type = var.bastion_instance_type
  ami           = data.aws_ami.amazon_linux.id

  subnet_id              = module.vpc.public_subnets[0]
  vpc_security_group_ids = [aws_security_group.bastion.id]
}

The bastion host is intended to be the single point of entry for any SSH traffic to other instances within the VPC’s private subnets. The configuration also includes an SSH group that scopes an ingress SSH traffic to the bastion to just the 192.168.0.0/16 CIDR block, a hypothetical CIDR representing your organization’s network.

Though this configuration references this module locally, in a larger organization, you would likely package this for distribution and usage in your Terraform registry. By including a bastion in the boilerplate of your networking configuration, you can establish a standard for SSH access to instances in your networks.

Define a precondition

The network module takes multiple input variables — one of which is bastion_instance_type — to allow users to account for anticipated usage and workloads. While you want to allow users to specify an instance type, you do not want to allow them to provision an instance that is too big. A precondition can verify that the instance type does not have more than 2 cores to keep your operating costs low.

First, add the data source below to the module configuration. It accesses the instance type details from the AWS provider.

data "aws_ec2_instance_type" "bastion" {
  instance_type = var.bastion_instance_type
}

Now, add the precondition to the aws_instance.bastion resource definition.

resource "aws_instance" "bastion" {
  instance_type = var.bastion_instance_type
  ami           = data.aws_ami.amazon_linux.id

  subnet_id              = module.vpc.public_subnets[0]
  vpc_security_group_ids = [aws_security_group.bastion.id]

  lifecycle {
    precondition {
      condition     = data.aws_ec2_instance_type.bastion.default_cores <= 2
      error_message = "Change the value of bastion_instance_type to a type that has fewer than 2 cores to avoid over provisioning."
    }
  }
}

Terraform evaluates preconditions before evaluating the surrounding block. In this case, it will check whether the configuration satisfies the condition before provisioning the bastion instance.

Deploy infrastructure

Navigate back to the root level of the repository directory. The root Terraform configuration uses the network module to create a bastion host and networking components including a VPC, subnets, a NAT gateway, and route tables.

It sets the values for input variables in the terraform.auto.tfvars file. The initial value for the bastion instance type is t2.2xlarge, which has 8 cores and will fail the precondition as expected.

bastion_instance_type = "t2.2xlarge"
aws_region            = "us-east-2"

Set your HCP Terraform organization name as an environment variable to configure your HCP Terraform integration.

$ export TF_CLOUD_ORGANIZATION=

Initialize your configuration. As part of initialization, Terraform creates your learn-terraform-validation-enforcement HCP Terraform workspace.

$ terraform init
Initializing modules...
- network in modules/network
Downloading registry.terraform.io/terraform-aws-modules/vpc/aws 3.14.0 for network.vpc...
- network.vpc in .terraform/modules/network.vpc

Initializing HCP Terraform...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v4.10.0...
- Installed hashicorp/aws v4.10.0 (signed by HashiCorp)

HCP Terraform has been successfully initialized!

You may now begin working with HCP Terraform. Try running "terraform plan" to
see any changes that are required for your infrastructure.

If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.

Now attempt to apply your configuration. The instance size you specified is too big, so the precondition will return an error.

$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-validation-enforcement/runs/run-gLh56su1pc4W52Vo

Waiting for the plan to start...

Terraform v1.2.2
on linux_amd64
Initializing plugins and modules...
data.aws_ami.amazon_linux: Reading...
module.network.data.aws_ami.amazon_linux: Reading...
module.network.data.aws_availability_zones.available: Reading...
module.network.data.aws_ec2_instance_type.bastion: Reading...
module.network.data.aws_ec2_instance_type.bastion: Read complete after 0s [id=t2.2xlarge]
module.network.data.aws_availability_zones.available: Read complete after 0s [id=us-east-2]
data.aws_ami.amazon_linux: Read complete after 1s [id=ami-0fe23c115c3ba9bac]
module.network.data.aws_ami.amazon_linux: Read complete after 1s [id=ami-0fe23c115c3ba9bac]
╷
│ Error: Resource precondition failed
│
│   on modules/network/main.tf line 68, in resource "aws_instance" "bastion":
│   68:       condition     = data.aws_ec2_instance_type.bastion.default_cores <= 2
│     ├────────────────
│     │ data.aws_ec2_instance_type.bastion.default_cores is 8
│
│ Bastion instances should not have more than 2 cores to avoid
│ overprovisioning.
╵
Operation failed: failed running terraform plan (exit 1)

The t2.2xlarge instance type has 8 cores, which would incur unnecessary cost to your bastion host. As a result, this Terraform run failed because it did not satisfy the preconditions defined in the network module.

Change the bastion_instance_type variable in terraform.auto.tfvars to t2.small.

bastion_instance_type = "t2.small"
aws_region            = "us-east-2"

Apply your configuration again. Respond yes to the prompt to confirm the operation.

$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-validation-enforcement/runs/run-JuiXz53wihnXUE18

Waiting for the plan to start...

Terraform v1.2.2
on linux_amd64
Initializing plugins and modules...
module.network.data.aws_ec2_instance_type.bastion: Reading...
module.network.data.aws_availability_zones.available: Reading...
module.network.data.aws_ami.amazon_linux: Reading...
data.aws_ami.amazon_linux: Reading...
module.network.data.aws_ec2_instance_type.bastion: Read complete after 0s [id=t2.small]
module.network.data.aws_availability_zones.available: Read complete after 0s [id=us-east-2]
data.aws_ami.amazon_linux: Read complete after 1s [id=ami-0fe23c115c3ba9bac]
module.network.data.aws_ami.amazon_linux: Read complete after 1s [id=ami-0fe23c115c3ba9bac]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:
##...
Plan: 22 to add, 0 to change, 0 to destroy.

Do you want to perform these actions in workspace "learn-terraform-validation-enforcement"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

##...
Apply complete! Resources: 22 added, 0 changed, 0 destroyed.

The precondition you defined let you query AWS to retrieve the most up to date information about an instance type's resource allocations, making your configuration more dynamic. You also could have used variable validation to catch the violation. However, that would require researching all available instance types and their capacities and listing the acceptable instance types in your configuration, making it less flexible.

Configure Styra run task

Configuration-level validation such as variable constraints and pre- and postconditions let you enforce standards from within your written configuration. However this enforcement relies on module authors including preconditions and postconditions in module definitions, and other users consuming those modules to provision infrastructure. To enforce and control infrastructure in a whole workspace, you can use HCP Terraform run tasks. A run task uses third party tools to determine whether the proposed changes in a Terraform operation satisfy predefined requirements, and whether to warn the user about a violation or to block the operation entirely.

Styra’s HCP Terraform run task lets you integrate OPA checks into your infrastructure management workflow. OPA allows you to use a standard policy-as-code framework across different parts of your system. The run task specifically lets you enforce infrastructure standards without requiring your users to write their infrastructure configuration in a specific way. The Styra run task works similarly to Sentinel’s policy-as-code implementation. However, the run task can only apply to the workspaces you associate it with, while you can apply Sentinel policies across your HCP Terraform organization.

Styra DAS organizes your policies in collections. The entirety of your policy set is a Styra workspace, which can consist of multiple systems. In Styra, a system is a logical collection of policies that is grouped by the way that you provision resources, such as Kubernetes or Terraform resources. To configure an HCP Terraform run task that uses the Styra integration, you must first create a Terraform system.

In your Styra dashboard, click the + next to Systems to create a new system.

Select the Terraform system type and name your system tfc. Then, click Create system.

Styra pre-configures your new system with a dedicated directory for each of the common cloud providers you may define policies for, including AWS, GCP, and Azure.

Next, you need to grant Styra access to your HCP Terraform organization using an API token. Navigate to the API Tokens page in your HCP Terraform organization’s settings. Then, click on Create an organization token. HCP Terraform will not display this token again, so keep this tab open.

Back in your Styra dashboard, navigate to your workspace’s settings, then click Terraform Integration. Enter your HCP Terraform organization name and the organization API token you just created, then click Save changes. Leave this tab open to complete the final configuration step later.

Styra created a policy check run task in your HCP Terraform organization. In a new tab, navigate to your learn-terraform-validation-enforcement HCP Terraform workspace.

Under the workspace’s Settings, select Run Tasks. Click the plus sign (+) next to the styra-das-policy-check-* run task to associate it with the current workspace.

Select the Mandatory enforcement level, then click Create.

Copy the workspace ID, found at the top of the page underneath the workspace name.

Back in your Styra dashboard, click Add system mapping in your Styra workspace’s Terraform integration settings.

Under DAS Terraform system, select the tfc system you created earlier. Paste in the workspace ID you just copied into the HCP Terraform workspaces field, then click Save changes.

You are now ready to define policies in your Styra DAS. To keep this tutorial simple, you will manage policies in the Styra UI. However, Styra can also use GitHub repositories as the source for your policies.

Define an infrastructure configuration policy check

When using Styra DAS to manage your OPA configuration, you can define your own custom policies or use existing policies from the public policy library. Paid users can also use compliance packs, which are collections of policies that enforce industry best practices and standards. Pre-defined policies make it easy for you to quickly add compliance checks across your organization.

In your Styra dashboard, navigate to the policy/aws/ec2/rules.rego file in your tfc system. Styra created this directory for you when you added the tfc system.

Click on the Add rule button, which allows you to select predefined rules from the policy library to add to your system. Type in security group and select the Restrict ingress from public IPs rule.

Styra added the following policy definition to your system, which will prevent Terraform from creating any security groups that allow ingress traffic from the public internet.

enforce[decision] {
  data.global.systemtypes["terraform:1.0"].library.provider.aws.network.security_group.ingress_restrict_public_access.v1.ingress_restrict_public_access[message]
  decision := {
    "allowed": false,
    "message": message
  }
}

Update the policy mode to Enforce to prevent the HCP Terraform workspace from creating any publicly accessible security groups. Then, click Publish and confirm for the policy to take effect.

Trigger policy violation

The networking resources you provisioned earlier include a bastion host configured with a security group that restricts ingress traffic to your organization’s internal network. Imagine a hypothetical scenario in which an engineer is troubleshooting a production incident and attempts to circumvent this restriction by adding a more permissive security group to the bastion host.

To simulate this, add the following configuration to your main.tf file.

data "aws_instance" "bastion" {
  instance_id = module.network.bastion_instance_id
}

resource "aws_security_group" "bastion" {
  name   = "bastion_ssh_new"
  vpc_id = module.network.vpc_id

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_network_interface_sg_attachment" "bastion" {
  security_group_id = aws_security_group.bastion.id
  network_interface_id = data.aws_instance.bastion.network_interface_id
}

This configuration defines the data.aws_instance.bastion data source to access the bastion host’s attributes, defines a security group that allows public ingress traffic, and associates the security group with the bastion’s network interface.

Run terraform apply to attempt to create these resources. The run task will fail because the security group is too permissive.

$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-validation-enforcement/runs/run-vHGE4qjtrwP6Tx24

Waiting for the plan to start...

Terraform v1.2.2
on linux_amd64
Initializing plugins and modules...
##...
Plan: 2 to add, 0 to change, 0 to destroy.


Run Tasks (post-plan):

All tasks completed! 0 passed, 1 failed

│ styra-das-policy-check-1oj519 ⸺   Errored (Mandatory)
│ 412 Precondition Failed
│ Details: https://1oj519.svc.styra.com/settings/terraform-integration
│
│
│ Overall Result: Failed

------------------------------------------------------------------------

╷
│ Error: the run failed because the run task, styra-das-policy-check-1oj519, is required to succeed

After Terraform generated the execution plan for your resource changes, it sent the details to Styra to verify that the proposed resources satisfy your defined policies. As expected, your policy check failed and prevented Terraform from creating the overly-permissive security group. Navigate to the run in your HCP Terraform workspace, then click on the Details to get more information about the run task failure.

After a few minutes, Styra will display the decision details for the run, including the reason for the failure. Expand the decision details to see the input for the run, which contains the Terraform execution plan and all proposed resource changes.

Define a policy for organization practices

In addition to placing guardrails on infrastructure configuration, you may wish to enforce standards around your organization’s workflows themselves. One common practice is to prevent infrastructure deployments on Fridays in order to lower the risk of production incidents before the weekend.

In your Styra dashboard, click the three dots next to the policy directory, then Add policy to create a new policy.

Enter tfc/runs as the path and rules.rego as the module name. Then, click Add.

In the tfc/runs/rules.rego file, add the following rule to prevent operations on a given day of the week. Replace Friday with the current day of the week.

enforce[decision] {
  message := "No deployments allowed on Fridays"

  time.weekday(time.now_ns()) == "Friday"

  decision := {
    "allowed": false,
    "message": message
  }
}

Then, click Publish for the policy to take effect.

Now trigger another run to see this policy in effect.

$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-validation-enforcement/runs/run-JKdodpoAoGSf75Bv

Waiting for the plan to start...

Terraform v1.2.2
on linux_amd64
Initializing plugins and modules...
##...
Run Tasks (post-plan):

All tasks completed! 0 passed, 1 failed

│ styra-das-policy-check-1oj519 ⸺   Failed (Mandatory)
│ Found 1 policy failure. Click details for the full decision log and remediation steps.
│ Details: https://1oj519.svc.styra.com/systems/c4f8cfdf7fa84acf8e6ed105483924af/decisions?filter=decision_id=9fda2998-0fc8-4914-bd31-340d124f22dc
│
│
│ Overall Result: Failed

------------------------------------------------------------------------

╷
│ Error: the run failed because the run task, styra-das-policy-check-1oj519, is required to succeed
│
│
╵

If you navigate to the decision details for this run, Styra will show that the plan failed both of your defined rules.

Modify policy

In some cases, blocking all Friday infrastructure deployments may be too limiting. In the event of an outage or production incident, you may want to allow senior members of your team to make necessary changes, regardless of the day. To do so, you can define a list of HCP Terraform users as an exception to the rule.

The HCP Terraform plan payload contains a styra-tfc-webhook field with details about the run, including the HCP Terraform user name of the user who initiated the operation.

  "styra-tfc-webhook": {
##...
      "run_created_by": "USERNAME",
##...
    },

Update your deployment policy to check whether the user is a member of the exceptions list. Replace Friday with the current day of the week and USERNAME with your own HCP Terraform username to add yourself to the list of overrides.

package policy.tfc.runs

allowed_users := {
  "USERNAME"
}

enforce[decision] {
  user := input["styra-tfc-webhook"].run_created_by
  message := sprintf("User %s is not permitted to trigger deployments on Fridays", [user])

  time.weekday(time.now_ns()) == "Friday"
  not allowed_users[user] 

  decision := {
    "allowed": false,
    "message": message
  }
}

Publish the policy changes.

Now, update your security group configuration to fix the overly permissive security group.

data "aws_instance" "bastion" {
  instance_id = module.network.bastion_instance_id
}

resource "aws_security_group" "bastion" {
  name   = "bastion_ssh_new"
  vpc_id = module.network.vpc_id

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["172.16.0.0/12"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Trigger another Terraform run. This time, your plan will succeed.

$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-validation-enforcement/runs/run-JKdodpoAoGSf75Bv

Waiting for the plan to start...

Terraform v1.2.2
on linux_amd64
Initializing plugins and modules...
##...
Run Tasks (post-plan):
All tasks completed! 1 passed, 0 failed
│ styra-das-policy-check-1oj519 ⸺   Passed
│ The Styra DAS OPA run was successful.
│ Details: https://1oj519.svc.styra.com/systems/c4f8cfdf7fa84acf8e6ed105483924af/decisions?filter=decision_id=d3a675bd-6368-458a-87c2-59360b1d938e
│
│
│ Overall Result: Passed
------------------------------------------------------------------------

Do you want to perform these actions in workspace "learn-terraform-validation-enforcement"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Using HCP Terraform run tasks and Styra, you defined two different policies that enforced infrastructure configuration standards and organizational practices.

Clean up infrastructure

Destroy the resources you created as part of this tutorial to avoid incurring unnecessary costs. Respond yes to the prompt to confirm the operation.

$ terraform destroy
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-validation-enforcement/runs/run-iHhaPP2ybRF8Ppjd

Waiting for the plan to start...

Terraform v1.2.2
on linux_amd64
Initializing plugins and modules...
##...
Plan: 0 to add, 0 to change, 22 to destroy.


Run Tasks (post-plan):

All tasks completed! 1 passed, 0 failed

│ styra-das-policy-check-1oj519 ⸺   Passed
│ The Styra DAS OPA run was successful.
│ Details: https://1oj519.svc.styra.com/systems/c4f8cfdf7fa84acf8e6ed105483924af/decisions?filter=decision_id=d3a675bd-6368-458a-87c2-59360b1d938e
│
│
│ Overall Result: Passed

------------------------------------------------------------------------


Do you really want to destroy all resources in workspace "learn-terraform-validation-enforcement"?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes
##...

Then, delete your learn-terraform-validation-enforcement workspace in HCP Terraform.

Optionally, delete the run task from your HCP Terraform organization and the tfc system from Styra DAS.

Next steps

In this tutorial, you used Terraform preconditions and HCP Terraform run tasks to define multiple guardrails in your infrastructure provisioning workflow. Configuration-level validation such as pre- and postconditions allow you to prevent users from provisioning resources on the basis of attributes in your written configuration. HCP Terraform run tasks give you broader scope of control, and let you use both public and custom policies to enforce your organization’s standards on specific workspaces.

To learn more about how Terraform features can help you validate your infrastructure configuration, check out the following resources:

• Learn how to manage your infrastructure costs in HCP Terraform.
• Learn how to use HCP Terraform run tasks and HCP Packer to ensure machine image compliance.
• Review the available Styra policy library rules.