Introducing AWS CloudFormation Hooks invoked via AWS Cloud Control API (CCAPI)

Today we are announcing the integration of AWS CloudFormation Hooks with AWS Cloud Control API (CCAPI). This integration enables the use of hooks to validate the configuration of resources being provisioned through CCAPI. In this blog post, we will explore the integration between CloudFormation Hooks and CCAPI by configuring an existing hook to work with CCAPI and then test that hook using the AWS CLI and Terraform.

Understanding CloudFormation Hooks

CloudFormation Hooks integrate seamlessly with your CloudFormation and CCAPI requests to perform validation of your resource configuration during resource create and update operations. You can create hooks using AWS Lambda, AWS CloudFormation Guard rules, or using code and the CloudFormation Command Line Interface (CFN-CLI). A hook can be triggered on change sets, entire stack templates, or by each resource and it will return back any discovered misconfiguration information. Hooks can be configured to warn or fail on the operation allowing you to prevent any misconfigured resources from being deployed in your account. Some key benefits of using CloudFormation Hooks with CCAPI include:

• Enforcing security best practices
• Applying organizational policies to resource deployments
• Optimizing resource configurations for cost and performance
• Standardize validation across different infrastructure as code solutions like CloudFormation, Terraform (Terraform AWS Cloud Control Provider), and Pulumi (AWS Cloud Control)

Prerequisites

For this post we are going to use the new AWS CloudFormation Guard (Guard) hook AWS::Hooks::GuardHook. Guard is an open-source policy-as-code tool that allows you to validate your infrastructure configurations against company policy guidelines. It provides a domain-specific language (DSL) for writing rules to check both required and prohibited resource configurations. The new AWS::Hooks::GuardHook allows you to use the Guard DSL inside of a hook so you can easily implement your organizations guidelines. The result is you can use the same Guard rules in our local development environment, continuous integration and continuous deployment pipelines, and at deployment time (using hooks). To learn more about AWS::Hooks::GuardHook you can look at the blog.

This is what the configuration of the current Guard hook looks like.

{
    "CloudFormationConfiguration": {
        "HookConfiguration": {
            "HookInvocationStatus": "ENABLED",
            "FailureMode": "FAIL",
            "TargetOperations": [
                "RESOURCE"
            ],
            "TargetFilters": {
                "Actions": [
                    "CREATE",
                    "UPDATE"
                ]
            },
            "Properties": {
                "ruleLocation": {
                    "uri": "s3://<my-guard-hook-config-bucket>/s3-guard.zip"
                },
                "logBucket": "<my-guard-hook-logging-bucket>",
            }
        }
    }
}

This hook has been configured to log the Guard validation report to an Amazon Simple Storage Service (S3) bucket. Additionally, this hook is configured to use a rule from the AWS CloudFormation Guard registry. This rule will validate that an S3 bucket is using versioning. This hook is configured with an alias named My::Hooks::Guard.

Here is the rule for reference. This rule will validate that the property VersioningConfiguration is provided and that its value is Enabled.

let s3_buckets_versioning_enabled = Resources.*[ Type == 'AWS::S3::Bucket' ]
rule S3_BUCKET_VERSIONING_ENABLED when %s3_buckets_versioning_enabled !empty {
  %s3_buckets_versioning_enabled.Properties.VersioningConfiguration exists
  %s3_buckets_versioning_enabled.Properties.VersioningConfiguration.Status == 'Enabled'
  <<
    Guard Rule Set: ABS-CCIGv2-Standard
    Controls: section4b-design-and-secure-the-cloud-14-standard-workloads,section4b-design-and-secure-the-cloud-15-standard-workloads    
    Violation: S3 Bucket Versioning must be enabled.
    Fix: Set the S3 Bucket property VersioningConfiguration.Status to 'Enabled' .
  >>
}

Configuring the hook to work with CCAPI

This announcement adds a new hook target that can easily be configured on your existing or new hooks. To configure the hook to work with CCAPI you will edit the configuration to include a new TargetOperations value of CLOUD_CONTROL. This hook is only enabled to execute on CREATE and UPDATE operations. Additionally HookInvocationStatus is ENABLED which will execute the hook and FailureMode will tell the hook to FAIL the operation if the resource is not compliant.

{
    "CloudFormationConfiguration": {
        "HookConfiguration": {
            "HookInvocationStatus": "ENABLED",
            "FailureMode": "FAIL",
            "TargetOperations": [
                "RESOURCE",
                "CLOUD_CONTROL"
            ],
            "TargetFilters": {
                "Actions": [
                    "CREATE",
                    "UPDATE"
                ]
            },
            "Properties": {
                "ruleLocation": {
                    "uri": "s3://<my-guard-hook-confg-bucket>/s3-guard.zip"
                },
                "logBucket": "<my-guard-hook-logging-bucket>",
            }
        }
    }
}

By using TargetOperations of ["RESOURCE", "CLOUD_CONTROL"] the Guard rules will work the same across CloudFormation resource operations and CCAPI operations.

Testing the hook using AWS CLI

Test your hook using the AWS CLI which allows us to create, update, delete, and list resources.

1. Start by creating a S3 bucket using CCAPI. In this example you are providing no properties for creating the S3 bucket. Run the command aws cloudcontrol create-resource --type-name AWS::S3::Bucket --desired-state {}
   Response:

   {
       "ProgressEvent": {
           "TypeName": "AWS::S3::Bucket",
           "RequestToken": "2c7b6f5e-4083-4ef8-9a23-5c81472540b1",
           "Operation": "CREATE",
           "OperationStatus": "IN_PROGRESS",
           "EventTime": "2024-11-05T09:41:38.072000-08:00"
       }
   }

2. Get the request status by using the RequestToken from the response above. Run the command aws cloudcontrol get-resource-request-status --request-token 2c7b6f5e-4083-4ef8-9a23-5c81472540b1

   {
       "ProgressEvent": {
           "TypeName": "AWS::S3::Bucket",
           "RequestToken": "2c7b6f5e-4083-4ef8-9a23-5c81472540b1",
           "HooksRequestToken": "4a193a00-4c76-41fe-87b8-75b838f00bbe",
           "Operation": "CREATE",
           "OperationStatus": "FAILED",
           "EventTime": "2024-11-05T09:41:40.785000-08:00",
           "StatusMessage": "Request [4a193a00-4c76-41fe-87b8-75b838f00bbe] failed \ndue to the following failed invocations: [My::Hooks::Guard]"
       },
       "HooksProgressEvent": [
           {
               "HookTypeName": "My::Hooks::Guard",
               "HookTypeVersionId": "00000006",
               "HookTypeArn": "arn:aws:cloudformation:eu-central-1:123456789012:type/hook/My-Hooks-Guard/00000001/aws-hooks/AWS-Hooks-GuardHook/00000001.00000005",
               "InvocationPoint": "PRE_PROVISION",
               "HookStatus": "HOOK_COMPLETE_FAILED",
               "HookEventTime": "2024-11-05T09:41:38.978000-08:00",
               "HookStatusMessage": "Template failed validation, the following rule(s) failed: S3_BUCKET_VERSIONING_ENABLED. Full output was written to s3://<my-guard-hook-logging-bucket>/cfn-guard-validate-report/AWS--S3--Bucket-4a193a00-4c76-41fe-87b8-75b838f00bbe-RESOURCE-AWS--S3--Bucket-CREATE-PRE_PROVISION/1730828427591.json",
               "FailureMode": "FAIL"
           }
       ]
   }

   In the response you will see all hooks that were executed and their response in relation to the request. This response shows that the hook My::Hooks::Guard failed because of the rule S3_BUCKET_VERSIONING_ENABLED . You are also provided a s3 location for where the full Guard output is stored.

3. You can get the Guard results file by using the following command. Replace <path-from-previous-output> with the path provided in the previous output. Run the command aws s3 cp s3://<path-from-previous-output> -
   Response:

   [
       {
           "name": "STDIN",
           "metadata": {},
           "status": "FAIL",
           "not_compliant": [
               {
                   "Rule": {
                       "name": "S3_BUCKET_VERSIONING_ENABLED",
                       "metadata": {},
                       "messages": {
                           "custom_message": null,
                           "error_message": null
                       },
                       "checks": [
                           {
   ...

   We truncated the output as it can be very verbose.

Testing the hook using Terraform

The Terraform AWS Cloud Control Provider allows you to manage AWS resources using CCAPI and Terraform. By leveraging this provider you get the benefit of using hooks to validate the configuration of Terraform provisioned resources.

1. Create a new Terraform configuration file named main.tf with the following content:

   terraform {
       required_providers {
           awscc = {
               source  = "hashicorp/awscc"
               version = "~> 1.20"
           }
       }
   }
   
   resource "awscc_s3_bucket" "example" {}

2. Run the following commands to initialize Terraform and create an execution plan. Run the command terraform init followed by terraform plan.
3. Apply the configuration by running. Run the command terraform apply

   Response:

   ...
   awscc_s3_bucket.example: Creating...
   ╷
   │ Error: AWS SDK Go Service Operation Incomplete
   │ 
   │   with awscc_s3_bucket.example,
   │   on main.tf line 14, in resource "awscc_s3_bucket" "example":
   │   14: resource "awscc_s3_bucket" "example" {
   │ 
   │ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: Request [d417b05b-9eff-46ef-b164-08c76aec1801] failed 
   │ due to the following failed invocations: [My::Hooks::Guard]. ErrorCode: 
   ╵

   In this response you can see that the hook My::Hooks::Guard failed and the request token is d417b05b-9eff-46ef-b164-08c76aec1801

4. You can get details on the hook invocation by running the command aws cloudformation list-hook-results --hook-target TargetType=CLOUD_CONTROL,TargetId=d417b05b-9eff-46ef-b164-08c76aec1801Response:

   {
       "TargetType": "CLOUD_CONTROL",
       "TargetId": "d417b05b-9eff-46ef-b164-08c76aec1801",
       "HookResults": [
           {
               "InvocationPoint": "PRE_PROVISION",
               "FailureMode": "FAIL",
               "TypeName": "My::Hooks::Guard",
               "TypeVersionId": "00000006",
               "Status": "HOOK_COMPLETE_FAILED",
               "HookStatusReason": "Template failed validation, the following rule(s) failed: S3_BUCKET_VERSIONING_ENABLED. Full output was written to s3://my-company-guard-hooks-eu-central-1/cfn-guard-validate-report/AWS--S3--Bucket-d417b05b-9eff-46ef-b164-08c76aec1801-RESOURCE-AWS--S3--Bucket-CREATE-PRE_PROVISION/1730829108790.json"
           }
       ]
   }

   As with the AWS CLI you now know what rule failed and additional you have the S3 bucket location for the Guard log file.

5. You can get the Guard results file by running the command aws s3 cp s3://<path-from-previous-output> -. Replace <path-from-previous-output> with the path provided in the previous output.Response:

   [
       {
           "name": "STDIN",
           "metadata": {},
           "status": "FAIL",
           "not_compliant": [
               {
                   "Rule": {
                       "name": "S3_BUCKET_VERSIONING_ENABLED",
                       "metadata": {},
                       "messages": {
                           "custom_message": null,
                           "error_message": null
                       },
                       "checks": [
                           {
   ...

   We truncated the output as it can be very verbose.

6. Let’s correct our S3 bucket configuration in main.tf

   terraform {
       required_providers {
           awscc = {
               source  = "hashicorp/awscc"
               version = "~> 1.20"
           }
       }
   }
   
   resource "awscc_s3_bucket" "example" {
       versioning_configuration = {
           status = "Enabled"
       }
   }

7. Try the deployment again by running terraform applyResponse:

   ...
   awscc_s3_bucket.example: Creating...
   awscc_s3_bucket.example: Still creating... [10s elapsed]
   awscc_s3_bucket.example: Creation complete after 20s [id=rjuzykvh6oum2jeuq42xsxets-evilftb6rutb]
   Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

   Conclusion

   CloudFormation Hooks provide a powerful way to enforce best practices and compliance for your AWS resources. By leveraging CloudFormation Hooks and the Cloud Control API you can create consistent validation of your resources before deployment across many of your infrastructure as code solutions.