Imperative Infrastructure as Code using AWS CDK

When we talk about infrastructure as code we often talk about declarative frameworks. Frameworks like Terraform or CloudFormation often come to mind first. Declarative approaches allow you to define what you want to be created and the tool provisions the resources for you. The general idea is that you declare what you want your final state to be and the tool does the work to get you there.

But, this is only one approach.

Another approach to creating your infrastructure via code is via imperative options. Unlike declarative where we specify what, an imperative approach specifies how. Instead of saying “I want these 3 AWS resources”, an imperative approach says “Here is how to create these 3 AWS resources”.

The difference between the two is subtle, one focuses on obtaining the end state. The other focuses on defining how to get the end state.

There are advantages and disadvantages to both approaches.

Declarative approaches have the advantage of operating on the end state. This means by nature they tend to know the current state of your infrastructure. If in a declarative framework I provision 10 EC2 instances and later need 15 instead of 10, it knows I only need 5 more. Declarative infrastructure as code tends to be in languages that we don’t use every day like HCL, YAML, or JSON. This also means they can be tricky to reuse or modularize to share across projects.

Imperative approaches can often use languages we already know. We can define our infrastructure in the code we use every day like Python, TypeScript, or even Ruby. This means all the tools we have for testing, reuse, and sharing are usable here. But, it also means that we often lose the notion of the end state. Taking our example from above, 10 EC2 instances to 15 in an imperative framework creates 15 new EC2 instances instead of adding 5 more to our existing 10. This is because the imperative approach defines how to get the final state we want, not what the final state should be.

The imperative approach has been on the rise recently. Many folks would prefer that infrastructure code is in the same language they use every day.

So in this post, we are going to take a look at how we can define some of our AWS infrastructure using the new project, AWS Cloud Development Kit.

Prerequisites

Before we can start defining some infrastructure using the CDK, we need to complete a few prerequisites.

First, we need to install AWS CDK via npm:

$ npm install -g aws-cdk
$ cdk --version
1.6.1 (build a09203a)

Now that we have the CDK installed, let’s configure a sample project that we can start experimenting in. For this post, we are going to make use of Typescript, but the CDK is available in Python, JavaScript, Java, and .NET.

$ mkdir cdk-post
$ cd cdk-post
$ cdk init sample-app --language=typescript
Applying project template sample-app for typescript
Initializing a new git repository...
Executing npm install...
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.

# Useful commands

 * `npm run build`   compile typescript to js
 * `npm run watch`   watch for changes and compile
 * `cdk deploy`      deploy this stack to your default AWS account/region
 * `cdk diff`        compare deployed stack with current state
 * `cdk synth`       emits the synthesized CloudFormation template

Using CDK for Infrastructure as Code

If we take a look at the sample app that cdk created we should see a file at lib/cdk-post-stack.ts. When we open that file we should see that there is some code in it that provisions an SQS queue and an SNS topic.

import sns = require('@aws-cdk/aws-sns')
import subs = require('@aws-cdk/aws-sns-subscriptions')
import sqs = require('@aws-cdk/aws-sqs')
import cdk = require('@aws-cdk/core')

export class CdkPostStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props)

    const queue = new sqs.Queue(this, 'CdkPostQueue', {
      visibilityTimeout: cdk.Duration.seconds(300),
    })

    const topic = new sns.Topic(this, 'CdkPostTopic')

    topic.addSubscription(new subs.SqsSubscription(queue))
  }
}

What we have here is a Stack, a collection of AWS resources that should get created, maintained, and removed together. In a CDK world, an application or service can consist of one or more stacks. We can see how this stack gets created by CDK by taking a look at bin/cdk-post.ts.

#!/usr/bin/env node
import cdk = require('@aws-cdk/core')
import {CdkPostStack} from '../lib/cdk-post-stack'

const app = new cdk.App()
new CdkPostStack(app, 'CdkPostStack')

Here we see that an App gets created via cdk and the CdkPostStack we looked at before gets attached to it. Let’s go ahead and deploy this sample app via the cdk on our command line.

$ cdk deploy
IAM Statement Changes
┌───┬─────────────────────┬────────┬─────────────────┬───────────────────────────┬─────────────────────────────────────────────────────┐
│   │ Resource            │ Effect │ Action          │ Principal                 │ Condition                                           │
├───┼─────────────────────┼────────┼─────────────────┼───────────────────────────┼─────────────────────────────────────────────────────┤
│ + │ ${CdkPostQueue.Arn} │ Allow  │ sqs:SendMessage │ Service:sns.amazonaws.com │ "ArnEquals": {                                      │
│   │                     │        │                 │                           │   "aws:SourceArn": "${CdkPostTopic}"                │
│   │                     │        │                 │                           │ }                                                   │
└───┴─────────────────────┴────────┴─────────────────┴───────────────────────────┴─────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See http://bit.ly/cdk-2EhF7Np)

Do you wish to deploy these changes (y/n)? y
CdkPostStack: deploying...
CdkPostStack: creating CloudFormation changeset...
 0/6 | 17:36:05 | CREATE_IN_PROGRESS   | AWS::CDK::Metadata     | CDKMetadata
 0/6 | 17:36:05 | CREATE_IN_PROGRESS   | AWS::SQS::Queue        | CdkPostQueue (CdkPostQueueBA7F3D07)
 0/6 | 17:36:05 | CREATE_IN_PROGRESS   | AWS::SNS::Topic        | CdkPostTopic (CdkPostTopic28394E2B)
 0/6 | 17:36:05 | CREATE_IN_PROGRESS   | AWS::SQS::Queue        | CdkPostQueue (CdkPostQueueBA7F3D07) Resource creation Initiated
 0/6 | 17:36:05 | CREATE_IN_PROGRESS   | AWS::SNS::Topic        | CdkPostTopic (CdkPostTopic28394E2B) Resource creation Initiated
 1/6 | 17:36:05 | CREATE_COMPLETE      | AWS::SQS::Queue        | CdkPostQueue (CdkPostQueueBA7F3D07)
 1/6 | 17:36:06 | CREATE_IN_PROGRESS   | AWS::CDK::Metadata     | CDKMetadata Resource creation Initiated
 2/6 | 17:36:06 | CREATE_COMPLETE      | AWS::CDK::Metadata     | CDKMetadata
 3/6 | 17:36:16 | CREATE_COMPLETE      | AWS::SNS::Topic        | CdkPostTopic (CdkPostTopic28394E2B)
 3/6 | 17:36:18 | CREATE_IN_PROGRESS   | AWS::SNS::Subscription | CdkPostQueue/CdkPostStackCdkPostTopic7A3E421F (CdkPostQueueCdkPostStackCdkPostTopic7A3E421F4679B27C)
 3/6 | 17:36:18 | CREATE_IN_PROGRESS   | AWS::SQS::QueuePolicy  | CdkPostQueue/Policy (CdkPostQueuePolicyC7FE0F0B)
 3/6 | 17:36:19 | CREATE_IN_PROGRESS   | AWS::SNS::Subscription | CdkPostQueue/CdkPostStackCdkPostTopic7A3E421F (CdkPostQueueCdkPostStackCdkPostTopic7A3E421F4679B27C) Resource creation Initiated
 3/6 | 17:36:19 | CREATE_IN_PROGRESS   | AWS::SQS::QueuePolicy  | CdkPostQueue/Policy (CdkPostQueuePolicyC7FE0F0B) Resource creation Initiated
 4/6 | 17:36:19 | CREATE_COMPLETE      | AWS::SNS::Subscription | CdkPostQueue/CdkPostStackCdkPostTopic7A3E421F (CdkPostQueueCdkPostStackCdkPostTopic7A3E421F4679B27C)
 5/6 | 17:36:19 | CREATE_COMPLETE      | AWS::SQS::QueuePolicy  | CdkPostQueue/Policy (CdkPostQueuePolicyC7FE0F0B)
 6/6 | 17:36:21 | CREATE_COMPLETE      | AWS::CloudFormation::Stack | CdkPostStack

 ✅  CdkPostStack

What do we see? Quite a few interesting things.

1. We see that cdk tells us about any IAM statements that are going to change. Showing that they want to highlight any security changes before proceeding.
2. The cdk asks for confirmation before creating the resources. You can bypass this if you want to auto-approve the creation by adding -y to the command.
3. There is a CloudFormation changeset that gets generated. This is interesting in terms of declarative versus imperative infrastructure as code, more on that later.

If we log into the AWS Console we should see that we now have SQS queue and an SNS topic that starts with CdkPostStack. Additionally, we should see that we have a CloudFormation stack that contains all our resources.

Now let’s go back to our sample application and make a few changes to see how updates are handled in CDK. For our example let’s add another SQS queue and change the timeout of our existing queue. So now our CdkPostStack should look like this.

export class CdkPostStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props)

    const queue = new sqs.Queue(this, 'CdkPostQueue', {
      visibilityTimeout: cdk.Duration.seconds(600),
    })

    const dev2Queue = new sqs.Queue(this, 'Dev2Queue', {
      visibilityTimeout: cdk.Duration.seconds(180),
    })

    const topic = new sns.Topic(this, 'CdkPostTopic')
    topic.addSubscription(new subs.SqsSubscription(queue))
  }
}

We now have a new queue, Dev2Queue, and we have updated our CdkPostQueue to have a visibility timeout of 600 seconds rather than 300. Let’s use CDK to update our infrastructure and see what happens.

$ cdk deploy -y
CdkPostStack: deploying...
CdkPostStack: creating CloudFormation changeset...
 0/5 | 12:40:55 | CREATE_IN_PROGRESS   | AWS::SQS::Queue        | Dev2Queue (Dev2Queue5997490B)
 0/5 | 12:40:56 | UPDATE_IN_PROGRESS   | AWS::SQS::Queue        | CdkPostQueue (CdkPostQueueBA7F3D07)
 0/5 | 12:40:56 | CREATE_IN_PROGRESS   | AWS::SQS::Queue        | Dev2Queue (Dev2Queue5997490B) Resource creation Initiated
 1/5 | 12:40:56 | CREATE_COMPLETE      | AWS::SQS::Queue        | Dev2Queue (Dev2Queue5997490B)
 2/5 | 12:40:56 | UPDATE_COMPLETE      | AWS::SQS::Queue        | CdkPostQueue (CdkPostQueueBA7F3D07)
 2/5 | 12:41:00 | UPDATE_COMPLETE_CLEA | AWS::CloudFormation::Stack | CdkPostStack
 3/5 | 12:41:01 | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | CdkPostStack

 ✅  CdkPostStack

Awesome! What we see after this deploy is that our new queue is created and our existing queue gets updated as expected. We were able to make changes in our code and see them get reflected in a declarative way via CloudFormation.

But wait, what does this mean?

AWS CDK blurs the lines

It’s worth taking a pause at this point and revisiting our imperative versus declarative discussion from earlier. AWS CDK is an imperative tool. As we saw we can create our infrastructure via Typescript.

But, when we ran cdk deploy we saw that a CloudFormation changeset gets created. CloudFormation is a declarative infrastructure as code tool.

This is where the debate between the two approaches for provisioning your AWS resources gets confusing. Behind the scenes, CDK is still leveraging CloudFormation to do the provisioning. In a way, we are getting all the benefits of the declarative approach with all the benefits of the imperative approach as well.

We saw further evidence of this when we updated our existing queue and created a new queue. Our entire stack wasn’t recreated as you would expect in a strict imperative approach. Instead, we saw a declarative update happen, the queue got updated in place and our new queue got created.

CDK is providing us with an imperative interface that we can use to represent our infrastructure as code. Underneath the hood, it is still making use of the declarative framework, CloudFormation.

What else can CDK give us?

It can give us everything that our normal programming patterns and practices allow. Why? Because it is a framework that we can use in the languages we already know.

This means that we can do things like test the code that provisions our infrastructure. A problem that has proven to be somewhat tricky in a declarative world like Terraform or CloudFormation.

Modularizing and sharing code that creates common pieces of infrastructure is possible in AWS CDK. In a CDK world, these are referred to as constructs. Where a construct can consist of one or more AWS resources that need to get created together. As we saw in our example we could create a construct that others could reuse to provision both queues and our SNS topic. That said, declarative frameworks like Terraform have this notion as well in the form of modules.

Conclusion

Some folks are very excited about clear imperative approaches for representing their infrastructure in code that they use every day. Other folks are die-hard declarative fans that are OK with learning another language to gain the benefits of a stateful approach.

But as we saw in our look at AWS Cloud Development Kit (CDK), the debate between the two may not be all that important anyway. There are still pros and cons to each, but those tend to be more tool-specific than method specific. What we saw with CDK is that they blur the line between the two. Represent your code in your everyday languages and they will handle the declarative part behind the scenes using CloudFormation.

The key takeaway here is that representing your infrastructure in code is a win all around. What CDK shows us is that more tools are being created to increase the adoption of this practice. If the imperative approach isn’t your jam, that’s fine. But, you should find the tool and method that does work for you and get to work using it. Any infrastructure as code is still better than no infrastructure as code.