General best practices say that we should isolate our development environments from our production environments. In terms of Amazon Web Services, this often means that we configure entirely separate accounts for each environment. Essentially, environments are segregated at an account level.
If we need to create a new account for a new environment, most of us will typically do this manually. We might also configure billing alarms and set up a process to track our spend. Not to bad right? One account created manually isn’t too bad if we only need to do it once in a while.
But, this can be cumbersome. What if we have multiple products and need to provision new environments every month or every week? Creating our accounts manually becomes a full-time job. Each account has to be created, configured, and continously managed.
As organizations scale in AWS, this process can become an inhibitor to bringing new accounts online. This continous management also means we have to manage the spend of multiple AWS accounts. When they are all separate and not easily manageable this could cause surprises at the end of the month.
Luckily, there are strategies we can use to optimize and even automate a large portion of this process.
In this post, we are going to explore how we can use AWS Organizations to consolidate all of the billing into one master account. This will also give us the ability to manage each child account using a role from the master account. We are also going to use a bit of Terraform to demonstrate how we can automate the account creation process.
Billing in Amazon Web Services
For those not familiar with AWS, you are billed for the resources you use in a given month. At the end of the month we receive an invoice in our email that tells us what our spend was. We also tend to have billing alarms configured so we know if we are spending more than we intended before we get the bill at the end of the month.
For one account, this is all pretty straight forward.
But each new account we create must be managed in this way as well. Each account will have its own invoice and billing alarms. I don’t know about you, but I don’t want to go through ten different invoices or configure ten different billing alarms.
AWS Organizations simplifies this problem greatly for us. Using the organizations’ service we can consolidate billing across our accounts so that we get one bill at the end of the month. Not only will this consolidate our billing, but we can create one billing alarm to rule them all. We can also configure roles on each account that allow users from the master account to manage child accounts.
Let’s setup AWS Organizations so that we can link our existing account and future accounts to it.
Setting up your AWS Organization
The remainder of this post assumes that you have one AWS account already created. We are going to call this account the master account. Think of this as the top level account that additional accounts are going to roll their billing up to.
From the AWS Console of your master account, navigate to AWS Organizations. Now we can set up our organization.
- Select “My Organizations”.
- Click “Create Organization”.
- Select the option, “Enable only consolidated billing”. There are other features of AWS Organizations that we could enable, but consolidated billing is the only one we need right now.
- Click “Create”.
We now have our top-level AWS organization. That was simple enough, but sometimes clicking buttons in the AWS Console is a bit of a pain. That’s OK, you can use Terraform to do the exact same thing with one resource.
provider "aws" {
region = "us-west-2"
}
resource "aws_organizations_organization" "top-level-org" {
feature_set = "CONSOLIDATED_BILLING"
}
When you run terraform apply with this resource, your AWS CLI must be configured to use your master account
credentials. Once complete, you will have an organization configured in your master account that our other accounts can
reside under.
Provisioning accounts
Sticking to the Terraform approach we can provision a new AWS account by adding additional resources to the same
template where our top-level-org resource is. Let’s add a new account to our template.
provider "aws" {
region = "us-west-2"
}
resource "aws_organizations_organization" "top-level-org" {
feature_set = "CONSOLIDATED_BILLING"
}
resource "aws_organizations_account" "test-account" {
name = "test-account"
email = "[email protected]"
role_name = "admin"
}
We now have our top-level organization, top-level-org and an additional account associated with an email address. We
also see that this account has a role of admin associated with it. This is the role that users in the master account
can assume to access the new member account.
With this template, we have a top-level organization and one additional account. Additionally, we have configured our master account to handle all of the consolidated billing. Allowing us to track our costs from a single account.
A few things to note about what we just created using Terraform:
- The email associated with each separate AWS account must not be associated with any other account.
- The member accounts provisioned via the Organizations service are not configured to be standalone.
The second bullet point introduces some odd behaviors in terms of automation. Since they are not standalone, they cannot be automatically removed from the organization or destroyed until they are.
In order to configure the account to be standalone, you must access it as the root user which is the email address associated with the account. If this a brand new account you don’t have the password for that user.
You must go through the reset password flow for the root user using the login details specific to that account. Once that is complete you can log in to that account as the root user and configure it to be standalone. Typically standalone means entering billing information that account.
Honestly, this feels a bit clunky and appears to be a rough spot in AWS Organizations that could use some polish. It feels odd to have to go through some manual steps to configure the account to be standalone if we ever want to delete it or remove it from the organization.
That said if we are provisioning an account for a new environment for a specific product we are likely not going to be deleting it anytime soon.
Final thoughts
AWS Organizations greatly simplifies the provisioning, management, and billing of new and existing AWS accounts. If we need to launch a new account for a new environment of our product or service we can add it to our Terraform template, apply it to our master account, and were done.
Before we would have had to manually create the account, configure billing alarms and continously manage it. For one additional account, that’s probably not to bad. For three, five, or even ten additional accounts, it becomes quite a pain point.
There are some rough spots in AWS Organizations that might be painful if you are needing to frequently remove or delete accounts. But, outside of that, the service can really make managing and monitoring the costs associated with each account very streamlined.