AWS CloudFormation Concepts

External

• https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-whatis-concepts.html

Internal

• AWS CloudFormation
• AWS CloudFormation Resource Types
• CloudFormation Intrinsic Functions

Overview

CloudFormation models, configures and set up resources, grouped in stacks. The resources and their high-level dependencies are declared in templates. A template describes desired state, not a set of operations to perform. If a resource is defined in the template it will be created. If a resource already exists, it will not be created, but can be updated if its properties change. If a resource is removed from the template it will be deleted. One of the greatest benefits of templates and AWS CloudFormation is the ability to create a set of resources that work together to create an application or solution. After the stack is created, the resources are started and are left running. Deleting the stack deletes all resources in the stack. CloudFormation is one of the tools that can be used to manage generic Infrastructure as Code stacks.

Caution

ThroughtWorks Technology Radar: AWS CloudFormation is a proprietary declarative language to provision AWS infrastructure as code. Handwriting CloudFormation files is often a default approach to bootstrap AWS infrastructure automation. Although this might be a sensible way to start a small project, our teams, and the industry at large, have found that handwritten CloudFormation simply does not scale as the infrastructure grows. Noticeable pitfalls of handwritten CloudFormation files for large projects include poor readability, lack of imperative constructs, limited parameter definition and usage, and lack of type checking. Addressing these shortfalls has led to a rich ecosystem of both open-source and custom tooling. We find Terraform a sensible default that not only addresses shortfalls of CloudFormation but also has an active community to add the latest AWS features and fix bugs. In addition to Terraform, you can choose from many other tools and languages:

• https://www.terraform.io
• https://github.com/cloudtools/troposphere
• https://github.com/cloudreach/sceptre
• https://github.com/capitalone/stack-deployment-tool
• https://pulumi.io

CloudFormation as AWS Service

CloudFormation is an AWS service, named "cloudformation.amazonaws.com".

Bootstrapping Application via WS CloudFormation

https://s3.amazonaws.com/cloudformation-examples/BoostrappingApplicationsWithAWSCloudFormation.pdf

Names

CloudFormation Name Type

Security

• Controlling access: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-template.html
• CloudFormation IAM Template Actions: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-template.html#using-iam-template-actions
• CloudFormation Service Role: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-servicerole.html

Logging

• Logging CloudFormation Calls: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-api-logging-cloudtrail.html

Stack

Working with Stacks

A stack is a collection of AWS resources that can be managed (create, update or delete) as a single unit. Also see Infrastructure as Code stack. A stack is an instantiation of a template: all resources in a stack are defined by the stack's AWS CloudFormation template. Because AWS CloudFormation treats the stack resources as a single unit, they must all be created or deleted successfully for the stack to be created or deleted. If a resource cannot be created, AWS CloudFormation rolls the stack back and automatically deletes any resources that were created. If a resource cannot be deleted, any remaining resources are retained until the stack can be successfully deleted. The instantiation process can be configured with input parameters. The creation process involves submitting the template to CloudFormation. CloudFormation makes the underlying service calls to AWS to provision and configure the resources. CloudFormation can only perform the actions the user has permissions for. After all resources have been created CloudFormation reports that the stack has been created. If the stack creation fails, CloudFormation rolls back the changes, by deleting the resources that have been created.

Stack Name

A stack name must contain only letters, numbers and dashes.

Stack States

• CREATE_IN_PROGRESS
• UPDATE_ROLLBACK_COMPLETE
• UPDATE_ROLLBACK_FAILED. See Continue Rolling Back an Update.

Stack Updates

To make changes to running resources in a stack, the stack can be updated by modifying the stack template, or providing different parameter values, and submitting the stack template and the new parameter values, if any, to CloudFormation with an operation like aws cloudformation update-stack. There is no need to create a new stack and delete the old one. An update operation creates a change set. The change set list proposed changes. CloudFormation compares submitted changes with the current state of the stack and updates only the changed resources. Resources that have not changed run without disruption during the update process.

If a resource has been updated, there are several update behaviors:

• Update with no interruption. CloudFormation updates the resource without disrupting operation of that resource and without changing the resource's physical ID. This is equivalent with reconfiguring the resource while the resource is running.
• Update with some interruption. CloudFormation updates the resource with some interruption and retains the physical ID. This is equivalent with rebooting the resource.
• Replacement CloudFormation recreates the resource during an update, which also generates a new physical ID. The replacement resource is created first, references from other dependent resources are changed to point to the replacement resource, and then the old resource is deleted.

Which behavior is employed depends on the type of configuration change, and it should be documented individually by the resource. See Resource Types.

If a resource that was specified in the previous version of the stack template is not present in the stack templated submitted to the update operation, the resource will be deleted after the update operation completes successfully. Updates can cause interruptions. If a stack update fails, CloudFormation rolls back the changes to restore the stack to the last known working state.

There are two methods for updating stacks:

• Direct Updates. In this case, the changes are submitted to CloudFormation and CloudFormation immediately deploys them. This is what aws cloudformation update-stack does.
• Creating and Executing Change Sets. In this case, you can preview the changes CloudFormation will make to your stack, and then decide whether to apply those changes. This method is preferred when you make sure CloudFormation does not make unintentional changes.

If the template includes one or more nested stacks, CloudFormation also initiates an update for every nested stack. However, CloudFormation updates only those resources in the nested stacks that have changes specified in corresponding templates.

The progress of an update operation can be monitoring by watching the console's events.

An update in progress can be canceled with aws cloudformation cancel-update-stack.

Change Set

A change set is a JSON-formatted document that summarize the changes CloudFormation will make to a stack.

Detailed instructions available here, to process:

• Updating Stacks Using Change Sets
• Creating a Change Set
• Viewing a Change Set
• Executing a Change Set
• Deleting a Change Set
• Example Change Sets

Stack Policy

Prevent Updates to Stack Resources

Nested Stack

AWS::CloudFormation::Stack

A nested stack is a stack created as part of other stacks. Nested stacks support modular infrastructure with CloudFormation. As the infrastructure grows, common patterns can emerge in which the same components are declared in the same way in multiple templates. These components can be separated out into dedicated templates. This way, different templates can be mixed and matched, but use nested stacks to create a single, unified stack. Nested stacks are stacks that create other stacks.

TODO: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-nested-stacks.html.

Cross-Stack References and Dependencies between Stacks

Outputs

Template

Learn Template Basics

Template Anatomy

AWS CloudFormation Template Formats

A template is a JSON or YAML file that contains declaration of AWS resources that make up a stack. It can be seen as a blueprint for building resources. The template describes what resources are needed, and AWS CloudFormation provisions those resources in an orderly and predictable fashion. Multiple resources can be specified and configured to work together, to create an application or solution. The resources are created in parallel where possible. AWS CloudFormation deals with failure and transient issues. Since templates are text files, they can be version controlled. CloudFormation takes care of checking references to resources in the template and also checks references to existing resources to ensure that they exist in the region where you are creating the stack. If your template refers to a dependent resource that does not exist, stack creation fails.

Template Examples

• Template samples per service
• Template snippets per service

Template Structure

Template Anatomy

---
AWSTemplateFormatVersion: "version date"

Description:
  <String>

Metadata:
  <template metadata>

Parameters:
  <set of parameters>

Mappings:
  <set of mappings>

Conditions:
  <set of conditions>

Transform:
  <set of transforms>

Resources:
  <set of resources - the only required top-level object>

Outputs:
  <set of outputs>

AWSTemplateFormatVersion

Format Version

AWSTemplateFormatVersion represents the version of the template format. If not specified, loudFormation will use the latest version.

AWSTemplateFormatVersion: '2010-09-09'

Input Parameters

Parameters

The input parameters are declared in a template's Parameters object. A parameter contains a list of attributes that define its value and constraints against its value. The only required attribute is Type, which can be a String, Number or an AWS-specific type.

Parameters:
  WebServerPort:
    Default: 8888
    Description: TCP/IP port for the WordPress web server
    Type: Number
    MinValue: 1
    MaxValue: 65535

For AWS-specific parameter types, AWS CloudFormation validates input values against existing values in the user's AWS account and in the region where he or she is creating the stack before creating any stack resources.

Type

String

The following attributes can be declared constraints: MinLength, MaxLength, Default, AllowedValues and AllowedPattern.

Number

The following attributes can be declared constraints: MinValue, MaxValue, Default and AllowedValue.

AWS-Specific Parameter Types

AWS-Specific Parameter Types

These include availability zone names, image IDs, instance IDs, security group names, subnet IDs, etc.

SSM Parameter Types

SSM Parameter Types

Also see:

AWS Systems Manager Parameter Store

Default

A default value for the parameter. Specifying a default value is optional, but if no "Default" is provided, the stack creation will fail unless a value is provided at the creation time.

Unfortunately, the Default value cannot refer other parameters, the declaration of "SomeOtherParam" parameter will be rejected as syntax error:

SomeParam:
  Type: String
   Default: 'something'

SomeOtherParam:
  Type: String
   Default: !Sub ${SomeParam}

Validation Constraints

• MinLength
• MaxLength
• Default
• AllowedValues
• AllowedPattern

Parameter Description

The parameter description is important, as it will show up in the dynamically-generated CloudFormation console (wizard) while creating the stack. Specifying examples and details is a good idea.

NoEcho

For sensitive information, the "NoEcho" attribute can be used to prevent a parameter value from being displayed in the console, command line tools, or API.

 GitHubPersonalAccessCode:
   Type: String
   NoEcho: true

Metadata

Metadata

Metadata section is optional, and provides information about the template itself. These details may include implementation details for specific resources.

Keys:

• AWS::CloudFormation::Init
• AWS::CloudFormation::Interface: Defines the grouping and ordering of input parameters when they are displayed in the AWS CloudFormation console. By default, the AWS CloudFormation console alphabetically sorts parameters by their logical ID.
• AWS::CloudFormation::Designer: describes how resources are laid out in the AWS CloudFormation Designer.

Conditions

Conditions

A condition is a top level template construct that evaluates to "true" or "false", and it is used to define the circumstances under which entities are created or configured. Conditions are useful when templates are reused, and those templates should create some resources in some contexts, and other resources in different context. The classical example is "production" versus "test" environments.

Conditions are declared in the "Conditions" section of the template. Each condition declaration has a logical ID, which is used to refer to the condition throughout the template, and a body that consists in condition functions that eventually evaluate to a final "true" or "false" value:

Conditions:
  ConditionLogicalID:
    conition-function-expression

Conditions:
  SomeConditions: !Equals [ 'something', !Ref SomeParameter ]

The conditions are referred, using their logical ID, in resource and output declarations, using the "Condition" key.

Outputs:
  SomeOutput:
     Condition: SomeCondition
     ...
Resources:
  SomeResource:
     Condition: SomeCondition
     ...

Example:

Parameters:
  SomeParameter:
    Type: String
    Default: blue

Conditions:
  CreateLogGroup: !Equals ['blue', !Ref SomeParameter]

Outputs:
  SomeOutput:
    Condition: CreateLogGroup
    Value: !Ref LogGroup
    Export:
      Name: log-group-name

Resources:
  LogGroup:
    Type: AWS::Logs::LogGroup
    Condition: CreateLogGroup
    Properties:
      LogGroupName: test-log-group

At stack creation or update, CloudFormation evaluates all the conditions in the template before creating resources.

On stack creation, if the condition evaluates to true, the resources guarded by the condition are created, and the outputs guarded by the condition are exported. If the condition evaluates to false, the resources are not created and the outputs are not exported. On stack update, the conditions are re-evaluated and resources that are still associated with true conditions are updated. Resources that become associated with false conditions are deleted.

Note that during a stack update, conditions cannot be updated by themselves. Conditions can be updated only when changes that add, modify or delete resources are also included.

More about condition functions is available here:

Condition Functions

Mappings

Mappings

More here. Mappings are logically similar to "switch" statements. Also see Fn::FindInMap

Resources

Resources

"Resources" is the only required top-level template element. It must contain at least one resource. However, even if Resources is required and must contain at least one resource, that does necessarily mean that the stack has to create resources. If resources are guarded by conditions and those conditions evaluate to false, no resources are created, and that is a perfectly valid outcome.

The resources are declared starting with their logical name.

Resources:
  ResourceName: # ... also known as Logical ID
    Type: AWS::ProductIdentifier::ResourceType
    Properties: ...

A resource must have a Type attribute, which defines the kind of AWS resource should be created.

AWS::ProductIdentifier::ResourceType

Full list of resource types:

AWS Resource and Property Types Reference

Resource declarations use a Properties attribute to specify the information used to create a resource. Resources are declared in template using logical names. When the resource is crated, a physical name is generated for it.

Resource Name, Logical ID

The name used to declare a resource definition within the template represents the logical name of that resource. It is also referred to as "resource name" or "logical ID". When AWS CloudFormation creates the resource, it generates a physical name that is based on the combination of the logical name, (sometimes) the stack name, and a unique ID. Examples of logical names to generated physical names mapping:

Resource Logical ID		Resource Physical ID
BuildBucket	→	stack-name-buildbucket-2a3et4c9f3bas
CodeBuildProject	→	CodeBuildProject-apCEy5I1KyH8
Pipeline	→	stack-name-Pipeline-24RCYXM52UE6A

Resource Physical ID

Upon successful creation, a resource will get a physical ID, which can be obtained from the "Resources" tab of the stack.

Resource Types

AWS CloudFormation Resource Types

DependsOn

DependsOn

The DependsOn attribute specifies that one resource must be created after another.

DeletionPolicy

DeletionPolicy Attribute

If a stack is deleted, but some of the resources are to be retained, a deletion policy can be used. Specifies how CloudFormation should handle the resource deletion.

Metadata

Metadata

The Metadata attribute specifies structured data with a resource.

Outputs

Outputs

Exporting Stack Output Values

Listing Stacks That Import an Exported Output Value

The "Output" section lists the stack's exports, or values associated with various elements of the stack that are needed in other stacks. The stack's outputs are also referred to as "stack properties". Once a dependent stack uses an exported value, a cross-stack reference is created, and the stack that uses the value becomes dependent on the stack that exports the value.

Outputs:
  OutputLogicalID:
    Description: Information about the value exported as "Export.Name"
    [Condition: ...]
    Value: value-to-return
    Export:
      Name: export-name

Outputs:
  CodeBuildServiceRoleArn:
    Description: The ARN of the CodeBuild service role created by this stack.
    Value: !GetAtt CodeBuildServiceRole.Arn
    Export:
      Name: elysium-codebuild-service-role-arn

Resources:
  CodeBuildServiceRole:
    Type: AWS::IAM::Role
    ...

Each output is identified in the declaring stack by its Output Logical ID. The declaration contains the exported Value, an Export Name and a Description. The value of an output can include literals, parameter references, pseudo-parameters, a mapping value, or intrinsic functions. The Export Name is the name used by other stacks to look up the exported value. The best logical representation of an Output is a key/value pair in a virtual map shared between all stacks within the same account and region. The exported Value is keyed by its Export Name. When the Fn::ImportValue function is used in a dependent stack, the exported Value is returned to the dependent stack. Once the stack is created, the Outputs can be inspected with:

aws cloudformation describe-stacks --stack-name stack-name

or by navigating to CloudFormation -> Stacks -> stack-name -> Outputs in AWS Console.

The cross-stack dependency is declared in the dependent stack by using the Fn::ImportValue function and passing the Export Name as argument, as follows:

Resources:
  CodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      ...
      ServiceRole:
        Fn::ImportValue: elysium-codebuild-service-role-arn

Once the cross-stack reference is established, the Export Name can be used to list the stack that depend on it:

aws cloudformation list-imports --export-name elysium-codebuild-service-role-arn

The result of the command is a list of stacks that depend on the specified export.

Stack dependency is useful when AWS resources are grouped based on lifecycle and ownership, and stacks are built in such a way that they use resources managed by another stack. These dependencies resources can be hard-coded in the dependent stack, or input parameters can be used to specify those. However, this makes the template difficult to reuse. The alternative is to export dependency resources and use cross-stack references. TO PROCESS: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-crossstackref.html

Cross-stack references cannot be created across regions - which probably means that the virtual map that maintains the exports is not replicated across regions. A stack cannot be deleted if stack references one of its outputs: if stack B uses stack A (relies on outputs produced by stack A), then stack A cannot be deleted:

Export us-west-2-CodeBuildServiceRole cannot be deleted as it is in use by B.

An output value that is referenced by another stack cannot be modified or removed.

Function

AWS CloudFormation has a number of intrinsic functions that can be used to refer to other resources and their properties.

Intrinsic Functions

Ref:, Fn::Join:, Fn::Split:, etc:

Intrinsic Functions

Condition Functions

Fn::And:, Fn::Equals:, Fn::If:, Fn::Not:, Fn::Or:

Condition Functions

Capabilities

Acknowledging IAM Resources in AWS CloudFormation Templates

Some stack templates might include resources that can affect permissions - for example, IAM users and roles. For those stacks, the capability of creating these resources must be explicitly acknowledged, by specifying one of these capabilities:

• CAPABILITY_IAM: the capability to create anonymous IAM resources.
• CAPABILITY_NAMED_IAM: the capability to create IAM resources with custom names.

Capabilities can be configured as command line options for CloudFormation commands. The capabilities are also relevant at the deploy stage of a CodePipeline release pipeline, if the deployment is delegated to CloudFormation.

Drift

TODO: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-stack-drift.html.

CloudFormation Service Role

AWS CloudFormation Service Role

If (and when) creating a service role, CloudFormation is identified as "cloudformation.amazonaws.com".

Variable Substitution

"Variables", in lack of a better name:

The following can be substituted in Strings with the Sub function.

Resource Logical ID

Resources:
  MyECRepository:
    ...

!Sub 'this will be replaced with the name of the ${MyECRepository}'

Also see, above: Resource Name, Logical ID.

Pseudo-Parameters

Pseudo-parameters are names CloudFormation resolves implicitly, based on the context the template is processed within, when the stack is created. The current value of those pseudo-parameters can be obtained in the template with:

!Sub ${PseudoParameterName}

or

!Ref PseudoParameterName

Stack Policy

A stack policy is a JSON document that describes what update actions can be performed on designated resources.

TO PROCESS: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/protect-stack-resources.html

Continuous Delivery with CloudFormation and CodePipeline

CodePipeline Concepts - Continuous Delivery with CloudFormation and CodePipeline

Amazon API Gateway Deployment with CloudFormation

Amazon API Gateway Deployment with CloudFormation

Best Practices

CloudFormation Best Practices

Handling Dates in Templates

https://stackoverflow.com/a/15649630/7817789

Recipes

• Obtaining the ARN of a resource created by the template with GetAtt or with Sub.