AWS CloudFormation Concepts: Difference between revisions
| Line 12: | Line 12: | ||
CloudFormation models, configures and set up [[#Resource|resources]], grouped in [[#Stack|stacks]]. The resources and their high-level dependencies are declared in [[#Template|templates]]. A template describes desired state, not a set of operations to perform. If a resource is defined in the template it will be created. If a resource already exists, it will not be created, but can be updated if its properties change. If a resource is removed from the template it will be deleted. One of the greatest benefits of templates and AWS CloudFormation is the ability to create a set of resources that work together to create an application or solution. After the stack is created, the resources are started and are left running. Deleting the stack deletes all resources in the stack. | CloudFormation models, configures and set up [[#Resource|resources]], grouped in [[#Stack|stacks]]. The resources and their high-level dependencies are declared in [[#Template|templates]]. A template describes desired state, not a set of operations to perform. If a resource is defined in the template it will be created. If a resource already exists, it will not be created, but can be updated if its properties change. If a resource is removed from the template it will be deleted. One of the greatest benefits of templates and AWS CloudFormation is the ability to create a set of resources that work together to create an application or solution. After the stack is created, the resources are started and are left running. Deleting the stack deletes all resources in the stack. | ||
=Caution= | |||
ThroughtWorks Technology Radar: AWS CloudFormation is a proprietary declarative language to provision AWS infrastructure as code. Handwriting CloudFormation files is often a default approach to bootstrap AWS infrastructure automation. Although this might be a sensible way to start a small project, our teams, and the industry at large, have found that handwritten CloudFormation simply does not scale as the infrastructure grows. Noticeable pitfalls of handwritten CloudFormation files for large projects include poor readability, lack of imperative constructs, limited parameter definition and usage, and lack of type checking. Addressing these shortfalls has led to a rich ecosystem of both open-source and custom tooling. We find Terraform a sensible default that not only addresses shortfalls of CloudFormation but also has an active community to add the latest AWS features and fix bugs. In addition to Terraform, you can choose from many other tools and languages: | |||
* https://www.terraform.io | |||
* https://github.com/cloudtools/troposphere | |||
* https://github.com/cloudreach/sceptre | |||
* https://github.com/capitalone/stack-deployment-tool | |||
* https://pulumi.io | |||
=CloudFormation as AWS Service= | =CloudFormation as AWS Service= | ||
Revision as of 20:43, 9 May 2019
External
Internal
Overview
CloudFormation models, configures and set up resources, grouped in stacks. The resources and their high-level dependencies are declared in templates. A template describes desired state, not a set of operations to perform. If a resource is defined in the template it will be created. If a resource already exists, it will not be created, but can be updated if its properties change. If a resource is removed from the template it will be deleted. One of the greatest benefits of templates and AWS CloudFormation is the ability to create a set of resources that work together to create an application or solution. After the stack is created, the resources are started and are left running. Deleting the stack deletes all resources in the stack.
Caution
ThroughtWorks Technology Radar: AWS CloudFormation is a proprietary declarative language to provision AWS infrastructure as code. Handwriting CloudFormation files is often a default approach to bootstrap AWS infrastructure automation. Although this might be a sensible way to start a small project, our teams, and the industry at large, have found that handwritten CloudFormation simply does not scale as the infrastructure grows. Noticeable pitfalls of handwritten CloudFormation files for large projects include poor readability, lack of imperative constructs, limited parameter definition and usage, and lack of type checking. Addressing these shortfalls has led to a rich ecosystem of both open-source and custom tooling. We find Terraform a sensible default that not only addresses shortfalls of CloudFormation but also has an active community to add the latest AWS features and fix bugs. In addition to Terraform, you can choose from many other tools and languages:
- https://www.terraform.io
- https://github.com/cloudtools/troposphere
- https://github.com/cloudreach/sceptre
- https://github.com/capitalone/stack-deployment-tool
- https://pulumi.io
CloudFormation as AWS Service
CloudFormation is an AWS service, named "cloudformation.amazonaws.com".
Bootstrapping Application via WS CloudFormation
Names
Security
- Controlling access: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-template.html
- CloudFormation IAM Template Actions: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-template.html#using-iam-template-actions
- CloudFormation Service Role: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-servicerole.html
Logging
- Logging CloudFormation Calls: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-api-logging-cloudtrail.html
Stack
A stack is a collection of AWS resources that can be managed (create, update or delete) as a single unit. A stack is an instantiation of a template: all resources in a stack are defined by the stack's AWS CloudFormation template. Because AWS CloudFormation treats the stack resources as a single unit, they must all be created or deleted successfully for the stack to be created or deleted. If a resource cannot be created, AWS CloudFormation rolls the stack back and automatically deletes any resources that were created. If a resource cannot be deleted, any remaining resources are retained until the stack can be successfully deleted. The instantiation process can be configured with input parameters. The creation process involves submitting the template to CloudFormation. CloudFormation makes the underlying service calls to AWS to provision and configure the resources. CloudFormation can only perform the actions the user has permissions for. After all resources have been created CloudFormation reports that the stack has been created. If the stack creation fails, CloudFormation rolls back the changes, by deleting the resources that have been created.
Stack Name
A stack name must contain only letters, numbers and dashes.
Stack States
- CREATE_IN_PROGRESS
- UPDATE_ROLLBACK_COMPLETE
- UPDATE_ROLLBACK_FAILED. See Continue Rolling Back an Update.
Stack Updates
To make changes to running resources in a stack, the stack can be updated by modifying the stack template, or providing different parameter values, and submitting the stack template and the new parameter values, if any, to CloudFormation with an operation like aws cloudformation update-stack. There is no need to create a new stack and delete the old one. An update operation creates a change set. The change set list proposed changes. CloudFormation compares submitted changes with the current state of the stack and updates only the changed resources. Resources that have not changed run without disruption during the update process.
If a resource has been updated, there are several update behaviors:
- Update with no interruption. CloudFormation updates the resource without disrupting operation of that resource and without changing the resource's physical ID. This is equivalent with reconfiguring the resource while the resource is running.
- Update with some interruption. CloudFormation updates the resource with some interruption and retains the physical ID. This is equivalent with rebooting the resource.
- Replacement CloudFormation recreates the resource during an update, which also generates a new physical ID. The replacement resource is created first, references from other dependent resources are changed to point to the replacement resource, and then the old resource is deleted.
Which behavior is employed depends on the type of configuration change, and it should be documented individually by the resource. See Resource Types.
If a resource that was specified in the previous version of the stack template is not present in the stack templated submitted to the update operation, the resource will be deleted after the update operation completes successfully. Updates can cause interruptions. If a stack update fails, CloudFormation rolls back the changes to restore the stack to the last known working state.
There are two methods for updating stacks:
- Direct Updates. In this case, the changes are submitted to CloudFormation and CloudFormation immediately deploys them. This is what aws cloudformation update-stack does.
- Creating and Executing Change Sets. In this case, you can preview the changes CloudFormation will make to your stack, and then decide whether to apply those changes. This method is preferred when you make sure CloudFormation does not make unintentional changes.
If the template includes one or more nested stacks, CloudFormation also initiates an update for every nested stack. However, CloudFormation updates only those resources in the nested stacks that have changes specified in corresponding templates.
The progress of an update operation can be monitoring by watching the console's events.
An update in progress can be canceled with aws cloudformation cancel-update-stack.
Change Set
A change set is a JSON-formatted document that summarize the changes CloudFormation will make to a stack.
Detailed instructions available here, to process:
- Updating Stacks Using Change Sets
- Creating a Change Set
- Viewing a Change Set
- Executing a Change Set
- Deleting a Change Set
- Example Change Sets
UpdatePolicy
Stack Policy
Nested Stack
A nested stack is a stack created as part of other stacks. As the infrastructure grows, common patterns can emerge in which the same components are declared in the same way in multiple templates. These components can be separated out into dedicated templates. This way, different templates can be mixed and matched, but use nested stacks to create a single, unified stack. Nested stacks are stacks that create other stacks.
TODO: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-nested-stacks.html.
Root Stack
Cross-Stack References and Dependencies between Stacks
Template
A template is a JSON or YAML file that contains declaration of AWS resources that make up a stack. It can be seen as a blueprint for building resources. The template describes what resources are needed, and AWS CloudFormation provisions those resources in an orderly and predictable fashion. Multiple resources can be specified and configured to work together, to create an application or solution. The resources are created in parallel where possible. AWS CloudFormation deals with failure and transient issues. Since templates are text files, they can be version controlled. CloudFormation takes care of checking references to resources in the template and also checks references to existing resources to ensure that they exist in the region where you are creating the stack. If your template refers to a dependent resource that does not exist, stack creation fails.
Template Examples
Template Structure
---
AWSTemplateFormatVersion: "version date"
Description:
<String>
Metadata:
<template metadata>
Parameters:
<set of parameters>
Mappings:
<set of mappings>
Conditions:
<set of conditions>
Transform:
<set of transforms>
Resources:
<set of resources - the only required top-level object>
Outputs:
<set of outputs>
AWSTemplateFormatVersion
AWSTemplateFormatVersion represents the version of the template format. If not specified, loudFormation will use the latest version.
AWSTemplateFormatVersion: '2010-09-09'
Description
Input Parameters
The input parameters are declared in a template's Parameters object. A parameter contains a list of attributes that define its value and constraints against its value. The only required attribute is Type, which can be a String, Number or an AWS-specific type.
Parameters:
WebServerPort:
Default: 8888
Description: TCP/IP port for the WordPress web server
Type: Number
MinValue: 1
MaxValue: 65535
For AWS-specific parameter types, AWS CloudFormation validates input values against existing values in the user's AWS account and in the region where he or she is creating the stack before creating any stack resources.
Type
String
The following attributes can be declared constraints: MinLength, MaxLength, Default, AllowedValues and AllowedPattern.
Number
The following attributes can be declared constraints: MinValue, MaxValue, Default and AllowedValue.
List<Number>
CommaDelimitedList
AWS-Specific Parameter Types
These include availability zone names, image IDs, instance IDs, security group names, subnet IDs, etc.
SSM Parameter Types
Default
A default value for the parameter. Specifying a default value is optional, but if no "Default" is provided, the stack creation will fail unless a value is provided at the creation time.
Unfortunately, the Default value cannot refer other parameters, the declaration of "SomeOtherParam" parameter will be rejected as syntax error:
SomeParam:
Type: String
Default: 'something'
SomeOtherParam:
Type: String
Default: !Sub ${SomeParam}
Validation Constraints
- MinLength
- MaxLength
- Default
- AllowedValues
- AllowedPattern
Parameter Description
The parameter description is important, as it will show up in the dynamically-generated CloudFormation console (wizard) while creating the stack. Specifying examples and details is a good idea.
NoEcho
For sensitive information, the "NoEcho" attribute can be used to prevent a parameter value from being displayed in the console, command line tools, or API.
GitHubPersonalAccessCode: Type: String NoEcho: true
Metadata
Metadata section is optional, and provides information about the template itself. These details may include implementation details for specific resources.
Keys:
- AWS::CloudFormation::Init
- AWS::CloudFormation::Interface: Defines the grouping and ordering of input parameters when they are displayed in the AWS CloudFormation console. By default, the AWS CloudFormation console alphabetically sorts parameters by their logical ID.
- AWS::CloudFormation::Designer: describes how resources are laid out in the AWS CloudFormation Designer.
Conditions
A condition is a top level template construct that evaluates to "true" or "false", and it is used to define the circumstances under which entities are created or configured. Conditions are useful when templates are reused, and those templates should create some resources in some contexts, and other resources in different context. The classical example is "production" versus "test" environments.
Conditions are declared in the "Conditions" section of the template. Each condition declaration has a logical ID, which is used to refer to the condition throughout the template, and a body that consists in condition functions that eventually evaluate to a final "true" or "false" value:
Conditions:
ConditionLogicalID:
conition-function-expression
Conditions: SomeConditions: !Equals [ 'something', !Ref SomeParameter ]
The conditions are referred, using their logical ID, in resource and output declarations, using the "Condition" key.
Outputs:
SomeOutput:
Condition: SomeCondition
...
Resources:
SomeResource:
Condition: SomeCondition
...
Example:
Parameters:
SomeParameter:
Type: String
Default: blue
Conditions:
CreateLogGroup: !Equals ['blue', !Ref SomeParameter]
Outputs:
SomeOutput:
Condition: CreateLogGroup
Value: !Ref LogGroup
Export:
Name: log-group-name
Resources:
LogGroup:
Type: AWS::Logs::LogGroup
Condition: CreateLogGroup
Properties:
LogGroupName: test-log-group
At stack creation or update, CloudFormation evaluates all the conditions in the template before creating resources.
On stack creation, if the condition evaluates to true, the resources guarded by the condition are created, and the outputs guarded by the condition are exported. If the condition evaluates to false, the resources are not created and the outputs are not exported. On stack update, the conditions are re-evaluated and resources that are still associated with true conditions are updated. Resources that become associated with false conditions are deleted.
Note that during a stack update, conditions cannot be updated by themselves. Conditions can be updated only when changes that add, modify or delete resources are also included.
More about condition functions is available here:
Mappings
More here. Mappings are logically similar to "switch" statements. Also see Fn::FindInMap
Resources
"Resources" is the only required top-level template element. It must contain at least one resource. However, even if Resources is required and must contain at least one resource, that does necessarily mean that the stack has to create resources. If resources are guarded by conditions and those conditions evaluate to false, no resources are created, and that is a perfectly valid outcome.
The resources are declared starting with their logical name.
Resources:
ResourceName: # ... also known as Logical ID
Type: AWS::ProductIdentifier::ResourceType
Properties: ...
A resource must have a Type attribute, which defines the kind of AWS resource should be created.
AWS::ProductIdentifier::ResourceType
Full list of resource types:
Resource declarations use a Properties attribute to specify the information used to create a resource. Resources are declared in template using logical names. When the resource is crated, a physical name is generated for it.
Resource Name, Logical ID
The name used to declare a resource definition within the template represents the logical name of that resource. It is also referred to as "resource name" or "logical ID". When AWS CloudFormation creates the resource, it generates a physical name that is based on the combination of the logical name, (sometimes) the stack name, and a unique ID. Examples of logical names to generated physical names mapping:
| Resource Logical ID | Resource Physical ID | |
| BuildBucket | → | stack-name-buildbucket-2a3et4c9f3bas |
| CodeBuildProject | → | CodeBuildProject-apCEy5I1KyH8 |
| Pipeline | → | stack-name-Pipeline-24RCYXM52UE6A |
Resource Physical ID
Upon successful creation, a resource will get a physical ID, which can be obtained from the "Resources" tab of the stack.
Resource Types
DependsOn
The DependsOn attribute specifies that one resource must be created after another.
DeletionPolicy
If a stack is deleted, but some of the resources are to be retained, a deletion policy can be used. Specifies how CloudFormation should handle the resource deletion.
Metadata
The Metadata attribute specifies structured data with a resource.
Outputs
The "Output" section lists the stack's exports, or values associated with various elements of the stack that are needed in other stacks. The stack's outputs are also referred to as "stack properties". Once a dependent stack uses an exported value, a cross-stack reference is created, and the stack that uses the value becomes dependent on the stack that exports the value.
Outputs:
OutputLogicalID:
Description: Information about the value exported as "Export.Name"
[Condition: ...]
Value: value-to-return
Export:
Name: export-name
Outputs:
CodeBuildServiceRoleArn:
Description: The ARN of the CodeBuild service role created by this stack.
Value: !GetAtt CodeBuildServiceRole.Arn
Export:
Name: elysium-codebuild-service-role-arn
Resources:
CodeBuildServiceRole:
Type: AWS::IAM::Role
...
Each output is identified in the declaring stack by its Output Logical ID. The declaration contains the exported Value, an Export Name and a Description. The value of an output can include literals, parameter references, pseudo-parameters, a mapping value, or intrinsic functions. The Export Name is the name used by other stacks to look up the exported value. The best logical representation of an Output is a key/value pair in a virtual map shared between all stacks within the same account and region. The exported Value is keyed by its Export Name. When the Fn::ImportValue function is used in a dependent stack, the exported Value is returned to the dependent stack. Once the stack is created, the Outputs can be inspected with:
aws cloudformation describe-stacks --stack-name stack-name
or by navigating to CloudFormation -> Stacks -> stack-name -> Outputs in AWS Console.
The cross-stack dependency is declared in the dependent stack by using the Fn::ImportValue function and passing the Export Name as argument, as follows:
Resources:
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
...
ServiceRole:
Fn::ImportValue: elysium-codebuild-service-role-arn
Once the cross-stack reference is established, the Export Name can be used to list the stack that depend on it:
aws cloudformation list-imports --export-name elysium-codebuild-service-role-arn
The result of the command is a list of stacks that depend on the specified export.
Stack dependency is useful when AWS resources are grouped based on lifecycle and ownership, and stacks are built in such a way that they use resources managed by another stack. These dependencies resources can be hard-coded in the dependent stack, or input parameters can be used to specify those. However, this makes the template difficult to reuse. The alternative is to export dependency resources and use cross-stack references. TO PROCESS: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-crossstackref.html
Cross-stack references cannot be created across regions - which probably means that the virtual map that maintains the exports is not replicated across regions. A stack cannot be deleted if stack references one of its outputs: if stack B uses stack A (relies on outputs produced by stack A), then stack A cannot be deleted:
Export us-west-2-CodeBuildServiceRole cannot be deleted as it is in use by B.
An output value that is referenced by another stack cannot be modified or removed.
Function
AWS CloudFormation has a number of intrinsic functions that can be used to refer to other resources and their properties.
Intrinsic Functions
Ref:, Fn::Join:, Fn::Split:, etc:
Condition Functions
Fn::And:, Fn::Equals:, Fn::If:, Fn::Not:, Fn::Or:
Capabilities
Some stack templates might include resources that can affect permissions - for example, IAM users and roles. For those stacks, the capability of creating these resources must be explicitly acknowledged, by specifying one of these capabilities:
- CAPABILITY_IAM: the capability to create anonymous IAM resources.
- CAPABILITY_NAMED_IAM: the capability to create IAM resources with custom names.
Capabilities can be configured as command line options for CloudFormation commands. The capabilities are also relevant at the deploy stage of a CodePipeline release pipeline, if the deployment is delegated to CloudFormation.
Drift
TODO: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-stack-drift.html.
CloudFormation Designer
CloudFormation Service Role
If (and when) creating a service role, CloudFormation is identified as "cloudformation.amazonaws.com".
Variable Substitution
"Variables", in lack of a better name:
The following can be substituted in Strings with the Sub function.
Input Parameters (Template Parameters)
Resource Logical ID
Resources:
MyECRepository:
...
!Sub 'this will be replaced with the name of the ${MyECRepository}'
Also see, above: Resource Name, Logical ID.
Resource Attributes
Pseudo-Parameters
Pseudo-parameters are names CloudFormation resolves implicitly, based on the context the template is processed within, when the stack is created. The current value of those pseudo-parameters can be obtained in the template with:
!Sub ${PseudoParameterName}
or
!Ref PseudoParameterName
AWS::StackName
AWS::Region
AWS::AccountId
Stack Policy
A stack policy is a JSON document that describes what update actions can be performed on designated resources.
TO PROCESS: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/protect-stack-resources.html