Simple GitHub Simulated Shell Build Simulated Deployment AWS CodePipeline Pipeline
Internal
Overview
thalarion: an example of a simple pipeline to be created with CloudFormation, and that reads code from a GitHub repository, applies a trivial "build" transformation and "deploys" the final artifacts via a CloudFormation project-embedded stack.
Example
Prerequisites
- The example requires a GitHub repository to be available. We'll use https://github.com/ovidiuf/aws-pipeline-source-example. The repository contains buildspec metadata, that drives the build, and a CloudFormation stack template, which will drive the deployment.
- The CodeBuild, CodePipeline and CloudFormation service roles, required by the build service, which performs the build, the CloudFormation service, which performs the deployment, and CodePipeline service, which drives both of them, must be created in advanced and referred from the CloudFormation pipeline stack specification by their ARN, or by reference. I tried creating them as part of the same stack, but I got: "CodeBuild is not authorized to perform: sts:AssumeRole on ...". If they exist when the stack creation is attempted, it works. TODO: try to declare them in the same stack, experiment with dependencies, try to make this work.. An auxiliary CloudFormation stack that creates those roles is available here:
aws cloudformation create-stack --capabilities CAPABILITY_NAMED_IAM --stack-name thalarion-release-pipeline-prerequisites --template-body file://./thalarion-release-pipeline-prerequisites.yml
Procedure
CodeFormation Release Pipeline Stack
The CodePipeline pipeline, the delegate CodeBuild project, and the S3 bucket to keep the artifacts produced by the pipeline and the ECR repository that will hold images produced by the project will be created as part of one CodeFormation stack:
AWSTemplateFormatVersion: '2010-09-09'
Description: "A release pipeline stack. The service roles must be created in advance."
Parameters:
ProjectID:
Type: String
Default: 'thalarion'
Description: 'The ID of the project we create the release pipeline for, will be used as prefix for names of all created resources.'
GitHubRepositoryName:
Type: String
Default: 'aws-pipeline-source-example'
Description: "The GitHub repository name to pull this component from. Only the name of the repository should be specified here, the full URL will be assembled from GitHubOrganizationID"
GitHubOrganizationID:
Type: String
Default: 'ovidiuf'
Description: "The GitHub organization ID"
GitHubPersonalAccessCode:
Type: String
Default: 'Provide a GitHub Personal Access Code'
Branch:
Type: String
Default: master
Description: "The GitHub branch of the specified repository to pull from."
Buildspec:
Type: String
Default: buildspec.yml
Description: "The name of the CodeBuild buildspec file that should be present in the root of the source repository and must contain build instructions."
DeploymentStackTemplate:
Type: String
Default: deployment-stack.yml
Description: "The name of the CloudFormation stack template that should drive the deployment"
Resources:
EcrRepository:
Type: AWS::ECR::Repository
Properties:
RepositoryName: !Ref ProjectID
BuildBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub '${ProjectID}-release-pipeline'
AccessControl: BucketOwnerFullControl
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Name: !Ref ProjectID
Source:
Type: CODEPIPELINE
BuildSpec: !Ref Buildspec
Artifacts:
Type: CODEPIPELINE
ServiceRole:
Fn::ImportValue: !Sub '${ProjectID}-codebuild-service-role-ARN'
TimeoutInMinutes: 20
Environment:
Type: LINUX_CONTAINER
ComputeType: BUILD_GENERAL1_SMALL
Image: 'aws/codebuild/java:openjdk-8'
PrivilegedMode: true
EnvironmentVariables:
- Name: AWS_ACCOUNT_ID
Value: !Ref AWS::AccountId
- Name: PROJECT_ID
Value: !Ref ProjectID
- Name: ECR_REPOSITORY_URI
Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${EcrRepository}
#
# 'BUILD_BUCKET' and 'AWS_PIPELINE_NAME' will allow the CodeBuild to write content info
# the pipeline's S3 "folder".
#
- Name: BUILD_BUCKET
Value: !Ref BuildBucket
- Name: AWS_PIPELINE_NAME
Value: !Ref ProjectID
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: !Ref ProjectID
RoleArn:
Fn::ImportValue: !Sub '${ProjectID}-codepipeline-service-role-ARN'
ArtifactStore:
Type: S3
Location: !Ref BuildBucket
RestartExecutionOnUpdate: true
Stages:
- Name: Source
Actions:
- Name: !Sub 'github-pull-${Branch}'
ActionTypeId:
Category: Source
Provider: GitHub
Owner: ThirdParty
Version: '1'
Configuration:
Owner: !Ref GitHubOrganizationID
Repo: !Ref GitHubRepositoryName
Branch: !Ref Branch
OAuthToken: !Ref GitHubPersonalAccessCode
InputArtifacts: []
OutputArtifacts:
- Name: RepositorySourceTree
RunOrder: 1
- Name: Build
Actions:
- Name: !Sub '${Buildspec}-driven-CodeBuild'
ActionTypeId:
Category: Build
Provider: CodeBuild
Owner: AWS
Version: '1'
InputArtifacts:
- Name: RepositorySourceTree
OutputArtifacts:
- Name: BuildspecProducedFiles
Configuration:
ProjectName: !Ref CodeBuildProject
RunOrder: 1
- Name: Deploy
Actions:
- Name: !Sub '${DeploymentStackTemplate}-driven-deployment'
ActionTypeId:
Category: Deploy
Provider: CloudFormation
Owner: AWS
Version: '1'
InputArtifacts:
- Name: RepositorySourceTree
- Name: BuildspecProducedFiles
OutputArtifacts: []
Configuration:
StackName: !Ref ProjectID
TemplatePath: !Sub RepositorySourceTree::${DeploymentStackTemplate}
TemplateConfiguration: BuildspecProducedFiles::deployment-stack-config.json
ParameterOverrides: !Sub '{"BuildBucket": "${BuildBucket}"}'
ActionMode: CREATE_UPDATE
Capabilities: CAPABILITY_IAM
RoleArn:
Fn::ImportValue: !Sub '${ProjectID}-cloudformation-service-role-ARN'
RunOrder: 1
aws cloudformation create-stack --stack-name thalarion-release-pipeline --template-body file://.thalarion-release-pipeline.yml --parameters ParameterKey=GitHubPersonalAccessCode,ParameterValue=...
Buildspec
The GitHub repository should expose a builspec.yml in root. A simple example is available here:
Playground example:
CloudFormation Deployment Stack Template
Playground example:
The GitHub repository should expose a CloudFormation deployment stack template, which will be used by CloudFormation in the "deploy" stage of the pipeline to perform the deployment. This is a simple example:
AWSTemplateFormatVersion: '2010-09-09'
Description: "CloudFormation deployment template, will drive the deployment as part of this project's release pipeline."
Parameters:
#
# these must be set in the deployment stack configuration file, produced by a prior pipeline step, or by the "ParameterOverrides" configuration.
#
BuildBucket:
Type: String
MyConfigurationParameterA:
Type: String
MyConfigurationParameterB:
Type: String
Resources:
ServiceLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub '/playground/${MyConfigurationParameterA}-${MyConfigurationParameterB}'
RetentionInDays: 1