Simple GitHub Simulated Shell Build Simulated Deployment AWS CodePipeline Pipeline

From NovaOrdis Knowledge Base
Jump to navigation Jump to search

Internal

Overview

An example of a simple pipeline to be created with CloudFormation, and that reads code from a GitHub repository, applies a trivial "build" transformation and "deploys" the final artifacts via a CloudFormation project-embedded stack.

Prerequisites

  • The example requires a GitHub repository to be available. We'll use https://github.com/ovidiuf/aws-pipeline-source-example. The repository contains buildspec metadata, that drives the build, and a CloudFormation stack template, which will drive the deployment.
  • The CodeBuild, CodePipeline and CloudFormation service roles, required by the build service, which performs the build, the CloudFormation service, which performs the deployment, and CodePipeline service, which drives both of them, must be created in advanced and referred from the CloudFormation pipeline stack specification by their ARN, or by reference. I tried creating them as part of the same stack, but I got: "CodeBuild is not authorized to perform: sts:AssumeRole on ...". If they exist when the stack creation is attempted, it works. TODO: try to declare them in the same stack, experiment with dependencies, try to make this work.. An auxiliary CloudFormation stack that creates those roles is available here:
thalarion-release-pipeline-prerequisites.yml

Procedure

CodeFormation Release Pipeline Stack

The CodePipeline pipeline, the delegate CodeBuild project, and the S3 bucket to keep the artifacts produced by the pipeline and the ECR repository that will hold images produced by the project will be created as part of one CodeFormation stack: 

AWSTemplateFormatVersion: '2010-09-09'

Description: "A release pipeline stack. The service roles must be created in advance."

Parameters:

  ProjectID:
    Type: String
    Default: 'thalarion'
    Description: 'The ID of the project we create the release pipeline for, will be used as prefix for names of all created resources.'

  GitHubRepositoryName:
    Type: String
    Default: 'aws-pipeline-source-example'
    Description: "The GitHub repository name to pull this component from. Only the name of the repository should be specified here, the full URL will be assembled from GitHubOrganizationID"

  GitHubOrganizationID:
    Type: String
    Default: 'ovidiuf'
    Description: "The GitHub organization ID"

  GitHubPersonalAccessCode:
    Type: String
    Default: '*****'

  Branch:
    Type: String
    Default: master
    Description: "The GitHub branch of the specified repository to pull from."

  Buildspec:
    Type: String
    Default: buildspec.yml
    Description: "The name of the CodeBuild buildspec file that should be present in the root of the source repository and must contain build instructions."

  DeploymentStackTemplate:
    Type: String
    Default: cloudformation-deployment-stack.yml
    Description: "The name of the CloudFormation template that should drive the deployment"

Resources:

  EcrRepository:
    Type: AWS::ECR::Repository
    Properties:
      RepositoryName: !Ref ProjectID

  BuildBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub '${ProjectID}-release-pipeline'
      AccessControl: BucketOwnerFullControl

  CodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Ref ProjectID
      Source:
        Type: CODEPIPELINE
        BuildSpec: !Ref Buildspec
      Artifacts:
        Type: CODEPIPELINE
      ServiceRole:
        Fn::ImportValue: !Sub '${ProjectID}-codebuild-service-role-ARN'
      TimeoutInMinutes: 20
      Environment:
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_SMALL
        Image: 'aws/codebuild/java:openjdk-8'
        PrivilegedMode: true
        EnvironmentVariables:
          - Name: AWS_ACCOUNT_ID
            Value: !Ref AWS::AccountId
          - Name: PROJECT_ID
            Value: !Ref ProjectID
          - Name: ECR_REPOSITORY_URI
            Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${EcrRepository}

  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      Name: !Ref ProjectID
      RoleArn:
        Fn::ImportValue: !Sub '${ProjectID}-codepipeline-service-role-ARN'
      ArtifactStore:
        Type: S3
        Location: !Ref BuildBucket
      RestartExecutionOnUpdate: true
      Stages:
        - Name: Source
          Actions:
            - Name: !Sub 'github-pull-${Branch}'
              ActionTypeId:
                Category: Source
                Owner: ThirdParty
                Version: '1'
                Provider: GitHub
              Configuration:
                Owner: !Ref GitHubOrganizationID
                Repo: !Ref GitHubRepositoryName
                Branch: !Ref Branch
                OAuthToken: !Ref GitHubPersonalAccessCode
              InputArtifacts: []
              OutputArtifacts:
                - Name: 'sources'
              RunOrder: 1
        - Name: Build
          Actions:
            - Name: !Sub '${Buildspec}-driven-CodeBuild'
              ActionTypeId:
                Category: Build
                Owner: AWS
                Version: '1'
                Provider: CodeBuild
              InputArtifacts:
                - Name: 'sources'
              OutputArtifacts:
                - Name: 'build'
              Configuration:
                ProjectName: !Ref CodeBuildProject
              RunOrder: 1
        - Name: Deploy
          Actions:
            - Name: !Sub '${DeploymentStackTemplate}-driven-deployment'
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Version: '1'
                Provider: CloudFormation
              InputArtifacts:
                - Name: 'sources'
                - Name: 'build'
              OutputArtifacts: []
              Configuration:
                StackName: !Ref ProjectID
                TemplatePath: !Sub sources::${DeploymentStackTemplate}
                TemplateConfiguration: build::overrides.json
                ParameterOverrides: !Sub '{ "EcrRepository": "${EcrRepository}" }'
                ActionMode: CREATE_UPDATE
                Capabilities: CAPABILITY_IAM
                RoleArn:
                  Fn::ImportValue: !Sub '${ProjectID}-cloudformation-service-role-ARN'
              RunOrder: 1

Buildspec

The GitHub repository should expose a builspec.yml in root. A simple example is available here:

buildspec.yml Example

CloudFormation Deployment Stack Template

The GitHub repository should expose a CloudFormation deployment stack template, which will be used by CloudFormation in the "deploy" stage of the pipeline to perform the deployment. This is a simple example: 

AWSTemplateFormatVersion: '2010-09-09'

Description: "CloudFormation deployment template, will drive the deployment as part of this project's release pipeline."

Parameters:

  EcrRepository:
    Type: String

Resources:

  ServiceLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub '/playground/${EcrRepository}'
      RetentionInDays: 1
Retrieved from "https://kb.novaordis.com/index.php?title=Simple_GitHub_Simulated_Shell_Build_Simulated_Deployment_AWS_CodePipeline_Pipeline&oldid=56312"