AWS CodePipeline Concepts
External
- https://docs.aws.amazon.com/codepipeline/latest/userguide/concepts.html
- CreatePipeline API Request Reference
Internal
CodePipeline as AWS Service
CodePipeline is an AWS service, named "codepipeline.amazonaws.com".
Pipeline
A pipeline is a top-level AWS resource that provides CI/CD release pipeline functionality. It consist in a set of sequential stages, each stage containing one or more actions. A specific stage is always in a fixed position relative to other stages. However, actions within a stage can be executed sequentially, according their run order or in parallel. Stages and actions process artifacts, which "advance" along the pipeline. A pipeline can be created the following CloudFormation sequence:
Resources:
Pipeline:
Name: !Ref AWS::StackName
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: 'arn:aws:iam::777777777777:role/CodePipelineServiceRole-1'
ArtifactStore
Type: 'S3'
Location: 'experimental-s3-bucket-for-codepipeline'
...
Stages:
...
An example of a simple GitHub-based pipeline is available here:
Required Configuration
The pipeline requires a number of configuration properties:
RoleArn
The pipeline needs to be associated with a service role, which allows the codepipeline service to execute various actions required by pipeline operations.
ArtifactStore
The pipeline requires an artifact store, which provides the storage for transient and final artifacts that are processed by the various stages and actions. In most cases, the storage is provided by an Amazon S3 bucket. "Location" specifies the name of the bucket. When the pipeline is initialized, the codepipeline service creates a directory associated with the pipeline. The directory will have the same name as the pipeline. As the pipeline operates, sub-directories corresponding to various input and output artifacts declared by actions will be also created.
Optional Configuration
Optionally, a name can also be configured with a name:
Name
Optional parameter, that provides the physical ID for the pipeline. If not specified, a name will be generated, based on the value of the CloudFormation stack name. Recommended value:
Name: !Ref AWS::StackName
Stage
A pipeline must have at least 2 stages, one-stage pipeline will be considered invalid.
Action
An action is a task performed on an artifact, executed as part of the sequence in the stage of a pipeline. The action may occur in a specified order, or in parallel, depending on their configuration.
Syntactic and semantic details about actions can be found in:
Action Provider
Action Specification
Action Name
An action name must match the regular expression pattern: [A-Za-z0-9.@\-_]+ Cannot contain spaces.
Action Type ID
Input Artifacts
A list of input "artifacts", that actually correspond to names of sub-directories in the artifact store, into a directory named after the pipeline physical ID. The list may be empty.
Output Artifacts
A list of output "artifacts", that actually correspond to sub-directories in the artifact store, into a directory named after the pipeline physical ID. The list may be empty.
Configuration
Run Order
Action Types
Currently, six types of actions are supported:
Custom actions can also be developed.
Source
Resources:
MyPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
...
Stages:
- Name: Source
Actions:
- Name: !Sub 'source-action-${GitHubRepositoryName}-${Branch}'
ActionTypeId:
Category: Source
Owner: ThirdParty
Version: '1'
Provider: GitHub
InputArtifacts: []
OutputArtifacts:
- Name: 'sources'
Configuration:
Owner: 'novaordis-llc'
Repo: !Ref GitHubRepositoryName
Branch: !Ref Branch
OAuthToken: ...
RunOrder: 1
GitHub Authentication
Source Action Execution Mechanics
The action provider (GitHub) performs a repository clone and packages the content as a ZIP file in the artifact store, which in most cases is an S3 bucket. The ZIP file is placed into a <pipeline-physical-ID>/<SourceOutputArtifacts.Name>.
Build
External
- Build Action Integration
- Add an AWS CodeBuild Build Action to a Pipeline
- Use CodePipeline with AWS CodeBuild to Test Code and Run Builds
CodeBuild can be added as a build action to the build stage of the pipeline. Existing build projects can be used, or new ones can be created in the CodePipeline console.
Resources:
MyPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
...
Stages:
...
- Name: Build
Actions:
- Name: !Sub 'CodeBuild build driven by ${Buildspec}'
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
InputArtifacts:
- Name: 'sources'
OutputArtifacts:
- Name: 'build-files'
Configuration:
ProjectName: !Ref CodeBuildProject
RunOrder: 1
CodePipeline-Driven CodeBuild Builds
Test
Deploy
Resources:
MyPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
...
Stages:
...
- Name: Deploy
Actions:
- Name: Deploy
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: CloudFormation
InputArtifacts:
- Name: ContainerSrc
- Name: ContainerBuild
Configuration:
StackName: !Join ['-', [!Sub '${AWS::Region}', !Join ['-', !Split ['.', !Ref GitHubRepositoryName]], 'deploy']]
ActionMode: CREATE_UPDATE
Capabilities: CAPABILITY_IAM
TemplatePath: !Sub ContainerSrc::${SvcTemplate}
TemplateConfiguration: ContainerBuild::overrides.json
ParameterOverrides: !Sub '{ "ECRRepository": "${ECRRepository}", "EnvironmentName": "${CFEnvironment}", "Image": "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRRepository}", "DeploymentStackName": "${DeploymentStackName}", "GitHubRepo": "${GitHubRepositoryName}" }'
RoleArn:
Fn::ImportValue: !Sub '${AWS::Region}-CloudFormationDeploymentRole'
OutputArtifacts: []
RunOrder: 1
Notes to organize:
- The parameters specified in the "ParameterOverrides" must match with the sub-template parameters.
- If "override.json" is declared as "TemplateConfiguration" and the previous build task does not create the override.json file, the deployment stage will fail with an S3 error.