AWS CodeBuild Operations: Difference between revisions
(42 intermediate revisions by the same user not shown) | |||
Line 7: | Line 7: | ||
==Project configuration== | ==Project configuration== | ||
Select [[AWS_CodeBuild_Concepts#Build_Badge|Build | ====Project name==== | ||
====Description==== | |||
====Build badge==== | |||
For more details about the build badge, see Select [[AWS_CodeBuild_Concepts#Build_Badge|AWS CodeBuild Concepts - Build Badge]]. | |||
====Tags==== | |||
==Source== | ==Source== | ||
Source provider | ====Source provider==== | ||
GitHub | |||
====Repository==== | |||
Repository in my GitHub account. | Repository in my GitHub account. | ||
====GitHub repository==== | |||
====Connection Status==== | |||
Additional configuration | Additional configuration | ||
Git clone depth | ====Git clone depth==== | ||
Clone depth 1 | |||
====Build Status==== | |||
==Primary source webhook events== | ==Primary source webhook events== | ||
Line 25: | Line 44: | ||
==Environment== | ==Environment== | ||
Environment image | ====Environment image==== | ||
The options are "Managed image" or "Custom image". A managed image is always preferable if it provides all the build functionality you need. However, if you need extra functionality, you can build and use a custom image. | |||
'''Managed image configuration''': | |||
Operating system: Ubuntu | Operating system: Ubuntu | ||
Line 33: | Line 56: | ||
Runtime version: aws/codebuild/java:openjdk-8 | Runtime version: aws/codebuild/java:openjdk-8 | ||
{{Warn|The Java runtime may have an obsolete Gradle version. If that is the case, [[Gradle_Concepts#The_Gradle_Wrapper|initialize your project with gradlew]] and configure the buildspec file to use ./gradlew to build the project.}} | |||
'''Custom image configuration''': | |||
Environment type: Linux | |||
Image registry: Amazon ECR | |||
ECR account: My ECR account | |||
Amazon ECR Repository: "com.example/unity/unity-build-image" | |||
Amazon ECR image: latest | |||
Image pull credentials: Project service role. This is the service role associated with this build project. | |||
====Privileged==== | |||
This flag should be enabled if the build project builds Docker images. More details about elevated privileges: [[AWS_CodeBuild_Concepts#Privileged_Build|AWS CodeBuild Concepts - Elevated Privileges]]. | |||
====Service role==== | |||
New service role. As a name pattern, use ''project-name''-build-service-role. Eg. "themyscira-unity-build-service-role". The role will require specific permissions, depending on what the build is supposed to do. For more details on what permission the role may need, and how to grant them, see: {{Internal|AWS_CodeBuild_Concepts#Service_Role|CodeBuild Service Role}} | |||
<span id='Allow_AWS_CodeBuild_to_modify_this_service_role_so_it_can_be_used_with_this_build_project'></span>Allow AWS CodeBuild to modify this service role so it can be used with this build project. If selected, the console configuration code updates the role so it contains the appropriate policies. | <span id='Allow_AWS_CodeBuild_to_modify_this_service_role_so_it_can_be_used_with_this_build_project'></span>Allow AWS CodeBuild to modify this service role so it can be used with this build project. If selected, the console configuration code updates the role so it contains the appropriate policies. | ||
Line 47: | Line 88: | ||
unselect "Allow AWS CodeBuild to modify ..." | unselect "Allow AWS CodeBuild to modify ..." | ||
Environment variables | ====Timeout==== | ||
====Queued timeout==== | |||
====Certificate==== | |||
====VPC==== | |||
====Compute==== | |||
====Environment variables==== | |||
For more details see: {{Internal|AWS_CodeBuild_Concepts#Environment_Variables|CodeBuild Concepts - Environment Variables}} | For more details see: {{Internal|AWS_CodeBuild_Concepts#Environment_Variables|CodeBuild Concepts - Environment Variables}} | ||
<font color=darkgray>No user environment variables can start with CODEBUILD_ | |||
Name: CODEBUILD_SOURCE_VERSION, Value: "unity" (for GitHub, this is the branch name to be uses) | Name: CODEBUILD_SOURCE_VERSION, Value: "unity" (for GitHub, this is the branch name to be uses) | ||
</font> | |||
For personal account: | |||
* Name: AWS_ACCESS_KEY Value: ... | |||
* Name: SK Value: TBD | |||
For shared build account: | |||
* Name: AWS_ACCESS_KEY_ID | |||
* Name: AWS_SECRET_ACCESS_KEY | |||
Additional Configuration | Additional Configuration | ||
Timeout: 20 minutes. | Timeout: 20 minutes. | ||
==Buildspec== | |||
====Build specifications==== | |||
Use a buildspec file. | |||
Specify name of the file, if not buildspec.yml. The default is a buildspec.yml in the source code root directory. | |||
For gradle modules, the following works: | |||
./themyscira/buildspec.yml | |||
==Artifacts== | |||
==Logs== | |||
====CloudWatch==== | |||
Select CloudWatch logs. | |||
Group Name: /playground | |||
Stream Name: themyscira-unity-build | |||
You must enable CloudWatch logging if you want any kind of build logs, including the "console" log. | |||
=Run a Build Project= | |||
Run Build | |||
Source version: unity | |||
Environment variables override: | |||
SK: | |||
Start Build | |||
=Build an AWS CodeBuild Docker Image= | |||
{{External|https://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker-custom-image.html}} | |||
{{External|https://docs.aws.amazon.com/codebuild/latest/APIReference/API_ProjectEnvironment.html}} | |||
This procedure documents the process of creating a custom [[AWS_CodeBuild_Concepts#Environment_Image|environment image]]. | |||
git clone git@github.com:aws/aws-codebuild-docker-images.git | |||
cd aws-codebuild-docker-images/ubuntu/java/openjdk-8 | |||
docker build -t aws/codebuild/java:openjdk-8 . | |||
docker images | |||
REPOSITORY TAG IMAGE ID CREATED SIZE | |||
aws/codebuild/java openjdk-8 5490a2e1223f 2 minutes ago 1.6GB | |||
{{Warn|If custom build images need to be able to interact with Docker - for creating Docker images, for example -, the Docker server [[#Amend_the_Build_Specification_to_Explicitly_Start_the_Docker_Daemon|must be started manually in the build specification]], as shown below.}} | |||
=Amend the Build Specification to Explicitly Start the Docker Daemon= | |||
In case of custom environment images, even those based on an AWS CodeBuild image, built as shown [[#Build_an_AWS_CodeBuild_Docker_Image|above]], the docker daemon must be started explicitly in the build specification. This has to be done even if the "[[AWS_CodeBuild_Concepts#Privileged_Build|privileged build]]" flag is enabled, the docker daemon must be started explicitly in build spec - it is not started automatically as in the case of AWS-provided environment images. | |||
This sequence works for Ubuntu-based images: | |||
<syntaxhighlight lang='yaml'> | |||
version: 0.2 | |||
... | |||
phases: | |||
install: | |||
commands: | |||
- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2& | |||
- timeout 15 sh -c "until docker info; do echo .; sleep 1; done" | |||
build: | |||
... | |||
</syntaxhighlight> | |||
=Troubleshooting= | =Troubleshooting= | ||
{{External|https://docs.aws.amazon.com/codebuild/latest/userguide/troubleshooting.html}} | Generic troubleshooting advice: {{External|https://docs.aws.amazon.com/codebuild/latest/userguide/troubleshooting.html}} | ||
==Failed Status== | |||
For more details, go to: Build Project -> Build History -> Build Run -> Phase details | |||
==ECR AccessDeniedException== | |||
An error occurred (AccessDeniedException) when calling the GetAuthorizationToken operation: User: arn:aws:iam::673499572719:user/codebuild-p2 is not authorized to perform: ecr:GetAuthorizationToken on resource: * | |||
Add ecr:GetAuthorizationToken for the user in question to the build role. These are extremely lax permissions, it could be further tightened up: | |||
<syntaxhighlight lang='javascript'> | |||
{ | |||
... | |||
"Sid": "VisualEditor1", | |||
"Effect": "Allow", | |||
"Action": "ecr:*", | |||
"Resource": "*" | |||
} | |||
</syntaxhighlight> |
Latest revision as of 04:19, 15 March 2019
Internal
Create a Build Project
Project configuration
Project name
Description
Build badge
For more details about the build badge, see Select AWS CodeBuild Concepts - Build Badge.
Tags
Source
Source provider
GitHub
Repository
Repository in my GitHub account.
GitHub repository
Connection Status
Additional configuration
Git clone depth
Clone depth 1
Build Status
Primary source webhook events
Webhook - Rebuild every time a code change is pushed to this repository. Enable and experiment with that.
Environment
Environment image
The options are "Managed image" or "Custom image". A managed image is always preferable if it provides all the build functionality you need. However, if you need extra functionality, you can build and use a custom image.
Managed image configuration:
Operating system: Ubuntu
Runtime: Java
Runtime version: aws/codebuild/java:openjdk-8
The Java runtime may have an obsolete Gradle version. If that is the case, initialize your project with gradlew and configure the buildspec file to use ./gradlew to build the project.
Custom image configuration:
Environment type: Linux
Image registry: Amazon ECR
ECR account: My ECR account
Amazon ECR Repository: "com.example/unity/unity-build-image"
Amazon ECR image: latest
Image pull credentials: Project service role. This is the service role associated with this build project.
Privileged
This flag should be enabled if the build project builds Docker images. More details about elevated privileges: AWS CodeBuild Concepts - Elevated Privileges.
Service role
New service role. As a name pattern, use project-name-build-service-role. Eg. "themyscira-unity-build-service-role". The role will require specific permissions, depending on what the build is supposed to do. For more details on what permission the role may need, and how to grant them, see:
Allow AWS CodeBuild to modify this service role so it can be used with this build project. If selected, the console configuration code updates the role so it contains the appropriate policies.
If you get:
The policy's default version was not created by enhanced zero click role creation or was not the most recent version created by enhanced zero click role creation.
unselect "Allow AWS CodeBuild to modify ..."
Timeout
Queued timeout
Certificate
VPC
Compute
Environment variables
For more details see:
No user environment variables can start with CODEBUILD_
Name: CODEBUILD_SOURCE_VERSION, Value: "unity" (for GitHub, this is the branch name to be uses)
For personal account:
- Name: AWS_ACCESS_KEY Value: ...
- Name: SK Value: TBD
For shared build account:
- Name: AWS_ACCESS_KEY_ID
- Name: AWS_SECRET_ACCESS_KEY
Additional Configuration
Timeout: 20 minutes.
Buildspec
Build specifications
Use a buildspec file.
Specify name of the file, if not buildspec.yml. The default is a buildspec.yml in the source code root directory.
For gradle modules, the following works:
./themyscira/buildspec.yml
Artifacts
Logs
CloudWatch
Select CloudWatch logs.
Group Name: /playground
Stream Name: themyscira-unity-build
You must enable CloudWatch logging if you want any kind of build logs, including the "console" log.
Run a Build Project
Run Build
Source version: unity
Environment variables override:
SK:
Start Build
Build an AWS CodeBuild Docker Image
This procedure documents the process of creating a custom environment image.
git clone git@github.com:aws/aws-codebuild-docker-images.git cd aws-codebuild-docker-images/ubuntu/java/openjdk-8 docker build -t aws/codebuild/java:openjdk-8 . docker images REPOSITORY TAG IMAGE ID CREATED SIZE aws/codebuild/java openjdk-8 5490a2e1223f 2 minutes ago 1.6GB
If custom build images need to be able to interact with Docker - for creating Docker images, for example -, the Docker server must be started manually in the build specification, as shown below.
Amend the Build Specification to Explicitly Start the Docker Daemon
In case of custom environment images, even those based on an AWS CodeBuild image, built as shown above, the docker daemon must be started explicitly in the build specification. This has to be done even if the "privileged build" flag is enabled, the docker daemon must be started explicitly in build spec - it is not started automatically as in the case of AWS-provided environment images.
This sequence works for Ubuntu-based images:
version: 0.2
...
phases:
install:
commands:
- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2&
- timeout 15 sh -c "until docker info; do echo .; sleep 1; done"
build:
...
Troubleshooting
Generic troubleshooting advice:
Failed Status
For more details, go to: Build Project -> Build History -> Build Run -> Phase details
ECR AccessDeniedException
An error occurred (AccessDeniedException) when calling the GetAuthorizationToken operation: User: arn:aws:iam::673499572719:user/codebuild-p2 is not authorized to perform: ecr:GetAuthorizationToken on resource: *
Add ecr:GetAuthorizationToken for the user in question to the build role. These are extremely lax permissions, it could be further tightened up:
{
...
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "ecr:*",
"Resource": "*"
}