CI/CD Infrastructure Setup

External

• https://github.com/OpenShiftDemos/openshift-cd-demo

Internal

• OpenShift CI/CD Operations

Overview

This article describes the procedure to install a CI/CD pipeline based on a persistent Jenkins instance and auxiliary tools (Nexus, Gogs, SonarQube). The procedure was derived from the "CI/CD Demo - OpenShift Container Platform 3.6" https://github.com/OpenShiftDemos/openshift-cd-demo. The Jenkins instance will be a shared instance, deployed within its own dedicated "cicd" project to server any other project that may need CI/CD services.

Pre-Requisites

Create the "cicd" project to host the Jenkins instance and auxiliaries.

oc new-project cicd \
  --display-name="CI/CD" \
  --description="Shared CI/CD tools to provide release pipeline services for other projects"

Provision 6 1Gi persistent volumes to be used by Jenkins, Nexus, Gogs data, Gogs Postgres, Sonar and Sonar Postrgres, and a smaller one (512Mi) for Gogs config.

Deploy Jenkins

oc new-app jenkins-persistent --param=MEMORY_LIMIT=1Gi -e INSTALL_PLUGINS=analysis-core:1.92,findbugs:4.71,pmd:3.49,checkstyle:3.49,dependency-check-jenkins-plugin:2.1.1,htmlpublisher:1.14,jacoco:2.2.1,analysis-collector:1.52 -n cicd

Template

The template is available at https://github.com/NovaOrdis/playground/blob/master/openshift/auxiliary-tools/all-cicd.yaml. It is based on https://github.com/OpenShiftDemos/openshift-cd-demo/blob/ocp-3.6/cicd-template-with-sonar.yaml

oc process -f ./all-cicd.yaml \
  -p GOGS_PASSWORD=<gogs-password> \
  -p DEV_PROJECT=<dev-project-name> \
  -p STAGE_PROJECT=<stage-project-name> \

Since the entire project is dedicated to CI/CD, instead of creating individual service accounts for individual services, we use the project's default service account "default" and we elevate its privileges to "edit". For more details on CI/CD security considerations see CI/CD Security Considerations.

Before Running:

• Verify that all required images are available in the internal docker registry.
• Who creates the database for gogs and sonarqube postgreses?
• Modify all routes to make sure the are Edge terminate and they not allow not-https.
• Verify that -p V='K' works.

A special special service account ("system:serviceaccount:CICD:jenkins") will be created for Jenkins.

Additional components (Gogs, Sonar, Nexus) will also be deployed.

Create Required Image Streams

Create an Image Stream

Create Projects

Create the following projects:

1. A project for the CI/CD components, named "CICD":

oc new-project CICD --display-name="CI/CD pipeline with Jenkins"

2. A project to host development-stage containers and processes, named "dev":

oc new-project dev --display-name="Test Development Project"

3. A project to host publicly-accessible application produced by the CI/CD pipeline, named "stage":

oc new-project stage --display-name="Test Stage Project"

Grant Required Permissions

Jenkins components need to access the OpenShift API, so the service account that will run the Jenkins pod ("system:serviceaccount:CICD:jenkins") must be given appropriate permissions for the projects it must service:

Do we really need "admin" to "jenkins"?

oc policy add-role-to-user admin system:serviceaccount:CICD:jenkins

oc policy add-role-to-user edit system:serviceaccount:CICD:jenkins -n dev
oc policy add-role-to-user edit system:serviceaccount:CICD:jenkins -n stage

More details about Jenkins security considerations:

Jenkins Security Considerations

Provision a Persistent Volume

The template requires a persistent volume, which must be provisioned before the installation.

Persistent Volume Operations

Create Jenkins Components

oc project CICD

oc process -f ./cicd-template.yaml --param DEV_PROJECT=dev --param STAGE_PROJECT=stage \
| oc create -f - 

A template example for OpenShift 3.5 is available at https://github.com/OpenShiftDemos/openshift-cd-demo/blob/ocp-3.5/cicd-template.yaml. A version is also available here:

OpenShift CICD Template Example

Post-Install Adjustments

Adjust Readiness Probe Timeout

 oc set probe dc jenkins --readiness --initial-delay-seconds=500

Adjust Memory

oc project CICD
oc set resources dc/jenkins --limits=memory=1Gi

Verification

• Jenkins should start and be available at https://jenkins-cicd.apps.openshift.novaordis.io/
• Gogs should start and be available at https://gogs-cicd.apps.openshift.novaordis.io/
• Nexus should start and be available at https://nexus-cicd.apps.openshift.novaordis.io/