OpenShift CI/CD Concepts

External

• https://docs.openshift.com/container-platform/latest/using_images/other_images/jenkins.html
• https://blog.openshift.com/cicd-with-openshift/

Internal

• OpenShift Concepts
• OpenShift CI/CD Operations
• Jenkins

Overview

This article aggregates concepts related to the implementation of CI/CD pipelines in OpenShift. OpenShift relies on Jenkins to execute the core pipeline logic - pipeline is a Jenkins concept - so Jenkins is integrated with OpenShift by default. Integration details are discussed in the Jenkins Integration section. CI/CD pipelines can be declared natively in OpenShift as pipeline builds.

Jenkins Integration

Jenkins Service Initialization

When a project declares a pipeline build configuration, OpenShift looks for a project service named "jenkins". "jenkins" is the default value, but the name of the service can be configured as jenkinsPipelineConfig.serviceName in master-config.yml.

If no Jenkins service exists in the project and master-config.yml jenkinsPipelineConfig.autoProvisionEnabled is set to "true", which is the default, OpenShift executes the jenkinsPipelineConfig.templateName template, which it expected to be available in the jenkinsPipelineConfig.templateNamespace namespace. The default values are "jerkins-ephemeral" and "openshift", respectively. However, if the auto-provisioning is explicitly disabled, and no Jenkins service is defined in the project, a pipeline build configuration won't trigger any build, and if the build is start manually, it will just hang in the "New" state. No error message or events will be triggered. This behavior was noticed in OpenShift 3.6.

I have never got this running, the closest I got is described in Integrate a Service Running in a Different Project, but the build would not start: Individual projects may be configured to avoid starting their own Jenkins infrastructure and use a shared Jenkins instance, typically deployed in a dedicated CI/CD project. To do that, they must declare a service named "jenkins" or whatever the value of master-config.yml jenkinsPipelineConfig.serviceName service name is, which carries the endpoints of the shared Jenkins service.

Jenkins Images and Templates

OpenShift provides a certified Jenkins container image, developed under https://github.com/openshift/jenkins repository. OpenShift also comes with two standard templates that use this image to create Jenkins infrastructure objects. The "jenkins-ephemeral" and "jenkins-persistent" are available in the "openshift" namespace. The "persistent" Jenkins will save its state between the pod restarts, and needs a persistent volume with sufficient storage space to be available at the time of the installation.

oc -n openshift get templates | grep jenkins
jenkins-ephemeral       Jenkins service, without persistent storage....     
jenkins-persistent      Jenkins service, with persistent storage....

Upon instantiations, the Jenkins templates create a deployment configuration for the Jenkins pod (dc/jenkins), two services (jenkins and jenkins-jnlp) and a route that makes jenkins publicly accessible as "jenkins-project-name.apps... The deployment configuration then creates the Jenkins pod.

Jenkins Heap Sizing

The heap sizing is calculated by the container's CMD script (/usr/libexec/s2i/run) based on the OpenShift memory limit set in the deployment configuration, which propagates to the container as /sys/fs/cgroup/memory/memory.limit_in_bytes, and CONTAINER_HEAP_PERCENT.

If the JVM_ARCH environment variable is set at installation from template, the memory calculated as mentioned above. If not, and the memory of the container is less than 4GB, meaning max heap of 2GB, 32bit is forced to save space. The template comes with a small metaspace, which can be increased with "JAVA_GC_OPTS="-XX:+UseParallelGC -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:MaxMetaspaceSize=384m"

OpenShift Jenkins Plugins

Jenkins is ultimately the application that drives the pipeline logic so it first need to be told what to do, via its Groovy pipeline syntax, and then it needs to execute the actions, and while doing so, it needs to access and modify OpenShift resources. All these are implemented via a series of plugins, described below:

OpenShift Plugin for Jenkins (jenkins-plugin)

This is the oldest Jenkins/OpenShift integration plugin, which executes inside Jenkins and implements a series of REST flows that interface with the OpenShift master server via its exposed API. This plugin does not require the oc binary to be present on the host that executes the plugin logic.

OpenShift Plugin for Jenkins (jenkins-plugin)

OpenShift Jenkins Pipeline DSL Plugin (jenkins-client-plugin)

This plugin provides a Jenkins pipeline DSL syntax for interactions with OpenShit. This plugin needs the oc binary to be present on the nodes executing the script. The plugin seems to be newer than the OpenShift Plugin for Jenkins (jenkins-plugin). Plugin usage and syntax details are available here:

OpenShift Jenkins Pipeline DSL Plugin (jenkins-client-plugin)

jenkins-sync-plugin

https://github.com/openshift/jenkins-sync-plugin

This plugin keeps OpenShift build configuration and build objects in sync with jobs and builds managed by the OpenShift-integrated Jenkins instances.

The plugin insures that any changes operated on OpenShift pipeline build configuration is propagated to the Jenkins job with the same name. When an OpenShift user triggers a build based on a pipeline build configuration, the plugin starts the corresponding Jenkins job. While the Jenkins build that is associated with the job is running, any changes in the build are replicated in the corresponding OpenShift build object. The plugin also examine ConfigMaps looking for XML documents that correspond to Kubernetes plugin pod templates, and changes the configuration of the Kubernetes plugin to add, edit or remove pod templates based on those found in the ConfigMaps. The plugin monitors changes in image streams labeled with "role=jenkins-slave" and image stream tags with a "role=jenkins-slave" annotation and updates the Kubernetes plugin pod templates accordingly.

jenkins-openshift-login-plugin

https://github.com/openshift/jenkins-openshift-login-plugin

Kubernetes Plugin

https://plugins.jenkins.io/kubernetes

https://docs.openshift.org/latest/using_images/other_images/jenkins.html#using-the-jenkins-kubernetes-plug-in-to-run-jobs

The Kubernetes plugin is a Jenkins extension that uses a Kubernetes cluster to dynamically provision Jenkins agents, use them to run builds and then dispose of them. It does do by using Kubernetes scheduling mechanism to optimize the load. The Jenkins image that ships with OpenShift comes with the Kubernetes plugin pre-installed. The OpenShift Plugin for Jenkins allows specifying agents as slave pods.

Pipeline Logic Declaration

Pipeline logic is declared in Groovy, either inline in the build configuration, or as a Jenkinsfile stored in the source code repository. For syntax, see

Build Configuration

Inline Pipeline Logic Declaration

apiVersion: "v1"
kind: "BuildConfig"
metadata:
  name: "my-pipeline-build-configuration"
spec:
  strategy:
    type: JenkinsPipeline
    jenkinsPipelineStrategy:
      env:
        - name: ENV_VAR_1
          value: "..."
      jenkinsfile: |
        //
        // in-line pipeline code
        //
        ...

Jenkinsfile

Jenkinsfile contains the definition of the pipeline that drives an OpenShift pipeline build, written in Groovy. The OpenShift pipeline build configuration can be written in such a way to refer to it:

apiVersion: "v1"
kind: "BuildConfig"
metadata:
  name: "my-pipeline-build-configuration"
spec:
  source:
    git:
      uri: "https://....git"
  strategy:
    type: JenkinsPipeline
    jenkinsPipelineStrategy:
      env:
        - name: ENV_VAR_1
          value: "..."
      jenkinsfilePath: "jenkinsfile/path/relative/to/the/root/of/the/repository/Jenkinsfile"

Note that 'jenkinsfilePath' must contain the relative path and the name of the file.

A Jenkinsfile maintained in the source code and version controlled is the preferred way of specifying the pipeline logic.

Deplete

OpenShift CI/CD Concepts TODEPLETE