Spinnaker Running a Script with Run Job (Manifest)

External

• https://kb.armory.io/s/article/Run-a-Generic-Shell-Script-with-Spinnaker
• https://spinnaker.io/docs/guides/user/kubernetes-v2/run-job-manifest/
• https://blog.spinnaker.io/extending-spinnaker-with-kubernetes-and-containers-5d16ec810d81

Internal

• Spinnaker Concepts
• Run Job (Manifest)

Overview

The execution phase is implemented as a Run Job (Manifest) stage.

⚠️ Make sure you use Run Job (Manifest) and NOT Deploy (Manifest)!

Run Job (Manifest) only accepts Kubernetes Job manifests, so a ConfigMap to specify the script cannot be used. The functionality must be already present in the container image and can be controlled via command and arguments. The stage will deploy the job manifest and wait until it completes, thus gating the pipeline’s continuation on the job’s success or failure. The stage may optionally collect the Job output and inject it back into the pipeline. The Job will be automatically deleted upon completion.

An alternative to run configurable functionality is to use Running a Script with Deploy (Manifest), but in that case we lose the ability to inject output back into the pipeline.

Run Job (Manifest) Definition

Add a "Run Job (Manifest)" stage and name it "Script Job" or similar.

The Account is the target Kubernetes cluster.

In Manifest Configuration. set Manifest Source: Text, and use the following manifest. Replace 'REPLACE-WITH-TARGET-NAMESPACE-NAME' with the target namespace for the Job.

Note that the Job name will be available in kubectl -n <namespace> get job and it will also surface in pod names, so choose something meaningful.

apiVersion: batch/v1
kind: Job
metadata:
  name: script-job
  namespace: REPLACE-WITH-TARGET-NAMESPACE-NAME
spec:
  backoffLimit: 2
  template:
    spec:
      containers:
        - command:
            - kubectl
            - version
          image: 'bitnami/kubectl:1.12'
          name: script
      restartPolicy: Never

Configuration

backoffLimit

Kubernetes Job Manifest | backoffLimit

Viewing Execution Logs

https://spinnaker.io/docs/guides/user/kubernetes-v2/run-job-manifest/#viewing-execution-logs

The job execution log is captured and can be accessed in the pipeline Execution Details → Run Job Config → Logs → Console Output.

If the Spinnaker instance is configured as such, logs can be forwarded to dedicated infrastructure.

Also see:

Operations | Log Management

Generating and Using Execution Results

https://spinnaker.io/docs/guides/user/kubernetes-v2/run-job-manifest/#capturing-output

A job produces some output, and the Spinnaker "Run Job (Manifest)" stage can be configured to "capture it". Captured output will be available in the pipeline context for use in downstream stages. There are several ways to "capture output": from a job's container stdout or from an artifact produced by the job.

Capture Output from Container Stdout and Update Stage Outputs and Context

Output → Capture Output From → Logs. A Container Name must be specified.

⚠️ Once set, "Capture Output From: Logs" implies that the job output contains the expected syntax (SPINNAKER_PROPERTY_*=* or SPINNAKER_CONFIG_JSON=...). If that output is missing, the job will fail with "Exception ( Wait On Job Completion ) Expected properties file automation-logic but it was either missing, empty or contained invalid syntax". On the other hand, if additional output is present, it won't interfere with the parsing of the expected syntax.

SPINNAKER_PROPERTY_*=*

If the job sends to stdout SPINNAKER_PROPERTY_<key>=<value>, then <key>:<value>, where <key> is converted to lowercase, shows up in the stage output map .stages[stage_index].outputs, and also in the map .stages[stage_index].outputs.propertyFileContents. It also shows in the stage context map .stages[stage_index].context, and also in the map .stages[stage_index].context. propertyFileContents .

So, if the designated Job container sends this to stdout:

SPINNAKER_PROPERTY_COLOR=Blue

when the stage completes, it carries this in its JSON representation:

{
  "type": "PIPELINE",
  "stages": [
    {
      "type": "runJobManifest",
      "context": {
        "color": "Blue",
        "propertyFileContents": {
          "color": "Blue"
        }
      },
      "outputs": {
        "color": "Blue",
        "propertyFileContents": {
          "color": "Blue"
        }
      }
    }
  ]
}

The values can be accessed with:

${ execution.stages.?[ name == 'Script Job' ][0].outputs.color }

or:

${ execution.stages.?[ name == 'Script Job' ][0].context.color }

More key formats:

SPINNAKER_PROPERTY_COLOR.NUANCE=darkblue

surfaces in "output"/"context" as:

{
  "stages": [
    {
      "context": {
        "color.nuance": "Blue",
         ...
        }
     }
  ]
}

The SPEL lookup should be performed as follows::

${ execution.stages.?[ name == 'Script Job' ][0].context['color.nuance'] }

SPINNAKER_CONFIG_JSON

TO PROCESS: https://spinnaker.io/docs/guides/user/kubernetes-v2/run-job-manifest/#spinnaker_config_json

If the Job container sends this to stdout:

SPINNAKER_CONFIG_JSON ={"color":"Blue","size","small"}

when the stage completes, it carries the following structure in its JSON representation.

Note that the JSON content must be on just one line!

{
  "type": "PIPELINE",
  "stages": [
    {
      "type": "runJobManifest",
      "context": {
        "color": "Blue",
        "object": {
          "shape": "square",
          "size": "10"
        },
        "propertyFileContents": {
          "color": "Blue",
          "object": {
            "shape": "square",
            "size": "10"
          }
        }
      },
      "outputs": {
        "color": "Blue",
        "object": {
          "shape": "square",
          "size": "10"
        },
        "propertyFileContents": {
          "color": "Blue",
          "object": {
            "shape": "square",
            "size": "10"
          }
        }
      }
    }
  ]
}

The SPINNAKER_CONFIG_JSON map keys are merged into context, contents.propertyFileContents, outputs and outputs.propertyFileContents.

The values can be accessed with:

${ execution.stages.?[ name == 'Script Job' ][0].outputs.color }

or:

${ execution.stages.?[ name == 'Script Job' ][0].outputs.object.shape }

Inject a base64-encoded Embedded Artifact back into the Pipeline with SPINNAKER_CONFIG_JSON

The content generated by the Job can be packaged as a base64-encoded artifact and injected back into the pipeline, to be used by subsequent stages:

1. Generate the content in the Job and package it as a base64 element.

cat > /tmp/test.yaml <<EOF
a:
   b:
       c: 'something'
EOF
local content=$(cat /tmp/test.yaml | base64 | tr -d \\n) || exit 1

2. Generate the following JSON as value of SPINNAKER_CONFIG_JSON:

{
  "artifacts": [
    { "type":"embedded/base64",
      "name": "job_generated_artifact", 
      "reference": "${content}"}
  ]
}

3. Configure the 'Run Job (Manifest)' 'Produces Artifacts" section with a new artifact that "matches" the base64-encoded job output, which is available to the pipeline as "job_generated_artifact":

Upon the successful execution of the stage, the stage JSON will contain this:

{
  "type": "PIPELINE",
  "stages": [
    {
      "name": "Automation Logic",
      "context": {
        "artifacts": [
          {
            "reference": "YXBwbGVDb25uZWN0...",
            "name": "helm_chart_overlay",
            "type": "embedded/base64"
          }
        ]
      },
      "outputs": {
        "artifacts": [
          {
            "name": "helm_chart_overlay",
            "reference": "YXBwbGVDb25uZWN0...",
            "type": "embedded/base64"
          }
        ],
        "propertyFileContents": {
          "artifacts": [
            {
              "name": "helm_chart_overlay",
            "reference": "YXBwbGVDb25uZWN0...",
              "type": "embedded/base64"
            }
          ]
        }
      }
    },
    ...
  ],
}

4. Configure the next stage to use the artifact produced by this stage:

Job-Generated Artifacts

TO PROCESS: https://spinnaker.io/docs/guides/user/kubernetes-v2/run-job-manifest/#artifacts

Also see Inject a base64-encoded Embedded Artifact back into the Pipeline with SPINNAKER_CONFIG_JSON above.