Spinnaker Pipeline

External

• https://spinnaker.io/docs/concepts/pipelines/

Internal

• Spinnaker Concepts

Overview

Pipelines are the tools for deploying an application in Spinnaker. A pipeline consists of a sequence of stages. Parameters can be passed from stage to stage. Pipelines can be triggered manually via the UI or command line, or can be configured to be triggered by an event such as a Jenkins job completing, a new Docker image appearing in the registry, a cron schedule or a stage in another pipeline. The JSON-encoded state of an executed pipeline can be visualized as JSON by going to Pipelines → Select a pipeline → Select an Execution → Execution Details → View as JSON:

{
  "id": "112[...]X",
  "type": "PIPELINE",
  "application": "smoke",
  "name": "Test Docker Trigger",
  "buildTime": 1645824168644,
  "startTime": 1645824168714,
  "endTime": 1645824185005,
  "canceled": false,
  "limitConcurrent": true,
  "keepWaitingPipelines": false,
  "status": "SUCCEEDED",
  "origin": "api",
  "spelEvaluator": "v4",
  "pipelineConfigId": "35[...]a1",
  "authentication": {
    "user": "...",
    "allowedAccounts": []
  },
  "trigger": {
    "id": "...",
    "type": "docker",
    ...
  },
  "stages": [
    {
      "id": "11F[...]1",
      "refId": "2",
      "type": "bakeManifest",
      "name": "Render Helm",
      "startTime": 1645824168750,
      "endTime": 1645824171026,
      "status": "SUCCEEDED",
      "context": {...},
      "outputs": {...},
      "tasks": {...},
    },
    ...
  ],
  "notifications": [],
  "initialConfig": {},
  "systemNotifications": []
}

Pipeline Status

• Running
• Awaiting Judgement
• Terminal
• Succeeded
• Not Started
• Canceled
• Stopped
• Paused
• Buffered

Pipeline Context and Helper Properties

https://spinnaker.io/docs/guides/user/pipeline/expressions/#helper-properties

https://spinnaker.io/docs/reference/pipeline/expressions/#helper-properties

There is no "pipeline context" per se, the name comes from stage context. but the pipeline exposes a series of helper properties, variables which refer to global information about the current pipeline execution. The available helper properties are described below.

To quickly list available helper properties and stages, while editing a stage, enter enter a question mark (?) into the pipeline expression. The attributes of the helper properties can be queried at edit time while editing a stage and lingering with the cursor close to the helper property.

Also see:

Stage Context

execution

A variable that refers to the current pipeline execution, and it can be referred in an expression as ${execution}. execution has the following attributes:

• ${execution['id']}
• etc.

trigger

A variable that provides access to the pipeline trigger. It can be referred in an expression as ${trigger}. trigger has the following attributes:

• ${trigger['type']}
• ${trigger['user']}
• ${trigger['buildInfo']}
• etc.

Pipeline Configuration

A pipeline configuration section consists of the following sub-sections: Execution Options, Automated Triggers, Parameters, Notifications and Metadata:

Execution Options

Do not automatically cancel pipelines waiting in queue

If concurrent pipeline execution is disabled, then the pipelines that are in the waiting queue will get canceled when the next execution starts. The pipeline can be configured to keep them in the queue.

Parameters

A pipeline can define a set of parameters. They can added with "Add Parameter" in the UI. The parameters are useful in the following situations:

• when the pipeline is executed manually, in which case the parameters are present on the UI when launching the pipeline.
• when the pipeline is executed programmatically via a spin command.
• when the pipeline is executed as result of a Pipeline stage, to pass configuration to the pipeline

If the pipeline is thought about as a function, then the pipeline parameters are similar to function's parameters, and the values provided in the UI are the function arguments.

Declaring Parameters in the UI

This section describes how to create pipeline parameters using the UI. The same result can be achieved when creating the pipeline from command line.

Each parameter has a name and a label.

The name identifies the parameter in the pipeline state. For manual triggering via UI, the parameter name and value are available as:

{
  "type": "PIPELINE",
  "trigger": {
    "type": "manual",
    "parameters": {
      "helm_chart_version": "1.0.0"
    },
  },
  "stages": [
    [...]
  ],
  [...]
}

The label is displayed by the UI by the text control associated with the parameter, when the user triggers the pipeline manually and it should be composed in such a way that it provides additional information to the user.

If the parameter is marked as Required, then the UI displays the text control associated with the parameter with a "required" asterisk by the Label, and the "Run" button does not become available until all "required" parameter values are filled out. The "Description" is rendered when the cursor hovers over the question mark icon.

Each parameter may have options.

Pin all parameters or Pin Parameter If checked, the parameter(s) will be shown in a pipeline execution view, otherwise it will be collapsed by default.

Providing Parameter Values in the UI

If the pipeline is triggered manually from the UI and the pipeline has parameters, a "Select Execution Parameters" window will pop up, giving the user the chance to provide parameter values. If the parameters have default values, those values will pop up as default values in the UI.

Declaring Parameters in the JSON Representation of the Pipeline

If the pipeline is created programmatically from its JSON representation with spin pipeline save --file <file-name>, the pipeline parameters can be declared as such:

{
  [...]
  "parameterConfig": [
    {
      "name": "color",
      "default": "blue",
      "label": "The Color",
      "description": "The value of the color to be provided to the pipeline",
      "pinned": true,
      "required": true
    }
  ]
}

For details on how to create an entire pipeline programmatically, without a template, see:

Create Pipeline from Command Line with spin

Providing Parameters Values on Command Line

The equivalent of providing parameters in the "Select Execution Parameters" pop up window (see Providing Parameter Values in the UI, above) is to provide a parameter file when manually triggering the pipeline with spin pipeline execute with --parameter-file:

spin pipeline execute --application "<application-name>" --name "<pipeline-name>" --parameter-file "<parameter-file>"

The parameter file is a YAML or JSON file that contains key/value pairs corresponding to the pipeline's parameters:

color: green

The value of the parameters such provided are displayed in the UI under the "Parameters" section of the execution:

For more details, see:

Executing a Pipeline in Command Line

Using Parameters in Expressions

Once the pipeline is triggered manually, the parameters whose values are specified in the UI can be retrieved by the pipeline stages using SpEL:

"helm.com/something/something-else-${execution.trigger.parameters.helm_chart_version}.tgz"

The color is ${execution.trigger.parameters['color']}

For more details; see:

A tag set by a trigger

Pipeline Executions

A pipeline may have multiple executions, listed in the UI under the pipeline name.

Pipeline Template

Pipeline Template

Pipeline Variables

https://spinnaker.io/docs/guides/user/pipeline/expressions/

Research and document, it reoccurs often.

• Define a pipeline variable.
• Define the relationship between a pipeline variable and a pipeline parameter.
• Define a template variable.
• Define the relationship between a pipeline variable and a template variable.
• Why does ${templateVariables.<variable-name>} is not rendered? The template variable is defined. Is this it? https://github.com/spinnaker/spinnaker/issues/5720

Pipeline expressions allow you to dynamically set and access variables during pipeline execution.

Aslo see:

SpEL Expressions | Variable

Variable Creation

Variables can be created at runtime with "Evaluate Variables" stage. See:

'Evaluate Variables' Stage

Pipeline Operations

• List pipelines
• Create a pipeline
• Execute a pipeline
• Delete pipeline