Spinnaker Pipeline Template
External
- https://spinnaker.io/docs/concepts/pipelines/#pipeline-templates
- https://spinnaker.io/docs/reference/pipeline/templates/
Internal
Overview
Multiple pipeline instances can be created from a pipeline template by providing values to the variables defined in the template. Any changes made to the pipeline template even after it has been instantiated as one or more pipelines, are propagated to the instantiated pipelines automatically.
The Process of Creating a Pipeline from Template
The template is expressed as a JSON file, similar to the one provided in the Template Example below:
{
"schema": "v2",
"id": "blue-template",
"protect": false,
"metadata": {
"name": "Blue Template",
"description": "An experimental pipeline template",
"owner": "someone@example.com",
"scopes": [
"global"
]
},
"pipeline": {
"keepWaitingPipelines": false,
"spelEvaluator": "v4",
"limitConcurrent": true,
"stages": [
{
"refId": "1",
"name": "My Manual Judgment",
"type": "manualJudgment",
"failPipeline": true,
"isNew": true,
"judgmentInputs": [
{
"value": "Option 1 to Pick from Dropdown Box"
},
{
"value": "Option 2 to Pick from Dropdown Box"
}
],
"instructions": "Take a decision based on color: ${ templateVariables.color }",
"notifications": [],
"requisiteStageRefIds": []
}
]
},
"variables": [
{
"name": "color",
"type": "string",
"defaultValue": "blue",
"description": "A color"
}
]
}
The pipeline template optionally includes a variables section, that lists template variables (name, type, default value and description). Explain template variables.
The template is saved to the backend with:
spin pipeline-template save --file ".../template-file.json"
If a template with the same ID already exists, the value will be preserved, and the new version of the template will take priority. A list of existing versions can be obtained from the Pipeline Template UI → Version. The latest version is tagged with :latest. Each version gets a @sha256:01...00 tag. If pipelines based on this template exist, they will be updated.
The template such saved can be referred to instantiate pipelines by its id, and can be queried in the UI by its user-readable metadata.name. To get the state of the template from the backend storage:
spin pipeline-template get --id "<pipeline-template-id>"
The result is mostly identical with the source JSON, with a few fields like lastModifiedBy, updateTs updated to reflect the interaction with the backend.
Once saved into the backend, the template can be used to instantiate pipelines, by generating the pipeline JSON. Note that the following command does not store anything in the back end, it just generates the pipeline JSON representation, it is our responsibility to take that representation and create the pipeline:
spin pipeline-template use --application "<application-name>" --id "<pipeline-template-id>" --name "<pipeline-name>" [--values "<.../pipeline-variables.yaml>"] > ~/tmp/my-pipeline.json
This command generates a JSON representation of the pipeline at stdout. The representation looks similar to:
{
"schema": "v2",
"name": "some-pipeline",
"application": "some-application",
"template": {
"reference": "spinnaker://blue-template",
[...]
},
"parameters": [],
"variables": {},
"stages": []
[...]
}
The fact that is a pipeline instantiated from a template is obvious from the template.reference element, and the representation contains the application name and the pipeline name.
The --value command line parameter, followed by a YAML/JSON file containing variable names and values will cause those variable name and values to appear in the variables section of the rendered template. If the content of the pipeline-variables.yaml specified in the above command line example is:
size: large
then the pipeline representation will be:
{
"schema": "v2",
"name": "some-pipeline",
[...]
"variables": {
"size": "large"
}
}
Discuss parameters and variables.
The JSON representation of the pipeline can be used to instantiate the pipeline with:
spin pipeline save --file "<pipeline-json-representation>"
Upon saving, the pipeline instance gets a unique ID.
The pipeline can be obtained from the backend with:
spin pipeline list --application "<application-name>" | jq [...]
or:
spin pipeline get --application"<application-name>" --name "<pipeline-name>"
At this stage, the pipeline is managed by the backend and can be executed. If we access the pipeline via UI and "Edit Pipeline as JSON", we get two representations: a Configuration, which represents the pipeline in its persistent state, and a rendered pipeline.
Configuration
The "configuration" is the representation of a template-based pipeline, in its persistent state in the backend. Configurations can inject new stages into the final pipeline graph and inherit or override, or both, triggers, notifications, and parameters, additionally to those that come from the template. The configuration is similar to a pipeline not created from a template, but it additionally has variable bindings and a reference to the template:
{
"schema": "v2",
"type": "templatedPipeline",
"updateTs": "1685487935331",
"lastModifiedBy": "...",
"template": {
"reference": "spinnaker://blue-template",
[...]
},
"stages": [], # usually empty, if the configuration does not want to override the template
"parameters": [],
"variables": {}
}
Rendered Pipeline
The renewed pipeline is the final result of merging the configuration with the template, and it is what is going to be executed:
{
"keepWaitingPipelines": false,
"limitConcurrent": true,
"source": {
"type": "templatedPipeline",
"version": "v2"
},
"spelEvaluator": "v4",
"stages": [
{
"failPipeline": true,
"instructions": "Take a decision based on color: ${ templateVariables.color }",
"isNew": true,
"judgmentInputs": [
{
"value": "Option 1 to Pick from Dropdown Box"
},
{
"value": "Option 2 to Pick from Dropdown Box"
},
{
"value": "Option 3 to Pick from Dropdown Box"
}
],
"name": "My Manual Judgment",
"notifications": [],
"refId": "1",
"requisiteStageRefIds": [],
"type": "manualJudgment"
}
],
"templateVariables": {
"color": "blue"
},
"triggers": [],
"parameterConfig": []
}
Note how the template's variables become templateVariables in the rendered template.
Pipeline Execution
The pipeline can be executed with:
spin pipeline execute --application "<application-name>" --name "<pipeline-name>" [--parameter-file "<parameter-file>"]
More details about pipeline execution from command line:
Template
A template defines a parameterized pipeline, with the set of variables for which the users instantiating the template will provide values. The template does not have the pipeline configuration found on a pipeline instance. The template is useful in defining a pipeline pattern that can be instantiated as multiple concrete pipeline instances.
Template Example
Pipeline template JSON example:
{
"schema": "v2",
"id": "my-pipeline-template", # The pipeline instance references the template using this
"protect": <true | false>,
"metadata": {
"name": "My Template Name as seen in the UI",
"description": "<description>",
"owner": "example@example.com",
"scopes": ["global"] # Not used
},
"variables": [
{
"name": "<varName>",
"type": "<type>",
"defaultValue": <value>,
"description": "<description>"
}
],
"pipeline": { # Contains the templatized pipeline itself
"lastModifiedBy": "anonymous", # Not used
"updateTs": "0", # Not used
"parameterConfig": [], # Same as in a regular pipeline
"limitConcurrent": true, # Same as in a regular pipeline
"keepWaitingPipelines": false, # Same as in a regular pipeline
"description": "", # Same as in a regular pipeline
"triggers": [], # Same as in a regular pipeline
"notifications": [], # Same as in a regular pipeline
"stages": [ # Contains the templated stages
{
# This one is an example stage:
"waitTime": "${ templateVariables.waitTime }", # Templated field.
"name": "My Wait Stage",
"type": "wait",
"refId": "wait1",
"requisiteStageRefIds": []
}
]
}
}
Elements
schema
id
The identifier of this template. The pipeline instances generated from this template will references the template using the value of id:
{
[...]
"template": {
"artifactAccount": "...",
"reference": "spinnaker://my-pipeline-template",
"type": ".../pipelineTemplate"
},
[...]
}
The user readable name is given by metadata.name.
protect
'true' or 'false'
metadata
Metadata name
The name under which the template pipeline is listed in the UI. Note that while id is used to refer to the template while instantiating pipelines, the metadata.name is the "user readable" name.
scopes
Documentation says "not used" but an attempt save the pipeline template in the backend fails with "Missing required fields: metadata.scopes" if not present.
{
"metadata": {
"scopes": ["global"]
}
}
variables
A template variable is a variable defined in a pipeline template's variables section , whose value is determined when a pipeline is instantiated based on the template. A template variable is different from a pipeline variable, which vary per execution of the pipeline. The template variables declared in the pipeline template can be referred, using their name, in the pipeline stages, as follows:
{
"default": "${templateVariables.SomeVariableName}",
}
Variable name
Variable type
The type can be "string", ?
Variable defaultValue
Variable description
pipeline
The definition of the templatized pipeline.
parameterConfig
Same as in a regular pipeline.
stages
Contains the templatized stages.
keepWaitingPipelines
Same as in a regular pipeline. Default false.
limitConcurrent
Same as in a regular pipeline. Default true.
spelEvaluator
v4.
description
Same as in a regular pipeline.
triggers
Same as in a regular pipeline.
notifications
Same as in a regular pipeline.
lastModifiedBy
Not used.
updateTs
Not used.