Tekton Pipeline: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(41 intermediate revisions by the same user not shown)
Line 3: Line 3:
* https://github.com/tektoncd/pipeline/blob/main/docs/README.md
* https://github.com/tektoncd/pipeline/blob/main/docs/README.md
* https://github.com/tektoncd/pipeline/blob/main/docs/pipelines.md
* https://github.com/tektoncd/pipeline/blob/main/docs/pipelines.md
=Internal=
=Internal=
* [[Tekton_Concepts#Pipeline|Tekton Concepts]]
* [[Tekton_Concepts#Pipeline|Tekton Concepts]]
* [[Tekton_Task#Overview|Tekton Task]]
=Overview=
=Overview=
A '''pipeline''' defines a series of ordered [[Tekton_Task#Overview|tasks]]. Just like [[Tekton_Step#Overview|steps]], a task in a pipeline can use as [[#Input|input]] the [[#Output|output]] of a previously executed task. Tekton collects all the tasks, and based on the dependency relationships declared in definition of the pipeline, it orders the tasks in a [[Graph_Concepts#Directed_Acyclic_Graph_.28DAG.29|DAG]], and executes the graph in sequence. That results in the materialization of a number of task pods. The pipeline comes up with additional features, such as the capability to fan-out task execution, retry task execution or verify conditions that tasks must meet before proceeding. The pipeline is implemented as a Kubernetes custom resource.
A '''pipeline''' defines a series of ordered [[Tekton_Task#Overview|tasks]]. Just like [[Tekton_Step#Overview|steps]], a task in a pipeline can use as [[#Input|input]] the [[#Output|output]] of a previously executed task. Tekton collects all the tasks, and based on the dependency relationships declared in definition of the pipeline, it orders the tasks in a [[Graph_Concepts#Directed_Acyclic_Graph_.28DAG.29|DAG]], and executes the graph in sequence. That results in the materialization of a number of task pods. The pipeline comes up with additional features, such as the capability to fan-out task execution, retry task execution or verify conditions that tasks must meet before proceeding. The pipeline is implemented as a Kubernetes custom resource.
Line 56: Line 59:
   name: <font color=indigo><''pipeline-name''></font> <font color=teal>      # required field</font>
   name: <font color=indigo><''pipeline-name''></font> <font color=teal>      # required field</font>
  spec: <font color=teal>                        # required field</font>
  spec: <font color=teal>                        # required field</font>
  description: <font color=indigo>''some description''</font>
   <span id='_workspaces'></span>[[Tekton_Workspaces#Pipeline_Workspaces|workspaces]]:
   <span id='_workspaces'></span>[[Tekton_Workspaces#Pipeline_Workspaces|workspaces]]:
     - name: <font color=indigo>''<workspace-name>''</font>
     - name: <font color=indigo>''<workspace-name>''</font>
Line 66: Line 70:
       type: array
       type: array
       description:  <font color=indigo>''some description''</font>
       description:  <font color=indigo>''some description''</font>
  <span id='_resources'></span>resources:
   <span id='_tasks'></span>[[#Tasks|tasks]]: <font color=teal>                      # required field</font>
   <span id='_tasks'></span>[[#Tasks|tasks]]: <font color=teal>                      # required field</font>
    - name: <font color=indigo>''task1-name''</font>
      <span id='_taskRef'></span>taskRef:
        name: <font color=indigo>''task1-ref-name''</font>
    - name: <font color=indigo>''task2-name''</font>
      <span id='_taskRef'></span>taskRef:
        name: <font color=indigo>''task2-ref-name''</font>
    - ...
  <span id='_finally'></span>[[#Finally|finally]]:
    - name: <font color=indigo>''name''</font>
      taskRef:
        name: <font color=indigo>''task-ref-name''</font>
</font>
=<span id='Tasks'></span>Tasks in Pipeline=
The pipeline specification contains a mandatory list of <code>[[#_tasks|tasks]]</code>. The list specifies the tasks of this pipeline and the details of their execution. The pipeline definition must reference at least one task. Each task must be specified via a valid [[Kubernetes_API_Resources_Concepts#Names|Kubernetes name]] and a <code>[[#_taskRef|taskRef]]</code>.
<font size=-1>
kind: Pipeline
spec:
  tasks:
     - name: <font color=indigo>''task-name''</font>
     - name: <font color=indigo>''task-name''</font>
       <span id='_taskRef'></span>taskRef:
       <span id='_taskRef'></span>taskRef:
         name: <font color=indigo>''task-ref-name''</font>  
         name: <font color=indigo>''task-ref-name''</font>  
        <span id='pipeline_task_bundle'></span>[[Tekton_Bundle|bundle]]: <font color=indigo>docker.com/myrepo/mycatalog</font>
      <span id='_retries'></span>[[#Retries|retries]]: 3
      <span id='_timeout'></span>[[#Task_Timeout|timeout]]: "0h1m30s"
      <span id='_runAfter'></span>[[#runAfter|runAfter]]:
        - <font color=indigo>''some-task-name''</font>
        - <font color=indigo>''some-other-task-name''</font>
      <span id='__params'></span>[[Tekton_Parameters#Task_Declaration_in_Pipeline|params]]:
        - name: <font color=indigo>''param-name''</font>
          value: <font color=indigo>''param-value''</font>
      <span id='_matrix'></span>[[Tekton_Parameters#Parameter_Matrix|matrix]]:
        - name: <font color=indigo>''matrix-name''</font>
          value:
            - <font color=indigo>''value1''</font>
            - <font color=indigo>''value2''</font>
      <span id='_2_workspaces'></span>[[Tekton_Workspaces#Task_Declaration_in_Pipeline|workspaces]]:
        - name: <font color=indigo>''<name>''</font>
          workspace: <font color=indigo>''<workspace>''</font>
      resources:
        inputs:
          - name: <font color=indigo>''input-resource-name''</font>
              resource: <font color=indigo>''input-resource''</font>
        outputs:
          - name: <font color=indigo>''output-resource-name''</font>
              resource: <font color=indigo>''output-resource''</font>
  </font>
  </font>
==Execution Order==
<font color=darkkhaki>
To clarify. Is the order in which the tasks have been declared?
TO PROCESS: https://tekton.dev/docs/pipelines/pipelines/#configuring-the-task-execution-order
</font>
===<tt>runAfter</tt>===
{{External|https://tekton.dev/docs/pipelines/pipelines/#using-the-runafter-field}}
Use <code>runAfter</code> field to indicate that the task it shows up on must run after the tasks declared in the <code>runAfter</code> list. This order is enforced regardless of the order in which the tasks are specified in the pipeline definition.
==Retries==
A task execution can be retired a number of times if the if fails. The number of retires is specified using the <code>[[#_retries|retries]]</code> field in the task declaration in the pipeline.
https://tekton.dev/docs/pipelines/pipelines/#using-the-retries-field
==Conditional Execution with <tt>when</tt>==
<font color=darkkhaki>
TO PROCESS:
* https://tekton.dev/docs/pipelines/pipelines/#guard-task-execution-using-when-expressions
* https://tekton.dev/docs/pipelines/pipelines/#guarding-a-task-and-its-dependent-tasks
* https://tekton.dev/docs/pipelines/pipelines/#guarding-a-task-only
* [[Tekton_Concepts#Conditions|Conditions]] are deprecated.
</font>
==Task Timeout==
<code>[[#_timeout|timeout]]</code>
<font color=darkkhaki>
TO PROCESS:
* https://tekton.dev/docs/pipelines/pipelines/#configuring-the-failure-timeout
</font>
==Resources==
<font color=darkkhaki>TO PROCESS: https://tekton.dev/docs/pipelines/pipelines/#specifying-resources-in-pipelinetasks</font>
=Finally=
One or more tasks specified under the <code>[[#_finally|finally]]</code> section are guaranteed to be executed in parallel after all pipeline tasks have completed, regardless of success or failure.


=Tasks=
<font color=darkkhaki>https://tekton.dev/docs/pipelines/pipelines/#adding-finally-to-the-pipeline</font>
The pipeline specification contains a mandatory list of <code>[[#_tasks|tasks]]</code>. The list specifies the tasks of this pipeline and the details of their execution. The pipeline definition must reference at least one task. Each task must be specified via a valid name and a <code>[[#_taskRef|taskRef</code>.

Latest revision as of 04:16, 29 April 2022

External

Internal

Overview

A pipeline defines a series of ordered tasks. Just like steps, a task in a pipeline can use as input the output of a previously executed task. Tekton collects all the tasks, and based on the dependency relationships declared in definition of the pipeline, it orders the tasks in a DAG, and executes the graph in sequence. That results in the materialization of a number of task pods. The pipeline comes up with additional features, such as the capability to fan-out task execution, retry task execution or verify conditions that tasks must meet before proceeding. The pipeline is implemented as a Kubernetes custom resource.

Example

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: task1
spec:
  steps:
    - name: echo
      image: alpine
      script: |
        #!/bin/sh
        echo "this is Task 1 output"
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: task2
spec:
  steps:
    - name: echo
      image: alpine
      script: |
        #!/bin/sh
        echo "this is Task 2 output"
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: simple-pipeline
spec:
  tasks:
    - name: task1
      taskRef:
        name: task1
    - name: task2
      runAfter:
        - task1
      taskRef:
        name: task2

Manifest

apiVersion: tekton.dev/v1beta1 # required field
kind: Pipeline                 # required field
metadata:                      # required field
  name: <pipeline-name>        # required field
spec:                          # required field
  description: some description
  workspaces:
    - name: <workspace-name>
  params:
    - name: <some-string-param-name>
      type: string
      description: some description
      default: <some-default-value>
    - name: <some-array-param-name>
      type: array
      description:  some description
  resources:
  tasks:                       # required field
    - name: task1-name
      taskRef:
        name: task1-ref-name 
    - name: task2-name
      taskRef:
        name: task2-ref-name 
    - ...
  finally:
    - name: name 
      taskRef:
        name: task-ref-name 

Tasks in Pipeline

The pipeline specification contains a mandatory list of tasks. The list specifies the tasks of this pipeline and the details of their execution. The pipeline definition must reference at least one task. Each task must be specified via a valid Kubernetes name and a taskRef.

kind: Pipeline
spec:
  tasks:
    - name: task-name
      taskRef:
        name: task-ref-name 
        bundle: docker.com/myrepo/mycatalog
      retries: 3
      timeout: "0h1m30s"
      runAfter:
        - some-task-name
        - some-other-task-name
      params:
        - name: param-name 
          value: param-value 
      matrix:
        - name: matrix-name 
          value:
            - value1 
            - value2 
      workspaces:
        - name: <name>
          workspace: <workspace>
      resources:
        inputs:
          - name: input-resource-name 
             resource: input-resource 
        outputs:
          - name: output-resource-name 
             resource: output-resource 

Execution Order

To clarify. Is the order in which the tasks have been declared?

TO PROCESS: https://tekton.dev/docs/pipelines/pipelines/#configuring-the-task-execution-order

runAfter

https://tekton.dev/docs/pipelines/pipelines/#using-the-runafter-field

Use runAfter field to indicate that the task it shows up on must run after the tasks declared in the runAfter list. This order is enforced regardless of the order in which the tasks are specified in the pipeline definition.

Retries

A task execution can be retired a number of times if the if fails. The number of retires is specified using the retries field in the task declaration in the pipeline.

https://tekton.dev/docs/pipelines/pipelines/#using-the-retries-field

Conditional Execution with when

TO PROCESS:

Task Timeout

timeout

TO PROCESS:

Resources

TO PROCESS: https://tekton.dev/docs/pipelines/pipelines/#specifying-resources-in-pipelinetasks

Finally

One or more tasks specified under the finally section are guaranteed to be executed in parallel after all pipeline tasks have completed, regardless of success or failure.

https://tekton.dev/docs/pipelines/pipelines/#adding-finally-to-the-pipeline