Airflow XComs: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
Line 9: Line 9:


<font color=darkkhaki>When you call a [[Airflow Concepts#TaskFlow|TaskFlow]] function in the DAG file, rather than executing it, you will get an object representing the XCom for the result (an <code>XComArg</code>, that you can use as inputs to downstream tasks and operators.</font>
<font color=darkkhaki>When you call a [[Airflow Concepts#TaskFlow|TaskFlow]] function in the DAG file, rather than executing it, you will get an object representing the XCom for the result (an <code>XComArg</code>, that you can use as inputs to downstream tasks and operators.</font>
[[Airflow_Concepts#Variable|Variables]] are an alternative mechanism for tasks to share data. However, variables are global and should be used for overall configuration that covers the entire installation. To pass data to and from tasks, XComs should be preferred instead.


=Programming Model=
=Programming Model=

Revision as of 21:31, 11 July 2022

External

Internal

Overview

Tasks communicate using inputs and outputs, and XComs ("cross-communications") intermediates that.

When you call a TaskFlow function in the DAG file, rather than executing it, you will get an object representing the XCom for the result (an XComArg, that you can use as inputs to downstream tasks and operators.

Variables are an alternative mechanism for tasks to share data. However, variables are global and should be used for overall configuration that covers the entire installation. To pass data to and from tasks, XComs should be preferred instead.

Programming Model

Backends

https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html#custom-xcom-backends