Airflow Concepts: Difference between revisions
Jump to navigation
Jump to search
(→Task) |
(→DAG) |
||
| Line 8: | Line 8: | ||
{{External|https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html}} | {{External|https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html}} | ||
{{Internal|Graph_Concepts#Directed_Acyclic_Graph_.28DAG.29|Graph Concepts | Directed Acyclic Graph}} | {{Internal|Graph_Concepts#Directed_Acyclic_Graph_.28DAG.29|Graph Concepts | Directed Acyclic Graph}} | ||
==SubDAG== | |||
{{External|https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html#concepts-subdags}} | |||
=Task= | =Task= | ||
{{External|https://airflow.apache.org/docs/apache-airflow/stable/concepts/tasks.html}} | {{External|https://airflow.apache.org/docs/apache-airflow/stable/concepts/tasks.html}} | ||
Revision as of 00:57, 11 July 2022
External
Internal
Workflow
DAG
SubDAG
Task
Tasks have dependencies on each other.
Operator
Sensor
A Sensor is a subclass of Operator.
TaskFlow-decorated Task
Passing Data between Tasks
Tasks pass data among each other using:
- XComs, when the amount of metadata to be exchanged is small.
- Uploading and downloading large files from a storage service.
XComs
"Cross-communications".