Airflow

External

• https://airflow.apache.org/docs/apache-airflow/stable/start/index.html
• https://www.datarevenue.com/en-blog/airflow-vs-luigi-vs-argo-vs-mlflow-vs-kubeflow
• Release Notes https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html

Internal

• Workflow Orchestration Tools

Overview

Airflow is a platform to programmatically author, schedule and monitor workflows. The workflows are expressed as DAGs of tasks.

Subjects

• Concepts
• Operations
• Programming Model

Organizatorium

• Users schedule batch jobs in Airflow.
• Celery Executor: https://airflow.apache.org/docs/apache-airflow/stable/executor/celery.html
• Kubernetes Executor introduced in Airflow 1.10.0
• Dag Sync