Apache Airflow: Orchestrating Hybrid Workloads in the Cloud

2021-01-21T12:14:07+00:00

This article describes a hybrid approach that we use to manage a data lake by handling heterogeneous workloads with the help of Apache Airflow, Kubernetes and Apache Spark on EMR.

At inovex we use Apache Airflow as a scheduling and orchestration tool in a wide range of d