Apache Airflow
Open-source platform to author, schedule, and monitor data pipelines as Python DAGs
FreeOpen source
Open-source platform to author, schedule, and monitor data pipelines as Python DAGs
Airflow defines pipelines as Python DAGs built from operators. It handles scheduling and backfills and has a web UI with logs for monitoring. It is Apache 2.0 licensed and self-hosted. Managed versions are available as Google Cloud Composer and Amazon MWAA.
Durable, code-first workflow orchestration that survives failures
Fair-code workflow automation you can self-host, with a visual builder and code when you need it
Open-source framework for Python ML and AI pipelines with artifact lineage
Open-source developer platform for scripts, workflows, and internal apps