A Modern Replacement For Airflow

Mage is an open-source data pipeline tool for transforming and integrating data. Mage is a modern replacement for Airflow.

Stars: 2.9k
License:  Apache-2.0
Languages: Python(61.1%), TypeScript(33.9%)
Link: https://github.com/mage-ai/mage-ai

Key Features

Orchestration

Schedule and manage data pipelines with observability.

Notebook

Interactive Python, SQL, & R editor for coding data pipelines.

Data Integrations

Synchronize data from 3rd party sources to your internal destinations.

Streaming Pipelines

Ingest and transform real-time data.

DBT

Build, run, and manage your DBT models with Mage.

Mage Gives Your Data Team Magical Powers

• Integrate and synchronize data from 3rd party sources.
• Build real-time and batch pipelines to transform data using Python, SQL, and R.
• Run, monitor, and orchestrate thousands of pipelines without losing sleep

1. Build

Have you met anyone who said they loved developing in Airflow? That’s why we designed an easy developer experience that you’ll enjoy.

• Easy developer experience: Start developing locally with a single command or launch a dev environment in your cloud using Terraform.
• Language of choice: Write code in Python, SQL, or R in the same data pipeline for ultimate flexibility.
• Engineering best practices built-in: Each step in your pipeline is a standalone file containing modular code that’s reusable and testable with data validations. No more DAGs with spaghetti code.

2. Preview

Stop wasting time waiting around for your DAGs to finish testing. Get instant feedback from your code each time you run it.