Space is limited
Course logo

Effective Data Orchestration with Airflow

This course provides an introduction to Airflow, starting with foundations and moving through advanced features such as sensors and third party operators. Throughout the course you'll build standard ingestion, model training, and data warehouse-based pipelines, and work with both the Taskflow API and the Astro SDK. This course will cover features through Airflow 2.5, the latest release covering a wide range of new improvements.

Instructor profile photo
Henry Weller
Data Engineer at MongoDB
Instructor profile photo
Mike Shwe
Senior Director of Product Management at Astronomer
Real-world projects that teach you industry skills.
Learn alongside a small group of your professional peers
Part-time program with 2 live events per week:
Monday @ 5:00 PM UTC
Project Session
Wednesday @ 5:00 PM UTC
Next Cohort
July 10, 2023
4 weeks
US$ 400
or included with membership

Course taught by expert instructors

Instructor Photo
Affiliation logo

Henry Weller

Data Engineer at MongoDB

Henry is an experienced system builder working on making data infrastructure useful in various domains. He is currently a Data Engineer at MongoDB, and has held previous roles at Kevala, where he built a modern data stack orchestrated in Airflow, and Osaro, where he wrote control software for autonomous robotic systems.

Instructor Photo
Affiliation logo

Mike Shwe

Senior Director of Product Management at Astronomer

Mike Shwe is an experienced product leader currently serving as Senior Director of Product Management at Astronomer, a driving force behind Apache Airflow. Mike was previously a Product Manager and Technical Program Manager at Google, Director of Data Engineering at Metaweb Technologies, and VP Products at M-Factor.

The course

Learn and apply skills with real-world projects.

Who is it for?
  • Software engineers and data scientists new to the field of data engineering who are interested in learning how to build modern data pipelines.

  • Data engineers looking to level up their ETL skills, and adopt a general framework for their data use cases.

  • Ability to write Python and work with documented libraries

  • Familiarity with web applications, Docker basics, and the command line

  • Nice to have: Familiarity with Kubernetes, working within cloud compute environments such as AWS/GCS

Not ready?

Try these prep courses first

  • DAGs
  • Operators, Tasks and Task Groups
  • Intro to Taskflow API
  • Astro CLI
  • Airflow Architecture
  • Airflow UI
Create a basic deployment of a cluster and your first useful DAG for processing timeseries energy data using the Taskflow API.
  • Sensors
  • XComs
  • Custom XCom Backends
  • Dynamic Task Mapping
Build an energy price prediction ML pipeline and use XComs and dynamic task mapping to select the best model and save it to GCS.
  • Hooks and Dedicated Operators
  • Data Warehouse Transformations
  • Large Volume Workflows
  • Control Plane vs Data Plane
Use dedicated BigQuery Operators to create external tables and views from data, perform SQL transformations and joins across data sources.
  • Astro SDK
  • Data Warehouse Transformations Revisited
  • Linking Pipelines Together Using Datasets
  • Making Airflow more accessible to data analysts and SQL experts
Write an Astro SDK DAG that tightly couples Airflow tasks with data assets using File and Table abstractions.

Real-world projects

Work on projects that bring your learning to life.
Made to be directly applicable in your work.

Live access to experts

Sessions and Q&As with our expert instructors, along with real-world projects.

Network & community

Core reviews a study groups. Share experiences and learn alongside a global network of professionals.

Support & accountability

We have a system in place to make sure you complete the course, and to help nudge you along the way.

Course success stories

Learn together and share experiences with other industry professionals

As a product management professional with 20+ years of experience, I was nervous and excited to enroll in a Masters level course. Henry, a consummate teacher, played a tremendous role in helping me navigate a tough curriculum and be successful in my endeavor to learn. On many occasions, Henry helped clarify foundational elements needed for the course, reviewed specific lecture material, taught concepts from supplementary reading material suggested in the course and even helped me learn other reference material that would reinforce the learning.

Mohan RangarajanDirector of Advanced Analytics

Henry uses Airflow to implement data pipeline that handle large volumes of data ingestion, he shared his wealth of practical Airflow knowledge with colleagues to help them build modern data pipeline that are future proof. Henry is very knowledgeable on leveraging different Airflow operators that help improve productivity and performance.

David LaiSenior Software Engineering Manager at Kevala

Frequently Asked Questions

Still not sure?

Get in touch and we'll help you decide.

Keep in touch for updates, discounts, and new courses.

Questions? Ask us anything at

© 2021-2022 CoRise Education