Space is limited
Course logo

Effective Data Orchestration with Airflow

This course provides an introduction to Airflow, starting with foundations and moving through advanced features such as sensors and third party operators. Throughout the course you'll complete projects focused on real-world projects, and be introduced to the Astro SDK. This course will cover Airflow 2.4, the latest release covering a wide range of new improvements.

Instructor profile photo
Henry Weller
Data Engineer at Kevala
Instructor profile photo
Mike Shwe
Senior Director of Product Management at Astronomer
Real-world projects that teach you industry skills.
Learn alongside a small group of your professional peers
Part-time program with 2 live events per week:
Next Cohort
February 6, 2023
Duration
4 weeks
Price
US$ 400
or included with membership

Course taught by expert instructors

Instructor Photo
Affiliation logo

Henry Weller

Data Engineer at Kevala

Henry is an experienced system builder working on solving climate problems with data. He is currently a Data Engineer at Kevala, Inc, where he has designed and built a modern data stack orchestrated in Airflow including Spark, BigQuery, and Google Data Studio. He previously wrote control software for autonomous robotic systems at Osaro, and built a general vendor data pipeline system at Apple. He received his bachelor’s and master’s in Mechanical Engineering and Robotics at Stanford.

Instructor Photo
Affiliation logo

Mike Shwe

Senior Director of Product Management at Astronomer

Mike Shwe is an experienced product leader currently serving as Senior Director of Product Management at Astronomer, a driving force behind Apache Airflow. Mike was previously a Product Manager and Technical Program Manager at Google, Director of Data Engineering at Metaweb Technologies, and VP Products at M-Factor.

The course

Learn and apply skills with real-world projects.

Project
Create a basic deployment of a cluster, a Hello World DAG, and your first useful DAG using the Taskflow API.
    Learn
    • DAGs
    • Operators, Tasks and Task Groups
    • Intro to Taskflow API
    • Astro CLI
    • Airflow Architecture
    • Airflow UI
    Project
    Refactor existing DAG to use dedicated operators, and layer on additional functionality using Sensors, Hooks, Xcoms and different types of Executors,
      Learn
      • Hooks and Dedicated Operators
      • Sensors
      • XComs
      • Custom XCom Backends
      • Dynamic Task Mapping
      Project
      Add a data warehouse for creating external and native tables from data, and performing joins across data sources.
        Learn
        • Linking Disparate Data Sources
        • Stateful vs. Stateless DAG Execution
        • Data Warehouse Operations
        Project
        Refactor existing pipelines using Astro SDK. Create a structure between multiple DAGS, such that one is triggered by another updating a dataset.
          Learn
          • Astro SDK
          • Performing ELT Transformations with cloud-agnostic code
          • Linking Pipelines Together Using Datasets

          Real-world projects

          Work on projects that bring your learning to life.
          Made to be directly applicable in your work.

          Live access to experts

          Sessions and Q&As with our expert instructors, along with real-world projects.

          Network & community

          Core reviews a study groups. Share experiences and learn alongside a global network of professionals.

          Support & accountability

          We have a system in place to make sure you complete the course, and to help nudge you along the way.

          Course success stories

          Learn together and share experiences with other industry professionals

          As a product management professional with 20+ years of experience, I was nervous and excited to enroll in a Masters level course. Henry, a consummate teacher, played a tremendous role in helping me navigate a tough curriculum and be successful in my endeavor to learn. On many occasions, Henry helped clarify foundational elements needed for the course, reviewed specific lecture material, taught concepts from supplementary reading material suggested in the course and even helped me learn other reference material that would reinforce the learning.

          Mohan RangarajanDirector of Advanced Analytics

          Henry uses Airflow to implement data pipeline that handle large volumes of data ingestion, he shared his wealth of practical Airflow knowledge with colleagues to help them build modern data pipeline that are future proof. Henry is very knowledgeable on leveraging different Airflow operators that help improve productivity and performance.

          David LaiSenior Software Engineering Manager at Kevala

          This course is for...

          Software engineers and data scientists new to the field of data engineering who are interested in learning how to build modern data pipelines.

          Data engineers looking to level up their ETL skills, and adopt a general framework for their data use cases.

          Prerequisites

          Ability to write Python and work with documented libraries

          Familiarity with web applications, Docker basics, and the command line

          Nice to have: Familiarity with Kubernetes, working within cloud compute environments such as AWS/GCS

          Frequently Asked Questions

          Keep in touch for updates, discounts, and new courses.

          Questions? Ask us anything at hello@corise.com

          © 2021-2022 CoRise Education