MLOps: From Models to Production
Acquire the skills to build effective real-world ML systems (bootstrapping datasets, improving label quality, experimentation, model evaluation, deployment and observability) with hands-on projects. This course will help you bridge the gap between state-of-the-art ML modeling, and building real-world ML systems.
Nihit Desai
CTO of Refuel.AI (ex-Facebook, Stanford)
Nihit Desai is the CTO and co-founder of Refuel.AI, an early stage ML infrastructure startup. Prior to this, he was a Staff Engineer at Facebook where he led ML efforts for content moderation. In prior roles, he has worked on large scale recommender systems at Instagram, and on search quality at LinkedIn. He holds a Masters degree in Computer Science from Stanford, specializing in Artificial Intelligence.
Learn the skills and best practices for building effective real-world ML systems. Some examples include: bootstrapping data for new ML projects, improving data & label quality, experiment tracking, model evaluation, deployment and observability. The goal of this course is to help bridge the gap between state-of-the-art ML modeling, and building real-world ML systems.
Week by week, we will develop an understanding of the role of MLOps in building and maintaining real-world ML systems with hands-on projects using Python (e.g. Sklearn, Numpy and Pytorch) and industry-standard infrastructure tools (e.g. Docker). Position yourself to capitalize on this growing area as a member of our beta cohort.
- Archetypes of real-world ML applications
- The production ML lifecycle
- Why data quality and quantity are critical for real-world ML success
- Exploratory data analysis
- Model training & hyperparameter optimization
- Fine-tuning state-of-the-art pretrained transformer models for NLP tasks
- Designing good model evaluation metrics
- Model underfitting and overfitting: what are they, and how to address them
- Behavioral testing for ML models
- Establish bounds on model performance with human annotation baseline
- Behavioral testing for ML models
- Testing for statistical properties of datasets
- Options for deploying models online: common scenarios & tradeoffs
- Feature Stores
- Good practices to ensure production stability: gated rollouts, shadow mode deployment, online experimentation, and easy rollbacks
- Wrap the model and data pipeline in a python FastAPI web service
- Containerize the service using Docker
- Basic integration testing for containerized service
- Deploy the service and test it online
- How MLOps practices evolve as a function of team and company maturity
- Logging and monitoring infrastructure for ML applications
- Data and concept drift in Machine Learning
- CI/CD for ML models
- Statistical data and concept drift measures
- Model performance measurement
- Outlier detection
Nihit has a rare set of skills and experiences - building large-scale ML production systems at top companies, along with a solid and rigorous research background. Along with that, he is great at distilling and passing on his hard-won insights and knowledge. I've learned a lot from his newsletter and the talks he's given to large audiences at Upstart - so I know first-hand how valuable and practical this class will be, and can't think of a better instructor!
Nihit has extensive experience building ML systems for recommendations, ranking and integrity problems at Facebook and LinkedIn. His expertise lies not only in developing and improving deep learning techniques but also in working with large scale systems that scale to billions of users. It’s a combination of both these skill sets that makes him a great fit to teach an MLOps course that requires an in-depth understanding of ML fundamentals and the ability to build out scalable systems that deal with constantly growing and ever-changing datasets in the real-world.
Nihit combines a deep theoretical understanding of ML with hands-on practical knowledge from having built large-scale search, recommender, and decisioning ML systems at the most impactful Internet companies. If I had to learn how to go from an idea to a working, scalable ML system, there would be no better instructor than Nihit!
Software engineers who want to build production systems that integrate ML
Data scientists who want to get hands on experience with the production ML lifecycle
Students/recent college grads who want to learn about building and shipping ML applications
Knowledge of basic machine learning concepts.
Familiarity with software development in Python.
Recommended: Familiarity with Docker, cloud ecosystems such as AWS.
