This course marries scalable machine learning systems with deep learning, and helps students to work on problems of how to scale machine learning and deep learning. We will look at scaling problems from architecture (serverless machine learning), data (scaling feature engineering and training with Big data), and compute (data/model parallel deep learning). At the end of the course students will be familiar with the main deep learning algorithms and know how to scale machine learning systems on the cloud and apply them on massive datasets. This course has a system-based focus, that is, student will learn not only the theory of machine learning and deep learning, but also the practical aspects of building large scale systems that take advantage of machine learning and deep learning. The course is divided into two parts. The focus of the first part is to introduce deep learning, and the goal of the second part is to present the scalable machine learning approaches.