EECS 498: Principles of Machine Learning, Fall 2022

Course Instructor: Prof. Laura Balzano, Prof. Qing Qu, Prof. Lei Ying

(listed in alpha beta order, please contact Prof. Qing Qu for more details)

Teaching Assistant: Alexander Ritchie

Title: Principles of Machine Learning

Course Time: Mon/Wed 3:00 PM – 4:30 PM,  3 credit hour

Office Hour: Wed 3:30 PM – 5:00 PM

Prerequisite: EECS 351, or EECS 301, or any linear algebra courses

Notice: This is an entry-level machine learning course targeted for senior undergraduate and junior master students. This course is a little bit more emphasis on mathematical principles in comparison to EECS 445. Students outside the ECE program interested in machine learning are welcome as well!

Overview: The class will cover basic principles in machine learning, such as unsupervised learning (e.g., clustering, mixture models, dimension reduction), supervised learning (e.g., regression, classification, neural networks & deep learning), and reinforcement learning. For each topic, key algorithmic ideas/intuitions and basic theoretical insights will be highlighted.

Course Materials: slides and videos will be accessed via Canvas (TBA). Tentative topics that will be covered in this course are supervised learning, unsupervised learning, and reinforcement learning:

• Basics of probability, linear algebra, and optimization
• Regression and linear prediction
• Support vector machines and kernel methods
• Deep neural networks
• Dimension reduction: PCA, autoencoder
• Clustering (Kmeans, Mixture of Gaussians, EM)
• Representation learning: nonnegative matrix factorization, dictionary learning
• Basics of Online/reinforcement learning

Assessment: (i) homework assignment (5 in total, 40%), (ii) mid-term exam (30%), (iii) Final project (30%)

Textbook: We recommend the following books and articles, although we will not follow them closely.

• Foundations of Machine Learning, by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar.
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction, by Trevor Hastie, Robert Tibshirani, and Jerome Friedman.
• Deep Learning, by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.
• Mathematics for Machine Learning, by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong.
• Linear Algebra and Optimization for Machine Learning, by Charu C. Aggarwal.

Related courses:

• EECS 445. Introduction to Machine Learning
• EECS 453. Applied Matrix Algorithms for Signal Processing, Data Analysis, and Machine Learning
• EECS 505. Computational Data Science and Machine Learning
• EECS 545. Machine Learning

Syllabus: (tentative)

Week	Date	Topic	Contents	Instructor	Homework, Review
Week-1-1	08/30	Introduction	Course overview	Qing Qu
Week-1-2	09/01	Supervised Learning	Introduction to supervised learning, linear models, regularization	Qing Qu	Linear Algebra Review
Week-2-1	09/06	Labor Day	No class
Week-2-2	09/08	Supervised Learning	Logistic Regression	Qing Qu	Probability Review, HW1 Release
Week-3-1	09/13	Supervised Learning	Logistic Regression continued, optimization methods	Qing Qu
Week-3-2	09/15	Supervised Learning	Optimization methods	Qing Qu	Python Review
Week-4-1	09/20	Supervised Learning	Linear classification, support vector machine (SVM) I	Qing Qu
Week-4-2	09/22	Supervised Learning	Linear classification, support vector machine (SVM) II	Qing Qu	HW1 Due, HW2 Release
Week-5-1	09/27	Supervised Learning	Dual SVM	Qing Qu
Week-5-2	09/29	Supervised Learning	Nonlinear models, kernel methods	Qing Qu
Week-6-1	10/04	Supervised Learning	Introduction to deep neural networks I	Qing Qu
Week-6-2	10/06	Supervised Learning	Introduction to deep neural networks II	Qing Qu	HW2 Due,  HW3 Release
Week-7-1	10/11	Supervised Learning	Introduction to deep neural networks III	Qing Qu
Week-7-2	10/13	Unsupervised Learning	Introduction to unsupervised learning, clustering problem, K-means	Qing Qu
Week-8-1	10/18	Fall Study Day	No class
Week-8-2	10/20	Unsupervised Learning	K-means, mixtures of Gaussian, expectation maximization	Qing Qu	HW3 Due
Week-9-1	10/25	Midterm Review	Midterm Review	Qing Qu
Week-9-2	10/27	Midterm	Midterm	Qing Qu
Week-10-1	11/01	Unsupervised Learning	Dimension reduction, PCA	Qing Qu
Week-10-2	11/03	Unsupervised Learning	Dimension reduction II	Qing Qu	HW4 Release, Project Proposal Due
Week-11-1	11/08	Unsupervised Learning	Representation learning, matrix factorization	Qing Qu
Week-11-2	11/10	Unsupervised Learning	Representation learning, matrix factorization	Qing Qu
Week-12-1	11/15	Unsupervised Learning	Autoencoder, self-supervised learning, GAN	Qing Qu
Week-12-2	11/17	Unsupervised Learning	Autoencoder, self-supervised learning, GAN	Qing Qu	HW4 Due, HW5 Release
Week-13-1	11/22	Reinforcement Learning	TBD	Lei Ying
Week-13-2	11/24	Thanksgiving	no class
Week-14-1	12/01	Reinforcement Learning	TBD	Lei Ying
Week-14-2	12/03	Reinforcement Learning	TBD	Lei Ying	HW5 Due
Week-15-1	12/06	Final	Final project preparation	Qing Qu
Week-15-2	12/08	Final	Final project presentation	Qing Qu