6.881 Optimization for Machine Learning

INTRODUCTION

Welcome to 6.881. This is the homepage for the course: Optimization for Machine Learning (OPTML) that I am teaching (second time) in SPRING 2021. OPTML covers topics from convex, nonconvex, continuous, and combinatorial optimization, especially motivated by the needs of problems and applications in Machine Learning. The topics covered range from foundational material to research-edge topics.

ORGANIZATION

Lectures: TTh 11:00AM–12:30PM
Virtual Location: Canvas Site
Instructor: Suvrit Sra
Teaching Assistants: Kwangjun Ahn
TA Office Hours: Thu, 2pm-3pm
Grading policy: Homeworks (45%), Project (50%); Peer-review (5%)
PSET Policy
Project Guidelines

Schedule (Subject to change)

Check Canvas Site for the course.

L#	Date	Topic	Links	Notes
1	Feb 16	Foundations: convex sets, functions	L01	HW1 out
2	Feb 18	Foundations: conjugates, subdifferentials.	L02
3	Feb 23	Foundations: weak and strong duality	L03
4	Feb 25	Foundations: Optimality conditions, KKT	L04
5	Mar 02	Nonconvex optimality & stationarity	L05	HW1 In; HW2 Out
6	Mar 04	Tractable nonconvex problems	L06
	Mar 09	NO CLASS		NO CLASS
7	Mar 11	First-order methods: gradient descent and SGD	L07
8	Mar 16	Accelerated gradient, subgradient method	L08
9	Mar 18	Faster stochastic methods, variance reduction	L09	HW2 In; HW3 Out
	Mar 23	NO CLASS		NO CLASS
10	Mar 25	Frank-Wolfe methods	L10
11	Mar 30	Operator splitting: proximal gradient, DR, and more	L11
12	Apr 01	Coordinate descent, BCD, Altmin	L12	HW3 In; HW4 Out
13	Apr 06	The EM method, CCCP, and related ideas	L13
14	Apr 08	Gradient flows and diffusion (guest lecture)	L14
15	Apr 13	Min-Max Problems I (convex)	L15
	Apr 15	CANCELED		CANCELED
	Apr 20	NO CLASS		NO CLASS
16	Apr 22	Min-Max Problems II (nonconvex)	L16
17	Apr 27	Non-Euclidean Optimization I	L17	HW4 In
18	Apr 29	Non-Euclidean Optimization II	L18
19	May 04	Neural network optimization I	L19
20	May 06	Neural network optimization II	L20
21	May 11	Interior point methods intro	L21
22	May 13	Submodular optimization intro	L22
23	May 18	Project Presentations – I
24	May 20	Project Presentations – II