OPTML++

• Basic convex analysis and optimization
  • Basics of convex sets and functions
  • Key convex functions
  • Constructing and recognizing convex functions
  • Some convex geometric problems
• Scalable convex optimization
  • First-order methods
  • Parallel
  • Distributed
  • Online learning -- maybe Sasha Rakhlin can help!
  • Sublinear methods
  • Incremental gradient methods
  • Stochastic gradient methods
  • Large-scale Semidefinite programming
• Scalable nonconvex optimization
  • online optimization
  • dictionary learning models and algorithms
  • deep learning algorithms
  • nonconvex SGD, incremental gradient, and other algorithms
• Non-Euclidean optimization
  • Hadamard manifolds
  • Riemannian manifolds
  • Metric spaces of nonpositive curvature
  • Wasserstein metric based problems (probability, geometry)
  • Explicit examples, such as large scale Gaussian mixtures
• Interplay of convex and combinatorial optimization
  • Submodular problems -- potentially led by Stefanie Jegelka
  • Max-flow, graph algorithms -- potentially led by Aleksander Mądry
  • Convex relaxations
• Applications
  • Large-scale regression problems
  • Optimization in deep learning
  • Dictionary learning
  • Applications from machine learning, statistics
  • Applications from finance, signal processing, control, information theory, ...
• Software, design and implementation
  • Data Science platforms: AWS, Azure, Spark, MLBase, Parameter Server, Dato
  • Lessons in implementing asynchronous optimization algorithms
  • Multicore CPU considerations
  • GPU and GPGPU considerations
  • Delay sensitive algorithms and design
  • Other useful software and hardware ideas
• Sublinear algorithms, randomized methods
  • Randomized linear algebra
  • Property testing
  • Hashing, SimHash, dimensionality reduction, etc.
  • Approximation algorithms
• Exploiting symmetry in optimization
  • Optimizing over permutations
  • Schur-convexity, majorization
  • Optimizing over groups
  • Scattering transforms, application to deep learning
• Large-scale sampling, MCMC
  • Parallel Gibbs sampling procedures
  • Large-scale multivariate gaussians
  • MCMC via the optimization lens
  • Scalable determinantal sampling
• Open problems in optimization, ML, statistics, Information theory
  • Entropy inequalities
  • Convexity conjectures
  • Global optimization problems