When: March 10, 2016, 4:00 PM
Location: 4130, Discovery Building, 330 N. Orchard Street, Discovery Building
Stochastic Optimization for High-Dimensional Large-Scale Learning
As the scale and dimensionality of data continue to grow in many applications of data analytics (e.g., bioinformatics, finance, computer vision, medical informatics), it becomes critical to develop efficient and effective algorithms to solve large-scale machine learning problems. In this talk we introduce novel methods to cope with large number of samples or huge number of features. We first discuss a new paradigm for optimization, dubbed mixed optimization (a.k.a stochastic optimization with variance reduction), which interpolates between stochastic and full gradient methods and is able to i) achieve faster convergence rate in stochastic optimization, and ii) condition number independent convergence rate in deterministic optimization. We then consider learning from high-dimensional data and consider a recovery problem, i.e., how to accurately recover the solution to the optimization problem in the original high-dimensional space based on the solution learned from the subspace spanned by random projections. We present a simple algorithm that uses the dual solution to the low-dimensional optimization problem to recover the solution to the original optimization problem.