Machine Learning Seminar
Speaker: Rahul Mazumder, MIT Sloan School of Management
Title: New directions in solving structured nonconvex problems in multivariate statistical learning
Refreshments available at 2 pm.
Date: Mon, 09 Apr 2018, 2:15 pm – 3:15 pm
Location: 1400 BPS Bldg.
Nonconvex problems arise frequently in modern applied statistics and machine learning, posing outstanding challenges from a computational and statistical viewpoint. Continuous especially convex optimization, has played a key role in our computational understanding of (relaxations or approximations of) these problems. However, some other well-grounded techniques in mathematical optimization (for example, mixed integer optimization) have not been explored to their fullest potential. When the underlying statistical problem becomes difficult, simple convex relaxations and/or greedy methods have shortcomings. Fortunately, many of these can be ameliorated by using estimators that can be posed as solutions to structured discrete optimization problems. To this end, I will demonstrate how techniques in modern computational mathematical optimization (especially, discrete optimization) can be used to address the canonical problem of best-subset selection and cousins. I will describe how recent algorithms based on local combinatorial optimization can lead to high quality solutions in times comparable to (or even faster than) the fastest algorithms based on L1-regularization. I will also discuss the relatively less understood low Signal to Noise ratio regime, where usual subset selection performs unfavorably from a statistical viewpoint; and propose simple alternatives that rely on nonconvex optimization. If time permits, I will outline instances in robust statistics (least median squares/trimmed squares), nonparametric function estimation and low-rank factor analysis where nonconvex problems arise naturally, and techniques based on mathematical optimization seem to be promising.
Rahul Mazumder is an Assistant Professor in the Operations Research and Statistics group at MIT Sloan School of Management, where he is also affiliated with the Operations Research Center and a core faculty member of MIT's Center for Statistics. Before joining MIT, he was an Assistant Professor at Columbia University (Dept of Statistics), where he was also affiliated with the Data Science Institute. He completed a Ph.D. in Statistics from Stanford University under the supervision of Trevor Hastie. He is a recipient of the Office of Naval Research Young Investigator Award (2018).