Automatic selection of hyper-parameters via the use of softened profile likelihood

Abstract

We extend a heuristic method for automatic dimensionality selection, which maximizes a profile likelihood to identify "elbows" in scree plots. Our extension enables researchers to make automatic choices of multiple hyper-parameters simultaneously. To facilitate our extension to multi-dimensions, we propose a "softened" profile likelihood. We present two distinct parameterizations of our solution and demonstrate our approach on elastic nets, support vector machines, and neural networks. We also briefly discuss applications of our method to other data-analytic tasks than hyper-parameter selection.

Categories