Automatic selection of hyper-parameters via the use of softened profile likelihood
Published: Oct 29, 2025
Last Updated: Oct 29, 2025
Authors:Gengyang Chen, Mu Zhu
Abstract
We extend a heuristic method for automatic dimensionality selection, which maximizes a profile likelihood to identify "elbows" in scree plots. Our extension enables researchers to make automatic choices of multiple hyper-parameters simultaneously. To facilitate our extension to multi-dimensions, we propose a "softened" profile likelihood. We present two distinct parameterizations of our solution and demonstrate our approach on elastic nets, support vector machines, and neural networks. We also briefly discuss applications of our method to other data-analytic tasks than hyper-parameter selection.