Linear regression with known noise distribution up to a scale: The reward of not using the OLSE
Abstract
While the ordinary least squares estimator (OLSE) is still the most used estimator in linear regression models, other estimators can be more efficient when the error distribution is not Gaussian. In this paper, our goal is to evaluate this efficiency in the case of the Maximum Likelihood estimator (MLE) when the noise distribution belongs to a scale family. Under some regularity conditions, we show that (\beta_n,s_n), the MLE of the unknown regression vector \beta_0 and the scale s_0 exists and give the expression of the asymptotic efficiency of \beta_n over the OLSE. For given three scale families of densities, we quantify the true statistical gain of the MLE as a function of their deviation from the Gaussian family. To illustrate the theory, we present simulation results for different settings and also compare the MLE to the OLSE for the real market fish dataset.