Exploding variance of means of exponentials: least-squares to the rescue

Estimating log-sum-exp functions, which are common in machine learning for normalizing probabilistic models and approximating maxima, suffers from exploding variance that grows exponentially with the magnitude of the potential function values. The author proposes leveraging least-squares regression methods as an alternative approach that maintains the advantages of log-sum-exp functions while avoiding their severe variance and computational challenges. This framework aims to combine the statistical efficiency and computational simplicity of least-squares with the desirable properties of log-sum-exp for probabilistic modeling.