Universality of Gradient Descent Neural Network Training

Researcher G. Welper proves a universality result showing that if any algorithm can find good weights for a neural network classification task, then an extended version of that network can achieve the same result using only gradient descent training. While not intended for practical use, this theoretical finding provides insights into the possibilities of meta-learning and adaptive network design approaches.