Neural Networks Are Already Clean

Researchers studying neural networks have found that polysemanticity—neurons appearing to perform multiple unrelated functions—is not caused by superposition as previously believed, but rather by ordinary, fixable issues like recursive generalization. This suggests neural networks are fundamentally more interpretable than the field assumed, and mechanistic interpretability can proceed by directly studying networks rather than first untangling them using decomposition methods like sparse autoencoders.