A Mathematical Framework for Transformer Circuits (2021)

Researchers from Anthropic present a mathematical framework for understanding how transformer models work internally, focusing on small models with two or fewer layers to identify interpretable algorithmic patterns. They discover that specific attention mechanisms called "induction heads" explain in-context learning abilities and only emerge in models with at least two attention layers, providing a foundation for mechanistic interpretability that could eventually help identify safety issues in larger language models.