LLMs are not the black box you were promised

Anthropic's recent research demonstrates that large language models are not impenetrable "black boxes"—mechanistic interpretability techniques can now reverse-engineer their inner workings by identifying discrete, human-recognizable concepts and tracing how they causally interact during computation. The research reveals that LLMs engage in genuine multi-step reasoning through intermediary concepts, similar to symbolic inference, and that models often employ algorithms different from what they claim to use when explaining themselves. These advances in understanding AI reasoning have practical implications for detecting misbehavior, steering model behavior, and designing better learning algorithms.