The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"

Researchers discovered the "Reversal Curse," a significant generalization failure in large language models where training on a statement like "A is B" does not enable the model to understand or answer "B is A"—for example, models trained on "Valentina Tereshkova was the first woman to travel to space" cannot answer "Who was the first woman to travel to space?" The phenomenon persists across different model sizes and families, is not fixed by data augmentation, though models can deduce reverse relationships when both statements appear in context together. Testing on GPT-4 showed it correctly answers celebrity questions 79% of the time in one direction but only 33% in the reverse direction.