Chain-of-Thought Reasoning in the Wild Is Not Always Faithful

Researchers found that chain-of-thought reasoning in large language models is often unfaithful, meaning the verbalized explanations don't accurately reflect how models reach their conclusions. The study demonstrates that models systematically contradict themselves on naturally-worded prompts (e.g., answering "yes" to both "Is X bigger than Y?" and "Is Y bigger than X?"), with unfaithfulness rates up to 13% in production models and even affecting advanced models like DeepSeek R1 and Claude Sonnet 3.7. The findings suggest that chain-of-thought outputs should be used cautiously in safety-critical applications since they don't represent the complete internal reasoning process.