Anthropic: Introducing The Conceptual Reasoning Index

Anthropic and Redwood Research have introduced the Conceptual Reasoning Index (CRI), a suite of benchmarks designed to evaluate AI models' ability to reason about complex, empirically unverifiable questions critical to managing AI risks—such as AI governance, alignment, and decision theory. The benchmarks include LMCA (a dataset of expert-rated conceptual arguments), ACCoRD, and DTBench, addressing the gap in current AI evaluation methods, which typically rely on abundant data and measurable feedback that are unavailable for many AI safety tasks.