ARC.

The AI2 Reasoning Challenge (ARC) is a benchmark dataset created by the Allen Institute for AI to evaluate artificial intelligence systems' ability to answer grade-school level science questions that require reasoning beyond simple fact retrieval. The dataset consists of multiple-choice questions drawn from standardized tests, split into ARC-Easy and ARC-Challenge sets based on difficulty. ARC has become a standard evaluation metric for measuring AI progress in commonsense reasoning and scientific understanding.

ARC questions are specifically chosen to require multi-step reasoning, scientific knowledge, and the ability to eliminate incorrect answers through logical deduction rather than pattern matching. The ARC-Easy set contains questions that existing information retrieval methods could solve, while ARC-Challenge focuses on questions that stumped retrieval-based systems when the benchmark was created. Questions cover topics like physics, chemistry, biology, and earth science, testing whether models can apply scientific principles to novel scenarios rather than simply memorizing facts.

Performance on ARC serves as a key indicator of model capabilities, with scores often reported alongside other major benchmarks in AI research papers and model evaluations. While modern large language models have achieved high scores on ARC-Easy, ARC-Challenge remains more difficult and continues to differentiate between model capabilities. The benchmark highlights the ongoing challenge of building AI systems that can truly reason about the physical world rather than rely solely on statistical patterns in training data.