Evaluation

GSM8K

GSM8K contains 8,500 elementary school math word problems. Evaluation is exact-match of final answers. This tests arithmetic and reasoning. GSM8K is very challenging without prompting techniques (chain-of-thought helps significantly). Pass rates vary widely by model. GSM8K shows reasoning ability better than raw knowledge. Recent models achieve 90%+ with chain-of-thought. GSM8K is standard for evaluating math reasoning.