> ML_BENCHMARK // GSM8K-GRADE-SCHOOL-MATH_v1.0
GSM8K (Grade School Math Multi-step Reasoning)
Mathematical Reasoning · task-text-generation · near-saturation
Mathematical Reasoningnear-saturationmoderate
Evaluation Protocol
8-shot chain-of-thought generation followed by numerical regex answer extraction on 1,319 test problems.
Baseline & Metrics
Canonical Baseline:GPT-3: 35.0% | GPT-4: 92.0% | DeepSeek-R1: 97.2%
Evaluated Metrics:
Strict Match Accuracy (%)
Contamination & Leakage Risks
High contamination in frontier training sets; prompt phrasing variations test true arithmetic generalization.
Reproducibility Concerns
Minor differences in numerical parsing regex can alter reported scores by 1-2%.
CONNECTED DATASETGSM8K (Grade School Math 8K)Mathematical Reasoning & Code Generation · 8,500 grade school math word problems with step-by-step solutions
View Dataset Spec