Skip to main content

> ML_BENCHMARK // MATH-COMPETITION-BENCHMARK_v1.0

MATH (Hendrycks High School Competition Benchmark)

Mathematical Reasoning · task-text-generation · active-headroom

Mathematical Reasoningactive-headroomhigh-vram-dependent
Official LeaderboardBack to Benchmarks

Evaluation Protocol

5-shot chain-of-thought generation over 5,000 challenging competition math problems with SymPy symbolic equivalence checking.

Baseline & Metrics

Canonical Baseline:GPT-4 (original): 42.5% | Claude 3.5 Sonnet: 71.1% | DeepSeek-R1: 90.8%
Evaluated Metrics:
Exact Match Accuracy on Final Answer (%)

Contamination & Leakage Risks

Contamination present; symbolic math verification mitigates memorized formatting quirks.

Reproducibility Concerns

Symbolic equivalence checking requires precise SymPy simplification rules for complex algebraic expressions.

CONNECTED DATASETMATH: Measuring Mathematical Problem Solving (Hendrycks et al. 2021)Advanced Mathematical Reasoning · 12,500 high school competition math problems (AMC 10, AMC 12, AIME)
View Dataset Spec