- Services
Expert-written science questions that PhD researchers can barely solve and Google searches cannot answer.
GPQA tests whether a model can reason through hard graduate-level science problems on its own. The questions are written by domain PhDs and explicitly checked to be "Google-proof," which means a smart non-expert with 30 minutes and a search engine still cannot solve them. That setup measures real subject-matter reasoning, not retrieval or pattern matching against the open web.
Each question is multiple choice with one correct answer and a fixed set of distractors. Models are evaluated zero-shot or with a short reasoning prompt. Scores are reported as percent of questions answered correctly. The harder "Diamond" subset of 198 questions is the slice most labs publish numbers on.
Benchmark data sourced from: Hugging Face, Arena.ai, Artificial Analysis.
A particle is in a 1D infinite potential well of width L. If the particle is in the ground state, what is the probability of finding it in the region between L/4 and 3L/4?
| # | Model | Lab | Source | Score |
|---|---|---|---|---|
| 01 | GPT-5.6 Sol | OpenAI | Closed | 94.1 |
| 02 | Gemini 3.1 Pro Preview | Closed | 94.1 | |
| 03 | Kimi K3 | Moonshot AI | Open | 93.5 |
| 04 | GPT-5.5 | OpenAI | Closed | 93.5 |
| 05 | GPT-5.2 | OpenAI | Closed | 93.2 |
| 06 | Grok 4.5 | xAI | Closed | 93.1 |
| 07 | MiniMax M3 | MiniMax | Open | 92.9 |
| 08 | Qwen3.8-Max | Alibaba | Open | 92.8 |
Ad Tools for AI BuildersAll the hardware and tools you need to run private AI at your office or at home.Shop Now | ||||
| 09 | GPT-5.4 | OpenAI | Closed | 92.8 |
| 10 | Claude Fable 5 | Anthropic | Closed | 92.6 |
| 11 | GPT-5.6 Terra | OpenAI | Closed | 92.5 |
| 12 | Qwen 3.7 Max | Alibaba | Closed | 92.3 |
| 13 | Gemini 3.5 Flash | Closed | 92.2 | |
| 14 | Claude Opus 4.8 | Anthropic | Closed | 92.0 |
| 15 | GPT-5.4 High | OpenAI | Closed | 92.0 |
A human PhD in the relevant field scores about 65% on the Diamond subset, while a non-expert with Google scores around 34%. Frontier models in 2026 are in the 80–90% range; mid-tier open-source models land between 45% and 65%.
Every question was checked by non-expert validators who had open web access and 30 minutes per question. Only questions that the validators could not solve made it into the final dataset, so a model has to reason rather than retrieve.
No. MMLU is broad and shallow: high school through college knowledge across 57 subjects. GPQA is narrow and deep: graduate-level reasoning in three hard sciences. Strong MMLU scores do not guarantee strong GPQA scores.
Partially. Subject knowledge sets a floor, but the questions require multi-step reasoning that textbook memorization alone cannot solve. The best scores correlate with chain-of-thought ability, not training corpus size.
84 model(s) with undisclosed parameter counts not shown. Most closed-source labs do not publish model size.
