Research-grade scientific coding problems drawn from real physics, biology, and chemistry work.
SciCode asks an agent to write code that solves real scientific problems, the kind a computational researcher would actually implement. Problems are broken into subproblems that build on each other, so the agent has to carry context and produce code that runs and returns the right numbers.
Each subproblem is graded by running the generated code against hidden test cases. The strict main score requires every subproblem in a problem to pass. Scores report both subproblem accuracy and the harder full-problem success rate.
| # | Agent System | Model | Score |
|---|---|---|---|
| 01 | Scicode Tool Calling Agent | o3 Medium (April 2025) | 9.2 |
| 02 | Scicode Zero Shot Agent | o4-mini Low (April 2025) | 9.2 |
| 03 | Scicode Tool Calling Agent | Claude Opus 4.1 (August 2025) | 7.7 |
| 04 | Scicode Tool Calling Agent | Claude Opus 4.1 High (August 2025) | 6.9 |
| 05 | Scicode Tool Calling Agent | GPT-5 Medium (August 2025) | 6.2 |
| 06 | Scicode Zero Shot Agent | o4-mini High (April 2025) | 6.2 |
| 07 | HAL Generalist Agent | o4-mini Low (April 2025) | 6.2 |
| 08 | Scicode Zero Shot Agent | GPT-4.1 (April 2025) | 6.2 |
| 09 | Scicode Tool Calling Agent | Claude-3.7 Sonnet High (February 2025) | 4.6 |
| 10 | Scicode Tool Calling Agent | o4-mini High (April 2025) | 4.6 |
| 11 | Scicode Tool Calling Agent | o4-mini Low (April 2025) | 4.6 |
| 12 | Scicode Zero Shot Agent | o3 Medium (April 2025) | 4.6 |
| 13 | Scicode Tool Calling Agent | Claude-3.7 Sonnet (February 2025) | 3.1 |
| 14 | Scicode Tool Calling Agent | Claude Sonnet 4.5 (September 2025) | 3.1 |
| 15 | HAL Generalist Agent | Claude-3.7 Sonnet High (February 2025) | 3.1 |
| 16 | HAL Generalist Agent | o3 Medium (April 2025) | 3.1 |
| 17 | Scicode Zero Shot Agent | Claude-3.7 Sonnet High (February 2025) | 3.1 |
| 18 | HAL Generalist Agent | Claude-3.7 Sonnet (February 2025) | 3.1 |
| 19 | Scicode Zero Shot Agent | DeepSeek V3 (March 2025) | 3.1 |
| 20 | Scicode Tool Calling Agent | Claude Sonnet 4.5 High (September 2025) | 1.5 |
| 21 | HAL Generalist Agent | GPT-4.1 (April 2025) | 1.5 |
| 22 | HAL Generalist Agent | o4-mini High (April 2025) | 1.5 |
| 23 | Scicode Tool Calling Agent | GPT-4.1 (April 2025) | 1.5 |
| 24 | Scicode Tool Calling Agent | Gemini 2.0 Flash (February 2025) | 1.5 |
| 25 | Scicode Zero Shot Agent | Gemini 2.0 Flash (February 2025) | 1.5 |
No. General coding benchmarks test software engineering. SciCode tests whether an agent can turn scientific descriptions into correct numerical code, which is a narrower and often harder skill.
Browse the other benchmarks on the leaderboard.
