Benchmarks · 2023

IFBench: Instruction Following Benchmark

Name: IFBench: Instruction Following Benchmark
Creator: Google Research
Published: 2023
Keywords: IFBench, AI benchmark, text model evaluation, Google Research

Tests whether a model obeys precise formatting, length, and constraint rules.

Open Dataset Read Paper

Models Tested

Top Score

81.3

Published

2023

Source

Google Research

How It Works

IFBench measures whether a model actually follows precise instructions. Each prompt has verifiable rules: respond in exactly four sentences, include the word "elephant" twice, output JSON with these exact keys. The benchmark cares about obedience, not creativity, which makes it a strong signal for production reliability.

Each prompt has one or more programmatic verifiers. The model’s output is checked rule-by-rule. The score is the fraction of rules satisfied, averaged across prompts.

Dataset size

500+ prompts with verifiable instructions like word counts, JSON formatting, language constraints, and section structure.

Mean score

60.0

Median score

67.7

Open / Closed

27 / 31

Top Scorers

#	Model	Lab	Source	Score
01	Grok 4.3	xAI	Closed	81.3
02	MiMo-V2.5-Pro	Xiaomi	Closed	79.9
03	DeepSeek-V4-Flash	DeepSeek	Open	79.2
04	Qwen3.5-397B-A17B	Alibaba	Open	78.8
05	Gemini 3.1 Flash Lite Preview	Google	Closed	77.2
06	Gemini 3.1 Pro Preview	Google	Closed	77.1
07	Qwen3.6 Max Preview	Alibaba	Closed	76.6

Score Distribution

Open vs Closed Source

Gap on IFBench:+2.1pts closed leads

Top Open-Source Models

1DeepSeek-V4-Flash79.2
2Qwen3.5-397B-A17B78.8
3DeepSeek-V4-Pro76.5

Top Closed-Source Models

1Grok 4.381.3
2MiMo-V2.5-Pro79.9
3Gemini 3.1 Flash Lite Preview

Score vs Parameter Count

30 model(s) with undisclosed parameter counts not shown. Most closed-source labs do not publish model size.

Average Score by Lab

MiniMax
73.7n = 2
Xiaomi
71.9n = 3
Moonshot AI
71.4n = 3
Alibaba
70.1n = 10
Z.ai
63.3n = 4

Most Correlated Benchmarks

SWE-Pro
+0.97n = 10
MATH-500
+0.88n = 12
LiveCodeBench
+0.85n = 26
AA Intelligence Index
+0.82n = 58
HLE

What It Captures Well

Programmatic scoring means no judge model is needed.
Strong predictor of production usefulness for structured output.
Catches models that are clever but disobedient.

Where It Falls Short

Does not measure response quality, only rule adherence.
A model can pass every rule and still give a bad answer.
Some rules are subjective (tone, formality).

Frequently Asked Questions

Why does IFBench matter for production?

Real applications need predictable output: JSON with specific keys, summaries with specific lengths, responses in specific languages. A high IFBench score means the model will hold up when the prompt has hard constraints.