Benchmarks · 2023

IFBench: Instruction Following Benchmark

Name: IFBench: Instruction Following Benchmark
Creator: Google Research
Published: 2023
Keywords: IFBench, AI benchmark, text model evaluation, Google Research

Tests whether a model obeys precise formatting, length, and constraint rules.

Open Dataset Read Paper

Models Tested

133

Top Score

82.9

Published

2023

Source

Google Research

Tools for AI Builders—All the hardware and tools you need to run private AI at your office or at home.Shop Now

How It Works

IFBench measures whether a model actually follows precise instructions. Each prompt has verifiable rules: respond in exactly four sentences, include the word "elephant" twice, output JSON with these exact keys. The benchmark cares about obedience, not creativity, which makes it a strong signal for production reliability.

Each prompt has one or more programmatic verifiers. The model’s output is checked rule-by-rule. The score is the fraction of rules satisfied, averaged across prompts.

Dataset size

500+ prompts with verifiable instructions like word counts, JSON formatting, language constraints, and section structure.

Mean score

56.6

Median score

55.7

Open / Closed

57 / 76

Top Scorers

#	Model	Lab	Source	Score
01	Grok 4.20 Beta 0309 Reasoning	xAI	Closed	82.9
02	MiniMax M3	MiniMax	Open	82.9
03	Grok 4.3	xAI	Closed	81.3
04	Grok 4.3 beta	xAI	Closed	81.3
05	Qwen 3.7 Max	Alibaba	Closed	80.5
06	MiMo-V2.5-Pro	Xiaomi	Closed	79.9
07	DeepSeek-V4-Flash	DeepSeek	Open	79.2
08	Qwen3.5-397B-A17B	Alibaba	Open	78.8
AdTools for AI BuildersAll the hardware and tools you need to run private AI at your office or at home.Shop Now
09	Gemini 3 Flash (Thinking Minimal)	Google	Closed	78.0
10	Gemini 3.1 Flash Lite Preview	Google	Closed	77.2
11	Gemini 3.1 Pro Preview	Google	Closed	77.1
12	Qwen3.6 Max Preview	Alibaba	Closed	76.6
13	DeepSeek-V4-Pro	DeepSeek	Open	76.5
14	Gemini 3.5 Flash	Google	Closed	76.3
15	GLM-5.1	Z.ai	Open	76.3

Score Distribution

Open vs Closed Source

Gap on IFBench:+0.1pts closed leads

Top Open-Source Models

1MiniMax M382.9
2DeepSeek-V4-Flash79.2
3Qwen3.5-397B-A17B78.8

Top Closed-Source Models

1Grok 4.20 Beta 0309 Reasoning82.9
2Grok 4.381.3
3Grok 4.3 beta

Score vs Parameter Count

Average Score by Lab

MiniMax
76.7n = 3
Xiaomi
71.9n = 3
NVIDIA
67.3n = 2
OpenAI
65.9n = 23

Most Correlated Benchmarks

SWE-Pro
+0.94n = 14
LiveCodeBench
+0.84n = 75
GPQA
+0.78n = 133
HLE
+0.78n = 133
AA LCR

What It Captures Well

Programmatic scoring means no judge model is needed.
Strong predictor of production usefulness for structured output.
Catches models that are clever but disobedient.

Where It Falls Short

Does not measure response quality, only rule adherence.
A model can pass every rule and still give a bad answer.
Some rules are subjective (tone, formality).

Frequently Asked Questions

Why does IFBench matter for production?

Real applications need predictable output: JSON with specific keys, summaries with specific lengths, responses in specific languages. A high IFBench score means the model will hold up when the prompt has hard constraints.