Lighter, multilingual variant of MMLU covering 14 languages and the original subject mix.
Global MMLU Lite tests broad academic knowledge across languages, not just English. Questions are translated from the original MMLU set into 14 languages, with cultural adjustments where literal translations would not work. It is the cleanest signal for how well a model holds up outside English.
Multiple-choice questions are presented in each target language. Models are evaluated zero-shot, and the score is percent correct, averaged across languages or reported per-language.
No scores yet for this benchmark.
Not enough scored models yet.
Not enough scored models yet.
Based on score correlations across our database.
