Benchmarks · 2026

Jevals Score: Jevals Score: HelpSteer2 Helpfulness Ratings

Name: Jevals Score: Jevals Score: HelpSteer2 Helpfulness Ratings
Creator: Jevals
Published: 2026
Keywords: Jevals Score, AI benchmark, decision model evaluation, Jevals

Rate how helpful a response is on an ordered scale. The hardest primitive: most models barely beat guessing.

Open Dataset

Models Tested

Top Score

9.2

Published

2026

Source

Jevals

Tools for AI Builders—All the hardware and tools you need to run private AI at your office or at home.Shop Now

How It Works

Score is the graded primitive: the model rates content on an ordered scale and returns a probability for each level. Jevals tests it on human helpfulness ratings from HelpSteer2. It is a proxy for lead scoring, quality review, and ranking.

Each item has a human gold label. Models return a probability for every option, and the Decision Score measures skill over the label prior using the Brier score (or ranked probability score for ordered scales). 100 is perfect, 0 matches guessing the label base rates, and negative means worse than guessing. Each run is repeated five times.

Dataset size

300 human-rated HelpSteer2 responses.

Mean score

9.2

Median score

9.2

Open / Closed

0 / 1

Top Scorers

#	Model	Lab	Source	Score
01	Jev 1.13	TypeSafe AI	Closed	9.2

Score Distribution

Open vs Closed Source

Top Open-Source Models

No models in this category.

Top Closed-Source Models

1Jev 1.139.2

Most Correlated Benchmarks

Not enough scored models yet.

What It Captures Well

Human gold labels, not labels from other LLMs.
Scores probabilities, so a confident wrong answer costs more than an unsure one.
One clean task per primitive makes it easy to see where a model is weak.

Where It Falls Short

One dataset per primitive, so results may not transfer to your domain.
Jev is the only System One model on the board so far; the rest are LLMs.

Frequently Asked Questions

Why are Jevals Score results so low?

Human helpfulness ratings are noisy and hard to predict. Jev 1.13 leads at 9.2, and several LLMs score below zero, which means worse than guessing the base rates.