- Services
A frozen, repeatable protocol that runs decision models against constrained LLMs and simple baselines.
The headline suite is 77-way intent classification on Banking77. The other suites probe failure modes: how many options a model accepts, whether it still works when label names are shuffled, and how it behaves when forced to be uncertain.
The protocol is frozen so reruns are comparable. Models are scored on accuracy, calibration (ECE), latency, and cost, next to keyword, majority, and random baselines.
| # | Model | Lab | Source | Score |
|---|---|---|---|---|
| 01 | Jev 1.13 | TypeSafe AI | Closed | 76.3% |
No models in this category.
Not enough scored models yet.
In this benchmark Jev accepted up to 255 options and rejected 256 and above. LLMs have no fixed cap but get slower and less accurate as the list grows.
Based on score correlations across our database.
