Terminal Bench: Terminal Bench

How It Works

Terminal Bench measures how well a model can drive a real shell. The agent is given a goal — install a package, debug a failing build, set up a service — and a sandboxed Linux environment. Success requires picking the right commands, parsing real output, recovering from errors, and finishing in a bounded number of steps. It is the most operational benchmark in this set, closer to "can this model run my devops" than any other.

Each task has a verifier — usually a script that checks the final filesystem and process state. The model agent runs commands, reads stdout, and iterates until it believes the task is complete. Scoring is task pass-rate. Token and step budgets vary by task; most leaderboards report results inside a fixed agent harness.