AISI publishes evaluation results with their experimental configurations

A benchmark score measures performance under a particular protocol. The benchmark's name does not describe all those conditions. In a September 22, 2026 announcement, EvalEval and the UK AI Security Institute describe sharing evaluation findings with context and configuration information through Evaluation Cards.

The release accompanies research into how inference compute and evaluation procedures shape results. Its Humanity's Last Exam example describes models solving additional tasks as token use increased when they received correctness feedback after attempts. That experimental condition matters: the model was being told whether its answers were correct.

Every Eval Ever provides a common reporting structure for evaluation information. Such records make differences between runs easier to investigate, but a shared schema does not make different experimental setups equivalent.

For model selection, compare tool access, execution allowance and feedback conditions before using a score difference to guide a purchase or deployment. The scientific uncertainty is specific: how much of the observed advantage would remain without the correctness feedback supplied during the experiment?

Source: the collaboration announcement, by Avijit Ghosh and coauthors · EvalEval · UK AI Security Institute.