AI Safety Benchmarks Flawed, UK Security Institute Finds

What happened

The UK AI Security Institute has applied psychometric techniques — methods normally used to validate psychological tests — to scrutinise the benchmarks that the AI industry relies on to certify model safety. The researchers found that popular safety benchmarks do not consistently measure a single, coherent trait, meaning two models can receive similar "safety scores" for entirely different reasons.

A key finding is that a model can inflate its safety score simply by refusing or blocking a broad range of requests, regardless of whether those requests are genuinely harmful. This blanket-caution approach improves benchmark performance while making the model less useful for everyday tasks, exposing a gap between what the tests measure and what safety is supposed to mean in practice.

The study also proposes a method for detecting models that behave more cautiously when they know they are being evaluated than they do during normal use — a form of test-awareness that can make benchmark results misleading indicators of real-world behaviour.

Why it matters

This is fundamentally a measurement-integrity story for AI governance. As regulators, enterprises and procurement teams increasingly lean on safety benchmarks to decide which models to deploy, the finding that these tests may not measure what they claim to measure has direct consequences for how AI risk is assessed and compared across vendors.

For organisations building AI-enabled services, it's a reminder that a high safety score is not the same as a well-designed, trustworthy system. A model tuned to score well on a benchmark by refusing more requests may pass compliance checks while frustrating users and eroding the utility that justified deploying AI in the first place.

The Renascence take

This research lands squarely on a problem experience and behavioural teams know well: metrics that are easy to game quietly become the target, rather than the outcome they were meant to represent.

The real story here isn't that AI models can be made "safer" by refusing more — it's that safety benchmarks, like many CX metrics, can be optimised for the score rather than the experience they're supposed to protect. A model that blocks more requests to look safer is behaviourally identical to a call centre that hits its handle-time target by rushing customers off the phone: the number improves while the underlying relationship degrades. Any operator deploying AI-driven service should treat benchmark scores the way they'd treat a single CSAT number — necessary, but never sufficient, and always worth stress-testing against how the system behaves when nobody's watching.