US State Laws Mandate Human Oversight in AI Healthcare Decisions

What happened

A wave of US state-level legislation is placing firm human-oversight requirements on artificial intelligence used in healthcare settings, with several of the most significant provisions scheduled to take effect in 2027. The laws target two particularly sensitive applications: automated decision-making in insurance claims and AI-delivered mental health therapy.

Under the emerging regulatory framework, insurers and healthcare platforms that rely on algorithmic systems to approve, deny or modify coverage decisions will be required to ensure a qualified human reviewer remains meaningfully in the loop. Similarly, AI tools that simulate therapeutic conversations — a fast-growing category of consumer mental health apps — face new restrictions on how autonomously they can operate when a user's wellbeing is at stake.

Why it matters

For anyone working in customer experience, this legislation is a concrete expression of what behavioral economists call automation bias — the tendency for people to over-trust algorithmic outputs, and the corresponding loss of perceived fairness when a consequential decision arrives without a human face attached to it. Healthcare is the highest-stakes service environment there is; when a claim is denied or a vulnerable person receives mental health guidance, the absence of human accountability doesn't just feel wrong to the recipient — research consistently shows it undermines trust in the entire institution. Legislators appear to be codifying what service designers have long argued: that automation should augment human judgement, not replace it at the moments that matter most.

For service and experience leaders beyond healthcare, the direction of travel is instructive. Regulators are beginning to treat "human in the loop" not as a design preference but as a compliance baseline. Organisations that have already embedded human review into high-stakes automated journeys will find themselves ahead of a curve that is likely to extend into financial services, social care and other sensitive sectors.

By the numbers

• 2027 is the effective date for several of the most consequential provisions across the new state laws.

The Renascence take

The instinct in many organisations will be to treat these laws as a compliance exercise — add a human sign-off step, document it, move on. That framing misses the deeper opportunity. What regulators are responding to is a genuine erosion of felt fairness in automated service journeys, and that erosion was happening long before any law was passed.

The real signal here is not legal risk — it is a customer trust deficit that automation alone created. Organisations that treat "human in the loop" as a checkbox will satisfy the letter of the law while continuing to deliver experiences that feel cold and opaque to the people on the receiving end. The more interesting question is: which touchpoints in your automated journeys carry the highest emotional weight, and have you deliberately designed human presence — not just human availability — into those moments? A 2027 deadline is a useful forcing function, but customer-obsessed operators should be asking this question today, not in three years.