Human-in-the-Loop White Paper
Human oversight is the safeguard of last resort for AI, and from late 2026 became a legal requirement for high-risk systems under the EU AI Act. Our research examines whether that safeguard actually works.
What is this research about?
As AI systems take on consequential decisions in healthcare, finance, employment, and law enforcement, regulators and organizations are relying on a “human in the loop” to catch the errors that every other safeguard has missed. BRITE Institute’s Human in the Loop project draws on decades of human factors research to examine what that reviewer actually needs in order to succeed. The project looks at automation bias, the limits of explainable AI, trust calibration, algorithm aversion, and the organizational pressures that shape whether a reviewer who spots an error will report it. It concludes that effective oversight depends on measurable conditions such as verifiability, workload, authority, and psychological safety, and it sets out a program to measure whether those conditions are being met.
Why is this research important?
Article 14 of the EU AI Act makes human oversight mandatory for high-risk AI systems, yet its main protection against reviewer failure is simply making reviewers aware of automation bias. Awareness is not a control. Eighty years of research, going back to studies of radar operators in the 1940s, shows that people cannot reliably sustain vigilance for rare errors hidden among a high volume of correct results, particularly under time pressure and when the system presents every output with the same confidence. When oversight is specified but not properly resourced, it exists on the process map but not in practice, and responsibility for outcomes is assigned to people who lack the time, information, or authority to influence them. At present, no regulator, vendor, auditor, or deploying institution can answer the most basic question in AI safety: when a system gets something wrong, how often does the human reviewer catch it?
What are the conclusions?
Our white paper, Human in the Loop, brings together evidence from systematic reviews, meta-analyses, and field and experimental studies to set out six key findings. Automation bias affects experts as well as novices. Verification is the deciding factor in whether oversight works. Explanations do not reliably improve review and can reduce engagement under pressure. Trust in a system persists even after the system’s reliability changes. Under-reliance on sound AI advice is as real a failure as over-reliance. And detecting an error and being willing to report it are separate capabilities with separate causes. From this evidence, the paper identifies seven conditions for effective oversight: verifiability, calibrated trust, authority, engagement by design, workload headroom, psychological safety, and failure-mode-specific training. Each of these can be measured, yet none currently is. To close this gap, BRITE Institute is building an AI Incident Reporting System coded for oversight performance.