Adversarial AI Agent using Realistic Patient Records
BRITE Institute is developing an adversarial AI agent to evaluate how safely commercial medical-support large language models respond to realistic patient records. The system will test whether these tools can identify important clinical information, manage uncertainty, recognize incomplete or contradictory data, and avoid unsafe recommendations when patient records are fragmented, unstructured, or imperfect.
What is this project about?
Medical AI systems are often evaluated using clean, well-organized datasets. Real patient records rarely look like this.
Critical patient information may be distributed across progress notes, laboratory reports, medication lists, discharge summaries, scanned documents, and messages from different providers. Important facts are oftenbe repeated, outdated, incorrectly entered, or missing entirely. A current allergy may appear only in an old note. A medication list may contain a drug the patient stopped taking months earlier.
These imperfections create a major safety challenge for AI systems that summarize patient records or answer clinical questions about them.
BRITE Institute is developing an adversarial testing agent that will systematically evaluate commercial medical-support LLMs using a controlled library of realistic, fictional patient records. These test records will be created and reviewed by medical experts and will represent the complexity and ambiguity of real clinical documentation.
The project will examine how model performance changes when records contain:
- Missing clinical information
- Incorrect or outdated information
- Contradictory entries
- Important facts buried in long narrative notes
- Duplicate or fragmented documentation
- Irrelevant information that may distract the model
- Differences between structured and unstructured data
- Clinically important information located far apart within the record
The goal is to determine not only whether an AI system produces a generally plausible answer, but whether it can respond safely when the information available to it is incomplete, inconsistent, or difficult to interpret.
Why is this important?
Commercial medical AI tools are increasingly being used to summarize charts, answer questions about patient records, support clinical documentation, and assist with medical decision-making.
An AI system may produce a confident and professionally written response while comitting critical errors that threaten patient safety. AI systems have been shown to repeatedly fail to notice that a medication is contraindicated, treat an outdated diagnosis as current, or invent missing information rather than acknowledging issues in the patient record. Because the output sounds credible, clinicians may not immediately recognize the error.
This project focuses on patient protection by creating an adversarial AI agent which can test these systems for safety and accuracy under realistic conditions.
The adverarial agent answers questions such as:
- Does the system recognize when essential information is missing?
- Does it ask for clarification or proceed using unsupported assumptions?
- Can it distinguish current information from outdated information?
- Can it identify contradictions within a long patient record?
- Does it preserve critical details when summarizing complex documentation?
- Does it fabricate facts that do not appear in the record?
- Does it appropriately communicate uncertainty?
- Does it prioritize clinically urgent information?
- Does performance decline as the record becomes longer or less organized?
- Are errors consistent across repeated tests or different AI products?
A structured adversarial evaluation can help identify weaknesses before development, procurement, or deployment. It can also help clinicians test systems they plan on using.
Where can it be applied?
1. Evaluation of Commercial Medical AI Products
The adversarial agent can test commercially available tools that summarize medical records, answer clinician questions, generate clinical documentation, or provide decision support. Standardized cases will make it possible to compare products under the same conditions.
2. Healthcare Procurement and Vendor Evaluation
Hospitals and healthcare systems can use the testing framework to evaluate whether an AI product performs safely on the types of records found within their own clinical environment. Results can support vendor selection, contracting, risk assessment, and implementation planning.
3. AI Development and Product Improvement
Developers can use adversarial test results to identify weaknesses in retrieval, summarization, reasoning, uncertainty communication, interface design, and model safeguards. The test library can also support regression testing after product updates.
What are this results?
This project is currently in development.
The long-term goal is to create a repeatable and scalable method for determining whether medical AI systems remain safe when confronted with the complexity of real clinical records—not merely idealized data prepared for a demonstration.
Research You Can Rely On
Did you know that many research findings are manipulated—or even outright false? Some estimates suggest that up to 90% of published research may be unreliable. Meanwhile, more than $167 billion in taxpayer money is spent annually on research and development.
At BRITE Institute, we believe research should do more than just look credible. It should be credible. That’s why we go above and beyond typical standards with rigorous practices that ensure honesty, transparency, and accuracy at every step. Below are just some of the ways we safeguard the integrity of our work:
Honest Statistics
BRITE Institute never p-hacks or manipulates data to achieve a desired outcome. If a paper relies on complex statistical analyses, we use an external statistician to ensure objectivity and validity.
Open and Transparent
BRITE Institute prioritizes transparency at every stage of the research process. Whenever possible, we publish our full data sets and use open access publishing.
Impact Centered
BRITE Institute does not publish for the sake of publishing. Our research is built with end-users in mind—whether it’s policy-makers, engineers, or community leaders—ensuring that findings are not only trustworthy but also actionable.