Medical LLMs ability to Handle Realistic Patient Records
Medical LLMs are often trained and evaluated using clean, well-organized patient records. Unfortunately, real patient records can have significant missing data - exceeding 30% missingness in some cases. What problems does this create for patients safety?
What is this research about?
Medical AI systems are often evaluated using clean, well-organized datasets. Real patient records rarely look like this.
In real patient records, information is messy. It is often distributed across progress notes, laboratory reports, medication lists, discharge summaries, scanned documents, and messages from different providers. Important facts are oftenbe repeated, outdated, incorrectly entered, or missing entirely. A current allergy may appear only in an old note. A medication list may contain a drug the patient stopped taking months earlier.
These imperfections create a major safety challenge for AI systems that summarize patient records or answer clinical questions about them.
The project will examine how model performance changes when records contain:
- Missing clinical information
- Incorrect or outdated information
- Contradictory entries
- Important facts buried in long narrative notes
- Duplicate or fragmented documentation
- Irrelevant information that may distract the model
- Differences between structured and unstructured data
- Clinically important information located far apart within the record
The goal is to determine how safely medical LLMs handle realistic patient data and if ongoing benchmarking can be done via adversarial agents.
Why is this research important?
Commercial medical AI tools are increasingly being used to summarize charts, answer questions about patient records, support clinical documentation, and assist with medical decision-making.
An AI system may produce a confident and professionally written response while comitting critical errors that threaten patient safety. AI systems have been shown to repeatedly fail to notice that a medication is contraindicated, treat an outdated diagnosis as current, or invent missing information rather than acknowledging issues in the patient record. Because the output sounds credible, clinicians may not immediately recognize the error.
This project focuses on patient protection by creating an adversarial AI agent which can test these systems for safety and accuracy under realistic conditions.
The adverarial agent answers questions such as:
- Does the system recognize when essential information is missing?
- Does it ask for clarification or proceed using unsupported assumptions?
- Can it distinguish current information from outdated information?
- Can it identify contradictions within a long patient record?
- Does it preserve critical details when summarizing complex documentation?
- Does it fabricate facts that do not appear in the record?
- Does it appropriately communicate uncertainty?
- Does it prioritize clinically urgent information?
- Does performance decline as the record becomes longer or less organized?
- Are errors consistent across repeated tests or different AI products?
A structured adversarial evaluation can help identify weaknesses before development, procurement, or deployment. It can also help clinicians test systems they plan on using.
What are the conclusions?
This project is in the final stages of data analysis and writing. A preprint will be availible in the next few weeks.