Offline Compact AI for Mobile Medical Reference
BRITE Institute developed a lightweight AI application for Android that lets users ask questions and consult locally stored medical references without an internet connection. We investigated the design and design limitations of offline LLMs on mobile devices.
What is this research about?
This project explores how small language models can support access to medical information in settings where internet access and professional assistance may be limited.
We developed an Android prototype that combines a compact language model with retrieval-augmented generation (RAG). This approach searches medical references stored on the phone and supplies relevant passages to the model to inform its answers. Users can examine supporting passages and source sections, helping them trace responses back to the underlying material.
The research investigates the practical challenges of making this system work within a phone’s limited memory and processing capacity. It examines document preparation, information retrieval, model compatibility, and methods for testing the complete application.
Why is this research important?
A phone may still work when the network does not. Disasters can disable communications infrastructure, while remote environments may have little connectivity to begin with. In these circumstances, cloud-based AI tools and online medical resources may be unavailable precisely when information is most needed.
An offline reference tool could help emergency responders, field medics, and people awaiting assistance consult medical guidance wherever they are. Achieving that goal requires more than fitting an AI model onto a phone: the system must find relevant evidence, preserve its meaning, and make its sources visible.
What are the conclusions?
The prototype produced nine engineering lessons for developing and evaluating offline mobile AI. A central finding was that weak answers can originate upstream of the language model. Poorly extracted documents, oversized passages, and missing source information can undermine the evidence supplied to the model; increasing model size does not resolve those problems.
More complex search methods also did not consistently improve results. Each component must demonstrate its value against the application’s accuracy, memory, and response-time requirements.
The project establishes a foundation for further development, while identifying the testing needed before wider use. Clinical reliability remains unproven. Next steps include clinician-reviewed evaluation and systematic testing on representative phones.