Building foundation models for understanding human pathology
When Dr. Faisal Mahmood set up his laboratory at the intersection of pathology—the scientific study of disease—and machine learning six years ago, he saw an opportunity to explore what was then a relatively uncharted area. Today, the Mahmood Lab at Mass General Brigham and the Harvard Medical School is a leader in the field, powered by a team of 30 researchers from a variety of educational backgrounds at Harvard and MIT.
The focus on digital and computational pathology
“Our particular focus has been on digital and computational pathology because it’s a sort of modality that has recently started to be digitized,” Mahmood says. “There’s large amounts of data that is collected. It’s a challenging and interesting problem because the images are very large and we don’t typically know what we’re going to find.”
Open source models driving innovation
The team credits open source models, including Meta’s DINO, with helping them do their work. Earlier this year, the team shared two open source foundation models for understanding pathology that outperform previous models on state-of-the-art tasks.
Utilizing diverse datasets for model training
Using diverse datasets, the team was able to train a generic feature extraction model that could then enable more than 30 clinical and diagnostic tasks, including disease detection and diagnosis, organ transplant assessment, and rare disease analysis. The model performed well across each use case.
PathChat: A groundbreaking tool
Building on the work of DINO, the team created a chatbot called PathChat, which enables open-ended question answering from pathology images and can act as a co-pilot for pathologists. PathChat is capable of disease diagnosis, triage, and generating a pathology report, improving patient outcomes and saving valuable time.
Continued collaboration and innovation
As the research continues, Dr. Mahmood emphasizes the importance of building on each others’ work in the open source community, sharing insights and breakthroughs to drive health advancements. The ongoing collaboration in the open source community is vital for delivering better care and outcomes for patients.
Subscribe to our newsletter to keep up with Meta AI news, events, research breakthroughs, and more.
