Advancing Health Research through Multimodal AI

Background

Multimodal AI has the potential to capture the complexity of biomedical and behavioral systems and improve clinical decision‑making, but realizing this promise requires new innovations in data fusion, model training, evaluation, and application. Ethical considerations—including privacy, fairness, accountability, and transparency—must be integrated throughout the entire lifecycle, from data selection and preparation to model deployment, with careful attention to stakeholder needs. Collaboration among researchers, patients, policymakers, the scientific community, and end users is essential to co‑create multimodal AI systems that reflect shared values and real‑world requirements. As AI advances faster than traditional funding and development timelines, more flexible and agile approaches to AI research and implementation are increasingly necessary.

Purpose

The purpose of this program is to develop ethically focused and data-driven multimodal AI approachesto more closely model, interpret, and predict complex biological, behavioral, and health systems and enhance our understanding of health and the ability to detect and treat human diseases.

Program Goals

  • Creation of ethics- and data-driven multimodal AI models for use in biomedical, behavioral, and/or clinical fields
  • Build portfolio of innovative projects that address systems level biomedical challenges using a co-design approach to multimodal AI that integrate the work of various stakeholder groups as appropriate
  • Inform considerations for the appropriate use of multimodal AI and take significant steps towards incorporation of ethical and co-design approach in multimodal AI lifecycle
  • Use translational or end use applications that will be identified and used as test cases for testing and evaluation

Expected Program Outputs

Mulitmodal AI Awards

U.S. map of Multimodal AI Awards, showing states shaded in a blue gradient based on the number of awardees and sub‑awardees, from light blue (1) to dark blue (8). States with higher counts include California, Texas, New York, Pennsylvania, and Georgia. Red location markers indicate specific awardee locations within those states. A legend at the bottom displays the numeric scale from 1 to 8.

Lead InstitutionSub-awardees
Brigham and Women's HospitalMassachusetts General Hospital, University of Washington
University of FloridaIndiana University, University at Buffalo
University of MichiganUniversity of California Los Angeles, Cornell Medicine, Vanderbilt University, University of South Florida
University of Wisconsin-MadisonMarshfield Clinic Research Institute, The Medical College of Wisconsin, The University of Chicago
Northwestern University at ChicagoCleveland Clinic, Cornell Medicine
University of Pennsylvania
University of Colorado DenverCleveland Clinic
University of California BerkeleyUCSF , Stanford University, University of Virginia
Baylor College of MedicineCredence Management Solutions, UT Health
Stanford University
Mayo Clinic ArizonaMayo Clinic (Minnesota - Rochester), Arizona State University
University of Texas Health Science CenterUT-Houston, Rice University, UNC-CH, UNC-Charlotte, NC State
University of North CarolinaUC San Diego, Moffitt Cancer Center, Wake Forest, Medical University of South Carolina, UT Houston
University of PittsburghDuke, UT MD Anderson Cancer Center
Stanford University
University of California, San Diego
Emory UniversityStanford University, Mayo Clinic Arizona
Children's Hospital of PhiladelphiaColumbia University, Boston Children’s Hospital

Research Focus

Contact Information

Please direct questions about Advancing Health Research through Multimodal AI to ODMultimodalAI@od.nih.gov

FAQs

Technical Scope

Application