UCSF Study Finds Multiview AI Improves Heart Scan Diagnostic Accuracy

Cardiac ultrasounds generate a constellation of imaging angles that cardiologists routinely cross-reference to diagnose heart disease. Until recently, AI tools could only process one angle at a time. A new deep neural network developed at UC San Francisco changes that, and a study published March 17 in Nature Cardiovascular Research shows the approach measurably improves diagnostic accuracy across several cardiac conditions.

The research, led by first author Joshua Barrios, PhD, an assistant professor in the UCSF Division of Cardiology, and senior author Geoffrey Tison, MD, MPH, a cardiologist and co-director of the UCSF Center for Biosignal Research, introduced a "multiview" deep neural network architecture that ingests multiple echocardiogram video views simultaneously. Rather than extracting a diagnosis from a single two-dimensional slice, the model learns patterns both within each individual view and across views together.

"Until now, AI has primarily been used to analyze one 2D view at a time — from either images or videos — which limits an AI algorithm's ability to learn disease-relevant information between views," Tison said.

The team trained demonstration models on three primary diagnostic tasks: detecting any left or right ventricular abnormality, detecting diastolic dysfunction, and detecting substantial valvular regurgitation. For each task, the multiview model drew from three predefined echocardiogram views selected as most clinically suitable for that condition, all taken from the same patient study. The team also trained models for more anatomically specific endpoints, including left ventricular size, left ventricular ejection fraction, right ventricular function and size, and individual valve regurgitations affecting the mitral, aortic, and tricuspid valves.

Training and internal validation data came from adult transthoracic echocardiograms acquired at UCSF between 2012 and 2020. That dataset was linked to structured diagnoses and measurements adjudicated by level 3 echocardiographers at the UCSF echo lab. The study also incorporated echocardiogram data from the Montreal Heart Institute for cross-site comparison, with collaborators from the Montreal Heart Institute's Division of Cardiology, École Polytechnique de Montréal, and the University of Montreal contributing to the research. Pixel data were extracted from DICOM files, with the echo imaging cone identified by masking pixels that showed intensity changes over time; erosion and dilation operations then stripped out smaller moving artifacts such as electrocardiogram waveforms.

Across the diagnostic tasks, the multiview models improved discrimination compared with single-view models trained on data from the same echo studies.

"Our multi-view neural network architecture is explicitly designed to enable the model to learn complex relationships between information in multiple imaging views," Barrios said. "We find that this approach improves performance for diagnostic tasks in echocardiography, but this new AI architecture can also be applied to other medical imaging modalities where multiple views contain complimentary information."

The rationale for focusing on echocardiography first is straightforward: cardiac ultrasound diagnoses routinely require synthesizing information across multiple views because no single angle captures the full picture of heart function. That same logic, the researchers argue, applies broadly to medical imaging fields where anatomy is three-dimensional but imaging data is collected in two-dimensional planes. The UCSF team's work, rooted in nearly a decade of patient echo data from Parnassus Heights and validated across an international dataset, positions the approach as a potential template for AI development well beyond cardiology.