Subject2Vec: Generative-Discriminative Approach from a Set of Image Patches to a Vector
Singla S, Gong M, Ravanbakhsh S, Sciurba F, Poszos B, Batmanghelich KN. Subject2Vec: Generative-Discriminative Approach from a Set of Image Patches to a Vector. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). 2018: 502-510. doi: 10.1007/978-3-030-00928-1_57. PubMed PMID: 30895278. PMCID: PMC6422035.
We propose an attention-based method that aggregates local image features to a subject-level representation for predicting disease severity. In contrast to classical deep learning that requires a ﬁxed dimensional input, our method operates on a set of image patches; hence it can accommodate variable length input image without image resizing. The model learns a clinically interpretable subject-level representation that is reﬂective of the disease severity. Our model consists of three mutually dependent modules which regulate each other: (1) a discriminative network that learns a ﬁxed-length representation from local features and maps them to disease severity; (2) an attention mechanism that provides interpretability by focusing on the areas of the anatomy that contribute the most to the prediction task; and (3) a generative network that encourages the diversity of the local latent features. The generative term ensures that the attention weights are non-degenerate while maintaining the relevance of the local regions to the disease severity. We train our model end-to-end in the context of a large-scale lung CT study of Chronic Obstructive Pulmonary Disease (COPD). Our model gives state-of-the art performance in predicting clinical measures of severity for COPD. The distribution of the attention provides the regional relevance of lung tissue to the clinical measurements.