Histologic diagnosis of Barrett's esophagus and esophageal malignancy via probe-based confocal laser endomicroscopy (pCLE) allows for real-time examination of epithelial architecture and targeted biopsy sampling. Although pCLE demonstrates high specificity, sensitivity remains low. This study employs deep learning architectures in order to improve the accuracy of pCLE in diagnosing esophageal cancer and its precursors. pCLE videos are curated and annotated as belonging to one of the three classes: squamous, Bar-rett`s (intestinal metaplasia without dysplasia), or dysplasia. We introduce two novel video architectures, AttentionPooling and Multi-Module AttentionPooling deep networks, that outperform other models and demonstrate a high degree of explainability.
IEEE ISBI / April 2020 / Coauthors: Pulido, J.V., Guleriai, S., Ehsan, L., Shah, T., Syed, S. and Brown, D.E.
Deep convolutional neural networks(CNNs) have been successful for a wide range of computer vision tasks, including image classification. A specific area of the application lies in digital pathology for pattern recognition in the tissue-based diagnosis of gastrointestinal(GI) diseases. This domain can utilize CNNs to translate histopathological images into precise diagnostics. This is challenging since these complex biopsies are heterogeneous and require multiple levels of assessment. This is mainly due to structural similarities in different parts of the GI tract and shared features among different gut diseases. Addressing this problem with a flat model that assumes all classes (parts of the gut and their diseases) are equally difficult to distinguish leads to an inadequate assessment of each class. Since the hierarchical model restricts classification error to each sub-class, it leads to a more informative model than a flat model. In this paper, we propose to apply the hierarchical classification of biopsy images from different parts of the GI tract and the receptive diseases within each. We embedded a class hierarchy into the plain VGGNet to take advantage of its layers' hierarchical structure. The proposed model was evaluated using an independent set of image patches from 373 whole slide images. The results indicate that the hierarchical model can achieve better results than the flat model for multi-category diagnosis of GI disorders using histopathological images.
IEEE ICHI / August 2020 / Coauthors: Sali, R., Adewole, S., Ehsan, L., Denson, L.A., Kelly, P., Amadi, B.C., Holtz, L., Ali, S.A., Moore, S.R., Syed, S. and Brown, D.E.
One of the greatest obstacles in the adoption of deep neural networks for new medical applications is that training these models typically require a large amount of manually labeled training samples. In this body of work, we investigate the semi-supervised scenario where one has access to large amounts of unlabeled data and only a few labeled samples. We study the performance of MixMatch and FixMatch-two popular semi-supervised learning methods-on a histology dataset. More specifically, we study these models' impact under a highly noisy and imbalanced setting. The findings here motivate the development of semi-supervised methods to ameliorate problems commonly encountered in medical data applications.
IEEE BIBE 2020 / October 2020 / Coauthors: Pulido, J.V., Guleria, S., Ehsan, L., Fasullo, M., Lippman, R., Mutha, P., Shah, T., Syed, S.* and Brown, D.E.
Image classification is central to the big data revolution in medicine. Improved information processing methods for diagnosis and classification of digital medical images have shown to be successful via deep learning approaches. As this field is explored, there are limitations to the performance of traditional supervised classifiers. This paper outlines an approach that is different from the current medical image classification tasks that view the issue as multi-class classification. We performed a hierarchical classification using our Hierarchical Medical Image classification (HMIC) approach. HMIC uses stacks of deep learning models to give particular comprehension at each level of the clinical picture hierarchy. For testing our performance, we use biopsy of the small bowel images that contain three categories in the parent level (Celiac Disease, Environmental Enteropathy, and histologically normal controls).
MDPI Information 2020 / 2020 / Coauthors: Kowsari, K., Sali, R., Ehsan, L., Adorno, W., Ali, A., Moore, S., Amadi, B., Kelly, P., Syed, S.* and Brown, D.E.
Hematoxylin and Eosin (H&E) stained Whole Slide Images (WSIs) are utilized for biopsy visualization-based diagnostic and prognostic assessment of diseases. Variation in the H&E staining process across different lab sites can lead to significant variations in biopsy image appearance. These variations introduce an undesirable bias when the slides are examined by pathologists or used for training deep learning models. To reduce this bias, slides need to be translated to a common domain of stain appearance before analysis. We propose a Self-Attentive Adversarial Stain Normalization (SAASN) approach for the normalization of multiple stain appearances to a common domain. This unsupervised generative adversarial approach includes self-attention mechanism for synthesizing images with finer detail while preserving the structural consistency of the biopsy features during translation. SAASN demonstrates consistent and superior performance compared to other popular stain normalization techniques on H&E stained duodenal biopsy image data.
AIDP ICPR 2021 / January 2021 / Coauthors: Shrivastava, A., Adorno, W., Sharma, Y., Ehsan, L., Ali, S.A., Moore, S.R., Amadi, B., Kelly, P., Syed, S.* and Brown, D.E.
Probe-based confocal laser endomicroscopy (pCLE) allows for real-time diagnosis of dysplasia and cancer in Barrett’s esophagus (BE) but is limited by low sensitivity. We deployed deep-learning-based image and video analysis in order to improve diagnostic accuracy of pCLE videos and biopsy images. Biopsy classification was conducted using two distinct approaches—a patch-level model and a whole-slide-image-level model. Gradient-weighted class activation maps (Grad-CAMs) were extracted from pCLE and biopsy models in order to determine tissue structures deemed relevant by the models. Our deep learning models achieved high diagnostic accuracy for both pCLE-based and histopathologic diagnosis of esophageal dysplasia and its precursors, similar to human accuracy in prior studies. These machine learning approaches may improve accuracy and efficiency of current screening protocols.
Nature Scientific Reports 2021 / 2021 / Coauthors: Guleria, S., Shah, T.U., Pulido, J.V., Fasullo, M., Ehsan, L., Lippman, R., Sali, R., Mutha, P., Cheng, L., Brown, D.E. and Syed, S.
We have demonstrated that dividing a WSI into clusters can improve the model training by exposing it to diverse discriminative features extracted from
the patches. We regularized the clustering mechanism by introducing a KL-divergence loss between the attention weights of patches in a cluster and the uniform distribution. The framework is optimized end-to-end on slide-level cross-entropy, patch-level cross-entropy, and KL-divergence loss.
MIDL 2021 / 2021 / Coauthors: Sharma, Y., Shrivastava, A., Ehsan, L., Moskaluk, C.A., Syed, S.* and Brown, D.E.
Eosinophilic Esophagitis (EoE) is an inflammatory esophageal disease which is increasing in prevalence. The diagnostic gold-standard involves manual review of a patient's biopsy tissue sample by a clinical pathologist for the presence of 15 or greater eosinophils within a single high-power field (400× magnification). Diagnosing EoE can be a cumbersome process with added difficulty for assessing the severity and progression of disease. We propose an automated approach for quantifying eosinophils using deep image segmentation. A U-Net model and post-processing system are applied to generate eosinophil-based statistics that can diagnose EoE as well as describe disease severity and progression. The goal is to find linkages that could potentially guide treatment plans for new patients at their initial disease diagnosis. A deep image classification model is further applied to discover features other than eosinophils that can be used to diagnose EoE. This is the first study to utilize a deep learning computer vision approach for EoE diagnosis and to provide an automated process for tracking disease severity and progression.
Bioimaging 2021 Best Paper Award / 2021 / Coauthors: Adorno III, W., Catalano, A., Ehsan, L., von Eckstaedt, H.V., Barnes, B., McGowan, E., Syed, S. and Brown, D.E.
Advancement in digital pathology and artificial intelligence has enabled deep learning-based computer vision techniques for automated disease diagnosis and prognosis. However, WSIs present unique computational and algorithmic challenges. WSIs are gigapixel-sized, making them infeasible to be used directly for training deep neural networks. These approaches require detailed pixel-level annotations for training the patch encoder. However, obtaining these annotations is time-consuming and tedious for medical experts. Transfer learning is used to address this gap and deep learning architectures pre-trained on ImageNet are used for generating patch-level representation. In this work, we compare the performance of features extracted from networks trained on ImageNet and histopathology data. We use an attention pooling network over these extracted features for slide-level aggregation. We investigate if features learned using more complex networks lead to gain in performance. We use a simple top-k sampling approach for fine-tuning framework and study the representation similarity between frozen and fine-tuned networks using Centered Kernel Alignment.
IEEE-BHI 2021 / 2021 / Coauthors: Sharma, Y., Ehsan, L., Syed, S. and Brown, D.E.
In this diagnostic study, a deep learning convolutional neural network was trained on 3118 images from duodenal biopsies of patients with environmental enteropathy, celiac disease, and no disease. The convolutional neural network achieved 93.4% case-detection accuracy, with a false-negative rate of 2.4%, and automatically learned microlevel features in duodenal tissue, such as alterations in secretory cell populations.
JAMA 2019 / 2019 / Coauthors: Syed, S., Al-Boni, M., Khan, M.N., Sadiq, K., Iqbal, N.T., Moskaluk, C.A., Kelly, P., Amadi, B., Ali, S.A., Moore, S.R. and Brown, D.E.
Artificial intelligence (AI), a discipline encompassed by data science, has seen recent rapid growth in its application to healthcare and beyond, and is now an integral part of daily life. Uses of AI in gastroenterology include the automated detection of disease and differentiation of pathology subtypes and disease severity. Although a majority of AI research in gastroenterology focuses on adult applications, there are a number of pediatric pathologies that could benefit from more research. As new and improved diagnostic tools become available and more information is retrieved from them, AI could provide physicians a method to distill enormous amounts of data into enhanced decision-making and cost saving for children with digestive disorders. This review provides a broad overview of AI and examples of its possible applications in pediatric gastroenterology.
JPGN 2020 / 2020 / Coauthors: Patel, V., Khan, M.N., Shrivastava, A., Sadiq, K., Ali, S.A., Moore, S.R., Brown, D.E. and Syed, S.
Our AI-based image analysis platform demonstrated high classification accuracy for small bowel enteropathies which was capable of identifying biologically relevant microscopic features and emulating human pathologist decision-making process. Grad-CAMs illuminated the otherwise “black box” of deep learning in medicine, allowing for increased physician confidence in adopting these new technologies in clinical practice.
JPGN 2021 / 2021 / Coauthors: Syed, S., Ehsan, L., Shrivastava, A., Sengupta, S., Khan, M., Kowsari, K., Guleria, S., Sali, R., Kant, K., Kang, S.J. and Sadiq, K.
Get the latest news
Subscribe to receive updates from the School of Data Science.