An Explainable Deep Learning Framework for Automated Classification of Ocular Diseases in a Big Data Environment

1 - Classification of ocular diseases using transfer learning approaches and glaucoma severity grading

Age-related macular degeneration (AMD) is the leading cause of irreversible blindness in developed countries. Identifying patients at high risk of progression to late AMD, the sight-threatening stage, is critical for clinical actions, including medical interventions and timely monitoring. Recently, deep-learning-based models have been developed and achieved superior performance for late AMD prediction. However, most existing methods are limited to the color fundus photography (CFP) from the last ophthalmic visit and do not include the longitudinal CFP history and AMD progression during the previous years' visits. Patients in different AMD subphenotypes might have various speeds of progression in different stages of AMD disease. Capturing the progression information during the previous years' visits might be useful for the prediction of AMD progression. In this work, we propose a Contrastive-Attention-based Time-aware Long Short-Term Memory network (CAT-LSTM) to predict AMD progression. First, we adopt a convolutional neural network (CNN) model with a contrastive attention module (CA) to extract abnormal features from CFPs. Then we utilize a time-aware LSTM (T-LSTM) to model the patients' history and consider the AMD progression information. The combination of disease progression, genotype information, demographics, and CFP features are sent to T-LSTM. Moreover, we leverage an auto-encoder to represent temporal CFP sequences as fixed-size vectors and adopt k-means to cluster them into subphenotypes. We evaluate the proposed model based on real-world datasets, and the results show that the proposed model could achieve 0.925 on area under the receiver operating characteristic (AUROC) for 5-year late-AMD prediction and outperforms the state-of-the-art methods by more than 3%, which demonstrates the effectiveness of the proposed CAT-LSTM. After analyzing patient representation learned by an auto-encoder, we identify 3 novel subphenotypes of AMD patients with different characteristics and progression rates to late AMD, paving the way for improved personalization of AMD management. The code of CAT-LSTM can be found at GitHub.

Retinal vessel responses to flickering light are different in various systemic and ocular diseases and can be improved after successful therapy. We investigated retinal vessel response to flickering light in age-related macular degeneration (AMD) patients before and after treatment with a single intravitreal bevacizumab (Avastin(®) ) injection. In 10 patients with exudative AMD [age: median (1.quartile; 3.quartile) 76.0 (73.5; 80.0) years], retinal vessel reactions were examined by Dynamic Vessel Analyser (DVA) before and 3 months after a single intravitreal application of bevacizumab (1.25 mg). A baseline measurement was followed by three consecutive monochromatic flicker stimulations (530-600 nm, 12.5 Hz, 20 seconds). Temporal retinal vessel reaction was analysed and compared with the reaction in healthy controls. Mean arterial dilation at the end of flicker was not different in all groups. For veins this parameter amounted to: pre-treatment, 2.6 (1.7; 3.9)%; post-treatment, 2.9 (2.4; 4.0)%; control, 4.3 (3.2; 5.7)%; significant: pre-treatment - control (Dunnett's procedure, p < 0.05). Maximal dilation occurred in arteries at: pre-treatment, 17.5 (14.8; 32.5) seconds; post-treatment, 18.0 (16.6; 30.6) seconds; control, 14.5 (10.8; 17.3) seconds. Both AMD groups were slower (p < 0.05): in veins at 17.0 (14.5; 20.0) seconds, 12.8 (8.6; 14.8) seconds and 18.5 (17.1; 19.9) seconds, respectively; significant post-treatment - control (p < 0.05). In the post-treatment AMD group arterial constriction after stimulation occurred more slowly compared with the control group (p < 0.05). Dynamic retinal arterial and venous reactions to flickering light are altered in AMD compared with controls. Three months after a single injection of a vascular endothelial growth factor inhibitor, the investigated retinal dynamic vascular parameters were not altered in our study.

Using Deep Learning Models to Characterize Major Retinal Features on Color Fundus Photographs

RETRACTED: Deep learning network (DL-Net) based classification and segmentation of multi-class retinal diseases using OCT scans

Age-related macular degeneration (AMD) is a complex eye disorder affecting millions worldwide. This article uses deep learning techniques to investigate the relationship between AMD, genetics and optical coherence tomography (OCT) scans. The cohort consisted of 332 patients, of which 235 were diagnosed with AMD and 97 were controls with no signs of AMD. The genome-wide association studies summary statistics utilized to establish the polygenic risk score (PRS) in relation to AMD were derived from the GERA European study. A PRS estimation based on OCT volumes for both eyes was performed using a proprietary convolutional neural network (CNN) model supported by machine learning models. The method's performance was assessed using numerical evaluation metrics, and the Grad-CAM technique was used to evaluate the results by visualizing the features learned by the model. The best results were obtained with the CNN and the Extra Tree regressor (MAE = 0.55, MSE = 0.49, RMSE = 0.70, R2 = 0.34). Extending the feature vector with additional information on AMD diagnosis, age and smoking history improved the results slightly, with mainly AMD diagnosis used by the model (MAE = 0.54, MSE = 0.44, RMSE = 0.66, R2 = 0.42). Grad-CAM heatmap evaluation showed that the model decisions rely on retinal morphology factors relevant to AMD diagnosis. The developed method allows an efficient PRS estimation from OCT images. A new technique for analysing the association of OCT images with PRS of AMD, using a deep learning approach, may provide an opportunity to discover new associations between genotype-based AMD risk and retinal morphology.

The use of machine learning (ML) has been increasing rapidly in the medical imaging field, including computer-aided diagnosis (CAD), radiomics, and medical image analysis. Recently, an ML area called deep learning emerged in the computer vision field and became very popular in many fields. It started from an event in late 2012, when a deep-learning approach based on a convolutional neural network (CNN) won an overwhelming victory in the best-known worldwide computer vision competition, ImageNet Classification. Since then, researchers in virtually all fields, including medical imaging, have started actively participating in the explosively growing field of deep learning. In this paper, the area of deep learning in medical imaging is overviewed, including (1) what was changed in machine learning before and after the introduction of deep learning, (2) what is the source of the power of deep learning, (3) two major deep-learning models: a massive-training artificial neural network (MTANN) and a convolutional neural network (CNN), (4) similarities and differences between the two models, and (5) their applications to medical imaging. This review shows that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is the learning of image data directly without object segmentation or feature extraction; thus, it is the source of the power of deep learning, although the depth of the model is an important attribute. The class of ML with image input (or image-based ML) including deep learning has a long history, but recently gained popularity due to the use of the new terminology, deep learning. There are two major models in this class of ML in medical imaging, MTANN and CNN, which have similarities as well as several differences. In our experience, MTANNs were substantially more efficient in their development, had a higher performance, and required a lesser number of training cases than did CNNs. "Deep learning", or ML with image input, in medical imaging is an explosively growing, promising field. It is expected that ML with image input will be the mainstream area in the field of medical imaging in the next few decades.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

Recent advancements in deep learning, particularly convolutional networks, have rapidly become the preferred methodology for analyzing medical images, facilitating tasks like disease segmentation, classification, and pattern quantification. Central to these advancements is the capacity to leverage hierarchical feature representations acquired solely from data. This comprehensive review meticulously examines a variety of deep learning techniques applied across diverse healthcare domains, delving into the intricate realm of medical imaging to unveil concealed patterns through strategic deep learning methodologies. Encompassing a range of diseases, including Alzheimer’s, breast cancer, brain tumors, glaucoma, heart murmurs, retinal microaneurysms, colorectal liver metastases, and more, the analysis emphasizes contributions succinctly summarized in a tabular form.The table provides an overview of various deep learning approaches applied to different diseases, incorporating methodologies, datasets, and outcomes for each condition.Notably, performance metrics such as accuracy, specificity, sensitivity, and other crucial measures underscore the achieved results. Specifically, an in-depth discussion is conducted on the Convolutional Neural Network (CNN) owing to its widespread adoption as a paramount tool in computer vision tasks. Moreover, an exhaustive exploration encompasses deep learning classification approaches, procedural aspects of medical image processing, as well as a thorough examination of key features and characteristics. At the end, we delve into a range of research challenges and put forth potential avenues for future improvements in the field.

This study aimed to investigate the correspondence between intraretinal hyperreflective foci (IHRF) identified on optical coherence tomography (OCT) B-scans with hyperpigmentation on colour fundus photography (CFP) or hyperreflectivity on infrared reflectance (IR) images in eyes with age-related macular degeneration (AMD). Flash CFP, IR images and OCT B-scans obtained at the same visit were evaluated. Individual IHRF identified on OCT B-scans were assessed for the qualitative presence or absence of a hypotransmission tail into the choroid. The corresponding IR image obtained at the time of OCT acquisition was analysed for the presence or absence of hyperreflectivity in this region. The IR images were manually registered to the CFP image, and CFP images were inspected for the presence or absence of hyperpigmentation at the location of IHRF. From 122 eyes, a total of 494 IHRF were evaluated. For the primary analysis of qualitative presence or absence of hyperpigmentation on CFP and hyperreflectivity on IR at the locations corresponding to IHRF on OCT, 301 (61.0%) of the IHRFs demonstrated evidence of hyperpigmentation on CFP, while only 115 (23.3%) showed evidence of hyperreflectivity on IR. The qualitative determination of the presence or absence of an abnormality on CFP or IR were significantly different (p < 0.0001). 327 (66.2%) of the IHRF showed hypotransmission, and 80.4% of these IHRF showed hyperpigmentation on CFP, though only 23.9% (p < 0.0001) demonstrated hyperreflectivity on IR. Less than two-thirds of IHRF evident on OCT manifest as hyperpigmentation on colour photos, though IHRF with posterior shadowing are more likely to be evident as pigment. IR imaging appears to be even more poorly sensitive for visualizing IHRF.

Predictive Performance of an Updated Polygenic Risk Score for Age-Related Macular Degeneration

In recent years, deep learning (DL) has revolutionized medical imaging, offering notable promise in orthopedic diagnostics. This systematic review explores how DL, particularly Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), are applied to detect and analyze orthopedic anomalies such as fractures, spinal deformities, and foot deformities. We reviewed 63 peer-reviewed studies published between 2017 and 2025, analyzing their methodologies, datasets, performance metrics, and clinical relevance. The findings reveal significant advancements in fracture classification and vertebral labeling, though challenges persist for subtle anomalies and less-represented deformities. Despite encouraging results, limitations include small sample sizes, lack of external validation especially for rare pathologies. We conclude by identifying research gaps and proposing future directions for developing robust, clinically integrated DL tools to enhance diagnostic accuracy, data diversity, and anomaly detection in complex orthopedic scenarios.

The growing use of deep learning models in diagnosing medical images has resulted in impressive gains in terms of the accuracy in diagnosis. Still, the lack of transparency in these models tends to increase the resistance of these models in clinical use. This paper will fill the accuracy-interpretability gap by creating explainable deep learning models that do not affect predictive accuracy. An explainability methodology was incorporated into a convolutional neural network (CNN) architecture using the state-of-the-art explainability methods, Gradient-weighted Class Activation Mapping (Grad-CAM), Local Interpretable Model-Agnostic Explanations (LIME), and Integrated Gradients, to visualize model reasoning and locate important diagnostic regions of medical images. Benchmark radiology and histopathology data sets were evaluated experimentally to assess their accuracy and F1-score, interpretability measures, and the level of clinician agreement. The proposed model attained a diagnostic accuracy of 95.8 and an interpretability score of 0.89, which indicates a high similarity of model explanations and expert annotations. Using an explainability tool compared to the control group led to a marked increase in clinical trust and comprehension of model predictions. The results highlight that deep learning systems can be highly diagnostic and, at the same time, produce a transparent decision-making process. The study can be considered significant to developing human-centric artificial intelligence in healthcare as it contributes to accountability, reliability, and interpretability in medical image analysis.

Purpose: The purpose was to compare between color fundus photography (CFP) images and fundus autofluorescence (FAF) images of cases with geographical atrophy (GA) about size. To evaluate the relation between different phenotypes of FAF changes and best corrected visual acuity (BCVA) in cases with late dry age-related macular degeneration (AMD) (GA). Materials and Methods: This study was conducted on 18 eyes of 18 patients aged 55 years. Patients unwilling to participate in the study, suffering from hereditary fundus diseases, had previous laser photocoagulation treatment for any cause, were excluded. BCVA using Snellen chart was measured. Retinal imaging including CFP with 50° field camera: Topcon TRC-50 IX - and FAF using spectralis Heidelberg retinal angiograph (HRA) + optical coherence tomography (HRA, Heidelberg Engineering, Germany) were done. All images were optimized and then manually measured using image analysis software (Adobe Photoshop version CS6; Inc., San Jose, California, USA). Images of fundus autofluorescence in cases of late dry AMD were classified to phenotypes at the junctional zone according to the classification of abnormal FAF patterns. Results: According to sex distribution, 33.3% (6 eyes) were males and 66.7% (12 eyes) were females . The mean age of the study participants was 72.89 ± 9.09 years. About surface area of GA, the mean surface area of GA by FAF was 71094.56 ± 21490.53 pixels and by color fundus camera were 46236.56 ± 13153.46 pixels. About FAF phenotypes in late dry AMD cases, Twelve eyes (66.7%) had diffuse pattern, 11.1% (2 eyes) had a none pattern (no specific pattern), and 22.2% (4 eyes) had focal pattern. Color fundus camera underestimated the surface area of GA in cases of late dry AMD. BCVA was best in cases with no specific pattern of autofluorescence at the junctional zone of the GA, followed by cases with focal pattern of hyperautofluorescence while cases with diffuse increases of autofluorescence at the junctional zone showed the worst VA. Conclusions: CFP underestimated the size of the GA, as compared with FAF measured sizes. BCVA was best in cases with no specific pattern of autofluorescence at the junctional zone of the GA, followed by cases with focal pattern of hyperautofluorescence while cases with diffuse increases of autofluorescence at the junctional zone showed the worst VA.

This work reports a deep-learning based registration algorithm that aligns multi-modal retinal images collected from longitudinal clinical studies to achieve accuracy and robustness required for analysis of structural changes in large-scale clinical data. Deep-learning networks that mirror the architecture of conventional feature-point-based registration were evaluated with different networks that solved for registration affine parameters, image patch displacements, and patch displacements within the region of overlap. The ground truth images for deep learning-based approaches were derived from successful conventional feature-based registration. Cross-sectional and longitudinal affine registrations were performed across color fundus photography (CFP), fundus autofluorescence (FAF), and infrared reflectance (IR) image modalities. For mono-modality longitudinal registration, the conventional feature-based registration method achieved mean errors in the range of 39-53 µm (depending on the modality) whereas the deep learning method with region overlap prediction exhibited mean errors in the range 54-59 µm. For cross-sectional multi-modality registration, the conventional method exhibited gross failures with large errors in more than 50% of the cases while the proposed deep-learning method achieved robust performance with no gross failures and mean errors in the range 66-69 µm. Thus, the deep learning-based method achieved superior overall performance across all modalities. The accuracy and robustness reported in this work provide important advances that will facilitate clinical research and enable a detailed study of the progression of retinal diseases such as age-related macular degeneration.