Efficient multimodel method based on transformers and CoAtNet for Alzheimer's diagnosis

Multimodal neuroimaging has gained traction in Alzheimer's Disease (AD) diagnosis by integrating information from multiple imaging modalities to enhance classification accuracy. However, effectively handling heterogeneous data sources and overcoming the challenges posed by multiscale transform methods remains a significant hurdle. This article proposes a novel approach to address these challenges. To harness the power of diverse neuroimaging data, we employ a strategy that leverages optimized convolution techniques. These optimizations include varying kernel sizes and the incorporation of instance normalization, both of which play crucial roles in feature extraction from magnetic resonance imaging (MRI) and positron emission tomography (PET) images. Specifically, varying kernel sizes allow us to adapt the receptive field to different image characteristics, enhancing the model's ability to capture relevant information. Furthermore, we employ transposed convolution, which increases spatial resolution of feature maps, and it is optimized with varying kernel sizes and instance normalization. This heightened resolution facilitates the alignment and integration of data from disparate MRI and PET data. The use of larger kernels and strides in transposed convolution expands the receptive field, enabling the model to capture essential cross-modal relationships. Instance normalization, applied to each modality during the fusion process, mitigates potential biases stemming from differences in intensity, contrast, or scale between modalities. This enhancement contributes to improved model performance by reducing complexity and ensuring robust fusion. The performance of the proposed fusion method is assessed on three distinct neuroimaging datasets, which include: Alzheimer's Disease Neuroimaging Initiative (ADNI), consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, AD, and Early Mild Cognitive); Open Access Series of Imaging Studies (OASIS), consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, Mild Dementia, Very Mild Dementia); and whole-brain atlas neuroimaging (AANLIB) (consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, AD). To evaluate the quality of the fused images generated via our method, we employ a comprehensive set of evaluation metrics, including Structural Similarity Index Measurement (SSIM), which assesses the structural similarity between two images; Peak Signal-to-Noise Ratio (PSNR), which measures how closely the generated image resembles the ground truth; Entropy (E), which assesses the amount of information preserved or lost during fusion; the Feature Similarity Indexing Method (FSIM), which assesses the structural and feature similarities between two images; and Edge-Based Similarity (EBS), which measures the similarity of edges between the fused and ground truth images. The obtained fused image is further evaluated using a Mobile Vision Transformer. In the classification of AD vs. Cognitive Normal, the model achieved an accuracy of 99.00%, specificity of 99.00%, and sensitivity of 98.44% on the AANLIB dataset.

ABSTRACTMultimodal neuroimaging, combining data from different sources, has shown promise in the classification of the Alzheimer's disease (AD) stage. Existing multimodal neuroimaging fusion methods exhibit certain limitations, which require advancements to enhance their objective performance, sensitivity, and specificity for AD classification. This study uses the use of a Pareto‐optimal cosine color map to enhance classification performance and visual clarity of fused images. A mobile vision transformer (ViT) model, incorporating the swish activation function, is introduced for effective feature extraction and classification. Fused images from the Alzheimer's Disease Neuroimaging Initiative (ADNI), the Whole Brain Atlas (AANLIB), and Open Access Series of Imaging Studies (OASIS) datasets, obtained through optimized transposed convolution, are utilized for model training, while evaluation is achieved using images that have not been fused from the same databases. The proposed model demonstrates high accuracy in AD classification across different datasets, achieving 98.76% accuracy for Early Mild Cognitive Impairment (EMCI) versus LMCI, 98.65% for Late Mild Cognitive Impairment (LMCI) versus AD, 98.60% for EMCI versus AD, and 99.25% for AD versus Cognitive Normal (CN) in the ADNI dataset. Similarly, on OASIS and AANLIB, the precision of the AD versus CN classification is 99.50% and 96.00%, respectively. Evaluation metrics showcase the model's precision, recall, and F1 score for various binary classifications, emphasizing its robust performance.

The Accurate and the early detection of the neurodegenerative brain disorders such as the Alzheimer’s disease(AD) is crucial today for an effective patient care and prevention of disease progression. Computer Assisted Diagnosis (CAD) based on the neuroimaging data is an active research that was enhanced recent years by the deep learning methods. Brain modalities hold the key to achieve a comprehensive understanding of the brain changes and disorders. Magnetic Resonance Imaging(MRI) is a powerful brain modality and biomarker to capture the brain changes. It has been widely adopted for the AD diagnosis. Most of the recent studies are based on the Convolution Neural Network (CNN)using the MRI modality for the AD detection. However the CNN requires a huge amount of data. Furthermore, current CNN models cannot provide a good feature representation due to the lack of an adaptive channel weighting. The main drawback of the CNN network is its inability to learns the importance of the image features and the main correlations among them. Current methods that are based on the CNN does not ensure a valuable information extraction about which brain regions are related to the AD detection. To deal with these challenges, we proposed 2 contributions. Firstly, we used a Deep Convolution Generative Adversarial neural network(DCGAN) for data augmentation to enhance the training and to tackle the lack of data. We proposed also a new method for AD detection based on the integration of the Efficient Channel Attention(ECA-Net) into the Residual network. The ECA module improve the residual network by an attention mechanism to extract the most relevant features and ensure an adaptive weighting channel mechanism. Our method provide an efficient feature representation and extraction with low computational cost. The proposed method was validated using the ADNI (Alzheimer’s disease neuroimaging initiative) and the Open Access Series of Imaging Studies (OASIS). The experimental results outlines that our proposed method achieved the highest classification accuracy comparing with different methods by reaching an accuracy rate of 95%KeywordsAlzheimer’s diseaseData augmentationAttention mechanismDCGANECA-ResNet101

Alzheimer's disease (AD) is an incurable neurodegenerative disorder accounting for 70%-80% dementia cases worldwide. Although, research on AD has increased in recent years, however, the complexity associated with brain structure and functions makes the early diagnosis of this disease a challenging task. Resting-state functional magnetic resonance imaging (rs-fMRI) is a neuroimaging technology that has been widely used to study the pathogenesis of neurodegenerative diseases. In literature, the computer-aided diagnosis of AD is limited to binary classification or diagnosis of AD and MCI stages. However, its applicability to diagnose multiple progressive stages of AD is relatively under-studied. This study explores the effectiveness of rs-fMRI for multi-class classification of AD and its associated stages including CN, SMC, EMCI, MCI, LMCI, and AD. A longitudinal cohort of resting-state fMRI of 138 subjects (25 CN, 25 SMC, 25 EMCI, 25 LMCI, 13 MCI, and 25AD) from Alzheimer's Disease Neuroimaging Initiative (ADNI) is studied. To provide a better insight into deep learning approaches and their applications to AD classification, we investigate ResNet-18 architecture in detail. We consider the training of the network from scratch by using single-channel input as well as performed transfer learning with and without fine-tuning using an extended network architecture. We experimented with residual neural networks to perform AD classification task and compared it with former research in this domain. The performance of the models is evaluated using precision, recall, f1-measure, AUC and ROC curves. We found that our networks were able to significantly classify the subjects. We achieved improved results with our fine-tuned model for all the AD stages with an accuracy of 100%, 96.85%, 97.38%, 97.43%, 97.40% and 98.01% for CN, SMC, EMCI, LMCI, MCI, and AD respectively. However, in terms of overall performance, we achieved state-of-the-art results with an average accuracy of 97.92% and 97.88% for off-the-shelf and fine-tuned models respectively. The Analysis of results indicate that classification and prediction of neurodegenerative brain disorders such as AD using functional magnetic resonance imaging and advanced deep learning methods is promising for clinical decision making and have the potential to assist in early diagnosis of AD and its associated stages.

Purpose: Hard-to-interpret Black-box Machine Learning (ML) were often used for early Alzheimer's Disease (AD) detection. Methods: To interpret eXtreme Gradient Boosting (XGBoost), Random Forest (RF), and Support Vector Machine (SVM) black-box models a workflow based on Shapley values was developed. All models were trained on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and evaluated for an independent ADNI test set, as well as the external Australian Imaging and Lifestyle flagship study of Ageing (AIBL), and Open Access Series of Imaging Studies (OASIS) datasets. Shapley values were compared to intuitively interpretable Decision Trees (DTs), and Logistic Regression (LR), as well as natural and permutation feature importances. To avoid the reduction of the explanation validity caused by correlated features, forward selection and aspect consolidation were implemented. Results: Some black-box models outperformed DTs and LR. The forward-selected features correspond to brain areas previously associated with AD. Shapley values identified biologically plausible associations with moderate to strong correlations with feature importances. The most important RF features to predict AD conversion were the volume of the amygdalae, and a cognitive test score. Good cognitive test performances and large brain volumes decreased the AD risk. The models trained using cognitive test scores significantly outperformed brain volumetric models ($p<0.05$). Cognitive Normal (CN) vs. AD models were successfully transferred to external datasets. Conclusion: In comparison to previous work, improved performances for ADNI and AIBL were achieved for CN vs. Mild Cognitive Impairment (MCI) classification using brain volumes. The Shapley values and the feature importances showed moderate to strong correlations.

This letter proposes a novel way of using an ensemble of multiple deep convolutional neural networks (DCNNs) for Alzheimer's disease classification, based on magnetic resonance imaging (MRI) images. To create this ensemble of DCNNs, we propose to combine the use of multiple MRI projections (as input) with that of different DCNN architectures to increase the deep ensemble diversity. In particular, to find the optimal fusion weights of the DCNN members, we designed a novel deep ensemble generalization loss, which accounts for interaction and cooperation during the optimal weight search. The optimization framework, equipped with our ensemble generalization loss, was formulated and solved using the sequential quadratic programming. Through this method, we achieved optimal DCNN fusion weights (i.e., a high generalization performance). The experimental results showed that our proposed DCNN ensemble outperforms current deep learning-based methods: it is able to produce state-of-the-art results on the Alzheimer's disease neuroimaging initiative (ADNI) dataset.

Reducing variations in multi-center Alzheimer's disease classification with convolutional adversarial autoencoder.

Alzheimer's disease (AD) is one of the most common progressive neurodegenerative diseases. Structural magnetic resonance imaging (MRI) would provide abundant information on the anatomical structure of human organs. Fluorodeoxy‐glucose positron emission tomography (PET) obtains the metabolic activity of the brain. Previous studies have demonstrated that multi‐modality images could contribute to improve diagnosis of AD. However, these methods need to extract the handcrafted features that demand domain specific knowledge and image processing stage is time consuming. In order to tackle these problems, in this study, the authors propose a novel framework that ensembles three state‐of‐the‐art deep convolutional neural networks (DCNNs) with multi‐modality images for AD classification. In detail, they extract some slices from each subject of each modality, and every DCNN generates a probabilistic score for the input slices. Furthermore, a ‘dropout’ mechanism is introduced to discard low discrimination slices of the category probabilities. Then average reserved slices of each subject are acquired as a new feature. Finally, they train the Adaboost ensemble classifier based on single decision tree classifier with the MRI and PET probabilistic scores of each DCNN. Evaluations on Alzheimer's Disease Neuroimaging Initiative database show that the proposed algorithm has better performance compared to existing method, the algorithm proposed in this study significantly improved the classification accuracy.

Recent advancements in computer-aided diagnosis of Alzheimer's disease (AD) have primarily relied on unimodal data, particularly magnetic resonance imaging (MRI), to capture structural brain abnormalities. However, such single-source approaches often fail to account for genetic factors that play a critical role in AD onset and progression. To overcome this limitation, this study introduces an intermediate fusion architecture incorporating a cross-attention mechanism for AD classification using both MRI and genetic data. The proposed framework utilizes a ResNet-based convolutional neural network (CNN) to extract comprehensive structural representations from whole-brain MRI scans, while a Transformers encoder learns informative patterns from single nucleotide polymorphism (SNP) profiles. The features from both modalities are subsequently integrated through a cross-attention driven intermediate fusion module, enabling adaptive interaction between imaging and genetic domains to capture complementary and complex correlations. Experimental evaluations conducted on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset demonstrate that the proposed model achieves an overall classification accuracy of approximately 92,94%, significantly superior to unimodal models using only MRI or SNP data. The results demonstrate the effectiveness of the proposed cross-attention-based intermediate fusion strategy in enhancing multimodal representation learning, thereby improving the reliability of early and accurate Alzheimer's disease classification.

It is challenging to use incomplete multimodality data for Alzheimer's Disease (AD) diagnosis. The current methods to address this challenge, such as low-rank matrix completion (i.e., imputing the missing values and unknown labels simultaneously) and multi-task learning (i.e., defining one regression task for each combination of modalities and then learning them jointly), are unable to model the complex data-to-label relationship in AD diagnosis and also ignore the heterogeneity among the modalities. In light of this, we propose a new Maximum Mean Discrepancy (MMD) based Multiple Kernel Learning (MKL) method for AD diagnosis using incomplete multimodality data. Specifically, we map all the samples from different modalities into a Reproducing Kernel Hilbert Space (RKHS), by devising a new MMD algorithm. The proposed MMD method incorporates data distribution matching, pair-wise sample matching and feature selection in an unified formulation, thus alleviating the modality heterogeneity issue and making all the samples comparable to share a common classifier in the RKHS. The resulting classifier obviously captures the nonlinear data-to-label relationship. We have tested our method using MRI and PET data from Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset for AD diagnosis. The experimental results show that our method outperforms other methods.

During the recent years, there have been many studies implemented on the automatic diagnosis of Alzheimer's Disease (AD) using different methods. The focus of most of these studies has relied upon the detection of AD from neu-roimaging data. However, recognizing symptoms early as much as possible(Pre-detection) is crucial as disease modifying drugs will be most effective if administered early in the course of disease, before the occurrence of irreversible brain damages. Therefore, there is a high importance of utilizing automated techniques for the purpose of pre-detection of AD symptoms from such data. We report an experimental approach to evaluate the best pre-detection method of AD. Our study consists of two main experiments. Those two experiments were implemented using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Prior to our first experiment we have stated an assumption which is, if there is a successful AD detection method that will be successful in AD pre-detection also. Different studies have used different data sets and different diagnosing methods. Therefore we have verified the existing and the most successful detection method which is Support Vector Machine (SVMs) as the first experiment. According to the results obtained from the initial experiment (detection study) the sensitivity is 95.3%, the specificity is 71.4% and the accuracy is 84.4% with use of a SVM. Since those results were not successful, a deep learning based technique (Convolutional Neural Network) was proposed as the second experiment. The proposed Convolutional Neural Network (CNN) model was being tested using different image segmentation methods and different datasets. Finally the best image segmentation method obtained a high accuracy around 96% (sensitivity - 96%, specificity - 98%). And the CNN model remains unbiased to the dataset. Results of those experiments suggest an important role for early diagnosis of Alzheimer's disease using image processing and deep learning techniques.

A challenging point in neuroimaging is the diagnosis of Alzheimer's disease (AD) during its asymptomatic phase. Among all the biomarkers proposed in the literature, a measure of the hippocampal atrophy via Magnetic Resonance Imaging (MRI) seems to be one of the most reliable. Refined image processing techniques were already proposed to automatically extract the hippocampal boxes from images acquired with the standard full brain acquisition protocol suggested by the Alzheimer's Disease Neuroimaging Initiative (ADNI). In order to enhance this approach, here we propose a high resolution (HR) MRI protocol focused on the medial temporal lobe (MTL) mainly conceived for 1.5T MRI device, hereafter referred as MTL-HR protocol. A preliminary characterization of its behavior when compared to the standard ADNI protocol is also presented.

Polypathology is a major driver of heterogeneity in the clinical presentation and extent of neurodegeneration (N) in patients with Alzheimer's disease (AD). Beyond amyloid (A) and tau (T) pathologies, over half of patients with AD have concomitant pathology such as α-synuclein (S) in mixed AD with Lewy body disease (LBD). Patients with multiple aetiology dementia such as AD + LBD have faster progression and potentially differential responses to targeted treatments, although the diagnosis of AD + LBD can be challenging given the overlapping clinical and imaging features. Development and validation of improved in vivo biomarkers are required to study relationships between N and S and identify imaging patterns reflecting mixed AD + LBD pathologies. We hypothesized that individual proteinopathies, such as T and S, are associated with commensurate levels of N. Thus, we assessed biomarkers of A, T, N and S with PET, MRI and CSF seeding amplification assay (SAA) data to determine molecular presentations of mixed A+S+ versus A+S- cognitively impaired patients from the Alzheimer's Disease Neuroimaging Initiative (ADNI). Strikingly, A+S+ patients had parieto-occipital 18F-fluorodeoxyglucose hypometabolism (a measure of N) disproportionate to the degree of regional atrophy or T burden, highlighting worse hypometabolism associated with S+ status on SAA. Following up on this hypometabolic mismatch with CSF metabolite and proteome analyses, we found that A+S+ patients exhibited lower CSF levels of dopamine metabolites and synaptic markers like neuronal pentraxin-2 (NPTX2), suggesting that altered neurotransmission and neuron integrity contribute to this dissociation between metabolic PET and MRI. Potential confounders exist when studying relations between N, AD and LBD pathologies, including neuroinflammation and other non-Alzheimer's pathologies in mixed dementia, although our findings imply posterior hypometabolic mismatch is related more to S than vascular or TDP-43 pathology. Overall, A+S+ patients had posterior mismatch with excessive 18F-fluorodeoxyglucose hypometabolism relative to atrophy or T load, possibly reflecting impaired neuronal integrity. Further research must disentangle the impact of multiple proteinopathies and clinicopathologic factors on hypometabolism and atrophy. Cumulatively, patients with mixed AD + LBD aetiologies harbour a unique metabolic PET mismatch signature.

BackgroundRecent advances in deep learning have allowed multi‐omics data to be used to detect early stages of Alzheimer's disease (AD). The metabolome, as the end‐product of biological cascades including the genome, transcriptome, and proteome, may have informative elements that could serve as potential AD biomarkers. We report a novel two‐step deep learning method to detect AD and progression using lipidomics data.MethodWe used serum‐based cross‐sectional lipidome data with 781 lipids from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) including 216 cognitively normal (CN), 635 MCI, and 382 dementia (AD). Phenotype influence scores (PIS) was derived by deep learning‐based circling Sliding Window Association Test approach (Circling SWAT) (Fig. 1‐A), an extension of SWAT (Jo et al., 2022) with correlation heatmap and dendrogram analysis for omics data with minimal features. We used convolutional neural networks (CNN) to classify AD from CN based on top‐scoring metabolites. The AD classification model was used to predict conversion of MCI to AD within two years using 146 MCI‐convertors who developed AD and 190 MCI‐Stable who did not develop AD over two years from baseline. We removed two deDE lipids, deDE(18:2) and deDE(20:4), because their associations with AD were driven mainly by AD‐related medication.ResultPhenotype influence scores (PIS) were calculated for each lipid using circling SWAT to determine their impacts on AD. The metabolites with the highest PIS were LPE(22:6), PC(39:5), and PC(15‐MHDA_22:6). When CNN was applied to the top 50 lipids by PIS, AD and CN classification accuracy was 73.6% (Fig. 1‐B), which was 5.1% higher than the accuracy obtained using only age, sex, and APOE ε4. For MCI conversion to AD, the AD/CN classifier accuracy was 69.9% or 8.8% higher than a model using only age, sex, and APOE ε4.ConclusionThe circling SWAT method appears promising for the analysis of lipidome data as it was able to identify several AD‐related lipids and predict disease progression.

Background and Purpose: A majority studies on diagnosis of Alzheimer’s Disease (AD) are based on an assumption: the training and testing data are drawn from the same distribution. However, in the diagnosis of AD and mild cognitive impairment (MCI), this identical-distribution assumption may not hold. To solve this problem, we utilize the transfer learning method into the diagnosis of AD. Methods: The MR (Magnetic Resonance) images were segmented using spm-Dartel toolbox and registrated with Automatic Anatomical Labeling (AAL) atlas, then the gray matter (GM) tissue volume of the anatomical region were computed as characteristic parameter. The information gain was introduced for feature selection. The TrAdaboost algorithm was used to classify AD, MCI, and normal controls (NC) data from Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, meanwhile, the “knowledge” learned from ADNI was transferred to AD samples from local hospital. The classification accuracy, sensitivity and specificity were calculated and compared with four classical algorithms. Results: In the experiment of transfer task: AD to MCI, 177 AD and 40NC subjects were grouped as training data; 245 MCI and 45 remaining NC subjects were combined as testing data, the highest accuracy achieved 85.4%, higher than the other four classical algorithms. Meanwhile, feature selection that is based on information gain reduced the features from 90 to 7, controlled the redundancy efficiently. In the experiment of transfer task: ADNI to local hospital data, the highest accuracy achieved 93.7%, and the specificity achieved 100%. Conclusions: The experimental results showed that our algorithm has a clear advantage over classic classification methods with higher accuracy and less fluctuation.