A hybrid Transformer-CNN framework for uncertainty-guided semi-supervised multiclass eye disease classification with enhanced interpretability.

BackgroundAlzheimer's disease (AD) is a progressive neurodegenerative disorder marked by cognitive decline and memory impairment. Early and accurate detection is critical for clinical intervention. While machine and deep learning approaches have been widely used to predict AD progression, recent work emphasizes quantifying predictive uncertainty in high‐stakes medical contexts. However, many studies focus on limited genetic regions (e.g., APOE), highlighting the need for broader whole‐genome sequencing (WGS) analyses.MethodWe obtained 1,050 WGS datasets (443 cognitively normal, 607 AD‐diagnosed) from ADNI, ADNI‐WGS‐2, and ADSP‐FUS1‐ADNI‐WGS‐2. SNPs were extracted from a region containing the APOE gene on chromosome 19, then divided into fixed‐size windows (“tokens”) for a Transformer‐based classification model. Monte Carlo (MC) Dropout was applied during training and inference to enable multiple forward passes, providing predictive variance. Models with and without MC Dropout were compared using accuracy (ACC), area under the curve (AUC), F1 score, sensitivity, specificity, expected calibration error (ECE), and reliability diagrams. Predictions were further stratified into “Uncertain” (top 25% variance) and “Certain” (bottom 75%) to examine accuracy differences.ResultStratification by predictive variance revealed that the Uncertain group, with 53 samples, had an accuracy of 0.5472, while the Certain group, with 157 samples, had 0.6497. This indicates that Monte Carlo Dropout–derived variance effectively captures higher uncertainty. The Monte Carlo Dropout model showed a slight increase in accuracy from 0.6143 to 0.6238 and in area under the curve from 0.6644 to 0.6832 compared with baseline, but calibration worsened when the expected calibration error rose from 0.1024 to 0.1858. Sensitivity and specificity shifted from 0.7417 to 0.6833 and from 0.4444 to 0.5444, while the F1 score decreased from 0.6873 to 0.6749.ConclusionThese findings demonstrate the feasibility of using MC Dropout within a Transformer‐based WGS framework to estimate predictive uncertainty in AD classification. Variance‐based stratification effectively flagged samples with lower confidence. However, MC Dropout did not consistently improve calibration or overall accuracy, indicating a need for refining dropout settings, Transformer hyperparameters, and potentially Bayesian time‐series methods. Future work should also explore complementary approaches to balance classification accuracy with reliable uncertainty estimates.

Reliable detection of unknown signals is essential for ensuring the robustness of underwater acoustic sensing systems, particularly in maritime security and autonomous navigation. However, Conventional deep learning models often exhibit overconfidence when encountering unknown signals and are unable to quantify predictive uncertainty due to their deterministic inference process. To address these limitations, this study proposes a novelty detection framework that integrates an out-of-distribution detector for neural networks (ODIN) with Monte Carlo (MC) dropout. ODIN mitigates model overconfidence and enhances the separability between known and unknown signals through softmax probability calibration, while MC dropout introduces stochasticity via multiple forward passes to estimate predictive uncertainty-an element critical for stable sensing in real-world underwater environments. The resulting probabilistic outputs are modeled using Gaussian mixture models fitted to ODIN-calibrated softmax distributions of known classes. The Kullback-Leibler divergence is then employed to quantify deviations of test samples from known class behavior. Experimental evaluations on the DeepShip dataset demonstrate that the proposed method achieves, on average, a 9.5% and 5.39% increase in area under the receiver operating characteristic curve, and a 7.82% and 2.63% reduction in false positive rate at 95% true positive rate, compared to the MC dropout and ODIN baseline, respectively. These results confirm that integrating stochastic inference with ODIN significantly enhances the stability and reliability of novelty detection in underwater acoustic environments.

We introduce distributed NLI, a new NLU task with a goal to predict the distribution of human judgements for natural language inference. We show that by applying additional distribution estimation methods, namely, Monte Carlo (MC) Dropout, Deep Ensemble, Re-Calibration, and Distribution Distillation, models can capture human judgement distribution more effectively than the softmax baseline. We show that MC Dropout is able to achieve decent performance without any distribution annotations while Re-Calibration can give further improvements with extra distribution annotations, suggesting the value of multiple annotations for one example in modeling the distribution of human judgements. Despite these improvements, the best results are still far below the estimated human upper-bound, indicating that predicting the distribution of human judgements is still an open, challenging problem with a large room for improvements. We showcase the common errors for MC Dropout and Re-Calibration. Finally, we give guidelines on the usage of these methods with different levels of data availability and encourage future work on modeling the human opinion distribution for language reasoning.

Uncertainty assessment has gained rapid interest in medical image analysis. A popular technique to compute epistemic uncertainty is the Monte-Carlo (MC) dropout technique. From a network with MC dropout and a single input, multiple outputs can be sampled. Various methods can be used to obtain epistemic uncertainty maps from those multiple outputs. In the case of multi-class segmentation, the number of methods is even larger as epistemic uncertainty can be computed voxelwise per class or voxelwise per image. This paper highlights a systematic approach to define and quantitatively compare those methods in two different contexts: class-specific epistemic uncertainty maps (one value per image, voxel and class) and combined epistemic uncertainty maps (one value per image and voxel). We applied this quantitative analysis to a multi-class segmentation of the carotid artery lumen and vessel wall, on a multi-center, multi-scanner, multi-sequence dataset of Magnetic Resonance (MR) images. We validated our analysis over 144 sets of hyperparameters of a model. Our main analysis considers the relationship between the order of the voxels sorted according to their epistemic uncertainty values and the misclassification of the prediction. Under this consideration, the comparison of combined uncertainty maps reveals that the multi-class entropy and the multi-class mutual information statistically out-perform the other combined uncertainty maps under study (the averaged entropy, the averaged variance, the similarity Bhattacharya coefficient and the similarity Kullback-Leibler divergence). In a class-specific scenario, the one-versus-all entropy statistically out-performs the class-wise entropy, the class-wise variance and the one versus all mutual information. The class-wise entropy statistically out-performs the other class-specific uncertainty maps in term of calibration. We made a python package available to reproduce our analysis on different data and tasks.

This study evaluated a novel offline, AI-driven age-related macular degeneration (AMD) screening algorithm against fundus image-only grading and the standard of care (combined Spectral Domain-Optical Coherence Tomography (SD-OCT) and fundus image grading). Conducted prospectively at a South Asian tertiary eye hospital, this study utilized a validated smartphone-based non-mydriatic fundus camera to capture macula-centred images. The Medios AI's ability to detect referable AMD was compared to a reference standard image grading, using fundus images from the Zeiss Clarus 700 table-top camera and SD-OCT line scan across fovea. Three retina specialists provided blinded AMD diagnoses based on: (1) Zeiss Clarus 700 fundus images alone, and (2) combined SD-OCT and fundus images (standard of care). Referable AMD was defined as intermediate or advanced AMD. Among 984 eyes from 492 patients (mean age 61.8 ± 9.9 years), 52% had referable AMD. Inter-grader agreement was strong, with Cohen's Kappa scores of 0.81-0.84. The Medios AI's sensitivity and specificity for detecting referable AMD against fundus-only grading (n = 492) were 88.48% (95% CI: 84.04-92.03%) and 87% (95% CI: 81.86-91.11%), respectively. Against combined grading (n = 489), AI sensitivity was 90.62% (95% CI: 86.37-93.90%), and specificity was 85.41% (95% CI: 80.21-89.68%). False negatives were primarily intermediate AMD (71%), while 59% of false positives were early AMD. The novel, automated, offline AMD AI integrated on a smartphone fundus camera demonstrated robust performance in identifying referable forms of AMD, supporting its potential as an affordable and accessible screening solution.

Uncertainty aware predictive maintenance using a hybrid Transformer with Monte Carlo Dropout and conformal prediction

Heart disease remains a leading cause of mortality worldwide. Early prediction and diagnosis are crucial for effective treatment and management. This study employs the Random Forest algorithm to predict heart disease using a dataset comprising various clinical features. The methodology includes data preprocessing, feature encoding, model training, and evaluation. The model's performance is assessed using accuracy metrics and a confusion matrix. The results demonstrate the efficacy of the Random Forest algorithm in predicting heart disease, highlighting its potential as a reliable tool in clinical decision support systems.

BackgroundA large language model is a type of artificial intelligence (AI) model that opens up great possibilities for health care practice, research, and education, although scholars have emphasized the need to proactively address the issue of unvalidated and inaccurate information regarding its use. One of the best-known large language models is ChatGPT (OpenAI). It is believed to be of great help to medical research, as it facilitates more efficient data set analysis, code generation, and literature review, allowing researchers to focus on experimental design as well as drug discovery and development.ObjectiveThis study aims to explore the potential of ChatGPT as a real-time literature search tool for systematic reviews and clinical decision support systems, to enhance their efficiency and accuracy in health care settings.MethodsThe search results of a published systematic review by human experts on the treatment of Peyronie disease were selected as a benchmark, and the literature search formula of the study was applied to ChatGPT and Microsoft Bing AI as a comparison to human researchers. Peyronie disease typically presents with discomfort, curvature, or deformity of the penis in association with palpable plaques and erectile dysfunction. To evaluate the quality of individual studies derived from AI answers, we created a structured rating system based on bibliographic information related to the publications. We classified its answers into 4 grades if the title existed: A, B, C, and F. No grade was given for a fake title or no answer.ResultsFrom ChatGPT, 7 (0.5%) out of 1287 identified studies were directly relevant, whereas Bing AI resulted in 19 (40%) relevant studies out of 48, compared to the human benchmark of 24 studies. In the qualitative evaluation, ChatGPT had 7 grade A, 18 grade B, 167 grade C, and 211 grade F studies, and Bing AI had 19 grade A and 28 grade C studies.ConclusionsThis is the first study to compare AI and conventional human systematic review methods as a real-time literature collection tool for evidence-based medicine. The results suggest that the use of ChatGPT as a tool for real-time evidence generation is not yet accurate and feasible. Therefore, researchers should be cautious about using such AI. The limitations of this study using the generative pre-trained transformer model are that the search for research topics was not diverse and that it did not prevent the hallucination of generative AI. However, this study will serve as a standard for future studies by providing an index to verify the reliability and consistency of generative AI from a user’s point of view. If the reliability and consistency of AI literature search services are verified, then the use of these technologies will help medical research greatly.

BackgroundMigrants in host countries are at risk for the development of mental health conditions. The two aims of the study were to describe routine diagnoses of mental disorders among migrant patients at primary healthcare level and the associated risk factors, and to test the utility of an innovative migrant mental health assessment by evaluating whether the health professionals followed the recommendations proposed by the clinical decision support system (CDSS) tool. MethodsA cross-sectional study was carried out in eight primary care centres (PCCs) in four non-randomly selected health regions of Catalonia, Spain from March to December 2018. Routine health data and mental health diagnoses based on the International Classification of Diseases (10th edition), including mental, behavioural and neuro developmental disorders (F01-F99), symptoms and signs involving emotional state (R45), and sleep disorders (G47), were extracted from the electronic health records. The proportion of mental health conditions was estimated and logistic regression models were used to assess any possible association with mental health disorders. The utility of the mental health assessment was assessed with the proportion of questionnaires performed by health professionals for migrants fulfilling the mental health screening criterion (country of origin with an active conflict in 2017) and the diagnoses given to the screened patients. ResultsOf 14,130 migrants that visited any of the PCCs during the study period, 7,358 (52.1 %) were women with a median age of 38.0 years-old. There were 520/14,130 (3.7 %) migrant patients diagnosed with a mental disorder, being more frequent among women (342/7,358; 4.7 %, p-value < 0.001), migrants from Latin-America (177/3,483; 5.1 %, p < 0.001) and those who recently arrived in Spain (170/3,672; 4.6 %, p < 0.001). A lower proportion of mental disorders were reported in migrants coming from conflicted countries in 2017 (116/3,669, 3.2 %, p = 0.053).Out of the 547 mental health diagnoses reported in 520 patients, 69/14,130 (0.5 %) were mood disorders, 346/14,130 (2.5 %) anxiety disorders and 127/14,130 (0.9 %) sleeping disorders. Mood disorders were more common in migrants from Eastern Europe (25/2,971; 0.8 %, p < 0.001) and anxiety disorders in migrants from Latin-America (126/3,483; 3.6 %, p < 0.001), while both type of disorders were more often reported in women (p < 0.001).In the adjusted model, women (aOR: 1.5, [95 % CI 1.2–1.8, p < 0.001]), migrants with more than one visit to the health center during the study period (aOR: 4.4, [95 %CI 2.8–6.8, p < 0.001]) and who presented an infectious disease (aOR: 2.1, [95 %CI 1.5–3.1, p < 0.001]) had higher odds of having a mental disorder.Lastly, out of the 1,840 migrants coming from a conflicted country in 2017 who were attended in centres where the CDSS tool was implemented, 29 (1.6 %) had a mental health assessment performed and the tool correctly identified one individual. ConclusionsMental health is a condition that may be overlooked in migrants at primary healthcare. Interventions at this level of care must be reinforced and adapted to the needs and circumstances of migrants to ensure equity in health services.

Despite the success of Convolutional Neural Network-based Computer-Aided Diagnosis research, its clinical applications remain challenging. Accordingly, developing medical Artificial Intelligence (AI) fitting into a clinical environment requires identifying/bridging the gap between AI and Healthcare sides. Since the biggest problem in Medical Imaging lies in data paucity, confirming the clinical relevance for diagnosis of research-proven image augmentation techniques is essential. Therefore, we hold a clinically valuable AI-envisioning workshop among Japanese Medical Imaging experts, physicians, and generalists in Healthcare/Informatics. Then, a questionnaire survey for physicians evaluates our pathology-aware Generative Adversarial Network (GAN)-based image augmentation projects in terms of Data Augmentation and physician training. The workshop reveals the intrinsic gap between AI/Healthcare sides and solutions on Why (i.e., clinical significance/interpretation) and How (i.e., data acquisition, commercial deployment, and safety/feeling safe). This analysis confirms our pathology-aware GANs' clinical relevance as a clinical decision support system and non-expert physician training tool. Our findings would play a key role in connecting inter-disciplinary research and clinical applications, not limited to the Japanese medical context and pathology-aware GANs.

Despite the success of Convolutional Neural Network-based Computer-Aided Diagnosis research, its clinical applications remain challenging. Accordingly, developing medical Artificial Intelligence (AI) fitting into a clinical environment requires identifying/bridging the gap between AI and Healthcare sides. Since the biggest problem in Medical Imaging lies in data paucity, confirming the clinical relevance for diagnosis of research-proven image augmentation techniques is essential. Therefore, we hold a clinically valuable AI-envisioning workshop among Japanese Medical Imaging experts, physicians, and generalists in Healthcare/Informatics. Then, a questionnaire survey for physicians evaluates our pathology-aware Generative Adversarial Network (GAN)-based image augmentation projects in terms of Data Augmentation and physician training. The workshop reveals the intrinsic gap between AI/Healthcare sides and solutions on Why (i.e., clinical significance/interpretation) and How (i.e., data acquisition, commercial deployment, and safety/feeling safe). This analysis confirms our pathology-aware GANs’ clinical relevance as a clinical decision support system and non-expert physician training tool. Our findings would play a key role in connecting inter-disciplinary research and clinical applications, not limited to the Japanese medical context and pathology-aware GANs.

To develop and test a web-based Clinical Decision Support System (CDSS) tool, which integrated a new fuzzy multiple criteria classification methodology named PROAFTN in acute leukemia (AL) diagnosis. We have integrated a PROAFTN method and developed a web-based clinical decision support system using standard JSP, servlets, and XML technologies. All website data are database-driven; and the database system can handle data store, update, and retrieval instantly. Since the system was moved to a web server, we have started our experimental testing on 191 AL cases. The percentage of correct classification in this experimental testing was consistent with the proposed prototype. 96.4% of AL cases were correctly classified, proving that web-integration can be a promising tool for dissemination of CDSS tools. We found our system to be robust and capable of deployment for referring physicians. Our experimental results suggest that the Internet has promise as a means for distribution of CDSS tools. This system will help to: 1) make a "virtual" diagnosis and to compare its performances with given clinical diagnosis; 2) exchange health information between physicians and hematologists at the location and time of need; 3) assist online learning and simulate cases for training practitioners; 4) implement a strict security and access control for transmission of electronic health data through the Internet. The method will not replace specialists, but was developed to assist biologist-hematologists and general practitioners remotely in making decisions on medical diagnosis.

Binarized multi-gate mixture of Bayesian experts for cardiac syndrome X diagnosis: A clinician-in-the-loop scenario with a belief-uncertainty fusion paradigm

Precise probabilistic forecasts are fundamental for energy risk management, and there is a wide range of both statistical and machine learning models for this purpose. Inherent to these probabilistic models is some form of uncertainty quantification. However, most models do not capture the full extent of uncertainty, which arises not only from the data itself but also from model and distributional choices. In this study, we examine uncertainty quantification in state-of-the-art statistical and deep learning probabilistic forecasting models for electricity price forecasting in the German market. In particular, we consider distributional deep neural networks (DDNNs) and augment them with an ensemble approach, Monte Carlo (MC) dropout, LASSO-estimated autoregressive (LEAR), and conformal prediction to account for model uncertainty. Additionally, we consider the eXtreme Gradient Boosting (XGBoost) approach combined with quantile regression averaging (QRA), generalized autoregressive conditional heteroskedasticity (GARCH), and conformal prediction. Across a range of performance metrics, we find that the XGBoost-based models perform well in terms of probabilistic forecasting, irrespective of the uncertainty quantification method. Furthermore, we find that DDNNs benefit from incorporating both data and model uncertainty, improving both point and probabilistic forecasting. Uncertainty itself appears to be best captured by the models using conformal prediction. Overall, our extensive study shows that all models under consideration perform competitively. However, their relative performance depends on the choice of metrics for point and probabilistic forecasting.

Background In proton therapy, it is disputed whether synthetic computed tomography (sCT), derived from magnetic resonance imaging (MRI), permits accurate dose calculations. On the one hand, an MRI-only workflow could eliminate errors caused by, e.g., MRI-CT registration. On the other hand, the extra error would be induced due to an sCT generation model. This work investigated the systematic and random model error induced by sCT generation of a widely discussed deep learning model, pix2pix. Material and Methods An open-source image dataset of 19 patients with cancer in the pelvis was employed and split into 10, 5, and 4 for training, testing, and validation of the model, respectively. Proton pencil beams (200 MeV) were simulated on the real CT and generated sCT using the tool for particle simulation (TOPAS). Monte Carlo (MC) dropout was used for error estimation (50 random sCT samples). Systematic and random model errors were investigated for sCT generation and dose calculation on sCT. Results For sCT generation, random model error near the edge of the body (∼200 HU) was higher than that within the body (∼100 HU near the bone edge and <10 HU in soft tissue). The mean absolute error (MAE) was 49 ± 5, 191 ± 23, and 503 ± 70 HU for the whole body, bone, and air in the patient, respectively. Random model errors of the proton range were small (<0.2 mm) for all spots and evenly distributed throughout the proton fields. Systematic errors of the proton range were −1.0(±2.2) mm and 0.4(±0.9)%, respectively, and were unevenly distributed within the proton fields. For 4.5% of the spots, large errors (>5 mm) were found, which may relate to MRI-CT mismatch due to, e.g., registration, MRI distortion anatomical changes, etc. Conclusion The sCT model was shown to be robust, i.e., had a low random model error. However, further investigation to reduce and even predict and manage systematic error is still needed for future MRI-only proton therapy.