Multi-stage framework using transformer models, feature fusion and ensemble learning for enhancing eye disease classification

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Multiple Sclerosis (MS) is a chronic neuroinflammatory disease of the Central Nervous System (CNS) in which the body's immune system attacks and destroys the myelin sheath that protects nerve fibers, leading to a wide range of debilitating symptoms and causing disruption of axonal signal transmission. Accurate prediction, diagnosis, monitoring and treatment (PDMT) of MS are essential to improve patient outcomes. Recent advances in neuroimaging technologies, particularly electroencephalography (EEG), combined with machine learning (ML) techniques - including Deep Learning (DL) models - offer promising avenues for enhancing MS management. This systematic review synthesizes existing research on the application of ML and DL models to EEG data for MS. It explores the methodologies used, with a focus on DL architectures such as Convolutional Neural Networks (CNNs) and hybrid models, and highlights recent advancements in ML techniques and EEG technologies that have significantly improved MS diagnosis and monitoring. The review addresses the challenges and potential biases in using ML-based EEG analysis for MS. Strategies to mitigate these challenges, including advanced preprocessing techniques, diverse training datasets, cross-validation methods, and explainable Artificial Intelligence (AI), are discussed. Finally, the paper outlines potential future applications and trends in ML for MS management. This review underscores the transformative potential of ML-enhanced EEG analysis in improving MS management, providing insights into future research directions to overcome existing limitations and further improve clinical practice.

Background Accurate prediction of the remaining useful life (RUL) of assets is fundamental to the development of effective maintenance strategies and overall asset management. Despite significant advancements, there remains a notable gap in integrating fault detection and diagnostics (FDD) with RUL prediction models to create more comprehensive and accurate maintenance systems. One of the key challenges in this field is the limited ability of current models to generalize effectively across different types of equipment and varying operating conditions. This gap emphasizes the need for further research and innovation in developing robust and adaptable RUL prediction methodologies that can be applied broadly across diverse industrial scenarios. Methodology This review systematically evaluates the machine learning (ML) and deep learning (DL) techniques used for anomaly detection and RUL prediction, focusing on their efficacy and practical application. By adhering to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) criteria, the review identifies and addresses the deficiencies in existing models. It explores a range of machine learning and deep learning methods, including probabilistic approaches, hybrid models that combine multiple machine learning techniques, and neural networks designed to handle large-scale time-series data. The review also examines the potential for synergy between machine learning models and FDD, aiming to enhance the precision of equipment monitoring and the early detection of defects. The challenges of data variability, the irregularity in equipment deterioration, and the interpretability of complex models are highlighted. Results The analysis reveals that while current machine learning and deep learning models have made considerable strides in predicting the RUL of assets, significant challenges remain, particularly in their ability to generalize across various equipment types and operational contexts. Hybrid models and neural networks have shown promise in improving the accuracy of RUL predictions, especially when managing large, complex datasets. However, the irregular nature of equipment wears and tear, coupled with data variability, continues to pose significant challenges. The review highlights the need for more robust and adaptable models that can not only predict RUL more accurately but also integrate seamlessly with FDD systems to provide a more holistic approach to maintenance. Conclusion This comprehensive review focusses on the need for continued research in developing more integrated, generalizable, and efficient predictive maintenance systems. By exploring the application of AI in virtual assistants, the review suggests promising avenues for extending asset longevity and optimizing maintenance schedules. While current models offer valuable insights, they must evolve to address the identified gaps in generalizability and model interpretability.

The Internet of Things (IoT) has transformed various sectors by enabling connectivity and automation through smart devices. However, the expansion of IoT networks has introduced new security challenges, particularly concerning the detection and prevention of cyber threats. This study explores the application of machine learning (ML) models for enhancing network security in IoT-enabled smart systems. The primary objective was to evaluate the performance of different ML algorithms, including Convolutional Neural Networks (CNN), Random Forest (RF), Support Vector Machine (SVM), Decision Tree (DT), K-Means, and Autoencoder, in detecting and mitigating security threats in IoT environments. The study also aimed to identify the most effective model in terms of accuracy, precision, recall, and F1-score. The research methodology involved training and testing these models on publicly available IoT security datasets, followed by performance evaluation based on the key metrics. Statistical analysis, including One-Way ANOVA and post-hoc testing, was used to assess the significance of the performance differences across the models. The results revealed that CNN outperformed all other models, achieving the highest accuracy (96.7%), precision (95.8%), recall (97.2%), and F1-score (96.5%). Random Forest also demonstrated strong performance with an accuracy of 95.6% and a balanced evaluation across all metrics. In contrast, traditional models like SVM and DT showed comparatively lower performance, while Autoencoder struggled with classification tasks, particularly in real-time threat detection. Statistical analysis confirmed the significant differences in model performance. In conclusion, the study demonstrates the effectiveness of deep learning and ensemble models, particularly CNN and Random Forest, in enhancing IoT network security. The research highlights the importance of selecting the appropriate ML model based on the specific requirements of the IoT environment and computational constraints, and suggests that hybrid models and model optimization strategies may offer promising solutions for resource-constrained IoT systems.

IntroductionSeveral attempts have been made to enhance text-based sentiment analysis’s performance. The classifiers and word embedding models have been among the most prominent attempts. This work aims to develop a hybrid deep learning approach that combines the advantages of transformer models and sequence models with the elimination of sequence models’ shortcomings.MethodsIn this paper, we present a hybrid model based on the transformer model and deep learning models to enhance sentiment classification process. Robustly optimized BERT (RoBERTa) was selected for the representative vectors of the input sentences and the Long Short-Term Memory (LSTM) model in conjunction with the Convolutional Neural Networks (CNN) model was used to improve the suggested model’s ability to comprehend the semantics and context of each input sentence. We tested the proposed model with two datasets with different topics. The first dataset is a Twitter review of US airlines and the second is the IMDb movie reviews dataset. We propose using word embeddings in conjunction with the SMOTE technique to overcome the challenge of imbalanced classes of the Twitter dataset.ResultsWith an accuracy of 96.28% on the IMDb reviews dataset and 94.2% on the Twitter reviews dataset, the hybrid model that has been suggested outperforms the standard methods.DiscussionIt is clear from these results that the proposed hybrid RoBERTa–(CNN+ LSTM) method is an effective model in sentiment classification.

Objectives: Language detection is the process of identifying a language associated with a text. The proposed system aims to detect the Dravidian language that is associated with the given text using different machine learning and deep learning algorithms. The paper presents an empirical analysis of the results obtained using the different models. It also aims to evaluate the performance of a language agnostic model for the purpose of language detection. Method: An empirical analysis of Dravidian language identification in social media text using machine learning and deep learning approaches with k-fold cross validation has been implemented. The identification of Dravidian languages, including Tamil, Malayalam, Tamil Code Mix, and Malayalam Code Mix, is performed using both machine learning (ML) and deep learning algorithms. The machine learning algorithms used for language detection are Naive Bayes (NB), Multinomial Logistic Regression (MLR), Support Vector Machine (SVM), and Random Forest (RF). The supervised Deep Learning (DL) models used include BERT, mBERT and language agnostic models. Findings: The language agnostic model outperform all other models considering the task of language detection in Dravidian languages. The results of both the ML and DL models are analyzed empirically with performance measures like accuracy, precision, recall, and f1-score. The accuracy associated with different machine learning algorithms varies from 85% to 89%. It is evident from the experimental result that the deep learning model outperformed with an accuracy of 98%. Novelty: The proposed system emphasizes on the use of the language agnostic model to implement the process of detecting Dravidian languages associated with the given text which provides a promising result of 98% accuracy which is higher than the existing methodologies. Keywords: Language, Machine learning, Deep learning, Transformer model, Encoder, Decoder

Non-diagnostic ultrasound (US) of the gallbladder may be due to various factors. We aimed to evaluate the diagnostic performance of deep learning-based classification of gallbladder lesions on US images in patients with non-diagnostic US secondary to gallbladder factors. Consecutive patients with non-diagnostic US due to calculi within the gallbladder lumen, obscuring the detailed evaluation, were identified by a research fellow from a prospective database of patients with gallbladder lesions. The US reports and images were evaluated by a radiologist blinded to the final diagnosis. Patients who had the final pathological diagnosis based on fine-needle aspiration cytology, percutaneous or endoscopic biopsy, or surgical histopathology were included. Convolution neural networks (ResNet50, GBCNet), transformer models (vision transformer [ViT], RadFormer), and a hybrid model (MedViT) were trained on a public gallbladder dataset (GBCU dataset). The performance of these models for classifying gallbladder lesions into benign and malignant was tested on non-diagnostic (GB-RADS 0) US images. Training and validation cohorts (GBCU dataset) comprised 1004 and 251 images, respectively. The testing data (26 patients, mean age [SD]: 57.5 ±8.07 years, 17 female) comprised 304 images. The best performance for detection of GBC was achieved with GBCNet (sensitivity 51.1%, specificity 83.3%, area under the curve [AUC] 0.709) and MedViT (sensitivity 92.8%, specificity 50%, AUC 0.714). MedViT had the best accuracy (73.1%) for detecting benign gallbladder lesions. These results suggest that deep learning models can potentially stratify patients with non-diagnostic US. However, further improvement in the performance is needed to render this approach relevant in clinical practice.

Small and Medium-sized Enterprises (SMEs) in Sri Lanka are increasingly turning to social media to gain insights into consumer preferences; however, they frequently lack sophisticated analytical tools to derive actionable insights. This research introduces a hybrid deep learning framework aimed at sentiment and emotion analysis specifically for the Sri Lankan cake sector. Customer reviews and social media comments were gathered and processed through various methods including text cleaning, emoji translation, stopword elimination, tokenization, and sequence padding. Several deep learning models— Convolutional Neural Networks (CNN), Bidirectional Long Short-Term Memory (BiLSTM), and Gated Recurrent Units (GRU) were assessed, along with hybrid models that combined CNN and BiLSTM with GPT-2 embeddings. The CNN+GPT-2 model achieved the highest accuracy in sentiment classification at 96.14%, while BiLSTM performed best in emotion classification with an accuracy of 91.57%. In terms of trend forecasting, Long Short-Term Memory (LSTM) outshone both SARIMA and Prophet, yielding the lowest error metrics and the most reliable sentiment predictions. These results underscore the potential of hybrid deep learning and transformer-based models to effectively capture complex consumer sentiments and provide SMEs with practical, AI-driven insights. This study is among the first to implement such hybrid NLP models within a localized Sri Lankan context, presenting a scalable approach for business intelligence and informed decision-making.

Hybrid residual deep learning models with physical knowledge for improving plant transpiration estimation

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.

PurposeThe first transmission of coronavirus to humans started in Wuhan city of China, took the shape of a pandemic called Corona Virus Disease 2019 (COVID-19), and posed a principal threat to the entire world. The researchers are trying to inculcate artificial intelligence (Machine learning or deep learning models) for the efficient detection of COVID-19. This research explores all the existing machine learning (ML) or deep learning (DL) models, used for COVID-19 detection which may help the researcher to explore in different directions. The main purpose of this review article is to present a compact overview of the application of artificial intelligence to the research experts, helping them to explore the future scopes of improvement.MethodsThe researchers have used various machine learning, deep learning, and a combination of machine and deep learning models for extracting significant features and classifying various health conditions in COVID-19 patients. For this purpose, the researchers have utilized different image modalities such as CT-Scan, X-Ray, etc. This study has collected over 200 research papers from various repositories like Google Scholar, PubMed, Web of Science, etc. These research papers were passed through various levels of scrutiny and finally, 50 research articles were selected.ResultsIn those listed articles, the ML / DL models showed an accuracy of 99% and above while performing the classification of COVID-19. This study has also presented various clinical applications of various research. This study specifies the importance of various machine and deep learning models in the field of medical diagnosis and research.ConclusionIn conclusion, it is evident that ML/DL models have made significant progress in recent years, but there are still limitations that need to be addressed. Overfitting is one such limitation that can lead to incorrect predictions and overburdening of the models. The research community must continue to work towards finding ways to overcome these limitations and make machine and deep learning models even more effective and efficient. Through this ongoing research and development, we can expect even greater advances in the future.

What's New in Spine Surgery.

Snow depth is a critical parameter for characterizing snow dynamics and water resources, and its accurate inversion is essential for hydrological processes, climate studies, and disaster prevention in cold regions. Based on long-term daily ground meteorological observation data from the hydrological years 1961 to 2015 at two meteorological stations in Mohe and Mishan, Heilongjiang Province, China, this study integrates physical parameters of snow density and snow albedo from the ERA5-Land reanalysis data to systematically compare the performance of three machine learning and three deep learning models in retrieving daily snow depth. Four feature combination schemes were designed to evaluate the contributions of meteorological factors, lagged snow depth terms, and snow physical parameters. The results indicate that, for both machine learning and deep learning models, the first-order lagged value of snow depth is the most important variable determining prediction accuracy. In terms of model performance, machine learning methods excelled, with XGBoost performing particularly outstandingly, achieving optimal prediction accuracy and stability under the best feature combination (coefficient of determination, R2, reaching 0.989; root mean square error, RMSE, of 1.19 cm). Among deep learning methods, 1D CNN demonstrated strong local feature extraction capabilities, achieving prediction accuracy comparable to the best-performing machine learning model (R2 of 0.9878, RMSE of 1.26 cm). Notably, models specifically designed for time-series data, such as LSTM (R2 of 0.9848, RMSE of 1.41 cm), and the more complex 1D CNN-LSTM hybrid model (R2 of 0.9803, RMSE of 1.60 cm) did not show significant advantages in this study. This indicates that model complexity and predictive performance are not simply positively correlated. Through comprehensive analysis of data from both stations, this study demonstrates that a prediction framework centered on historical snow depth as the core driving factor, combined with key meteorological elements, is highly robust. Although the inclusion of ERA5-Land snow physical parameters did not significantly improve model accuracy, it provides important insights for the future development of hybrid models that integrate physical mechanisms with data-driven approaches. The findings offer an effective solution for reconstructing long-term snow depth time series and hold significant application value for simulating cryospheric hydrological processes and studying climate change.

Systematic Review of Machine Learning in Diagnosis of Myeloproliferative Neoplasia

Landslide is a devastating natural disaster, causing loss of life and property. It is likely to occur more frequently due to increasing urbanization, deforestation, and climate change. Landslide susceptibility mapping is vital to safeguard life and property. This article surveys machine learning (ML) models used for landslide susceptibility mapping to understand the current trend by analyzing published articles based on the ML models, landslide causative factors (LCFs), study location, datasets, evaluation methods, and model performance. Existing literature considered in this comprehensive survey is systematically selected using the ROSES protocol. The trend indicates a growing interest in the field. The choice of LCFs depends on data availability and case study location; China is the most studied location, and area under the receiver operating characteristic curve (AUC) is considered the best evaluation metric. Many ML models have achieved an AUC value > 0.90, indicating high reliability of the susceptibility map generated. This paper also discusses the recently developed hybrid, ensemble, and deep learning (DL) models in landslide susceptibility mapping. Generally, hybrid, ensemble, and DL models outperform conventional ML models. Based on the survey, a few recommendations and future works which may help the new researchers in the field are also presented.