Enhanced detection of diabetes mellitus using novel ensemble feature engineering approach and machine learning model

This work presents a system based on an ensemble of Convolutional Neural Networks (CNNs) and descriptors for bioimage classification that has been validated on different datasets of color images. The proposed system represents a very simple yet effective way of boosting the performance of trained CNNs by composing multiple CNNs into an ensemble and combining scores by sum rule. Several types of ensembles are considered, with different CNN topologies along with different learning parameter sets. The proposed system not only exhibits strong discriminative power but also generalizes well over multiple datasets thanks to the combination of multiple descriptors based on different feature types, both learned and handcrafted. Separate classifiers are trained for each descriptor, and the entire set of classifiers is combined by sum rule. Results show that the proposed system obtains state-of-the-art performance across four different bioimage and medical datasets. The MATLAB code of the descriptors will be available at https://github.com/LorisNanni.

While the rapid development of vaccines during the COVID-19 pandemic helped save billions of lives, a significant percentage of the population reported adverse reactions after vaccination. Post-vaccination surveys were conducted to understand the possible side effects. Analyzing and understanding the side effects enables the relevant stakeholders to gain more understanding of the vaccine and allows the individual to make informed decisions about whether to receive the vaccine. This work aims to identify a robust approach that can be used to predict the possibility of potential adverse symptoms. A dataset comprising 840 participants with 15 Post-Vaccination Symptoms was considered for the study. Synthetic Minority Oversampling TEchnique (SMOTE) was used to handle the class imbalance of the dataset. A combination of SMOTE and ensemble machine learning models was used to predict the adverse reactions to COVID-19 vaccines. The ensemble Machine Learning (ML) models that are considered for this study are Random Forest, Extreme Gradient Boosting Machine (XGBoost), Light Gradient Boosting Machine, and AdaBoost. The metrics accuracy, precision, recall and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) were used to measure the performance of the models. The dataset was pre-processed to handle missing values and one-hot encoding was applied to convert categorical variables into the numerical format. Insights into the data distribution and relationships between variables were gained through exploratory data and correlation analysis, respectively. Class imbalance in the target variables was addressed using SMOTE, resulting in a significantly improved F1-score and ROC-AUC score. Among the ensemble ML models, XGBoost delivered the best performance metrics. A combined performance score was calculated by averaging the F1-score and accuracy to identify the best model. XGBoost obtained the highest performance score among the ensemble ML models, and its performance is further enhanced by performing the threshold adjustment using the maximum F1- score strategy. The findings suggest that the combination of SMOTE and ensemble learning models with threshold adjustment provides a more efficient prediction of adverse effects after COVID-19 vaccination, aiding in healthcare decision-making.

Diagnosing Intracranial Hemorrhage (ICH) at an early stage is difficult since it affects the blood vessels in the brain, often resulting in death. We propose an ensemble of Convolutional Neural Networks (CNNs) combining Squeeze and Excitation-based Residual Networks with the next dimension (SE-ResNeXT) and Long Short-Term Memory (LSTM) Networks in order to address this issue. This research work primarily used data from the Radiological Society of North America (RSNA) brain CT hemorrhage challenge dataset and the CQ500 dataset. Preprocessing and data augmentation are performed using the windowing technique in the proposed work. The ICH is then classified using ensembled CNN techniques after being preprocessed, followed by feature extraction in an automatic manner. ICH is classified into the following five types: epidural, intraventricular, subarachnoid, intra-parenchymal, and subdural. A gradient-weighted Class Activation Mapping method (Grad-CAM) is used for identifying the region of interest in an ICH image. A number of performance measures are used to compare the experimental results with various state-of-the-art algorithms. By achieving 99.79% accuracy with an F-score of 0.97, the proposed model proved its efficacy in detecting ICH compared to other deep learning models. The proposed ensembled model can classify epidural, intraventricular, subarachnoid, intra-parenchymal, and subdural hemorrhages with an accuracy of 99.89%, 99.65%, 98%, 99.75%, and 99.88%. Simulation results indicate that the suggested approach can categorize a variety of intracranial bleeding types. By implementing the ensemble deep learning technique using the SE-ResNeXT and LSTM models, we achieved significant classification accuracy and AUC scores.

Background and Purpose:Accurate automated infarct segmentation is needed for acute ischemic stroke studies relying on infarct volumes as an imaging phenotype or biomarker that require large numbers of subjects. This study investigates whether an ensemble of convolutional neural networks (CNN) trained on multiparametric DWI maps outperforms single networks trained on solo DWI parametric maps.Materials and Methods:CNNs were trained on combinations of DWI, ADC, and low b-value-weighted images from 116 subjects. The performances of the networks (measured by Dice score, sensitivity and precision) were compared to one another and to ensembles of 5 networks. To assess the generalizability of the approach, the best performing model was applied to an independent evaluation cohort of 151 subjects. Agreement between manual and automated segmentations for identifying patients with large lesions volumes was calculated across multiple thresholds (21 cm3, 31 cm3, 51 cm3, and 70 cm3).ResultsAn ensemble of CNNs trained on DWI, ADC and low b-value-weighted images produced the most accurate acute infarct segmentation over individual networks (p<0.0001). Automated volumes correlated with manually measured volumes (Spearman’s ρ=0.91, p<0.0001) for the independent cohort. For the task of identifying patients with large lesion volumes, agreement between manual outlines and automated outlines was high (Cohen’s κ 0.86 to 0.90, p<0.0001).ConclusionAcute infarcts are more accurately segmented using ensembles of CNNs trained with multi-parametric maps than using a single model trained with a solo map. Automated lesion segmentation can perform with high agreement with manual techniques for identifying patients with large lesion volumes.

Big data and its production, management, and utilization are essential components in smart city planning. This paper presents a research framework for applying machine learning and deep learning using multiple big data sets on real estate. We built ensemble machine learning models to track property ownership variance in Austin, TX, USA. Then, the study employed the long short term memory (LSTM) model as a deep learning approach to forecasting property value in the same area. For model validation, root mean squared errors were calculated in both models. To avoid underfitting or overfitting of LSTM, we experimented with specific parameters settings. Bagging-based random forest machine learning model outperformed other ensemble machine learning models. Regarding property ownership variance, the Random Forest model’s highest feature importance generally comprised the race, age of residents, land use and built environment factors, number of schools, and neighborhood location. Our LSTM model predicted Austin to retain a rising curve in housing prices and identified which part of Austin experiences an increase or decrease in property value. The predictive models may help city planners to quantify and gain insights on future impacts of developing neighborhoods.

Handwritten digit recognition, a task in computer vision, is critical for applications such as postal automation, banking, and digitization of forms. Traditional approaches have leveraged statistical models, but the rise of deep learning, particularly Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM), has revolutionized the field. However, a comprehensive performance comparison of CNN and LSTM architectures in the context of handwritten digit classification remains underexplored. This study aimed to address this gap by evaluating and comparing CNN and LSTM models for the classification of handwritten digits. Two machine learning models – Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) were trained with the preprocessed handwritten digit dataset. The CNN model was designed with multiple convolutional and pooling layers, along with dropout for regularization. The LSTM model was designed with LSTM layers to capture sequential patterns in the data, followed by a dense layer for classification. The models were implemented in python, evaluated and compared based on accuracy, precision, recall and F1-score. The evaluation and comparison results indicate that CNN achieved 99.31% accuracy, 99.0% precision, 99.0% recall, and a 99.0% F1-score, while LSTM achieved 98.90% accuracy, 99.0% precision, 99.0% recall, and a 99.0% F1-score. The results demonstrated that CNN outperformed LSTM in terms of accuracy and misclassification errors, making it the optimal choice for image-based handwritten digit recognition. This finding underscores the efficiency of CNN in addressing challenges related to digit recognition, contributing to the advancement of automated digit classification systems and improving the accuracy of image-based classification tasks.

Long-term load forecasting is required for power system planning, long-term investments, and expansion policies. In this paper, an ensemble of Long Short Term Memory (LSTM) and Convolutional Neural Network (CNN) is proposed that minimizes the forecasting error for the long-term. The features are extracted from the decomposition of load series using Variational Mode Decomposition (VMD) into various timescale subseries' and are combined with the exogenous variables like temperature and rain. This data is then used to train a Fully Connected Neural Network (FCNN) that produces the final output. This ensemble is expected to provide an accurate forecast as local variations in the series are captured by CNN and long-term dependencies by LSTM, while the VMD decomposes the raw data into low and high-frequency subsequences. The efficacy of the methodology is assessed on the actual load data of Madhya Pradesh (MP) State, India. It is observed that the proposed method has higher forecasting accuracy.

Ensemble-based systems are well known to have the capacity to outperform individual approaches if the ensemble members are sufficiently accurate and diverse. This paper investigates how an efficient ensemble of deep convolutional neural networks (CNNs) can be created by forcing them to adjust their parameters during the training process to increase diversity in their decisions. As a new theoretical approach to reach this aim, we join the member neural architectures via a fully connected layer and insert a new correlation penalty term in the loss function to obstruct their similar operation. With this complementary term, we implement the standard guideline of ensemble creation to increase the members’ diversity for CNNs in a more detailed and flexible way than similar existing techniques. As for applicability, we show that our approach can be efficiently used in various classification tasks. More specifically, we demonstrate its performance in challenging medical image analysis and natural image classification problems. Besides the theoretical considerations and foundations, our experimental findings suggest that the proposed technique is competitive. Namely, on the one hand, the classification rate of the ensemble trained in this way outperformed all the individual accuracies of the state-of-the-art member CNNs according to the standard error functions of these application domains. On the other hand, it is also validated that the ensemble members get more diverse and their accuracies are raised by adding the penalization term. Moreover, we performed a full comparative analysis, including other state-of-the-art ensemble-based approaches recommended for the same classification tasks. This comparative study also confirmed the superiority of our method, as it overcame the current solutions.

Coupling convolutional neural networks with gated recurrent units to model illuminance distribution from light pipe systems

Sequence Classification is one of the on-demand research projects in the field of Natural Language Processing (NLP). Classifying a set of images or text into an appropriate category or class is a complex task that a lot of Machine Learning (ML) models fail to accomplish accurately and end up under-fitting the given dataset. Some of the ML algorithms used in text classification are KNN, Naïve Bayes, Support Vector Machines, Convolutional Neural Networks (CNNs), Recursive CNNs, Recurrent Neural Networks (RNNs), Long Short Term Memory (LSTM), etc. For this experimental study, LSTM and a few other algorithms were chosen for a more comparative study. The dataset used is the SMS Spam Collection Dataset from Kaggle and 150 more entries were additionally added from different sources. Two possible class labels for the data points are spam and ham. Each entry consists of the class label, a few sentences of text followed by a few useless features that are eliminated. After converting the text to the required format, the models are run and then evaluated using various metrics. In experimental studies, the LSTM gives much better classification accuracy than the other machine learning models. F1-Scores in the high nineties were achieved using LSTM for classifying the text. The other models showed very low F1-Scores and Cosine Similarities indicating that they had underperformed on the dataset. Another interesting observation is that the LSTM had reduced the number of false positives and false negatives than any other model.

Model optimization is a problem of great concern and challenge for developing an image classification model. In image classification, selecting the appropriate hyperparameters can substantially boost the model’s ability to learn intricate patterns and features from complex image data. Hyperparameter optimization helps to prevent overfitting by finding the right balance between complexity and generalization of a model. The ensemble genetic algorithm and convolutional neural network (EGACNN) are proposed to enhance image classification by fine-tuning hyperparameters. The convolutional neural network (CNN) model is combined with a genetic algorithm GA) using stacking based on the Modified National Institute of Standards and Technology (MNIST) dataset to enhance efficiency and prediction rate on image classification. The GA optimizes the number of layers, kernel size, learning rates, dropout rates, and batch sizes of the CNN model to improve the accuracy and performance of the model. The objective of this research is to improve the CNN-based image classification system by utilizing the advantages of ensemble learning and GA. The highest accuracy is obtained using the proposed EGACNN model which is 99.91% and the ensemble CNN and spiking neural network (CSNN) model shows an accuracy of 99.68%. The ensemble approaches like EGACNN and CSNN tends to be more effective as compared to CNN, RNN, AlexNet, ResNet, and VGG models. The hyperparameter optimization of deep learning classification models reduces human efforts and produces better prediction results. Performance comparison with existing approaches also shows the superior performance of the proposed model.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.

The security of information is among the greatest challenges facing organizations and institutions. Cybercrime has risen in frequency and magnitude in recent years, with new ways to steal, change and destroy information or disable information systems appearing every day. Among the types of penetration into the information systems where confidential information is processed is malware. An attacker injects malware into a computer system, after which he has full or partial access to critical information in the information system. This paper proposes an ensemble classification-based methodology for malware detection. The first-stage classification is performed by a stacked ensemble of dense (fully connected) and convolutional neural networks (CNN), while the final stage classification is performed by a meta-learner. For a meta-learner, we explore and compare 14 classifiers. For a baseline comparison, 13 machine learning methods are used: K-Nearest Neighbors, Linear Support Vector Machine (SVM), Radial basis function (RBF) SVM, Random Forest, AdaBoost, Decision Tree, ExtraTrees, Linear Discriminant Analysis, Logistic, Neural Net, Passive Classifier, Ridge Classifier and Stochastic Gradient Descent classifier. We present the results of experiments performed on the Classification of Malware with PE headers (ClaMP) dataset. The best performance is achieved by an ensemble of five dense and CNN neural networks, and the ExtraTrees classifier as a meta-learner.

Classification method for nailfold capillary images using an optimized sugeno fuzzy ensemble of convolutional neural networks.