Attention Based Encoder Architecture for Automatic Text Classification: A Case Study on Text-news Classification

Background and aimsThe birth weight of a newborn is a crucial factor that affects their overall health and future well-being. Low birth weight (LBW) is a widespread global issue, which the World Health Organization defines as weighing less than 2,500 g. LBW can have severe negative consequences on an individual’s health, including neonatal mortality and various health concerns throughout their life. To address this problem, this study has been conducted using BDHS 2017–2018 data to uncover important aspects of LBW using a variety of machine learning (ML) approaches and to determine the best feature selection technique and best predictive ML model.MethodsTo pick out the key features, the Boruta algorithm and wrapper method were used. Logistic Regression (LR) used as traditional method and several machine learning classifiers were then used, including, DT (Decision Tree), SVM (Support Vector Machine), NB (Naïve Bayes), RF (Random Forest), XGBoost (eXtreme Gradient Boosting), and AdaBoost (Adaptive Boosting), to determine the best model for predicting LBW. The model’s performance was evaluated based on the specificity, sensitivity, accuracy, F1 score and AUC value.ResultsResult shows, Boruta algorithm identifies eleven significant features including respondent’s age, highest education level, educational attainment, wealth index, age at first birth, weight, height, BMI, age at first sexual intercourse, birth order number, and whether the child is a twin. Incorporating Boruta algorithm’s significant features, the performance of traditional LR and ML methods including DT, SVM, NB, RF, XGBoost, and AB were evaluated where LR, had a specificity, sensitivity, accuracy and F1 score of 0.85, 0.5, 85.15% and 0.915. While the ML methods DT, SVM, NB, RF, XGBoost, and AB model’s respective accuracy values were 85.35%, 85.15%, 84.54%, 81.18%, and 84.41%. Based on the specificity, sensitivity, accuracy, F1 score and AUC, RF (specificity = 0.99, sensitivity = 0.58, accuracy = 85.86%, F1 score = 0.9243, AUC = 0.549) outperformed the other methods. Both the classical (LR) and machine learning (ML) models’ performance has improved dramatically when important characteristics are extracted using the wrapper method. The LR method identified five significant features with a specificity, sensitivity, accuracy and F1 score of 0.87, 0.33, 87.12% and 0.9309. The region, whether the infant is a twin, and cesarean delivery were the three key features discovered by the DT and RF models, which were implemented using the wrapper technique. All three models had the identical F1 score of 0.9318. However, “child is twin” was recognized as a significant feature by the SVM, NB, and AB models, with an F1 score of 0.9315. Ultimately, with an F1 score of 0.9315, the XGBoost model recognized “child is twin” and “age at first sex” as relevant features. Random Forest again beat the other approaches in this instance.ConclusionsThe study reveals Wrapper method as the optimal feature selection technique. The ML method outperforms traditional methods, with Random Forest (RF) being the most effective predictive model for Low-Birth-Weight prediction. The study suggests that policymakers in Bangladesh can mitigate low birth weight newborns by considering identified risk factors.

Detection of breast cancer remains an important global health issue, particularly in patients with dense breast tissue. Despite wide use of machine learning for breast cancer diagnosis, many existing studies rely on single-model evaluations and lack rigorous statistical validation, while transformer-based models remain insufficiently explored for structured tabular biomedical data. Using the Wisconsin Breast Cancer Diagnostic (WBCD) dataset, this study presents a comparative analysis of machine learning, deep learning, and transformer-based models for binary classification of breast masses as benign or malignant. A robust preprocessing and feature engineering pipeline was applied, and Support Vector Machine (SVM), Random Forest (RF), Multi-Layer Perceptron (MLP), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and BERT-based models were evaluated using a stratified 66–34% train–test split and stratified 5-fold cross-validation. Performance was assessed using standard classification metrics, Jaccard Index, AUC-ROC, false negative rate, and statistical significance testing (paired t-tests and Bayesian analysis). Three-layer RNN and LSTM models achieved the highest accuracy of 98.25%, showing statistically significant improvement over SVM (p = 0.042), while the optimized SVM demonstrated strong performance with superior computational efficiency. BERT-based models yielded lower accuracy (92.98%), reflecting domain mismatch between language-pretrained transformers and numerical tabular data. These findings indicate that deeper recurrent architectures provide superior diagnostic performance, while SVM remains a practical and efficient alternative for clinical deployment.

Industrial robots are widely used in critical tasks such as assembly, welding, and material handling as core components of modern manufacturing systems. For the reliable operation of these systems, early and accurate detection of execution failures is crucial. In this study, a comprehensive comparison of machine learning and deep learning methods is conducted for the classification of robot execution failures using data acquired from force-torque sensors. Three different feature engineering approaches are proposed. The first is a Baseline approach that includes 90 raw time-series features. The second is the Domain-6 approach, which consists of 6 basic statistical features per sensor (36 in total). The third is the Domain-12 approach, which comprises 12 comprehensive statistical features per sensor (72 in total). The domain features include the mean, standard deviation, minimum, maximum, range, slope, median, skewness, kurtosis, RMS, energy, and IQR. In total, ten classification algorithms are evaluated, including eight machine learning methods and two deep learning models: Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (KNN), Artificial Neural Network (ANN), Naive Bayes (NB), Decision Trees (DT), eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM-LGBM), as well as a One-Dimensional Convolutional Neural Network (CNN-1D) and Long Short-Term Memory (LSTM). For traditional machine learning algorithms, 5 × 5 nested cross-validation is used, whereas for deep learning models, 5-fold cross-validation with a 20% validation split is employed. To ensure statistical reliability, all experiments are repeated over 30 independent runs. The experimental results demonstrate that feature engineering has a decisive impact on classification performance. In addition, regardless of the feature set, the highest accuracy (93.85% ± 0.90) is achieved by the Naive Bayes classifier using the Baseline features. The Domain-12 feature set provides consistent improvements across many algorithms, with substantial performance gains. The results are reported using accuracy, precision, recall, and F1-score metrics and are supported by confusion matrices. Finally, permutation feature importance analysis indicates that the skewness features of the Fx and Fy sensors are the most critical variables for failure detection. Overall, these findings show that time-domain statistical features offer an effective approach for robot failure classification.

Fall prevention has always been a crucial topic for injury prevention. Research shows that real-time posture monitoring and subsequent fall prevention are important for the prevention of fall-related injuries. In this research, we determine a real-time posture classifier by comparing classical and deep machine learning classifiers in terms of their accuracy and robustness for posture classification. For this, multiple classical classifiers, including classical machine learning, support vector machine, random forest, neural network, and Adaboost methods, were used. Deep learning methods, including LSTM and transformer, were used for posture classification. In the experiment, joint data were obtained using an RGBD camera. The results show that classical machine learning posture classifier accuracy was between 75% and 99%, demonstrating that the use of classical machine learning classification alone is sufficient for real-time posture classification even with missing joints or added noise. The deep learning method LSTM was also effective in classifying the postures with high accuracy, despite incurring a significant computational overhead cost, thus compromising the real-time posture classification performance. The research thus shows that classical machine learning methods are worthy of our attention, at least, to consider for reuse or reinvention, especially for real-time posture classification tasks. The insight of using a classical posture classifier for large-scale human posture classification is also given through this research.

Objectives: Language detection is the process of identifying a language associated with a text. The proposed system aims to detect the Dravidian language that is associated with the given text using different machine learning and deep learning algorithms. The paper presents an empirical analysis of the results obtained using the different models. It also aims to evaluate the performance of a language agnostic model for the purpose of language detection. Method: An empirical analysis of Dravidian language identification in social media text using machine learning and deep learning approaches with k-fold cross validation has been implemented. The identification of Dravidian languages, including Tamil, Malayalam, Tamil Code Mix, and Malayalam Code Mix, is performed using both machine learning (ML) and deep learning algorithms. The machine learning algorithms used for language detection are Naive Bayes (NB), Multinomial Logistic Regression (MLR), Support Vector Machine (SVM), and Random Forest (RF). The supervised Deep Learning (DL) models used include BERT, mBERT and language agnostic models. Findings: The language agnostic model outperform all other models considering the task of language detection in Dravidian languages. The results of both the ML and DL models are analyzed empirically with performance measures like accuracy, precision, recall, and f1-score. The accuracy associated with different machine learning algorithms varies from 85% to 89%. It is evident from the experimental result that the deep learning model outperformed with an accuracy of 98%. Novelty: The proposed system emphasizes on the use of the language agnostic model to implement the process of detecting Dravidian languages associated with the given text which provides a promising result of 98% accuracy which is higher than the existing methodologies. Keywords: Language, Machine learning, Deep learning, Transformer model, Encoder, Decoder

In recent years, COVID-19 has been regarded as the most dangerous pandemic for several countries. On various social media platforms, such as Twitter, Facebook, and Instagram, a variety of rumours, hypes, and news are published. This might have a detrimental impact on people’s life. As a result, social media platforms have always had a difficult time authenticating this fake information. Different machine learning (ML) and deep learning (DL) classifiers were used in this work to categorize the continuing impacts of tweets and forecast their after-effects. Support vector machine (SVM), random forest (RF), decision tree (DT), and k-nearest neighbor (KNN) were used for classification, while AdaBoost and convolutional neural network (CNN) were utilized for future effects. The tweets dataset from Kaggle was used to train the SVM, RF, KNN, and DT models, which were then assessed on multiple evaluation criteria such as accuracy, precision, recall, and F1-score, using a 70 : 30 ratio. The CNN and AdaBoost, on the other hand, have been taught to detect the mean square error, root mean square error, and mean absolute error. With 0.74 and 0.73 percent score out of 1, respectively, RF and SVM exhibit the best accuracy in impact when classifying the outcomes on the obtained dataset. In terms of a regression problem, CNN beat the ADA Regressor across the board.

Detection of COVID-19 using deep learning techniques and classification methods

Artificial intelligence (AI)-based approaches are crucial in computer-aided diagnosis (CAD) for various medical applications. Their ability to quickly and accurately learn from complex data is remarkable. Deep learning (DL) models have shown promising results in accurately classifying Alzheimer's disease (AD) and its related cognitive states, Early Mild Cognitive Impairment (EMCI) and Late Mild Cognitive Impairment (LMCI), along with the healthy conditions known as Cognitively Normal (CN). This offers valuable insights into disease progression and diagnosis. However, certain traditional machine learning (ML) classifiers perform equally well or even better than DL models, requiring less training data. This is particularly valuable in CAD in situations with limited labeled datasets. In this paper, we propose an ensemble classifier based on ML models for magnetic resonance imaging (MRI) data, which achieved an impressive accuracy of 96.52%. This represents a 3-5% improvement over the best individual classifier. We evaluated popular ML classifiers for AD classification under both data-scarce and data-rich conditions using the Alzheimer's Disease Neuroimaging Initiative and Open Access Series of Imaging Studies datasets. By comparing the results to state-of-the-art CNN-centric DL algorithms, we gain insights into the strengths and weaknesses of each approach. This work will help users to select the most suitable algorithm for AD classification based on data availability.

Objective: Using methylation characteristics of human genes to construct machine learning predictive models for screening cervical cancer and precancerous lesions. Methods: Human DNA methylation detection was performed on 224 cervical exfoliated cell specimens from the Cancer Hospital of the Chinese Academy of Medical Sciences, Tianjin Central Hospital of Gynecology Obstetrics, Xinmi Maternal and Child Health Hospital of Henan Province, West China Second Affiliated Hospital of Sichuan University, and Heping Hospital Affiliated to Changzhi Medical College collected during April 2014 and March 2015. The hypermethylated gene fragments related to cervical cancer were selected by high-density, high-association, and hypermethylated gene fragment screening and the LASSO regression algorithm. Taking cervical intraepithelial neoplasia grade 2 (CIN2) or more severe lesions as the research outcome, machine learning predictive models based on the random forest (RF), naive Bayes (NB), and support vector machine (SVM) algorithm, respectively, were constructed. A total of 144 outpatient specimens were used as the training set and 80 cervical exfoliated cell specimens from women participating in the cervical cancer screening program were used as the test set to verify the predictive models. Using histological diagnosis results as the gold standard, the detection efficacy for CIN2 or more severe lesions of the three machine learning predictive models were compared with that of the human papilloma virus (HPV) detection and cytological diagnosis. Results: In the training set of 144 cases, there were 34 cases of HPV positivity, with a positive rate of 23.61%. Cytologically, there were 37 cases diagnosed as no intraepithelial lesion or malignancy (NILM), and 107 cases diagnosed as atypical squamous cells of undetermined significance (ASC-US) or above. Histologically, there were 28 cases without cervical intraepithelial neoplasia or benign cervical lesions, 31 cases of CIN1, 18 cases of CIN2, 31 cases of CIN3, and 36 cases of squamous cell carcinoma. Seven hypermethylated gene fragments were selected from 45 genes, and three machine learning prediction models based on the RF, NB, and SVM algorithm, respectively, were constructed. In the validation set of 80 cases, there were 28 cases of HPV positivity, with a positive rate of 35.00%. Cytologically, there were 65 cases diagnosed as NILM and 15 cases as ASC-US or above. Histologically, there were 39 cases without cervical intraepithelial neoplasia or benign cervical lesions, 10 cases of CIN1, 10 cases of CIN2, 11 cases of CIN3, and 10 cases of squamous cell carcinoma. In the validation set, the area under the curve (AUC) values of the RF model, NB model, SVM model, HPV detection, and cytological diagnosis of CIN2 or above were 0.90, 0.88, 0.82, 0.68, and 0.45, respectively. The DeLong test showed that there was no statistically significant difference in the AUC values between the RF, NB, and SVM models (all P＞0.05), and the AUC values of the RF and NB models were higher than that of HPV detection (both P＜0.01), and the AUC values of the RF, NB, and SVM models were higher than that of cytological diagnosis (all P＜0.01). Compared with the NB model, the sensitivity of the RF model was similar (80.65% vs. 77.42%), but the specificity of the NB model was much higher than that of the RF model (93.88% vs. 73.47%). Conclusion: Among the machine learning prediction models for cervical cancer and precancerous lesions constructed based on human DNA methylation, the NB model has good predictive performance for CIN2 and above lesions, and may be used for screening of cervical cancer and precancerous lesions.

Researchers have extensively employed machine learning (ML) approaches to establish forecasting systems in numerous domains like software faults, spam identification, disease foresting, and fraud recognition. The detection of individuals susceptible to heart disease (HD) could assist medical practitioners in their verdict concerning treatments and management. This study, therefore, is projected to examine the discriminative power of numerous prognosticators in the research to intensify the efficiency of HD discovery through their warning signs. A dimensionality reduction method called principal component analysis (PCA) and three machine learning (ML) classifiers which are Naïve Bayes (NB), random forest (RF), and support vector machine (SVM) are employed in the study. The three ML classifiers were implemented with PCA DR. The techniques’ performance was evaluated on a dataset attained from National Health Service (NHS) database. The system was evaluated using confusion matrix measures for instance, accuracy, precision, specificity, f1-score, detection rate, and false-positive rate (FPR). A comparative analysis was conducted between the three ML classifiers with PCA DR. It was concluded that the classifiers.KeywordsMachine learningPCASupport vector machineRandom forestNaïve Bayes

Machine Learning Techniques for Breast Cancer Prediction

Gesture recognition is a topic in computer science and language technology that aims to interpret human gestures with computer programs and many different algorithms. It can be seen as the way computers can understand human body language. Today, the main interaction tools between computers and humans are still the keyboard and mouse. Gesture recognition can be used as a tool for communication with the machine and interaction without any mechanical device such as a keyboard or mouse. In this paper, we present the results of a comparison of eight different machine learning (ML) classifiers in the task of human hand gesture recognition and classification to explore how to efficiently implement one or more tested ML algorithms on an 8-bit AVR microcontroller for on-line human gesture recognition with the intention to gesturally control the mobile robot. The 8-bit AVR microcontrollers are still widely used in the industry, but due to their lack of computational power and limited memory, it is a challenging task to efficiently implement ML algorithms on them for on-line classification. Gestures were recorded by using inertial sensors, gyroscopes, and accelerometers placed at the wrist and index finger. One thousand and eight hundred (1800) hand gestures were recorded and labelled. Six important features were defined for the identification of nine different hand gestures using eight different machine learning classifiers: Decision Tree (DT), Random Forests (RF), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Support Vector Machine (SVM) with linear kernel, Naïve Bayes classifier (NB), K-Nearest Neighbours (KNN), and Stochastic Gradient Descent (SGD). All tested algorithms were ranged according to Precision, Recall, and F1-score (abb.: P-R-F1). The best algorithms were SVM (P-R-F1: 0.9865, 0.9861, and 0.0863), and RF (P-R-F1: 0.9863, 0.9861, and 0.0862), but their main disadvantage is their unusability for on-line implementations in 8-bit AVR microcontrollers, as proven in the paper. The next best algorithms have had only slightly poorer performance than SVM and RF: KNN (P-R-F1: 0.9835, 0.9833, and 0.9834) and LR (P-R-F1: 0.9810, 0.9810, and 0.9810). Regarding the implementation on 8-bit microcontrollers, KNN has proven to be inadequate, like SVM and RF. However, the analysis for LR has proved that this classifier could be efficiently implemented on targeted microcontrollers. Having in mind its high F1-score (comparable to SVM, RF, and KNN), this leads to the conclusion that the LR is the most suitable classifier among tested for on-line applications in resource-constrained environments, such as embedded devices based on 8-bit AVR microcontrollers, due to its lower computational complexity in comparison with other tested algorithms.

Heating, ventilation, and air-conditioning (HVAC) systems are among the most energy-intensive components in commercial buildings. Accurate forecasting of HVAC energy consumption is essential for implementing energy-efficient control strategies. This study provides a detailed comparative analysis of classical machine learning (ML) and deep learning (DL) models for forecasting HVAC energy consumption using both predicted and actual indoor temperature data. Leveraging an open-source, high-resolution dataset from the Oak Ridge National Laboratory's FRP-2 multizone building, we implement a two-stage framework. The first stage involves temperature prediction using relative humidity and airflow data, while the second stage forecasts HVAC energy consumption using either predicted or actual temperature inputs. We evaluate Linear Regression, Random Forest, Support Vector Machine, GRU, and LSTM models. The results reveal that while models with actual temperature inputs yield superior accuracy (R² up to 0.884), models using predicted temperatures still achieve competitive performance. Our findings suggest that ML and DL models, particularly LSTM, offer promising capabilities for real-time energy forecasting in smart building environments.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

The triage of patients in the Emergency Department (ED) plays a critical role in determining the urgency and type of treatment that shall be administered. Therefore, an accurate prediction system for patient triage can be very helpful. This study aims to develop a Machine Learning (ML) classification model to predict triage decisions for patients admitted to the ED at UNS Hospital. Several classification models were evaluated, including Naïve Bayes (NB), Decision Trees (DT), Random Forest (RF), Support Vector Machines (SVM), and k-Nearest Neighbors (k-NN). To assess model performance, multiple metrics were employed: accuracy, precision, recall, F1-score, and the confusion matrix. Following an initial evaluation, hyperparameter tuning was conducted on the selected best-performing model to identify the optimal combination of parameters for improved predictive performance. RF emerged as the best-performing model for triage prediction. The parameters tuned included the number of estimators, criterion, maximum features, and maximum depth, using a 5-fold cross-validation strategy. The optimal parameter values were found to be 900 estimators, 'entropy' for the criterion, 'log2' for max features, and a maximum depth of 40. The results of this study demonstrate that hyperparameter tuning can significantly enhance model performance, reducing recall errors, improving the F1-score, and decreasing the number of mispredictions.