Deep Learning and Classical Machine Learning for code mapping in Heterogeneous Platforms

In online courses, discussion forums play a key role in enhancing student interaction with peers and instructors. Due to large enrolment sizes, instructors often struggle to respond to students in a timely manner. To address this problem, both traditional Machine Learning (ML) (e.g., Random Forest) and Deep Learning (DL) approaches have been applied to classify educational forum posts (e.g., those required urgent responses vs. that did not). However, there lacks an in-depth comparison between these two kinds of approaches. To better guide people to select an appropriate model, we aimed at providing a comparative study on the effectiveness of six frequently-used traditional ML and DL models across a total of seven different classification tasks centering around two datasets of educational forum posts. Through extensive evaluation, we showed that (i) the up-to-date DL approaches did not necessarily outperform traditional ML approaches; (ii) the performance gap between the two kinds of approaches can be up to 3.68% (measured in F1 score); (iii) the traditional ML approaches should be equipped with carefully-designed features, especially those of common importance across different classification tasks. Based on the derived findings, we further provided insights to help instructors and educators construct effective classifiers for characterizing educational forum discussions, which, ultimately, would enable them to provide students with timely and personalized learning support.

When assessing kidney biopsies, pathologists use light microscopy, immunofluorescence, and electron microscopy to describe and diagnose glomerular lesions and diseases. These methods can be laborious, costly, fraught with inter-observer variability, and can have delays in turn-around time. Thus, computational approaches can be designed as screening and/or diagnostic tools, potentially relieving pathologist time, healthcare resources, while also having the ability to identify novel biomarkers, including subvisual features. Here, we implement our recently published biomarker feature extraction (BFE) model along with 3 pre-trained deep learning models (VGG16, VGG19, and InceptionV3) to diagnose 3 glomerular diseases using PAS-stained digital pathology images alone. The BFE model extracts a panel of 233 explainable features related to underlying pathology, which are subsequently narrowed down to 10 morphological and microstructural texture features for classification with a linear discriminant analysis machine learning classifier. 45 patient renal biopsies (371 glomeruli) from minimal change disease (MCD), membranous nephropathy (MN), and thin-basement membrane nephropathy (TBMN) were split into training/validation and held out sets. For the 3 deep learningmodels, data augmentation and Grad-CAM were used for better performance and interpretability. The BFE model showed glomerular validation accuracy of 67.6% and testing accuracy of 76.8%. All deep learning approaches had higher validation accuracies (most for VGG16 at 78.5%) but lower testing accuracies. The highest testing accuracy at the glomerular level was VGG16 at 71.9%, while at the patient-level was InceptionV3 at 73.3%. The results highlight the potential of both traditional machine learning and deep learning-based approaches for kidney biopsy evaluation.

Small data, big challenges: Machine- and deep-learning strategies for data-limited drug discovery.

Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together.

The proliferation of Internet of Things (IoT) devices has created unprecedented cybersecurity vulnerabilities, with botnets emerging as a critical threat to network infrastructure. This study focuses on traditional machine learning and deep learning approaches, proposes a novel ensemble framework to address these issues, integrating Convolutional Neural Network (CNN), Bidirectional Long Short-Term Memory (BiLSTM), Random Forest (RF), and Logistic Regression (LR) via a weighted soft-voting mechanism. Our approach introduces a Quantile Uniform transformation to reduce feature skewness, a multi-layered feature selection method to enhance discriminative power, an individual performance of deep learning-traditional machine learning and a hybrid models (ensemble models) for robust detection. Evaluated on BOT-IOT, CICIOT2023, and IOT23 datasets, the framework achieves 100% accuracy on BOT-IOT, 99.2% on CICIOT2023, and 91.5% on IOT23, outperforming state-of-the-art models by up to 6.2%. These contributions advance IoT security by enabling scalable, high-performance detection adaptable to diverse network scenarios, with practical optimizations for real-world deployment.

This study investigates the feasibility of using tear sample analysis, based on protein corona formation on gold nanoparticles combined with electrospray ionization mass spectrometry (ESI-MS) and machine learning techniques, as a non-invasive approach for the detection of choroidal melanoma. The aim is to assess whether protein-nanoparticle interactions can support early and reliable identification of this ocular condition. Tear samples were collected using Schirmer strips from six healthy individuals and six patients diagnosed with choroidal melanoma, with subsequent augmentation to 18 samples per group. Gold nanoparticles (AuNPs, ~ 20 nm) were synthesized via citrate reduction and incubated with tear samples to form protein coronas, which were analyzed using ESI-MS. Eight statistical and entropy-based features (mean, variance, skewness, kurtosis, Shannon entropy, approximate entropy, sample entropy, and permutation entropy) were extracted from spectral data. Additionally, Continuous Wavelet Transform (CWT) with Mexican hat wavelet was applied to convert mass spectrometry data into 128 × 128 RGB images for deep learning analysis. Classification was performed using traditional machine learning models (Random Forest, Support Vector Machine, Decision Tree, Deep Neural Network) and transfer learning with pre-trained CNNs (VGG16, ResNet50, Xception), evaluated through 5-fold cross-validation. Significant differences in spectral intensity parameters were observed between healthy individuals and choroidal melanoma patients (p < 0.001), with notably lower Mean_Intensity values in cancer patients (56.41 ± 46.06 vs. 111.02 ± 10.01, Cohen’s d = 1.64). While m/z parameters showed moderate differences that didn’t reach statistical significance (p = 0.082), entropy-based features demonstrated strong discriminative power. Among traditional machine learning models, Random Forest achieved the highest accuracy (0.959 ± 0.003) and ROC AUC (0.993 ± 0.000) with remarkable computational efficiency (3.90 s per fold). For deep learning approaches using CWT-generated images, VGG16 demonstrated superior performance (Accuracy: 0.976 ± 0.008, ROC AUC: 0.997 ± 0.002) despite requiring significantly higher computational resources (1349.52 s per fold). This study demonstrates that tear sample analysis using protein corona formation on gold nanoparticles with ESI-MS and advanced machine learning techniques offers a promising non-invasive approach for choroidal melanoma detection with performance metrics that compare favorably to existing methods. The significant differences in spectral intensity parameters between groups suggest distinctive proteomic signatures that can be leveraged for diagnostic purposes. While both traditional machine learning and deep learning approaches achieved exceptional performance, each offers distinct advantages in terms of computational efficiency and feature extraction capabilities.

Backgrounds. Virtual reality (VR) simulates real-life events and scenarios and is widely utilized in education, entertainment, and medicine. VR can be presented in two dimensions (2D) or three dimensions (3D), with 3D VR offering a more realistic and immersive experience. Previous research has shown that electroencephalogram (EEG) profiles induced by 3D VR differ from those of 2D VR in various aspects, including brain rhythm power, activation, and functional connectivity. However, studies focused on classifying EEG in 2D and 3D VR contexts remain limited.Methods. A 56-channel EEG was recorded while visual stimuli were presented in 2D and 3D VR. The recorded EEG signals were classified using two machine learning approaches: traditional machine learning and deep learning. In the traditional approach, features such as power spectral density (PSD) and common spatial patterns (CSP) were extracted, and three classifiers-support vector machines (SVM), K-nearest neighbors (KNN), and random forests (RF)-were used. For the deep learning approach, a specialized convolutional neural network, EEGNet, was employed. The classification performance of these methods was then compared.Results. In terms of accuracy, precision, recall, and F1-score, the deep learning method outperformed traditional machine learning approaches. Specifically, the classification accuracy using the EEGNet deep learning model reached up to 97.86%.Conclusions. EEGNet-based deep learning significantly outperforms conventional machine learning methods in classifying EEG signals induced by 2D and 3D VR. Given EEGNet's design for EEG-based brain-computer interfaces (BCI), this superior classification performance suggests that it can enhance the application of 3D VR in BCI systems.

Image cyberbullying detection and recognition using transfer deep machine learning

With the development of information and technology, especially with the boom in big data, healthcare support systems are becoming much better. Patient data can be collected, retrieved, and stored in real time. These data are valuable and meaningful for monitoring, diagnosing, and further applications in data analysis and decision-making. Essentially, the data can be divided into three types, namely, statistical, image-based, and sequential data. Each type has a different method of retrieval, processing, and deployment. Additionally, the application of machine learning (ML) and deep learning (DL) in healthcare support systems is growing more rapidly than ever. Numerous high-performance architectures are proposed to optimize decision-making. As reliability and stability are the most important factors in the healthcare support system, enhancing the predicted performance and maintaining the stability of the model are always the top priority. The main idea of our study comes from ensemble techniques. Numerous studies and data science competitions show that by combining several weak models into one, ensemble models can attain outstanding performance and reliability. We propose three deep ensemble learning (DEL) approaches, each with stable and reliable performance, that are workable on the above-mentioned data types. These are deep-stacked generalization ensemble learning, gradient deep learning boosting, and deep aggregation learning. The experiment results show that our proposed approaches achieve more vigorous and reliable performance than traditional ML and DL techniques on statistical, image-based, and sequential benchmark datasets. In particular, on the Heart Disease UCI dataset, representing the statistical type, the gradient deep learning boosting approach dominates the others with accuracy, recall, F1-score, Matthews correlation coefficient, and area under the curve values of 0.87, 0.81, 0.83, 0.73, and 0.91, respectively. On the X-ray dataset, representing the image-based type, the deep aggregation learning approach shows the highest performance with values of 0.91, 0.97, 0.93, 0.80, and 0.94, respectively. On the Depresjon dataset, representing the sequence type, the deep-stacked generalization ensemble learning approach outperforms the others with values of 0.91, 0.84, 0.86, 0.8, and 0.94, respectively. Overall, we conclude that applying DL models using our proposed approaches is a promising method for the healthcare support system to enhance prediction and diagnosis performance. Furthermore, our study reveals that these approaches are flexible and easy to apply to achieve optimal performance.

Growth in system complexity increases the need for automated techniques dedicated to different log analysis tasks such as Log-based Anomaly Detection (LAD). The latter has been widely addressed in the literature, mostly by means of a variety of deep learning techniques. However, despite their many advantages, that focus on deep learning techniques is somewhat arbitrary as traditional Machine Learning (ML) techniques may perform well in many cases, depending on the context and datasets. In the same vein, semi-supervised techniques deserve the same attention as supervised techniques since the former have clear practical advantages. Further, current evaluations mostly rely on the assessment of detection accuracy. However, this is not enough to decide whether or not a specific ML technique is suitable to address the LAD problem in a given context. Other aspects to consider include training and prediction times as well as the sensitivity to hyperparameter tuning, which in practice matters to engineers. In this paper, we present a comprehensive empirical study, in which we evaluate a wide array of supervised and semi-supervised, traditional and deep ML techniques w.r.t. four evaluation criteria: detection accuracy, time performance, sensitivity of detection accuracy and time performance to hyperparameter tuning. Our goal is to provide much stronger and comprehensive evidence regarding the relative advantages and drawbacks of alternative techniques for LAD. The experimental results show that supervised traditional and deep ML techniques fare similarly in terms of their detection accuracy and prediction time on most of the benchmark datasets considered in our study. Moreover, overall, sensitivity analysis to hyperparameter tuning with respect to detection accuracy shows that supervised traditional ML techniques are less sensitive than deep learning techniques. Further, semi-supervised techniques yield significantly worse detection accuracy than supervised techniques.

Early detection of lung cancer remains a critical challenge in medical imaging, where accurate classification between benign and malignant pulmonary nodules can significantly impact patient outcomes. This study presents a comprehensive comparative analysis of six distinct machine learning approaches for lung cancer classification using 3D computed tomography (CT) data. The methodologies evaluated include 2D convolutional neural networks (CNN), 3D CNN architectures, multi-slice processing techniques, and traditional machine learning algorithms including Random Forest and Logistic Regression. Our experimental framework utilized synthetic CT volumes designed to replicate characteristic patterns of benign and malignant lung lesions. The synthetic dataset comprised 120 volumetric samples with distinct spatial patterns representing clinical features such as nodule morphology, tissue density variations, and anatomical distribution. Results demonstrate that traditional machine learning approaches, particularly Random Forest classifiers, achieved superior performance with accuracy rates exceeding 85%, while deep learning models showed variable performance depending on architecture complexity and data preprocessing strategies. The findings suggest that careful consideration of model architecture selection and data representation is crucial for effective lung cancer classification systems.

Detecting major depressive disorder presence using passively-collected wearable movement data in a nationally-representative sample

Communication-based train control (CBTC) systems are automated train control systems based on continuous and bidirectional train-ground communication. As one of the critical subsystems of the safety-demanding CBTC systems, the Data Communication System (DCS) is physically exposed to the environment. DCS faults may lead to delayed trains, stranded passengers, suspension of service, or even catastrophic losses of lives or assets. Therefore, it is quite desirable to identify the DCS faults and accordingly take effective maintenance measures. Most fault classification works are based on traditional Machine Learning (ML) techniques, such as Decision Tree (DT) and Bayesian Network. A substantial amount of fault information can be lost when we manually extract features from the system logs. This paper proposes a Deep Hybrid Learning (DHL) method to identify the DCS faults. A pre-trained Chinese Bidi-rectional Encoder Representation from Transformers (BERT) Deep Learning (DL) model is used to extract fault features from raw text logs automatically, and several traditional ML models are used to classify the system faults by the extracted features. Our study highlights the advantages of the DHL model in DCS fault diagnosis tasks. The intense simulation results illustrate that the proposed DHL model achieves substantial accuracy compared with traditional ML algorithms.

In this paper, the used methods and the results obtained by our team, entitled Emad, on the OffensEval 2019 shared task organized at SemEval 2019 are presented. The OffensEval shared task includes three sub-tasks namely Offensive language identification, Automatic categorization of offense types and Offense target identification. We participated in sub-task A and tried various methods including traditional machine learning methods, deep learning methods and also a combination of the first two sets of methods. We also proposed a data augmentation method using word embedding to improve the performance of our methods. The results show that the augmentation approach outperforms other methods in terms of macro-f1.

This study compares methodologies for fault classification in reciprocating compressors, focusing on traditional Machine Learning (ML) with classical feature extraction processes and one-dimensional Convolutional Neural Networks (1D-CNN) in Deep Learning (DL). Both techniques demonstrated viability by employing a dataset of compressor vibration signals encompassing ten fault classes. While ML achieved a classification accuracy of 86%, DL reached 90.709%, highlighting its superior learning and generalization abilities, although with longer training times. These findings suggest that, despite ML being effective when relevant prior knowledge is available, DL, particularly with 1D-CNN, offers enhanced fault classification performance for this study case at the expense of additional processing resources.