Evolutionary generation of adversarial examples for deep and shallow machine learning models

Automatic diagnosis based on clinical notes has become a popular research field recently, and many proposed deep learning models have achieved competitive performance in diseases inference. However, previous research reveals that deep learning models are susceptible to negligibly perturbed inputs named adversarial examples, which contradicts with the safety and reliability requirements of the medical domain. To analyze the vulnerability and robustness of current automatic diagnosis models, we investigate in the generation of adversarial text examples. The main challenges for generating adversarial text examples are divided into three parts. First, the word embedding space is discrete, which makes it hard to perturb as small as adversarial image examples generation. Second, previous adversarial example generation methods focus mainly on multi-class classification models, while automatic diagnosis is a multi-label classification task. Third, the semantic and medical meaning of clinical notes are vital in disease inference, and even small perturbations can change them to a large extent. In this paper, we address the three main challenges and propose Clinical-Attacker, a general framework for both white-box and black-box adversarial text examples generation against automatic diagnosis models. Experimental results on MIMIC-III dataset demonstrate that our framework can easily alter the predictions of automatic diagnosis models with the semantic and medical meaning preserved.

Accurately forecasting vessel motions is a critical step towards achieving fast and accurate intelligent vessel control systems. Intelligent vessel control relies on accurate predictions of vessel motion to make informed decisions regarding control, maneuvering, and positioning, particularly during times of exogenous loading caused by adverse weather conditions. Hence, by accurately forecasting vessel motion accurately, the control system can anticipate potential issues (i.e., excessive trim or roll) and prescribe corrective actions before they become problematic. In this study, the authors propose two approaches to address the problem of vessel motion forecasting. The first approach relies on classical shallow learning models, whereas the second approach involves the use of state-of-the-art deep learning models for improved accuracy at further forecast horizons. Unlike shallow models, deep models can learn the required features directly from the data and therefore do not require a priori knowledge or additional features engineering. By leveraging deep learning models, the authors show that vessel motions can be forecasted further into the future without a significant loss in accuracy, thereby improving the overall effectiveness of the intelligent vessel control system. To support their statements, the authors use real operational data and compare the performance of the shallow and deep learning models. The results show that deep learning outperforms shallow learning models in terms of accuracy without a significant increase in the computational demand. Additionally, the authors demonstrate that their models remain accurate even under adverse weather conditions, indicating that they have practical applicability for vessel motions forecasting and can potentially improve the overall effectiveness of intelligent vessel control systems.

This article investigates the identification of hate speech on social media using machine learning and deep learning techniques. The research uses metrics such as F-measure, AUC-ROC, precision, accuracy, and recall assessing the effectiveness of various tactics. The findings indicate that deep learning models, particularly the bidirectional long short-term memory (BiLSTM) architecture, consistently outperform other methods in categorization tasks. The research emphasizes the importance of sophisticated neural network designs in identifying the intricacies of hostile and offensive content online. The study offers insights for promoting early identification and prevention of cyberbullying, improving secure and inclusive online environments. Future research may explore real-time detection systems, hybrid approaches, or the integration of complementary components to enhance and improve innovative technology in tackling this significant social issue. A sample tweet was annotated by specialists who categorize tweets as hate speech, offensive language, or neutral. The researchers applied shallow learning methodologies and integrated word embeddings like Word2Vec and GloVe to enhance the efficacy of deep learning models. The results indicate that BiLSTM surpasses shallow learning methods in detecting hate speech on Twitter, highlighting the efficacy of deep learning approaches in recognizing and tracking hate speech on social media platforms. When comparing different deep learning and machine learning models on different datasets, the results reveal that deep learning techniques are usually more effective. A reasonably high level of accuracy is achieved by KNN and SVM among classical algorithms, whereas Naïve Bayes performs the poorest. While deep learning approaches provide better results, tree-based models such as Random Forest and Decision Trees offer more consistent accuracy. Models based on neural networks, such as LSTM, CNN, and BI-LSTM, perform well, with LSTM-based methods excelling in particular. The most successful strategy for classification problems is the model presented, which obtains the greatest accuracy, precision, recall, F1-score of 95%. The research aids in the development of advanced tools and methodologies to mitigate hate speech on social media and foster positive online interactions. Future research may investigate alternative deep learning architectures, such as transformers, to enhance hate speech detection efficacy. The advancement of interpretable AI methodologies for identifying hate speech and delivering transparent forecasts might enhance user confidence and facilitate better content moderation decisions.

The latest advancements of deep learning have resulted in a new era of natural language processing. The machines now possess an unparallel ability to interpret and engage with various tasks such as text classification, content generation and natural language understanding. This development extended to the analysis of human behavior, where deep learning models are used to decode human personality. Due to the rise of social media, generating huge amounts of textual data that reshaped communication patterns. Understanding personality traits is a challenging topic which helps us to explore the patterns of thoughts, feelings and behaviors which are helpful for recruitment, career counselling and consumers’ behavior for marketing, etc. In this research study, the main aim is to predict the human personality trait of agreeableness showing whether a person is emotional who feels a lot or thinker who is logical and has rational thinking. This behavior leads to analyzing them as cooperative, friendly and respecting difference of views. For comprehensive empirical analysis, shallow machine learning models, ensemble models, and deep learning technique including state of the art transformer-based models are applied on real-world dataset of MBTI. For feature engineering, textual features of TF-IDF and POS tagging and word embeddings such as word2vec, glove and sentence embeddings are explored. The results analysis shows the highest performance 91.57% with sentence embeddings utilizing Bi-LSTM algorithm that highlights the power of this study as compared to existing studies in the relevant literature.

With intentional feature perturbations to a deep learning model, the adversary generates an adversarial example to deceive the deep learning model. As an adversarial example has recently been considered in the most severe problem of deep learning technology, its defense methods have been actively studied. Such effective defense methods against adversarial examples are categorized into one of the three architectures: (1) model retraining architecture; (2) input transformation architecture; and (3) adversarial example detection architecture. Especially, defense methods using adversarial example detection architecture have been actively studied. This is because defense methods using adversarial example detection architecture do not make wrong decisions for the legitimate input data while others do. In this paper, we note that current defense methods using adversarial example detection architecture can classify the input data into only either a legitimate one or an adversarial one. That is, the current defense methods using adversarial example detection architecture can only detect the adversarial examples and cannot classify the input data into multiple classes of data, i.e., legitimate input data and various types of adversarial examples. To classify the input data into multiple classes of data while increasing the accuracy of the clustering model, we propose an advanced defense method using adversarial example detection architecture, which extracts the key features from the input data and feeds the extracted features into a clustering model. From the experimental results under various application datasets, we show that the proposed method can detect the adversarial examples while classifying the types of adversarial examples. We also show that the accuracy of the proposed method outperforms the accuracy of recent defense methods using adversarial example detection architecture.

Data is one of the most important limiting factors of deep machine learning (ML) model in drilling applications. Though a big size of historical data can be available, high-quality cleaned and labeled data is usually limited. In this case study, we show that with limited labeled data, physics-based data augmentation combined with unsupervised learning significantly improves both stability and accuracy in bit wear ML model. It provides a pathway to overcome labeled data shortage and field data quality limitations. Labeled bit wear data is usually limited because only the final bit dull state can be labeled using dull photos for the entire drilling distance. To overcome this difficulty, an encoder-decoder unsupervised ML framework based on Bi-directional LSTM architecture is first applied to the data of entire drilling distance to extract and maximize data learnings. Then, a physics-based wear estimator is implemented in the learned latent space to augment labeled wear dataset, guided by true wear labels and offset-well data. Weighting factor in loss function is applied to optimize augmented and true datasets in final supervised learning step. The proposed real-time bit wear model is built on a combination of physics-based data augmentation and unsupervised data learning method. The model is applied on multiple field bit runs. Results show that by implementing the unsupervised data learnings only, the prediction accuracy is improved by 30% compared to the baseline ML model. By combining physics-based augmentation, the accuracy is further improved by 10%. More importantly, adding physics-based data augmentation significantly reduces prediction variance and unphysical wear outputs, therefore improving prediction stability by more than 30%. It should be noted that the prediction stability of AI/ML model is crucial in real-time application and decision making. The results show physics-based data augmentation not only increases the size of label dataset and prevents model overfitting, but also applies physics-based guidance to the ML model effectively. It is learned weighting factor plays a crucial role in balancing loss contributions from true wear labels and physics-based labels. While low weighting factor of physics-based labels diminishes the augmented data, high weighting factor disrespects the true wear labels leading to a high prediction bias. Overall, the bit wear model study shows physic-based data augmentation combined with unsupervised data learning can effectively improve model accuracy, stability, and overcome labeled data shortage difficulty. The proposed paper shows a case study for the bit wear ML model using a combination of physics-based data augmentation and unsupervised data learning. While labeled data is one of the major challenges of ML models in many drilling applications, this study provides a pathway to improve both accuracy and stability of deep ML models with limited labeled data.

Recent years, Machine Learning has been widely used in malware analysis and achieved unprecedented success. However, deep learning models are found to be highly vulnerable to adversarial examples, which leads to the machine learning-based malware analysis methods vulnerable to malware makers. Exploring the attack algorithm can not only promote the generation of more effective malware analysis methods, but also can promote the development of the defense algorithm. Different machine learning models use different malware features as their classification basis, and accordingly there will be different attack methods against them. For malware visualization method, corresponding effective adversarial attack has not yet appeared. Most existing malware adversarial examples for malware visualization are generated at the feature level, and do not consider whether the generated adversarial examples can be executed and complete their original functions. In this paper, we explored how to modify an Android executable file without affecting its original functions and made it become an adversarial example. We proposed an executable adversarial examples attack strategy for machine learning-based malware visualization analysis. Experimental result shows that the executable adversarial examples we generated can be normally run on Android devices without affecting its original functions, and can confuse the malware family classifier with 93% success rate. We explored possible defense methods and hope to contribute to building a more robust malware classification method.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Breast cancer remains one of the most prevalent and deadly cancers affecting women worldwide. Early detection is crucial, particularly for younger women, as traditional screening methods like mammography often struggle with accuracy in cases of dense breast tissue. Infrared thermography offers a non-invasive imaging alternative that enhances early detection by capturing subtle thermal variations indicative of breast abnormalities. This study investigates and compares the performance of various deep learning and machine learning models in analyzing thermographic data to classify breast tissue as healthy, benign, or malignant. To maximize detection accuracy, data preprocessing, feature extraction, and dimensionality reduction were implemented to isolate distinguishing characteristics across tissue types. Leveraging advanced feature extraction and visualization techniques inspired by geospatial data methodologies, we evaluated several deep learning architectures and classical classifiers using the DRM-IR and Breast Thermography Mendeley thermal datasets. Among the tested models, the ResNet152 architecture combined with a Support Vector Machine (SVM) classifier delivered the highest performance, achieving 97.62% accuracy, 95.79% precision, 98.53% recall, 94.52% specificity, an F1 score of 97.16%, an area under the curve (AUC) of 99%, a latency of 0.06 s, and CPU utilization of 88.66%. These findings underscore the potential of integrating infrared thermography with advanced deep learning and machine learning approaches to significantly improve the accuracy and efficiency of breast cancer detection, supporting its role as a valuable tool for early diagnosis.

Accurate prediction of urban residential rents is of great importance for landlords, tenants, and investors. However, existing rents prediction models face challenges in meeting practical demands due to their limited perspectives and inadequate prediction performance. The existing individual prediction models often lack satisfactory accuracy, while ensemble learning models that combine multiple individual models to improve prediction results often overlook the impact of spatial heterogeneity on residential rents. To address these issues, this paper proposes a novel prediction model called GERPM, which stands for Geographically Weighted Stacking Ensemble Learning-Based Urban Residential Rents Prediction Model. GERPM comprehensively analyzes the influencing factors of residential rents from multiple perspectives and leverages a geographically weighted stacking ensemble learning approach. The model combines multiple machine learning and deep learning models, optimizes parameters to achieve optimal predictions, and incorporates the geographically weighted regression (GWR) model to consider spatial heterogeneity. By combining the strengths of deep learning and machine learning models and taking into account geographical factors, GERPM aims to improve prediction accuracy and provide robust predictions for urban residential rents. The model is evaluated using housing data from Nanjing, a major city in China, and compared with representative individual prediction models, the equal weight combination model, and the ensemble learning model. The experimental results demonstrate that GERPM outperforms other models in terms of prediction performance. Furthermore, the model’s effectiveness and robustness are validated by applying it to other major cities in China, such as Shanghai and Hangzhou. Overall, GERPM shows promising potential in accurately predicting urban residential rents and contributing to the advancement of the rental market.

Autistic people are often disadvantaged in employment, education, etc. In fact, autistic students/employees face several challenges navigating and communicating with their superiors and colleagues. Mobile applications for people with Autism Spectrum Disorder (ASD apps for short) have been increasingly being adapted to help autistic people manage their conditions and daily activities. User feedback analysis is an effective method that can be used to improve ASD apps' services. In this article, we investigate the usage of ASD apps to improve the quality of life for autistic students/employees based on user feedback analysis. For this purpose, we analyze user reviews suggested on highly ranked ASD apps for college students, and workers. A total of 97,051 reviews have been collected from 13 ASD apps available on Google Play and Apple App stores. The collected reviews have been classified into negative, positive, and neutral opinions. This analysis has been performed using machine learning and deep learning models. The best performances were provided by combining RNN and LSTM models with an accuracy of 96.58% and an AUC of 99.41%. Finally, we provide some recommendations to improve ASD apps to assist developers in upgrading the main services provided by their apps.

Introduction: Arthritis is one of the most common chronic diseases. Early detection of arthritis and its progression can facilitate early intervention measures, lowering disease severity in patients. As electronic health records (EHR) become more accessible, this study assesses whether general health information and arthritis-related questionnaires can be used in arthritis diagnosis, without the involvement of costly imaging methods. Therefore, we created deep learning (DL) and machine learning (ML) models to explore the feasibility of combining EHR and modern computational tools to diagnose arthritis. Methods: A total of 782 arthritis patients and 4014 control patients were identified from the Osteoarthritis Initiative (OAI) – a ten-year-long observational study that included patient EHR in five time points. Six hundred variables were filtered by random forest classifier followed by manual filtering. Data were split properly to training, testing and validation set, and the training set was balanced. Sequential, nonsequential DL models, and five independent DL models for each time points were used. The accuracy, positive prevalence value (PPV), negative prevalence value (NPV), and area under curve (AUC), were assessed and compared with four classical ML models. SHAP (SHapley Additive exPlanations) summary analysis was also conducted. Results: Sequential and non-sequential deep learning models showed accuracies of ~ 0.97, and the four classical machine learning approaches showed accuracies of above 0.9. High positive and negative predicted values (> 0.90) for all of the models suggested the potential clinical applicability of the model, while the SHAP analysis demonstrated its interpretability. Discussion: We tested various models and showed the ability to use machine learning methods for early diagnosis of arthritis with EHR. The models can be used as a screening tool to select susceptible patients for confirmatory tests such as X-ray and MRI. Identification of early disease states could facilitate protective measures that slow disease progression.

Driver drowsiness is a major factor that contributing to road accidents. Several researches are ongoing to detect driver drowsiness, but they suffer from the complexity and cost of the models. This paper introduces a hybrid artificial intelligence (AI)-driven framework integrating deep learning (DL) and machine learning (ML) models for real-time drowsiness detection. The system utilizes a robust DL model to classify driver states based on facial images and support vector machine (SVM) model is trained to develop a cost-efficient yet robust facial landmark detector to extract key features such as eye aspect ratio (EAR) and mouth aspect ratio (MAR). We also introduce a multi-stage decision fusion mechanism that combines convolutional neural network (CNN) probability scores with EAR/MAR thresholds to enhance detection reliability and reduce false positives. Experimental results demonstrate that the proposed model achieves 98% accuracy and F1-score, significantly outperforming traditional DL approaches. Additionally, the SVM-based landmark predictor shows improved efficiency with lower mean squared error (MSE) without having higher computational requirements.

Adversarial malware examples refer to the malwares that can evade the malware detector. Researching adversarial malware examples can help us find the vulnerability of malware detector and improve the defense ability of cyberspace. Considering the huge market share of android system, adversarial malware examples of android are studied in this paper. And an algorithm is proposed to find universal adversarial perturbations of malware. Such perturbation can be inserted the different malwares to generate adversarial examples. Then the effectiveness of algorithm is verified in the experiment. And three classic android malware detectors are used as targets. Experimental results show that universal adversarial perturbations for different machine learning models can be discovered via the proposed algorithm.KeywordsAdversarial malware examplesUniversal adversarial perturbationsAndroid platform

A robust and lightweight deep attention multiple instance learning algorithm for predicting genetic alterations.