Correlated Differential Privacy: Feature Selection in Machine Learning

To develop a hypothesis-free model that best predicts response to MTX drug in RA patients utilizing biologically meaningful genetic feature selection of potentially functional single nucleotide polymorphisms (pfSNPs) through robust machine learning (ML) feature selection methods. MTX-treated RA patients with known response were divided in a 4:1 ratio into training and test sets. From the patients' exomes, potential features for classifier prediction were identified from pfSNPs and non-genetic factors through ML using recursive feature elimination with cross-validation incorporating the random forest classifier. Feature selection was repeated on random subsets of the training cohort, and consensus features were assembled into the final feature set. This feature set was evaluated for predictive potential using six ML classifiers, first by cross-validation within the training set, and finally by analysing its performance with the unseen test set. The final feature set contains 56 pfSNPs and five non-genetic factors. The majority of these pfSNPs are located in pathways related to RA pathogenesis or MTX action and are predicted to modulate gene expression. When used for training in six ML classifiers, performance was good in both the training set (area under the curve: 0.855-0.916; sensitivity: 0.715-0.892; and specificity: 0.733-0.862) and the unseen test set (area under the curve: 0.751-0.826; sensitivity: 0.581-0.839; and specificity: 0.641-0.923). Sensitive and specific predictors of MTX response in RA patients were identified in this study through a novel strategy combining biologically meaningful and machine learning feature selection and training. These predictors may facilitate better treatment decision-making in RA management.

Feature selection in machine learning: A new perspective

This study utilized National Health and Nutrition Examination Survey (NHANES) data to investigate the associations between weight-adjusted waist index (WWI), body mass index (BMI), and chronic low back pain (CLBP) risk, and to develop machine learning models to assess the predictive capacity of WWI for CLBP. This cross-sectional analysis was based on NHANES 2009-2010 data. Weighted logistic regression models were employed to evaluate associations between WWI, BMI, and CLBP, with subgroup analyses, smooth curve fitting, and threshold effect analyses conducted to enhance result robustness. Receiver operating characteristic (ROC) curves were plotted to determine which indicator demonstrated stronger association with CLBP. Subsequently, permutation feature importance was applied for machine learning feature selection, random undersampling was utilized to address data imbalance, and the dataset was randomly divided into training and testing sets at a 7:3 ratio. Six machine learning algorithms were employed to predict CLBP occurrence and identify the optimal algorithm. A total of 4,687 participants were included. Significant differences were observed between CLBP and non-CLBP groups in age, diabetes prevalence, smoking status, BMI, WWI, and education level. Both WWI and BMI showed significant associations with CLBP; after covariate adjustment, WWI demonstrated stronger and more consistent associations across quartiles. Subgroup analyses, nonlinear analyses, and ROC analyses further supported these findings. Machine learning feature selection identified 19 variables, with the Random Forest model demonstrating optimal performance. Both WWI and BMI were associated with increased CLBP risk, with WWI potentially serving as a more sensitive predictive indicator. Prospective studies are needed to validate causal relationships. The Random Forest machine learning model demonstrated high accuracy in CLBP prediction.

Three-phase induction motors (IMs) are one of the most employed electric machines in industrial and household applications. Condition monitoring of these machines is essential to avoid unplanned maintenance and thereby enhance the availability. Artificial Intelligence (AI) technologies are emerging as an advanced tool for automating condition monitoring process to detect incipient faults at early stages. Machine Learning (ML) algorithms have been identified as a promising approach for condition monitoring of IMs and predicting maintenance to avoid failures. However, selecting the suitable ML algorithm for a given application is challenging because there is no predefined set of application-based algorithms. In addition, raw data processing and feature selection need careful attention to improve the accuracy of the results. This paper reviews supervised ML algorithms that can be used for condition monitoring of IMs and identifies their benefits and drawbacks. It then discusses how the dominant features from raw data can be selected through time domain and frequency domain analysis using the acoustic data collected from a three-phase induction motor. The study investigates classification accuracy of each ML algorithm and a procedure for selecting an algorithm based on the experimental results. Results of this study show that Support Vector Machines (SVM) algorithm outperforms other competing algorithms in condition monitoring of IMs when the dominant frequency components obtained through Fast Fourier Transform (FFT) are used as training data.

Recently, artificial intelligence (AI) domain increased to contain finance, education, health, mining, and education. Artificial intelligence controls the performance of systems that use new technologies, especially in the education environment. The multiagent system (MAS) is considered an intelligent system to facilitate the e-learning process in the educational environment. MAS is used to make interaction easily among agents, which supports the use of feature selection. The feature selection methods are used to select the important and relevant features from the database that could help machine learning algorithms produce high performance. This paper aims to propose an effective and suitable system for multiagent-based machine learning algorithms and feature selection methods to enhance the e-learning process in the educational environment which predicts pass or fail results. The univariate and Extra Trees feature selection methods are used to select the essential attributes from the database. Five machine learning algorithms named Decision Tree (DT), Logistic Regression (LR), Random Forest (RF), Naive Bayes (NB), and K-nearest neighbors algorithm (KNN) are applied to all features and selected features. The results showed that the learning algorithm that has been measured by the Extra Trees method has achieved the highest performance depending on the evaluation of cross-validation and testing.

The demand for electricity at home has increased in recent times globally, this high demand for continuous, stable and affordable power can be attributed to the demand for comfortable lifestyle of consumers but the quality and efficiency of the appliances being used remain questionable. Malfunctioning appliances usually show a power signature statistically different from their normal behavior, which can lead to higher energy consumption or more serious damages. As a result, numerous studies in recent times have been conducted on the household electrical appliance anomaly behaviors to find the root-cause of these anomalies using machine learning techniques and algorithms. This study attempted to undertake a systematic and critical review of ninety-two (92) research works reported in academic journals over fifteen (15) years (2006–2021) in the area of household electrical appliance anomaly detections and knowledge extraction using machine learning. The various techniques used in these reports were clustered based on machine learning-based techniques, statistical techniques and physical based approach techniques and the parameters adopted, such as machine learning algorithms, feature extraction approaches, anomaly detection levels, computing platforms and application scenarios. This clustering was done based on the following criteria: the nature of a dataset and the number of data sources used, the data timeframe, the machine learning algorithms used, machine learning task, used accuracy and error metrics and software packages used for modeling. For the number of data source used, the results revealed that 81.2% of documents reviewed used single sources and Autoregressive integrated moving average (ARIMA) was the highest implemented regression model (60.9%), the probability model that was mostly implemented was the Bayesian network. Furthermore, the study revealed that, root-mean-square error (RMSE) accounted 35% was the most used error metric among household appliance abnormal behaviors, followed by mean absolute percentage error MAPE which accounted 32%. The study further revealed that 46% of appliance abnormal detections was based on weather parameters, and historical energy consumption. Finally, we recap the challenges and limitations for further research in electrical appliance anomaly detections locally and globally.

The unmanned aerial vehicles (UAVs) have been extensively used in providing intelligence such as target tracking. In our field experiments, a pre-trained deep neural network (DNN) is deployed on the UAV to identify a target from the captured video frames and enable UAV to keep tracking. However, tracking in real time by the DNN requires a lot of computational resources. This motivates us to consider offloading this type of machine learning (ML) tasks to a mobile edge computing (MEC) server. Specifically, we propose a novel hierarchical ML tasks distribution framework for the UAV tracking system, where the UAV is embedded with lower layers of the pre-trained convolutional neural network (CNN) model due to its limited computing capability, while the MEC server with rich computing resources will handle the higher layers of the CNN model. An optimization problem is formulated to minimize the CNN inference delay while taking into account the communications delay, computing time, and ML error. Insights are provided to understand the tradeoff between communications and ML computing in offloading decisions. Numerical results demonstrate the effectiveness of the proposed ML tasks distribution framework with the optimized offloading strategy.

Recent technological advances allowed an exponential expansion of biological sequence data, and the extraction of meaningful information through Machine Learning (ML) algorithms. This knowledge improved the understanding of the mechanisms related to several fatal diseases, e.g., Cancer and COVID-19, helping to develop innovative solutions, such as CRISPR-based gene editing, coronavirus vaccine, and precision medicine. These advances benefit our society and economy, directly impacting peoples lives in various areas, such as health care, drug discovery, forensic analysis, and food analysis. Nevertheless, ML approaches applied to biological data require representative, quantitative, and informative features. Necessarily, as many ML algorithms can handle only numerical data, sequences need to be translated into a feature vector. This process, known as feature extraction, is a fundamental step for the elaboration of high-quality ML-based models in bioinformatics, by allowing the feature engineering stage, with the design and selection of suitable features. Feature engineering, ML algorithm selection, and hyperparameter tuning are often manual and time-consuming processes, requiring extensive domain knowledge, and performed manually by a human expert. To deal with this problem, we developed a new package, BioAutoML, which automatically runs an end-to-end ML pipeline. BioAutoML extracts numerical and informative features from biological sequence databases, automating feature selection, recommendation of ML algorithm(s), and tuning of hyperparameters, using Automated ML (AutoML). BioAutoML has two components, divided into four modules, (1) automated feature engineering (feature extraction and selection modules) and (2) Metalearning (algorithm recommendation and hyperparameter tuning modules). Our experimental results, assessing the relevance of our proposal, indicate robust results for different problem domains, such as SARS-CoV-2, anticancer peptides, HIV sequences, and non-coding RNAs. According to our systematic review, our proposal is innovative compared to available studies in the literature, being the first study to propose automated feature engineering and metalearning for biological sequences. BioAutoML has a high potential to significantly reduce the expertise required to use ML pipelines, aiding researchers in combating diseases, particularly in low- and middle-income countries. This initiative can provide biologists, physicians, epidemiologists, and other stakeholders with an opportunity for widespread use of these techniques to enhance the health and well-being of their communities.

Performance evaluation of machine learning techniques in predicting cumulative absolute velocity

The accurate estimation of aboveground biomass (AGB) in rubber plantations is essential for predicting rubber production and assessing carbon storage. Multispectral sensors mounted on unmanned aerial vehicles (UAVs) can obtain high spatiotemporal resolution imagery of rubber plantations, offering significant advantages in capturing fine structural details and heterogeneity. However, most previous studies primarily focused on developing biomass estimation models for rubber using machine learning (ML) algorithms in conjunction with feature selection methods based on UAV-acquired multispectral imagery. The reliance on feature selection methods limits the model’s generalizability, robustness, and predictive accuracy. In contrast, deep learning (DL) exhibits considerable promise in extracting features from high-resolution UAV-based multispectral imagery without the need for manual selection. Nonetheless, it remains unclear whether DL can surpass traditional ML methods in improving the AGB estimation accuracy in rubber plantations. To address this, our study evaluated the performance of three ML algorithms (random forest regression, RFR; XGBoost regression, XGBR; categorical boosting regression, CatBoost) combined with feature selection techniques and a deep convolutional neural network (DCNN) using multispectral imagery obtained from UAV for the AGB estimation of rubber plantations. The results indicate that the RFR combined with a principal component analysis (PCA) for feature selection yielded the best performance (R2 = 0.81, RMSE = 11.63 t/ha, MAE = 9.27 t/ha) between the three ML algorithms. Meanwhile, the DCNN model derived from the G, R, and NIR spectral bands achieved the highest estimation accuracy (R2 = 0.89, RMSE = 6.44 t/ha, MAE = 5.72 t/ha), where it outperformed the other ML methods. Our study highlights the great potential of combining UAV-based multispectral imagery with DL techniques to improve AGB estimation in rubber plantations, offering a new perspective for estimating the physiological and biochemical growth parameters of forests.

Use of machine learning (ML) and data mining (DM) algorithms has surfaced more often in the recent years for disease risk prediction problems in the healthcare communities. Several traditional feature selection models are combined with the DM and ML algorithms to improve accuracy of the disease risk prediction. In this study, a new Bio-inspired Ensemble Feature Selection (BEFS) model is introduced which is applied with the DM and ML algorithms. In the BEFS model, the most relevant and highly contributing features in the prediction are determined with a bio-inspired algorithm i.e., genetic algorithm, and an ensemble algorithm i.e., random forest algorithm. These important features obtained from the proposed model are then combined in various combinations and applied with the DM and ML algorithms, here logistic regression (LR) and random forest (RF), and the results obtained are promising. The experiment is executed using the famous ML language R. To accomplish this objective, the Breast Cancer Wisconsin (Diagnostic) dataset of UCI (University of California, Irvine) ML repository is utilized. In the experimental outcomes, the highest accuracy attained with the BEFS model is 96.49%, the AUC (Area Under Curve) achieved is 96%, and the sensitivity is 98.11%. These results, which greatly improve the disease risk prediction, are higher than several other existing works, while utilizing only six most relevant features out of the thirty two features of the dataset.

We propose a unifying FCA-based framework for some questions in data analysis and data mining, combining ideas from Rough Set Theory, JSM-reasoning, and feature selection in machine learning. Unlike the standard rough set model the indiscernibility relation in our paper is based on a quasi-order, not necessarily an equivalence relation. Feature selection, though algorithmically difficult in general, appears to be easier in many cases of scaled many-valued contexts, because the difficulties can at least partially be projected to the scale contexts. We propose a heuristic algorithm for this.

Human Action Recognition (HAR) is a vital part of the healthcare sector. Medical practitioners are experiencing difficulty in recognizing human physiological conditions due to the enormous quantity of sensory stimulation. Machine learning techniques help to predict human physiological conditions, providing medical practitioners to work more efficiently. Feature selection is an essential part of discovering new knowledge in the majority of real-world problems when there are a lot of features. Feature selection is extremely useful because it speeds up decisions and enhances classification performance. The importance of feature selection in machine learning is dimension reduction in a massive multivariate data collection. This paper presents an effective feature selection method known as the Enhanced Recursive Feature Elimination (EFRE) for selecting key features from the data set for HAR prediction. The experimental results reveal that the ERFE technique selects the most suitable features for HAR prediction. The performance of the proposed ERFE approach is tested using different performance evaluation metrics. The performance analysis shows that the ERFE method outperforms existing feature selection methods with 88% accuracy.

In recent years, proficiency in data science and machine learning (ML) became one of the most requested skills for jobs in both industry and academy. Machine learning algorithms typically require large sets of data to train the models and extensive usage of computing resources, both for training and inference. Especially for deep learning algorithms, training performances can be dramatically improved by exploiting Graphical Processing Units (GPUs). The needed skill set for a data scientist is therefore extremely broad, and ranges from knowledge of ML models to distributed programming on heterogeneous resources. While most of the available training resources focus on ML algorithms and tools such as TensorFlow, we designed a course for doctoral students where model training is tightly coupled with underlying technologies that can be used to dynamically provision resources. Throughout the course, students have access to a dedicated cluster of computing nodes on local premises. A set of libraries and helper functions is provided to execute a parallelized ML task by automatically deploying a Spark driver and several Spark execution nodes as Docker containers. Task scheduling is managed by an orchestration layer (Kubernetes). This solution automates the delivery of the software stack required by a typical ML workflow and enables scalability by allowing the execution of ML tasks, including training, over commodity (i.e. CPUs) or high-performance (i.e. GPUs) resources distributed over different hosts across a network. The adaptation of the same model on OCCAM, the HPC facility at the University of Turin, is currently under development.

A Distributed Denial of Service (DDoS) attack occurs when an attacker tries to disrupt a network, service or website by flooding huge numbers of packets on the internet traffic. Detecting DDoS attacks serves the goal of spotting and addressing them promptly to reduce their effects on the network, system or service being targeted. Detecting Distributed Denial of Service (DDoS) attacks is crucial, for people, companies and network managers. The detection of DDoS attacks has ranging uses in industries such as network security safeguarding websites, managing cloud services ensuring the security of online systems and services. Detecting DDoS attacks is essential for safeguarding infrastructure upholding service availability and guaranteeing the security of online systems and services. To achieve this objective, we proposed a framework to detect DDoS attacks including six steps. In step one, we start by gathering information, which includes network activity and system records, for operations as well as instances of DDoS attacks. Step two, we identify characteristics of the data collected such as patterns in network traffic, packet details, IP addresses, types of protocols used and more. Step three, we utilize algorithms for feature selection such as Salp Swarm Algorithm (SSA), Gray Wolf Algorithm (GWA), Particle Swarm Algorithm (PSO) to pinpoint the features that can distinguish between normal activities and DDoS attack patterns. After that in step four, we divide the processed dataset into sections for training and testing purposes to develop and assess the machine learning models such as SVM (support vector machine), and KNN (K-nearest neighbor). Step five we develop a classification model using machine learning techniques like decision trees, forests, support vector machines (SVM) logistic regression models or neural networks. Finally, we assess the effectiveness of models through metrics such as accuracy rates, precision levels, recall rates, and F1 scores. The results show that the proposed models achieve high results (99.9%). In summary detecting DDoS attacks is crucial for protecting networks, systems and online services against disruptions.