Intelligent Feature Engineering for Cybersecurity

Detecting loan fraud is a subject frequently raised in scientific research, and to which financial institutions attach great importance, given the losses they avoid by applying an efficient fraud detection. The purpose of this paper is to compare a machine learning algorithm with statistical models in fraud detection and describe a machine learning approach to loan fraud detection using data mining techniques. The paper proposes an approach using data mining techniques for machine learning in order to improve performance. In the training stage, the authors used two datasets. The first one is a raw data, and the second is a pre-processed data with feature engineering and selection methods. The results show a significant improvement in performance of all classifiers once tested on pre-processed data. Finally, they conclude that applying machine learning algorithms directly to raw data gives bad results for statistical model comparing to machine learning algorithms, and the proposed approach using feature engineering and selection techniques contributes to improve performance.

Evaluation of multispectral imaging for freeze damage assessment in strawberries using AI-based computer vision technology

Feature engineering involves extracting information from raw-data to use in machine learning or deep learning algorithms through feature transformation, feature generation or feature extraction, feature construction, feature selection, etc. Feature engineering optimizes the feature space dimensions, thereby reducing complexity. Hence, makes the data suitable for machine learning or deep learning applications: process prediction, detection accuracy, estimation accuracy, and quality of clustering and classi cation. Traditionally machine learning algorithms have been used in various wireless communication appli cations viz spectrum access and sharing, resource allocation, spectrum coverage, capacity optimization, signal intelligence, etc. This chapter presents the need for feature engineering and its application to machine learning and deep learning in wireless communication. The chapter also gives the details of the feature engineering assisted machine-learning algorithm for automatic modulation classi cation and path-loss prediction in wireless communication.KeywordsDeep learningFeature engineeringMachine learningWireless communication

Flame spectra contain useful information about combustion and hence the spectral features of flame radicals may be used to identify different biomass fuels. A technique for biomass fuel identification is proposed based on the spectral features of flame radicals, feature engineering and improved support vector machine. The spectral intensity signals of biomass flames and flame radicals (OH*(310.85nm), CN*(390.00nm), CH*(430.57nm) and C2*(515.23nm, 545.59nm)) were acquired using a spectrometer. Feature engineering was built, which can accurately reflect the characteristics of sample category, through feature extraction, feature selection based on Filter and feature learning based on dictionary learning. The support vector machine is used to build the identification model, where radial basis kernel parameter ? and error penalty factor C are optimized using an improved grid search algorithm. Experimental results from a laboratory-scale combustion rig show the effectiveness of the proposed method for the identification of biomass fuel.

Recent technological advances allowed an exponential expansion of biological sequence data, and the extraction of meaningful information through Machine Learning (ML) algorithms. This knowledge improved the understanding of the mechanisms related to several fatal diseases, e.g., Cancer and COVID-19, helping to develop innovative solutions, such as CRISPR-based gene editing, coronavirus vaccine, and precision medicine. These advances benefit our society and economy, directly impacting peoples lives in various areas, such as health care, drug discovery, forensic analysis, and food analysis. Nevertheless, ML approaches applied to biological data require representative, quantitative, and informative features. Necessarily, as many ML algorithms can handle only numerical data, sequences need to be translated into a feature vector. This process, known as feature extraction, is a fundamental step for the elaboration of high-quality ML-based models in bioinformatics, by allowing the feature engineering stage, with the design and selection of suitable features. Feature engineering, ML algorithm selection, and hyperparameter tuning are often manual and time-consuming processes, requiring extensive domain knowledge, and performed manually by a human expert. To deal with this problem, we developed a new package, BioAutoML, which automatically runs an end-to-end ML pipeline. BioAutoML extracts numerical and informative features from biological sequence databases, automating feature selection, recommendation of ML algorithm(s), and tuning of hyperparameters, using Automated ML (AutoML). BioAutoML has two components, divided into four modules, (1) automated feature engineering (feature extraction and selection modules) and (2) Metalearning (algorithm recommendation and hyperparameter tuning modules). Our experimental results, assessing the relevance of our proposal, indicate robust results for different problem domains, such as SARS-CoV-2, anticancer peptides, HIV sequences, and non-coding RNAs. According to our systematic review, our proposal is innovative compared to available studies in the literature, being the first study to propose automated feature engineering and metalearning for biological sequences. BioAutoML has a high potential to significantly reduce the expertise required to use ML pipelines, aiding researchers in combating diseases, particularly in low- and middle-income countries. This initiative can provide biologists, physicians, epidemiologists, and other stakeholders with an opportunity for widespread use of these techniques to enhance the health and well-being of their communities.

The fashion industry faces challenges in accurately predicting demand due to inherent uncertainty, leading to suboptimal inventory and financial losses. Machine learning (ML) offers a robust solution by analyzing large and complex data, identifying non-linear patterns, and providing more accurate predictions than conventional methods that rely on limited factors. This research aims to compare and evaluate the performance of six different ML models—XGBoost, SVM, RF, GBM, KNN, and NN, considering the influence of feature engineering and various data split ratios on predicting fashion product demand. KNN and NN were included due to distinct modeling approaches and competitive capabilities in identifying local and non-linear patterns across numerical, categorical, and time series data. Techniques such as feature extraction and selection and various data split ratios (70:30, 80:20, 90:10) were used. Using Adidas sales data, the models were evaluated based on Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). The results indicate that the XGBoost-based model with feature engineering consistently outperforms the other models across all data split ratios. Particularly, XGBoost with feature engineering at a data split ratio of 90:10 achieved the best performance with an RMSE of 4.46 and an MAE of 1.51. Analyzing model performance shows that the predictive ability of ML models is influenced by the implementation of feature engineering and the selection of the data split ratio. These results demonstrate the potential of using feature-engineered XGBoost models and optimized data ratios to mitigate the risk of stockouts or overstocks, and reduce financial losses and environmental waste.

Parkinson’s disease (PD) is a progressive neurological disorder that affects millions globally, posing significant challenges in early and accurate diagnosis. Recent advancements in machine learning (ML) offer promising approaches for addressing these challenges by enabling more precise and efficient PD predictions. This paper proposes an enhanced ML framework for PD prediction, integrating data balancing, feature selection, and explainable AI techniques. We evaluate nine different ML algorithms using a dataset of clinical and voice features. To address the class imbalance, we employ the Synthetic Minority Oversampling Technique (SMOTE) and NearMiss, comparing results to an imbalanced baseline. Feature engineering approaches, including Featurewiz, Tree based Feature Importance and the chi-square test, are utilized to identify key predictive features such as Pitch Period Entropy (PPE), Noise-to-Harmonic Ratio (NHR), and other voice biomarkers. Explainable AI (XAI) techniques (SHAP and LIME) interpret model decision-making and highlight influential features. The best-performing model, KNN with SMOTE, achieved 92% accuracy, F1-score 0.94, and a G-Mean of 0.95—demonstrating balanced, reliable PD detection. While some models achieved higher accuracy on imbalanced data (up to 97%), their performance lacked sensitivity and balance. Our findings suggest that combining SMOTE with feature engineering and XAI substantially enhances model fairness, performance, and interpretability. This research advances PD prediction by providing an accurate and interpretable ML-based diagnostic tool to support early diagnosis and better patient management.

The ability of music to spread joy and excitement across lives, makes it widely acknowledged as the human race's universal language. The phrase "music genre" is frequently used to group several musical styles together as following a shared custom or set of guidelines. According to their unique preferences, people now make playlists based on particular musical genres. Due to the determination and extraction of appropriate audio elements, music genre identification is regarded as a challenging task. Music information retrieval, which extracts meaningful information from music, is one of several real - world applications of machine learning. The objective of this paper is to efficiently categorise songs into various genres based on their attributes using various machine learning approaches. To enhance the outcomes, appropriate feature engineering and data pre-processing techniques have been performed. Finally, using suitable performance assessment measures, the output from each model has been compared. Compared to other machine learning algorithms, Random Forest along with efficient feature selection and hyperparameter tuning has produced better results in classifying music genres.

Principles, research status, and prospects of feature engineering for data-driven building energy prediction: A comprehensive review

BackgroundSepsis, an acute and potentially fatal systemic response to infection, significantly impacts global health by affecting millions annually. Prompt identification of sepsis is vital, as treatment delays lead to increased fatalities through progressive organ dysfunction. While recent studies have delved into leveraging Machine Learning (ML) for predicting sepsis, focusing on aspects such as prognosis, diagnosis, and clinical application, there remains a notable deficiency in the discourse regarding feature engineering. Specifically, the role of feature selection and extraction in enhancing model accuracy has been underexplored.ObjectivesThis scoping review aims to fulfill two primary objectives: To identify pivotal features for predicting sepsis across a variety of ML models, providing valuable insights for future model development, and To assess model efficacy through performance metrics including AUROC, sensitivity, and specificity.ResultsThe analysis included 29 studies across diverse clinical settings such as Intensive Care Units (ICU), Emergency Departments, and others, encompassing 1,147,202 patients. The review highlighted the diversity in prediction strategies and timeframes. It was found that feature extraction techniques notably outperformed others in terms of sensitivity and AUROC values, thus indicating their critical role in improving sepsis prediction models.ConclusionKey dynamic indicators, including vital signs and critical laboratory values, are instrumental in the early detection of sepsis. Applying feature selection methods significantly boosts model precision, with models like Random Forest and XG Boost showing promising results. Furthermore, Deep Learning models (DL) reveal unique insights, spotlighting the pivotal role of feature engineering in sepsis prediction, which could greatly benefit clinical practice.

In light of the escalating ubiquity of the Internet, the proliferation of cyber-attacks, coupled with their intricate and surreptitious nature, has significantly imperiled network security. Traditional machine learning methodologies inherently exhibit constraints in effectively detecting and classifying multifarious cyber threats. Specifically, the surge in high-dimensional network traffic data and the imbalanced distribution of classes exacerbate the predicament of ideal classification performance. Notably, the presence of redundant information within network traffic data undermines the accuracy of classifiers. To address these challenges, this study introduces a novel approach for intrusion detection classification which integrates advanced techniques of feature engineering and model optimization. The method employs a feature engineering approach that leverages mutual information maximum correlation minimum redundancy (mRMR) feature selection and synthetic minority class oversampling technique (SMOTE) to process network data. This transformation of raw data into more meaningful features effectively addresses the complexity and diversity inherent in network data, enhancing classifier accuracy by reducing feature redundancy and mitigating issues related to class imbalance and the detection of rare attacks. Furthermore, to optimize classifier performance, the paper applies the Optuna method to fine-tune the hyperparameters of the Catboost classifier, thereby determining the optimal model configuration. The study conducts binary and multi-classification experiments using publicly available datasets, including NSL_KDD, UNSW-NB15, and CICIDS-2017. Experimental results demonstrate that the proposed method outperforms traditional approaches regarding accuracy, recall, precision, and F-value. These findings highlight the method’s potential and performance in network intrusion detection.

Features engineered hardness and yield strength prediction of septenary refractory high-entropy alloys

Preharvest crop yield estimation is crucial for achieving food security and managing crop growth. Unmanned aerial vehicles (UAVs) can quickly and accurately acquire field crop growth data and are important mediums for collecting agricultural remote sensing data. With the rapid development of machine learning, especially deep learning, research on yield estimation based on UAV remote sensing data and machine learning has achieved excellent results. This paper systematically reviews the current research of yield estimation research based on UAV remote sensing and machine learning through a search of 76 articles, covering aspects such as the grain crops studied, research questions, data collection, feature selection, optimal yield estimation models, and optimal growth periods for yield estimation. Through visual and narrative analysis, the conclusion covers all the proposed research questions. Wheat, corn, rice, and soybeans are the main research objects, and the mechanisms of nitrogen fertilizer application, irrigation, crop variety diversity, and gene diversity have received widespread attention. In the modeling process, feature selection is the key to improving the robustness and accuracy of the model. Whether based on single modal features or multimodal features for yield estimation research, multispectral images are the main source of feature information. The optimal yield estimation model may vary depending on the selected features and the period of data collection, but random forest and convolutional neural networks still perform the best in most cases. Finally, this study delves into the challenges currently faced in terms of data volume, feature selection and optimization, determining the optimal growth period, algorithm selection and application, and the limitations of UAVs. Further research is needed in areas such as data augmentation, feature engineering, algorithm improvement, and real-time yield estimation in the future.

Feature engineering is treated as a crucial step in any intelligent system pipeline and practitioners often spend 70–80% of their time in this phase before modeling. Especially, in machine learning (ML), the performance of the classification systems deteriorates as irrelevant features are added, even when the features presented contain enough information about the problem. ML researchers often use feature selection and feature extraction techniques individually to filter out the redundant information from the data, but would not get desired results or performance due to choosing a single method only. In this article, we take an attempt to combine both feature selection and feature extraction methods that give an improvement in the prediction performance than using a single feature selection or feature extraction method. In the application of the proposed method, we consider the Indian liver disease patient dataset to classify liver disease using machine learning classifiers. In addition, ensemble classifiers also take into account reproducing and robustifying classification accuracy. The performance is evaluated and compared with different metrics such as accuracy, precision, recall, and F1-score. Using the proposed method, the KNN classifier can diagnosis 76.03% with a precision of 76.69%, recall 96.23%, and 85.36% F1-score which is much better than the individual feature selection and feature extraction approach. The proposed method can also be extended to medical datasets other than liver disease so that these types of deadly diseases can be correctly diagnosed for the betterment of humanity.

In the practical implementation of control of electromyography (sEMG) driven devices, algorithms should recognize the human’s motion from sEMG with fast speed and high accuracy. This study proposes two feature engineering (FE) techniques, namely, feature-vector resampling and time-lag techniques, to improve the accuracy and speed of least square support vector machine (LSSVM) for wrist palmar angle estimation from sEMG feature. The root mean square error and correlation coefficients of LSSVM with FE are 9.50 2.32 degree and 0.971 0.018 respectively. The average training time and average execution time of LSSVM with FE in processing 12600 sEMG points are 0.016 s and 0.053 s respectively. To evaluate the proposed algorithm, its estimation results are compared with those of three other methods, namely, LSSVM, radial basis function (RBF) neural network, and RBF with FE. Experimental results verify that introduction of time-lag into feature vector can greatly improve the estimation accuracy of both RBF and LSSVM; meanwhile the application of feature-vector resampling technique can significantly increase the training and execution speed of RBF neural network and LSSVM. Among different algorithms applied in this study, LSSVM with FE techniques performed best in terms of training and execution speed, as well as estimation accuracy.