Optimum-path forest stacking-based ensemble for intrusion detection

EEG signal classification for epilepsy diagnosis via optimum path forest – A systematic assessment

In this paper, the Optimum-Path Forest (OPF) classifier is applied in the classification of defects in cowhide, a problem of great evaluation complexity. The OPF classifier reduces a pattern classification problem to the problem of partitioning the vertices of a graph induced by its data set. The results revealed a competent performance compared to traditional classifiers, such as Support Vector Machines (SVM), Artificial Neural Networks-Perceptron Multilayer (MLP), Decision Trees (J48) and k-Nearest Neighbor (kNN).

Machine learning (ML) and image processing techniques have been applied together to various scenarios for the development of Intelligent Vehicles. Among these scenarios, pedestrian detection has received growing interest in recent years, since high concern for safety applications in traffic has arisen. Several ML methods were successfully applied to solve this problem. However, because pedestrian detection is in general computationally intensive, a good trade off between accuracy and processing time is desirable, particularly if the methods are directed to real-time applications. Optimum Path Forest (OPF) classifier is a recently developed non-parametric classifier method. This work contribution is the performance assessment of a novel OPF application to pedestrian detection. Results have shown that it is fast and competitive against established methods and a viable alternative to be considered for machine learning and pedestrian detection applications.

ABSTRACTThis paper reviews the activity-travel behaviour literature that employs Machine Learning (ML) techniques for empirical analysis and modelling. Machine Learning algorithms, which attempt to build intelligence utilizing the availability of large amounts of data, have emerged as powerful tools in the fields of pattern recognition and big data analysis. These techniques have been applied in activity-travel behaviour studies since the early ’90s when Artificial Neural Networks (ANN) were employed to model mode choice decisions. AMOS, an activity-based modelling system developed in the mid-’90s, has ANN at its core to model and predict individual responses to travel demand management measures. In the dawn of 2000, ALBATROSS, a comprehensive activity-based travel demand modelling system, was proposed by Arentze and Timmermans using Decision Trees. Since then researchers have been exploring ML techniques like Support Vector Machines (SVM), Decision Trees (DT), Neural Networks (NN), Bayes Classifiers, and more recently, Ensemble Learners to model and predict activity-travel behaviour. A large number of publications over the years and an upward trend in the number of published articles over time indicate that Machine Learning is a promising tool for activity-travel behaviour analysis and prediction. This article, first of its kind in the literature, reviews these studies and explores the trends in activity-travel behaviour research that apply ML techniques. The review finds that mode choice decisions have received wide attention in the literature on ML applications. It was observed that most of the studies identify the lack of interpretability as a serious shortcoming in ML techniques. However, very few studies have attempted to improve the interpretability of the models. Further, some studies report the importance of feature engineering in ML-based studies, but very few studies adopt feature engineering before model development. Spatiotemporal transferability of models is another issue that has received minimal attention in the literature. In the end, the paper discusses possible directions for future research in the area of activity-travel behaviour modelling using ML techniques.

In network security, intrusion detection plays an important role. Feature subsets obtained by different feature selection methods will lead to different accuracy of intrusion detection. Using individual feature selection method can be unstable in different intrusion detection scenarios. In this paper, the idea of ensemble is applied to feature selection to adjust feature subsets. Feature selection is converted into a two-category problem, and odd number of feature selection methods is used for voting method to decide whether a feature is required or discarded. In actual operation, mean decrease impurity, random forest classifier, stability selection, recursive feature elimination and chi-square test are used. Feature subsets obtained from them will be adjusted by our proposed method to get ensemble feature subsets. To test the performance, support vector machine, decision tree, knn and multi-layer perception are used to observe and compare the classification accuracy with ensemble feature subsets. Three intrusion detection data sets, including kddcup99, cidds-001 and unsw_nb15 are used in our experiments. The best result is reflected on cidds-001 with a 99.40% classification accuracy. The investigation shows that our method has a certain improvement on classification accuracy of intrusion detection.

The cyber assault has emerged as a serious digital threat. Day after day, hackers inflict severe financial harm on nations and people alike. As a result, threat detection has become increasingly critical for computer system security going forward. This research focused on seeing if it can recognize and anticipate dangers based on complicated instances by using machine learning. This can help identify and prevent attacks before they happen. Many well-known threat classification methods are tested in this work. It is centered around six of the most prominent machine learning (ML) techniques used in intrusion detection research: Random Forest, Naive Bayes, Artificial Neural Networks, K-nearest neighbor algorithm, Support Vector Machines, and bagging. In this study, a machine learning-based strategy for reliably predicting dos, R2L, U2R, probe, and overall attacks was established by assessing the accuracy of several ML tactics. Finding effective machine learning algorithms, as well as determining critical qualities that can provide the best outcomes, is a massive task for academics. This paper also examines the benefits and drawbacks of each method before applying it to the NSL-KDD dataset to evaluate how well they perform. The novelty in this research work lies in the fact that it utilizes various feature selection methods to find the set of features that have the greatest impact on classification performance. Analysis of the results using different performance metrics revealed improved accuracy. For the NSL-KDD dataset’s intrusion detection and attack type classification tasks, the highest detection rate was obtained with 99.86%.

Landslide disaster risk reduction necessitates the investigation of different geotechnical causal factors for slope failures. Machine learning (ML) techniques have been proposed to study causal factors across many application areas. However, the development of ensemble ML techniques for identifying the geotechnical causal factors for slope failures and their subsequent prediction has lacked in literature. The primary goal of this research is to develop and evaluate novel feature selection methods for identifying causal factors for slope failures and assess the potential of ensemble and individual ML techniques for slope failure prediction. Twenty-one geotechnical causal factors were obtained from 60 sites (both landslide and non-landslide) spread across a landslide-prone area in Mandi, India. Relevant causal factors were evaluated by developing a novel ensemble feature selection method that involved an average of different individual feature selection methods like correlation, information-gain, gain-ratio, OneR, and F-ratio. Furthermore, different ensemble ML techniques (Random Forest (RF), AdaBoost (AB), Bagging, Stacking, and Voting) and individual ML techniques (Bayesian network (BN), decision tree (DT), multilayer perceptron (MLP), and support vector machine (SVM)) were calibrated to 70% of the locations and tested on 30% of the sites. The ensemble feature selection method yielded six major contributing parameters to slope failures: relative compaction, porosity, saturated permeability, slope angle, angle of the internal friction, and in-situ moisture content. Furthermore, the ensemble RF and AB techniques performed the best compared to other ensemble and individual ML techniques on test data. The present study discusses the implications of different causal factors for slope failure prediction.

Cyber-security threats are a growing concern in networked environments. The development of Intrusion Detection Systems (IDSs) is fundamental in order to provide extra level of security. We have developed an unsupervised anomaly-based IDS that uses statistical techniques to conduct the detection process. Despite providing many advantages, anomaly-based IDSs tend to generate a high number of false alarms. Machine Learning (ML) techniques have gained wide interest in tasks of intrusion detection. In this work, Support Vector Machine (SVM) is deemed as an ML technique that could complement the performance of our IDS, providing a second line of detection to reduce the number of false alarms, or as an alternative detection technique. We assess the performance of our IDS against one-class and two-class SVMs, using linear and non- linear forms. The results that we present show that linear two-class SVM generates highly accurate results, and the accuracy of the linear one-class SVM is very comparable, and it does not need training datasets associated with malicious data. Similarly, the results evidence that our IDS could benefit from the use of ML techniques to increase its accuracy when analysing datasets comprising of non- homogeneous features.

A machine learning approach for classifying healthy and infarcted patients using heart rate variabilities derived vector magnitude

Known as one of the most significant neurodegenerative diseases of the central nervous system, Parkinson's disease (PD) has a combination of several symptoms, such as tremor, postural instability, loss of movements, depression, anxiety, and dementia, among others. For the medicine, to point an exam that can diagnose a patient with such illness is challenging due to the symptoms that are easily related to other diseases. Therefore, developing computational methods capable of identifying PD in its early stages has been of paramount importance in the scientific community. Thence, this paper proposes to use a deep neural network called ResNet-50 to learn the patterns and extract features from images draw by patients. Afterwards, the Optimum-Path Forest (OPF) classifier is employed to identify Parkinson's disease automatically, being the results compared against two well-known classifiers, i.e., Support Vector Machines and the Bayes, as well as the ones provided by ResNet-50 itself. The experiments showed promising results concerning OPF, reachinz over 96% of identification rate.

Diabetes is a major chronic syndrome caused by a series of metabolic abnormalities in which blood glucose levels are abnormally high for an indeterminate amount of time. It influences various organs in the human body, resulting in a variety of complex diseases such as stroke, renal disease, pulmonary embolism, eyesight, and so on. Diabetes Disorders (DD) are presently one of the healthcare top causes of mortality. Predictive analytics in the health care system is a huge obstacle, but if accurate early prediction is achieved, the potential risk and degree of diabetes may be significantly decreased. Machine learning (ML) techniques are now used to analyze medical datasets at an earlier stage of life in keeping people safe. In this research, we utilized several ML approaches notably Logistic Regression, Decision Tree (DT), XGBoost, Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Random Forest (RF) on PIMA Indian Diabetes Dataset in order to monitor and evaluate their performances in diabetes prediction. The performance of the various ML algorithms employed in this research suggests which algorithm is most suitable in diabetes prediction. It is observed that among all the models XGBoost had outperformed the other ML techniques with an accuracy of 80.73% while SVM was the second-best performing model with a classification accuracy of 80.21%. Thus, employing ML techniques, this study aims to assist doctors as well as clinicians in the early detection of diabetes.

Destructive cyber-attacks and cybercriminals are increasing with the increase in IoT (Internet of Things) devices globally. This ha/s led to the need for increase in security in IoT systems. Innovative and novel techniques are used by the intruders to accomplish malicious goals effectively through cyber-attacks. An Intrusion Detection System (IDS) is used for classification of attacks in IoT networks based on anomaly detection and machine learning techniques. Inefficiency is observed in the conventional machine learning models and intrusion detection techniques as the network technologies are unpredictable. Accurate identification of various anomalies is possible with deep learning models in several research segments. The input data along with its prominent characteristics may be categorized automatically for classification and anomaly detection using convolutional neural networks (CNN). Faster computations are enabled due to the performance efficiency of CNN. For IoT networks, an intrusion detection model based on anomaly detection is designed and developed in this paper. A multiclass classification framework is created initially using a CNN model. Further, 3D CNN is used for implementation of the proposed model. Various intrusion detection datasets from IoT networks are used for validation of the proposed CNN model. Pre-trained multiclass CNN model is used for implementation of multiclass and binary classification based on transfer learning. When compared to the conventional deep learning models, the proposed multiclass and binary classification framework has attained improved F1 score, recall, precision and accuracy.

The complexity of handling modern apps is causing older network architectures to struggle. As a result, Software Defined Networking (SDN) has developed into a paradigm-shifting network framework that enables administrators to exercise programmable control over whole network infrastructures. This development makes it easier to configure and operate large networks, but it also adds vulnerabilities, including the possibility of Distributed Denial of Service (DDoS) assaults that could take down the SDN controller. Integrating security tools like firewalls, intrusion detection and prevention systems, and anti-DDoS solutions is crucial to reducing this danger. In the context of SDN, leveraging the analytical power of Machine Learning (ML) algorithms to analyse network traffic patterns is a key tactic for fending off DDoS attacks. Effective defence systems are supported by these ML models, which can distinguish between normal network behaviour and DDoS assault traffic with ease. Decision trees (DT), support vector machines (SVM), and artificial neural networks (ANN) are examples of supervised ML techniques that have evolved into crucial tools in this field. These algorithms are developed utilising datasets with annotations that include examples of both legitimate and malicious network traffic. Once trained, the models may be used to instantly categorise incoming traffic, making it easier to identify and contain any DDoS attacks in real-time.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

The research on multiple classifiers systems includes the creation of an ensemble of classifiers and the proper combination of the decisions. In order to combine the decisions given by classifiers, methods related to fixed rules and decision templates are often used. Therefore, the influence and relationship between classifier decisions are often not considered in the combination schemes. In this paper we propose a framework to combine classifiers using a decision graph under a random field model and a game strategy approach to obtain the final decision. The results of combining Optimum-Path Forest (OPF) classifiers using the proposed model are reported, obtaining good performance in experiments using simulated and real data sets. The results encourage the combination of OPF ensembles and the framework to design multiple classifier systems.