A Hybrid Classifier Based on Rough Set Theory and Support Vector Machines

This paper reports the scaling behavior of the fuzzy c-means based classifier (FCMC) with many parameters. FCMC is a classifier based on clustering approaches. The classification accuracy on test sets (i.e., the generalization capability) is not necessarily improved by increasing the number of clusters. Especially when the number of training samples is relatively small, not only the classification boundary over-fits the data, but also covariance matrices and cluster centers are computed incorrectly, since the number of samples in each cluster becomes smaller. Hence, the test set accuracy deteriorates. The performance of FCMC with two clusters in each class and the number of training samples less than 1000, was reported in the literature. This paper reports the scaling behavior of FCMC by testing with variously-sized training samples. The number of clusters of FCMC is increased up to eight. The number of clusters used in this paper is not very large but the number of parameters is relatively large. So, the parameters are optimized to training sets. LibSVM is one of the widely known state of the art tools for support vector machines (SVM). The test set accuracy, training time and testing time (i.e., the detection time) of FCMC are compared with LibSVM by varying the size of training sets. FCMC shows a good generalization capability, though the parameters are optimized to training sets. When the number of training samples is increased by 10 times, the training time of FCMC increases by 10 times, but that of LibSVM increases by a factor of 100. The testing time is also much shorter than LibSVM when the size of the training set is large.

본 논문에서는 RST(Rough Set Theory)과 SVM(Support Vector Machine) 알고리즘을 이용한 RSIDS (RST and SVM based Intrusion Detection System)를 설계하였다. RSIDS는 PrePro(Preprocessing) 모듈, RRG(RST based Rule Generation) 모듈, 그리고 SAD(SVM based Attack Detection) 모듈로 구성된다. PrePro 모듈은 수집한 정보를 RSIDS의 데이터 형식에 맞게 변경한다. RRG 모듈은 공격 자료를 분석하여 공격 규칙을 생성하고, 그 규칙을 이용하여 대량화된 데이터에서 공격정보를 추출하고, 그리고 추출한 공격정보를 SAD 모듈에 전달한다. SAD 모듈은 추출된 공격 정보를 이용하여 공격을 탐지하여 관리자에게 통보한다. 그 결과, 기존의 SVM과 비교해볼 때, RSIDS는 평균 공격 탐지율 77.71%에서 85.28%로 향상되었으며, 평균 FPR은 13.25%에서 9.87%로 감소하였다. 따라서 RSIDS는 기존의 SVM을 이용한 공격 탐지 기법보다 향상되었다고 할 수 있다. This paper proposes a design of RSIDS(RST and SVM based Intrusion Detection System) using RST(Rough Set Theory) and SVM(Support Vector Machine) algorithm. The RSIDS consists of PrePro(PreProcessing) module, RRG(RST based Rule Generation) module, and SAD(SVM based Attack Detection) module. The PrePro module changes the collected information to the data format of RSIDS. The RRG module analyzes attack data, generates the rules of attacks, extracts attack information from the massive data by using these rules, and transfers the extracted attack information to the SAD module. The SAD module detects the attacks by using it, which the SAD module notifies to a manager. Therefore, compared to the existing SVM, the RSIDS improved average ADR(Attack Detection Ratio) from 77.71% to 85.28%, and reduced average FPR(False Positive ratio) from 13.25% to 9.87%. Thus, the RSIDS is estimated to have been improved, compared to the existing SVM.

Load forecasting is key for the optimal control of heating, ventilating, and air-conditioning. However, accurate forecasting modeling requires the ability of selecting relevant factors so that the influences of the irrelevant factors can be reduced substantially. This paper presents a novel hybrid method combining rough set theory and support vector machine network (RSSV). Rough set (RS) theory is applied to find the relevant factors of the load, which are used as the inputs of support vector machine (SVM) to predict cooling load. To improve the accuracy and enhance the robustness of load forecasting results, a general load prediction model synthesizing multi-RSSV (MRSSV) is presented so as to make full use of redundant information. Optimum principle is employed to deduce the weights of each RSSV model. Actual prediction results for a real HVAC system show that the MRSSV forecasting model, whose relative error turned out to be within 4%, may be better than those models using individual RSSV, recursive least squares (RLS), and autoregressive integrated moving average (ARIMA). In addition, individual RSSV forecasting results are also better than that of ARIMA.

In this paper, a classifier based on a combination of Rough Set and 1-v-1 (one-versus-one) Support Vector Machine for Content Based Image Retrieval system is presented. Some problems of 1-v-1 Support Vector Machine can be reduced using Rough Set. With Rough Set, a 1-v-1 Support Vector Machine can provide good results when dealing with incomplete and uncertain data and features. In addition, boundary region in Rough Set can reduce the error rate. Storage requirements are reduced when compared to the conventional 1-v-1 Support Vector Machine. This classifier has better semantic interpretation of the classification process. We compare our Content Based Image Retrieval system with other image retrieval systems that uses neural network, K-nearest neighbour and Support Vector Machine as the classifier in their methodology. Experiments are carried out using a standard Corel dataset to test the accuracy and robustness of the proposed system. The experiment results show the proposed method can retrieve images more efficiently than other methods in comparison.

Rough set theory (RST), support vector machine (SVM), and decision tree (DT) are brightly data mining methodologies for classification prediction tasks. While the accuracy for class prediction is highly emphasized, the ability to generate rules for decision support is also important in some practical applications. Studies have shown the ability of RST for feature selection while SVM and DT are significantly on their predictive power. Moreover, the ability of DT for rule generation is an attractive function. This study intents to integrate the advantages of RST, SVM and DT approaches to develop a hybrid data mining approach to improve the performance of class prediction as well as rule generation.

A hybrid approach of DEA, rough set and support vector machines for business failure prediction

Support vector machines (SVMs) has been accepted as a fashionable method in machine learning community. However, it cannot be easily scaled to handle large scale problems for its time and space complexity that is around quadratic with respect to the number of training samples. This paper proposes to combine distributed SVMs by simple Bayesian formalism rules (B-SVMs). B-SVMs randomly decomposes a large-scale task into many smaller and simpler sub-tasks in training phase and uses simple Bayesian formalism rules to make decision for final classification in test phase. B-SVMs was compared with single SVMs that is trained on entire training data set, parallel SVMs combined by majority voting (MV-SVMs), and one kind of fast modular SVMs (FM-SVMs). Experimental results on four problems show that B-SVMs can get higher accuracy than MV-SVMs and FM-SVMs does, the proposed algorithm can significantly reduce training and test time. More importantly, it produces test accuracy that is almost the same as single SVMs does.

The risk assessment of project is the important content for project management. This paper combines rough sets theory and support vector machine. The paper selects rough sets (RS) and support vector machine (SVM) algorithms to establish a new mathematical model for risk assessment of project. Using the rough sets to reduce numbers of indicators of risk factors, which reduces the dimensions of the input space. When treating the reduced data as the input space of SVM, we find that both the convergence speed and the assessment accuracy are enhanced. The results of Matlab simulation show the superiority of the model. The model based on rough sets and support vector machine can effectively help project managers for management of project risk.

There are usually a small amount and short sequence of prototype monitoring data on structural behavior during dangerous dam reinforcement. According to above data characteristics, some methods, such as support vector machine, particle swarm optimization, genetic algorithm, rough set, are combined to build the real-time monitoring model of safety status during structural reinforcement of dangerous dam. Firstly, the construction principle on standard support vector machine-based monitoring model of safety status is demonstrated. To improve the modeling accuracy and efficiency, particle swarm optimization and genetic algorithm are introduced to implement the support vector machine parameters optimization. Secondly, the ability on data mining in rough set is developed to determine the input vector of support vector machine. An approach is presented to obtain the simplified non-linear mapping relationship between dam behavior (effect-quantity) and its cause (influence-quantity). An index and its calculating formula are proposed to measure the influence-quantity importance. A real-time monitoring model based on rough set-support vector machine is established to describe reasonably the working mechanism during dangerous dam reinforcement. Lastly, the fitting and prediction capability of above monitoring model is demonstrated with an actual case.

Ordinal regression is one of the most important tasks of relation learning, and several techniques based on support vector machines (SVMs) have also been proposed for tackling it, but the scalability aspect of these approaches to handle large datasets still needs much of exploration. In this paper, we will extend the recent proposed algorithm Core Vector Machine (CVM) to the ordinal-class data, and propose a new algorithm named as Ordinal-Class Core Vector Machine (OCVM). Similar with CVM, its asymptotic time complexity is linear with the number of training samples, while the space complexity is independent with the number of training samples. We also give some analysis for OCVM, which mainly includes two parts, the first one shows that OCVM can guarantee that the biases are unique and properly ordered under some situation; the second one illustrates the approximate convergence of the solution from the viewpoints of objective function and KKT conditions. Experiments on several synthetic and real world datasets demonstrate that OCVM scales well with the size of the dataset and can achieve comparable generalization performance with existing SVM implementations.

This paper deals with the problem of fault detection and diagnosis in sensors considering erratic, drift, hard-over, spike, and stuck faults. The data set containing samples of the above mentioned fault signals was acquired as follows: normal data signals were obtained from a temperature-to-voltage converter by using an Arduino Uno microcontroller board and MATLAB. Then, faults were simulated in normal data to get 100 samples of each fault, in which one sample is composed of 1000 data elements. A support vector machine (SVM) was used for data classification in a one-versus-rest manner. The statistical time-domain features, extracted from a sample, were used as a single observation for training and testing SVM. The number of features varied from 5 to 10 to examine the effect on accuracy of SVM. Three different kernel functions used to train SVM include linear, polynomial, and radial-basis function kernels. The fault occurrence event in fault samples was chosen randomly in some cases to replicate a practical scenario in industrial systems. The results show that an increase in the number of features from 5 to 10 hardly increases the total accuracy of the classifier. However, using ten features gives the highest accuracy for fault classification in an SVM. An increase in the number of training samples from 40 to 60 caused an overfitting problem. The k-fold cross-validation technique was adopted to overcome this issue. The increase in number of data elements per sample to 2500 increases the efficiency of the classifier. However, an increase in the number of training samples to 400 reduces the capability of SVM to classify stuck fault. The receiver operating characteristics curve comparison shows the efficiency of SVM over a neural network.

Kernel machines have gained much popularity in applications of machine learning. Support vector machines (SVMs) are a subset of kernel machines and generalize well for classification, regression, and anomaly detection tasks. The training procedure for traditional SVMs involves solving a quadratic programming (QP) problem. The QP problem scales super linearly in computational effort with the number of training samples and is often used for the offline batch processing of data. Kernel machines operate by retaining a subset of observed data during training. The data vectors contained within this subset are referred to as support vectors (SVs). The work presented in this paper introduces a subset selection method for the use of kernel machines in online, changing environments. Our algorithm works by using a stochastic indexing technique when selecting a subset of SVs when computing the kernel expansion. The work described here is novel because it separates the selection of kernel basis functions from the training algorithm used. The subset selection algorithm presented here can be used in conjunction with any online training technique. It is important for online kernel machines to be computationally efficient due to the real-time requirements of online environments. Our algorithm is an important contribution because it scales linearly with the number of training samples and is compatible with current training techniques. Our algorithm outperforms standard techniques in terms of computational efficiency and provides increased recognition accuracy in our experiments. We provide results from experiments using both simulated and real-world data sets to verify our algorithm.

This paper presents the results of a research into the use of rough set (RS) for the precision coupling system modelling based on support vector machine (SVM). Because of the impact of multiple geometric parameters, there are more input variables in the precision coupling system modelling process. The high-dimensional data poses an interesting challenge to machine learning, as the presence of large numbers of redundant or highly correlated variables can seriously degrade modelling accuracy. In this study, a modelling method was developed based on rough set and support vector machine for precision coupling system. We used RS as the pre-processor for precision coupling system modelling, so as to realise dimension reduction of the high-dimensional data and improve the predictive performance of machine learning method, and then SVM was used for precision coupling system modelling. Experiments were carried out on a typical precision coupling, hydraulic valve. The results show that the use of RS method can improve the performance of machine learning in the modelling of high-dimensional data.

This study evaluates the performance of a Support Vector Machine (SVM) classifier to learn and detect changes in single- and multi-temporal X- and L-band Synthetic Aperture Radar (SAR) images under varying conditions. The purpose is to provide guidance on how to train a powerful learning machine for change detection in SAR images and to contribute to a better understanding of potentials and limitations of supervised change detection approaches. This becomes particularly important on the background of a rapidly growing demand for SAR change detection to support rapid situation awareness in case of natural disasters. The application environment of this study thus focuses on detecting changes caused by the 2011 Tohoku earthquake and tsunami disaster, where single polarized TerraSAR-X and ALOS PALSAR intensity images are used as input. An unprecedented reference dataset of more than 18,000 buildings that have been visually inspected by local authorities for damages after the disaster forms a solid statistical population for the performance experiments. Several critical choices commonly made during the training stage of a learning machine are being assessed for their influence on the change detection performance, including sampling approach, location and number of training samples, classification scheme, change feature space and the acquisition dates of the satellite images. Furthermore, the proposed machine learning approach is compared with the widely used change image thresholding. The study concludes that a well-trained and tuned SVM can provide highly accurate change detections that outperform change image thresholding. While good performance is achieved in the binary change detection case, a distinction between multiple change classes in terms of damage grades leads to poor performance in the tested experimental setting. The major drawback of a machine learning approach is related to the high costs of training. The outcomes of this study, however, indicate that given dynamic parameter tuning, feature selection and an appropriate sampling approach, already small training samples (100 samples per class) are sufficient to produce high change detection rates. Moreover, the experiments show a good generalization ability of SVM which allows transfer and reuse of trained learning machines.

In the fast-changing era of data communication, cyber-attacks and unauthorized access pose a significant threat to system and network security. Conventional Intrusion Detection Systems (IDS) are plagued by low accuracy and high false positives, which erode their usefulness in real-time intrusion detection. To overcome this issue, a new hybrid approach is proposed where Rough Set Theory (RST) is used for feature selection and Support Vector Machine (SVM) for classification. The main aim is to improve the detection accuracy while, at the same time, minimizing the complexity and dimensionality of data and processing. RST successfully filters out the most important features from the network traffic data and eliminates redundant or irrelevant attributes before forwarding the pre-processed data to the SVM classifier. This, besides lowering the processing complexity, also enhances the classification performance. The novelty lies in the combined application of RST and SVM, hence offering a more efficient alternative to existing methods like Principal Component Analysis (PCA). The proposed IDS exhibits enhanced detection accuracy with an appreciable reduction in false positives, hence offering a more efficient and reliable intrusion detection mechanism.