Fuzzy Data Mining in Higher Dimensions for Data Analysis

To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, the support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet a certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.

In this paper, a kernel-based nonlinear version of the oblique subspace projection (OBSP) operator is defined in terms of kernel functions. Input data are implicitly mapped into a high-dimensional kernel feature space by a nonlinear mapping, which is associated with a kernel function. The OBSP expression is then derived in the feature space, which is kernelized in terms of the kernel functions in order to avoid explicit computation in the high-dimensional feature space. The resulting kernelized OBSP algorithm is equivalent to a nonlinear OBSP in the original input space. Experimental results based on simulated hyperspectral data and real hyperspectral imagery shows that the kernel oblique subspace projection (KOBSP) outperforms the conventional OBSP.

Approaches the texture segmentation problem by clustering feature vectors created from a Gabor transform data block. Given an N*N image, the authors compute 24 Gabor transforms using Gabor kernels with six orientations and four sizes. This results in a Gabor data block composed of N/sup 2/ feature vectors of length 24. The feature vectors are then grouped based on their distribution in the high-dimensional feature space. The authors hypothesize that the pixels in a given group have similar characteristics, and thus are part of the same texture. Experimental results for segmenting a synthetic railroad track image were encouraging; a clear-cut segmentation of the image was obtained. By clustering Gabor features, the authors were able to segment an image into regions of uniform texture without prior knowledge of the types of texture, or the frequency and orientation characteristics of these textures. The clustering algorithm is a modified Kohonen self-organizing feature map. >

Stock market prediction is a very noisy problem and the use of any additional information to increase accuracy is necessary. In this paper, for the stock daily return prediction problem, the set of features is expanded to include indicators not only for the stock to be predicted itself but also a set of other stocks and currencies. Afterwards, different feature selection and classification methods are utilized for prediction. The daily close returns of the 3 most traded stocks (GARAN, THYAO, and ISCTR) in Borsa Istanbul (BIST) are predicted using indicators computed on those stocks, indicators for all the other stocks listed in the BIST100 index, and indicators on the dollar-gold prices. Twenty-five different indicators on daily stock prices are computed to form feature vectors for each trading day. These feature vectors are assigned class labels according to the daily close returns. Expanding the feature space with BIST100 stocks features results in a high dimensional feature space, with possibly noisy or irrelevant features. Therefore, feature selection methods are utilized to select the most informative features. In order to determine relevance scores of features, fast filter-based methods, gain ratio and relief, are used. Experiments are performed based on individual stock features, dollar-gold features (DG), BIST100 stock features (BIST100), and a combination of BIST100 and DG with and without feature selection. Using the gain ratio feature selection with a gradient boosting machine (GBM), the movements of GARAN stock were predicted with an accuracy of 0.599 and an F-measure of 0.614. For THYAO, the relief feature selection with the GBM gave an accuracy of 0.558, and for ISCTR, the gain ratio feature selection with logistic regression achieved an accuracy of 0.581. It was found that using BIST100 stock features boosts classification results for all stocks in terms of accuracy.

Highlights A low time complexity and high accuracy recognition method of tomato’s disease stages was proposed.Small number of disease image samples can support a stable and high accuracy method with less computation time and less memory cost.Provided a solution to establish a real-time and low energy consumption disease recognize system.Abstract. Disease stages recognition of tomato is important for the timely diagnosis and prevention of tomato diseases. In this article, the color and texture features were extracted, and the kernel mutual subspace method (KMSM) was introduced to establish a rapid Tomato’s disease stage recognition method. Firstly, the color and textural features were extracted from tomato leaf and mapped to a high-dimensional space using a Gaussian kernel function. Secondly, a nonlinear disease feature subspace was established by applying principal component analysis (PCA) based on the mapped high-dimensional space. Finally, disease stages are recognized by calculating the canonical angles between the testing subspace and reference sub-spaces. To validate recognition rate of the proposed method, we conducted 10-fold cross-validation for experiments using the tomato disease sets contained in PlantVillage and artificial intelligence (AI) Challenger 2018 datasets. Also compared with the support vector machine (SVM) and VGG 16 methods, the results show that, accuracy of those three evaluated methods on the PlantVillage datasets are 99.34%, 89.2%, and 97%, respectively; and accuracy on the AI Challenger 2018 datasets are 98.66%, 71.14%, and 73.93%, respectively. Moreover, average training and recognition time of the proposed method is 0.1496 and 0.008 s, respectively, which is faster than SVM and VGG16 methods. In conclusion, the proposed method can be carried out in real-time recognition intelligent equipment which requires less memory space, less computation time, and with higher recognition rate, and low energy consumption. Keywords: Color and textural feature, High dimensional feature space, Kernel mutual subspace method, Tomato’s disease stages recognition,.

We develop a face recognition algorithm which is insensitive to gross variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face under varying illumination direction lie in a 3-D linear subspace of the high dimensional feature space — if the face is a Lambertian surface without self-shadowing. However, since faces are not truly Lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. Rather than explicitly modeling this deviation, we project the image into a subspace in a manner which discounts those regions of the face with large deviation. Our projection method is based on Fisher's Linear Discriminant and produces well separated classes in a low-dimensional subspace even under severe variation in lighting and facial expressions. The Eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed “Fisherface” method has error rates that are significantly lower than those of the Eigenface technique when tested on the same database.

Clustering high dimensional data with sparse features is challenging because pairwise distances between data items are not informative in high dimensional space. To address this challenge, we propose two novel semi-supervised clustering methods that incorporate prior knowledge in the form of pairwise cluster membership constraints. In particular, we project high-dimensional data onto a much reduced-dimension subspace, where rough clustering structure defined by the prior knowledge is strengthened. Metric learning is then performed on the subspace to construct more informative pairwise distances. We also propose to propagate constraints locally to improve the informativeness of pairwise distances. When the new methods are evaluated using two real benchmark data sets, they show substantial improvement using only limited prior knowledge.

The k-nearest neighbor (k-NN) is a popular non-parametric benchmark classification algorithm to which new classifiers are usually compared. It is used in numerous applications, some of which may involve thousands of data vectors in a possibly very high dimensional feature space. For real-time classification a hardware implementation of the algorithm can deliver high performance gains by exploiting parallel processing and block pipelining. We present two different linear array architectures that have been described as soft parameterized IP cores in VHDL. The IP cores are used to synthesize and evaluate a variety of array architectures for a different k-NN problem instances and Xilinx FPGAs. It is shown that we can solve efficiently, using a medium size FPGA device, very large size classification problems, with thousands of reference data vectors or vector dimensions, while achieving very high throughput. To the best of our knowledge, this is the first effort to design flexible IP cores for the FPGA implementation of the widely used k-NN classifier.

The paper proposes a new approach in data censoring, which allows correcting diagnostic errors in the data sets in case when these samples are described in high-dimensional feature spaces. Considering this case as a separate task is explained by the fact that in high-dimensional spaces most of the methods of outliers detection and data filtering, both statistical and metric, stop working. At the same time, for the tasks of medical diagnostics, given the complexity of the objects and phenomena studied, a large number of descriptive characteristics are the norm rather than the exception. To solve this problem, an approach that focuses on local similarity between objects belonging to the same class and uses the function of rival similarity (FRiS function) as a measure of similarity has been proposed. In this approach for efficient data cleaning from misclassified objects, the most informative and relevant low-dimensional feature subspace is selected, in which the separability of classes after their correction will be maximal. Class separability here means the similarity of objects of one class to each other and their dissimilarity to objects of another class. Cleaning data from class errors can consist both in their correction and removing the objects-outliers from the data set. The described method was implemented as a FRiS-LCFS algorithm (FRiS Local Censoring with Feature Selection) and tested on model and real biomedical problems, including the problem of diagnosing prostate cancer based on DNA microarray analysis. The developed algorithm showed its competitiveness in comparison with the standard methods for filtering data in high-dimensional spaces.

Orthogonal discriminant analysis algorithms have recently been proposed. However, these methods don't address the singularity problem in the high dimensional feature space. In this paper, we present a new method called Direct Orthogonal Discriminant Analysis (DODA), which is able to extract all the orthogonal discriminant vectors simultaneously in the high-dimensional feature space and does not suffer the singularity problem. This method is very simple and easy to be implemented. Experimental results show that the proposed method is very competitive in comparison with some existing dimensionality reduction algorithms.

The massive photometric data collected from multiple large-scale sky surveys offer significant opportunities for measuring distances of celestial objects by photometric redshifts. However, catastrophic failure is an unsolved problem with a long history and it still exists in the current photometric redshift estimation approaches (such as the k-nearest neighbor (KNN) algorithm). In this paper, we propose a novel two-stage approach by integration of KNN and support vector machine (SVM) methods together. In the first stage, we apply the KNN algorithm to photometric data and estimate their corresponding zphot. Our analysis has found two dense regions with catastrophic failure, one in the range of zphot ϵ [0.3, 1.2] and the other in the range of zphot ϵ [1.2, 2.1]. In the second stage, we map the photometric input pattern of points falling into the two ranges from their original attribute space into a high dimensional feature space by using a Gaussian kernel function from an SVM. In the high dimensional feature space, many outliers resulting from catastrophic failure by simple Euclidean distance computation in KNN can be identified by a classification hyperplane of SVM and can be further corrected. Experimental results based on the Sloan Digital Sky Survey (SDSS) quasar data show that the two-stage fusion approach can significantly mitigate catastrophic failure and improve the estimation accuracy of photometric redshifts of quasars. The percents in different |δz| ranges and root mean square (rms) error by the integrated method are 83.47%, 89.83%, 90.90% and 0.192, respectively, compared to the results by KNN (71.96%, 83.78%, 89.73% and 0.204).

In this paper, a feature combining method which can be used in gender classification has been proposed. This method is based on examinating the importance of the pixel regions on face images. In this study, after the analysing commonly used three feature extraction methods (Local binary patterns, discrete cosine transform, histogram of oriented gradients) dimension reduction is achieved via eliminating the redundant face pixels. Then, a new feature vector is obtained by combining the regions considered to be important for each method. When the feature vector's dimension is reducted, it yields the highest success rate with 95.1% over the 1275 face images.

This paper introduces the Fast Iterative Rule-based Linguistic Classifier (FaIRLiC), a Genetic Fuzzy Rule-Based Classification System (GFRBCS) which targets at reducing the structural complexity of the resulting rule base, as well as its learning algorithm's computational requirements, especially when dealing with high-dimensional feature spaces. The proposed methodology follows the principles of the iterative rule learning (IRL) approach, whereby a rule extraction algorithm (REA) is invoked in an iterative fashion, producing one fuzzy rule at a time. The REA is performed in two successive steps: the first one selects the relevant features of the currently extracted rule, whereas the second one decides the antecedent part of the fuzzy rule, using the previously selected subset of features. The performance of the classifier is finally optimized through a genetic tuning post-processing stage. Comparative results in a hyperspectral remote sensing classification as well as in 12 real-world classification datasets indicate the effectiveness of the proposed methodology in generating high-performing and compact fuzzy rule-based classifiers, even for very high-dimensional feature spaces.

Data mining techniques can be used to discover useful information by exploring and analyzing data. The aim of this article is to propose a new fuzzy-data mining method to find a compact set consisting of fuzzy if-then classification rules with high classification capability using the genetic algorithm. Furthermore, for not reducing the usefulness of the proposed method for classification problems with high dimensional feature space, the curse dimensionality resulting from the grid partition is overcome in the proposed method by employing the principal component analysis to reduce the dimensions. Through computer simulations, it can be seen that the proposed method is comparable to the other fuzzy classification methods on the well-known iris data, the appendicitis data, and the cancer data.

This paper proposes an efficient indexing structure for CBMR (Content-Based Multimedia Retrieval), called HBI (Hierarchical Bitmap Index), in which each object is represented as a bitmap of size 2 · d · l bits, where d is the number of dimensions of object's feature vector and l is the number of bitmaps. In this bitmap representation, the feature (or attribute) value of object at each dimension is represented with a set of two bits each of which indicates whether it is relatively high ('11'), low ('00'), or neither ('01') compared to the feature values of other objects at a hierarchical organized interval. Using these compact representations of feature vectors, a lot of irrelevant objects could be quickly filtered-out by a couple of simple XOR operations, and it helps to reduce the filtering process of similarity search in high-dimensional data space. It also presents an optimization algorithm, called FQD (Filtering by Query Difference), for the similarity search with relevance feedback that reuses the previously calculated distances between the original query and all objects in the database when filtering the irrelevant objects in the successive search with modified query. It helps to further reduce the search time of CBMR with relevance feedback. Experimental results show that the similarity search using HBI is about 2 ~ 3 times faster than VA-File while guaranteeing the exact solutions, and FQD for relevance feedback helps to further reduce the elapsed time of successive similarity search compared to the one for the first search.