Maximum inter-class and marginal discriminant embedding (MIMDE) for feature extraction and classification

Graph embedding (GE) frameworks are used for extracting the discriminative features of hyperspectral images (HSIs). However, it is difficult to select a proper neighborhood size for graph construction. To overcome this difficulty, a semisupervised feature extraction (FE) method, called semisupervised collaborative representation graph embedding (SCRGE), is proposed. The proposed algorithm utilizes collaborative representation (CR) to obtain the collaborative coefficients of labeled and unlabeled samples. Then, a semisupervised graph is constructed using the collaborative coefficients of the labeled samples within the same class and the collaborative coefficients of the unlabeled samples, and an interclass graph is constructed using the collaborative coefficients of the labeled samples in different classes. Finally, a projection matrix for FE is obtained by embedding these graphs into a low-dimensional space. SCRGE not only inherits the merits of CR to reveal the collaborative reconstructive properties of data but also enhances intraclass compactness and interclass separability to improve the discriminating power for classification. Experimental results on three real HSIs datasets demonstrate that SCRGE outperforms other state-of-the-art FE methods in terms of classification accuracy.

The recently proposed Marginal Fisher Analysis (MFA) achieved high performance for face recognition. However, MFA suffers from the small sample size problem and the optimal basis vectors obtained by the MFA are nonorthogonal. In this paper, we present a new method called Direct Orthogonal Marginal Fisher Analysis (DOMFA), which is able to extract all the orthogonal discriminant vectors simultaneously in the high-dimensional feature space without pre-processing using PCA and does not suffer the small sample size problem. Experimental results on ORL database indicate that the proposed DOMFA method achieves better recognition rate than the MFA method and some other orthogonal feature extraction algorithms.

Local sensitive discriminative broad learning system for hyperspectral image classification

In recent years, a variety of manifold-based learning dimensionality reduction techniques have been proposed, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, marginal fisher analysis (MFA) achieved high performance for face recognition. However, the optimal basis vectors obtained by MFA are non-orthogonal and MFA usually deteriorates when labeled information is insufficient. In order to resolve these problems, we present a new method called orthogonal semi-supervised marginal fisher analysis (OSMFA), which not only extracts all the orthogonal discriminant vectors but also preserves the global structure of labeled and unlabeled samples to learn a better subspace for classification. Experimental results on ORL database demonstrate the effectiveness of the proposed algorithm.

The use of effective distance functions has been explored for many data mining problems including clustering, nearest neighbor search, and indexing. Recent research results show that if the Pearson variation of the distance distribution converges to zero with increasing dimensionality, the distance function will become unstable (or meaningless) in high dimensional space even with the commonly used Lp metric on the Euclidean space. This result has spawned many subsequent studies. We first comment that although the prior work provided the sufficient condition for the unstability of a distance function, the corresponding proof has some defects. Also, the necessary condition for unstability (i.e., the negation of the sufficient condition for the stability) of a distance function, which is required for function design, remains unknown. Consequently, we first provide in this paper a general proof for the sufficient condition of unstability. More importantly, we go further to prove that the rapid degradation of Pearson variation for a distance distribution is in fact a necessary condition of the resulting unstability. With the result, we will then have the necessary and the sufficient conditions for unstability, which in turn imply the sufficient and necessary conditions for stability. This theoretical result derived leads to a powerful means to design a meaningful distance function. Explicitly, in light of our results, we design in this paper a meaningful distance function, called Shrinkage-Divergence Proximity (abbreviated as SDP), based on a given distance function. It is empirically shown that the SDP significantly outperforms prior measures for its being stable in high dimensional data space and robust to noise, and is thus deemed more suitable for distance-based clustering applications than the priorly

In modern database applications the similarity or dissimilarity of complex objects is examined by performing distance-based queries (DBQs)on data of high dimensionality. The R-tree and its variations are commonly cited multidimensional access methods that can be used for answering such queries. Although, the related algorithms work well for low-dimensional data spaces, their performance degrades as the number of dimensions increases (dimensionality curse). In order to obtain accept- able response time in high-dimensional data spaces, algorithms that ob- tain approximate solutions can be used. Three approximation techniques (α-allowance, N-consider and M-consider) and the respective recursive branch-and-bound algorithms for DBQs are presented and studied in this paper. We investigate the performance of these algorithms for the most representative DBQs (the K-nearest neighbors query and the K-closest pairs query) in high-dimensional data spaces, where the point data sets are indexed by tree-like structures belonging to the R-tree family:R*- trees and X-trees. The searching strategy is tuned according to several parameters, in order to examine the trade-off between cost (I/O activ- ity and response time) and accuracy of the result. The outcome of the experimental evaluation is the derivation of the outperforming DBQ ap- proximate algorithm for large high-dimensional point data sets.

It is not desirable in the performance perspective of search algorithms to partition a high dimensional data space by dividing all the dimensions. This is because the number of cells resulted from partitioning explodes as the number of partitioning dimensions increases, thus making any search method based on space partitioning impractical. To address this problem, we propose an algorithm to dynamically select partitioning dimensions based on a data sampling method for efficient similarity join processing. Futhermore, a disk-based plane sweeping method is proposed to minimize the cost of joins between the partitioned cells. The experimental results show that the proposed schemes substantially improve the performance of the partition-based similarity joins in high dimensional data spaces.

In this paper, we propose a new supervised learning algorithm, which is named the Generalized Marginal Fisher Analysis (GMFA), to utilize the advantages of the Marginal Fisher Analysis (MFA) and the Generalized Singular Value Decomposition (GSVD) techniques for face recognition. The experimental results on several standard face databases demonstrate that GMFA outperforms LDA/Fisherface, LDA/GSVD and MFA.

A dataset can be clustered by merging the bucket indices that come from the random projection of locality sensitive hashing functions. It should be noted that for this to work the merging interval must be calculated first. To improve the feasibility of large scale data clustering in high dimensional space we propose an enhanced Locality Sensitive Hashing Clustering Method. Firstly, multiple hashing functions are generated. Secondly, data points are projected to bucket indices. Thirdly, bucket indices are clustered to get class labels. Experimental results showed that on synthetic datasets this method achieves high accuracy at much improved cluster speeds. These attributes make it well suited to clustering data in high dimensional space.

This paper introduces a new method of Photomosaic. In this method, we propose to use tiled images that can be rotated in a restricted range. The tiled images are selected from a database. The selection of an image is done by a hashing method based on principal component analysis of a database. After computing the principal components of the database, various kinds of hash tables based on the linear combination of the principal component are prepared beforehand. Using our hashing method, we can reduce the computation time for selecting the tiled images based on the approximated nearest neighbor searching in consideration of a distribution of data in a database. We demonstrate the effectiveness of our hashing method by using a huge number of data in high dimensional space and better looking results of our tiling in experimental results.

The paper proposes a new approach in data censoring, which allows correcting diagnostic errors in the data sets in case when these samples are described in high-dimensional feature spaces. Considering this case as a separate task is explained by the fact that in high-dimensional spaces most of the methods of outliers detection and data filtering, both statistical and metric, stop working. At the same time, for the tasks of medical diagnostics, given the complexity of the objects and phenomena studied, a large number of descriptive characteristics are the norm rather than the exception. To solve this problem, an approach that focuses on local similarity between objects belonging to the same class and uses the function of rival similarity (FRiS function) as a measure of similarity has been proposed. In this approach for efficient data cleaning from misclassified objects, the most informative and relevant low-dimensional feature subspace is selected, in which the separability of classes after their correction will be maximal. Class separability here means the similarity of objects of one class to each other and their dissimilarity to objects of another class. Cleaning data from class errors can consist both in their correction and removing the objects-outliers from the data set. The described method was implemented as a FRiS-LCFS algorithm (FRiS Local Censoring with Feature Selection) and tested on model and real biomedical problems, including the problem of diagnosing prostate cancer based on DNA microarray analysis. The developed algorithm showed its competitiveness in comparison with the standard methods for filtering data in high-dimensional spaces.

Unbalanced cluster solutions are affected by very different cluster sizes, with some clusters being very large while others contain almost no data. We demonstrate that this phenomenon is connected to ‘hubness’, a recently discovered general problem of machine learning in high dimensional data spaces. Hub objects have a small distance to an exceptionally large number of data points, and anti-hubs are far from all other data points. In an empirical study of K-medoids clustering we show that hubness gives rise to very unbalanced cluster sizes resulting in impaired internal and external evaluation indices. We compare three methods which reduce hubness in the distance spaces and show that with the balancing of the clusters evaluation indices improve. This is done using artificial and real data sets from diverse domains.

Summary The principal component analysis (PCA) is exists the problem of lack of probability model and the absence of higher-order statistics information. Therefore，in order to overcome its shortcomings，this paper studies the method that can overcome two disadvantages of the principal component analysis (PCA) — the probability kernel principal component analysis (PKPCA) which is based on Bayesian theory and kernel principal component analysis (KPCA). First，the sample data are mapped to the high dimensional feature space，then define probability model of the data in high-dimensional space，and finally，expectation maximization (EM) estimated is used to get the best results. This method has both the advantage of probability analysis and kernel principal component analysis (KPCA). It is able to effectively adapt to more complex reservoir conditions and can realize the non-linear probability analysis. The probability of kernel principal component analysis (PKPCA) is applied to reservoir prediction of the oilfield. The predicted results show that the method can improve the precision attribute optimization，while improving the accuracy of the forecasts of reservoir.

Feature selection only using wrapper method in high-dimensional data space is always time-consuming. A new feature selection method, named fast static particle swarm optimization, is proposed for tackling this problem. It treats the whole initial feature set as a static particle swarm in which no new particle would be generated in high dimensional space, and the proposed method takes filter and wrapper strategy to pick out the most discriminative feature particle subset. Compared with the existing methods, experimental results show that the proposed method is faster than the existing methods in frontal face detection, and the detection error rate is lower than them on average.

Fisher discrimination dictionary sparse learning (FDDL) has led to interesting image recognition results where the Fisher discrimination criterion is subject to the coding coefficients. But Fisher discrimination criterion has the limitations of data distribution assumptions and does not consider the local manifold structure of the coding coefficients. In this paper, we will introduce a novel Fisher discrimination sparse learning based on graph embedding (GE-FDSL) scheme. First, we utilizes graph embedding framework to define intra-class compact matrix and inter-class separable matrix imposed on the coding coefficients of training samples to preserving the intra-class compactness and the inter-class separability for the training samples, which simultaneously consider the local manifold structure and label information of the coding coefficients. Then, a new Fisher discrimination criterion based on graph embedding is added to the object function of the sparse coding problem so that the coding coefficients have more discriminative power, where the dictionary atoms in the sparse coding model are associated with the class labels so that the reconstructed error is applied to classification. This method can learn a structured dictionary and sparse coefficients, and in the meantime, it will also keep the local manifold structure of the coding coefficients. So, they will be more discriminative. Experiments on many image databases show that the our algorithm has good classification and recognition performance.