Integrating Global and Local Structures in Semi-supervised Discriminant Analysis

Principal Component Analysis Network (PCANet) is a feature learning algorithm which is widely used in face recognition and object classification. However, original PCANet still has some shortages. One is that PCA algorithm only extracts features by considering the global structure. The other lies in that the original PCANet only employs one particular single layer convolutional results, which loses the information of other convolutional layers. In this paper, we propose a new simple and efficient convolutional neural network called global and local structure network (GLSNet) to address the problems. The network extracts the features both from the global structure and the local structure of the original data space. Specifically, a principal component analysis (PCA) convolutional layer which learns the filters by PCA algorithm is used to remove the noises and redundant information at the first stage. Then at the second stage, another PCA convolution is added to extract features by considering the global structure. As for the local structure, we use the neighborhood preserving embedding (NPE) algorithm to learn the convolutional filters. At the output stage, the global structure feature extracted by PCA convolution and the local structure feature extracted by NPE convolution is concatenated as a united feature. Furthermore, the first layer convolutional feature is also taken into consideration to obtain shallow-level information. Finally, these features are concatenated as a united feature, and a spatial pyramid pooling layer is followed to pool above the united features. To test the effectiveness of the proposed algorithm, the experiments on some image datasets, including three types: human face dataset, object dataset, and handprinted dataset, proceeded. And it performs better than the original PCANet and some improvement algorithms of PCANet, such as PLDANet, and MMPCANet.

A semi-supervised linear dimensionality reduction method based on side information and neighbourhood preserving is proposed. In this problem, only must-link constraints (pairs of instances belong to the same class) and cannot-link constraints (pairs of instances belong to different classes) are given. The proposed neighbourhood preserving based semi-supervised dimensionality reduction algorithm can not only preserve the must-link and cannot-link constraints but can preserve the local structure of the input data in the low dimensional embedding subspace. Experimental results on several datasets demonstrate the effectiveness of the method.

Tracking is recently formulated as a problem of discriminating the object from its nearby background, where the classifier is updated by new samples successively arriving during tracking. Depending on whether labeling the samples or not, the tracker can be designed in a supervised or semi-supervised manner. This paper proposes a novel semi-supervised algorithm for tracking by combining Semi-supervised Discriminant Analysis (SDA) with an online boosting framework. Using the local geometric structure information from the samples, the SDA-based weak classifier is made more robust to outliers. Meanwhile, we design an incremental updating mechanism for SDA so that it can adapt to appearance changes. We further propose an Extended SDA (ESDA) algorithm, which gives better discrimination ability. Results on several challenging video sequences demonstrate the effectiveness of the method.

Anchor-based multi-view graph clustering (AMVGC) has received abundant attention owing to its high efficiency and the capability to capture complementary structural information across multiple views. Intuitively, a high-quality anchor graph plays an essential role in the success of AMVGC. However, the existing AMVGC methods only consider single-structure information, i.e., local or global structure, which provides insufficient information for the learning task. To be specific, the over-scattered global structure leads to learned anchors failing to depict the cluster partition well. In contrast, the local structure with an improper similarity measure results in potentially inaccurate anchor assignment, ultimately leading to sub-optimal clustering performance. To tackle the issue, we propose a novel anchor-based multi-view graph clustering framework termed Efficient Multi-View Graph Clustering with Local and Global Structure Preservation (EMVGC-LG). Specifically, a unified framework with a theoretical guarantee is designed to capture local and global information. Besides, EMVGC-LG jointly optimizes anchor construction and graph learning to enhance the clustering quality. In addition, EMVGC-LG inherits the linear complexity of existing AMVGC methods respecting the sample number, which is time-economical and scales well with the data size. Extensive experiments demonstrate the effectiveness and efficiency of our proposed method.

Deep multi-view subspace clustering has achieved promising performance compared with other multi-view clustering. However, existing deep multi-view subspace clustering only considers the global structure for all views, and they ignore the local geometric structure among each view. In addition, they cannot learn discriminative feature on different clusters of different views, i.e., inter-cluster difference. To solve these problems, in this paper, we propose a novel Deep Multi-view Subspace Clustering with Unified and Discriminative Learning (DMSC-UDL). DMSC-UDL combines global and local structures with self-expression layer. The global and local structures help each other forward and achieve small distance between samples of the same cluster. To make samples in different clusters of different views farther, DMSC-UDL uses a discriminative constraint between different views. In this way, DMSC-UDL makes the same cluster's samples have large weights, while different clusters’ samples have small weights. Thus, it can learn a better shared connection matrix for multi-view clustering. Extensive experimental results reveal that the proposed multi-view clustering method is superior to several state-of-the-art multi-view clustering methods in terms of performance.

A semantics-based method for density-based clustering with constraints imposed by geographical background knowledge is proposed. In this paper, we apply an ontological approach to the DBSCAN (Density-Based Geospatial Clustering of Applications with Noise) algorithm in the form of knowledge representation for constraint clustering. When used in the process of clustering geographic information, semantic reasoning based on a defined ontology and its relationships is primarily intended to overcome the lack of knowledge of the relevant geospatial data. Better constraints on the geographical knowledge yield more reasonable clustering results. This article uses an ontology to describe the four types of semantic constraints for geographical backgrounds: “No Constraints”, “Constraints”, “Cannot-Link Constraints”, and “Must-Link Constraints”. This paper also reports the implementation of a prototype clustering program. Based on the proposed approach, DBSCAN can be applied with both obstacle and non-obstacle constraints as a semi-supervised clustering algorithm and the clustering results are displayed on a digital map.

The diesel engine fault data is nonlinear and it’s difficult to extract the characteristic information. Kernel Principal Component Analysis (KPCA) is used to extract features of nonlinear data, only considering global structure. Kernel Locality Preserving Projection (KLPP) considers the local feature structure. So an improved algorithm for global and local structure preserving is proposed to extract the feature of data. The improved feature extraction algorithm combining KPCA and KLPP, avoids the loss of information considering the global and local feature structure and then uses the modified K-NN algorithm for fault classification. In this paper, the software AVL BOOST is used to simulate the faults of diesel engine. The simulation experiments indicate the proposed method can extract the feature vectors effectively, and shows good classification performance.

Semisupervised Discriminant Analysis (SDA) aims at dimensionality reduction with both limited labeled data and copious unlabeled data, but it may fail to discover the intrinsic geometry structure when the data manifold is highly nonlinear. The kernel trick is widely used to map the original nonlinearly separable problem to an intrinsically larger dimensionality space where the classes are linearly separable. Inspired by low-rank representation (LLR), we proposed a novel kernel SDA method called low-rank kernel-based SDA (LRKSDA) algorithm where the LRR is used as the kernel representation. Since LRR can capture the global data structures and get the lowest rank representation in a parameter-free way, the low-rank kernel method is extremely effective and robust for kinds of data. Extensive experiments on public databases show that the proposed LRKSDA dimensionality reduction algorithm can achieve better performance than other related kernel SDA methods.

Link prediction based on bipartite networks can not only mine hidden relationships between different types of nodes, but also reveal the inherent law of network evolution. Existing bipartite network link prediction is mainly based on the global structure that cannot analyze the role of the local structure in link prediction. To tackle this problem, this paper proposes a deep link-prediction (DLP) method by leveraging the local structure of bipartite networks. The method first extracts the local structure between target nodes and observes structural information between nodes from a local perspective. Then, representation learning of the local structure is performed on the basis of the graph neural network to extract latent features between target nodes. Lastly, a deep-link prediction model is trained on the basis of latent features between target nodes to achieve link prediction. Experimental results on five datasets showed that DLP achieved significant improvement over existing state-of-the-art link prediction methods. In addition, this paper analyzes the relationship between local structure and link prediction, confirming the effectiveness of a local structure in link prediction.

Discriminative feature extraction based on sparse and low-rank representation

Supervised feature extraction algorithms usually require lots of labeled samples to achieve good performance. However, labeling the samples is often time-consuming and even impractical. Therefore, in this paper, a semi-supervised manifold local Fisher discriminant analysis (SMLFDA) is proposed to take advantage of unlabeled samples as well as labeled samples. The proposed algorithm utilizes local scatter matrix and manifold structure to extract the information from labeled and unlabeled samples, respectively, which significantly improves the accuracy of successive classification application when labeled samples are insufficient. In addition, an exponential form weighting coefficient is proposed to further improve the classification performance. Experiments of hyperspectral classification demonstrate the effectiveness of the proposed semi-supervised feature extraction algorithm.

Adaptive semi-supervised dimensionality reduction with sparse representation using pairwise constraints

Local structures and global structures of the data set are both important information for learning from data. However, most manifold learning algorithms, such as LLE, Laplacian Eigenmap, LTSA, et.al paid great attention to preserving the local structures of data set, but neglected the global structure of the data set. ISOMap considers both the local structures and global structures; however, the constraint of preserving the global manifold distances is so strict that ISOMap would fail on some manifolds that cannot isometrically map to a lower dimensional Euclidean space. In this paper, we proposed a new method - Distance Penalization Embedding, which preserves the global structures of data sets in a more flexible way under the constraint of local structure preserving. Experimental results on the data sets with high nonlinearity show good performances of the proposed method.

Traditional dimensionality reduction algorithms such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA) and Locality Preserving Projections (LPP) have been employed in many fields such as biometrics patter recognition. But those methods only effectively preserve the global Euclidean structure or local structure of data set. In this paper, a novel unsupervised subspace method called Maximum Margin Subspace Projection (MMSP) is proposed. MMSP aims at preserving the local structure on the data manifold while maximize the global information of the samples simultaneously by maximizing the margin between the global structure and local structure of data manifold. Thus two abilities of manifold learning and classification have been combined into the proposed of our MMSP algorithm. Extensive experimental results on face databases demonstrate the effectiveness of the proposed algorithm.

The maintenance of Case-Based Reasoning (CBR) systems has attracted increasing interest within current research since they proved high-quality results in different real-world domains. This kind of systems stores previous experiences, which are described by a vocabulary (e.g., attributes), incrementally in a case base. Actually, the vocabulary presents one among the most important maintenance targets, since it highly contributes in providing accurate solutions and in improving systems’ performance, especially within high-dimensional domains. However, there is no policy, in the literature, that offers the ability to exploit prior knowledge (e.g., given by domain-experts) during the maintenance of features describing cases. In this paper, we propose a flexible policy for the most relevant attribute selection based on the attribute clustering concept. This new policy is able, on the one hand, to manage uncertainty using the belief function theory based tools, and on the other hand, to make use of domain-experts knowledge in form of pairwise constraints: If two attributes offer the same information without any added-value, then a Must-link constraint between them is generated. Otherwise, if there is no relation between them and they offer different information, then a Cannot-link constraint between them is created.