Infinite Gaussian Mixture Autoencoder for Adaptive Deep Unsupervised Clustering

Feature selection is an important task in machine learning, pattern recognition and data mining. This paper proposed a new feature selection method for classification, named SD, which is based on scatter matrix used in linear discriminant analysis. The main feature of SD is its simplicity and independency of learning algorithms. High-dimensional data samples are first projected into a lower dimensional subspace of the original feature space by means of a linear transformation matrix, which can be attained according to the scatter degree of each feature, and then the scatter degree is used to measure the importance of each feature. A comparison of SD and some popular feature selection methods (information gain and chi <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -test) is conducted, and the results of experiment carried out on 19 data sets show the advantages of SD.

Many real-world optimization problems are characterized by multiple conflicting objectives, which are known as multi-objective optimization problems (MOPs). In the last two decades, evolutionary algorithms have shown promising performance in solving various MOPs, and a large number of multi-objective evolutionary algorithms (MOEAs) have been proposed. In order to determine the most suitable MOEA for a specific MOP, it is usually necessary to perform experiments to compare the performance of multiple candidate MOEAs. In 2017, an evolutionary multi-objective optimization platform was proposed by us, called PlatEMO, which provides the source codes of many state-of-the-art MOEAs and helps researchers perform batch experiments on these MOEAs. In this work, we illustrate the method of using the newest version of PlatEMO to solve MOPs in applications, by means of a case study on the feature selection problem, which is an important and difficult task in machine learning and data mining. This paper details the method of adding the feature selection problem to PlatEMO, and presents the experimental results of eight MOEAs on nine datasets in feature selection.

Classification is a common task in Machine Learning and Data Mining. Some classification problems are called hierarchical classification problems because they need to take into account a hierarchical taxonomy which establishes an order between involved classes. The protein's function prediction is considered a hierarchical classification problem because their functions are arranged in a hierarchical taxonomy of classes. This paper presents an algorithm for hierarchical classification using the jumping emerging patterns approach. Jumping emerging patterns have been used to flat classification and in this work we explore its adoption in a hierarchical classification scenario. The proposed algorithm was evaluated in eight real datasets, compared against two other recent hierarchical classification algorithms from the literature and also with flat k-nearest neighbor classification algorithm. Preliminary results showed that the proposed approach is an alternative for hierarchical classification, having as main advantages the simplicity and understandability with good accuracy.

Community detection, which discovers densely-connected groups of nodes in networks, is a fundamental task in machine learning and data mining. Compared with plain network, community detection in attributed network presents more challenges. Several recent embedding-based methods have achieved promising community detection performance on some real attributed networks. However, there is limited understanding of how to effectively learn the combination of heterogeneity in the joint space of topology and attribute in unsupervised scenarios. In this paper, we propose an end-to-end network embedding method. By employing the high order graph convolutional networks, our method encodes the topological structure and node attributes to learn compact representations, i.e., community membership. The decoder on the other side, we reconstruct the global topological structure based on the learned community membership and stochastic block model. We further employ a self-training module, which takes the “confident” link assignments as soft labels to guide the optimizing procedure. Experiments show our method has achieved the sate-of-the-art performance on three popular datasets.

Classification is one of the main tasks in machine learning, data mining, and pattern recognition. Compared with the extensively studied automation approaches, the interactive approaches, centered on human users, are less explored. This chapter studies interactive classification at 3 levels. At the philosophical level, the motivations and a process-based framework of interactive classification are proposed. At the technical level, a granular computing model is suggested for re-examining not only existing classification problems, but also interactive classification problems. At the application level, an interactive classification system (ICS), using a granule network as the search space, is introduced. ICS allows multi-strategies for granule tree construction, and enhances the understanding and interpretation of the classification process. Interactive classification is complementary to the existing classification methods.

Learning classifiers from imbalanced data is still one of challenging tasks in machine learning and data mining. Data difficulty factors referring to internal and local characteristics of class distributions deteriorate performance of standard classifiers. Many of these factors may be approximated by analyzing the neighbourhood of the learning examples and identifying different types of examples from the minority class. In this paper, we follow recent research on developing such methods for assessing the types of examples which exploit either k-nearest neighbours or kernels. We discuss the approaches to tune the size of both kinds of neighborhoods depending on the data set characteristics and evaluate their usefulness in series of experiments with real-world and synthetic data sets. Furthermore, we claim that the proper analysis of these neighborhoods could be the basis for developing new specialized algorithms for imbalanced data. To illustrate it, we study generalizations of over-sampling in pre-processing methods and neighbourhood based ensembles.

Deep clustering is a fundamental task in machine learning and data mining that aims at learning clustering-oriented feature representations. In previous studies, most of deep clustering methods follow the idea of self-supervised representation learning by maximizing the consistency of all similar instance pairs while ignoring the effect of feature redundancy on clustering performance. In this paper, to address the above issue, we design a dual mutual information constrained clustering method named DMICC which is based on deep contrastive clustering architecture, in which the dual mutual information constraints are particularly employed with solid theoretical guarantees and experimental validations. Specifically, at the feature level, we reduce the redundancy among features by minimizing the mutual information across all the dimensionalities to encourage the neural network to extract more discriminative features. At the instance level, we maximize the mutual information of the similar instance pairs to obtain more unbiased and robust representations. The dual mutual information constraints happen simultaneously and thus complement each other to jointly optimize better features that are suitable for the clustering task. We also prove that our adopted mutual information constraints are superior in feature extraction, and the proposed dual mutual information constraints are clearly bounded and thus solvable. Extensive experiments on five benchmark datasets show that our proposed approach outperforms most other clustering algorithms. The code is available at https://github.com/Li-Hyn/DMICC.

Discretization of the value range of a numerical feature is a common task in data mining and machine learning. Optimal multivariate discretization is in general computationally intractable. We have proposed approximation algorithms with performance guarantees for training error minimization by axis-parallel hyperplanes. This work studies their efficiency and practicability. We give efficient implementations to both greedy set covering and linear programming approximation of optimal multivariate discretization. We also contrast the algorithms empirically to an efficient heuristic discretization method.

It is essential to identify similarity between graphs for various tasks in data mining, machine learning and pattern recognition. Graph edit distance (GED) is the most popular graph similarity measure thanks to its flexibility and versatility. In this paper, we study the problem of top-<i>k</i> graph similarity search, which finds <i>k</i> graphs most similar to a given query graph under the GED measure. We propose incremental GED computation algorithms that compute desired GED lower and upper bounds. Based on the algorithms, we develop novel search frameworks to address the top-<i>k</i> search problem. Our frameworks are also designed to use a state-of-the art indexing technique to speed up top-<i>k</i> search. By conducting extensive experiments on real datasets, we show that the proposed frameworks significantly improve the performance of top-<i>k</i> graph similarity search.

Application of Chi-square discretization algorithms to ensemble classification methods

The unmanned aerial vehicles (UAVs) have been extensively used in providing intelligence such as target tracking. In our field experiments, a pre-trained deep neural network (DNN) is deployed on the UAV to identify a target from the captured video frames and enable UAV to keep tracking. However, tracking in real time by the DNN requires a lot of computational resources. This motivates us to consider offloading this type of machine learning (ML) tasks to a mobile edge computing (MEC) server. Specifically, we propose a novel hierarchical ML tasks distribution framework for the UAV tracking system, where the UAV is embedded with lower layers of the pre-trained convolutional neural network (CNN) model due to its limited computing capability, while the MEC server with rich computing resources will handle the higher layers of the CNN model. An optimization problem is formulated to minimize the CNN inference delay while taking into account the communications delay, computing time, and ML error. Insights are provided to understand the tradeoff between communications and ML computing in offloading decisions. Numerical results demonstrate the effectiveness of the proposed ML tasks distribution framework with the optimized offloading strategy.

Graph edit distance is an error tolerant matching technique emerged as a powerful and flexible graph matching paradigm that can be used to address different tasks in pattern recognition, machine learning and data mining; it represents the minimum-cost sequence of basic edit operations to transform one graph into another by means of insertion, deletion and substitution of vertices and/or edges. A widely used method for exact graph edit distance computation is based on the A* algorithm. To overcome its high memory load while traversing the search tree for storing pending solutions to be explored, we propose a depth-first graph edit distance algorithm which requires less memory and searching time. An evaluation of all possible solutions is performed without explicitly enumerating them all. Candidates are discarded using an upper and lower bounds strategy. A solid experimental study is proposed; experiments on a publicly available database empirically demonstrated that our approach is better than the A* graph edit distance computation in terms of speed, accuracy and classification rate.

Proposing a classifier ensemble framework based on classifier selection and decision tree

Classifying binary imbalanced streaming data is a significant task in both machine learning and data mining. Previously, online area under the receiver operating characteristic (ROC) curve (AUC) maximization has been proposed to seek a linear classifier. However, it is not well suited for handling nonlinearity and heterogeneity of the data. In this paper, we propose the kernelized online imbalanced learning (KOIL) algorithm, which produces a nonlinear classifier for the data by maximizing the AUC score while minimizing a functional regularizer. We address four major challenges that arise from our approach. First, to control the number of support vectors without sacrificing the model performance, we introduce two buffers with fixed budgets to capture the global information on the decision boundary by storing the corresponding learned support vectors. Second, to restrict the fluctuation of the learned decision function and achieve smooth updating, we confine the influence on a new support vector to its -nearest opposite support vectors. Third, to avoid information loss, we propose an effective compensation scheme after the replacement is conducted when either buffer is full. With such a compensation scheme, the performance of the learned model is comparable to the one learned with infinite budgets. Fourth, to determine good kernels for data similarity representation, we exploit the multiple kernel learning framework to automatically learn a set of kernels. Extensive experiments on both synthetic and real-world benchmark data sets demonstrate the efficacy of our proposed approach.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2016 SIAM International Conference on Data Mining (SDM)Robust Unsupervised Feature Selection on Networked DataJundong Li, Xia Hu, Liang Wu, and Huan LiuJundong Li, Xia Hu, Liang Wu, and Huan Liupp.387 - 395Chapter DOI:https://doi.org/10.1137/1.9781611974348.44PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract Feature selection has shown its effectiveness to prepare high-dimensional data for many data mining and machine learning tasks. Traditional feature selection algorithms are mainly based on the assumption that data instances are independent and identically distributed. However, this assumption is invalid in networked data since instances are not only associated with high dimensional features but also inherently interconnected with each other. In addition, obtaining label information for networked data is time consuming and labor intensive. Without label information to direct feature selection, it is difficult to assess the feature relevance. In contrast to the scarce label information, link information in networks are abundant and could help select relevant features. However, most networked data has a lot of noisy links, resulting in the feature selection algorithms to be less effective. To address the above mentioned issues, we propose a robust unsupervised feature selection framework NetFS for networked data, which embeds the latent representation learning into feature selection. Therefore, content information is able to help mitigate the negative effects from noisy links in learning latent representations, while good latent representations in turn can contribute to extract more meaningful features. In other words, both phases could cooperate and boost each other. Experimental results on real-world datasets demonstrate the effectiveness of the proposed framework. Previous chapter Next chapter RelatedDetails Published:2016eISBN:978-1-61197-434-8 https://doi.org/10.1137/1.9781611974348Book Series Name:ProceedingsBook Code:PRDT16Book Pages:1-867