An exemplar-based clustering using efficient variational message passing

The high dimensionality of hyperspectral images brings a heavy burden for image processing. Band selection is a common technique for dimensionality reduction. Since the labels of hyperspectral images are difficult to collect, a new semisupervised band selection method based on affinity propagation (AP) is proposed. AP, an exemplar-based clustering method, is famous due to fast execution time and low reconstruction error. For band selection, AP involves two key issues: band correlation and band preference. In this paper, a new normalized trivariable mutual information (normalized TMI, NTMI) is devised to measure band correlation for classification. NTMI considers not only band redundancy but also band synergy, and overcomes the sensitivity of TMI to the discriminative abilities of bands. Band preference is defined by the discriminative ability and informative amount of each band. Since the clustering methods are easily disturbed by noisy bands, a new statistical-based method for band correlation and band preference is devised. It can automatically remove noisy bands beforehand by exploiting the continuity property of bands. Finally, the proposed method can select highly discriminative and informative bands, and remove highly redundant bands. Experimental results on hyperspectral images demonstrate the effectiveness of the proposed semisupervised band selection method.

Many important real-world applications of machine learning, statistical physics, constraint programming and information theory can be formulated using graphical models that involve determinism and cycles. Accurate and efficient inference and training of such graphical models remains a key challenge. Markov logic networks (MLNs) have recently emerged as a popular framework for expressing a number of problems which exhibit these properties. While loopy belief propagation (LBP) can be an effective solution in some cases; unfortunately, when both determinism and cycles are present, LBP frequently fails to converge or converges to inaccurate results. As such, sampling based algorithms have been found to be more effective and are more popular for general inference tasks in MLNs. In this paper, we introduce Generalized arc-consistency Expectation Maximization Message-Passing (GEM-MP), a novel message-passing approach to inference in an extended factor graph that combines constraint programming techniques with variational methods. We focus our experiments on Markov logic and Ising models but the method is applicable to graphical models in general. In contrast to LBP, GEM-MP formulates the message-passing structure as steps of variational expectation maximization. Moreover, in the algorithm we leverage the local structures in the factor graph by using generalized arc consistency when performing a variational mean-field approximation. Thus each such update increases a lower bound on the model evidence. Our experiments on Ising grids, entity resolution and link prediction problems demonstrate the accuracy and convergence of GEM-MP over existing state-of-the-art inference algorithms such as MC-SAT, LBP, and Gibbs sampling, as well as convergent message passing algorithms such as the concave---convex procedure, residual BP, and the L2-convex method.

Clustering data into meaningful subsets is a major task in scientific data analysis. To date, various strategies ranging from model-based approaches to data-driven schemes, have been devised for efficient and accurate clustering. One important class of clustering methods that is of a particular interest is the class of exemplar-based approaches. This interest primarily stems from the amount of compressed information encoded in these exemplars that effectively reflect the major characteristics of the respective clusters. Affinity propagation (AP) has proven to be a powerful exemplar-based approach that refines the set of optimal exemplars by iterative pairwise message updates. However, a critical limitation is its inability to capitalize on known networked relations between data points often available for various scientific datasets. To mitigate this shortcoming, we propose geometric-AP, a novel clustering algorithm that effectively extends AP to take advantage of the network topology. Geometric-AP obeys network constraints and uses max-sum belief propagation to leverage the available network topology for generating smooth clusters over the network. Extensive performance assessment reveals a significant enhancement in the quality of the clustering results when compared to benchmark clustering schemes. Especially, we demonstrate that geometric-AP performs extremely well even in cases where the original AP fails drastically.

Affinity Propagation (AP) is a recently proposed clustering technique, widely used in literature, which finds the cluster center by exchanging real messages between pairs of data. These messages are calculated on the basis of similarity matrix. Accordingly, the similarity matrix is considered an essential procedure of the AP. Negative Euclidean distance is used as a similarity measure, in order to construct the similarity matrix of the AP. However, most data points lie in the non-Euclidean space which becomes difficult for the Euclidean distance to acquire the real data structure. The performance of AP might be degraded if this drawback occurs. A clustering method is proposed here called Intuitionistic Fuzzy Affinity Propagation (IFAP) that uses an intuitionistic fuzzy similarity measure to construct the similarity matrix among data points. Subsequently, the similarity matrix is fed into the AP procedure, and the cluster center will emerge after a couple of iterations. The numerical experiment is demonstrated. Results show that the IFAP outperforms the other clustering method.

We present an iterative and unsupervised learning approach for the speaker verification task. In conventional speaker verification, Probabilistic Linear Discriminant Analysis (PLDA) has been widely used as a supervised backend. However, PLDA requires fully labeled training data, which is often difficult to obtain in reality. To automatically retrieve the speaker labels of unlabeled training data, we propose to use the Affinity Propagation (AP) - a clustering method that takes pairwise data similarity as input - to generate the labels for the PLDA modeling. We further propose an iterative refinement strategy that incrementally updates the similarity input of the AP clustering with the previous iteration's PLDA scoring outputs. Moreover, we evaluate the performance of different PLDA scoring methods for the multiple enrollment task and show that the generalized hypothesis testing achieves the best results. Experiments were conducted on the NIST SRE 2010 and the 2014 i-vector challenge database. The results show that our proposed iterative and unsupervised PLDA model learning approach outperformed the cosine similarity baseline by 35% relatively.

Affinity propagation (AP) algorithm, as a novel clustering method, does not require the users to specify the initial cluster centers in advance, which regards all data points as potential exemplars (cluster centers) equally and groups the clusters totally by the similar degree among the data points. But in many cases there exist some different intensive areas within the same data set, which means that the data set does not distribute homogeneously. In such situation the AP algorithm cannot group the data points into ideal clusters. In this paper, we proposed an extended AP clustering algorithm to deal with such a problem. There are two steps in our method: firstly the data set is partitioned into several data density types according to the nearest distances of each data point; and then the AP clustering method is, respectively, used to group the data points into clusters in each data density type. Two experiments are carried out to evaluate the performance of our algorithm: one utilizes an artificial data set and the other uses a real seismic data set. The experiment results show that groups are obtained more accurately by our algorithm than OPTICS and AP clustering algorithm itself.

By grouping nodes with similar attributes into clusters, the energy efficiency and lifespan of wireless sensor networks(WSNs) can be improved effectively. However, the number of clusters needs to be set in advance, and the optimal cluster heads (CHs) are difficult to determine, which will undoubtedly reduce the network’s overall performance. Therefore, a new clustering method using Affinity Propagation and Chaotic Lion swarm Optimization is proposed in this paper to form optimal clusters, which is called APCLO. In APCLO, Affinity Propagation (AP) is used to construct a cluster topology, forming the initial clustering according to the remaining energies of the nodes and the similarity of the distances between nodes. Moreover, the initial CHs are also determined simultaneously by AP. In order to eliminate the outliers from the initial CHs, Chaotic Lion Optimization(CLO) is presented to find the best CHs, in which a fitness function is set according to residual energy and distance to the BS, and chaotic map is used to speed up the convergence of CLO. Simulation results show that the APCLO protocol is superior to the comparison protocols in terms of energy consumption, network throughput, convergence speed, and lifetime. For network lifespan, it increases by 20.1%, 11.2%, and 13.5% respectively.

Biometric identification is getting more and more popular as the demand of security increases. Iris is a promising biometric because of its stability and uniqueness. In this paper, a novel iris analysis method is proposed by using Affinity Propagation (AP) Algorithm. AP algorithm is a new clustering method by transforming the input matrix of similarity between pairs of data points. The proposed method is evaluated using the Chinese Academy of Sciences-Institute of Automation (CASIA) iris image database, the similarity between two irises is measured by their negative Hamming Distance. Experiments indicate that the algorithm has excellent solution in iris analysis, when AP algorithm is used as preprocess in an iris identification system, it can greatly decrease the time complexity of iris recognition.

Clustering techniques have been applied to categorize documents on Web and extract knowledge from Web. In this paper, we introduce a novel clustering method into Web page clustering, which is an extension of affinity propagation (AP). This method is called partition adaptive affinity propagation (PAAP), which can automatically rerun AP procedure to yield optimal clustering results and eliminate number oscillations if they occur. Experiments are carried out to compare PAAP with K-means and AP on ten different Web page data sets. The results verify that PAAP can find better clusters when compared with similar methods. And the results also demonstrate that PAAP is robust and effective when clustering Web pages.

Face recognition using message passing based clustering method

We introduce a new framework for feature grouping based on factor graphs, which are graphical models that encode interactions among arbitrary numbers of random variables. The ability of factor graphs to express interactions higher than pairwise order (the highest order encountered in most graphical models used in computer vision) is useful for modeling a variety of pattern recognition problems. In particular, we show how this property makes factor graphs a natural framework for performing grouping and segmentation, which we apply to the problem of finding text in natural scenes. We demonstrate an implementation of our factor graph-based algorithm for finding text on a Nokia camera phone, which is intended for eventual use in a camera phone system that finds and reads text (such as street signs) in natural environments for blind users.

Factor graphs (FG) are graphical models with origins in coding theory. The sum-product and the max-product algorithms (SPA/MPA), which operate by message passing in an FG, subsume a great variety of algorithms in coding, signal processing, and artificial intelligence. This paper aims at showing that it is possible to give an FG/SPA interpretation of one of the best data compression algorithms, namely the context-tree weighting (CTW) method. An arithmetic encoder/decoder needs the conditional probabilities of the next symbol can be obtained by performing the SPA on the FG which represents the joint pmf/pdf of all occuring random variables. Once the CTW algorithm is formulated in the FG/SPA-framework, new extensions of the algorithm become readily apparent.

Levels Of Service (LOS) evaluation criteria for off-street pedestrian facilities are not well defined in urban Indian context; hence an in-depth research is carried out in this regard. Defining Pedestrian Level of Service (PLOS) criteria is basically a classification problem; therefore a comparative study is made using three methods of clustering i.e. Affinity Propagation (AP), Self-Organizing Map (SOM) in Artificial Neural Network (ANN) and Genetic AlgorithmFuzzy (GA-Fuzzy) clustering. Pedestrian data are used on validation measure of clustering method to obtain optimal number of cluster used in defining PLOS categories. To decide the most suitable algorithm applicable in defining PLOS criteria for urban off-street facilities in Indian context, Wilk’s Lambda is used on results of the three clustering methods. It is observed from the analysis that GA-Fuzzy is the most suitable clustering analysis among the three methods. With the help of GA-Fuzzy clustering analysis the ranges of the four measuring parameters (average pedestrian space, flow rate, speed of pedestrian and volume to capacity ratio) are defined by using the data collected from two mid-sized cities located in the state of Odisha, India. It is also observed that at >16.53 m2/ped average space, ≤0.061 ped/sec/m flow rate, >1.21 speed and ≤0.34 v/c ratio pedestrians can move in their desired path at LOS ‘A’ without changing movements and it is the best condition for off-street facilities. But in the pedestrian facility having ≤4.48 m2/ped average space, >0.146 ped/sec/m flow rate, ≤0.62 average speed and >1.00 v/c ratio, pedestrian movement is severely restricted and frequent collision among users occurs. The ranges of the parameters used for LOS categories found in this study for Indian cities are different from that mentioned in HCM (Highway Capacity Manual 2010) because of differences in population density, traffic flow condition, geometric structure and some other factors.

Hierarchical clustering, a traditional clustering method, has been getting attention again. Among several reasons, a credit goes to a recent paper by Dasgupta in 2016 that proposed a cost function that quantitatively evaluates hierarchical clustering trees. An important question is how to combine this recent advance with existing successful clustering methods. In this paper, we propose a hierarchical clustering method to minimize the cost function of clustering tree by incorporating existing clustering techniques. First, we developed an ensemble tree-search method that finds an integrated tree with reduced cost by integrating multiple existing hierarchical clustering methods. Second, to operate on large and arbitrary shape data, we designed an efficient hierarchical clustering framework, called integrating divisive and ensemble-agglomerate (IDEA) by combining it with advanced clustering techniques such as nearest neighbor graph construction, divisive-agglomerate hybridization, and dynamic cut tree. The IDEA clustering method showed better performance in minimizing Dasgupta's cost and improving accuracy (adjusted rand index) over existing cost-minimization-based, and density-based hierarchical clustering methods in experiments using arbitrary shape datasets and complex biology-domain datasets.

Human activity data discovery from triaxial accelerometer sensor: Non-supervised learning sensitivity to feature extraction parametrization