Genetic Programming for Evolving Similarity Functions for Clustering: Representations and Analysis

Content Based Image Retrieval (CBIR) is a process in which for a given query image similar images will be retrieved based on the image content similarity. Image content refers to its visual features, which are mathematical representations of a digital image. The image retrieval task primarily depends on image feature extraction and similarity measurement between the feature vectors. The performance of CBIR process not only decided by the optimum features extracted from the image but also on the proper choice of the similarity/dissimilarity measures (distance metrics). As the image features broadly diversified in terms of color, texture and shape based, using the same distance metric may not work well for all of these features. In this paper, first we presented overview of geometric and statistical distance metrics used in CBIR along with the comparative analysis of these measures on color and texture features. Color features extracted by computing color histograms in HSV space and texture features by wavelet decompositions. Geometrical distances such as Manhattan, Chebyshev, Euclidean, statistical distance metrics such as Cosine Similarity, Chi-square, KullbackLeibler, Jeffrey and cumulative statistical distance metrics of Kolmogorov-Smirnov, Cramer von Mises and Earthmover's distances were analyzed for feature similarity. We gave certain conclusions on the performance of all these distance metrics in terms of Mean Average Precision (MAP) and Recall rates with color and texture features respectively.

Mining, discovering, and integrating process-oriented services has attracted growing attention in the recent years. Workflow precedence graph and workflow block structures are two important factors for comparing and mining processes based on distance similarity measure. Some existing work has done on comparing workflow designs based on their precedence graphs. However, there lacks of standard distance metrics for comparing workflows that contain complex block structures such as parallel OR, parallel AND. In this paper we present a quantitative approach to modeling and capturing the similarity and dissimilarity between different workflow designs, focusing on similarity and dissimilarity between the block structures of different workflow designs. We derive the distance-based similarity measures by analyzing the workflow block structure of the participating workflow processes in four consecutive phases. We first convert each workflow dependency graph into a block tree by using our block detection algorithm. Second, we transform the block tree into a binary tree to provide a normalized reference structure for distance based similarity analysis. Third, we construct a binary branch vector by encoding the binary tree. Finally, we calculate the distance metric between two binary branch vectors.KeywordsBusiness ProcessBinary TreeDependency GraphBusiness Process ManagementBlock TreeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The Recognition and Classification of Objects Based on the Modified Distance Metric

The security and integrity of computer systems and networks highly depend on malware detection. In the realm of malware detection, the K-Nearest Neighbors (KNN) algorithm is a well-liked and successful machine learning algorithm. However, the choice of an acceptable distance metric parameter has a significant impact on the KNN algorithm's performance. This study tries to improve malware detection by adjusting the KNN algorithm's distance metric parameter. The distance metric greatly influences the similarity or dissimilarity between instances in the feature space. The KNN algorithm for malware detection can be more accurate and effective by carefully choosing or modifying the distance metric. This paper analyzes multiple distance metrics, including Minkowski distance, Manhattan distance, and Euclidean distance. These metrics account for the traits of malware samples while capturing various aspects of similarity. The effectiveness of the KNN algorithm is evaluated using the MalMem-2022 malware dataset, and the results are broken down into these three-distance metrics. The experimental findings show that, among the three distance metric parameters, the Euclidean and Minkowski distance metric parameters considerably produced the best outcomes with binary classification. While with multiclass classification, the KNN algorithm has achieved the highest outcomes using Manhattan distance.

Robust microarray data feature selection using a correntropy based distance metric learning approach

This paper represents a new approach for face recognition that incorporates Prewitt edge detection, Gabor filter and Zernike moments to transform the image into a unified domain. On this joint domain, five distance metrics are constructed using Schoenberg transform for the purpose of defining efficient similarity measures for holistic face recognition. The proposed Schoenberg similarity applies Schoenberg transform to the logarithm of five existing distance metrics: Minkowski, City-Block, Euclidean, Soergel and Lorentzian metrics. The constructed Schoenberg logarithmic metrics are called SL-Minkowski, SL-City-Block, SL-Euclidean, SL-Soergel and SL-Lorentzian. These distance metrics are utilized as similarity measures after being normalized over the range [0,1] for fair comparison with existing measures. The proposed Schoenberg system can resist three problems: Change in illumination, pose and facial expression. Simulation results show that the proposed distance measures have superior performance as compared to the classical metrics: Structural Similarity Index Measure (SSIM) and Feature-based Similarity Measure (FSIM). Performance criteria are the recognition rate and the recognition confidence, defined as the similarity difference between the best match and the second-best match in the face database.

Feature selection and construction are important pre-processing techniques in data mining. They allow not only dimensionality reduction but also classification accuracy and efficiency improvement. While feature selection consists in selecting a subset of relevant features from the original feature set, feature construction corresponds to the generation of new high-level features, called constructed features, where each one of them is a combination of a subset of original features. However, different features can have different abilities to distinguish different classes. Therefore, it may be more difficult to construct a better discriminating feature when combining features that are relevant to different classes. Based on these definitions, feature construction could be seen as a BLOP (Bi-Level optimization Problem) where the feature subset should be defined in the upper level and the feature construction is applied in the lower level by performing mutliple followers, each of which generates a set class dependent constructed features. In this paper, we propose a new bi-level evolutionary approach for feature construction called BCDFC that constructs multiple features which focuses on distinguishing one class from other classes using Genetic Programming (GP). A detailed experimental study has been conducted on six high-dimensional datasets. The statistical analysis of the obtained results shows the competitiveness and the outperformance of our bi-level feature construction approach with respect to many state-of-art algorithms.

In the current years, advancements in medical imaging have prompted the rise of some massive databases, including the pictures from a different scope of modalities. Content based image retrieval (CBIR) is a kind of search that selects examples from an image set that have a similar content to an input query image. Each CBIR system consists of two substantial factors, feature extraction and similarity measure. Based on the type of these parts, many CBIR systems are proposed. In this paper we present not only how to design a CBIR system for MRI brain Tumor images, but also a comparison between the different implemented CBIR systems and ours. Furthermore, in this research, using a DML (Distance Metric Learning) instead of the rigid distance metric such as Euclidean distance as the Similarity measure has helped the researchers to obtain a better mAP (mean Average Precision) as the result. The experimental results of our CBIR system show that it has a better performance than other rivals in the field. The obtained mean average precision of our CBIR system is 92.41 that is significant in MRI brain tumor.

Clustering have been proven to be an effective technique for finding data instances with similar characteristics. Such algorithms are based on the notion of distance between data points, often computed using Euclidean metric. That is why, clustering algorithms are mostly applicable to the data sets comprising of numerical values. However, the real life data often consist of features which are categorical in nature. For example, to identify abnormal behavior or a cyberattack in a network, we usually examine packet headers which contain categorical values such as source and destination IP addresses, source and destination port numbers, upper layer protocols, etc. Euclidean metric is not applicable to such data sets because it cannot compute the distance between categorical variables. To address this problem, similarity functions have been designed to determine the relationship between given categorical values. Similarity defines how closely related the objects are to one another. Often similarity could be thought of as opposite to distance where similar objects have high value, while dissimilar objects have low or zero value. In this paper we explored accuracy of various similarity functions using the Partitioning Around Medoids (PAM) clustering algorithm. We tested similarity functions on several data sets to determine their ability to correctly predict the class labels. We also examined the applicability of various similarity functions to different types of data sets.

Feature construction and feature selection are common pre-processing techniques to obtain smaller but better discriminating feature sets than the original ones. These two techniques are essential in high-dimensional data with thousands or tens of thousands of features where there may exist many irrelevant and redundant features. Genetic programming (GP) is a powerful technique that has shown promising results in feature construction and feature selection. However, constructing multiple features for high-dimensional data is still challenging due to its large search space. In this paper, we propose a GP-based method that simultaneously performs multiple feature construction and feature selection to automatically transform high-dimensional datasets into much smaller ones. Experiment results on six datasets show that the size of the generated feature set is less than 4% of the original feature set size and it significantly improves the performance of K-Nearest Neighbour, Naive Bayes and Decision Tree algorithms on 15 out of 18 comparisons. Compared with the single feature construction method using GP, the proposed method has better performance on half cases and similar on the other half. Comparisons between the constructed features, the selected features and the combination of both constructed and selected features by the propose method reveal different preferences of the three learning algorithms on these feature sets.

Nearest Neighbour (NN) classification is a widely-used, effective method for both binary and multi-class problems. It relies on the assumption that class conditional probabilities are locally constant. However, this assumption becomes invalid in high dimensions, and severe bias can be introduced, which degrades the performance of the method. The employment of a locally adaptive distance metric becomes crucial in order to keep class conditional probabilities approximately uniform, whereby better classification performance can be attained. This paper presents a locally adaptive distance metric for NN classification based on a supervised learning algorithm (Genetic Programming) that learns a vector of feature weights for the features composing an instance query. Using a weighted Euclidean distance metric, this has the effect of adaptive neighbourhood shapes to query locations, stretching the neighbourhood along the directions for which the class conditional probabilities don’t change much. Initial empirical results on a set of real-world classification datasets showed that the proposed method enhances the generalisation performance of standard NN algorithm, and that it is a competent method for pattern classification as compared to other learning algorithms.KeywordsOutput VectorNear NeighbourRadial Basis Function NetworkFeature WeightQuery PatternThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Simulated annealing spectral clustering algorithm for image segmentation

The similarity metric between videos is integral to several key tasks, including video retrieval, classification and recommendation. Since there is no standard criterion for the similarity measurement between videos except measuring manually, it is difficult to collect large training dataset for distance metric learning algorithms. Moreover, the existing distance metric learning (DML) methods for multimedia data suffer from two critical limitations: (1) they typically attempt to learn a distance function on the single label setting, in which each item is only labeled with single label; (2) they are often designed for learning distance metrics on low-level features, which ignore the semantic similarity of the multimedia data. To address these problems, in this paper, we propose a novel framework of Intermediate Semantics based Distance Learning (ISDL) for video clips, which aims to integrate semantics of multiple modals optimally for distance metric learning. In particular, the proposed framework: (1) generates the training pairs automatically; (2) defines multi-modal concepts for similarity measure among videos; (3) learns the distance metric for video clips based on the intermediate semantics. We conduct an extensive set of experiments to evaluate the performance of the proposed algorithms, and the results validate the effectiveness of our proposed approach.

In this work, the agglomerative hierarchical clustering and K-means clustering algorithms are implemented on small datasets. Considering that the selection of the similarity measure is a vital factor in data clustering, two measures are used in this study - cosine similarity measure and Euclidean distance - along with two evaluation metrics - entropy and purity - to assess the clustering quality. The datasets used in this work are taken from UCI machine learning depository. The experimental results indicate that k-means clustering outperformed hierarchical clustering in terms of entropy and purity using cosine similarity measure. However, hierarchical clustering outperformed k-means clustering using Euclidean distance. It is noted that performance of clustering algorithm is highly dependent on the similarity measure. Moreover, as the number of clusters gets reasonably increased, the clustering algorithms’ performance gets higher.

Clustering methods are commonly used for fMRI (functional Magnetic Resonance Imaging) data analysis. Based on an effective clustering algorithm called Affinity Propagation (AP) and a new defined similarity measure, we present a method for detecting activated brain regions. In the proposed method, autocovariance function values and the Euclidean distance metric of time series are firstly calculated and combined into a new similarity measure, then the AP algorithm with the measure is carried out on all time series of data, and at last regions with which their cross-correlation coefficients are greater than a threshold are taken as activations. Without setting the number of clusters in advance, our method is especially appropriate for the analysis of fMRI data collected with a periodic experimental paradigm. The validity of the proposed method is illustrated by experiments on a simulated dataset and a benchmark dataset. It can detect all activated regions in the simulated dataset accurately, and its error rate is smaller than that of K-means. On the benchmark dataset, the result is very similar to SPM.