Coordinate Descent Optimized Trace Difference Model for Joint Clustering and Feature Extraction

Feature selection plays a crucial role in scientific research and practical applications. In the real world applications, labeling data is time and labor consuming. Thus, unsupervised feature selection methods are desired for many practical applications. Linear discriminant analysis (LDA) with trace ratio criterion is a supervised dimensionality reduction method that has shown good performance to improve classifications. In this paper, we first propose a unified objective to seamlessly accommodate trace ratio formulation and K-means clustering procedure, such that the trace ratio criterion is extended to unsupervised model. After that, we propose a novel unsupervised feature selection method by integrating unsupervised trace ratio formulation and structured sparsity-inducing norms regularization. The proposed method can harness the discriminant power of trace ratio criterion, thus it tends to select discriminative features. Meanwhile, we also provide two important theorems to guarantee the unsupervised feature selection process. Empirical results on four benchmark data sets show that the proposed method outperforms other sate-of-the-art unsupervised feature selection algorithms in all three clustering evaluation metrics.

Clustering is a group of unsupervised statistical techniques commonly used in many disciplines. Considering their applications to fish abundance data, many technical details need to be considered to ensure reasonable interpretation. However, the reliability and stability of the clustering methods have rarely been studied in the contexts of fisheries. This study presents an intensive evaluation of three common clustering methods, including hierarchical clustering (HC), K-means (KM), and expectation-maximization (EM) methods, based on fish community surveys in the coastal waters of Shandong, China. We evaluated the performances of these three methods considering different numbers of clusters, data size, and data transformation approaches, focusing on the consistency validation using the index of average proportion of non-overlap (APN). The results indicate that the three methods tend to be inconsistent in the optimal number of clusters. EM showed relatively better performances to avoid unbalanced classification, whereas HC and KM provided more stable clustering results. Data transformation including scaling, square-root, and log-transformation had substantial influences on the clustering results, especially for KM. Moreover, transformation also influenced clustering stability, wherein scaling tended to provide a stable solution at the same number of clusters. The APN values indicated improved stability with increasing data size, and the effect leveled off over 70 samples in general and most quickly in EM. We conclude that the best clustering method can be chosen depending on the aim of the study and the number of clusters. In general, KM is relatively robust in our tests. We also provide recommendations for future application of clustering analyses. This study is helpful to ensure the credibility of the application and interpretation of clustering methods.

Object recognition is a key research area in the field of image processing and computer vision, which recognizes the object in an image and provides a proper label. In the paper, three popular feature descriptor algorithms that are Scale Invariant Feature Transform (SIFT), Speeded Up Robust Feature (SURF) and Oriented Fast and Rotated BRIEF (ORB) are used for experimental work of an object recognition system. A comparison among these three descriptors is exhibited in the paper by determining them individually and with different combinations of these three methodologies. The amount of the features extracted using these feature extraction methods are further reduced using a feature selection (k-means clustering) and a dimensionality reduction method (Locality Preserving Projection). Various classifiers i.e. K-Nearest Neighbor, Naive Bayes, Decision Tree, and Random Forest are used to classify objects based on their similarity. The focus of this article is to present a study of the performance comparison among these three feature extraction methods, particularly when their combination derives in recognizing the object more efficiently. In this paper, the authors have presented a comparative analysis view among various feature descriptors algorithms and classification models for 2D object recognition. The Caltech-101 public dataset is considered in this article for experimental work. The experiment reveals that a hybridization of SIFT, SURF and ORB method with Random Forest classification model accomplishes the best results as compared to other state-of-the-art work. The comparative analysis has been presented in terms of recognition accuracy, True Positive Rate (TPR), False Positive Rate (FPR), and Area Under Curve (AUC) parameters.

The growth of data today poses a challenge in management and inference. While feature extraction methods are capable of reducing the size of the data for inference, they do not help in minimizing the cost of data storage. On the other hand, feature selection helps to remove the redundant features and therefore is helpful not only in inference but also in reducing management costs. This work presents a novel parallel feature selection approach for classification, namely Parallel Feature Selection using Trace criterion (PFST), which scales up to very large datasets. Our method uses trace criterion, a measure of class separability used in Fisher's Discriminant Analysis, to evaluate feature usefulness. We analyzed the criterion's desirable properties theoretically. Based on the criterion, PFST rapidly finds important features out of a set of features for big datasets by first making a forward selection with early removal of seemingly redundant features parallelly. After the most important features are included in the model, we check back their contribution for possible interaction that may improve the fit. Lastly, we make a backward selection to check back possible redundant added by the forward steps. We evaluate our methods via various experiments using Linear Discriminant Analysis as the classifier on selected features. The experiments show that our method can produce a small set of features in a fraction of the amount of time by the other methods under comparison. In addition, the classifier trained on the features selected by PFST not only achieves better accuracy than the ones chosen by other approaches, but can also achieve better accuracy than the classification on all available features.

Joint clustering and feature learning methods have shown remarkable performance in unsupervised representation learning. However, the training schedule alternating between feature clustering and network parameters update leads to unstable learning of visual representations. To overcome this challenge, we propose Online Deep Clustering (ODC) that performs clustering and network update simultaneously rather than alternatingly. Our key insight is that the cluster centroids should evolve steadily in keeping the classifier stably updated. Specifically, we design and maintain two dynamic memory modules, i.e., samples memory to store samples' labels and features, and centroids memory for centroids evolution. We break down the abrupt global clustering into steady memory update and batch-wise label re-assignment. The process is integrated into network update iterations. In this way, labels and the network evolve shoulder-to-shoulder rather than alternatingly. Extensive experiments demonstrate that ODC stabilizes the training process and boosts the performance effectively.

K-Means is arguably the most popular data analysis method. The method outputs a partition of the entity set into clusters and centroids representing them. It is very intuitive and usually requires just a few pages to get presented. This text includes a number of less popular subjects that are important when using K-Means for real-world data analysis: Data standardization, especially, at mixed scales Innate tools for interpretation of clusters Analysis of examples of K-Means working and its failures Initialization – the choice of the number of clusters and location of centroids sVersions of K-Means such as incremental K-Means, nature inspired K-Means, and entity-centroid “medoid” methods are presented. Three modifications of K-Means onto different cluster structures are given:. Fuzzy K-Means for finding fuzzy clusters, Expectation-Maximization (EM) for finding probabilistic clusters, and Kohonen self-organizing maps (SOM) that tie up the sought clusters to a visually convenient two-dimensional grid. Equivalent reformulations of K-Means criterion are described – they can yield different algorithms for K-Means. One of these is explained at length: K-Means extends Principal component analysis to the case of binary scoring factors, which yields the so-called Anomalous cluster method, a key to an intelligent version of K-Means with automated choice of the number of clusters and their initialization.

Clustering is one of unsupervised classification method, that is, it classifies a data set into some clusters without any external criterion. Typical clustering methods, e.g. k-means (KM) and fuzzy c-means (FCM) are constructed based on optimization of the given objective function. Many clustering methods as well as KM and FCM are formulated as optimization problems with typical objective functions and constraints. The objective function itself is also an evaluation guideline of results of clustering methods. Considering together with its theoretical extensibility, there is the great advantage to construct clustering methods in the framework of optimization. From the viewpoint of optimization, some of the authors proposed an even-sized clustering method based on optimization (ECBO), which is with strengthened constraints of cluster size, and constructed some variations of ECBO. The constraint considered in ECBO is that each cluster size is K or K + 1. ECBO is based on KM and its algorithm is constructed as iterative optimization. The belongingness of each object to clusters are calculated by the simplex method in each iteration. The numerical experiments show that ECBO has higher classification accuracy than other similar clustering methods. It is considered that ECBO has the advantage in the viewpoint of clustering accuracy, cluster size, and optimization framework than other similar methods. However, the constraint of cluster sizes of ECBO is strict so that it may be inconvenient in case that the partition results, of which each cluster size need not be strictly even, but uneven, is desirable. Moreover, it is expected that new clustering algorithms of which each cluster size can be controlled can deal with more various datasets. In this paper, we first propose two new clustering algorithms based on ECBO. Each cluster size can be controlled in the proposed algorithms. Next, we estimate the new clustering algorithms through some numerical experiments.

In recent years, deep learning has shown significant potential in bearing fault diagnosis. However, challenges remain, including suboptimal signal quality under intricate conditions and the impact of network architecture on model performance. This study proposes a joint feature extraction method that combines a multi-scale residual convolutional neural network with a position-encoded transformer and integrating a transfer learning (TL) strategy to address the aforementioned issues. First, multi-scale convolutional layers are utilized to capture the details and local features from raw signals. To complement this, a transformer with embedded positional encoding learns global dependencies while retaining position information, effectively compensating the position shift issue. To further enhance the model’s generalization capability, a data augmentation strategy is designed and implemented, diversifying the training data. Furthermore, a TL strategy predicated on model fine-tuning is applied to mitigate the reliance on a substantial quantity of labeled data. Experiments conducted on two datasets, including Case Western Reserve University (CWRU) dataset and Self-Collected two-stage gear drive test bench dataset, that featuring diverse working conditions and bearing types. The proposed model achieved 99.92% accuracy on the CWRU dataset and demonstrated strong robustness across different operating conditions. With a model size of approximately 1.45MB, it maintains high diagnostic performance while ensuring computational efficiency. This intelligent diagnosis technique can also be applied to other rotating machinery, such as wind power, locomotive and various other fields owing to its robust feature extraction abilities.

ABSTRACTTraditional clustering methods focus on grouping subjects or (dependent) variables assuming independence between the variables. Clusters formed through these approaches can potentially lack homogeneity. This article proposes a joint clustering method by which both variables and subjects are clustered. In each joint cluster (in general composed of a subset of variables and a subset of subjects), there exists a unique association between dependent variables and covariates of interest. To this end, a Bayesian method is designed, in which a semi-parametric model is used to evaluate any unknown relationships between possibly correlated variables and covariates of interest, and a Dirichlet process is used to cluster subjects. Compared to existing clustering techniques, the major novelty of the method exists in its ability to improve the homogeneity of clusters, along with the ability to take the correlations between variables into account. Via simulations, we examine the performance and efficiency of the proposed method. Applying the method to cluster allergens and subjects based on the association of wheal size in reaction to allergens with age, we found that a certain pattern of allergic sensitization to a set of allergens has a potential to reduce the occurrence of asthma.

Patient-Specific Quality Assurance in Radiotherapy (PSQA) demands high-resolution dosimetry to verify accurate dose delivery in personalized intensity-modulated radiotherapy (IMRT) and volumetric-modulated arc therapy (VMAT) treatments. A novel optically stimulated luminescence (OSL) film dosimeter made with BaFBr:Eu2+ phosphor, offers submm spatial resolution. However, its energy-dependent response, requires corrections. Previously, a correction was proposed for a class of prostate cancer treatments assuming similar OSL energy response within the class. This study explored other class-specific corrections using a comprehensive radiotherapy treatment dataset. New classes were formed based on the similarity of treatment parameters without the need for user-based classifications. The dataset comprised 101 IMRT and VMAT treatment plans for three different Varian linac types (2 × Halcyon, 2 × TrueBeam, and 1 × TrueBeam STx). The treatment classes are based on a K-means clustering algorithm, that utilizes twelve quantitative treatment parameters expressed in principal components. Within cluster sum square (WCSS) was used to find the optimal number of classes and prevent data-overfitting. This objective assignment to classes was compared with three independent manual classifications by experienced medical physicists and dosimetrist. Additionally, a random class assignment was conducted for comparison. The adjusted-random-index (ARI) measured the similarity between classification methods. The OSL film, produced by Agfa N.V., was calibrated using a 6 MV TrueBeam linac. It was then used to measure treatments in an MULTICube phantom (IBA). Readout was performed in a CR-15 scanner. The local dose difference distribution between the measurement and treatment was characterized using a rational function. Class-specific corrections were developed by averaging the parameters of the rational function for each class as determined by the clustering, manual, and random classification methods. Dosimetric performances were evaluated within 20% and 50% isodose lines (D20% and D50%) before and after correction. The clustering method identified eight clusters (WCSS=119 silhouette=0.6) when representing data in three principal components, that is, 75% of the data variance. No significant similarity was found between clustering results and manual classification methods (ARI<0.01). Manual classifications are subject to interoperator variability. In fact, we found moderate similarity between classes and variations in the number of classes, ranging from 9 to 16. Uncorrected global dose difference (%) had mean value 0.9%±4.1% within D20%, with 47 and 34 treatments resulting in dose difference below 3% within D20% and D50%, respectively. After class-specific correction, the clustering method had mean dose differences (%) -0.2%±2.0%. The removal of the skewness in the corrected pixel-to-pixel dose difference distribution indicated an effective reduction of the OSL over-response. 88 and 74 treatments had corrected mean dose difference below 3% within D20% and D50%, respectively. Similar average dosimetric improvements were found only for the 16 manual class-solution, which however still showed a moderate skewness (0.1) after correction. Both, automated and manual class assignment preform better than the random assignment. Eight treatment class-solutions corrected the energy-dependent response of an OSL film used for PSQA measurements. Clustering classification methods, based on quantitative treatment information, yielded better dosimetric results compared to qualitative classification techniques.

A plant's reflectance can vary significantly depending on the type of stressors affecting it. Parasitic nematode species such as Meloidogyne incognita and Rotylenchulus reniformis are two of the leading nematode species affecting cotton plants. There is a need to detect the type of nematode in order to start proper nematode management program. Use of remotely sensed hyperspectral data could be one of the choices for species identification but, remotely sensed hyperspectral data are usually associated with high dimensions and requires some sort of dimensionality reduction without losing vital information. Some of the standard feature extraction and dimensionality reduction methods widely used nowadays are DWT and Self-Organized Maps (SOM) based methods. In this paper, authors explore the possibility of combining two above mentioned feature extraction and dimensionality reduction methods to extract feature for better classification accuracies pertaining to this study. The accuracies were then compared with the accuracies obtained using features extracted from DWT and SOM-based methods separately. For the entire analysis, SOM-supervised classification method was used.

Consensus clustering methods are motivated by the success of combining multiple classifiers in many areas. In this paper, graph-based consensus clustering is used to improve the quality of chemical compound clustering by enhancing the robustness, novelty, consistency and stability of individual clusterings. For this purpose, Hyper-Graph Partitioning Algorithm (HGPA) [1], was applied. The clustering is evaluated based on the ability to separate actives from inactives molecules in each cluster and the results were compared with the Ward's clustering method. The chemical dataset MDL Drug Data Report (MDDR) database has been used for experiments. The MDL Drug Data Report (MDDR) database consists of 102516 molecules. For the experiments, the dataset DS1 was chosen from the MDDR database. This dataset has been used for many virtual screening experiments [2-4]. The dataset DS1contains 10 heterogeneous activity classes (8568 molecules). For the clustering experiments, two 2D fingerprint descriptors will be used which are developed by Scitegic's Pipeline Pilot [5]. These are 120-bit ALOGP and 1024-bit extended connectivity fingerprints (ECFP_4). The results were evaluated based on the effectiveness of the methods to separate actives from non-actives molecules using QPI- (for quality partition index) measure, which was devised by Varin et al. [6]. As defined by [7], an active cluster as a non-singleton cluster for which the percentage of active molecules in the cluster is greater than the percentage of active molecules in the dataset as a whole. Let p be the number of actives in active clusters, q the number of inactives in active clusters, r the number of actives in inactive clusters (i.e., clusters that are not active clusters) and s the number of singleton actives. The high value occurs when the actives are clustered tightly together and separated from the inactive molecules. Then the quality partition index, QPI, is defined to be: QPI=pp+q+r+s (1) Then, the results will be compared with Ward's individual clustering method, the standard clustering method for chemoinformatics applications. The generation process has been done by multiple run of K-means algorithms, each with random initialization of cluster centroids. The number of partitions generated in this step is ranged between n = 5 to n = 50, with 5-times step. Then, all the generated partitions were combined using HGPA to obtain the consensus partition. This process is done for each fingerprint (ALOGP and ECFP_4). The mean of QPI values are averaged over the ten activity classes of the datasets. Tables ​Tables1,1, ​,22 show the effectiveness of MDDR dataset clustering using ALOGP and ECFP_4 fingerprints. The best PQI value of consensus clustering methods for each column has been bold-faced for ease of reference. Table 1 Effectivenss of clustering of high diverse MDDR dataset: ALOGP Fingerprint. Table 2 Effectivenss of clustering of high diverse MDDR dataset: ECFP_4 Fingerprint. Visual inspection of the results enables comparisons to be made between the effectiveness of clustering of MDDR datasets and Ward's method, the best of choice clustering method for chemoinformatics applications. In addition, ten times of consensus clustering, for each fingerprint were observed in order to study the effectiveness of consensus clustering with different ensemble sizes. The results show that HGPA consensus clustering gives robust and novel result when K-means algorithm is run 20-50 times using ALOGP. The performance of consensus clustering outperforms the Wards' method. For consensus of dataset which represented by ECFP_4 fingerprint, the best QPI values of consensus clustering are obtained from ensembles of size n = 20-50. The performance of consensus clustering gives robust results which are better than overall performance of individual clusterings. The values of QPI in both datasets for consensus clustering are close to the Wards method. The consensus clustering, HGPA, provide stable clusters by decreasing the sensitivity to noise and outliers. The average percentages of singleton clusters of individual clusterings compared with consensus clustering for both fingerprints. The results show that consensus clustering partition the datasets with average percentage of singleton equal to zero, which is much better than individual clusterings and Wards' method. For example, 16.72% of molecules of DS1 are clustered as singletons when Wards method is applied on ALOGP fingerprint with number of clusters equal to 1000 clusters. Finally we conclude that graph-based consensus clustering can improve the effectiveness of chemical compounds clustering. The performance of consensus clustering is more robust, novel, stable, consistent, and out-perform Wards' method in case of using ALOGP fingerprint. By using ECFP_4 fingerprint, consensus clustering methods provide more robust, stable, and consistent clustering and close to the Wards clustering results. The experiments reported here suggest that graph-based consensus clustering can improve the quality of individual clustering by using the efficient algorithm, K-means algorithm, to generate the ensemble with size (20-50) for both structurally diverse chemical datasets.

Clustering is an unsupervised learning approach that explores data and seeks groups of similar objects. Many classical clustering models such as k-means and DBSCAN are based on heuristics algorithms and suffer from local optimal solutions and numerical instability. Recently convex clustering has received increasing attentions, which leverages the sparsity inducing norms and enjoys many attractive theoretical properties. However, convex clustering is based on Euclidean distance and is thus not robust against outlier features. Since the outlier features are very common especially when dimensionality is high, the vulnerability has greatly limited the applicability of convex clustering to analyze many real-world datasets. In this paper, we address the challenge by proposing a novel robust convex clustering method that simultaneously performs convex clustering and identifies outlier features. Specifically, the proposed method learns to decompose the data matrix into a clustering structure component and a group sparse component that captures feature outliers. We develop a block coordinate descent algorithm which iteratively performs convex clustering after outliers features are identified and eliminated. We also propose an efficient algorithm for solving the convex clustering by exploiting the structures on its dual problem. Moreover, to further illustrate the statistical stability, we present the theoretical performance bound of the proposed clustering method. Empirical studies on synthetic data and real-world data demonstrate that the proposed robust convex clustering can detect feature outliers as well as improve cluster quality.

This paper proposes a phonetic feature extraction and recognition model in Korean pronunciation exercises based on AdaBoot moments. Feature extraction algorithms have a great influence on speech emotion recognition algorithms, among which Mel-frequency Cepstral Coefficients (MFCC) is the most commonly used feature in speech emotion recognition. The feature extraction method and the influence of SVM kernel function and parameter selection on the recognition result are studied, and the existing speech feature extraction algorithms and their respective advantages and disadvantages are analyzed, as well as the influence of different kernel functions, kernel parameters and penalty parameters on the recognition performance. An improved grid optimization method is used to further improve the recognition time of voice information.

Productivity in agriculture plays a major role in economic expansion. Because plant disease is a widespread occurrence, plant disease detection is an important problem in the world of agriculture. Plants do suffer a significant consequence if the required care is not taken at the beginning, which affects the amount, quality or productivity of the relevant products. Because it can detect disease symptoms at the earliest stage and reduces the labour required for large crop farm tracking, the automated plant disease detection system is more advantageous. In order to detect plant diseases, this paper proposes a novel, four‐step methodology that consists of improved deep joint image segmentation, feature extraction (which includes LGXP, MBP, colour feature and hierarchy of skeleton feature extraction) and detection via hybrid DL classifier, specifically improved RNN with the transfer learning process and GoogleNet. By averaging the classifiers' results scores, the final detection result is calculated. In terms of several performance metrics, the suggested work's effectiveness is verified in comparison to the traditional models. In contrast to the SVM (79.5597), KNN (59.2767), LSTM (78.1446), GoogleNet (79.4025), CNN (77.6729), and CAE + CNN (80.1886), the F‐measure of the IRNN‐TL is 91.1949.