Nonnegative matrix factorization with combined kernels for small data representation

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

In this paper, a novel multiple kernel learning (MKL) algorithm is proposed for the classification of hyperspectral images. The proposed MKL algorithm adopts a two-step strategy to learn a multiple kernel machine. In the first step, unsupervised learning is carried out to learn a combined kernel from the predefined base kernels. In our algorithms, low-rank nonnegative matrix factorization (NMF) is used to carry out the unsupervised learning and learn an optimal combined kernel. Furthermore, the kernel NMF (KNMF) is introduced to substitute NMF for enhancing the ability of the unsupervised learning with the predefined base kernels. In the second step, the optimal kernel is embedded into the standard optimization routine of support vector machine (SVM). In addition, we address a major challenge in hyperspectral data classification, i.e., using very few labeled samples in a high-dimensional space. Experiments are conducted on three real hyperspectral datasets, and the experimental results show that the proposed algorithms, especially for KNMF-based MKL, achieve the outstanding performance for hyperspectral image classification with few labeled samples when compared with several state-of-the-art algorithms.

In this paper, we extend the original non-negative matrix factorization (NMF) to kernel NMF (KNMF). The advantages of KNMF over NMF are: 1) it could extract more useful features hidden in the original data through some kernel-induced nonlinear mappings; 2) it can deal with data where only relationships (similarities or dissimilarities) between objects are known; 3) it can process data with negative values by using some specific kernel functions (e.g. Gaussian). Thus, KNMF is more general than NMF. To further improve the performance of KNMF, we also propose the SpKNMF, which performs KNMF on sub-patterns of the original data. The effectiveness of the proposed algorithms is validated by extensive experiments on UCI datasets and the FERET face database.

In this paper, we proposed an incremental kernel non-negative matrix factorization (IKNMF) to reduce the computing scale in hyperspectral unmixing. Kernel non-negative matrix factorization (KNMF) is an extended non-negative matrix factorization (NMF) able to capture nonlinear dependency features in data matrix through kernel functions. In KNMF algorithm, the size of kernel matrices is closely associated with the input data scale. To reduce calculation and storage of large matrices, we extend KNMF by introducing partition matrix theory. The decomposition results of data matrices are derived from smaller scale matrices incrementally. Experiments are conducted on synthetic hyperspectral images with multiple sizes, and the experimental results show that the proposed algorithm have effect in saving calculation and memory resource without degrading the unmixing performance.

This paper attempts to represent the mapped data in the radial basis function (RBF) feature space under non-negativity constraints and develops a RBF kernel based non-negative matrix factorization (KNMF-RBF) algorithm. Based on an objective function with Frobenius norm, we obtain the multiplicative update rules of our KNMF-RBF approach using kernel theory and gradient descent method. The proposed KNMF-RBF method is theoretically shown to be convergence by means of the constructed auxiliary function. Our approach is successfully applied to face recognition in the FERET face database. Experimental results demonstrate that the proposed KNMF-RBF method surpasses some kernel based methods. The results are encouraging.

Sentence similarity is widely used in various natural language tasks such as natural language inference, paraphrase identification, and question answering. However, a variety of linguistic expressions and ambiguities of words in sentences make it difficult to measure sentence similarity. Many studies show that using local features or global features of a sentence will produce satisfactory sentence representations that can be utilized to measure sentence similarity. Local features reflect the relationships of adjacent words for each sentence and the sequence information of a sentence are usually expressed by global features. However, local features lack abilities to capture sequence information while a small amount of extracted global features is not enough to produce sentence representations with good qualities. In this paper, we propose A Hybrid Model combining Local and Global features into a Siamese Network (HM-LGSN) for sentence similarity calculation. We first propose a new convolution neural network architecture called group convolution neural network to extract the most representative local features (or word semantic features). Then we combine these new features with pre-trained embeddings of words as input to the Bidirectional Gated Recurrent Units to extract global features of sentences. Finally, we select the global features to form sentence representations and calculate sentence similarity through Manhattan distance. The experimental results on SICK, MSRVID, STS-B datasets show that the accuracy of our proposed model is significantly improved by combining local features and global features.

We present a numerical algorithm for nonnegative matrix factorization (NMF) problems under noisy separability. An NMF problem under separability can be stated as one of finding all vertices of the ...

Face recognition is a challenging issue in field of multi-science, main contents of research is how to make computer have ability of face recognition face recognition technology involved in a lot, which is a key feature extraction and classification method, this paper focuses on study of related theory. Non-negative matrix factorization trapped MF) algorithm and local non-negative matrix factorization (LNMF) algorithm is a feature extraction method based on local features, has been successfully used in face recognition NMF algorithm in face recognition rate is low, although LNMF algorithm to a certain extent, improve recognition rate, but its price is to increase number of iterations. In addition, two algorithms have failed to solve good nonlinear separable problems the kernel method combined with LNMF algorithm, kernel local non-negative matrix factorization (KLNMF) algorithm, first by a nonlinear transformation of original space to high-dimensional space, making samples linearly separable, and then use LNMF algorithm to extract face features. In classification part, paper presents decision rules of classification of their own, and design based on NMF subspace classifier. DOI : http://dx.doi.org/10.11591/telkomnika.v12i3.4356 Full Text: PDF

In this paper, we present a novel muscle synergy extraction method based on multivariate curve resolution–alternating least squares (MCR-ALS) to overcome the limitation of the nonnegative matrix factorization (NMF) method for extracting non-sparse muscle synergy, and we study its potential application for evaluating motor function of stroke survivors. Nonnegative matrix factorization (NMF) is the most widely used method for muscle synergy extraction. However, NMF is susceptible to components’ sparseness and usually provides inferior reliability, which significantly limits the promotion of muscle synergy. In this study, MCR-ALS was employed to extract muscle synergy from electromyography (EMG) data. Its performance was compared with two other matrix factorization algorithms, NMF and self-modeling mixture analysis (SMMA). Simulated data sets were utilized to explore the influences of the sparseness and noise on the extracted synergies. As a result, the synergies estimated by MCR-ALS were the most similar to true synergies as compared with SMMA and NMF. MCR-ALS was used to analyze the muscle synergy characteristics of upper limb movements performed by healthy (n = 11) and stroke (n = 5) subjects. The repeatability and intra-subject consistency were used to evaluate the performance of MCR-ALS. As a result, MCR-ALS provided much higher repeatability and intra-subject consistency as compared with NMF, which were important for the reliability of the motor function evaluation. The stroke subjects had lower intra-subject consistency and seemingly had more synergies as compared with the healthy subjects. Thus, MCR-ALS is a promising muscle synergy analysis method for motor function evaluation of stroke patients.

We investigated the application of multiresolution global and local texture features to reduce false-positive detection in a computerized mass detection program. One hundred and sixty-eight digitized mammograms were randomly and equally divided into training and test groups. From these mammograms, two datasets were formed. The first dataset (manual) contained four regions of interest (ROIs) selected manually from each of the mammograms. One of the four ROIs contained a biopsy-proven mass and the other three contained normal parenchyma, including dense, mixed dense/fatty, and fatty tissues. The second dataset (hybrid) contained the manually extracted mass ROIs, along with normal tissue ROIs extracted by an automated Density-Weighted Contrast Enhancement (DWCE) algorithm as false-positive detections. A wavelet transform was used to decompose an ROI into several scales. Global texture features were derived from the low-pass coefficients in the wavelet transformed images. Local texture features were calculated from the suspicious object and the peripheral subregions. Linear discriminant models using effective features selected from the global, local, or combined feature spaces were established to maximize the separation between masses and normal tissue. Receiver Operating Characteristic (ROC) analysis was conducted to evaluate the classifier performance. The classification accuracy using global features were comparable to that using local features. With both global and local features, the average area, Az, under the test ROC curve, reached 0.92 for the manual dataset and 0.96 for the hybrid dataset, demonstrating statistically significant improvement over those obtained with global or local features alone. The results indicated the effectiveness of the combined global and local features in the classification of masses and normal tissue for false-positive reduction.

Nonnegative matrix factorization and I-divergence alternating minimization

In the harsh underground environment, the transmission gear in the rocker arm of the shearer is susceptible to impact load, frequently leading to failures that result in economic losses or casualties. Therefore, real-time and accurate gear fault diagnosis is crucial. Existing diagnostic models struggle with insufficient feature extraction and low diagnostic accuracy when addressing nonlinear, nonsmooth signals with strong noise in this environment. Therefore, an innovative parallel integration model of Swin Transformer and multiscale convolution is proposed to extract both global and local features under different fault modes and noise conditions. A spatial interaction block (SIB) is introduced into the Swin Transformer branch to enhance global information acquisition. In multiscale convolutional neural network (CNN) branches, depthwise separable convolutions with different expansion coefficients are utilized for multiscale extraction of local features to reduce computational complexity. Considering the differences between global and local features, a feature interaction module (FIM) is added between the two branches to realize the interaction between local and global features, considerably improving the model diagnosis performance. Furthermore, to meet the high real-time diagnosis requirements, a cloud-edge collaboration (CEC) framework is designed to realize the timely alarm of gear faults through the efficient collaboration between edge nodes and the industrial cloud platform. Using the vibration data collected from the rocker arm loading test bed of Taizhong Coal Machinery and the gear data provided by Southeast University for validation, the results show that the diagnostic precision of the proposed method can reach up to 99.55%. Even when the signal-to-noise ratio (SNR) is −2 db, the accuracy is still 93.67%, which is superior to other advanced comparison models, and significantly reduces the completion time of fault diagnosis tasks, and has strong anti-noise and real-time diagnosis capabilities.

In this study, we investigated how shapes are classified based on local and global features by four representative convolutional neural networks (CNNs), i.e., AlexNet, VGG, ResNet and Inception. While the local features are based on simple components, such as orientation of line segment, the global features are based on the whole object, such as whether an object has a hole. For example, solid triangles and solid squares are differentiated by local features, solid circles and rings are differentiated by a global feature. Two sets of experiments were performed in this research. In the first experiment, we examined how the four CNNs pre-trained on ImageNet (with transfer learning) learned to differentiate the regular shapes (equilateral-triangles, squares, circles and rings). Our results showed that the pre-trained CNNs exhibited faster learning rates in the tasks discriminating the local features than in the tasks discriminating the global feature. However, the transfer learning of discriminating the global feature in regular shapes were better generalized to irregular shapes than the transfer learning of discriminating local features. In the second experiment, the CNNs were trained from scratch (with random weights initialization) to discriminate local and global features in regular and irregular shapes. Different from the transfer learning, the CNNs exhibited faster learning rates in discriminating the global feature than the local features. Similar to transfer learning, the CNNs exhibited excellent learning generalization to discriminating the global feature of irregular shapes, but poor learning generalization to discriminating the local features in the irregular shapes. The overarching goal of this research is to create a paradigm and benchmark to directly compare how the CNNs and primate visual systems process geometrical invariants. In contrast to the ImageNet approach which employs natural images to train CNNs, we employed the “ShapeNet” approach which features geometrical shapes with well-defined properties. The ShapeNet approach will not only help elucidate the strengths and limitations of CNN computation, but also provide insights into visual information processing in the primates.

This article uses the projected gradient method (PG) for a non-negative matrix factorization problem (NMF), where one or both matrix factors must have orthonormal columns or rows. We penalize the orthonormality constraints and apply the PG method via a block coordinate descent approach. This means that at a certain time one matrix factor is fixed and the other is updated by moving along the steepest descent direction computed from the penalized objective function and projecting onto the space of non-negative matrices. Our method is tested on two sets of synthetic data for various values of penalty parameters. The performance is compared to the well-known multiplicative update (MU) method from Ding (2006), and with a modified global convergent variant of the MU algorithm recently proposed by Mirzal (2014). We provide extensive numerical results coupled with appropriate visualizations, which demonstrate that our method is very competitive and usually outperforms the other two methods.

Camouflaged object detection (COD) aims to accurately detect objects that blend in with their backgrounds. While deep learning-based methods have significantly improved detection accuracy, the persistent similarity in chromatic and textural characteristics between camouflaged objects and their environments continues to pose substantial challenges, particularly in complex scenarios. Existing COD methods still encounter two primary challenges: (1) the difficulty in effectively fusing detailed features with semantic representations, leading to suboptimal balance between local and global features in the fused features; (2) progressive information dilution in conventional top-down decoder architectures, resulting in imprecise boundary localization. To address these dual challenges of multi-scale feature enhancement and semantic information dilution, we propose a D ual-Branch C ross-Layer I nformation F low Network (DCIFNet) for camouflaged object detection in complex scenes. Specifically, our framework integrates a hybrid CNN-ViT architecture for parallel extraction of localized patterns and global contextual features. Subsequently, a global local fusion (GLF) module is employed to iteratively and adaptively fuse the global and local features at each stage using gating weights. Additionally, a novel Dense Connection Decoder (DCD) module is designed to mitigate the information dilution problem associated with traditional top-down decoders. Extensive experiments show that DCIFNet outperforms other mainstream COD models in most performance metrics on four widely used benchmark datasets. On the COD10K dataset, compared with the latest SOTA work of PRNet, DCIFNet improves the metrics of \({S_{\alpha}}\) , \({E_{\phi}}\) , and \(F_{\beta}^{w}\) by 0.11 \(\%\) , 0.11 \(\%\) , and 1.38 \(\%\) , respectively, demonstrating a significant performance enhancement. The code is available https://github.com/WateverOk/DCIFNet .