A Convex Model for Support Vector Distance Metric Learning.

A convex multi-class model via distance metric learning based class-to-instance confidence

This paper focuses on semi-supervised clustering, where the goal is to cluster a set of data-points given a set of similar/dissimilar examples. These examples provide instance-level equivalence/in-equivalence constraints (e.g., similar pairs belong to the same cluster while dissimilar pairs belong to different clusters), but in order to aid final clustering we must propagate them to feature-space level constraints (i.e., how similar are two regions in the feature space?). An increasingly popular approach to accomplish this is by learning distance metrics over the feature space that are guided by the instance-level constraints. Inspired by the success of recent bag-of-words models, we utilize codewords (or visual-words) as building blocks. Our proposed technique learns non-parametric distance metrics over codewords from these equivalence (and optionally, in-equivalence) constraints, which we are then able to propagate back to compute a dissimilarity measure between any two points in the feature space. There are two significant advances over previous work. First, unlike past efforts on global distance metric learning which try to transform the entire feature space so that similar pairs are close, we transform modes in data distribution or pockets of the feature space. This transformation is non-parametric and thus allows arbitrary non-linear deformations of the feature space. Second, while most Mahalanobis metrics are learnt using Semi-Definite Programming (SDP), our proposed solution is developed as a Linear Program (LP) and in practice, is extremely fast. Finally, we provide quantitative analysis on image datasets (MSRC, Corel) where ground-truth segmentation is available, and show that our learnt metrics can significantly improve clustering accuracy.

Recent years have witnessed extensive studies on distance metric learning (DML) for improving similarity search in multimedia information retrieval tasks. Despite their successes, most existing DML methods suffer from two critical limitations: (i) they typically attempt to learn a linear distance function on the input feature space, in which the assumption of linearity limits their capacity of measuring the similarity on complex patterns in real-world applications; (ii) they are often designed for learning distance metrics on uni-modal data, which may not effectively handle the similarity measures for multimedia objects with multimodal representations. To address these limitations, in this paper, we propose a novel framework of online multimodal deep similarity learning (OMDSL), which aims to optimally integrate multiple deep neural networks pretrained with stacked denoising autoencoder. In particular, the proposed framework explores a unified two-stage online learning scheme that consists of (i) learning a flexible nonlinear transformation function for each individual modality, and (ii) learning to find the optimal combination of multiple diverse modalities simultaneously in a coherent process. We conduct an extensive set of experiments to evaluate the performance of the proposed algorithms for multimodal image retrieval tasks, in which the encouraging results validate the effectiveness of the proposed technique.

Partial multi-label learning (PML) models the scenario where each training instance is annotated with a set of candidate labels, and only some of the labels are relevant. The PML problem is practical in real-world scenarios, as it is difficult and even impossible to obtain precisely labeled samples. Several PML solutions have been proposed to combat with the prone misled by the irrelevant labels concealed in the candidate labels, but they generally focus on the smoothness assumption in feature space or low-rank assumption in label space, while ignore the negative information between features and labels. Specifically, if two instances have largely overlapped candidate labels, irrespective of their feature similarity, their ground-truth labels should be similar; while if they are dissimilar in the feature and candidate label space, their ground-truth labels should be dissimilar with each other. To achieve a credible predictor on PML data, we propose a novel approach called PML-LFC (Partial Multi-label Learning with Label and Feature Collaboration). PML-LFC estimates the confidence values of relevant labels for each instance using the similarity from both the label and feature spaces, and trains the desired predictor with the estimated confidence values. PML-LFC achieves the predictor and the latent label matrix in a reciprocal reinforce manner by a unified model, and develops an alternative optimization procedure to optimize them. Extensive empirical study on both synthetic and real-world datasets demonstrates the superiority of PML-LFC.

In an indoor positioning system (IPS), transfer learning (TL) methods are commonly used to predict the location of mobile devices under the assumption that all training instances of the target domain are given in advance. However, this assumption has been criticized for its shortcomings in dealing with the problem of signal distribution variations, especially in a dynamic indoor environment. The reasons are: collecting a sufficient number of training instances is costly, the training instances may arrive online, the feature spaces of the target and source domains may be different, and negative knowledge may be transferred in the case of a redundant source domain. In this work, we proposed an online heterogeneous transfer learning (OHetTLAL) algorithm for IPS-based RSS fingerprinting to improve the positioning performance in the target domain by fusing both source and target domain knowledge. The source domain was refined based on the target domain to avoid negative knowledge transfer. The co-occurrence measure of the feature spaces (Cmip) was used to derive the homogeneous new feature spaces, and the features with higher weight values were selected for training the classifier because they could positively affect the location prediction of the target. Thus, the objective function was minimized over the new feature spaces. Extensive experiments were conducted on two real-world scenarios of datasets, and the predictive power of the different modeling techniques were evaluated for predicting the location of a mobile device. The results have revealed that the proposed algorithm outperforms the state-of-the-art methods for fingerprint-based indoor positioning and is found robust to changing environments. Moreover, the proposed algorithm is not only resilient to fluctuating environments but also mitigates the model's overfitting problem.

A nearest-neighbor search model for distance metric learning

This paper describes polynomial kernel subspace approach to Isolated Word Recognition (IWR) systems. Auditory motivated wavelet packet transform is used to derive the desirable speech features. This approach represents speech features as the projection of the Wavelet Packet Parameters (WPP) mapped into a feature space via a non-linear mapping onto the principal components called Kernel Fisher Discriminant (KFD). The nonlinear mapping between the input space and the feature space is implicitly performed using the kernel-trick. This nonlinear mapping using KFD increases the discrimination ability of a pattern classifier. The use of Mel-scale based and Bark-scale based wavelet packet trees for feature extraction process adds human auditory perception behavior to enhance the classification performance. Experimental results show that the proposed kernel based techniques are computationally efficient and performs well with less training data.

Multi-view learning machines aim to process multi-view data set which consists of instances with different views. In recent years, the scale of multi-view data set is more and more larger and traditional multi-view learning machines which should store the entire data set cannot process them with the limitation of computation and storage ability. In order to overcome such a disadvantage, one-pass multi-view (OPMV) framework has been proposed and it goes through instances only once without storing the entire data set. But most multi-view data sets are semi-supervised, i.e., some training instances of the data sets are labeled while others are unlabeled. Furthermore, for a real-world multi-view data set, labeling instances is a high-cost task. In this paper, we propose semi-supervised OPMV (SSOPMV) framework for solving this issue. In SSOPMV, we develop an approach to generate additional unlabeled instances which possess useful discriminant information and apply them into the original OPMV framework along with the labeled and original unlabeled training instances. We verify the effectiveness of the SSOPMV with some real-world semi-supervised multi-view data sets used theoretically and empirically.

Hierarchical GAN-Tree and Bi-Directional Capsules for multi-label image classification

Partial label (PL) learning tackles the problem where each training instance is associated with a set of candidate labels, among which only one is the true label. In this paper, we propose a novel generative model PL-CGAN, which tackles the partial label learning problem with Conditional Generative Adversarial Nets (CGAN). Specially, PL-CGAN introduces a teacher model to refine a more reliable soft label vector of each training instance by iteratively ensembling the current learned prediction network with the formal one in an online manner. Besides, it adopts a MixUp data augmentation scheme to prevent the prediction network from overfitting to the noisy labels. In addition, it deploys a CGAN to generate training instances with its corresponding label vectors, which form a feature alignment based on consistency cost to enhance it’s label refinement capacity. Extensive experiments are conducted on synthesized and real-world partial label learning datasets, while the proposed approach demonstrates the state-of-the-art performance for partial label learning.

Viewing to monitoring the continuous casting process, a nonlinear fault detection method based on kernel principal component analysis (KPCA) was introduced. KPCA can efficiently compute principal components in high-dimensional feature spaces by means of integral operators and nonlinear kernel functions, which is to first map the input space into a feature space via nonlinear mapping and then to compute the principal components in that feature space. Based on T2 and SPE charts in feature space, principal component analysis(PCA)can be used to detect faults.. The simulation results show that the proposed approach effectively captured the nonlinear relationship in the process variables and showed superior process monitoring performance compared to linear PCA.

The aim of this study is to build Machine Learning models to evaluate the effect of blast furnace slag (BFS) and waste tire rubber powder (WTRP) on the compressive strength of cement mortars. In order to develop these models, 12 different mixes with 288 specimens of the 2, 7, 28, and 90 days compressive strength experimental results of cement mortars containing BFS, WTRP and BFS+WTRP were used in training and testing by Random Forest, Ada Boost, SVM and Bayes classifier machine learning models, which implement standard cement tests. The machine learning models were trained with 288 data that acquired from experimental results. The models had four input parameters that cover the amount of Portland cement, BFS, WTRP and sample ages. Furthermore, it had one output parameter which is compressive strength of cement mortars. Experimental observations from compressive strength tests were compared with predictions of machine learning methods. In order to do predictive experimentation, we exploit R programming language and corresponding packages. During experimentation on the dataset, Random Forest, Ada Boost and SVM models have produced notable good outputs with higher coefficients of determination of R2, RMS and MAPE. Among the machine learning algorithms, Ada Boost presented the best R2, RMS and MAPE values, which are 0.9831, 5.2425 and 0.1105, respectively. As a result, in the model, the testing results indicated that experimental data can be estimated to a notable close extent by the model.

Electrical faults in the power system refer to a variety of abnormalities in the power equipment, and these faults may lead to equipment damage or even cause dangerous events such as fires and explosions, which pose a serious threat to people's lives and property. Therefore, it is very important to find and deal with these faults in time. In this paper, based on the electrical fault dataset of power system, SVM, decision tree, KNN and random forest model are used to detect electrical faults. After the confusion matrix results are analyzed, most of the predictions of the four models are correct, and only a few instances that should not be predicted as electrical faults are predicted as electrical faults. All the four machine learning models achieved more than 99% accuracy in detecting electrical faults in the power system and were able to detect electrical faults very accurately. Among them, the SVM model has the highest prediction accuracy of 99.6%, followed by the KNN model prediction accuracy of 99.5%. In summary, this paper adopts machine learning method to detect electrical faults in power systems, and achieves better results in the experimental results. This method can help power system supervisors find and deal with faults in time, and improve the safety and stability of the power system. In the future, we can further study how to optimize the model to improve the detection accuracy and apply it in practical production.

Parkinson’s disease (PD) is a complex disorder characterized by several motor and non-motor symptoms that worsen over time, and that differ from person to another. In the early stages, when the symptoms are often incomplete, the diagnosis becomes difficult and at times, the subject may remain undiagnosed. This difficulty is a strong motivation for computer-based assessment tools that can aid in the early diagnosing and predicting the progression of PD. Handwriting’s deterioration, vocal and eye movement impairments may be ones of the earliest indicators for the onset of the illness. A language independent model to detect PD at early stages by using multimodal signals has not been enough addressed. Due to the lack of multimodal and multilingual databases, database which includes online handwriting, speech signals, and eye movement’s recordings have been recently collected. After succeeding in building language independent models for PD early diagnosis using pure handwriting or speech, we propose in this work language independent models based on bimodal analyses (handwriting and speech), where both SVM and deep learning models are studied. Our experiments show that classification accuracy up to 100% can be obtained by our SVM model through handwriting/speech bimodal analysis.

COVID-19 belongs to the genus of Coronaviridae, has emerged and affected the whole world. A virus without a vaccine creates unexpected havoc in human life, financial and economic systems of every country around the world. This disease carries a heavy workload for doctors and health workers. This workload can be reduced by machine learning and the development of computer-aided diagnostic systems. Any scientific study of this disease will help manage this disease as soon as possible. In this experiment, support vector machine, decision tree, and neural network classifiers have been built for COVID-19 dataset and the accuracies among those classifiers have been analyzed by ANOVA test. First, we performed data preprocessing to handle the missing values and categorical data. Second, we performed feature selection using chi-squared statistics and mutual-information statistics methods. Third, we split the data into 4 train-sets and test-sets for performing 4-fold cross validation. And lastly, we built SVM, DT, and NN models and started the experiment. In the experiment without feature selection, accuracy results among SVM, DT, and NN is stastistically significant (reject the null hypothesis). Otherwise, in the experiment with feature selection, accuracy results among SVM, DT, and NN is not stastistically significant (accept the null hypothesis). Both experiments are tested by ANOVA test. In this experiments, neural network classifiers (both without and with feature selection) have better accuracy result (98,16% and 97,08%) than SVM (96,63% and 96,72%) and DT (98,08% and 96,89%) for the dataset used in this experiment.