Learning Compact and Effective Distance Metrics with Diversity Regularization

In real tasks, usually a good classification performance can only be obtained when a good distance metric is obtained; therefore, distance metric learning has attracted significant attention in the past few years. Typical studies of distance metric learning evaluate how to construct an appropriate distance metric that is able to separate training data points from different classes or satisfy a set of constraints (e.g., must-links and/or cannot-links). It is noteworthy that this task becomes challenging when there are only limited labeled training data points and no constraints are given explicitly. Moreover, most existing approaches aim to construct a global distance metric that is applicable to all data points. However, different data points may have different properties and may require different distance metrics. We notice that data points in real tasks are often connected by physical links (e.g., people are linked with each other in social networks; personal webpages are often connected to other webpages, including nonpersonal webpages), but the linkage information has not been exploited in distance metric learning. In this article, we develop a pairwised specific distance (PSD) approach that exploits the structures of physical linkages and in particular captures the key observations that nonmetric and clique linkages imply the appearance of different or unique semantics, respectively. It is noteworthy that, rather than generating a global distance, PSD generates different distances for different pairs of data points; this property is desired in applications involving complicated data semantics. We mainly present PSD for multi-class learning and further extend it to multi-label learning. Experimental results validate the effectiveness of PSD, especially in the scenarios in which there are very limited labeled training data points and no explicit constraints are given.

Multivariate longitudinal data are used in a variety of research areas not only because they allow to analyze time trajectories of multiple indicators, but also to determine how these trajectories are influenced by other covariates. In this article, we propose a mixture of longitudinal factor analyzers. This model could be used to extract latent factors representing multiple longitudinal noisy indicators in heterogeneous longitudinal data and to study the impact of one or several covariates on these latent factors. One of the advantages of this model is that it allows for measurement non-invariance, which arises in practice when the factor structure varies between groups of individuals due to cultural or physiological differences. This is achieved by estimating different factor models for different latent classes. The proposed model could also be used to extract latent classes with different latent factor trajectories over time. Other advantages of the model include its ability to take into account heteroscedasticity of errors in the factor analysis model by estimating different error variances for different latent classes. We first define the mixture of longitudinal factor analyzers and its parameters. Then, we propose an EM algorithm to estimate these parameters. We propose a Bayesian information criterion to identify both the number of components in the mixture and the number of latent factors. We then discuss the comparability of the latent factors obtained between subjects in different latent groups. Finally, we apply the model to simulated and real data of patients with chronic postoperative pain.

Latent factor models are valuable in bioinformatics for accounting for unmeasured variation alongside observed covariates. Yet many methods struggle to separate known effects from latent structure and to handle losses beyond standard regression. We present a unified framework that augments row and column predictors with a low-rank latent component, jointly modeling measured effects and residual variation. To remove ambiguity in estimating observed and latent effects, we impose a carefully designed set of orthogonality constraints on the coefficient and latent factor matrices, relative to the spans of the predictor matrices. These constraints ensure identifiability, yield a decomposition in which the latent term captures only variation unexplained by the covariates, and improve interpretability. An efficient algorithm handles general non-quadratic losses via surrogates with monotone descent. Each iteration updates the latent term by truncated singular value decomposition of a doubly projected residual and refines coefficients by projections. The number of latent factors is selected by applying an elbow rule to a degrees-of-freedom-adjusted information criterion. A parametric bootstrap provides valid inference on feature-outcome associations under the regularized low-rank structure. Applied to real pharmacogenomic data, the method recovers biologically coherent gene-drug associations missed by standard factor models, such as the EGFR-inhibitor link, highlights novel candidates with plausible mechanisms, and reveals gene programs aligned with compound modes of action, including a latent unfolded-protein-response module affecting drug sensitivity. These results support the framework’s utility for precision oncology, yielding stronger biomarkers for patient stratification and deeper insight into drug resistance mechanisms.

Data mining is a collection of methods used to extract useful information from large data bases. Cluster Analysis refers to the grouping of a set of data points into clusters. Most widely used partitioning methods are K-means and Fuzzy c-means (FCM) algorithms. However, they suffer from the difficulties such as random selection of initial centre values and handling outlier data points. Most of the existing clustering methods use the Euclidean distance metric. The modified fuzzy c-means algorithm (MFCM) is efficient in handling outlier data points. In this paper, a new hybrid algorithm is proposed to solve the limitations of the traditional clustering methods. The hybrid K-MFCM algorithm is tested on four real world bench mark data sets from UCI machine learning repository with various distance metrics including Euclidean, City Block and Chessboard. The cluster centroid values of hybrid algorithm are calculated for various data sets. The experimental results show that the hybrid algorithm gives good results in terms of objective function value and better fuzzy cluster validity results for chessboard distance metric than other distance metrics.

Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator's consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm as well as approximate MML estimation procedures such as variational inference (VI) are computationally time-consuming when the sample size and the number of latent factors are very large. In this work, we investigate a deep learning-based VI algorithm for exploratory item factor analysis (IFA) that is computationally fast even in large data sets with many latent factors. The proposed approach applies a deep artificial neural network model called an importance-weighted autoencoder (IWAE) for exploratory IFA. The IWAE approximates the MML estimator using an importance sampling technique wherein increasing the number of importance-weighted (IW) samples drawn during fitting improves the approximation, typically at the cost of decreased computational efficiency. We provide a real data application that recovers results aligning with psychological theory across random starts. Via simulation studies, we show that the IWAE yields more accurate estimates as either the sample size or the number of IW samples increases (although factor correlation and intercepts estimates exhibit some bias) and obtains similar results to MH-RM in less time. Our simulations also suggest that the proposed approach performs similarly to and is potentially faster than constrained joint maximum likelihood estimation, a fast procedure that is consistent when the sample size and the number of items simultaneously tend to infinity.

Recommender systems can help to relieve the dilemma called information overload. Collaborative filtering is a primary approach based on collective historical ratings to recommend items to users. One of the most competitive collaborative filtering algorithm is matrix factorization. In this paper, we proposed an alternative method. It aims to make users be spatially close to items they like and be far away from items they dislike, by connecting matrix factorization and distance metric learning. The metric and latent factors are trained simultaneously and then used to generate reliable recommendations. The experiments conducted on the real-world datasets have shown that, compared with methods only based on factorization, our method has advantage in terms of accuracy.

Seize the (intra)day: Features selection and rules extraction for tradings on high-frequency data

ABSTRACTIn this article, we respond to ‘A comment on geographically weighted regression with parameter-specific distance metrics’ by Oshan et al. (2019), published in this journal, where several concerns on the parameter-specific distance metric geographically weighted regression (PSDM GWR) technique are raised. In doing so, we review the developmental timeline of the multiscale geographically weighed regression modelling framework with related and equivalent models, including flexible bandwidth GWR, conditional GWR and PSDM GWR. In our response, we have tried to answer all the concerns raised in terms of applicability, veracity, interpretability and computational efficiency of the PSDM GWR model.

Anomaly Detection Using Adaptive Suppression

In the literature there are only few papers concerned with classification methods for multi-way arrays. The most common procedure, by far, is to unfold the multi-way data array into an ordinary matrix and then to apply the traditional multivariate tools for classification. As opposed to unfolding the data several possibilities exist for building classification models more directly based on the multi-way structure of the data. As an example, multi-way partial least squares discriminant analysis has been used as a supervised classification method, another alternative that has been investigated is to perform classification using Fisher's LDA or SIMCA on the score matrix from e.g. a PARAFAC or a Tucker model. Despite a few attempts of applying such multi-way classification approaches, no-one has looked into how such models are best built and implemented. In this work, the SIMCA method is extended to three-way arrays. Included in this work is also actual code that will work on general multi-way arrays rather than just three-way arrays. In analogy with two-way SIMCA, a decomposition model is separately built for the multi-way data for each class, using multi-way decomposition method such as PARAFAC or Tucker3. In the choice of the best class dimensionality, i.e. number of latent factors, both the results of cross-validation but mainly the sensitivity/specificity values are evaluated. In order to estimate the class limits for each class model, orthogonal and score distances are considered, and different statistics are implemented and tested to set confidence limits for these two parameters. Classification performance using different definitions of class boundaries and classification rules, including the use of cross-validated residuals and scores is compared. The proposed N-SIMCA methodology and code, besides simulated data sets of varying dimensionality, has been tested on two case studies, concerning food authentication tasks for typical food products.

The efficacy of family-based approaches to mixture model-based clustering and classification depends on the selection of parsimonious models. Current wisdom suggests the Bayesian information criterion (BIC) for mixture model selection. However, the BIC has well-known limitations, including a tendency to overestimate the number of components as well as a proclivity for, often drastically, underestimating the number of components in higher dimensions. While the former problem might be soluble through merging components, the latter is impossible to mitigate in clustering and classification applications. In this paper, a LASSO-penalized BIC (LPBIC) is introduced to overcome this problem. This approach is illustrated based on applications of extensions of mixtures of factor analyzers, where the LPBIC is used to select both the number of components and the number of latent factors. The LPBIC is shown to match or outperform the BIC in several situations.

The problem of the determination of the number of latent factors is analysed within a generalized factor model, where nonzero elements out of the diagonal of the variance-covariance matrix of error terms are admitted. In this framework, it is proved that classical methods for the selection of the factor model such as likelihood ratio test and information criteria are biased and systematically overestimate the number of factors. A Monte Carlo experiment shows the different behavior of classical methods and of two alternative criteria which have been developed for the generalized model.

Surface enhanced infrared absorption spectroscopy (SEIRA) as a green analytical chemistry approach: Coating of recycled aluminum TLC sheets with citrate capped silver nanoparticles for chemometric quantitative analysis of ternary mixtures as a green alternative to the traditional methods

Variable selection is important in fine tuning partial least squares (PLS) regression models. This study introduces a novel variable weighting method for PLS regression where the univariate response variable y is used to guide the variable weighting in a recursive manner—the method is called recursive weighted PLS or just rPLS. The method iteratively reweights the variables using the regression coefficients calculated by PLS. The use of the regression vector to make up the weights is a reasonable idea from the fact that the weights in the regression vector ideally reflect the importance of the variables. In contrast to many other variable selection methods, the rPLS method has the advantage that only one parameter needs to be estimated: the number of latent factors used in the PLS model. The rPLS model has the fascinating output that it, under normal conditions, converges to a very limited number of variables (useful for interpretation), but it will exhibit optimal regression performance before convergence, normally including covarying neighbor variables. This study examines the properties of rPLS by application to a near‐infrared spectroscopy dataset of feed samples predicting the protein content and to a metabolomics dataset modeling a reference metabolic parameter (creatinine) from nuclear magnetic resonance spectra of human urine. Copyright © 2013 John Wiley & Sons, Ltd.

Distributed computation of the knn graph for large high-dimensional point sets