Sparse regularization discriminant analysis for face recognition

Sequential projection pursuit (SPP) is proposed to detect inhomogeneities (clusters) in high-dimensional analytical data. Such inhomogeneities indicate that there are groups of objects (samples) with different chemical characteristics. The method is compared with principal component analysis (PCA). PCA is generally applied to visually explore structure in high-dimensional data, but is not specifically used to find clustering tendency. Projection pursuit (PP) is specifically designed to find inhomogeneities, but the original method is computationally very intensive. SPP combines the advantages of both methods and overcomes most of their weak points. In this method, latent variables are obtained sequentially according to their importance measured by the entropy index. This involves an optimization step, which is achieved by using a genetic algorithm. The performance of the method is demonstrated and evaluated, first on simulated data sets, and then on near-infrared and gas chromatography data sets. It is shown that SPP indeed reveals more easily information about inhomogeneities than PCA.

There is a growing interest in subspace learning techniques for face recognition; however, the excessive dimension of the data space often brings the algorithms into the curse of dimensionality dilemma. In this paper, we present a novel approach to solve the supervised dimensionality reduction problem by encoding an image object as a general tensor of second or even higher order. First, we propose a discriminant tensor criterion, whereby multiple interrelated lower dimensional discriminative subspaces are derived for feature extraction. Then, a novel approach, called k-mode optimization, is presented to iteratively learn these subspaces by unfolding the tensor along different tensor directions. We call this algorithm multilinear discriminant analysis (MDA), which has the following characteristics: 1) multiple interrelated subspaces can collaborate to discriminate different classes, 2) for classification problems involving higher order tensors, the MDA algorithm can avoid the curse of dimensionality dilemma and alleviate the small sample size problem, and 3) the computational cost in the learning stage is reduced to a large extent owing to the reduced data dimensions in k-mode optimization. We provide extensive experiments on ORL, CMU PIE, and FERET databases by encoding face images as second- or third-order tensors to demonstrate that the proposed MDA algorithm based on higher order tensors has the potential to outperform the traditional vector-based subspace learning algorithms, especially in the cases with small sample sizes.

In this paper, we present a Sub-pattern based Multi-manifold Discriminant Analysis (SpMMDA) algorithm for face recognition. Unlike existing Multi-manifold Discriminant Analysis (MMDA) approach which is based on holistic information of face image for recognition, SpMMDA operates on sub-images partitioned from the original face image and then extracts the discriminative local feature from the sub-images separately. Moreover, the structure information of different sub-images from the same face image is considered in the proposed method with the aim of further improve the recognition performance. Extensive experiments on three standard face databases (Extended YaleB, CMU PIE and AR) demonstrate that the proposed method is effective and outperforms some other sub-pattern based face recognition methods.

In this paper, a robust face recognition algorithm is proposed, which is based on the elastic graph matching (EGM) and discriminative feature analysis algorithm. We introduce a cost function for the EGM taking account of variations in face pose and facial expressions, and propose its optimization procedure. Our proposed cost function uses a set of Gabor-wavelet-based features, called robust jet, which are robust against the variations. The robust jet is defined in terms of discrete Fourier transform coefficients of Gabor coefficients. To cope with the difference between face poses of test face and reference faces, 2 x 2 warping matrix is incorporated in the proposed cost function. For the discriminative feature analysis, linear projection discriminant analysis and kernel-based projection discriminant analysis are introduced. These methods are motivated to solve the small-size problem of training samples. The basic idea of PDA is that a class is represented by a subspace spanned by some training samples of the class instead of using sample mean vector, that the distance from a pattern to a class is defined by using the error vector between the pattern and its projection to the subspace representing the class, and that an optimum feature selection rule is developed using the distance concept in a similar way as in the conventional linear discriminant analysis. In order to evaluate the performance of our face recognition algorithm, we carried out some experiments using the well-known FERET face database, and compared the performance with recently developed approaches. We observed that our algorithm outperformed the compared approaches.

Contemporary single-cell technologies produce data with a vast number of variables at a rapid pace, making large volumes of high-dimensional data available. The exploratory analysis of such high dimensional data can be aided by intuitive low dimensional visualizations. In this work, we investigate how both discrete and continuous structures in single cell data can be captured using the recently proposed dimensionality reduction method SONG, and compare the results with commonly used methods UMAP and PHATE. Using simulated and real-world datasets, we observed that SONG preserves a variety of patterns including discrete clusters, continuums, and branching structures. More importantly, SONG produced more/equally insightful visualizations compared to UMAP and PHATE in all considered datasets. We also quantitatively validate the high-dimensional pairwise distance preservation ability of these visualization methods in the low dimensional space for the generated visualizations.

Hunting for significance: Bayesian classifiers under a mixture loss function

Segmentation has long been a cornerstone of strategic marketing, grounded in the recognition that markets are inherently heterogeneous and that effective strategies must reflect meaningful differences in customer needs and responses. However, its relevance and practical applicability have been widely debated in recent years, alongside the emergence of numerous alternative approaches. Historically, advances in computational power have enabled increasingly sophisticated segmentation techniques. More recently, rapid progress in artificial intelligence (AI), data availability, and methods has opened up new possibilities while also raising the question of whether segmentation in its traditional form may become obsolete. Against this backdrop, this Special Issue on the revitalization of segmentation was conceived. It aims to encourage scholars to reassess established approaches, incorporate technological developments, and develop novel frameworks. The contributions advance theory guided and data enabled segmentation to improve targeting, pricing, and the estimation of segment specific effects. Modern machine learning techniques help uncover latent structures in high dimensional data, enhancing segment formation and predictive performance, while emerging approaches leverage unstructured data such as text to identify segment level drivers and strategic groups. At the same time, the growing reliance on data intensive methods highlights the importance of data quality, validation, and governance to avoid misguided decisions. Overall, the contributions in this Special Issue demonstrate that segmentation is far from obsolete. On the contrary, the opportunities for segmentation research are greater than ever, calling on researchers to revitalize the field.

Purpose: Hyperperfusion detected on arterial spin labeling (ASL) cerebral blood flow (CBF) images acquired after acute ischemic stroke (AIS) onset has been shown to be significantly associated with development of intracerebral hemorrhage. ASL data is often too noisy for accurate voxel-based analysis alone. The purpose of this study was to develop a machine learning model for the voxel-based detection of tissue at risk for hemorrhagic transformation (HT) utilizing multiple MRI modalities. Materials and Methods: The present study utilized clinical MRI and ASL image data acquired from 67 AIS patients shortly after endovascular therapy. A novel regional cuboid sampling framework was developed for machine learning training, in which local cuboids were extracted from the CBF map, DWI, FLAIR, and TSE images before being matched with GRE-based manually drawn bleed groundtruth delineations. Kernel spectral regression (KSR) uses the eigenvectors of the graph representation to reveal low dimensional structure in high dimensional data. A two-layer feed-forward neural network was then built with 10 neurons in the sigmoid hidden layer and trained with scaled conjugate gradient backpropagation to classify cuboid inputs into likelihood of HT. Results: The proposed multimodal regional framework reached an accuracy of 80.59 ± 3% in detecting hemorrhage with KSR on our dataset (better than a voxel-based prediction on CBF with 72.80 ± 5% accuracy). Using the neural network training, the framework reached an improved accuracy of 95.1% ± 0.6%. Figure 1 shows how the regions with high likelihood of hemorrhage match well with the manually drawn regions in the reference GRE map. Conclusion: Kernel spectral regression and neural networks can provide more accurate detection of tissues at risk for HT. Although CBF can inform AIS patient clinical outcome, the addition of multi-modal MRI data into the regional cuboid framework substantially improves the voxel-based HT detection accuracy.

Understanding the structure of multidimensional patterns, especially in unsupervised cases, is of fundamental importance in data mining, pattern recognition and machine learning. Several algorithms have been proposed to analyze the structure of high dimensional data based on the notion of manifold learning. These algorithms have been used to extract the intrinsic characteristics of different types of high dimensional data by performing nonlinear dimensionality reduction. Most of these algorithms rely on the Euclidean metric and manually choosing a neighborhood size. They cannot recover the intrinsic geometry of data manifold automatically and accurately for some data sets. In this paper, we propose an adaptive version of ISOMAP that integrates the advantages of local and global manifold learning algorithms. Faster convergence rate of this algorithm is obtained by replacing the Euclidean metric with the arc length metric as the sum of second-order approximation to the geodesic distance. Our experiments on synthetic data as well as real world images demonstrate that our proposed algorithm can achieve better performance than that of ISOMAP and solve some harder data sets that are impossible for existing global methods.

We propose a novel deep learning framework for counterfactual inference on the COMPAS dataset, utilizing a multi-task CNN-LSTM architecture. The model jointly predicts multiple outcome types: (i) count outcomes with zero inflation, modeled using zero-inflated Poisson (ZIP), zero-inflated negative binomial (ZINB), and negative binomial (NB) distributions; (ii) time-to-event outcomes, modeled via the Cox proportional hazards model. To effectively leverage the structure in high-dimensional tabular data, we integrate functional data analysis (FDA) techniques by transforming covariates into smooth functional representations using B-spline basis expansions. Specifically, we construct a pseudo-temporal index over predictor variables and fit basis expansions to each subject’s feature vector, yielding a low-dimensional set of coefficients that preserve smooth variation while reducing noise. This functional representation enables the CNN-LSTM model to capture both local and global temporal patterns in the data, including treatment-covariate interactions. Our approach estimates both population-average and individual-level treatment effects (ATE and CATE) for each outcome and evaluates predictive performance using metrics such as Poisson deviance, root mean squared error (RMSE), and the concordance index (C-index). Statistical inference on treatment effects is supported via bootstrap-based confidence intervals and hypothesis testing. Overall, this comprehensive framework facilitates flexible modeling of heterogeneous treatment effects in structured, high-dimensional data, advancing causal inference methodologies in criminal justice and related domains.

Symmetric low-rank representation with adaptive distance penalty for semi-supervised learning

Over the years, many dimensionality reduction algorithms have been proposed for learning the structure of high dimensional data by linearly or non-linearly transforming it into a low-dimensional space. Some techniques can keep the local structure of data, while the others try to preserve the global structure. In this paper, we propose a linear dimensionality reduction technique that characterizes the local and global properties of data by firstly applying k-means algorithm on original data, and then finding the projection by simultaneously globally maximizing the between-cluster scatter matrix and locally minimizing the within-cluster scatter matrix, which actually keeps both local and global structure of data. Low complexity and structure preserving are two main advantages of the proposed technique. The experiments on both artificial and real data sets show the effectiveness and novelty of proposed algorithm in visualization and classification tasks.

Persistent homology (PH) is a robust method to compute multi-dimensional geometric and topological features of a dataset. Because these features are often stable under certain perturbations of the underlying data, are often discriminating, and can be used for visualization of structure in high-dimensional data and in statistical and machine learning modeling, PH has attracted the interest of researchers across scientific disciplines and in many industry applications. However, computational costs may present challenges to effectively using PH in certain data contexts, and theoretical stability results may not hold in practice. In this paper, we define, implement, and investigate a simplicial complex construction for computing persistent homology of Euclidean point cloud data, which we call the Delaunay-Rips complex (DR). By only considering simplices that appear in the Delaunay triangulation of the point cloud and assigning the Vietoris-Rips weights to simplices, DR avoids potentially costly computations in the persistence calculations. We document and compare a Python implementation of DR with other simplicial complex constructions for generating persistence diagrams. By imposing sufficient conditions on point cloud data, we are able to theoretically justify the stability of the persistence diagrams produced using DR. When the Delaunay triangulation of the point cloud changes under perturbations of the points, we prove that DR-produced persistence diagrams exhibit instability. Since we cannot guarantee that real-world data will satisfy our stability conditions, we demonstrate the practical robustness of DR for persistent homology in comparison with other simplicial complexes in machine learning applications. We find in our experiments that using DR in an ML-TDA pipeline performs comparatively well as using other simplicial complex constructions.

New independent component analysis tools for time series

Modern infrastructures such as transportation and communication networks are large-scale systems with complex dependence structures between various sub-systems. Human-interpretable performance targets in such systems are often represented in terms of lower-dimensional projections of the high-dimensional state space. We consider the problem of designing control strategies for high-dimensional systems that lack a detailed model. To do so, we leverage the ability of copulas to represent dependant structures in high-dimensional data, and approximate the state space model through inverse sampling. We demonstrate the applicability of the control policies obtained from our methodology through a data-driven case study of controlling flight delays within the US air transportation network.