Sequential projection pursuit using genetic algorithms for data mining of analytical data.

In recent year, stock investor in Indonesia increased rapidly, so it is required to do analysis about the stock that helps the investor in their investment plan. Clustering is beneficial to select the appropriate stock for investors. Unfortunately, stock prices keep varying from time to time. Consequently, it is not an easy work to select the stock for investment. In addition, stock price time series data are high dimensional data that influenced by many factors. In this study, high dimensional data are obtained by the time frame of each factor. Therefore, it is important to use a suitable technique to cluster high dimensional data. This paper presents High Dimensional Data Clustering (HDDC), a model-based clustering based on Gaussian Mixture Model, using the Expectation-Maximization (EM) algorithm. HDDC via EM algorithm gives a more robust result, and it possible to make an additional assumption. Moreover, this paper combines a high-dimensional clustering technique HDDC via EM algorithm and the most popular feature extraction technique Principal Component Analysis (PCA). This paper comparing methods of clustering technique HDDC and the combination between HDDC and PCA to know the most effective method which gives better result in clustering high-dimensional time series data. The 155 data features are reduced to 7 principal components using PCA analysis. Despite PCA has increased the time efficiency of building the model, clustering technique HDDC via EM algorithm enables to handle the high-dimensional data better than the combination with PCA.

An improved optimization algorithm and a Bayes factor termination criterion for sequential projection pursuit

High dimensional data clustering is a difficult task due to the curse of dimensionality. Traditional clustering methods usually fail to produce meaningful results for high dimensional data. Hypergraph partition is believed to be a promising method for dealing with this challenge. In this work, a new high dimensional clustering method called Merging Dense SubGraphs (MDSG) is proposed. A graph G is first constructed from the data by defining an adjacency relationship between the data points using Shared k Nearest Neighbors (SNN). Then a hypergraph is created from the graph G by defining the hyperedges to be all the maximal cliques in the graph. After the hypergraph is produced, an improved hypergraph partitioning method is used to produce the final clustering results. The proposed MDSG method is evaluated on several real high dimensional handwritten datasets, and the experimental results show that the proposed method is superior to the traditional clustering method and other hypergraph partition methods for high dimensional handwritten data clustering.

Contemporary single-cell technologies produce data with a vast number of variables at a rapid pace, making large volumes of high-dimensional data available. The exploratory analysis of such high dimensional data can be aided by intuitive low dimensional visualizations. In this work, we investigate how both discrete and continuous structures in single cell data can be captured using the recently proposed dimensionality reduction method SONG, and compare the results with commonly used methods UMAP and PHATE. Using simulated and real-world datasets, we observed that SONG preserves a variety of patterns including discrete clusters, continuums, and branching structures. More importantly, SONG produced more/equally insightful visualizations compared to UMAP and PHATE in all considered datasets. We also quantitatively validate the high-dimensional pairwise distance preservation ability of these visualization methods in the low dimensional space for the generated visualizations.

Principal Component Analysis (PCA) is a common exploratory tool used to evaluate large complex data sets. The resulting lower-dimensional representations are often valuable for pattern visualization, clustering, or classification of the data. However, PCA cannot be applied directly to many -omics data sets generated by newer technologies such as label-free mass spectrometry due to large numbers of non-random missing values. Here we present a sequential projection pursuit PCA (sppPCA) method for defining principal components in the presence of missing data. Our results demonstrate that this approach generates robust and informative low-dimensional data representations compared to commonly used imputation approaches.

Exploratory chemometric analysis of the classification of pharmaceutical substances based on chromatographic data

The aim of projection pursuit (PP) (Friedman and Tukey 1974, Huber 1985, Jones and Sibson 1987) is to find low (1-3)-dimensional projections showing the most interesting views of high-dimensional data. To reach this goal, many projections are calculated and rated by an objective function, a so called projection index I. Dependent on I, this method is able to characterise the high-dimensional data from different points of view (Fig. 6.3.1). To find good projections it is necessary to maximize I. There are several ways to do this: Gradient Algorithms (Friedman 1987), Simulated Annealing (Montanari and Guglielmi 1996) or Genetic Algorithms (Crawford 1991, Guo et al. 2000). The application of PP has some advantages over other methods. Although the projections are linear, the index may be non-linear, facilitating the detection of non-linear connections in the data. By choosing different and suitable indices, PP is able to provide various views of the data. One important feature of the index functions is robustness; this enables PP to be insensitive to outliers. Projection vectors from previous analysis can be used to view and analyse another data set from the same viewpoint. The multivariate analysis method PP can be used to classify data, comparable to supervised and unsupervised learning in artificial neural networks, but with reproducible results.

Motivation: There are a number of well-established methods such as principal component analysis (PCA) for automatically capturing systematic variation due to latent variables in large-scale genomic data. PCA and related methods may directly provide a quantitative characterization of a complex biological variable that is otherwise difficult to precisely define or model. An unsolved problem in this context is how to systematically identify the genomic variables that are drivers of systematic variation captured by PCA. Principal components (PCs) (and other estimates of systematic variation) are directly constructed from the genomic variables themselves, making measures of statistical significance artificially inflated when using conventional methods due to over-fitting.Results: We introduce a new approach called the jackstraw that allows one to accurately identify genomic variables that are statistically significantly associated with any subset or linear combination of PCs. The proposed method can greatly simplify complex significance testing problems encountered in genomics and can be used to identify the genomic variables significantly associated with latent variables. Using simulation, we demonstrate that our method attains accurate measures of statistical significance over a range of relevant scenarios. We consider yeast cell-cycle gene expression data, and show that the proposed method can be used to straightforwardly identify genes that are cell-cycle regulated with an accurate measure of statistical significance. We also analyze gene expression data from post-trauma patients, allowing the gene expression data to provide a molecularly driven phenotype. Using our method, we find a greater enrichment for inflammatory-related gene sets compared to the original analysis that uses a clinically defined, although likely imprecise, phenotype. The proposed method provides a useful bridge between large-scale quantifications of systematic variation and gene-level significance analyses.Availability and implementation: An R software package, called jackstraw, is available in CRAN.Contact: jstorey@princeton.edu

One of the main challenges that the industry faces when dealing with massive data for failure diagnosis is high dimensionality of such data. This can be tackled by dimensionality reduction method such as principal components analysis, which usually results in an improved fault diagnosis. Other available techniques include auto-encoders and its variants denoising auto-encoders and sparse auto-encoders. Most recently, variational auto-encoders are one of the most promising techniques for unsupervised learning with successful applications in image processing and speech recognition. Differently from other auto-encoder methods, variational auto-encoders use variational inference to generate a latent representation of the data and impose a distribution over the latent variables and the data itself. In this article, we propose a fully unsupervised deep variational auto-encoder-based approach for dimensionality reduction in fault diagnosis and explore the variational auto-encoder capabilities for such a task. This is achieved by comparing the latent representations provided by variational auto-encoders to the ones from principal components analysis as well as when no reduction is performed in ball bearings’ fault classification using vibration signals. To tackle massive sensor data, we propose different architectures for the variational auto-encoder’s encoder and decoder that are based on deep neural networks and deep convolutional neural networks. Experiments are also carried out by varying the data preprocessing methods for generating spectrograms and hand-engineering features as well as the use of raw vibration data, the architecture of the neural networks fault classifiers operating on the latent representations from variational auto-encoder and principal components analysis, the degree of data dimension reduction, and the size of the available labeled data used for training the fault classifiers. The results show that variational auto-encoders are a competent and promising tool for dimensionality reduction for use in fault diagnosis and worth further exploring their capabilities beyond vibration signals of ball bearing elements.

We propose a novel deep learning framework for counterfactual inference on the COMPAS dataset, utilizing a multi-task CNN-LSTM architecture. The model jointly predicts multiple outcome types: (i) count outcomes with zero inflation, modeled using zero-inflated Poisson (ZIP), zero-inflated negative binomial (ZINB), and negative binomial (NB) distributions; (ii) time-to-event outcomes, modeled via the Cox proportional hazards model. To effectively leverage the structure in high-dimensional tabular data, we integrate functional data analysis (FDA) techniques by transforming covariates into smooth functional representations using B-spline basis expansions. Specifically, we construct a pseudo-temporal index over predictor variables and fit basis expansions to each subject’s feature vector, yielding a low-dimensional set of coefficients that preserve smooth variation while reducing noise. This functional representation enables the CNN-LSTM model to capture both local and global temporal patterns in the data, including treatment-covariate interactions. Our approach estimates both population-average and individual-level treatment effects (ATE and CATE) for each outcome and evaluates predictive performance using metrics such as Poisson deviance, root mean squared error (RMSE), and the concordance index (C-index). Statistical inference on treatment effects is supported via bootstrap-based confidence intervals and hypothesis testing. Overall, this comprehensive framework facilitates flexible modeling of heterogeneous treatment effects in structured, high-dimensional data, advancing causal inference methodologies in criminal justice and related domains.

With the rapid development of hardware and software technology, modern industry has produced a large amount of high-dimensional unlabeled data, such as pictures and videos. As clusters of these data sets may exist in some subspaces, traditional algorithms are no longer applicable. The related algorithms based on sparse subspace can find clusters in the subspace, which solves the problem of high data dimension. However, their clustering process based on only one single feature, which results in their performance being particularly sensitive to this single view. Affected by the integrated algorithm, a large number of multi-view methods began to emerge. These methods improve the clustering performance by integrating the subspace expressions of multiple views, but the problem is that the complementary information of multiple views cannot be fully considered. In addition, the problem of non-uniform distribution in clusters also exists in high-dimensional data sets. In this paper, based on the multi-view subspace clustering method, a clustering algorithm based on tensor low rank expression is proposed to solve the clustering problem of high-dimensional datasets. On the one hand, this paper solves the problem of noise and data corruption by combining the method of $\ell _{2,1} $ norms, which transform the optimization problem of solving multi-view subspace expression into a low-rank expression problem of tensor to fully consider the complementarities between views. On the other hand, the proposed scheme solves the problem of non- uniform distribution in clusters in high-dimensional data by combining with the density peak algorithm based on hierarchical strategy. Experiment results show the effectiveness of the algorithm.

Due to the intricate relationship between different dimensions of high-dimensional data, subspace analysis is often conducted to decompose dimensions and give prominence to certain subsets of dimensions, i.e. subspaces. Exploring and comparing subspaces are important to reveal the underlying features of subspaces, as well as to portray the characteristics of individual dimensions. To date, most of the existing high-dimensional data exploration and analysis approaches rely on dimensionality reduction algorithms (e.g. principal component analysis and multi-dimensional scaling) to project high-dimensional data, or their subspaces, to two-dimensional space and employ scatterplots for visualization. However, the dimensionality reduction algorithms are sometimes difficult to fine-tune and scatterplots are not effective for comparative visualization, making subspace comparison hard to perform. In this article, we aggregate high-dimensional data or their subspaces by computing pair-wise distances between all data items and showing the distances with matrix visualizations to present the original high-dimensional data or subspaces. Our approach enables effective visual comparisons among subspaces, which allows users to further investigate the characteristics of individual dimensions by studying their behaviors in similar subspaces. Through subspace comparisons, we identify dominant, similar, and conforming dimensions in different subspace contexts of synthetic and real-world high-dimensional data sets. Additionally, we present a prototype that integrates parallel coordinates plot and matrix visualization for high-dimensional data exploration and incremental dimensionality analysis, which also allows users to further validate the dimension characterization results derived from the subspace comparisons.

Latent autoimmune diabetes in adults (LADA) which accounts for more than 10% of all cases of diabetes is characterized by onset after age 30, absence of ketoacidosis, insulin independence for at least 6 months, and presence of circulating islet-cell antibodies. Its marked heterogeneity in clinical features and immunological markers suggests the existence of multiple mechanisms underlying its pathogenesis. The principal component (PC) analysis is a statistical approach used for finding patterns in data of high dimension. In this study the PC analysis was applied to a set of variables from a cohort of Sardinian LADA patients to identify a smaller number of latent patterns. A list of 11 variables including clinical (gender, BMI, lipid profile, systolic and diastolic blood pressure and insulin-free time period), immunological (anti-GAD65, anti-IA-2 and anti-TPO antibody titers) and genetic features (predisposing gene variants previously identified as risk factors for autoimmune diabetes) retrieved from clinical records of 238 LADA patients referred to the Internal Medicine Unit of University of Sassari, Italy, were analyzed by PC analysis. The predictive value of each PC on the further development of insulin dependence was evaluated using Kaplan-Meier curves. Overall 4 clusters were identified by PC analysis. In component PC-1, the dominant variables were: BMI, triglycerides, systolic and diastolic blood pressure and duration of insulin-free time period; in PC-2: genetic variables such as Class II HLA, CTLA-4 as well as anti-GAD65, anti-IA-2 and anti-TPO antibody titers, and the insulin-free time period predominated; in PC-3: gender and triglycerides; and in PC-4: total cholesterol. These components explained 18, 15, 12, and 12%, respectively, of the total variance in the LADA cohort. The predictive power of insulin dependence of the four components was different. PC-2 (characterized mostly by high antibody titers and presence of predisposing genetic markers) showed a faster beta-cells failure and PC-3 (characterized mostly by gender and high triglycerides) and PC-4 (high cholesterol) showed a slower beta-cells failure. PC-1 (including dislipidemia and other metabolic dysfunctions), showed a mild beta-cells failure. In conclusion variable clustering might be consistent with different pathogenic pathways and/or distinct immune mechanisms in LADA and could potentially help physicians improve the clinical management of these patients.

There has been an intense recent activity in embedding of very high-dimensional and nonlinear data structures, much of it in the data science and machine learning literature. We survey this activity in four parts. In the first part, we cover nonlinear methods such as principal curves, multidimensional scaling, local linear methods, ISOMAP, graph-based methods and diffusion mapping, kernel based methods and random projections. The second part is concerned with topological embedding methods, in particular mapping topological properties into persistence diagrams and the Mapper algorithm. Another type of data sets with a tremendous growth is very high-dimensional network data. The task considered in part three is how to embed such data in a vector space of moderate dimension to make the data amenable to traditional techniques such as cluster and classification techniques. Arguably, this is the part where the contrast between algorithmic machine learning methods and statistical modeling, represented by the so-called stochastic block model, is at its greatest. In the paper, we discuss the pros and cons for the two approaches. The final part of the survey deals with embedding in R2, that is, visualization. Three methods are presented: t-SNE, UMAP and LargeVis based on methods in parts one, two and three, respectively. The methods are illustrated and compared on two simulated data sets; one consisting of a triplet of noisy Ranunculoid curves, and one consisting of networks of increasing complexity generated with stochastic block models and with two types of nodes.

Fault detection has been recognized in the semiconductor industry as an effective component of advanced process control framework in increasing yield and product quality. Principal component analysis (PCA) has been applied widely to semiconductor manufacturing process monitoring. However, the unique characteristics of semiconductor processes - high dimension of data, nonlinearity in most batch processes, and multimodal batch trajectories due to multiple operating conditions - significantly limit applicability of PCA to semiconductor manufacturing. This paper proposes a manifold learning algorithm, local and nonlocal preserving projection (LNPP), for feature extraction. Different from PCA, which aims to discover the global structure of Euclidean space, LNPP can find a good linear embedding that preserves local and nonlocal information. This may enable LNPP to find meaningful low-dimensional information hidden in high-dimensional observations. The Gaussian mixture model (GMM) is applied to handle process data with nonlinearity or multimodal features. GMM-based Mahalanobis distance is proposed to assess process states, and a Bayesian inference-based method is proposed to provide the process failure probability. A variable replacing-based contribution analysis method is developed to identify the process variables that are responsible for the onset of process fault. The proposed monitoring model is demonstrated through its application to a batch semiconductor etch process.