A Gaussian Process Model for Unsupervised Analysis of High Dimensional Shape Data

In recent year, stock investor in Indonesia increased rapidly, so it is required to do analysis about the stock that helps the investor in their investment plan. Clustering is beneficial to select the appropriate stock for investors. Unfortunately, stock prices keep varying from time to time. Consequently, it is not an easy work to select the stock for investment. In addition, stock price time series data are high dimensional data that influenced by many factors. In this study, high dimensional data are obtained by the time frame of each factor. Therefore, it is important to use a suitable technique to cluster high dimensional data. This paper presents High Dimensional Data Clustering (HDDC), a model-based clustering based on Gaussian Mixture Model, using the Expectation-Maximization (EM) algorithm. HDDC via EM algorithm gives a more robust result, and it possible to make an additional assumption. Moreover, this paper combines a high-dimensional clustering technique HDDC via EM algorithm and the most popular feature extraction technique Principal Component Analysis (PCA). This paper comparing methods of clustering technique HDDC and the combination between HDDC and PCA to know the most effective method which gives better result in clustering high-dimensional time series data. The 155 data features are reduced to 7 principal components using PCA analysis. Despite PCA has increased the time efficiency of building the model, clustering technique HDDC via EM algorithm enables to handle the high-dimensional data better than the combination with PCA.

Due to the intricate relationship between different dimensions of high-dimensional data, subspace analysis is often conducted to decompose dimensions and give prominence to certain subsets of dimensions, i.e. subspaces. Exploring and comparing subspaces are important to reveal the underlying features of subspaces, as well as to portray the characteristics of individual dimensions. To date, most of the existing high-dimensional data exploration and analysis approaches rely on dimensionality reduction algorithms (e.g. principal component analysis and multi-dimensional scaling) to project high-dimensional data, or their subspaces, to two-dimensional space and employ scatterplots for visualization. However, the dimensionality reduction algorithms are sometimes difficult to fine-tune and scatterplots are not effective for comparative visualization, making subspace comparison hard to perform. In this article, we aggregate high-dimensional data or their subspaces by computing pair-wise distances between all data items and showing the distances with matrix visualizations to present the original high-dimensional data or subspaces. Our approach enables effective visual comparisons among subspaces, which allows users to further investigate the characteristics of individual dimensions by studying their behaviors in similar subspaces. Through subspace comparisons, we identify dominant, similar, and conforming dimensions in different subspace contexts of synthetic and real-world high-dimensional data sets. Additionally, we present a prototype that integrates parallel coordinates plot and matrix visualization for high-dimensional data exploration and incremental dimensionality analysis, which also allows users to further validate the dimension characterization results derived from the subspace comparisons.

As a fundamental but difficult topic in computer vision, 3D object segmentation has various applications in medical image analysis, autonomous vehicles, robotics, virtual reality, lithium battery image analysis, etc. In the past, 3D segmentation was performed using hand-made features and design techniques, but these techniques could not generalize to vast amounts of data or reach acceptable accuracy. Deep learning techniques have lately emerged as the preferred method for 3D segmentation jobs as a result of their extraordinary performance in 2D computer vision. Our proposed method used a CNN-based architecture called 3D UNET, which is inspired by the famous 2D UNET that has been used to segment volumetric image data. To see the internal changes of composite materials, for instance, in a lithium battery image, it is necessary to see the flow of different materials and follow the directions analyzing the inside properties. In this paper, a combination of 3D UNET and VGG19 has been used to conduct a multiclass segmentation of publicly available sandstone datasets to analyze their microstructures using image data based on four different objects in the samples of volumetric data. In our image sample, there are a total of 448 2D images, which are then aggregated as one 3D volume to examine the 3D volumetric data. The solution involves the segmentation of each object in the volume data and further analysis of each object to find its average size, area percentage, total area, etc. The open-source image processing package IMAGEJ is used for further analysis of individual particles. In this study, it was demonstrated that convolutional neural networks can be trained to recognize sandstone microstructure traits with an accuracy of 96.78% and an IOU of 91.12%. According to our knowledge, many prior works have applied 3D UNET for segmentation, but very few papers extend it further to show the details of particles in the sample. The proposed solution offers a computational insight for real-time implementation and is discovered to be superior to the current state-of-the-art methods. The result has importance for the creation of an approximately similar model for the microstructural analysis of volumetric data.

Contemporary single-cell technologies produce data with a vast number of variables at a rapid pace, making large volumes of high-dimensional data available. The exploratory analysis of such high dimensional data can be aided by intuitive low dimensional visualizations. In this work, we investigate how both discrete and continuous structures in single cell data can be captured using the recently proposed dimensionality reduction method SONG, and compare the results with commonly used methods UMAP and PHATE. Using simulated and real-world datasets, we observed that SONG preserves a variety of patterns including discrete clusters, continuums, and branching structures. More importantly, SONG produced more/equally insightful visualizations compared to UMAP and PHATE in all considered datasets. We also quantitatively validate the high-dimensional pairwise distance preservation ability of these visualization methods in the low dimensional space for the generated visualizations.

Data visualization is one of the techniques to understand the patterns of data in graphical methods. Data visualization is an effective tool for transforming raw data into actionable insights and facilitating data-driven decision-making. High-dimensional synthetic data are datasets created artificially with an abundance of attributes or aspects. This type of synthetic data is particularly helpful for attempting to assess machine learning algorithms and data analysis techniques in scenarios with a large number of factors. The method can be difficult because of the more complicated nature of high-dimensional data, but it is necessary for a variety of applications, such as testing machine learning algorithms, evaluating data analysis techniques, and exploring model behaviour in high-dimensional spaces. These trained high-dimensional synthetic data are given to the visualization techniques to produce graphical representation and better decision-making models. This chapter elaborates on visualizing synthetic high-dimensional data for better understanding by common men.

In standard Principal Component Analysis (PCA) comes one problem in addressing the set of incomplete data. The standard PCA procedure on incomplete data is to eliminate (listwise deletion procedure) or using the mean of the variable, this procedure may result in loss information from these observations. Another method used is to integrate Expectation Maximization (EM) to the method of Probabilistic Principal Component Analysis (PPCA). But PPCA can produce overfitting response prediction. In this study discussed the Variational Bayesian Principal Component Analysis (VBPCA) which is a method of development of PPCA method by incorporating prior information from the distribution of the principal components of the model parameters. From the simulation studies by eliminating the data through the concept of missing at random (MAR), obtained results that the value of the correlation scores principal components complete data with the principal component score predicted results PPCA method is superior when compared with VBPCA, as well as to the value of the correlation scores for the various percentages are generally incomplete data. However, judging from the size of a match between the response to predictions by the size normalized root mean square error of prediction (NRMSEP) VBPCA method produces better than PPCA.

This chapter covers the state-of-the-art multivariate statistical methods designed for high dimensional multiset omics data analysis. Recent biotechnological developments have enabled large-scale measurement of various biomolecular data, such as genotypic and phenotypic data, dispersed over various omics domains. An emergent research direction is to analyze these data sources using an integrated approach to better model and understand the underlying biology of complex disease conditions. However, comprehensive analysis techniques that can handle both the size and complexity, and at the same time can account for the hierarchical structure of such data, are lacking. An overview of some of the developments in multivariate techniques for high-dimensional omics data analysis, highlighting two well-known multivariate methods, canonical correlation analysis (CCA) and redundancy analysis (RDA), is provided in this chapter. Penalized versions of CCA are widespread in the omics data analysis field, and there is recent work on multiset penalized RDA that is applicable to multiset omics data. How these methods meet the statistical challenges that come with high-dimensional multiset omics data analysis and help to further our understanding of the human condition in terms of health and disease are presented. Additionally, the current challenges to be resolved in the field of omics data analysis are discussed.

Uncovering driver genes is crucial for understanding heterogeneity in cancer. L1-type regularization approaches have been widely used for uncovering cancer driver genes based on genome-scale data. Although the existing methods have been widely applied in the field of bioinformatics, they possess several drawbacks: subset size limitations, erroneous estimation results, multicollinearity, and heavy time consumption. We introduce a novel statistical strategy, called a Recursive Random Lasso (RRLasso), for high dimensional genomic data analysis and investigation of driver genes. For time-effective analysis, we consider a recursive bootstrap procedure in line with the random lasso. Furthermore, we introduce a parametric statistical test for driver gene selection based on bootstrap regression modeling results. The proposed RRLasso is not only rapid but performs well for high dimensional genomic data analysis. Monte Carlo simulations and analysis of the “Sanger Genomics of Drug Sensitivity in Cancer dataset from the Cancer Genome Project” show that the proposed RRLasso is an effective tool for high dimensional genomic data analysis. The proposed methods provide reliable and biologically relevant results for cancer driver gene selection.

Numerous variable selection techniques have been developed for complete high-dimensional data but very few of them for censored data. The techniques for complete data must be modified if censoring is present. In this paper, we consider the variable selection technique for accelerated failure time (AFT) models by extending the ranking-based variable selection (RBVS) algorithm and its iterative procedure as proposed in the work of Baranowski et al. through the Stute’s weighted least square technique. Simulation studies are conducted to demonstrate the performance of the proposed methods. We further illustrate the performance of this method with a mantle cell lymphoma microarray example. When there is no correlation among the covariates, the proposed method outperforms the iterative sure independence screening and stability selection methods in terms of overall performance for high-dimensional data. Real data analysis also suggests that the proposed method can be chosen for high-dimensional censored data analysis in parallel to other methods in the literature.

High-dimensional data is a key study object for both machine learning (ML) and information visualization. In the field of visualization, dimensionality reduction (DR) methods, also known as projections, are one of the most frequently used classes of techniques for visually exploring large and high-dimensional datasets. In ML, high-dimensional data is generated and processed by classifiers and regressors, which increasingly require visualization for explanation and exploration. This thesis focuses on a recent visualization technique called decision maps. A decision map is a 2D image that visualizes the decision boundaries of a classifier in the data space, and can be used to explain and improve the behavior of ML classifiers. Constructing decision maps essentially involves a DR method and its inverse process (inverse projection). As such, ML techniques can help to create decision maps by providing improved (inverse) projections. We begin with a case study applying decision maps to explain the classification of mineral deposit genesis. Our findings show that decision maps provide extra insights into the mineral classification model, aiding geologists in interpreting the model. However, we also identified gaps in current decision map techniques that present opportunities for improvement. Following this case study, we conducted a comprehensive evaluation of three notable decision map techniques. Our evaluation shows that each technique has unique advantages and disadvantages. Our results can guide users in selecting the most suitable technique for specific tasks. A particularly salient finding of this evaluation was that all tested decision maps exhibit a surface-like behavior when applied to a 3D dataset. We explored the aforementioned surface-like behavior of decision maps across more scenarios. By estimating the intrinsic dimensionality of the maps, we found that existing decision map methods cover only a small portion of the intrinsic dimensionality of high-dimensional data spaces. This finding highlights fundamental limitations in all current approaches to constructing decision maps. To address these limitations, we propose a novel approach for computing inverse projections with the support of ML. Our method allows users to interactively control the location of the inversely projected points, thus also of visualizations like decision maps, within the high-dimensional space. In this way, users can explore larger parts of the data space, bypassing the practical limitations imposed by the aforementioned surface-like behavior of decision maps. We demonstrate the effectiveness of our approach through its application to a style transfer task. Finally, we introduce an accelerated computation method for decision maps. Our method significantly reduces the computation time for both basic decision maps and enhanced variations thereof such as gradient maps. The acceleration facilitates the further deployment of such visualizations in interactive visual analytics workflows for classifier engineering.

This doctoral thesis presents a framework for machine based interpretation of Malaysian Sign Language (MSL). The goal is to explore and develop necessary core algorithms that are capable of recognizing hand signs, based on the monocular/2D visual input of a low-cost hardware (Digital Webcam), without the assistance of visual-markers on hands. Successful hand sign interpretation with the aforementioned constrained visual system is challenging. Economical hardware devices often provide noisy/cluttered raw data with limited useful information. Non-involvement of visual markers imposes the requirement of additional strategies to disambiguate occlusions and background complexities. Natural variations in hand posture/gesture performances further convolute the recognition task. Addressing these challenges require computationally heavy algorithms in image processing, machine learning and pattern recognition. Thus vision-based sign language translator frameworks have to be multifaceted and versatile, in order to address innumerable practical concerns. In this research, hand postures interpretation is achieved by statistical projection methods. Analysis of high dimensional hand shape data is a complex task. Thus reducing dimensions via known projection methods aids in alleviating some of these complexities. Locality Preserving Projection (LPP) has been employed to accomplish this goal, while Principal Component Analysis (PCA) has been used as the benchmark method. Both methods were extensively tested against a hand posture database of 24 alphabet signs from MSL. As for the hand gesture interpretation, parameter-less Gaussian process based machine learning methods are proposed, as an alternative to current status quo. A variation of the Gaussian latent variable model – Gaussian Process Dynamic Model (GPDM) – has been employed for hand gesture interpretation. This has been comparatively studied against the established Hidden Markov Model (HMM). These machine learning methods were thoroughly tested against a hand gesture database of 64 signs from MSL. Grammatical sentence construction offers both intelligible sentences, and a mean of identifying erroneous sign classifications. In this thesis, a proof-of-concept on: use of Context Free Grammar (CFG) in successful parsing of meaningful sentences based on recognised gesture words, and how it can be extended in identifying (and correcting) erroneous sign classifications, is demonstrated with 92 sentences and the output of GPDM gesture classification. The novelty and contribution of this research: this is the first undertaking of markerless vision-based translation for MSL. Both LPP and GPDM are introduced to the sign language research field as an outcome of this investigation. The preliminary study and demonstration of grammatical sentence construction for translated gesture signs, is the first of its kind to the best of author's knowledge.

Gaussian mixture model (GMM) techniques are popular for speaker identification. Theoretically, each Gaussian function should have a full covariance matrix. However, the diagonal covariance matrix is usually used because the inverse of diagonal covariance matrix can be easily calculated via expectation maximization (EM) algorithm. This paper proposes a new probabilistic principal component analysis (PPCA) model for speaker identification. The full covariance of speaker’s data is considered. This model is originated from factor analysis theory. The probability distributions using PPCA are well defined. In particular, GMM and PPCA are found to be equivalent when using diagonal covariance matrix. In this study, we derive a novel PPCA model selection and establish models for different speakers. Applying PPCA model selection, we can dynamically determine the numbers of speech features and mixture components. Experiments show that PPCA achieves desirable speaker recognition performance with proper model regularization.

Dimensionality reduction combined with particle swarm optimization algorithm for solving high-dimensional nuclear data target accuracy evaluation problem

Clustering high dimensional data with sparse features is challenging because pairwise distances between data items are not informative in high dimensional space. To address this challenge, we propose two novel semi-supervised clustering methods that incorporate prior knowledge in the form of pairwise cluster membership constraints. In particular, we project high-dimensional data onto a much reduced-dimension subspace, where rough clustering structure defined by the prior knowledge is strengthened. Metric learning is then performed on the subspace to construct more informative pairwise distances. We also propose to propagate constraints locally to improve the informativeness of pairwise distances. When the new methods are evaluated using two real benchmark data sets, they show substantial improvement using only limited prior knowledge.

We propose using graph theoretic results to develop an infrastructure that tracks movement from a display of one set of variables to another. The illustrative example throughout is the real-time morphing of one scatterplot into another. Hurley and Oldford (J Comput Graph Stat 2010) made extensive use of the graph having variables as nodes and edges indicating a paired relationship between them. The present paper introduces several new graphs derivable from this one whose traversals can be described as particular movements through high dimensional spaces. These are connected to known results in graph theory and the graph theoretic results applied to the problem of visualizing high-dimensional data.