Sparse representations of high dimensional neural data

The development of advanced recording and measurement devices in scientific fields is producing high-dimensional time series data. Vector autoregressive (VAR) models are well suited for inferring Granger-causal networks from high dimensional time series data sets, but accurate inference at scale remains a central challenge. We have recently introduced a flexible and scalable statistical machine learning framework, Union of Intersections (UoI), which enables low false-positive and low false-negative feature selection along with low bias and low variance estimation, enhancing interpretation and predictive accuracy. In this paper, we scale the UoI framework for VAR models (algorithm UoI V AR ) to infer network connectivity from large time series data sets (TBs). To achieve this, we optimize distributed convex optimization and introduce novel strategies for improved data read and data distribution times. We study the strong and weak scaling of the algorithm on a Xeon-phi based supercomputer (100,000 cores). These advances enable us to estimate the largest VAR model as known (1000 nodes, corresponding to 1M parameters) and apply it to large time series data from neurophysiology (192 neurons) and finance (470 companies).

In recent year, stock investor in Indonesia increased rapidly, so it is required to do analysis about the stock that helps the investor in their investment plan. Clustering is beneficial to select the appropriate stock for investors. Unfortunately, stock prices keep varying from time to time. Consequently, it is not an easy work to select the stock for investment. In addition, stock price time series data are high dimensional data that influenced by many factors. In this study, high dimensional data are obtained by the time frame of each factor. Therefore, it is important to use a suitable technique to cluster high dimensional data. This paper presents High Dimensional Data Clustering (HDDC), a model-based clustering based on Gaussian Mixture Model, using the Expectation-Maximization (EM) algorithm. HDDC via EM algorithm gives a more robust result, and it possible to make an additional assumption. Moreover, this paper combines a high-dimensional clustering technique HDDC via EM algorithm and the most popular feature extraction technique Principal Component Analysis (PCA). This paper comparing methods of clustering technique HDDC and the combination between HDDC and PCA to know the most effective method which gives better result in clustering high-dimensional time series data. The 155 data features are reduced to 7 principal components using PCA analysis. Despite PCA has increased the time efficiency of building the model, clustering technique HDDC via EM algorithm enables to handle the high-dimensional data better than the combination with PCA.

In real-world grasping tasks, unstable grasping will lead to sliding between the object and the robotic nd-effectors. When the target object owns special properties (e.g. glasses, etc.), sliding may cause damage to both the object and the grasping environment, so it is necessary to predict and classify whether sliding occurs at the moment of grasping. Aiming at predicting sliding, this paper proposes a sliding prediction solution for time series tactile data obtained by crude sensor Xela and designs a novel neural network: the force motion tracking network. It predicts the variation trend of the frictional force at the moment of contact, and then concatenate the collected and predicted friction force data as the input of the LSTM network to classify the contact state (whether there is slippage). In this paper, 660 sets of tactile time series data are collected, and we process high dimensional tactile time series data into video data. This data processing method can be applied to other similar tactile sensors. Meanwhile, we also verify the proposed model, the mean square error of our force tracking network is much smaller than ConvLSTM, and the prediction accuracy of our network can reach to 93.5%,which is higher than other methods.

This paper presents a new and novel hybrid modeling method for the segmentation of high dimensional time-series data using the mixture of the sparse principal components regression (MIX-SPCR) model with information complexity () criterion as the fitness function. Our approach encompasses dimension reduction in high dimensional time-series data and, at the same time, determines the number of component clusters (i.e., number of segments across time-series data) and selects the best subset of predictors. A large-scale Monte Carlo simulation is performed to show the capability of the MIX-SPCR model to identify the correct structure of the time-series data successfully. MIX-SPCR model is also applied to a high dimensional Standard & Poor’s 500 (S&P 500) index data to uncover the time-series’s hidden structure and identify the structure change points. The approach presented in this paper determines both the relationships among the predictor variables and how various predictor variables contribute to the explanatory power of the response variable through the sparsity settings cluster wise.

Upper respiratory tract infections (URTIs) represent a large strain on primary health resources. To mitigate URTI transmission and public health burdens, it is important to pre-empt and provide forward guidance on URTI burden, while taking into account various facets which influence URTI transmission. This is so that appropriate public health measures can be taken to mitigate strain on primary care resources. This study describes a new approach to forecasting URTIs which can be used for national public health resource planning. Specifically, using environmental and disease data comprising more than 1000 dimensions, we developed sub-models which optimizes model explainability, in-sample model fit, predictive accuracy and combines many weaker predictors over a 2-month time horizon to generate direct, point forecasts over a 1-8 week ahead forecast horizon. Predictive performance was evaluated using rolling out-of-sample forecast assessment within both periods with/without structural breaks in transmission over the period of 2012-2022. We showed that forecast combinations of 5 other forecasting models had better and more consistent predictive performance than other modelling approaches, over periods with and without structural breaks in transmission dynamics. Furthermore, epidemiological analysis on high dimensional data was enabled using post-selection inference, to show the dynamic association between lower temperature, increases in past relative humidity and absolute humidity and increased URTIs attendance. The methods proposed can be used for outbreak preparedness and guide healthcare resource planning, in both stable periods of transmission and periods where structural breaks in data occur.

Due to technological advances there is the possibility to collect datasets of growing size and dimension. On the other hand, standard techniques do not allow the easy management of large dimensional data and new techniques need to be considered in order to find useful results. Another relevant problem is the information loss due to the aggregation in large data sets. We need to take into account this information richness present in the data which could be hidden in the data visualization process. Our proposal - which contributes to the literature on temporal data mining - is to use some new types of time series defined as the beanplot time series in order to avoid the aggregation and to cluster original high dimensional time series effectively. In particular we consider the case of high dimensional time series and a clustering approach based on the statistical features of the beanplot time series.

Recently, anomaly detection has acquired a realistic response from data mining scientists as a graph of its reputation has increased smoothly in various practical domains like product marketing, fraud detection, medical diagnosis, fault detection and so many other fields. High dimensional data subjected to outlier detection poses exceptional challenges for data mining experts and it is because of natural problems of the curse of dimensionality and resemblance of distant and adjoining points. Traditional algorithms and techniques were experimented on full feature space regarding outlier detection. Customary methodologies concentrate largely on low dimensional data and hence show ineffectiveness while discovering anomalies in a data set comprised of a high number of dimensions. It becomes a very difficult and tiresome job to dig out anomalies present in high dimensional data set when all subsets of projections need to be explored. All data points in high dimensional data behave like similar observations because of its intrinsic feature i.e., the distance between observations approaches to zero as the number of dimensions extends towards infinity. This research work proposes a novel technique that explores deviation among all data points and embeds its findings inside well established density-based techniques. This is a state of art technique as it gives a new breadth of research towards resolving inherent problems of high dimensional data where outliers reside within clusters having different densities. A high dimensional dataset from UCI Machine Learning Repository is chosen to test the proposed technique and then its results are compared with that of density-based techniques to evaluate its efficiency.

Discovering dominant patterns and exploring dynamic behaviors especially critical state transitions and tipping points in high-dimensional time-series data are challenging tasks in study of real-world complex systems, which demand interpretable data representations to facilitate comprehension of both spatial and temporal information within the original data space. This study proposes a general and analytical ultralow-dimensionality reduction method for dynamical systems named spatial-temporal principal component analysis (stPCA) to fully represent the dynamics of a high-dimensional time-series by only a single latent variable without distortion, which transforms high-dimensional spatial information into one-dimensional temporal information based on nonlinear delay-embedding theory. The dynamics of this single variable is analytically solved and theoretically preserves the temporal property of original high-dimensional time-series, thereby accurately and reliably identifying the tipping point before an upcoming critical transition. Its applications to real-world datasets such as individual-specific heterogeneous ICU records demonstrate the effectiveness of stPCA, which quantitatively and robustly provides the early-warning signals of the critical/tipping state on each patient.

Background/Objectives: This paper aims to develop a novel intelligent recommender system suitable for high dimensional data where multiple factor variables influence on multiple response variables. Methods/Statistical Analysis: This paper suggests that the structure of the cause-and-effect relations among the variables can be represented in a simple form called structured association network model (SANM). Based on the SANM and conventional data mining techniques such as association rule mining and naive Bayesian classifier, the proposed recommender system computes three novel recommendation scores for each response variables, and the variables with high scores can be selected for recommendation. Findings: For illustration, the proposed recommender system has been applied to a mass health examination result data set. Owing to its simple structure, a SANM for a given data set can be easily obtained by simply identifying the factor and the response variables, and the experiment results revealed that the proposed system can identify the recommendable items for the individuals more effectively than the traditional classification techniques such as naive Bayesian classifier. Consequently, we can conclude that the proposed recommender system can deal with high dimensional transaction data set in more effective manner than the traditional approaches where the underlying semantic relationships among the variables are not considered. Applications/Improvements: The proposed recommender system is useful for evaluation of the potential risks of specific diseases. Moreover, the recommendation scores can also be used as a tool for feature construction.

Pavement distress images are typical high dimensional nonlinear data. Manifold learning algorithms can find the intrinsic characteristic hidden in the distress images, which helps to better recognize them. Unlike most of manifold learning algorithms, multilayer autoencoders have solved the data reconstructed problem through building a bi-directional mapping between the high dimensional data and the low dimensional data. An automatic pavement distress image recognition method based on multilayer autoencoders was proposed, which combined the image processing method and multilayer autoencoders. In the method, the distress images were firstly processed with the image processing method. Then the images were reduced dimensions and reconstructed with multilayer autoencoders. Lastly, the distress type was recognized through the network. Experiments showed that the recognition accuracy with the proposed method was great higher than that with the BP neural network.

Fuzzy partition based soft subspace clustering and its applications in high dimensional data

High dimensional data clustering is a difficult task due to the curse of dimensionality. Traditional clustering methods usually fail to produce meaningful results for high dimensional data. Hypergraph partition is believed to be a promising method for dealing with this challenge. In this work, a new high dimensional clustering method called Merging Dense SubGraphs (MDSG) is proposed. A graph G is first constructed from the data by defining an adjacency relationship between the data points using Shared k Nearest Neighbors (SNN). Then a hypergraph is created from the graph G by defining the hyperedges to be all the maximal cliques in the graph. After the hypergraph is produced, an improved hypergraph partitioning method is used to produce the final clustering results. The proposed MDSG method is evaluated on several real high dimensional handwritten datasets, and the experimental results show that the proposed method is superior to the traditional clustering method and other hypergraph partition methods for high dimensional handwritten data clustering.

In describing high dimensional discrete response data, mathematical and statistical issues arise that require multivariate procedures that are not based on normal distributions, that is, the mathematical representation of high dimensional discrete response data (Event Spaces) requires a representation in lower dimensional parameter spaces consistent with the discrete properties of the Event Space. Mapping discrete responses to latent discrete classes has the limitation of not representing real individual variation within the categories. The use of a fuzzy partition model is proposed which describes individuals in terms of partial membership in multiple latent categories which represents bounded discrete event spaces with significant third and higher order moments. We discuss statistical issues arising in identifying both the deterministic and the stochastic variation of data when applications involve systematic variation due to observed and unobserved variables. We present an empirical Bayes-maximum likelihood estimation scheme for the application of the fuzzy partition models.

Outlier detection aims to find a data sample that is significantly different from other data samples. Various outlier detection methods have been proposed and have been shown to be able to detect anomalies in many practical problems. However, in high dimensional data, conventional outlier detection methods often behave unexpectedly due to a phenomenon called the curse of dimensionality. In this paper, we compare and analyze outlier detection performance in various experimental settings, focusing on text data with dimensions typically in the tens of thousands. Experimental setups were simulated to compare the performance of outlier detection methods in unsupervised versus semi-supervised mode and uni-modal versus multi-modal data distributions. The performance of outlier detection methods based on dimension reduction is compared, and a discussion on using k-NN distance in high dimensional data is also provided. Analysis through experimental comparison in various environments can provide insights into the application of outlier detection methods in high dimensional data.

High-dimensional time series data appear in many scientific areas in the current data-rich environment. Analysis of such data poses new challenges to data analysts because of not only the complicated dynamic dependence between the series, but also the existence of aberrant observations, such as missing values, contaminated observations, and heavy-tailed distributions. For high-dimensional vector autoregressive (VAR) models, we introduce a unified estimation procedure that is robust to model misspecification, heavy-tailed noise contamination, and conditional heteroscedasticity. The proposed methodology enjoys both statistical optimality and computational efficiency, and can handle many popular high-dimensional models, such as sparse, reduced-rank, banded, and network-structured VAR models. With proper regularization and data truncation, the estimation convergence rates are shown to be almost optimal in the minimax sense under a bounded (2+2ϵ)th moment condition. When ϵ≥1, the rates of convergence match those obtained under the sub-Gaussian assumption. Consistency of the proposed estimators is also established for some ϵ∈(0,1), with minimax optimal convergence rates associated with ϵ. The efficacy of the proposed estimation methods is demonstrated by simulation and a U.S. macroeconomic example.