High-dimensional sparse vine copula regression with application to genomic prediction.

Future automated driving systems will require a comprehensive scene understanding. Considering these systems in an urban environment it becomes immediately clear that reasoning about the future behavior and trajectories of pedestrians represents one major challenge. In this paper we focus on predicting the pedestrians' time-to-cross when approaching a crosswalk. Due to the complexity of the underlying model, we propose a data-driven approach that by means of regression models learns the target variable. Instead of utilizing a standard mean regression, we propose the use of Quantile Regression. We show that this special type of regression is more suited to describe the variability of real world pedestrian trajectories. We examine and compare two approaches: Linear Quantile Regression and Quantile Regression Forest, which is an extended version of Random Forests. We present evaluations with real data and a detailed analysis emphasizing strengths and weaknesses of quantile regression for the target application.

DBFS: An effective Density Based Feature Selection scheme for small sample size and high dimensional imbalanced data sets

There is growing interest in the field of digital soil mapping (DSM) to quantify the underlying uncertainty of point predictions through different probabilistic prediction models. Uncertainty in DSM is often described in the format of a prediction interval (PI). Yet, PIs or uncertainty estimates in general are only of value for end users if they have good quality, i.e. when they are reliable. Reliability refers to the consistency between predicted conditional probabilities and observed frequencies of independent validation data. Ideally, PIs are also sharp, which refers to the concentration of probabilistic information, i.e. the narrowness of conditional probability distributions. The prediction interval coverage probability (PICP) is currently used in DSM to assess the reliability of PIs but it is ignorant to a potential one-sided bias of its bounding quantiles. Therefore, we propose to complement the current validation procedure with new metrics suggested in the broader probabilistic literature. This includes metrics that do not only evaluate uncertainty estimates in PI format but also quantiles or full conditional probability distributions. The newly proposed metrics are the quantile coverage probability (QCP), the probability integral transform (PIT) and so-called proper scoring rules for relative comparisons. Examples of scoring rules are the continuous ranked probability score (CRPS), which can be decomposed into a reliability part (RELI) and the interval score (IS). Sharpness can be evaluated through the prediction interval width (PIW). We illustrated the use of the various metrics in a case-study using soil pH data from The Land Use and Coverage Area Frame Survey (LUCAS). Thereby, uncertainty estimates of five different models were compared: Kriging with external drift (KED), quantile regression forest (QRF), quantile regression post-processing of a random forest (QRPP RF), quantile regression neural network (QRNN) and a reference null-model (NM).  KED, NM and QRPP RF showed very good reliability according to QCP, PICP, PIT and RELI. QRF was slightly pessimistic in the centre and QRNN very overoptimistic at the edges of the conditional probability distributions. Despite this, QRF performed best according to mean CRPS and mean IS because it produced fewer outliers at the edges. As expected, NM had the lowest sharpness, i.e. the largest PIW values. Sharpness of the other models was overall similar but QRNN had sharper predictions at the edges and QRF was less sharp in the centre of the conditional probability distributions. Lastly, we also generated PIW maps to indicate the spatial uncertainty of the five prediction models. The spatial variability of PIW was larger for QRF, QRNN and QRPP RF in comparison to KED. Whereas with NM, PIW was completely uniform. 

Ensembles used for probabilistic weather forecasting tend to be biased and underdispersive. This paper proposes a statistical method for postprocessing ensembles based on quantile regression forests (QRF), a generalization of random forests for quantile regression. This method does not fit a parametric probability density function (PDF) like in ensemble model output statistics (EMOS) but provides an estimation of desired quantiles. This is a nonparametric approach that eliminates any assumption on the variable subject to calibration. This method can estimate quantiles using not only members of the ensemble but any predictor available including statistics on other variables. The method is applied to the Météo-France 35-member ensemble forecast (PEARP) for surface temperature and wind speed for available lead times from 3 up to 54 h and compared to EMOS. All postprocessed ensembles are much better calibrated than the PEARP raw ensemble and experiments on real data also show that QRF performs better than EMOS, and can bring a real gain for human forecasters compared to EMOS. QRF provides sharp and reliable probabilistic forecasts. At last, classical scoring rules to verify predictive forecasts are completed by the introduction of entropy as a general measure of reliability.

Recently, anomaly detection has acquired a realistic response from data mining scientists as a graph of its reputation has increased smoothly in various practical domains like product marketing, fraud detection, medical diagnosis, fault detection and so many other fields. High dimensional data subjected to outlier detection poses exceptional challenges for data mining experts and it is because of natural problems of the curse of dimensionality and resemblance of distant and adjoining points. Traditional algorithms and techniques were experimented on full feature space regarding outlier detection. Customary methodologies concentrate largely on low dimensional data and hence show ineffectiveness while discovering anomalies in a data set comprised of a high number of dimensions. It becomes a very difficult and tiresome job to dig out anomalies present in high dimensional data set when all subsets of projections need to be explored. All data points in high dimensional data behave like similar observations because of its intrinsic feature i.e., the distance between observations approaches to zero as the number of dimensions extends towards infinity. This research work proposes a novel technique that explores deviation among all data points and embeds its findings inside well established density-based techniques. This is a state of art technique as it gives a new breadth of research towards resolving inherent problems of high dimensional data where outliers reside within clusters having different densities. A high dimensional dataset from UCI Machine Learning Repository is chosen to test the proposed technique and then its results are compared with that of density-based techniques to evaluate its efficiency.

Network intrusion detection system are significant features that contribute to the enterprises and organization network success. In the past decade, Intrusion Detection system (IDS) using several methods have been proposed and implemented to safeguard the networks within an organization to be reliable, secure, and readily available. Although, some of these methods are built utilizing machine learning concepts. IDS utilizing machine learning techniques are accurate, effective and efficient in spotting attacks. However, the machine learning model performance reduces with high dimensional intrusion dataset. Therefore, it is sacrosanct to implement an efficient feature selection technique that can generate a great impact on the classification stage. Also, the feature selection phase is the most serious phase in IDS based on machine learning. This phase is expensive both in time and efforts. Furthermore, most of the proposed machine learning intrusion detection system suffered from low detection accuracy and high false positive rate when the models are experimented on high dimensional dataset. The aim of this paper is to propose a binary firefly algorithm (BFFA) based feature selection for IDS. We first performed normalization in the first stage of the model. Subsequently, the BFFA algorithm was used for feature selection stage. We adopted random forest algorithm for the classification phase. The experiment was performed on high dimensional University of New South Wales-NB 2015 (UNSW-NB15) dataset with seventy five percent of the data used for training the model and twenty percent for testing. The findings showed an accuracy of 99.72%, detection rate of 99.84%, precision of 99.27%, recall of 99.84% and F-score of 99.56%. The results were gauge with the state-of-the-art results and our results were found outstanding.

Hydrological post-processing using stacked generalization of quantile regression algorithms: Large-scale application over CONUS

The genomic prediction of phenotypes and breeding values in animals and plants has developed rapidly into its own research field. Results of genomic prediction studies are often difficult to compare because data simulation varies, real or simulated data are not fully described, and not all relevant results are reported. In addition, some new methods have been compared only in limited genetic architectures, leading to potentially misleading conclusions. In this article we review simulation procedures, discuss validation and reporting of results, and apply benchmark procedures for a variety of genomic prediction methods in simulated and real example data. Plant and animal breeding programs are being transformed by the use of genomic data, which are becoming widely available and cost-effective to predict genetic merit. A large number of genomic prediction studies have been published using both simulated and real data. The relative novelty of this area of research has made the development of scientific conventions difficult with regard to description of the real data, simulation of genomes, validation and reporting of results, and forward in time methods. In this review article we discuss the generation of simulated genotype and phenotype data, using approaches such as the coalescent and forward in time simulation. We outline ways to validate simulated data and genomic prediction results, including cross-validation. The accuracy and bias of genomic prediction are highlighted as performance indicators that should be reported. We suggest that a measure of relatedness between the reference and validation individuals be reported, as its impact on the accuracy of genomic prediction is substantial. A large number of methods were compared in example simulated and real (pine and wheat) data sets, all of which are publicly available. In our limited simulations, most methods performed similarly in traits with a large number of quantitative trait loci (QTL), whereas in traits with fewer QTL variable selection did have some advantages. In the real data sets examined here all methods had very similar accuracies. We conclude that no single method can serve as a benchmark for genomic prediction. We recommend comparing accuracy and bias of new methods to results from genomic best linear prediction and a variable selection approach (e.g., BayesB), because, together, these methods are appropriate for a range of genetic architectures. An accompanying article in this issue provides a comprehensive review of genomic prediction methods and discusses a selection of topics related to application of genomic prediction in plants and animals.

Distributed feature selection (DFS) strategy for microarray gene expression data to improve the classification performance

This paper presents an evolutionary approach for mining diverse and more informative subgroups focused on high dimensional data sets. Subgroup Discovery (SD) is an important tool for knowledge discovery that aims to identify sets of features that distinguish a target group from the others (e.g. successful from unsuccessful treatments). At the same time, to extract information from high dimensional data sets becomes more natural. One of the first and most efficient SD heuristics focused on high dimensional data is the SSDP. However, this model deals superficially with diverse/redundancy in top-k subgroups, which can result in poor information for users. This work presents SSDP+, an extension of the SSDP model to provide diversity in a way that explore the relation between subgroups order to 2enerate a more informative set of patterns.

Radial Visualization technique is a non linear dimensionality reduction method. Radial Visualization projects multivariate data in the 2-dimensional visual space inside the unit circle. Radial Visualization supports display both the samples and the attributes that provides useful information of data structures. In this article, we introduced a new variant of Radial Visualization for visualizing high dimensional data set that named Arc Radial Visualization. The new proposal that modified Radial Visualization supported more space to display high dimensional datasets. Our method provides an improvement in visualizing cluster structures of high dimensional data sets on the Radial Visualization. We present our proposal method with two quality measurements and proved the effectiveness of our approach for several real datasets.

Because of the poor performance of the Random Forest algorithm in processing the classification of high-dimensional unbalanced data, a Hybrid Samping&Feature Selection Random Forest optimization strategy (Hybrid Samping&Feature Selection Random Forest (HF_RF) is proposed in this paper. First, from the data level, the high-dimensional unbalanced data set is preprocessed by SMOTE algorithm combined with random undersampling to achieve balanced unbalanced data. At the same time, the clustering algorithm is combined with SMOTE algorithm to improve the processing ability of the algorithm for negative samples; On the algorithm level, through the Relief F algorithm, different weight values are given to the preprocessed high-dimensional data, irrelevant and redundant features are eliminated, and high-dimensional data is reduced for dimensionality; Finally, the weighted voting principle is used to further elevate the predictive performance of HF_RF. The experimental results show that compared with the traditional algorithm, the proposed algorithm has higher indicators when dealing with high-dimensional unbalanced data, which proves that the HF_RF proposed in this paper is The correctness of the algorithm and its effectiveness in improving the classification performance of high-dimensional unbalanced data.

Tracking recurrence of correlation structure in neuronal recordings

High dimensional data clustering is a difficult task due to the curse of dimensionality. Traditional clustering methods usually fail to produce meaningful results for high dimensional data. Hypergraph partition is believed to be a promising method for dealing with this challenge. In this work, a new high dimensional clustering method called Merging Dense SubGraphs (MDSG) is proposed. A graph G is first constructed from the data by defining an adjacency relationship between the data points using Shared k Nearest Neighbors (SNN). Then a hypergraph is created from the graph G by defining the hyperedges to be all the maximal cliques in the graph. After the hypergraph is produced, an improved hypergraph partitioning method is used to produce the final clustering results. The proposed MDSG method is evaluated on several real high dimensional handwritten datasets, and the experimental results show that the proposed method is superior to the traditional clustering method and other hypergraph partition methods for high dimensional handwritten data clustering.

With the rapid development of hardware and software technology, modern industry has produced a large amount of high-dimensional unlabeled data, such as pictures and videos. As clusters of these data sets may exist in some subspaces, traditional algorithms are no longer applicable. The related algorithms based on sparse subspace can find clusters in the subspace, which solves the problem of high data dimension. However, their clustering process based on only one single feature, which results in their performance being particularly sensitive to this single view. Affected by the integrated algorithm, a large number of multi-view methods began to emerge. These methods improve the clustering performance by integrating the subspace expressions of multiple views, but the problem is that the complementary information of multiple views cannot be fully considered. In addition, the problem of non-uniform distribution in clusters also exists in high-dimensional data sets. In this paper, based on the multi-view subspace clustering method, a clustering algorithm based on tensor low rank expression is proposed to solve the clustering problem of high-dimensional datasets. On the one hand, this paper solves the problem of noise and data corruption by combining the method of $\ell _{2,1} $ norms, which transform the optimization problem of solving multi-view subspace expression into a low-rank expression problem of tensor to fully consider the complementarities between views. On the other hand, the proposed scheme solves the problem of non- uniform distribution in clusters in high-dimensional data by combining with the density peak algorithm based on hierarchical strategy. Experiment results show the effectiveness of the algorithm.