Improved partitioning technique for density cube-based spatio-temporal clustering method

There is a strong need for synthetic yet realistic distribution system test data sets that are as diverse, large, and complex to solve as real systems. Such data sets can facilitate the development of advanced algorithms and the assessment of emerging distributed energy resources while avoiding the need to acquire proprietary critical infrastructure or private data. Such synthetic data sets, however, are useful only if they are realistic enough to look and behave similarly to actual systems. This paper presents a comprehensive framework for validating synthetic distribution data sets using a three-pronged statistical, operational, and expert validation approach. It also presents a set of statistical and operational metric targets for achieving realistic data sets based on detailed characterization of more than 10,000 real U.S. utility feeders. The paper demonstrates the use of the proposed validation approach to validate three large-scale synthetic data sets developed by the authors representing Santa Fe, New Mexico; Greensboro, North Carolina; and the San Francisco Bay Area, California.

Clustering is a useful exploratory technique for the analysis of gene expression data. Many different heuristic clustering algorithms have been proposed in this context. Clustering algorithms based on probability models offer a principled alternative to heuristic algorithms. In particular, model-based clustering assumes that the data is generated by a finite mixture of underlying probability distributions such as multivariate normal distributions. The issues of selecting a 'good' clustering method and determining the 'correct' number of clusters are reduced to model selection problems in the probability framework. Gaussian mixture models have been shown to be a powerful tool for clustering in many applications. We benchmarked the performance of model-based clustering on several synthetic and real gene expression data sets for which external evaluation criteria were available. The model-based approach has superior performance on our synthetic data sets, consistently selecting the correct model and the number of clusters. On real expression data, the model-based approach produced clusters of quality comparable to a leading heuristic clustering algorithm, but with the key advantage of suggesting the number of clusters and an appropriate model. We also explored the validity of the Gaussian mixture assumption on different transformations of real data. We also assessed the degree to which these real gene expression data sets fit multivariate Gaussian distributions both before and after subjecting them to commonly used data transformations. Suitably chosen transformations seem to result in reasonable fits. MCLUST is available at http://www.stat.washington.edu/fraley/mclust. The software for the diagonal model is under development. kayee@cs.washington.edu. http://www.cs.washington.edu/homes/kayee/model.

The matrix factorization (MF) technique has been widely adopted for solving the rating prediction problem in recommender systems. The MF technique utilizes the latent factor model to obtain static user preferences (user latent vectors) and item characteristics (item latent vectors) based on historical rating data. However, in the real world, user preferences are not static but full of dynamics. Though there are several previous works that addressed this time-varying issue of user preferences, it seems (to the best of our knowledge) that none of them are specifically designed for tracking concept drift in individual user preferences . Motivated by this, we develop a temporal MF approach for tracking concept drift in each individual user latent vector. There are two key innovative steps in our approach: 1) we develop a modified stochastic gradient descent method to learn an individual user latent vector at each time step and 2) by Lasso regression, we learn a linear model for the transition of the individual user latent vectors. We test our method on a synthetic data set and several real data sets. In comparison with the original MF, our experimental results show that our temporal method is able to achieve lower root mean square errors (RMSEs) for both the synthetic and real data sets. One interesting finding is that the performance gain in RMSE is mostly from those users who indeed have concept drift in their user latent vectors at the time of prediction. In particular, for the synthetic data set and the Ciao data set, there are quite a few users with that property and the performance gains for these two data sets are roughly 20% and 5%, respectively.

In this Article, the targeted adjustment of the relative density of laser additive manufactured components made of AlSi10Mg is considered. The interest in demand-oriented process parameters is steadily increasing. Thus, shorter process times and lower unit costs can be achieved with decreasing component densities. Especially when hot isostatic pressing is considered as a post-processing step. In order to be able to generate process parameters automatically, a model hypothesis is learned via artificial neural networks (ANN) for a density range from 70% to almost 100%, based on a synthetic dataset with equally distributed process parameters and a statistical test series with 256 full factorial combined instances. This allows the achievable relative density to be predicted from given process parameters. Based on the best model, a database approach and supervised training of concatenated ANNs are developed to solve the inverse parameter prediction problem for a target density. In this way, it is possible to generate a parameter prediction model for the high-dimensional result space through constraints that are shown with synthetic test data sets. The presented concatenated ANN model is able to reproduce the origin distribution. The relative density of synthetic data can be predicted with an R2-value of 0.98. The mean build rate can be increased by 12% with the formulation of a hint during the backward model training. The application of the experimental data shows increased fuzziness related to the big data gaps and a small number of instances. For practical use, this algorithm could be trained on increased data sets and can be expanded by properties such as surface quality, residual stress, or mechanical strength. With knowledge of the necessary (mechanical) properties of the components, the model can be used to generate appropriate process parameters. This way, the processing time and the amount of scrap parts can be reduced.

In view of an optimal exploitation of the Swarm data set, we have prepared and tested software dedicated to the determination of accurate core magnetic field models and of the Euler angles between the magnetic sensors and the satellite reference frame. The dedicated core field model estimation is derived directly from the GFZ Reference Internal Magnetic Model (GRIMM) inversion and modeling family. The data selection techniques and the model parameterizations are similar to what were used for the derivation of the second (Lesur et al., 2010) and third versions of GRIMM, although the usage of observatory data is not planned in the framework of the application to Swarm. The regularization technique applied during the inversion process smoothes the magnetic field model in time. The algorithm to estimate the Euler angles is also derived from the CHAMP studies. The inversion scheme includes Euler angle determination with a quaternion representation for describing the rotations. It has been built to handle possible weak time variations of these angles. The modeling approach and software have been initially validated on a simple, noise-free, synthetic data set and on CHAMP vector magnetic field measurements. We present results of test runs applied to the synthetic Swarm test data set.

Dimensionality reduction is essential in multidimensional data mining since the dimensionality of real time data could easily reach higher dimensions. Most recent efforts on dimensionality reduction, however, are not adequate to multidimensional data due to lack of scalability. In this paper, we improve the performance of the fractional cuckoo search algorithm by utilizing the FCM function as objective function and we utilize the FCM operator which helps to improve the fitness value of worse solution. The fractional cuckoo search (FCS) algorithm is modified to reduce the dimension and select the best dimension. Once the high dimensional data is reduced in to low dimensional data, then the data is supplied to the clustering algorithm to make the partition easily. Finally, the experimentation is made with synthetic and real datasets and has been we have proved that efficiency of the FCM-FCS algorithm is 1.21% better than FPSO algorithm on iris dataset. The proposed algorithm is 0.66% better than FPSO 4.91% better than FCS on wine dataset and 3.4% better than FPSO 0.1% better than FCS on synthetic dataset.

We study the ability of statistical tests to identify nonrandom features of earthquake catalogs, with a focus on the global earthquake record since 1900. We construct four types of synthetic data sets containing varying strengths of clustering, with each data set containing on average 10,000 events over 100 years with magnitudes above M = 6. We apply a suite of statistical tests to each synthetic realization in order to evaluate the ability of each test to identify the sequences of events as nonrandom. Our results show that detection ability is dependent on the quantity of data, the nature of the type of clustering, and the specific signal used in the statistical test. Data sets that exhibit a stronger variation in the seismicity rate are generally easier to identify as nonrandom for a given background rate. We also show that we can address this problem in a Bayesian framework, with the clustered data sets as prior distributions. Using this new Bayesian approach, we can place quantitative bounds on the range of possible clustering strengths that are consistent with the global earthquake data. At M = 7, we can estimate 99th percentile confidence bounds on the number of triggered events, with an upper bound of 20% of the catalog for global aftershock sequences, with a stronger upper bound on the fraction of triggered events of 10% for long‐term event clusters. At M = 8, the bounds are less strict due to the reduced number of events. However, our analysis shows that other types of clustering could be present in the data that we are unable to detect. Our results aid in the interpretation of the results of statistical tests on earthquake catalogs, both worldwide and regionally.

General fuzzy C-means clustering algorithm using Minkowski metric

Problem statement: The processing of the XML queries needs an efficient indexing method. The index generation includes the labeling of the nodes. But for the dynamic data, when the changes are made, the re-computation of the labeling is needed. Approach: Based on the structural indexing used the performance of the query processing system will be affected. The dynamic XML document allows inserting, deleting and updating operations. Results: Suppose for the frequently changed documents, the indexes will also be changed. i.e., it requires the re-computing the labels frequently. This leads to an inconvenient system. To avoid this problem, in the scheme New Labeling Scheme for XML (NLSX), only the small, capital letters and digits are used to generate persistent labels for the nodes in the document. In the proposed system New Labeling Scheme for XML using Unicode Characters, characters from Unicode Characters (NLSXU) are used. Thus it can provide more combinations of the characters for persistent labeling the nodes in the document so that it will very much reduce the space needed to store the labels. Using the proposed scheme NLSXU, the index size of the real world data sets will be greatly reduced by 81% of the existing scheme NLSX. The results shows that the proposed scheme NLSXU will reduce the size of the indexes of the synthetic data sets up to 26, 34, 71 and 95% than the NLSX, LSDX, GRP and SP schemes respectively. Also when compared to LSDX scheme, the NLSXU will reduce the time taken for generating the labels by 96 and 80% for the real world datasets and the synthetic data sets respectively. Conclusion: Finally when compared to NLSX scheme, the NLSXU will reduce the time taken for generating the labels by 66 and 15% for the real world datasets and the synthetic data sets respectively. Thus it will improve the performance of the query system.

Application of a novel hybrid method for spatiotemporal data imputation: A case study of the Minqin County groundwater level

Abstract. Considering the critical role of trajectory data in Big Data era for dynamic geographical processes, human behaviour analysis and meteorological prediction, trajectory clustering has attracted growing attention. Many literatures have discussed the spatiot emporal clustering method of simple trajectories (i.e., has no branches, e.g. vehicle trajectories), yet there are few researches for clustering complex trajectories (i.e., has at least one split and/or merger and/or split -merger branch, e.g. ocean eddy trajectories, rainstorm trajectories). For addressing this issue, we propose a Process-Oriented Spatiotemporal Clustering Method (POSCM) for clustering complex trajectory data. The POSCM includes three parts: the first uses the semantic of process-sequence-state to represent the complex trajectories; the second proposes a Hierarchical Similarity Measurement Method (HSMM) to get the similarity between any two complex trajectories; in the last step, the complex trajectories clustering pattern is extracted through density-based clustering algorithm. Experiments on simulated trajectories are used to evaluate the POSCM and demonstrate the advantage by comparing against that of the VF2 algorithm. The POSCM is applied to the sea surface temperature abnormal variations trajectories from January 1950 to December 2017 in the Pacific Ocean. As shown in this case study, some new mined spatiotemporal patterns can provide new references for understanding the behaviours of marine abnormal variations under the background of the global change.

Fuzzy C-Means clustering based on dual expression between cluster prototypes and reconstructed data

Clustering analysis, a key step for many data mining problems, can be applied to various fields. However, no matter what kind of clustering method, noise points have always been an important factor affecting the clustering effect. In addition, in spectral clustering, the construction of affinity matrix affects the formation of new samples, which in turn affects the final clustering results. Therefore, this study proposes a noise cutting and natural neighbors spectral clustering method based on coupling P system (NCNNSC-CP) to solve the above problems. The whole algorithm process is carried out in the coupled P system. We propose a natural neighbors searching method without parameters, which can quickly determine the natural neighbors and natural characteristic value of data points. Then, based on it, the critical density and reverse density are obtained, and noise identification and cutting are performed. The affinity matrix constructed using core natural neighbors greatly improve the similarity between data points. Experimental results on nine synthetic data sets and six UCI datasets demonstrate that the proposed algorithm is better than other comparison algorithms.

We propose a novel density estimation method using both the k-nearest neighbor (KNN) graph and the potential field of the data points to capture the local and global data distribution information respectively. The clustering is performed based on the computed density values. A forest of trees is built using each data point as the tree node. And the clusters are formed according to the trees in the forest. The new clustering method is evaluated by comparing with three popular clustering methods, K-means++, Mean Shift and DBSCAN. Experiments on two synthetic data sets and one real data set show that our approach can effectively improve the clustering results.

This paper proposes a method for clustering asynchronous events generated upon scene activities by a dynamic 3D vision system. The inherent detection of moving objects offered by the dynamic stereo vision system comprising a pair of dynamic vision sensors allows event-based stereo vision in real-time and a 3D representation of moving objects. The clustering method exploits the sparse spatio-temporal representation of sensor's events for real-time detection and separation between moving objects. The method makes use of density and distance metrics for clustering asynchronous events generated by scene dynamics (changes in the scene). It has been evaluated on clustering the events of moving persons across the sensor field of view. Tests on real scenarios with more than 100 persons show that the resulting asynchronous events can be successfully clustered and the persons can be detected.