Enhanced shared nearest neighbor clustering approach using fuzzy for teleconnection analysis

Fuzzy partition based soft subspace clustering and its applications in high dimensional data

In modern database applications the similarity or dissimilarity of complex objects is examined by performing distance-based queries (DBQs)on data of high dimensionality. The R-tree and its variations are commonly cited multidimensional access methods that can be used for answering such queries. Although, the related algorithms work well for low-dimensional data spaces, their performance degrades as the number of dimensions increases (dimensionality curse). In order to obtain accept- able response time in high-dimensional data spaces, algorithms that ob- tain approximate solutions can be used. Three approximation techniques (α-allowance, N-consider and M-consider) and the respective recursive branch-and-bound algorithms for DBQs are presented and studied in this paper. We investigate the performance of these algorithms for the most representative DBQs (the K-nearest neighbors query and the K-closest pairs query) in high-dimensional data spaces, where the point data sets are indexed by tree-like structures belonging to the R-tree family:R*- trees and X-trees. The searching strategy is tuned according to several parameters, in order to examine the trade-off between cost (I/O activ- ity and response time) and accuracy of the result. The outcome of the experimental evaluation is the derivation of the outperforming DBQ ap- proximate algorithm for large high-dimensional point data sets.

The sparsity and the problem of the curse of dimensionality of high-dimensional data, which make the most of the traditional clustering algorithm, lose action in high-dimensional space. Therefore, clustering of data in high-dimensional space is becoming the hot research areas. By utilizing the subtractive clustering as initialized method, and combine with the revised clustering validation indices, this paper offers a subspace clustering algorithm for automatically determining the optimal number of clusters on high dimensional data. The experiment results show that the proposed clustering algorithm can get better cluster validation performance than that of conventional indices.

In order to solve the problem that traditional grid-based clustering techniques lack of the capability of dealing with data of high dimensionality, we propose an intersecting grid partition method and a density estimation method. The partition method can greatly reduce the number of grid cells generated in high dimensional data space and make the neighbor-searching easily. On basis of the two methods, we propose grid-based clustering algorithm (GCOD), which merges two intersecting grids according to density estimation. The algorithm requires only one parameter and the time complexity is linear to the size of the input data set or data dimension. The experimental results show that GCOD can discover arbitrary shapes of clusters and scale well.

Feature selection has been widely used as a pre-processing step that helps to optimise the performance of data-driven intrusion/anomaly detection systems in achieving their tasks. For example, when grouping the data into normal and outlier groups, the existence of redundant and non-representative features would reduce the accuracy of classifying the data points and would also increase the processing time. Therefore, feature selection is applied as a pre-processing step for anomaly detection systems in order to optimize their classification accuracy and running time. Most of the existing feature selection methods have limitations when dealing with high-dimensional data, as they search different subsets of features to find accurate representations of all features. Obviously, searching for different combinations of features is computationally very expensive, which makes existing work not efficient for high-dimensional data. The work carried out here, which relates to the design of a similaritybased unsupervised feature selection method for an efficient and accurate anomaly detection (UFSAD), tackles mainly the selection of reduced set of representative features from high-dimensional data without the data class labels. The selected features should improve the accuracy and performance of anomaly detection systems due to the elimination of redundant and non-representative features. The proposed UFSAD method extends the k-mean clustering algorithm to partition the features into k clusters based on a similarity measure (e.g. PCC - Pearson Correlation Coefficient, LSRE - Least Square Regression Error or MICI - Maximal Information Compression Index) in order to accurately partition the features. Then the proposed centroid-based feature selection method is used, where the feature with the closest similarity to its cluster centroid is selected as the representative feature while others are discarded. Extensive experimental work has shown that UFSAD can generate a reduced representative and non-redundant feature set that achieves good classification accuracy in comparison with well-known unsupervised features selection methods.

A scientific visualization tool, based on a virtual reality system, has been created to explore remotely sensed data. The main aim is to find possible relationships between results of different spectral unmixing techniques in order to improve image classifications. In particular, data from fuzzy classifiers and linear spectral unmixing techniques, has been considered. The tool offers the possibility to use different representations for high dimensional data due to the fact that none of the existing representations is always effective to visualise multi-dimensional data. Furthermore, the possibility to visually link different representations of the same data, by using a technique called data brushing, will help the analyst in to get more insight into the structures present in the high dimensional data space and, to find relationships between the data spaces. The exploratory tool has been based on a virtual reality system in order to create new human-computer interaction paradigms. Furthermore, it has been used to assess the usability of immersive computer interfaces, called virtual environments, in data exploration. The main advantages in using this kind of interface has been an increased interface space and a more natural human-computer interaction which means more information content requiring less time to become familiar with the tool.

Various machine learning, cloud computing and big data models are most utilized control parameters in software applications. Therefore these systems are most demand at current trending decades. But these models are needs very low data access time, speed for process. Day to day life data storage servers and devices are costly and hardware complex, so dimensionality is increases with rapid manner. Any type of optimization techniques access takes more time consumption for high dimensional data. So, more applications related problems are occurs only at high dimensional data space does not at low space dimensional data. In this research proposes a dimensional reduction technique with Logistic regression (LR) model. This LR model is most helpful for dimensional reduction and clustering problems. LRML method has reduced the dimensional data size and achieved thee efficiency by 95.3% and ratio of reduction by 35.76%.

Multi-level adaptive neuro-fuzzy inference system-based reconstruction of 1D ISOMAP representations

In spatial database applications the similarity or dissimilarity of complex objects is examined by performing distance-based queries (DBQs) on data of high dimensionality (a generalization of spatial data). The R-tree and its variations are commonly cited as multidimensional access methods that can be used for answering such queries. Although the related algorithms work well for low-dimensional data spaces, their performance degrades as the number of dimensions increases (dimensionality curse). To obtain acceptable response time in high-dimensional data spaces, algorithms that obtain approximate solutions can be used. In this chapter, we review the most important approximation techniques for reporting sufficiently good results quickly. We focus on the design choices of efficient approximate DBQ algorithms that minimize the response time and the number of I/O operations over tree-like structures. The chapter concludes with possible future research trends in the approximate computation of DBQs.

Dimensionality reduction combined with particle swarm optimization algorithm for solving high-dimensional nuclear data target accuracy evaluation problem

Nowadays, the influence of machine learning algorithms on our lives is booming gradually, and gradient descent (GD) method is a typical big data analysis method that can be applied to various machine learning algorithms. Traditional GD method has its advantages in dealing with the optimal parameter finding issue for partial loss functions, but the optimal parameter finding process may be trapped into local minimum sometimes if the loss function with certain convexity; moreover, traditional GD cannot meet the requirement of high accuracy. Fractional order GD, which is the core of fixed memory step GD, has better performance by comparing with traditional GD, but still have some problems in the aspect of convergence. On the other hand, t-distributed machine learning algorithm for dimensionality reduction, can reduce the dimensionality of data in high-dimensional space to two-dimensional or three-dimensional space, and can convert the original data into more comprehensible images. In order to solve the problem mentioned above reasonably, in this paper, a fixed-memory step GD method combined with t-distributed stochastic neighbor embedding (t-SNE) is employed to replace the traditional GD method, so as to keep the consistency of low-dimensional data being mapped from high-dimensional data, and to achieve efficient data visualization. The effectiveness of proposed t-SNE algorithm based on fixed memory step gradient descent method has been verified finally via numerical experiment.

Area Under the ROC Curve (AUC) is an objective indicator of evaluating classification performance for imbalanced data. In order to deal with large-scale imbalanced streaming data, especially high-dimensional sparse data, this paper proposes a Sparse Stochastic Online AUC Optimization (SSOAO) method. Specifically, we first turn the standard online AUC optimization problem into a stochastic saddle point problem, then optimizing AUC by solving stochastic saddle point problem through AdaGrad optimizer. A sparse regularization term is also added for learning sparse data with high dimension. Comprehensive evaluation has been carried out on the recent benchmark. The experimental results show that the proposed SSOAO has the comparable performance on low-dimensional data, and outperforms other popular AUC optimization methods on high-dimensional sparse imbalanced streaming data. Both time and space complexity for model updating are reduced from \(O(d^2)\) to O(d), which equal to the data dimension.

Modified fuzzy clustering with segregated cluster centroids

Indexing shapes in image databases using the centroid–radii model

This paper presents a new approach to multiple objects tracking that combines particle filters and k-means. The ap-proach has been tested under an important real world sit-uation, related to pharmacological development, which has also proved to serve as an interesting ground-truth dataset provider for the evaluation of tracking algorithms. The obtained results are then compared to other models. The promising results of these experiments are presented. 1