Robust Convex Clustering Analysis

Tremendous growth of Location-based technologies resulted in the generation of a huge volume of spatial data, which needs to be analyzed to get potentially important patterns. The spatial patterns extracted can be used to design a better infrastructure ensuring reliable service coverage. Trajectory data is one variant of spatial data that are generated by moving objects travelling across. It is represented as a sequence of spatial coordinates (latitude, longitude) of a location. Trajectory clustering tries to group similar spatial data points to extract the most common movement behaviors. Trajectory data poses major challenges including uncertainty, sampling rate, representation, relationships, spatial autocorrelation, serialization, redundancy, and triviality, which makes it hard to apply traditional clustering algorithms over trajectory data. In this paper, K-Means and DBSCAN (Density-based spatial clustering of applications with noise) clustering algorithms are analyzed using different similarity measures like Euclidean, Hausdorff and Haversine distances with the help of index measures say Adjusted Rand Index (ARI) and Fowlkes-Mallows scores (FMS). Experiment is carried out over two different trajectory datasets and it is proved that usage of Haversine distance for clustering is efficient than Euclidean and Hausdorff distances in terms of spatial trajectory data.

As the foundation of global communication networks, optical fibers are vulnerable to various disruptive events, including mechanical damage, such as cuts, and malicious physical layer breaches, such as eavesdropping via fiber bending. Traditional monitoring methods often fail to identify subtle or novel anomalies, stimulating the proliferation of Machine Learning (ML) techniques for detection of threats before they cause significant harm. In this paper, we evaluate the performance of Semi-Supervised Learning (SSL) and Unsupervised Learning (USL) approaches for detecting various abnormal events, such as fiber bending and vibrations, by analyzing polarization signatures with minimal reliance on labeled data. We experimentally collect thirteen polarization signatures on three different types of fiber cable and process them using One-Class Support Vector Machine (OCSVM) as an SSL, and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) as a USL algorithm for anomaly detection. We introduce tailored evaluation metrics designed to guide hyper-parameter tuning and capture generalization over different anomaly types, detection consistency, and robustness to false positives, enabling practical deployment of OCSVM and DBSCAN in optical fiber security. Our findings demonstrate DBSCAN as a strong contender to detect previously unseen threats in scenarios where labeled data are not available, despite some variability in performance between different scenarios, with F1 score values between 0.615 and 0.995. In contrast, OCSVM, trained on normal operating conditions, maintains high F1 scores of 0.98 to 0.998, demonstrating accurate detection of complex anomalies in optical networks.

We propose an original learning approach for image classification problems. Recognizing semantic events in video requires to preliminary learn the different classes of events. This first stage is crucial since it conditions the further classification results. In video content analysis, the task is especially difficult due to the high infra-class variability and to noisy measurements. We then represent each class by the centers of several sub-classes (or clusters) thanks to a robust partitional clustering algorithm which can be applied in parallel to a (non-predefined) number of classes. Our clustering technique overcome three main limitations of standard K-means methods: sensitivity to initialization, choice of the number of clusters and influence of outliers. Moreover, it can process the training data in an incremental way. Experimental results on sports videos are reported.

In data-intensive environment, data mining and data analysis is a big challenge. It analyses basic algorithms, such as k-means algorithm and Density-Based Spatial Clustering of Applications with Noise (DBSCAN). And both algorithms are implemented on MapReduce of Hadoop. Further, an improved Data-intensive Density Base Distributed Clustering (IDBDC) algorithm is proposed which are based on k-means algorithm and DBSCAN algorithm. And IDBDC algorithm is implemented on MapReduce of Hadoop as well. The experimental results show that the improved IDBDC algorithm has better performance.

The use of machine learning as a basis for making informed public policy decisions is advancing; however, its application in the area of social protection in developing societies remains under-researched. The chapter offers the novel methodology of combining an Unsupervised Machine Learning K-means clustering algorithm with descriptive statistics to explore big data collected using the household survey in order to form clusters of households. The shaped clusters represent various welfare regimes that will respond to the second research question of the study. This chapter compares density-based spatial clustering of applications with noise (DBSCAN) and K-means clustering to investigate the most efficient approach. The results revealed that K-means clustering is an efficient method to explore the survey dataset. Hence, the clusters formed using the K-means clustering approach showed common patterns relating to the insecurities associated with loss of income or property, unemployment, disasters, or disease etc. that households faced, and to the access to formal and informal social protection in each cluster. The chapter concludes that by using the K-means clustering unsupervised machine learning method, big data can be explored efficiently to target social protection interventions better, with the aim of achieving optimal results to increase the welfare of the poor.

Background: This study focuses on the latest knowledge regarding Micro, Small and Medium Enterprises (MSMEs) as a current central issue. These enterprises have shown their significance in providing employment opportunities and contributing to the country’s economy. However, MSMEs face various challenges that must be addressed to optimize their outcomes. Understanding the characteristics of this group was crucial in formulating effective strategies. Objective: This study proposed to cluster or combine micro, small, and medium enterprises (MSMEs) data in a particular area based on asset value and turnover. As a result, this study aimed to gain insights into the MSME landscape in the area and provided valuable information for decision-makers and stakeholders. Methods: This study utilized two methods, namely the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) method and the K-Means method. These methods were chosen for their distinct capabilities. DBSCAN was selected for its ability to handle noisy data and identify clusters with diverse forms, while K-Means was chosen for its popularity and ability to group data based on proximity. The study used a dataset containing MSME information, including asset values and turnover, collected from various sources. Results: The outcomes encompassed identifying clusters of MSMEs based on their closeness in the feature space within a specific region. Optimizing the clustering outcomes involved modifying algorithm parameters like epsilon and minimum points for DBSCAN and the number of clusters for K-Means. Furthermore, this study attained a deeper understanding of the arrangement and characteristics of MSME clusters in the region through a comparative analysis of the two methodologies. Conclusion: This study offered perspectives on clustering MSMEs based on asset value and turnover in a specific region. Employing DBSCAN and K-Means methodologies allowed researchers to depict the MSME landscape and grasp the business attributes of these enterprises. These results could aid in decision-making and strategic planning concerning the advancement of the MSME sector in the mentioned area. Future study may investigate supplementary factors and variables to deepen comprehension of MSME clusters and promote regional growth and sustainability. Keywords: Asset Value, Clustering, DBSCAN, K-Means, Turnover

This study proposes a novel robust unit commitment (UC) framework with data-driven disjunctive uncertainty sets for volatile wind power generation, which integrates a two-stage adaptive robust UC model with machine learning techniques to flexibly capture the uncertainty space of the wind power forecast errors with disjunctive structures. K-means clustering and density-based spatial clustering of applications with noise (DBSCAN) are applied for clustering, and data-driven disjunctive uncertainty sets are constructed as the union of multiple basic uncertainty sets following the clustering results. The problem is formulated into a two-stage adaptive robust UC model with data-driven disjunctive uncertainty sets and with a multi-level optimization structure. A tailored decomposition-based optimization algorithm is developed to facilitate the solution process. A numerical experiment and a UC case study on the IEEE 39-bus system are presented to demonstrate the effectiveness of the proposed approach. In both applications, using the proposed disjunctive uncertainty sets can effectively reduce the price of robustness compared to the conventional "one-set-fits-all" approach. Furthermore, the disjunctive uncertainty sets using DBSCAN tend to provide less conservative solutions than those using K-means, implying that DBSCAN can handle the outliers and noise of the uncertainty data more efficiently.

Clustering algorithm is one of the most important algorithms in unsupervised learning. For density‐based spatial clustering of applications with noise (DBSCAN) density clustering algorithm, the selection of neighborhood radius and minimum number is the key to get the best clustering results. Aiming at the problems of traditional DBSCAN algorithm, such as the neighborhood radius and the minimum number of points, this article puts forward two classifications based on K‐means algorithm, and gets two clustering centers. Where calculated between two data points and the cluster center‐to ‐center distance, clustering, distance, statistics in a distance of data points within the scope of the search, the number of data points corresponding to the maximum distance value, and thus the parameters for the DBSCAN algorithm to estimate and selection of initial radius of neighborhood with the minimum number of clustering start critical value. When the parameters are iterated and optimized continuously, the data are divided into clusters, and the most suitable neighborhood radius and the minimum point number are obtained. The experimental data analysis show that the improved algorithm reduces the human factors in the traditional algorithm and improves the efficiency, so as to get the accurate clustering results.

In order to better mine information from the data of automatic identification system (AIS) and scientifically perceive the water traffic situation, several common clustering algorithms, such as K-means algorithm and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm, are studied and analyzed, and the advantages and disadvantages of each algorithm and applicable scenarios are compared. Taking the actual ship's AIS trajectory data as an example, each algorithm is used to verify its characteristics, which provides convenience for further information mining of trajectory data.

The density based notion for clustering approach is used widely due to its easy implementation and ability to detect arbitrary shaped clusters in the presence of noisy data points without requiring prior knowledge of the number of clusters to be identified. Density-based spatial clustering of applications with noise (DBSCAN) is the first algorithm proposed in the literature that uses density based notion for cluster detection. Since most of the real data set, today contains feature space of adjacent nested clusters, clearly DBSCAN is not suitable to detect variable adjacent density clusters due to the use of global density parameter neighborhood radius Nrad and minimum number of points in neighborhood Npts. So the efficiency of DBSCAN depends on these initial parameter settings, for DBSCAN to work properly, the neighborhood radius must be less than the distance between two clusters otherwise algorithm merges two clusters and detects them as a single cluster. Through this paper: 1) We have proposed improved version of DBSCAN algorithm to detect clusters of varying density adjacent clusters by using the concept of neighborhood difference and using the notion of density based approach without introducing much additional computational complexity to original DBSCAN algorithm. 2) We validated our experimental results using one of our authors recently proposed space density indexing (SDI) internal cluster measure to demonstrate the quality of proposed clustering method. Also our experimental results suggested that proposed method is effective in detecting variable density adjacent nested clusters.

The K-means algorithm is one of the widely used clustering algorithms in the image classification systems. However, the K-Means algorithm is easily trapped into the local optimal solutions. Several optimization techniques have been proposed to solve this problem such as genetic algorithms, simulated annealing and swarm intelligence. In this paper, we develop hybrid techniques using different particle swarm optimization (PSO) heuristics to optimize the k-means algorithm and examine the reliability of parametric values for different variants of PSO and k-means algorithms. These PSO heuristics include linear inertia reduction, constriction factor, and dynamic inertia and maximum velocity reduction. The performance of these hybridization of PSO and the k-means algorithms was tested on the image segmentation. These PSO heuristics can make the K-means algorithm more stable for finding better solutions and less dependent on the initial cluster centers based on the preliminary experimental results.

In fingerprint-based localization systems, clustering fingerprint databases is a proposed technique for improving localization accuracy while reducing localization time. Among various clustering algorithms, density-based spatial clustering of applications with noise (DBSCAN) stands out for its robustness to outliers and ability to accommodate fingerprint databases of various shapes. However, the clustering performance of the DBSCAN algorithm is heavily influenced by the type of similarity measure metric used, with most researchers using distance-based metrics. This paper aims to enhance DBSCAN clustering by using a pattern-based metric known as the context similarity coefficient (CSC) instead of distance-based metrics. The CSC metric examines received signal strength (RSS) measurement patterns that form fingerprint vectors and assesses both linear and non-linear relationships between these vectors to determine similarity. Four publicly available fingerprint databases were used to evaluate the clustering performance with silhouette scores as a performance metric. The performance of the DBSCAN algorithm with the CSC metric is determined and compared to Euclidean and Manhattan distances as similarity measure metrics. Simulation results indicate that achieving good clustering performance with the DBSCAN algorithm requires generating three or fewer clusters. The proposed CSC metric demonstrated the best clustering performance in two of four fingerprint databases and the second-best in another. However, computational complexity comparisons reveal that the CSC metric is highly computationally intensive and is suggested to be used on small to medium-sized fingerprint databases generated using an odd number of wireless APs deployed in a non-uniform or non-grid-like distribution.

Facies identification from basic well logs is essential for optimizing well operation or constructing reservoir model. The sequential nature of well log data makes the Hidden Markov Model (HMM) a suitable unsupervised machine learning approach for clustering. This study investigates the effectiveness of HMM combined with window filtering for unsupervised facies detection and compares its performance with the widely used K-Means Clustering (KMC) method. Synthetic gamma ray (GR), density (DEN), and neutron (NEU) logs, along with geologist-guided facies in a channel depositional environment, were used as input data. HMM identifies facies by modeling hidden geological states based on sequential well log data. Transition probabilities capture facies relationship across depths. A mode-based moving window filter smoothed the initial results by reducing fluctuations, while preserving facies boundaries. Validation was performed by comparing the results with reference facies using accuracy, adjusted rand index (ARI), normalized mutual information (NMI), and visual assessment. Both HMM and KMC clustering revealed thin non-geological facies, primarily at the facies boundaries. In the middle of facies, HMM mitigated thin facies more effectively than KMC. These thin facies arose from fluctuations in the well logs. Visually, HMM provided more geologically realistic results, despite KMC sometimes showed slightly better performance metrics. To address thin artificial facies, a moving window filter was applied. This slightly improved clustering performance metrics but significantly enhanced geological consistency by removing most artificial facies at boundaries or within facies, particularly in the smoothed HMM. Smoothed HMM achieved slightly better or comparable accuracy and NMI to smoothed KMC. However, minor thin facies persisted in both methods, more notably in KMC, due to significant well log fluctuations. The filter effectively smoothed the facies by reducing variations in thin facies across small depth intervals, ultimately enhancing the facies continuity. This improved facies distribution coherency with geological expectation, particularly for the smoothed HMM. HMM with post-processing plays a crucial role in identifying geologically consistent facies while maintaining or slightly improving clustering performance metrics. Unsupervised machine learning that incorporates sequential information is vital for facies clustering from well log data. The proposed workflow which integrates HMM with window filtering, enhances clustering outcomes by producing more geologically consistent result and more interpretable facies. This approach aids geoscientists in reservoir characterization and modeling, enabling more informed decision-making.

The techniques behind Data mining play a significant task in health care field. The techniques are mainly involved in prediction-based computing. A wide variety of algorithms are available in data mining. Data mining has many modern data analysis techniques like classification and prediction which may be utilized for this purpose. Classification is a data mining approach that allocates items in a group to target categories or classes. It also may be used to label a target item into any one of the classes identified. Among many available classification techniques, clustering is one of the unsupervised machine learning approaches that could be used for creating clusters as features to enhance classification models. There are various clustering algorithms available like K-mean clustering, AGNES clustering, etc. For disease prediction system K-means algorithm is used among all techniques available in prediction system. K-Mean algorithm creates clusters and groups of data properly. Many researchers have worked in the data clustering field and various clustering techniques have been used by them. In this work at first, Knowledge-based data are created by using factor analysis. After that K-mean algorithm is used on the result of knowledge-based data and different number of cluster points are assumed, then we get the Euclidean distance function with the simple matching dissimilarity measure. Once the cluster was formed, another method was used to forecast the resultant values of the data in the cluster. The proposed approach was tested on heart disease dataset and found efficient in this domain.

The main influencing factors of the clustering effect of the k-means algorithm are the selection of the initial clustering center and the distance measurement between the sample points. The traditional k-mean algorithm uses Euclidean distance to measure the distance between sample points, thus it suffers from low differentiation of attributes between sample points and is prone to local optimal solutions. For this feature, this paper proposes an improved k-means algorithm based on evidence distance. Firstly, the attribute values of sample points are modelled as the basic probability assignment (BPA) of sample points. Then, the traditional Euclidean distance is replaced by the evidence distance for measuring the distance between sample points, and finally k-means clustering is carried out using UCI data. Experimental comparisons are made with the traditional k-means algorithm, the k-means algorithm based on the aggregation distance parameter, and the Gaussian mixture model. The experimental results show that the improved k-means algorithm based on evidence distance proposed in this paper has a better clustering effect and the convergence of the algorithm is also better.