Noise identification with the k-means algorithm

Class noise detection based on software metrics and ROC curves

The presence of a substantial number of noisy instances in a given dataset may adversely affect the hypothesis learnt from that data. Removing noisy instances prior to the construction of a classifier has been shown to improve the classification ability of a learner on new data. This paper introduces a novel technique for identifying observations with class noise in a dataset using frequent itemsets. For the given dataset, each instance is assigned a NoiseFactor, indicating a relative likelihood that it contains class noise. A frequent itemset is a set of instances with common attribute values which contains at least as many instances as a user-defined minimum support threshold. Consequently, the set of frequent itemsets contains information related to the structure and dependence between the attributes. Each frequent itemset is assigned a class, based on the proportion of instances within the itemset from each class. Instances that are contained in itemsets that have a large proportion of instances from the other class are identified as noisy. The technique proposed in this paper is analyzed in numerous case studies using real-world software measurement datasets with either inherent or injected noise. A comparison is provided with two well-known techniques for the identification of class noise: Classification Filter and Ensemble Filter. The results demonstrate that this new algorithm is very effective at identifying instances with class noise.

This paper presents an integrated approach by considering chemical reaction optimization (CRO) and functional link artificial neural networks (FLANNs) for building a classifier from the dataset with missing value, inconsistent records, and noisy instances. Here, imputation is carried out based on the known value of two nearest neighbors to address dataset plagued with missing values. The probabilistic approach is used to remove the inconsistency from either of the datasets like original or imputed. The resulting dataset is then given as an input to boosted instance selection approach for selection of relevant instances to reduce the size of the dataset without loss of generality and compromising classification accuracy. Finally, the transformed dataset (i.e., from non-imputed and inconsistent dataset to imputed and consistent dataset) is used for developing a classifier based on CRO trained FLANN. The method is evaluated extensively through a few bench-mark datasets obtained from University of California, Irvine (UCI) repository. The experimental results confirm that our preprocessing tasks along with integrated approach can be a promising alternative tool for mitigating missing value, inconsistent records, and noisy instances.

PurposeThe target of this research is to develop a mathematical model which combines the Resource-Constrained Multi-Project Scheduling Problem (RCMPSP) and the Multi-Skilled Resource-Constrained Project Scheduling Problem (MSRCPSP). Due to the importance of resource management, the proposed formulation comprises resource leveling considerations as well. The model aims to simultaneously optimize: (1) the total time to accomplish all projects and (2) the total deviation of resource consumptions from the uniform utilization levels.Design/methodology/approachThe K-Means (KM) and Fuzzy C-Means (FCM) clustering methods have been separately applied to discover the clusters of activities which have the most similar resource demands. The discovered clusters are given to the scheduling process as priori knowledge. Consequently, the execution times of the activities with the most common resource requests will not overlap. The intricacy of the problem led us to incorporate the KM and FCM techniques into a meta-heuristic called the Bi-objective Symbiosis Organisms Search (BSOS) algorithm so that the real-life samples of this problem could be solved. Therefore, two clustering-based algorithms, namely, the BSOS-KM and BSOS-FCM have been developed.FindingsComparisons between the BSOS-KM, BSOS-FCM and the BSOS method without any clustering approach show that the clustering techniques could enhance the optimization process. Another hybrid clustering-based methodology called the NSGA-II-SPE has been added to the comparisons to evaluate the developed resource leveling framework.Practical implicationsThe practical importance of the model and the clustering-based algorithms have been demonstrated in planning several construction projects, where multiple water supply systems are concurrently constructed.Originality/valueReviewing the literature revealed that there was a need for a hybrid formulation that embraces the characteristics of the RCMPSP and MSRCPSP with resource leveling considerations. Moreover, the application of clustering algorithms as resource leveling techniques was not studied sufficiently in the literature.

Objectives: To develop a machine learning-based model to select the most important features from a high-dimensional dataset to classify the patterns at high accuracy and reduce their dimensionality. Methods: The proposed feature selection method (FSIFC) forms and combines feature clusters incrementally and produces feature subsets each time. The method uses K-means clustering and Mutual Information (MI) to refine the feature selection process iteratively. Initially, two clusters of features are formed using K-means clustering (K=2) by taking features as the basis of clustering instead of taking the patterns (a traditional way). From these two clusters, the features with the highest MI value in each cluster are kept in a feature subset. Classification accuracies (CA) of the feature subset are calculated using three classifiers namely Support Vector Machines (SVM), Random Forest (RF), and k-nearest Neighbor (knn). The process is repeated by incrementing the value of K i.e. number of clusters; until a maximum user-defined value of K is reached. The best value of CA obtained from these trials is recorded and the corresponding feature set is finally accepted. Findings: The proposed method is demonstrated using ten datasets and the results are compared with the existing published results using three classifiers to determine the method's performance. The ten datasets are classified with average CAs of 92.72%, 93.13%, and 91.5%, using the SVM, RF, and K-NN classifiers respectively. The proposed method selects a maximum of thirty features from the datasets. In terms of selecting the most effective and the smallest feature sets, the proposed method outperforms eight other feature selection methods considering CAs. Novelty: The proposed model applies feature reduction using combined feature clustering and filter methods in an incremental way. This provides an improved selection of relevant features while removing those which are irrelevant at different trials. Keywords: Feature selection, High-dimensional datasets, K-means algorithm, Mutual information, Machine learning

We are developing computerized feature extraction and classification methods to analyze malignant and benign pulmonary nodules in three-dimensional (3-D) thoracic CT images. Internal structure features were derived from CT density and 3-D curvatures to characterize the inhomogeneous of CT density distribution inside the nodule. In the classification step, we combined an unsupervised k-means clustering (KMC) procedure and a supervised linear discriminate (LD) classifier. The KMC procedure classified the sample nodules into two classes by using the mean CT density values for two different regions such as a core region and a complement of the core region in 3-D nodule image. The LD classifier was designed for each class by using internal structure features. The stepwise procedure was used to select the best feature subset from multi-dimensional feature spaces. The discriminant scores output from the classifier were analyzed by receiver operating characteristic (ROC) method and the classification accuracy was quantified by the area, Az, under the ROC curve. We analyzed a data set of pulmonary nodules in this study. The results of this study indicate the potential of combining the KMC procedure and the LD classifier for computer-aided classification of pulmonary nodules.

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for development of a digital twin of gamma radiation background, and for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. In one segment of work, we developed a gamma background estimation model using a Longshort term memory (LSTM) network for one-step CPS time series prediction. The LSTM model was validated with two data sets of measurements from two independent NaI detectors positioned on a mobile platform. The data sets contained background radiation only and no orphan isotope sources. The LSTM model was constructed and tested using data from one of the detectors. Performance of the LSTM model was validate through one-step prediction of CPS time series of another NaI detector without re-training. This approach allows to create a digital twin for nuclear background estimation. Using LSTM, it could be possible to detect a source through subtraction of the estimated counts from the measured background. In another segment of work, we investigated detection of gamma emitting sources in the presence of complex background using unsupervised machine learning. Spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137Cs source, and another dataset contained 131I source.

Attribute reduction is a basic problem of data mining. In an attribute reduction algorithm, significance measures of attributes, search strategy, and stopping criterion are the three key points. We assume that the attribute which has a bigger maximal information coefficient (MIC) with the class label is more important to discriminate classes. An experiment is designed to examine the relationship between MIC and the classification accuracy. The experiment uses KNN and SVM algorithm to classify four datasets from UCI. Each attribute is used to classify their dataset, and the corresponding accuracies are recorded. Linear fitting models display the relationship of these accuracies and the MIC scores between each attribute and class label. The results of four datasets show that the MIC scores between each feature and class label have a positive linear relationship with classification accuracy. Furthermore, omitting some of the attributes with lower MIC value would not lessen the classification accuracy significantly.

Trustworthiness models to categorize and prioritize code for security improvement

A novel semi-supervised clustering algorithm is proposed that synergizes the benefits of supervised and unsupervised learning methods. Data are clustered using an unsupervised learning technique biased toward producing clusters as pure as possible in terms of class distribution. These clusters can then be used to predict the class of future points. For example, in database marketing this technique can be used to identify and characterize segments of the customer population likely to respond to a specific promotion. One key additional benefit of this approach is that it allows unlabeled data with unknown class to be used to improve classification accuracy. The objective function of a traditional clustering technique, cluster dispersion in the K-means algorithm, is modified to minimize both the within-cluster variance of the input attributes and a measure of cluster impurity based on the class labels. Minimizing the within-cluster variance of the examples is a form of capacity control to prevent overfitting...

The precise knowledge of the load weight of each operation of the quayside container crane is important for accurately assessing the service life of the crane. The load weight is directly related to the vibration intensity. Through the study on the vibration of the hoist motor of the crane in radial and axial directions, we can classify the load using K-means clustering algorithm and quantitative statistical analysis. Vibration in radial direction is significantly and positively correlated with that in axial direction by correlation analysis, which means that we can use the data only in one of the directions to carry out the study improving then the efficiency without degrading the accuracy of load classification. The proposed method can well represent the real-time working condition of the crane.

Remotely sensed imagery accounts for sensor specific information. The following paper deals with making use of data from multiple sources with similar temporal resolution to improve classification accuracy. This was done by clustering five masks or samples of 100 × 100 pixels selected randomly from multispectral data from Landsat TM and evaluation of cluster quality to find the number of naturally occurring clusters. This was followed by clustering the entire study area Landsat TM data using k-means algorithm and evaluation of the resulting cluster quality using silhouette coefficient to identify loosely classified pixels and mean silhouette value (threshold of the scene). Hyper-spectral data from Hyperion was used for only the loosely classified pixels identified above and was clustered using the k-means algorithm. Finally, soft decision level fusion method was applied to the clustering output from HS data with good quality clusters (clusters with silhouette coefficient above the mean) from the multi-spectral imagery to produce final classification maps. In the fused imagery, the overall Classification accuracy and Kappa Statistics increased significantly as compared to the multispectral imagery. Cluster validity indices like Silhouette coefficient is used to evaluate cluster quality and predict naturally occurring clusters. The decision level fusion of selective data from multiple sources has exhibited better classification results at reduced computational overheads.

The paper proposed a hybrid two-stage method of support vector machines (SVM) to increase its performance in classification accuracy. In this model, a filtering stage of the k nearest neighbor (kNN) rule was employed to collect information from training observations and re-evaluate balance weights for the observations based on their influences. The balance weights changed the policy of the discrete class label. A novel idea of real-valued class labels for transferring the balance weights was therefore proposed. Embedded in the class label, the weights given as the penalties of the uncertain outliers in the classification were considered in the quadratic programming of SVM, and produced a different hyperplane with higher accuracy. The adoption of kNN rule in the filtering stage has the advantage to distinguish the uncertain outliers in an independent way. The results showed that the classification accuracy of the hybrid model was higher than that of the classical SVM.

Aiming at the problems of the classical data classification method, this paper proposes a method using genetic algorithm and K-means algorithm to classify data. In order to improve the effectiveness of data analysis, considering that the classical K-means algorithm is easy to be influenced by the initial cluster center with random selection, this paper improves the K-means algorithm by using the method of optimizing the initial cluster center. This paper first uses the sorted neighborhood method (SNM) to preprocess the data, and then the K-means algorithm is used to cluster data. In order to improve the accuracy of the K-means algorithm, this paper optimizes the initial cluster center, and unifies the genetic algorithm for the data dimensionality reduction. The experimental results show that the proposed method has higher classification accuracy than the classical data classification method has.

Support Vector Machines (SVMs) ensembles have been widely used to improve classification accuracy in complicated pattern recognition tasks. In this work we propose to apply an ensemble of SVMs coupled with feature-subset selection methods to aleviate the curse of dimensionality associated with expression-based classification of DNA microarray data. We compare the single SVM classifier to SVM ensembles applying two different feature-subset selection techniques, namely random selection and k-means clustering, the base classifiers are combined using either majority vote or SVM fusion. Two real-world benchmarks datasets are used to evaluate and compare the performance. Experimental results show that the SVM ensemble of SVM base classifiers using k-means clustering for feature-subset selection and employing an SVM combiner achieve the best classification accuracy, and that feature-subset-selection methods can have considerable impact on the classification accuracy.