Class noise detection using frequent itemsets

Noise correction to improve data and model quality for crowdsourcing

Effective vector representation has been proven useful for transaction classification and clustering tasks in Cyber-Physical Systems. Traditional methods use heuristic-based approaches and different pruning strategies to discover the required patterns efficiently. With the extensive and high dimensional availability of transactional data in cyber-physical systems, traditional methods that used frequent itemsets (FIs) as features suffer from dimensionality, sparsity, and privacy issues. In this paper, we first propose a federated learning-based embedding model for the transaction classification task. The model takes transaction data as a set of frequent item-sets. Afterward, the model can learn low dimensional continuous vectors by preserving the frequent item-sets contextual relationship. We perform an in-depth experimental analysis on the number of high dimensional transactional data to verify the developed models with attention-based mechanism and federated learning. From the results, it can be seen that the designed model can help and improve the decision boundary by reducing the global loss function while maintaining both security and privacy.

DEFINITION Let I be a set of binary-valued attributes, called items. A set X ⊆ I is called an itemset. A transaction database D is a multiset of itemsets, where each itemset, called a transaction, has a unique identifier, called a tid. The support of an itemset X in a dataset D, denoted sup(X), is the fraction of transactions in D where X appears as a subset. X is said to be a frequent itemset in D if sup(X) ≥ minsup, where minsup is a user defined minimum support threshold. An (frequent) itemset is called closed if it has no (frequent) superset having the same support. An association rule is an expression A ⇒ B, where A and B are itemsets, and A ∩ B = ∅. The support of the rule is the joint probability of a transaction containing both A and B, given as sup(A ⇒ B) = P (A ∧ B) = sup(A ∪ B). The confidence of a rule is the conditional probability that a transaction contains B, given that it contains A, given as: conf(A ⇒ B) = P (B|A) = P (A∧B) P (A) = sup(A∪B) sup(A) . A rule is frequent if the itemset A∪B is frequent. A rule is confident if conf ≥ minconf , where minconf is a user-specified minimum threshold. The aim of non-redundant association rule mining is to generate a rule basis, a small, non-redundant set of rules, from which all other association rules can be derived.

Frequent itemsets mining is one of the interesting applications of data mining. Recently data mining has got a great deal of attention due to the explosive growth in data and the economic and scientific need for turning such data into useful information. However, the traditional frequent itemsets mining algorithms have become inefficient to work with large datasets effectively on a single machine due to computational power and memory limits. Current methods prefer to control the execution time and output by using higher minimum support thresholds, which lead to less candidates and frequent itemsets. In this paper, an improved-version of Apriori like HFDM-EB algorithm that can deal with lower minimum support thresholds is proposed for mining frequent itemsets over big transactional data on Hadoop framework and by utilizing compressed bitmaps. The experimental results show that the improved algorithm is efficient and scalable for mining frequent itemsets in big data.

Vertical itemset mining is an important frequent pattern mining problem with broad applications. It is challenging since one may need to examine a combinatorial explosive number of possible patterns of items of a dataset in a traditional horizontal algorithm. Since high dimensional datasets typically contain a large number of columns and a small number of rows, vertical itemset mining algorithms, which extract the frequent itemsets of dataset by producing all combination of rows ids, are a good alternative for horizontal algorithms in mining frequent itemsets from high dimensional dataset. Since a rowset can be simply produced from its subsets by adding a new row id to a sub rowset, many bottom up vertical itemset mining algorithms are designed and represented in the literature. However, bottom up vertical mining algorithms suffer from a main drawback. Bottom-up algorithms start the process of generating and testing of rowsets from the small rowsets and go on to the larger rowsets, whereas the small rowsets cannot produce a frequent itemsets because they contain less than minimum support threshold number of rows. In this paper, we described a new efficient vertical top down algorithm called VTD (Vertical Top Down) to conduct mining of frequent itemsets in high dimensional datasets. Our top down approach employed the minimum support threshold to prune the rowsets which any itemset could not be extracted from them. Several experiments on real bioinformatics datasets showed that VTD is orders of magnitude better than previous closed pattern mining algorithms. Our performance study showed that this algorithm outperformed substantially the best former algorithms.

The presence of noise in a measurement dataset can have a negative effect on the classification model built. More specifically, the noisy instances in the dataset can adversely affect the learnt hypothesis. Removal of noisy instances will improve the learnt hypothesis; thus, improving the classification accuracy of the model. A clustering-based noise detection approach using the k-means algorithm is presented. We present a new metric for measuring the potentiality (noise factor) of an instance being noisy. Based on the computed noise factor values of the instances, the clustering-based algorithm is then used to identify and eliminate p% of the instances in the dataset. These p% of instances are considered the most likely to be noisy among the instances in the dataset - the p% value is varied from 1% to 40%. The noise detection approach is investigated with respect to two case studies of software measurement data obtained from NASA software projects. The two datasets are characterized by the same thirteen software metrics and a class label that classifies the program modules as fault-prone and not fault-prone. It is shown that as more noisy instances are removed, classification accuracy of the C4>5 learner improves. This indicates that the removed instances are most likely noisy instances that attributed to poor classification accuracy.

Bioinformatics has contributed to a different form of datasets called as high dimensional datasets. The high dimensional datasets are characterized by a large number of features and a small number of samples. The traditional algorithms expend most of the running time in mining large number of small and mid-size items which does not enclose valuable and significant information. The recent research focused on mining large cardinality itemsets called as colossal itemsets which are significant to many applications, especially in the field of bioinformatics. The existing frequent colossal itemset mining algorithms are unsuccessful in discovering complete set of significant frequent colossal itemsets. The mined colossal itemsets from existing algorithms provide erroneous support information which affects association analysis. Mining significant frequent colossal itemsets with accurate support information helps in attaining a high-level accuracy of association analysis. The proposed work highlights a novel pre-processing technique and bottom-up row enumeration algorithm to mine significant frequent colossal itemsets with accurate support information. A novel pre-processing technique efficiently utilizes minimum support threshold and minimum cardinality threshold to prune irrelevant samples and features. The experiment results demonstrate that the proposed algorithm has high accuracy over existing algorithms. Performance study indicates the efficiency of the pre-processing technique.

Research in machine learning, data mining and related areas has produced a wide variety of algorithms for cost-sensitive (CS) classification, where instead of maximizing the classification accuracy, minimizing the misclassification cost becomes the objective. However, these methods assume that training sets do not contain significant noise, which is rarely the case in real-world environments. In this paper, we systematically study the impacts of class noise on CS learning, and propose a cost-guided class noise handling algorithm to identify noise for effective CS learning. We call it cost-guided iterative classification filter (CICF), because it seamlessly integrates costs and an existing classification filter (C. Brodley and M. Friedl, 1999) for noise identification. Instead of putting equal weights to handle noise in all classes in existing efforts, CICF puts more emphasis on expensive classes, which makes it especially successful in dealing with datasets with a large cost-ratio. Experimental results and comparative studies from real-world datasets indicate that the existence of noise may seriously corrupt the performance of CS classifiers, and by adopting the proposed CICF algorithm, we can significantly reduce the misclassification cost of a CS classifier in noisy environments.

This paper presents a new efficient algorithm for mining frequent closed itemsets. It enumerates the closed set of frequent itemsets by using a novel compound frequent itemset tree that facilitates fast growth and efficient pruning of search space. It also employs a hybrid approach that adapts search strategies, representations of projected transaction subsets, and projecting methods to the characteristics of the dataset. Efficient local pruning, global subsumption checking, and fast hashing methods are detailed in this paper. The principle that balances the overhead of search space growth and pruning is also discussed. Extensive experimental evaluations on real world and artificial datasets showed that our algorithm outperforms CHARM by a factor of five and is one to three orders of magnitude more efficient than CLOSET and MAFIA.

Maximal frequent item sets are one of several condensed representations of frequent item sets, which store most of the information contained in frequent item sets using less space, thus being more suitable for stream mining. This paper focuses on mining maximal frequent item sets approximately over a stream landmark model. We separate the continuously arriving transactions into sections, and the mining results are indexed by an extended direct update tree, thus, a simple but effective algorithm named SMIS is proposed. In our algorithm, we employ the Chern off Bound to perform the maximal frequent item set mining in a false negative manner, which can reduce the memory cost, as well guarantee our algorithm conducting with an incremental fashion. Our experimental results on two synthetic datasets and two real world datasets show that SMIS achieves much reduced memory cost in comparison with the state-of-the-art algorithm with a 100 percent precision.

Mining generalized association rules among items in the presence of taxonomy and with nonuniform minimum support has been recognized as an important model in the data mining community. In real applications, however, the work of discovering interesting association rules is an iterative process; the analysts have to continuously adjust the constraint of minimum support to discover real informative rules. How to reduce the response time for each remining process thus becomes a crucial issue. In this paper, we examine the problem of maintaining the discovered multi-supported generalized association rules when the multiple minimum support constraint is refined and propose a novel algorithm called RGA_MSR to accomplish the work. By keeping and utilizing the set of frequent itemsets and negative border, and adopting vertical intersection counting strategy, the proposed RGA_MSR algorithm can significantly reduce the computation time spent on rediscovery of frequent itemsets and has very good performance.

In this paper, we present an improved method for predicting missing attribute values in data sets. We make use of frequent itemsets, generated from the association rules algorithm, displaying the correlations between different items in a set of transactions. In particular, we consider a database as a set of transactions and each data instance as an itemset. Then frequent itemsets can be used as a knowledge base to predict missing attribute values. Our approach integrates the RSFit method based on rough sets theory that produces faster predictions by considering similarities of attribute value pairs, but only for those attributes contained in the core or reduct of the data set. Using empirical studies on UCI and other real world data sets, we demonstrate a significant increase in prediction accuracy obtained from our new integrated approach, referred to as ItemRSFit.

Association rules, introduced by Agrawal, Imielinski and Swami (1993), provide useful means to discover associations in data. The problem of mining association rules in a database is defined as finding all the association rules that hold with more than a user-given minimum support threshold and a user-given minimum confidence threshold. According to Agrawal, Imielinski and Swami, this problem is solved in two steps: 1. Find all frequent itemsets in the database. 2. For each frequent itemset I, generate all the association rules I’ÞI\I’, where I’ÌI.

Class noise detection based on software metrics and ROC curves

Objectives: To develop a memory efficient, incremental and interactive distributed FPM having less communication and synchronization overhead with good load balancing capability, to analyze the dynamic transactional data in a distributed database. Methods/Analysis: This technique adopts prefix based equivalence class partitioning scheme to generate frequent item sets without generating local frequent sets with low memory consumption. This approach uses a range of support values to update the frequent patterns with less time complexity. This paper proposes distributed FPM techniques with both count distributed and compressed data distributed parallel approaches. The performance of the algorithms are tested and compared with popular distributed FPM algorithms using standard datasets. Findings: To deal with the massive dynamic data stored in distributed databases, this approach develops three distributed frequent set generation algorithms, which update frequent patterns by reusing the previously stored pattern information with no complex calculations or data structures. The proposed approaches also provide the user with the facility to interactively adjust the minimum support value as per their own conveniences by keeping the nearly frequent itemsets with the help of two minimum support thresholds (low, high). Measures have been taken to reduce the additional itemset storage and computations as well as to achieve good load balancing with low communication and synchronization overhead. Since the proposed algorithms adopt prefix based equivalent class partitioning technique at each n-itemset level and undergo four levels of itemset filtering to remove infrequent items from each class before calculating the individual item count, the inter node communication required is less in this approach. To eliminate the drawbacks of both count and data distribution approaches one of the algorithms proposed adopts a hybrid approach which distributes the compressed data only once, hence communication overhead is less compared with other DD algorithms. Conclusion/Application: The proposed distributed techniques reduce memory utilization and itemset comparisons compared to the existing approaches. The performances are tested and evaluated for market analysis and online credit card fraud detection applications.