Closed Itemset Mining: A graph theory perspective

DEFINITION Let I be a set of binary-valued attributes, called items. A set X ⊆ I is called an itemset. A transaction database D is a multiset of itemsets, where each itemset, called a transaction, has a unique identifier, called a tid. The support of an itemset X in a dataset D, denoted sup(X), is the fraction of transactions in D where X appears as a subset. X is said to be a frequent itemset in D if sup(X) ≥ minsup, where minsup is a user defined minimum support threshold. An (frequent) itemset is called closed if it has no (frequent) superset having the same support. An association rule is an expression A ⇒ B, where A and B are itemsets, and A ∩ B = ∅. The support of the rule is the joint probability of a transaction containing both A and B, given as sup(A ⇒ B) = P (A ∧ B) = sup(A ∪ B). The confidence of a rule is the conditional probability that a transaction contains B, given that it contains A, given as: conf(A ⇒ B) = P (B|A) = P (A∧B) P (A) = sup(A∪B) sup(A) . A rule is frequent if the itemset A∪B is frequent. A rule is confident if conf ≥ minconf , where minconf is a user-specified minimum threshold. The aim of non-redundant association rule mining is to generate a rule basis, a small, non-redundant set of rules, from which all other association rules can be derived.

Discovering interesting patterns in transactional databases is often a challenging area by the length of patterns and number of transactions in data mining, which is prohibitively expensive in both time and space. Closed itemset mining is introduced from traditional frequent pattern mining and having its own importance in data mining applications. Recently, regular itemset mining gained lot of attention in data mining research because of its occurrence behavior. In this paper we propose a new method called RCP-method (regular-closed pattern method) to mine regular-closed itemsets in transactional database by using vertical data format. Our RCP-method generates complete set of regular-closed patterns in transactional databases for a user given regularity threshold and support. Our experimental results show that this method is efficient in memory and execution time.

Since the era of data explosion, data mining in transactional databases has become more and more important. There are many data mining techniques like association rule mining, the most important and well-researched one. Furthermore, frequent itemset mining is one of the fundamental but time-consuming steps in association rule mining. Most of the algorithms used in literature find frequent itemsets on search space items having at least a minsup and are not reused for mining next time. To deal with this problem, NOV-FI algorithms are proposed as a new approach in order to quickly detect frequent itemsets from transactional databases using an array of co-occurrences and occurrences of kernel item in at least one transaction. NOV-FI algorithms are easily expanded in distributed systems. Finally, the experimental results show that the proposed algorithms perform better than other existing algorithms.

In data mining finding out the frequent itemsets is one of the very essential topics. Data mining helps in identifying the best knowledge for different decision makers. Frequent itemset generation is the precondition and most time-consuming method for association rule mining. In this paper we suggest a new algorithm for frequent itemset detection that works with datasets in distributed manner. The proposed algorithm brings in a new method to find frequent itemset not including the necessitate to create candidate itemsets. The proposed approach could be implemented using horizontal representation for transaction datasets and allocating prime value. It explores all the frequent itemset that is present in the input and according to the support the maximum frequent itemset is identified. It was applied on different transactions database and compared with well-known algorithms: FP-Growth and Parallel Apriori with different support levels. The try out showed that the proposed algorithm attain major time improvement over both algorithms.

Mining frequent itemsets is a fundamental task in data mining. Unfortunately the number of frequent itemsets describing the data is often too large to comprehend. This problem has been attacked by condensed representations of frequent itemsets that are subcollections of frequent itemsets containing only the frequent itemsets that cannot be deduced from other frequent itemsets in the subcollection, using some deduction rules. In this paper we review the most popular condensed representations of frequent itemsets, study their relationship to transaction databases and each other, examine their combinatorial and computational complexity, and describe their relationship to other important concepts in combinatorial data analysis, such as Vapnik-Chervonenkis dimension and hypergraph transversals.

Finding frequent itemsets is a popular data mining problem, aiming to extract hidden patterns from a transactional database. Several bio-inspired approaches to solve this problem have been proposed to overcome the poor performance of exact algorithms, such as Apriori and FPGrowth. Approaches based on genetic algorithms are among the most efficient ones from the point of view of runtime performance, but they are still inefficient in terms of solution’s quality, i.e., the number of frequent itemsets discovered. To deal with this issue, we propose in this paper a new genetic algorithm for finding frequent itemsets called GA-Apriori, in which the crossover and mutation operators are defined by taking into account the Apriori heuristic principle. The results of our evaluation show that GA-Apriori outperforms other approaches to frequent itemset mining based on genetic algorithms, especially when dealing with large instances. The experiments also show that GA-Apriori is competitive with exact approaches in terms of the number of frequent itemsets discovered.

The discovery of frequent itemsets is one of the very important topics in data mining. Frequent itemset discovery techniques help in generating qualitative knowledge which gives business insight and helps the decision makers. In the Big Data era the need for a customizable algorithm to work with big data sets in a reasonable time becomes a necessity. In this paper we propose a new algorithm for frequent itemset discovery that could work in distributed manner with big datasets. Our approach is based on the original Buddy Prima algorithm and the Greatest Common Divisor (GCD) calculation between itemsets which exist in the transaction database. The proposed algorithm introduces a new method to parallelize the frequent itemset mining without the need to generate candidate itemsets and also it avoids any communication overhead between the participated nodes. It explores the parallelism abilities in the hardware in case of single node operation. The proposed approach could be implemented using map-reduce technique or Spark. It was successfully applied on different size transactions DBs and compared with two well-known algorithms: FP-Growth and Parallel Apriori with different support levels. The experiments showed that the proposed algorithm achieves major time improvement over both algorithms especially with datasets having huge number of items.

Every element of the transaction in a transaction database may contain the components such as item number, quantity, cost of the item bought and some other relevant information of the customer. Most of the association rules mining algorithms to discover frequent itemsets do not consider the components such as quantity, cost etc. In a large database it is possible that even if the itemset appears in a very few transactions, it may be purchased in a large quantity. Further, this may lead to very high profit. Therefore these components are the most important information and without which it may cause the lose of information. This motivated us to propose a parallel algorithm to discover all frequent itemsets based on the quantity of the item bought in a single scan of the database. This method achieves its efficiency by applying two new ideas. Firstly, transaction database is converted into an abstraction called Weighted Tree that prevents multiple scanning of the database during the mining phase. This data structure is replicated among the parallel nodes. Secondly, for each frequent item assigned to a parallel node, an item tree is constructed and frequent itemsets are mined from this tree based on weighted minimum support.

Most retail companies have implemented computer-based information systems for recording sales transaction data. In the implementation of information systems, the data collected in the database is processed limited to making reports such as sales reports and inventory reports. Database generated from computer-based information systems can be further processed to obtain more valuable information. One strategy for using sales transaction data is to analyze consumer spending patterns. Consumer spending patterns can be in the form of associations of items that are often purchased simultaneously. The association between goods can be determined using the frequent itemset search technique. The Fp-growth algorithm is an algorithm that can be used to determine frequent itemsets in a data set. This article describes the results of implementing the FP-Growth algorithm in the consumer shopping pattern analysis application. The resulting shopping pattern is in the form of goods that are often purchased simultaneously by consumers. From the results of the application of the fp-growth algorithm, it was found that the minimum value of support had an effect, namely the smaller the input value of support, the more pairs of items were obtained. The application of the FP-Growth algorithm in determining frequent itemsets in association data mining can find customer spending habits in buying goods simultaneously.

Association rules mining is one of the most important problems in knowledge discovery and data mining. The goal of it is to acquire consumption habits of customers by discovering the relationships between items from a transaction database that has a large number of transactions and items. The most compute intensive process for ARM is to find out the frequent 1-itemsets and 2-itemsets. In this paper, we propose a quantum algorithm for finding out the frequent 1-itemsets and 2-itemsets. In our algorithm, to mine the frequent 1-itemsets efficiently, we use the technique of amplitude amplification. To mine the frequent 2-itemsets efficiently, we propose a new tomography scheme, i.e., pure-state-based quantum state tomography. It is shown that our algorithm is potential to achieve exponential speedup in the number of transactions and polynomial speedup in the number of items over the classical algorithm.

Frequent itemset mining is a fundamental problem in data mining area because frequent itemsets have been extensively used in reasoning, classifying, clustering, and so on. To mine frequent itemsets, previous algorithms based on a prefix tree structure have to construct many prefix trees, which is very time-consuming. In this paper, we propose a novel frequent itemset mining algorithm called DPT (Dynamic Prefix Tree) which uses only one prefix tree. We first introduce the concept of the post-conditional database of an itemset, and analyze the distribution of an itemset’s post-conditional database in a prefix tree representing a database. Subsequently, we illuminate how DPT adjusts the prefix tree to mine frequent itemsets and give three optimization techniques. An interesting advantage of DPT is that the algorithm can directly output a prefix tree representing all frequent itemsets after slight modifications. Using only one dynamic prefix tree, DPT avoids the high cost of constructing many prefix trees and thus gains significant performance improvement. Experimental results show that DPT remarkably outperforms previous algorithms with respect to running time and memory usage, and that a prefix tree representing all frequent itemsets DPT outputs can be used more efficient than a list representing them previous algorithms output.

Mining frequent itemsets is an important research task for knowledge discovery, which is to discover the groups of items appearing always together excess of a user specified threshold from a transaction database. However, there may be many frequent itemsets existing in a transaction database, such that it is difficult to make a decision for a decision maker. Recently, mining frequent closed itemsets becomes a major research issue. The reason is that all frequent itemsets can be derived from frequent closed itemsets. In addition, the transactions in a database will increase constantly. It is a challenge that how to update the previous frequent closed itemsets from the increased transactions. In this paper, we propose an efficient algorithm MRFI for incrementally mining frequent closed itemsets without scanning original database. MRFI algorithm generates frequent closed itemsets by performing some operations on the previous closed itemsets and the added transactions without doing any searching operation. Finally, the experimental results show that MRFI algorithm performs much better than the previous approaches.

In this chapter, we present granule based inter association mining to reduce the complexity of intertransaction association rule mining. To compare with other methods, our method can reduce the width of sliding windows. It uses granules to replace extended item sets. Thus, we do not need to consider too many combinations of extended items. We also propose the concept of precision in order to evaluate the effectiveness of intertransaction association rule mining. The experiments show that the proposed method is promising.

Ghaggar river basin of northern India has been experiencing flash floods many times during the last three decades despite locating in semi-arid/arid regions. Identification of the flood risk zone of a basin is essential where the availability of flood-related data is limited. The present study proposed the graph theory to identify the flood risk zone using spatial database of the digital elevation model, land use and land cover, soil cover, population, rainfall, drainage density, distance from roads, rivers, and flood shelters. Further, the results obtained from graph theory were compared with the results obtained by Analytical Hierarchy Process. It is observed that graph theory can also identify the flood risk zone of a basin precisely with acceptable accuracy. Around 94.1% of the study area is observed to have a high flood risk using Analytical Hierarchy Process, whereas 100% area is observed to have a high flood risk using graph theory.

Mining association rules is an important task. Past transaction data can be analyzed to discover customer purchasing behaviors such that the quality of business decision can be improved. The association rules describe the associations among items in the large database of customer transactions. However, the size of the database can be very large. It is very time consuming to find all the association rules from a large database, and users may be only interested in the associations among some items. Moreover, the criteria of the discovered rules for the user requirements may not be the same. Many uninteresting association rules for the user requirements can be generated when traditional mining methods are applied. Hence, a data mining language needs to be provided such that users can query only interesting knowledge to them from a large database of customer transactions. A data mining language is presented. From the data mining language, users can specify the interested items and the criteria of the rules to be discovered. Also, an efficient data mining technique is proposed to extract the association rules according to the users requests.