Mining Frequent Itemsets in Association Rule Mining Using Improved SETM Algorithm

Data mining has a great role for making market revenue by its decisive quality. Frequent Itemsets Mining (FIM) is very focused and emerging field of research of data mining. .FIM is used in association rule mining and other data analytic tasks. Frequent itemsets are patterns those occur very frequently in dataset. This paper provides efficacious and in-depth study of FIM algorithms. A comparative study of existing FIM algorithms and their variations are analyzed to find the limitations of these algorithms. Apriori ,Fp Growth, Eclat are basic methodologies for Frequent itemset mining. But there is issue for efficiency and memory consumption. Initially FIM was started in 1990’s and first algorithm, Apriori has drawback of scanning database multiple time, so complexity and memory consumption was high. Then Fp-Growth algorithm required scanning of dataset twice to form Fp-tree so huge memory consumption was required to hold the fp tree for Frequent Itemset Mining. After it Parallel frequent mining (shared and non-shared memory) overcome the drawback of FP-Growth algorithms by balancing the workload of single node to achieve efficiency. Now in present era of big dataset with voluminous and growing data at every moment, more prominent, efficient and scalable algorithm is required as complexity of these algorithms exponentially increases with the growth of rapid data .So objective of the study is in-depth literature review of Frequent Itemsets Mining algorithms to develop an efficient FIM algorithm.

Since the era of data explosion, data mining in transactional databases has become more and more important. There are many data mining techniques like association rule mining, the most important and well-researched one. Furthermore, frequent itemset mining is one of the fundamental but time-consuming steps in association rule mining. Most of the algorithms used in literature find frequent itemsets on search space items having at least a minsup and are not reused for mining next time. To deal with this problem, NOV-FI algorithms are proposed as a new approach in order to quickly detect frequent itemsets from transactional databases using an array of co-occurrences and occurrences of kernel item in at least one transaction. NOV-FI algorithms are easily expanded in distributed systems. Finally, the experimental results show that the proposed algorithms perform better than other existing algorithms.

SufRec, an algorithm for mining association rules: Recursivity and task parallelism

Online mining of data streams is an important data mining problem with broad applications. However, it is also a difficult problem since the streaming data possess some inherent characteristics. In this paper, we propose a new single-pass algorithm, called DSM-FI (data stream mining for frequent itemsets), for online incremental mining of frequent itemsets over a continuous stream of online transactions. According to the proposed algorithm, each transaction of the stream is projected into a set of sub-transactions, and these sub-transactions are inserted into a new in-memory summary data structure, called SFI-forest (summary frequent itemset forest) for maintaining the set of all frequent itemsets embedded in the transaction data stream generated so far. Finally, the set of all frequent itemsets is determined from the current SFI-forest. Theoretical analysis and experimental studies show that the proposed DSM-FI algorithm uses stable memory, makes only one pass over an online transactional data stream, and outperforms the existing algorithms of one-pass mining of frequent itemsets.

Finding the frequent itemset is one of the most investigated extents of data mining. The Apriori algorithm is the most established algorithm for frequent itemset mining, but it has issues regarding scanning frequent databases and generating a large amount of candidate sets. To solve these issues, an Improved Apriori algorithm was proposed. We examined the data structure, implementation, and algorithmic features that mainly focus on frequent itemset mining. We are representing an Improved Apriori algorithm on Spark in which simple and scalable implementation is done to achieve a faster process with lower support thresholds. We examined the improved Apriori algorithm on Extended Bakery Dataset and Retail Dataset. The results show execution time was reduced by 40% and 57% compared with the original Apriori algorithm.

Index-BitTableFI: An improved algorithm for mining frequent itemsets

Analyzing navigation history of a web site and that of a user reported in log files and web click stream helps in determining the components that shall be needed in the near future and fetch them in advance. Identifying the frequently visited websites from such data enhances the process. Processing and analyzing the vast amounts of click stream data and web logs that are generated at very high rates in a smart and cost-efficient way is a daunting challenge to the data mining community. Mining frequent itemsets plays an important role in analyzing such data streams. In spite of the existence of many such mining algorithms, more time and space efficient algorithms for mining frequent patterns are the need of the hour and are attracting wide attention in web click-stream analysis in recent years. In this paper an effective algorithm for mining frequent itemsets from a time-sensitive sliding window is proposed. It is a one-pass algorithm that uses a circular queue to implement the sliding window. The time-sensitive sliding window stores the web click stream data of various sessions of the web users. The proposed one-pass algorithm, FIM_CQTimeSWin has three phases: representation of time unit, maintenance of the sliding window and generation of frequent itemsets in the current sliding window.

Frequent pattern mining is the most important phase of association rule mining process because of its time and space complexity. Several methods have attempted to improve the performance of association rule mining by enhancing frequent pattern mining efficiency. Due to the large size of the data-sets and huge amounts of data which should be mined, many parallel and distributed mining approaches have been introduced to divide data-sets or to distribute mining processes between multiple processors or computers and thus, improve the efficiency of the mining process. In this paper, we propose a hadoop-based parallel implementation of PrePost+ algorithm for frequent itemset mining. In our parallel approach, the process of constructing N-Lists of itemsets has been distributed between the mappers and the operation of the final pruning process and extracting frequent itemsets has been carried out by reducers in a map-reduce parallel programming model. The experimental results show that our hadoop-based PrePost+(HBPrePost+) algorithm outperforms one of the best existing parallel methods of frequent itemset mining (PARMA) in terms of execution time.

Frequent itemset mining is an important problem in the data mining area. Extensive efforts have been devoted to developing efficient algorithms for mining frequent itemsets. However, not much attention is paid on managing the large collection of frequent itemsets produced by these algorithms for subsequent analysis and for user exploration. In this paper, we study three structures for indexing and querying frequent itemsets: inverted files, signature files and CFP-tree. The first two structures have been widely used for indexing general set-valued data. We make some modifications to make them more suitable for indexing frequent itemsets. The CFP-tree structure is specially designed for storing frequent itemsets. We add a pruning technique based on length-2 frequent itemsets to make it more efficient for processing superset queries. We study the performance of the three structures in supporting five types of containment queries: exact match, subset/superset search and immediate subset/superset search. Our results show that no structure can outperform other structures for all the five types of queries on all the datasets. CFP-tree shows better overall performance than the other two structures.

The step of mining frequent itemsets in database is the essential step and most expensive in the process of mining association rules in data mining task, many algorithms of mining frequent itemsets have been proposed to improve the performance of Apriori Algorithm. In this paper, we have introduced an optimization in the phase of generation pruning of candidates by a new strategy for the calculation of frequent itemsets based on approximate values of supports exact the itemsets. We have evaluated our algorithm AprioriMin against three popular frequent itemsets mining algorithms — Apriori and FP-growth, Close using two data sets with a variety of minimum support.

This paper shows a mathematical foundation for almost important features in the problem of discovering knowledge by association rules. The class of frequent itemsets and the association rule set are partitioned into disjoint classes by two equivalence relations based on closures. Thanks to these partitions, efficient parallel algorithms for mining frequent itemsets and association rules can be obtained. Practically, one can mine frequent itemsets as well as association rules just in the classes that users take care of. Then, we obtain structures of each rule class using corresponding order relations. For a given relation, each rule class splits into two subsets of basic and consequence. The basic one contains minimal rules and the consequence one includes in the rules that can be deducted from those minimal rules. In the rest, we consider association rule mining based on order relation min. The explicit form of minimal rules according to that relation is shown. Due to unique representations of frequent itemsets through their generators and corresponding eliminable itemsets, operators for deducting all remaining rules are also suggested. Experimental results show that mining association rules based on relation min is better than the ones based on relations of minmin and minMax in terms of reduction in mining times as well as number of basic rules.

PrePost+: An efficient N-lists-based algorithm for mining frequent itemsets via Children–Parent Equivalence pruning

Mining frequent itemsets in databases is an important and widely studied problem in data mining research. The problem of mining frequent itemsets is usually solved by constructing candidates of itemsets, and identifying those itemsets that meet the requirement of frequent itemsets. This paper proposes a novel algorithm based on BitTable (or bitmap) representation of the data. Data - related to frequent itemsets - are stored in spare matrices. Simple matrix and vector multiplications are used to calculate the support of the potential n+1 itemsets. The main benefit of this approach is that only bitmaps of the frequent itemsets are generated. The concept is simple and easily interpretable and it supports a compact and effective implementation (in MATLAB). An application example related to the BMS-WebView-1 benchmark data is presented to illustrate the applicability of the proposed algorithm.

In this chapter, we present granule based inter association mining to reduce the complexity of intertransaction association rule mining. To compare with other methods, our method can reduce the width of sliding windows. It uses granules to replace extended item sets. Thus, we do not need to consider too many combinations of extended items. We also propose the concept of precision in order to evaluate the effectiveness of intertransaction association rule mining. The experiments show that the proposed method is promising.

Frequent itemset mining and association rule mining are fundamental problems in data mining. Despite of the intensive and continuous researches on frequent itemset mining, one essential and not completely solved drawback still remains. The drawback is pattern explosion: a huge number of frequent itemsets, or patterns, will be often derived. The association rule mining takes over the same drawback since each association rule consists of two frequent itemsets. One promising solution for alleviating this essential drawback practically as a post-processing is to rank the patterns and rules from various aspects and to identify which one to be examined first. In this paper, we propose to utilize the technology of representation learning to rank frequent itemsets and association rules. More specifically, we develop several evaluation measures for frequent itemsets and association rules by using the vector representations of items, transactions and frequent patterns which are obtained by existing representation learning techniques. By employing vector representations of three kinds of objects, various rankings reflecting different aspects can expect to be obtained.