A Simple But Effective Stream Maximal Frequent Itemset Mining Algorithm

The association rule mining can be divided into two steps. The first step is to find out all frequent itemsets, whose occurrences are greater than or equal to the user-specified threshold. The second step is to generate reliable association rules based on all frequent itemsets found in the first step. Identifying all frequent itemsets in a large database dominates the overall performance in the association rule mining. In this paper, we propose an efficient hash-based method, HMFS, for discovering the maximal frequent itemsets. The HMFS method combines the advantages of both the DHP (Direct Hashing and Pruning) and the Pincer-Search algorithms. The combination leads to two advantages. First, the HMFS method, in general, can reduce the number of database scans. Second, the HMFS can filter the infrequent candidate itemsets and can use the filtered itemsets to find the maximal frequent itemsets. These two advantages can reduce the overall computing time of finding the maximal frequent itemsets. In addition, the HMFS method also provides an efficient mechanism to construct the maximal frequent candidate itemsets to reduce the search space. We have implemented the HMFS method along with the DHP and the Pincer-Search algorithms on a Pentium III 800 MHz PC. The experimental results show that the HMFS method has better performance than the DHP and the Pincer-Search algorithms for most of test cases. In particular, our method has significant improvement over the DHP and the Pincer-Search algorithms when the size of a database is large and the length of the longest itemset is relatively long.

Discovering frequent itemsets is a key problem in important data mining applications, such as the discovery of association rules, strong rules, episodes, and minimal keys. Typical algorithms for solving this problem operate in a bottom-up breadth-first search direction. The computation starts from frequent 1-itemsets (minimal length frequent itemsets) and continues until all maximal (length) frequent itemsets are found. During the execution, every frequent itemset is explicitly considered. Such algorithms perform reasonably well when all maximal frequent itemsets are short. However, performance drastically decreases when some of the maximal frequent itemsets are relatively long. We present a new algorithm which combines both the bottom-up and top-down directions. The main search direction is still bottom-up but a restricted search is conducted in the top-down direction. This search is used only for maintaining and updating a new data structure we designed, the maximum frequent candidate set. It is used to prune candidates in the bottom-up search. As a very important characteristic of the algorithm, it is not necessary to explicitly examine every frequent itemset. Therefore it performs well even when some maximal frequent itemsets are long. As its output, the algorithm produces the maximum frequent set, i.e., the set containing all maximal frequent itemsets, which therefore specifies immediately all frequent itemsets. We evaluate the performance of the algorithm using a well-known benchmark database. The improvements can be up to several orders of magnitude, compared to the best current algorithms.

When it comes to association rule mining, all frequent itemsets are first found, and then the confidence level of association rules is calculated through the support degree of frequent itemsets. As all non-empty subsets in frequent itemsets are still frequent itemsets, all frequent itemsets can be acquired only by finding all maximal frequent itemsets (MFIs), whose supersets are not frequent itemsets. In this study, an algorithm, named right-hand side expanding (RHSE), which can accurately find all MFIs, was proposed. First, an Expanding Operation was designed, which, starting from any given frequent itemset, could add items using certain rules and form some supersets of given frequent itemsets. In addition, these supersets were all MFIs. Next, this operator was used to add items by taking all frequent 1-itemsets as the starting point alternately, and all MFIs were found in the end. Due to the special design of the Expanding Operation, each MFI could be found. Moreover, the path found was unique, which avoided the algorithm redundancy in temporal and spatial complexity. This algorithm, which has a high operating rate, is applicable to the big data of high-dimensional mass transactions as it is capable of avoiding the computing redundancy and finding all MFIs. In the end, a detailed experimental report on 10 open standard transaction sets was given in this study, including the big data calculation results of million-class transactions.

Frequent itemsets (FIs) mining from uncertain database is a very popular research area nowadays. Many algorithms have been proposed to mine FI from uncertain database. But in typical FI mining process, all the FIs have to be mined individually, which needs a huge memory. Four trees are proposed in this paper which are: (i) maximal frequent itemset from uncertain database (MFU) tree which contains only the maximal frequent itemsets generated from uncertain database, (ii) closed frequent itemset from uncertain database (CFU) tree which contains only closed frequent itemsets generated from uncertain database, (iii) maximal frequent itemset from uncertain data stream (MFUS) tree which contains maximal frequent itemsets generated from uncertain data stream and (iv) closed frequent itemset from uncertain data stream (CFUS) tree which contains closed frequent itemsets generated from uncertain data stream. Experimental results are also presented which show that maximal and closed frequent itemsets mining requires less time and memory than typical frequent itemsets mining.

In the discipline of data mining, association rule mining is an important study topic that focuses on discovering the relationships between database attributes. The maximum frequent itemset comprises the information of all frequent itemsets, which is one of the important difficulties in mining association rules, and certain data mining applications just need to mine the maximum frequent itemsets. As a result, analyzing the maximum frequent itemset mining technique is practical. Considering this, the research introduces FP-MFIA, a new maximum frequent itemset mining approach based on the FP-tree, which is inspired by the data structure of the frequent pattern tree and the idea that the maximum frequent itemset implies all frequent itemsets. First, the FP-MFIA constructs a one-way FP-tree structure, which only has pointers from the root to the leaves, so that only two scans of the FP-tree are required by the FP-MFIA. On the other hand, it redefines a data storage structure MFI-list for maximum frequent itemsets. It can quickly release unnecessary nodes in the FP-tree after scanning it. In this way, not only the information required by the maximum frequent itemsets can be quickly mined but also the space required for storing the maximum frequent itemsets can be reduced, which greatly improves the mining efficiency. Finally, experiments were conducted to compare the mining efficiency of the novel FP-MFIA algorithm to the IDMFIA and DMFIA algorithms. We can see from the findings that the FP-MFIA algorithm is more efficient than the other two techniques.

Mining maximal frequent itemsets is a fundamental and important issue in many data mining application. A new depth-first search algorithm for mining maximal frequent itemsets called DFMFI (depth-first search for maximal frequent itemsets) is proposed, which can reduce the number of candidate itemsets and the cost of support counting. DFMFI projects the dataset information stored by the compressed FP-tree into the conditional matrix, and improves efficiency of support counting by using vector logic operation. Global 2-itemset pruning and local extension pruning used to prune the search space effectively. The experiments results verify the efficiency and advantage of this DFMFI.

Association rule has been an area of active research in the field of knowledge discovery. Data mining researchers had improved upon the quality of association rule mining for business development by incorporating influential factors like value (utility), quantity of items sold (weight) and more for the mining of association patterns. In this paper, we propose an efficient approach to find maximal frequent item set first. Most of the algorithms in literature used to find minimal frequent item first, then with the help of minimal frequent item sets derive the maximal frequent item sets. These methods consume more time to find maximal frequent item sets. To overcome this problem, we propose a navel approach to find maximal frequent item set directly using the concepts of subsets. The proposed method is found to be efficient in finding maximal frequent item sets.

The proposed algorithm, used for mining maximal frequent itemsets, finds interesting correlations between data in an efficient manner and can be helpful in case of large databases where large frequent itemsets contain large number of attributes. The algorithm used, works efficiently when the number of items in the large frequent itemsets is close to the number of total attributes in the dataset, or if the number of items in the large frequent itemsets is predetermined. The salient feature of the proposed approach is that it always automatically satisfies downward closure property, which states that all subsets of a frequent set must also be frequent. Owing to bit-set representation of dataset and utilization of unique list structure, the process of counting the occurrences of itemsets becomes faster, which ultimately enhances the overall mining performance.

Data mining is the practice to search large amount of data to discover data patterns.Data mining uses mathematical algorithms to group the data and evaluate the future events.Association rule is a research area in the field of knowledge discovery.Many data mining researchers had improved upon the quality of association rule for business development by incorporating influential factors like utility, number of items sold and for the mining of association data patterns.In this paper, we propose an efficient algorithm to find maximal frequent itemset first.Most of the association rule algorithms used to find minimal frequent item first, then with the help of minimal frequent itemsets derive the maximal frequent itemsets, these methods consume more time to find maximal frequent itemsets.To overcome this problem, we propose a new approach to find maximal frequent itemset directly using the concepts of subsets.The proposed method is found to be efficient in finding maximal frequent itemsets.

Privacy preserving data mining is a vibrant area in data mining. The sharing of data between the organizations is found to be beneficial for business growth. However, privacy policies and threats prevent the data owners from sharing the data for mini

In this paper we present a novel hybrid (arraybased layout and vertical bitmap layout) database representation approach for mining complete Maximal Frequent Itemset (MFI) on sparse and large datasets. Our work is novel in terms of scalability, item search order and two horizontal and vertical projection techniques. We also present a maximal algorithm using this hybrid database representation approach. Different experimental results on real and sparse benchmark datasets show that our approach is better than previous state of art maximal algorithms.

This paper addressed the insecurity and the inefficiency of privacy preserving association rule mining in vertically partitioned data. We presented a privacy preserving maximal frequent itemsets mining algorithm in vertically partitioned data. The algorithm adopted a more secure vector dot protocol which used an inverse matrix to hide the original input vector, and without any site revealing privacy vector. The mining strategy was based on depth-first search for the maximal frequent itemsets. Performance analysis and experimental analysis showed that the algorithm possessed higher security and efficiency.

In the relational model, inclusion dependencies (INDs) convey many information on data semantics. They generalize foreign keys, which are very popular constraints in practice. However, one seldom knows the set of satisfied INDs in a database. The IND discovery problem in existing databases can be formulated as a data-mining problem. We underline that the exploration of IND expressions from most general (smallest) INDs to most specific (largest) INDs does not succeed whenever large INDs have to be discovered. To cope with this problem, we introduce a new algorithm, called Zigzag, which combines the strength of levelwise algorithms (to find out some smallest INDs) with an optimistic criteria to jump more or less to largest INDs. Preliminary tests, on synthetic databases, are presented and commented on. It is worth noting that the main result is general enough to be applied to other data-mining problems, such as maximal frequent itemsets mining.

Mining maximal frequent itemsets is very important in many data mining applications. How to improve the efficiency and effectiveness of mining algorithm has become an interesting issue in the world. In this paper, we introduce a new method to solve this problem, which is based on graph theory. Firstly, the concept of directed itemsets graph and the trifurcate linked list storage structure are proposed. Secondly, the mining algorithm of maximal frequent itemsets based on this kind of structure is presented and developed. Finally, through some experimental analyses on the standard data sets, the results have demonstrated that the proposed mining algorithm can realize one time database scanning and improve time-storage efficiency a lot.

Frequent itemset mining is a popular and important first step in the analysis of data arising in a broad range of applications. The traditional “exact” model for frequent itemsets requires that every item occur in each supporting transaction. However, real data is typically subject to noise and measurement error. To date, the effect of noise on exact frequent pattern mining algorithms have been addressed primarily through simulation studies, and there has been limited attention to the development of noise tolerant algorithms. In this paper we propose a noise tolerant itemset model, which we call approximate frequent itemsets (AFI). Like frequent itemsets, the AFI model requires that an itemset has a minimum number of supporting transactions. However, the AFI model tolerates a controlled fraction of errors in each item and each supporting transaction. Motivating this model are theoretical results (and a supporting simulation study presented here) which state that, in the presence of even low levels of noise, large frequent itemsets are broken into fragments of logarithmic size; thus the itemsets cannot be recovered by a routine application of frequent itemset mining. By contrast, we provide theoretical results showing that the AFI criterion is well suited to recovery of block structures subject to noise. We developed and implemented an algorithm to mine AFIs that generalizes the level-wise enumeration of frequent itemsets by allowing noise. We propose the noise-tolerant support threshold, a relaxed version of support, which varies with the length of the itemset and the noise threshold. We exhibit an Apriori property that permits the pruning of an itemset if any of its sub-itemset is not sufficiently supported. Several experiments presented demonstrate that the AFI algorithm enables better recoverability of frequent patterns under noisy conditions than existing frequent itemset mining approaches. Noise-tolerant support pruning also renders an order of magnitude performance gain over existing methods.