Probabilistic frequent itemset mining in uncertain databases

Itemset mining is an important subfield of data mining, which consists of discovering interesting and useful patterns in transaction databases. The traditional task of frequent itemset mining is to discover groups of items (itemsets) that appear frequently together in transactions made by customers. Although itemset mining was designed for market basket analysis, it can be viewed more generally as the task of discovering groups of attribute values frequently cooccurring in databases. Because of its numerous applications in domains such as bioinformatics, text mining, product recommendation, e‐learning, and web click stream analysis, itemset mining has become a popular research area. This study provides an up‐to‐date survey that can serve both as an introduction and as a guide to recent advances and opportunities in the field. The problem of frequent itemset mining and its applications are described. Moreover, main approaches and strategies to solve itemset mining problems are presented, as well as their characteristics are provided. Limitations of traditional frequent itemset mining approaches are also highlighted, and extensions of the task of itemset mining are presented such as high‐utility itemset mining, rare itemset mining, fuzzy itemset mining, and uncertain itemset mining. This study also discusses research opportunities and the relationship to other popular pattern mining problems, such as sequential pattern mining, episode mining, subgraph mining, and association rule mining. Main open‐source libraries of itemset mining implementations are also briefly presented.WIREs Data Mining Knowl Discov2017, 7:e1207. doi: 10.1002/widm.1207This article is categorized under:Algorithmic Development > Association RulesTechnologies > Association Rules

Finding frequent itemsets is a popular data mining problem, aiming to extract hidden patterns from a transactional database. Several bio-inspired approaches to solve this problem have been proposed to overcome the poor performance of exact algorithms, such as Apriori and FPGrowth. Approaches based on genetic algorithms are among the most efficient ones from the point of view of runtime performance, but they are still inefficient in terms of solution’s quality, i.e., the number of frequent itemsets discovered. To deal with this issue, we propose in this paper a new genetic algorithm for finding frequent itemsets called GA-Apriori, in which the crossover and mutation operators are defined by taking into account the Apriori heuristic principle. The results of our evaluation show that GA-Apriori outperforms other approaches to frequent itemset mining based on genetic algorithms, especially when dealing with large instances. The experiments also show that GA-Apriori is competitive with exact approaches in terms of the number of frequent itemsets discovered.

Since the era of data explosion, data mining in transactional databases has become more and more important. There are many data mining techniques like association rule mining, the most important and well-researched one. Furthermore, frequent itemset mining is one of the fundamental but time-consuming steps in association rule mining. Most of the algorithms used in literature find frequent itemsets on search space items having at least a minsup and are not reused for mining next time. To deal with this problem, NOV-FI algorithms are proposed as a new approach in order to quickly detect frequent itemsets from transactional databases using an array of co-occurrences and occurrences of kernel item in at least one transaction. NOV-FI algorithms are easily expanded in distributed systems. Finally, the experimental results show that the proposed algorithms perform better than other existing algorithms.

In order to generate synthetic basket datasets for better benchmark testing, it is important to integrate characteristics from real-life databases into the synthetic basket datasets. The characteristics that could be used for this purpose include the frequent itemsets and association rules. The problem of generating synthetic basket datasets from frequent itemsets is generally referred to as inverse frequent itemset mining. In this paper, we show that the problem of approximate inverse frequent itemset mining is NP-complete. Then we propose and analyze an approximate algorithm for approximate inverse frequent itemset mining, and discuss privacy issues related to the synthetic basket dataset. In particular, we propose an approximate algorithm to determine the privacy leakage in a synthetic basket dataset.

Many applications such as intelligent tutoring system (ITS) use data that are better represented as binary data. This paper presents a novel algorithm called MBER (Mining Binary Data Efficiently by Reduced AND operations) for finding frequent itemsets in a binary dataset using matrix algebra operations. Frequent itemsets are sets of items in a transactional database that occur together frequently (defined by a user-given threshold value called minimum support). Existing algorithms that operate on binary data, such as ABBM, generate frequent itemsets by performing exhaustive AND operations using brute force method. MBER, on the other hand, generates frequent itemsets using a novel technique in which it first uses matrix algebra operations to find those transactions that have m common items in them (called as potential transactions) and then performs AND operations on only such potential transactions. This reduces the total number of AND operations required considerably (by less than a quarter) and thereby improves the efficiency of the algorithm. MBER also shows a significant improvement over traditional algorithms that generate frequent itemsets, such as Apriori, by eliminating the need to (i) scan the database more than once and (ii) to generate large number of candidate itemsets. This paper concludes by a proof of correctness of MBER and a discussion on evaluating it.

<p>Frequent item set mining emphases on excavating item sets which follow frequently in a transactional database. These item sets are advantageous in professional decision making. Investigators have projected numerous processes to mine rare item sets, however it is still stimulating. In current studies, to catch rare item sets, customer demarcated single threshold value ought be fixed small adequate which outcomes in generation of enormous quantity of duplicate item sets. The rare item sets may not initiate if a high threshold value is fixed. In this study, an exertion is generated to analyse the rare items for discovery of all probable rare item sets from the transaction database. To mine rare item sets effectually, a method has been anticipated that would permit customer to identify many lowest support that imitates the items and their diverse occurrences in list.<strong> </strong>Investigational results shows that novel methodology is more righteous and vigorous than the present systems.</p>

In this work we focus on the problem of frequent itemset mining on large, out-of-core data sets. After presenting a characterization of existing out-of-core frequent itemset mining algorithms and their drawbacks, we introduce our efficient, highly scalable solution. Presented in the context of the FPGrowth algorithm, our technique involves several novel I/O-conscious optimizations, such as approximate hash-based sorting and blocking, and leverages recent architectural advancements in commodity computers, such as 64-bit processing. We evaluate the proposed optimizations on truly large data sets,up to 75GB, and show they yield greater than a 400-fold execution time improvement. Finally, we discuss the impact of this research in the context of other pattern mining challenges, such as sequence mining and graph mining.

Mining frequent itemsets in databases is an important and widely studied problem in data mining research. The problem of mining frequent itemsets is usually solved by constructing candidates of itemsets, and identifying those itemsets that meet the requirement of frequent itemsets. This paper proposes a novel algorithm based on BitTable (or bitmap) representation of the data. Data - related to frequent itemsets - are stored in spare matrices. Simple matrix and vector multiplications are used to calculate the support of the potential n+1 itemsets. The main benefit of this approach is that only bitmaps of the frequent itemsets are generated. The concept is simple and easily interpretable and it supports a compact and effective implementation (in MATLAB). An application example related to the BMS-WebView-1 benchmark data is presented to illustrate the applicability of the proposed algorithm.

In order to solve these problems how to easily generate candidate frequent item sets and fast compute support of candidate item sets, an algorithm of association rules mining based on binary has been introduced. However, one presented binary mining algorithm is only suitable for mining some relative short frequent item sets since the way of generating candidate item sets is also similar to Apriori, another is only suitable for mining long frequent item sets, which generates candidate item sets by up-down search strategy. And so aiming to mining general frequent item sets, this paper proposes an algorithm of association rules double search mining based on binary, which is different from tradition association rules mining algorithm based on double search strategy. The algorithm doesn’t use combination of set theory to generate candidate item sets but binary logic operation that is also used to compute support of candidate item sets, which can use character digital to reduce the number of scanned transaction. The algorithm gets rid of shortage about some presented algorithms based on binary. The experiment based on above three algorithms indicates that the efficiency about double search strategy is fast and efficient when mining general frequent item sets which aren’t confined.

Mining frequent itemsets is an important research task for knowledge discovery, which is to discover the groups of items appearing always together excess of a user specified threshold from a transaction database. However, there may be many frequent itemsets existing in a transaction database, such that it is difficult to make a decision for a decision maker. Recently, mining frequent closed itemsets becomes a major research issue. The reason is that all frequent itemsets can be derived from frequent closed itemsets. In addition, the transactions in a database will increase constantly. It is a challenge that how to update the previous frequent closed itemsets from the increased transactions. In this paper, we propose an efficient algorithm MRFI for incrementally mining frequent closed itemsets without scanning original database. MRFI algorithm generates frequent closed itemsets by performing some operations on the previous closed itemsets and the added transactions without doing any searching operation. Finally, the experimental results show that MRFI algorithm performs much better than the previous approaches.

Current paper submits an altogether novel method to identify Frequent Itemset which applies approach of statistics further based on association rule mining which otherwise a commonly applied method in transactional database to trace high frequency itemset in data mining arena. The algorithm designs frequency distribution table, Algorithm applies the formulas of class intervals to design frequency distribution table. It calculates the frequency if item sets present in the database. It functions in two phases. Phase 1 is termed as filtering and debulking phase. Phase 2 is termed as application step alongside comparison step among class interval techniques. Threshold values are calculated within the algorithm to find frequent item sets. Threshold numeric is subjected on to the formed table showing frequency distribution and simultaneously products listed are also selected. It generates pairs by applying combination technique. Database is the alone requirement as input. Current paper therefore addresses the worth of identifying and searching frequent item sets in huge transactional database.KeywordsAssociation ruleData miningStatistical techniquesTransactional database

With the rapid progress in information technology, the data mining technique has been exploited in various applications. The association rule(hereafter, AR) mining, one of the most popular data mining techniques, is to find the frequent itemsets which occur commonly in transaction database. Of the various AR algorithms, the Apriori is most popular, and it has been continuously improved during the past decade. Even with recent version, however, it is very time consuming for the Apriori-based algorithms to count frequent itemset since, basically for each k-size item set, we need to compute its support on-the-fly.

Association rules mining is one of the most important problems in knowledge discovery and data mining. The goal of it is to acquire consumption habits of customers by discovering the relationships between items from a transaction database that has a large number of transactions and items. The most compute intensive process for ARM is to find out the frequent 1-itemsets and 2-itemsets. In this paper, we propose a quantum algorithm for finding out the frequent 1-itemsets and 2-itemsets. In our algorithm, to mine the frequent 1-itemsets efficiently, we use the technique of amplitude amplification. To mine the frequent 2-itemsets efficiently, we propose a new tomography scheme, i.e., pure-state-based quantum state tomography. It is shown that our algorithm is potential to achieve exponential speedup in the number of transactions and polynomial speedup in the number of items over the classical algorithm.

This paper presents the design and the implementation of an architecture for the analysis of data streams in distributed environments. In particular, data stream analysis has been carried out for the computation of items and item sets that exceed a frequency threshold. The mining approach is hybrid, that is, frequent items are calculated with a single pass, using a sketch algorithm, while frequent item sets are calculated by a further multi-pass analysis. The architecture combines parallel and distributed processing to keep the pace with the rate of distributed data streams. In order to keep computation close to data, miners are distributed among the domains where data streams are generated. The paper also reports the experimental results obtained with a prototype of the architecture, tested on a Grid composed of two domains handling two different data streams.

DEFINITION Let I be a set of binary-valued attributes, called items. A set X ⊆ I is called an itemset. A transaction database D is a multiset of itemsets, where each itemset, called a transaction, has a unique identifier, called a tid. The support of an itemset X in a dataset D, denoted sup(X), is the fraction of transactions in D where X appears as a subset. X is said to be a frequent itemset in D if sup(X) ≥ minsup, where minsup is a user defined minimum support threshold. An (frequent) itemset is called closed if it has no (frequent) superset having the same support. An association rule is an expression A ⇒ B, where A and B are itemsets, and A ∩ B = ∅. The support of the rule is the joint probability of a transaction containing both A and B, given as sup(A ⇒ B) = P (A ∧ B) = sup(A ∪ B). The confidence of a rule is the conditional probability that a transaction contains B, given that it contains A, given as: conf(A ⇒ B) = P (B|A) = P (A∧B) P (A) = sup(A∪B) sup(A) . A rule is frequent if the itemset A∪B is frequent. A rule is confident if conf ≥ minconf , where minconf is a user-specified minimum threshold. The aim of non-redundant association rule mining is to generate a rule basis, a small, non-redundant set of rules, from which all other association rules can be derived.