HybridMiner: Mining Maximal Frequent Itemsets Using Hybrid Database Representation Approach

Discovering frequent itemsets is a key problem in important data mining applications, such as the discovery of association rules, strong rules, episodes, and minimal keys. Typical algorithms for solving this problem operate in a bottom-up breadth-first search direction. The computation starts from frequent 1-itemsets (minimal length frequent itemsets) and continues until all maximal (length) frequent itemsets are found. During the execution, every frequent itemset is explicitly considered. Such algorithms perform reasonably well when all maximal frequent itemsets are short. However, performance drastically decreases when some of the maximal frequent itemsets are relatively long. We present a new algorithm which combines both the bottom-up and top-down directions. The main search direction is still bottom-up but a restricted search is conducted in the top-down direction. This search is used only for maintaining and updating a new data structure we designed, the maximum frequent candidate set. It is used to prune candidates in the bottom-up search. As a very important characteristic of the algorithm, it is not necessary to explicitly examine every frequent itemset. Therefore it performs well even when some maximal frequent itemsets are long. As its output, the algorithm produces the maximum frequent set, i.e., the set containing all maximal frequent itemsets, which therefore specifies immediately all frequent itemsets. We evaluate the performance of the algorithm using a well-known benchmark database. The improvements can be up to several orders of magnitude, compared to the best current algorithms.

The association rule mining can be divided into two steps. The first step is to find out all frequent itemsets, whose occurrences are greater than or equal to the user-specified threshold. The second step is to generate reliable association rules based on all frequent itemsets found in the first step. Identifying all frequent itemsets in a large database dominates the overall performance in the association rule mining. In this paper, we propose an efficient hash-based method, HMFS, for discovering the maximal frequent itemsets. The HMFS method combines the advantages of both the DHP (Direct Hashing and Pruning) and the Pincer-Search algorithms. The combination leads to two advantages. First, the HMFS method, in general, can reduce the number of database scans. Second, the HMFS can filter the infrequent candidate itemsets and can use the filtered itemsets to find the maximal frequent itemsets. These two advantages can reduce the overall computing time of finding the maximal frequent itemsets. In addition, the HMFS method also provides an efficient mechanism to construct the maximal frequent candidate itemsets to reduce the search space. We have implemented the HMFS method along with the DHP and the Pincer-Search algorithms on a Pentium III 800 MHz PC. The experimental results show that the HMFS method has better performance than the DHP and the Pincer-Search algorithms for most of test cases. In particular, our method has significant improvement over the DHP and the Pincer-Search algorithms when the size of a database is large and the length of the longest itemset is relatively long.

Maximal frequent item sets are one of several condensed representations of frequent item sets, which store most of the information contained in frequent item sets using less space, thus being more suitable for stream mining. This paper focuses on mining maximal frequent item sets approximately over a stream landmark model. We separate the continuously arriving transactions into sections, and the mining results are indexed by an extended direct update tree, thus, a simple but effective algorithm named SMIS is proposed. In our algorithm, we employ the Chern off Bound to perform the maximal frequent item set mining in a false negative manner, which can reduce the memory cost, as well guarantee our algorithm conducting with an incremental fashion. Our experimental results on two synthetic datasets and two real world datasets show that SMIS achieves much reduced memory cost in comparison with the state-of-the-art algorithm with a 100 percent precision.

When it comes to association rule mining, all frequent itemsets are first found, and then the confidence level of association rules is calculated through the support degree of frequent itemsets. As all non-empty subsets in frequent itemsets are still frequent itemsets, all frequent itemsets can be acquired only by finding all maximal frequent itemsets (MFIs), whose supersets are not frequent itemsets. In this study, an algorithm, named right-hand side expanding (RHSE), which can accurately find all MFIs, was proposed. First, an Expanding Operation was designed, which, starting from any given frequent itemset, could add items using certain rules and form some supersets of given frequent itemsets. In addition, these supersets were all MFIs. Next, this operator was used to add items by taking all frequent 1-itemsets as the starting point alternately, and all MFIs were found in the end. Due to the special design of the Expanding Operation, each MFI could be found. Moreover, the path found was unique, which avoided the algorithm redundancy in temporal and spatial complexity. This algorithm, which has a high operating rate, is applicable to the big data of high-dimensional mass transactions as it is capable of avoiding the computing redundancy and finding all MFIs. In the end, a detailed experimental report on 10 open standard transaction sets was given in this study, including the big data calculation results of million-class transactions.

In the discipline of data mining, association rule mining is an important study topic that focuses on discovering the relationships between database attributes. The maximum frequent itemset comprises the information of all frequent itemsets, which is one of the important difficulties in mining association rules, and certain data mining applications just need to mine the maximum frequent itemsets. As a result, analyzing the maximum frequent itemset mining technique is practical. Considering this, the research introduces FP-MFIA, a new maximum frequent itemset mining approach based on the FP-tree, which is inspired by the data structure of the frequent pattern tree and the idea that the maximum frequent itemset implies all frequent itemsets. First, the FP-MFIA constructs a one-way FP-tree structure, which only has pointers from the root to the leaves, so that only two scans of the FP-tree are required by the FP-MFIA. On the other hand, it redefines a data storage structure MFI-list for maximum frequent itemsets. It can quickly release unnecessary nodes in the FP-tree after scanning it. In this way, not only the information required by the maximum frequent itemsets can be quickly mined but also the space required for storing the maximum frequent itemsets can be reduced, which greatly improves the mining efficiency. Finally, experiments were conducted to compare the mining efficiency of the novel FP-MFIA algorithm to the IDMFIA and DMFIA algorithms. We can see from the findings that the FP-MFIA algorithm is more efficient than the other two techniques.

Frequent itemsets (FIs) mining from uncertain database is a very popular research area nowadays. Many algorithms have been proposed to mine FI from uncertain database. But in typical FI mining process, all the FIs have to be mined individually, which needs a huge memory. Four trees are proposed in this paper which are: (i) maximal frequent itemset from uncertain database (MFU) tree which contains only the maximal frequent itemsets generated from uncertain database, (ii) closed frequent itemset from uncertain database (CFU) tree which contains only closed frequent itemsets generated from uncertain database, (iii) maximal frequent itemset from uncertain data stream (MFUS) tree which contains maximal frequent itemsets generated from uncertain data stream and (iv) closed frequent itemset from uncertain data stream (CFUS) tree which contains closed frequent itemsets generated from uncertain data stream. Experimental results are also presented which show that maximal and closed frequent itemsets mining requires less time and memory than typical frequent itemsets mining.

This paper addressed the insecurity and the inefficiency of privacy preserving association rule mining in vertically partitioned data. We presented a privacy preserving maximal frequent itemsets mining algorithm in vertically partitioned data. The algorithm adopted a more secure vector dot protocol which used an inverse matrix to hide the original input vector, and without any site revealing privacy vector. The mining strategy was based on depth-first search for the maximal frequent itemsets. Performance analysis and experimental analysis showed that the algorithm possessed higher security and efficiency.

Association rule has been an area of active research in the field of knowledge discovery. Data mining researchers had improved upon the quality of association rule mining for business development by incorporating influential factors like value (utility), quantity of items sold (weight) and more for the mining of association patterns. In this paper, we propose an efficient approach to find maximal frequent item set first. Most of the algorithms in literature used to find minimal frequent item first, then with the help of minimal frequent item sets derive the maximal frequent item sets. These methods consume more time to find maximal frequent item sets. To overcome this problem, we propose a navel approach to find maximal frequent item set directly using the concepts of subsets. The proposed method is found to be efficient in finding maximal frequent item sets.

Uncertain data are inherent in some important applications. Although a considerable amount of research has been dedicated to modeling uncertain data and answering some types of queries on uncertain data, how to conduct advanced analysis on uncertain data remains an open problem at large. In this paper, we tackle the problem of skyline analysis on uncertain data. We propose a novel probabilistic skyline model where an uncertain object may take a probability to be in the skyline, and a p-skyline contains all objects whose skyline probabilities are at least p (0?<?p???1). Computing probabilistic skylines on large uncertain data sets is challenging. We develop a bounding-pruning-refining framework and three algorithms systematically. The bottom-up algorithm computes the skyline probabilities of some selected instances of uncertain objects, and uses those instances to prune other instances and uncertain objects effectively. The top-down algorithm recursively partitions the instances of uncertain objects into subsets, and prunes subsets and objects aggressively. Combining the advantages of the bottom-up algorithm and the top-down algorithm, we develop a hybrid algorithm to further improve the performance. Our experimental results on both the real NBA player data set and the benchmark synthetic data sets show that probabilistic skylines are interesting and useful, and our algorithms are efficient on large data sets.

The proposed algorithm, used for mining maximal frequent itemsets, finds interesting correlations between data in an efficient manner and can be helpful in case of large databases where large frequent itemsets contain large number of attributes. The algorithm used, works efficiently when the number of items in the large frequent itemsets is close to the number of total attributes in the dataset, or if the number of items in the large frequent itemsets is predetermined. The salient feature of the proposed approach is that it always automatically satisfies downward closure property, which states that all subsets of a frequent set must also be frequent. Owing to bit-set representation of dataset and utilization of unique list structure, the process of counting the occurrences of itemsets becomes faster, which ultimately enhances the overall mining performance.

There is no reasonable scientific basis for selecting the excellent teachers of the school’s courses. To solve the practical problem, we firstly give a series of normalization models for defining the key attributes of teachers’ professional foundation, course difficulty coefficient, and comprehensive evaluation of teaching. Then, we define a partial weight function to calculate the key attributes, and obtain the partial recommendation values. Next, we construct a highly sparse Teaching Recommendation Factorization Machines (TRFMs) model, which takes the 5-tuples relation including teacher, course, teachers’ professional foundation, course difficulty, teaching evaluation as the feature vector, and take partial recommendation value as the recommendation label. Finally, we design a novel Top-N excellent teacher recommendation algorithm based on TRFMs by course classification on the highly sparse dataset. Experimental results show that the proposed TRFMs and recommendation algorithm can accurately realize the recommendation of excellent teachers on a highly sparse historical teaching dataset. The recommendation accuracy is superior to that of the three-dimensional tensor decomposition model algorithm which also solves sparse datasets. The proposed method can be used as a new recommendation method applied to the teaching arrangements in all kinds of schools, which can effectively improve the teaching quality.

Data mining is the practice to search large amount of data to discover data patterns.Data mining uses mathematical algorithms to group the data and evaluate the future events.Association rule is a research area in the field of knowledge discovery.Many data mining researchers had improved upon the quality of association rule for business development by incorporating influential factors like utility, number of items sold and for the mining of association data patterns.In this paper, we propose an efficient algorithm to find maximal frequent itemset first.Most of the association rule algorithms used to find minimal frequent item first, then with the help of minimal frequent itemsets derive the maximal frequent itemsets, these methods consume more time to find maximal frequent itemsets.To overcome this problem, we propose a new approach to find maximal frequent itemset directly using the concepts of subsets.The proposed method is found to be efficient in finding maximal frequent itemsets.

Introduction: Recommendation systems (RS) are pivotal in enhancing personalized user experiences, particularly in the educational domain, where tailored book recommendation systems (BRS) support individualized learning. Despite their importance, selecting an optimal recommendation algorithm remains challenging, especially in scenarios involving large and sparse datasets. Objectives: This study aims to evaluate the performance of four recommendation algorithms—Random Baseline, Popular, User-Based Collaborative Filtering (UBCF), and Singular Value Decomposition (SVD)—to identify their strengths and tradeoffs regarding accuracy and computational efficiency. The goal is to provide actionable insights for selecting appropriate algorithms for educational platforms and similar applications. Methods: The evaluation employed a dataset containing over 10,000 books, 53,424 users, and 981,756 ratings. The algorithms were assessed using Root Mean Square Error (RMSE) to measure predictive accuracy and training time to indicate computational efficiency. The analysis addressed challenges such as data sparsity and rating biases. Results: The results demonstrated that SVD achieved the highest accuracy (RMSE: 0.950), effectively uncovering latent relationships in sparse datasets. UBCF, with a slightly lower accuracy (RMSE: 1.020), offered a balance between accuracy and computational efficiency, making it suitable for real-time applications. Conversely, simpler algorithms like Random Baseline and Popular exhibited faster training times but significantly lower predictive accuracy, highlighting the limitations of non-personalized methods. Conclusions: This study underscores the tradeoffs between accuracy and efficiency among recommendation algorithms. While SVD is ideal for accuracy-driven applications, UBCF provides a practical alternative for scenarios requiring computational efficiency. The findings have substantial implications for educational platforms and e-commerce, where personalized recommendations enhance user satisfaction and engagement. Future research should focus on integrating deep learning models and expanding evaluation criteria, including user satisfaction and diversity, to further improve the performance of recommendation systems.

Large and sparse datasets with a lot of missing values are common in the big data era, such as user behaviors over a large number of items. Classification in such datasets is an important topic for machine learning and data mining. Practically, naive Bayes is still a popular classification algorithm for large sparse datasets, as its time and space complexity scales linearly with the size of non-missing values. However, several important questions about the behavior of naive Bayes are yet to be answered. For example, how different mechanisms of data missing, data sparsity, and the number of attributes systematically affect the learning curves and convergence? In this paper, we address several common data missing mechanisms and propose novel data generation methods based on these mechanisms. We generate large and sparse data systematically, and study the entire AUC (Area Under ROC Curve) learning curve and convergence behavior of naive Bayes. We not only have several important experiment observations, but also provide detailed theoretic studies. Finally, we summarize our empirical and theoretic results as an intuitive decision flowchart and a useful guideline for classifying large sparse datasets in practice.

Temperature profoundly affects ecology, a fact ever more evident as the ability to measure thermal environments increases and global changes alter these environments. The spatial structure of thermalscapes is especially relevant to the distribution and abundance of ectothermic organisms, but the ability to describe biothermal relationships at extents and grains relevant to conservation planning has been limited by small or sparse data sets. Here, we combine a large occurrence database of >23 000 aquatic species surveys with stream microclimate scenarios supported by an equally large temperature database for a 149 000-km mountain stream network to describe thermal relationships for 14 fish and amphibian species. Species occurrence probabilities peaked across a wide range of temperatures (7.0-18.8°C) but distinct warm- or cold-edge distribution boundaries were apparent for all species and represented environments where populations may be most sensitive to thermal changes. Warm-edge boundary temperatures for a native species of conservation concern were used with geospatial data sets and a habitat occupancy model to highlight subsets of the network where conservation measures could benefit local populations by maintaining cool temperatures. Linking that strategic approach to local estimates of habitat impairment remains a key challenge but is also an opportunity to build relationships and develop synergies between the research, management, and regulatory communities. As with any data mining or species distribution modeling exercise, care is required in analysis and interpretation of results, but the use of large biological data sets with accurate microclimate scenarios can provide valuable information about the thermal ecology of many ectotherms and a spatially explicit way of guiding conservation investments.