Multi-dimensional semantic clustering of large databases for association rule mining

Inter-transactional association rules for multi-dimensional contexts for prediction and their application to studying meteorological data

One of the most important data mining problems is mining association rules. In this paper we consider discovering association rules from large transaction databases. The problem of discovering association rules can be decomposed into two sub-problems: find large itemsets and generate association rules from large itemsets. The second sub-problem is easier one and the complexity of discovering association rules is determined by complexity of discovering large itemsets. In this paper, we suggest Apriori-based algorithm for discovering large itemsets. Actually, we suggest a new procedure for large itemsets generation which is more efficient than the appropriate procedure of the original Apriori algorithm. For its implementation, we suggest a modified sort-merge-join algorithm, which is more efficient than nested-loop-join algorithm, which is suggested in the original Apriori algorithm. Besides, we propose a way in which Apriori Multiple finishes in just two iterations.

In this chapter, we present granule based inter association mining to reduce the complexity of intertransaction association rule mining. To compare with other methods, our method can reduce the width of sliding windows. It uses granules to replace extended item sets. Thus, we do not need to consider too many combinations of extended items. We also propose the concept of precision in order to evaluate the effectiveness of intertransaction association rule mining. The experiments show that the proposed method is promising.

Association rule mining (ARM) is a commonly encountred data mining method. There are many approaches to mining frequent rules and patterns from a database and one among them is heuristics. Many heuristic approaches have been proposed but, to the best of our knowledge, there is no comprehensive literature review on such approaches, yet with only a limited attempt. This gap needs to be filled. This paper reviews heuristic approaches to ARM and points out their most significant strengths and weaknesses. We propose eight performance metrics, such as execution time, memory consumption, completeness, and interestingness, we compare approaches against these performance metrics and discuss our findings. For instance, comparison results indicate that SRmining, PMES, Ant‐ARM, and MDS‐H are the fastest heuristic ARM algorithms. HSBO‐TS is the most complete one, while SRmining and ACS require only one database scan. In addition, we propose a parameter, named GT‐Rank for ranking heuristic ARM approaches, and based on that, ARMGA, ASC, and Kua emerge as the best approaches. We also consider ARM algorithms and their characteristics as transactions and items in a transactional database, respectively, and generate association rules that indicate research trends in this area.This article is categorized under: Algorithmic Development > Association Rules Technologies > Association Rules Fundamental Concepts of Data and Knowledge > Motivation and Emergence of Data Mining

In this paper, we examine the issue of mining association rules among items in a large database of sales transactions. The mining of association rules can be mapped into the problem of discovering large itemsets where a large itemset is a group of items which appear in a sufficient number of transactions. The problem of discovering large itemsets can be solved by constructing a candidate set of itemsets first and then, identifying, within this candidate set, those itemsets that meet the large itemset requirement. Generally this is done iteratively for each large k-itemset in increasing order of k where a large k-itemset is a large itemset with k items. To determine large itemsets from a huge number of candidate large itemsets in early iterations is usually the dominating factor for the overall data mining performance. To address this issue, we propose an effective hash-based algorithm for the candidate set generation. Explicitly, the number of candidate 2-itemsets generated by the proposed algorithm is, in orders of magnitude, smaller than that by previous methods, thus resolving the performance bottleneck. Note that the generation of smaller candidate sets enables us to effectively trim the transaction database size at a much earlier stage of the iterations, thereby reducing the computational cost for later iterations significantly. Extensive simulation study is conducted to evaluate performance of the proposed algorithm.

Mining top- k frequent closed itemsets over data streams using the sliding window model

In the recent years, the growth of online marketplace explodes and has become the new norm of shopping. Access to the internet allows consumers to visit local and global online marketplace. By using online shopping, consumers can view the latest products and comparing products' prices. Online marketplace offers flexibility and eases consumers in so many ways. On the contrary, it also has some limitations. Dynamic pricing allows sellers to enhance their marketing strategy by ensuring price competitive with other sellers. The frequent occurrence of real-time price changing limits the user to get the best deal. This paper discusses the analysis of price changes using Association Rule Mining. Price-ChARM finds a frequent pattern of price changes in corresponding to different timelines. On the online marketplace, due to multiple sellers can sell the same item with different offers such as delivery speed and dynamic pricing, the comparison can be quite tricky. We collected data from two well-known portals and we represent the dataset as set of prices for different products for the purpose of frequent itemset mining. We implement the Apriori algorithm and use a total amount of 3,960 records. We generate association rules from the frequent itemsets found in the records and visualize the confidence of 0.9 rules. The association rules represent the pattern of price changes in the portals. This study eases consumer shopping experiences by understanding the trend of price changes to provide a better decision in making a purchase. Â

Data mining aims to develop a new generation of tools to intelligently assist humans in analyzing mountains of data. Over the past few years, great progress has been made in both research and applications of data mining. Data mining systems have helped many businesses by exposing previously unknown patterns in their databases, which were used to improve profits, enhance customer services, and ultimately achieve a competitive advantage. In this paper, we present our unique data mining system DM-II (Data Mining Integration and Interestingness). DM-II is a PC-based system working in Windows 95/98/NT environment. Apart from the normal components of a data mining system, DM-II has a number of unique and advanced sub-systems. These sub-systems have been applied in many real-life applications, including education applications, insurance application, accident application, disease application, drug screening application, image classification, etc. Here, we focus on discussing the following sub-systems: CBA: CBA originally stands for Classification-Based on Associations [lo]. It has now been extended with a number of other advanced features. In all, CBA has the following capabilities: Building classifiers using association rules: Traditionally, association rule mining and classifier building are regarded as two different data mining tasks. CBA unifies the two tasks. It is able to ,generate association rules and builds a classifier using a special subset of the association rules. What is significant is that CBA, in general, produces more accurate classifiers compared to the state-ofthe-art classification system C4.5 [ 161. It also helps to solve some outstanding problems with the existing classification systems. Pruning and summarizing the discovered associations: One major problem with association

The growing of ageing population causes many major negative impacts especially on pension plans and annuity providers. One of the impacts is on the exposure to longevity risk. There are various methods have been previously developed in order to quantify and classify longevity risk. However, there are some major drawbacks to these methods, especially in long-term mortality risk exposures studies. Therefore, this study is conducted in order to observe the potentiality of Association Rules Mining (ARM) in overcoming these drawbacks. The results show that ARM has an advantage of generating a less complex longevity profile based on the generated association rules; and can be used as an alternative method to the intricated statistical methods in profiling longevity risk exposure.

Internet of Things (IoT)-backed smart shopping carts are generating an extensive amount of data in shopping markets around the world. This data can be cleaned and utilized for setting business goals and strategies. Artificial intelligence (AI) methods are used to efficiently extract meaningful patterns or insights from such huge amounts of data or big data. One such technique is Association Rule Mining (ARM) which is used to extract strategic information from the data. The crucial step in ARM is Frequent Itemsets Mining (FIM) followed by association rule generation. The FIM process starts by tuning the support threshold parameter from the user to produce the number of required frequent patterns. To perform the FIM process, the user applies hit and trial methods to rerun the aforesaid routine in order to receive the required number of patterns. The research community has shifted its focus towards the development of top-K most frequent patterns not using the support threshold parameter tuned by the user. Top-K most frequent patterns mining is considered a harder task than user-tuned support-threshold-based FIM. One of the reasons why top-K most frequent patterns mining techniques are computationally intensive is the fact that they produce a large number of candidate itemsets. These methods also do not use any explicit pruning mechanism apart from the internally auto-maintained support threshold parameter. Therefore, we propose an efficient TKIFIs Miner algorithm that uses depth-first search strategy for top-K identical frequent patterns mining. The TKIFIs Miner uses specialized one- and two-itemsets-based pruning techniques for topmost patterns mining. Comparative analysis is performed on special benchmark datasets, for example, Retail with 16,469 items, T40I10D100K and T10I4D100K with 1000 items each, etc. The evaluation results have proven that the TKIFIs Miner is at the top of the line, compared to recently available topmost patterns mining methods not using the support threshold parameter.

This research demonstrates the application of association rule mining to spatio‐temporal data. Association rule mining seeks to discover associations among transactions encoded in a database. An association rule takes the form A→B where A (the antecedent) and B (the consequent) are sets of predicates. A spatio‐temporal association rule occurs when there is a spatio‐temporal relationship in the antecedent or consequent of the rule. As a case study, association rule mining is used to explore the spatial and temporal relationships among a set of variables that characterize socioeconomic and land cover change in the Denver, Colorado, USA region from 1970–1990. Geographic Information Systems (GIS)‐based data pre‐processing is used to integrate diverse data sets, extract spatio‐temporal relationships, classify numeric data into ordinal categories, and encode spatio‐temporal relationship data in tabular format for use by conventional (non‐spatio‐temporal) association rule mining software. Multiple level association rule mining is supported by the development of a hierarchical classification scheme (concept hierarchy) for each variable. Further research in spatio‐temporal association rule mining should address issues of data integration, data classification, the representation and calculation of spatial relationships, and strategies for finding ‘interesting’ rules.

Being aware of the waiting time for selling residential units is one of the important issues in the housing sector for the majority of people, especially investors. There are several factors affecting the waiting time for selling residential units. Determining the influential factors on the time period of selling real estates can lead to an informed decision making by real estate consultants, sellers as well as those seeking to buy real estates. Using a real estate database in Iran, the present paper proposes a two-module procedure. The first module deals with implementation of association rule mining. Using the well-known association rule mining techniques namely FP-Growth, several association rules have been extracted which indicate the effective factors on the waiting time for selling residential units. Generated association rules have been evaluated based on metrics such as support, confidence and lift and finally the best rules are selected. The main objective of the second module is to develop a fuzzy inference system which can determine the factors influencing the waiting time for selling residential units from historical data, so that the model can be used to estimate the time it to sell the property for a real estate agency. Several IF-THEN rules are extracted from this module. Extracted rules can be used by real estate agencies as well as buyers and sellers of residential units to make better decisions in their investments. In conclusion section, a number of suggestions for future studies are presented. For example, machine learning algorithms such as neural networks, decision trees, etc. can also be used to predict the duration of residential units’ sale. The main objective of the second module is to develop a fuzzy inference system which can learn about the factors that influence the waiting time for selling residential units from historical data, so that the model can be used to estimate the time it takes to sell the property for a real estate agency. Several IF-THEN rules are extracted from this module. Extracted rules can be used by real estate agencies as well as buyers and sellers of residential units to make better decisions in their investments.

The Apriori algorithm is a widely used method in association rule learning for mining frequent item sets in a dataset and generating association rules. This research focuses on the application of the Apriori algorithm to a dataset containing the presence or absence of 10 diseases in patients. The aim is to discover patterns and associations between these diseases which could provide insights into disease co-occurrence and help in the development of preventive strategies. The study demonstrates the potential of data mining techniques in the healthcare domain specifically in understanding disease correlations.

Interactions between human and viral proteins have proved to be the major cause of several critical ailments and research directions in the recent past have focused on predicting potential viralhost protein relations through computational techniques. This research aimed at detecting probable interactions between HIV-1 and human proteins by generating all possible high -confident (>80%) associations between the host and viral protein and utilizing the association rules to predict new interactions. The FP-Growth algorithm was analyzed and found to evolve the exhaustive set of high confidence association rules that were mined further to isolate probable and significant HIV1-Human predictions with improved accuracy, sensitivity and specificity compared to previous work. The identified HIV1-Human protein interactions were further investigated using Gene Ontology based and DAVID functional annotation tool to establish their biological and therapeutic merits. The superiority of the proposed approach to previously applied computational techniques has been discussed. AIDS is one of the most dreaded diseases and we believe the proposed approach and the predicted interactions would be instrumental in expediting biological and molecular researchers towards formulating drugs for AIDS therapy and the biological functionality of the predicted interactions would enable timely diagnosis of the presence of the infectious viral protein and its replication in the host. Keywords: AIDS, association rule mining, data mining, FP-growth algorithm, HIV1-human PPI.

Privacy is very important in Data Mining. Association rule mining, is a important technique, has already been applied in a wide range of areas. Large databases cannot be secured by encryption or other techniques because of complexities. These security issues can be divided into two categories: data hiding and knowledge hiding. This paper deals with the problem of association rule mining which preserves the confidentiality of each database i.e. Knowledge hiding. Knowledge hiding is concerned with the sanitization of confidential knowledge from the data. There are many methods to solve this problem. The one, presented in the paper is called support-based and confidence-based blocking schemes. Apriori algorithm is used for mining rule from database. Knowledge hiding is achieved using ISL and DSR. The improvement over sensitive rule hiding is proposed to offer more accuracy and security with smaller negative impact. In ever changing business environment like enterprises e-businesses, the old fashioned disclosure and database inference protection techniques are not capable to ensure complete data privacy. Privacy is important for the on line disclosure of private information Organizations should be able to evaluate the risk of disclosing information and should adopt new more efficient approaches for information disclosure control, in order to maintain their competitive edge in the market. Not only the data but also the hidden knowledge in the data should be made secure. The sensitive information can be extracted in the form of association rules with association rule mining tools. But this can jeopardize the privacy of the customers. Some other rule could be very critical for the company itself such as buying patterns of very rich customers. For example, an Internet based company may give up selling hardware and may concentrate on selling books and videos. Therefore a rule relating the customer buying patterns of hardware may no longer be sensitive for that company. It is desirable to apply a trivial algorithm that hides data by deleting it randomly. Data Mining is the process of discovering new patterns from large data sets involving methods from statistics and artificial intelligence but also database management. Privacy has become an important issue in Data Mining. Many methods have been brought out to solve this problem. This paper focuses on the problem of association rule mining, which hides the knowledge from each database. This paper reviews the major method of knowledge hiding. Association rule mining, as a very important technique, has already been applied in a wide range of areas. Knowledge hiding is concerned with the sanitization of confidential knowledge from the data. As a result of association rule mining, many useful association rules will be discovered, but at the same time, many privacy rules will also be exposed which do not want others to know. To solve this, limit the mining process, in order to keep these sensitive rules being hidden. There are so many methods to solve this problem. The one that is covered in the paper is just one kind of them, called support-based and confidence-based blocking schemes. Algorithm used For Knowledge Hiding is ISL algorithm or DSR algorithm (2). The main disadvantage of a block in algorithm is the fact that the dataset, apart from the blocked values, is not distorted. Thus, an adversary can disclose the hidden rules by identifying those generating item sets that contain question marks and lead to rules with a maximum confidence that lies above the minimum confidence threshold. If the number of these rules is small then the probability of identifying the sensitive ones among them becomes high. To avoid this problem, a method for selectively removing individual values from a database is introduced to prevent the discovery of a set of rules, while preserving the data for other applications. The problem of building privacy preserving algorithms is considered for one category of data mining techniques, the association rule mining. New metrics is introduced in order to demonstrate how security issues can be taken into consideration in the general framework of association rule mining (1).