A Pattern Decomposition Algorithm for Data Mining of Frequent Patterns

Frequent pattern mining is a classic data mining technique, generally applicable to a wide range of application domains, and a mature area of research. The fundamental challenge arises from the combinatorial nature of frequent itemsets, scaling exponentially with respect to the number of unique items. Apriori-based and FPTree-based algorithms have dominated the space thus far. Initial phases of this research relied on the Apriori algorithm and utilized a distributed computing environment; we proposed the Cartesian Scheduler to manage Apriori's candidate generation process. To address the limitation of bottom-up frequent pattern mining algorithms such as Apriori and FPGrowth, we propose the Frequent Hierarchical Pattern Tree (FHPTree): a tree structure and new frequent pattern mining paradigm. The classic problem is redefined as frequent hierarchical pattern mining where the goal is to detect frequent maximal pattern covers. Under the proposed paradigm, compressed representations of maximal patterns are mined using a top-down FHPTree traversal, FHPGrowth, which detects large patterns before their subsets, thus yielding significant reductions in computation time. The FHPTree memory footprint is small; the number of nodes in the structure scales linearly with respect to the number of unique items. Additionally, the FHPTree serves as a persistent, dynamic data structure to index frequent patterns and enable efficient searches. When the search space is exponential, efficient targeted mining capabilities are paramount; this is one of the key contributions of the FHPTree. This dissertation will demonstrate the performance of FHPGrowth, achieving a 300x speed up over state-of-the-art maximal pattern mining algorithms and approximately a 2400x speedup when utilizing FHPGrowth in a distributed computing environment. In addition, we allude to future research opportunities, and suggest various modifications to further optimize the FHPTree and FHPGrowth. Moreover, the methods we offer will have an impact on other data mining research areas including contrast set mining as well as spatial and temporal mining.

Mining top- k co-occurrence items with sequential pattern

Closed frequent similar pattern mining: Reducing the number of frequent similar patterns without information loss

Mining of frequent patterns is one of the popular knowledge discovery and data mining (KDD) tasks. It also plays an essential role in the mining of many other patterns such as correlation, sequences, and association rules. Hence, it has been the subject of numerous studies since its introduction. Most of these studies find all the frequent patterns from collection of precise data, in which the items within each datum or transaction are definitely known and precise. However, there are many real-life situations in which the user is interested in only some tiny portions of these frequent patterns. Finding all frequent patterns would then be redundant and waste lots of computation. This calls for constrained mining , which aims to find only those frequent patterns that are interesting to the user. Moreover, there are also many real-life situations in which the data are uncertain. This calls for uncertain data mining . In this article, we propose algorithms to efficiently find constrained frequent patterns from collections of uncertain data .

Extraction of frequent patterns from a transactional database is a fundamental task in data mining. Its applications include association rules, time series, etc. The Apriori approach is a commonly used generate-and-test approach to obtain frequent patterns from a database with a given threshold. Many parallel and distributed methods have been proposed for frequent pattern mining (FPM) to reduce computation time. However, most of them require a Cluster system or Grid system. In this study, a graphic processing unit (GPU) was used to perform FPM with a GPU-FPM to speed-up the process. Because of GPU hardware delimitations, a compact data structure was designed to store an entire database on GPU. In addition, MemPack and CLProgram template classes were also designed. Two datasets with different conditions were used to verify the performance of GPU-FPM. The experimental results showed that the speed-up ratio of GPU-FPM can achieve 14.857 with 16 times of threads.

Frequent pattern mining is a basic problem in data mining and knowledge discovery. The discovered patterns can be used as the input for analyzing association rules, mining sequential patterns, recognizing clusters, and so on. However, discovering frequent patterns in large scale datasets is an extremely time consuming task. Most research in the area of association rule discovery has focused on the method of efficient frequent pattern discovery, e.g. Park, Chen & Yu (1995); Savasere, Omiecinski & Navathe (1995); Han, Pei & Yin (2000); Pei, Han, Lu, Nishio, Tang & Yang (2001). When seeking all associations that satisfy constraints on support and confidence, once frequent patterns have been identified, generating the association rules is trivial. In the last decade, various algorithms have been proposed on this problem, e.g. maximal pattern mining - Grahne & Zhu (2003); closed pattern mining - Pei, Han & Mao (2000); Grahne & Zhu (2003); mining the most interesting frequent patterns - Fu, Kwong & Tang (2000); Han, Wang, Lu & Tzvetkov (2002, 2005); Hirate, Iwahashi & Yamana (2004). However, some challenges are still existed and need to be overcome. In this paper, a mathematical space will be introduced with some new related concepts and propositions to design a new algorithm for solving frequent patterns mining problem. It is hoped that such an improved algorithm will be simple to implement and more efficient.

Data Mining: Mining Frequent Patterns, Associations Rules, and Correlations

Recommender systems are chiefly renowned for their applicability in e-commerce sites and social media. For system optimization, this work introduces a method of behaviour pattern mining to analyze the person's mental stability. With the utilization of the sequential pattern mining algorithm, efficient extraction of frequent patterns from the database is achieved. A candidate sub-sequence generation-and-test method is adopted in conventional sequential mining algorithms like the Generalized Sequential Pattern Algorithm (GSP). However, since this approach will yield a huge candidate set, it is not ideal when a large amount of data is involved from the social media analysis. Since the data is composed of numerous features, all of which may not have any relation with one another, the utilization of feature selection helps remove unrelated features from the data with minimal information loss. In this work, Frequent Pattern (FP) mining operations will employ the Systolic tree. The systolic tree-based reconfigurable architecture will offer various benefits such as high throughput as well as cost-effective performance. The database's frequently occurring item sets can be found by using the FP mining algorithms. Numerous research areas related to machine learning and data mining are fascinated by feature selection since it will enable the classifiers to be swift, more accurate, and cost-effective. Over the last ten years or so, there have been significant technological advancements in heuristic techniques. These techniques are beneficial because they improve the search procedure's efficiency, albeit at the potential sacrifice of completeness claims. A new recommender system for mental illness detection was based on features selected using River Formation Dynamics (RFD), Particle Swarm Optimization (PSO), and hybrid RFD-PSO algorithm is proposed in this paper. The experiments use the depressive patient datasets for evaluation, and the results demonstrate the improved performance of the proposed technique.

A common task in data mining is the visualization of multivariate objects on scatterplots, allowing human observers to perceive subtle inter-relations in the dataset such as outliers, groupings or other regularities. Least- squares multidimensional scaling (MDS) is a well known Exploratory Data Analysis family of techniques that produce dissimilarity or distance preserving layouts in a nonlinear way. In this framework, the issue of visualizing large multidimensional datasets through MDS-based methods is addressed. An original scheme providing very accurate layouts of large datasets is introduced. It is a compromise between the computational complexity O(N5/2) and the accuracy of the solution that makes it suitable both for visualization of fairly large datasets and preprocessing in pattern recognition tasks.

As an important data mining and knowledge discovery task, association rule mining searches for implicit, previously unknown, and potentially useful pieces of information—in the form of rules revealing associative relationships—that are embedded in the data. In general, the association rule mining process comprises two key steps. The first key step, which mines frequent patterns (i.e., frequently occurring sets of items) from data, is more computationally intensive than the second key step of using the mined frequent patterns to form association rules. In the early days, many developed algorithms mined frequent patterns from traditional transaction databases of precise data such as shopping market basket data, in which the contents of databases are known. However, we are living in an uncertain world, in which uncertain data can be found almost everywhere. Hence, in recent years, researchers have paid more attention to frequent pattern mining from probabilistic databases of uncertain data. In this paper, we review recent algorithmic development on mining uncertain data in these probabilistic databases for frequent patterns. © 2011 John Wiley & Sons, Inc. WIREs Data Mining Knowl Discov 2011 1 316–329 DOI: 10.1002/widm.31This article is categorized under: Algorithmic Development > Association Rules

In knowledge management process, data mining technique can be used to extract and discover the valuable and meaningful knowledge from a large amount of data. Nowadays, data mining has given a great deal of concern and attention in the information industry and in society as a whole. This technique is an approach that is currently receiving great attention in data analysis and it has been recognized as a newly emerging analysis tool (Osei-Bryson, 2010; Park, 2001; Sinha, 2008; Tso & Yau, 2007; Wan, 2009; Zanakis, 2005; Zhuang et al., 2009). Additionally, among the major tasks in data mining are classification and prediction; concept description; rule association; cluster analysis; outlier analysis; trend and evaluation analysis; statistical analysis and others. Classification and prediction tasks are among the popular tasks in data mining; and widely used in many areas especially for trend analysis and future planning. In fact, classification technique is supervised learning, which is the class level or prediction target is already known. As a result, the classification model which is represented through rules structures will be constructed in the classification process. In this case, the constructed model will be representing the precious knowledge and it can be used for future planning. There are many areas which adapted this approach to solve their problems such as in finance, medical, marketing, stock, telecommunication, manufacturing, health care, customer relationship and etc. However, the data mining application has not attracted much attention from people in Human Resource (HR) field (Chien & Chen, 2008; Ranjan, 2008). Besides that, in our previous study, most of the prediction applications are used to predict stock, demand, rate, risk, event and others; but there are quite limited studies on human prediction. In addition prediction applications are mainly developed in business and industrious fields; and quite restricted studies involved human talent in an organization (Jantan et al., 2009). HR data can provide a rich resource for knowledge discovery and for decision support system development. Recently, an organization has to struggle effectively in term of cost, quality, service or innovation. All these depend on having enough right people with the right skills, employed

Analysis of Transaction Database Using Apriori Algorithm. This study examines the use of the Apriori algorithm for analysing transaction databases. The Apriori algorithm is a fundamental technique in data mining that allows for the efficient discovery of frequent patterns and association rules in large datasets. The Apriori algorithm employs a two-step approach. Initially, it identifies frequent items in the database based on a user-defined minimum support threshold. Subsequently, it generates association rules that describe relationships between these frequent items based on metrics such as confidence and lift. This paper provides an in-depth explanation of the Apriori algorithm, emphasizing its strengths and limitations. Additionally, it presents various applications of the Apriori algorithm in real-world scenarios, such as shopping cart analysis, cross-selling and upselling, and customer segmentation. The significance of this study lies in its comprehensive analysis of the Apriori algorithm and its practical relevance in diverse data mining tasks. It serves as a valuable resource for researchers, practitioners, and anyone seeking to understand the fundamentals and applications of association rule mining. Keywords: Datamining, OLTP, Apriori algorithm, Item sets and Market Basket Analysis

Big data analysis and mining aims to discover implicit, previously unknown, and potentially useful information and knowledge from big databases that contain high volumes of valuable veracious data collected or generated at a high velocity from a wide variety of data sources. Among different big data mining tasks, this chapter focuses on big data analysis and mining for frequent patterns. By relying on the MapReduce programming model, researchers only need to specify the “map” and “reduce” functions to discover frequent patterns from (1) big databases of precise data in a breadth-first manner or in a depth-first manner and/or from (2) big databases of uncertain data. Such a big data analysis and mining process can be sped up. The resulting (constrained or unconstrained) frequent patterns mined from big databases provide users with new insights and a sound understanding of users' patterns. Such knowledge is useful is many real-life information science and technology applications.

The subject of the paper is methods of classification of data mining and data processing tasks in the economy, as well as their classification characteristics. Methodology. The research used the general scientific methodology of analysis, synthesis, generalization, comparison. Taxonomy methods are used in the compilation of the classification. The selection of actual economic problems is carried out by researching scientific publications on analysis and data processing. The purpose of the article is to compile an actual classification of data mining and data processing tasks in the economy and to refine the terminology of this branch of science. Results. The classification of data mining tasks is proposed, consisting of four levels. All economic tasks of data mining are divided into two large groups: predictive and descriptive. Each group is subdivided into several classes, which combine tasks with similar taxonomic features. These are the classes of tasks classification, regression, clustering, link analysis, and outlier analysis. Classes of tasks are divided into types. An important criterion for this is the dimensionality of the input data representation. It means the number of neighbours for each individual data element. The data can be presented in the form of series, matrices, and graphs. The methods of analysing each form of data presentation vary considerably. The definition of data processing is clarified and a classification of relevant tasks is proposed. At the top level of classification, the data processing tasks are divided into two groups, depending on whether the order of the elements in the input data changes or not. The main classes of data processing tasks are identified, such as ranking, sorting, filtering, cleansing, recovery, and quantization. Practical use. The results of the research can be used to model intellectual decision-support systems. They allow improving the processes of problem formulation in data mining and data processing tasks. Also, it helps in the formalization of the procedures for selecting methods of their solutions. This leads to an increase in economic efficiency.

BackgroundMany studies report about detection and functional characterization of cis-regulatory motifs in untranslated regions (UTRs) of mRNAs but little is known about the nature and functional role of their distribution. To address this issue we have developed a computational approach based on the use of data mining techniques. The idea is that of mining frequent combinations of translation regulatory motifs, since their significant co-occurrences could reveal functional relationships important for the post-transcriptional control of gene expression. The experimentation has been focused on targeted mitochondrial transcripts to elucidate the role of translational control in mitochondrial biogenesis and function.ResultsThe analysis is based on a two-stepped procedure using a sequential pattern mining algorithm. The first step searches for frequent patterns (FPs) of motifs without taking into account their spatial displacement. In the second step, frequent sequential patterns (FSPs) of spaced motifs are generated by taking into account the conservation of spacers between each ordered pair of co-occurring motifs. The algorithm makes no assumption on the relation among motifs and on the number of motifs involved in a pattern. Different FSPs can be found depending on different combinations of two parameters, i.e. the threshold of the minimum percentage of sequences supporting the pattern, and the granularity of spacer discretization. Results can be retrieved at the UTRminer web site: . The discovered FPs of motifs amount to 216 in the overall dataset and to 140 in the human subset. For each FP, the system provides information on the discovered FSPs, if any. A variety of search options help users in browsing the web resource. The list of sequence IDs supporting each pattern can be used for the retrieval of information from the UTRminer database.ConclusionComputational prediction of structural properties of regulatory sequences is not trivial. The presented data mining approach is able to overcome some limits observed in other competitive tools. Preliminary results on UTR sequences from nuclear transcripts targeting mitochondria are promising and lead us to be confident on the effectiveness of the approach for future developments.