A Platform for Parallel Data Mining on Cluster System

Parallel data mining algorithms are extensively used to mine and discover hidden knowledge from varied, unrelated data. Parallel data mining algorithms provide advantages such as reduced training time, less execution time, and less memory requirement. There are several issues in executing parallel data mining algorithms in a distributed environment. It is crucial to partition the data among processors such that there is minimal data dependency, proper synchronization, communication overhead, work load balancing among nodes in distributed processors and disk IO cost. Few of these issues can be resolved when parallel data mining algorithms are executed on Apache framework called Hadoop Map Reduce. Hadoop Map Reduce provides improved performance, reduced communication cost, reduced execution time, reduced training time, and reduced IO access. This paper proposes a novel framework that aims at enhancing the aforementioned advantages in terms of scalability by increasing the number of nodes in the Hadoop cluster and analyzing the performance of classification algorithms like K-Nearest Neighbor, Naive Bayes and Decision Tree. This parallel framework could be extended to other fields of biotechnology where prediction on large datasets is essential.

In information systems in particular business Intelligence systems, for data produced at physically distributed locations most traditional data mining approaches require data to be transmitted to a single location for centralized processing and mining. However, the continual transmission of a large number of data to a central location must be impractical and expensive. Thus, distributed and parallel data mining algorithms and applications were rapidly developed. The paper surveys the-state-of-the art in approaches and applications of distributed computing environment. The goal is to summarize a brief introduction to this field with pointers for further exploration.

Most of the research in parallel data mining and machine learning algorithms is focused on improving the efficiency of existing algorithms. However, our focus is the improvement of the solution quality, or model accuracy. We are looking for "smart" strategies to invest parallel compute resources in order to achieve a better exploration of the search space by exploring several solutions in parallel, referred to as Widening. In this paper, we discuss the theoretical properties of a neighborhood-based Widening using a type of neighborhoods, optimality neighborhoods and contrast this communicationless approach to the straightforward beam-like Top-k Widening approach, which requires communication. We show a bound on the number of parallel workers needed for the communicationless approach to guarantee that it has a solution of the same quality as the Top-k approach. In addition to the theoretical comparison, we experimentally compare these two approaches in terms of running time and quality of final solution, using a widened version of the greedy algorithm for set cover problem.

Sequential pattern mining is applicable in a wide range of applications since many types of data sets are in a time related format. Besides mining sequential patterns in a single dimension, mining multidimensional sequential patterns can give us more informative and useful patterns. Due to the huge increase in data volume and also quite large search space, efficient solutions for finding patterns in multidimensional sequence data are nowadays very important. For this reason, developing a parallel algorithm is necessary. In this paper, we present a multidimensional sequence model and a parallel algorithm follows the level-wise approach and all participating processors or workers generate candidate sequence and count their supports independently. Simulation experiments show good load balancing and scalable and acceptable speedup over different processors and problem sizes. dense data sets such as telecommunications, where there are many and long frequent patterns, performance of these algorithms degrades incredibly. On the one side, parallel and distributed computing is expected to relieve current mining methods from the sequential bottleneck, providing the ability to scale to massive datasets, and improving the response time. Achieving good performance on today's multiprocessor systems is a non-trivial task. The main challenges include synchronization and communication minimization, work-load balancing, find good data layout and data decomposition, and disk I/O minimization, which is especially important for data mining (4). On the other side, the basic single dimension can not satisfy the requirement of multi-attribute analysis, which is often the case in actual system practice. To address this problem, multidimensional sequence pattern mining is developed. The most time consuming operation in the discovery process of sequential patterns is the computation of the frequency of the occurrences of interesting subsequences in the sequence database. However, the number of sequential patterns grows exponentially and the task of finding all sequential patterns requires a lot of computational resources, which make it an ideal candidate for parallel processing. However, sequential pattern mining suffers from a scalability problem in both memory use and computational time when a dataset size is large. To perform mining on large datasets, we propose to parallelize this task. We demonstrate that our parallel algorithm can effectively alleviate the scalability problem. In this paper, we present a model for multidimensional sequence pattern and then propose a parallel algorithm for mining sequential patterns from multidimensional sequence data. The algorithm is based on the data parallelism strategy of pattern growth. The experimental results show that our parallel algorithms usually achieve good load balancing and scalability.

As we are living in the era of big data, huge volumes of a wide variety of complex data-which can be of different levels of veracity-are generated or collected at a high velocity from rich sources of data in various real-life applications. A rich source of these big data sources is the Internet of Things (IoT), which include a collection of sensors, smartphones and other mobile devices, wearable devices, as well as other "things" that are capable to operate within the existing Internet infrastructure. Embedded in these big data are valuable knowledge and useful information. Hence, the research problem of data mining from big IoT data have drawn attention of many researchers as it aims to discover implicit, previously unknown and potentially useful information and knowledge from the data. For instance, frequent pattern mining finds sets of frequently co-occurring items in the IoT domains. Associative classification discovers rules revealing relationships among items within the frequent patterns and their associations with the corresponding class labels. Induction based classification uses decision tree or random forest to learn from old big IoT for classifying or making predictions on new data. Over the past quarter of a century, many serial, distributed, parallel, and MapReduce-based (Hadook-based and Spark-based) big data mining algorithms have been proposed. These algorithms are run in local computers, distributed and parallel environments, clusters, grids, clouds and/or data centers. In this paper, we review some of these algorithms, discuss issues and research prospective in mining classification patterns from these big IoT data in fog. Our case study on a real-life application shows the feasibility on classifying real-life big IoT data over fog for urban analytics.

The goal of data mining algorithm is to discover useful information embedded in large databases. Frequent itemset mining and sequential pattern mining are two important data mining problems with broad applications. Perhaps the most efficient way to solve these problems sequentially is to apply a pattern-growth algorithm, which is a divide-and-conquer algorithm [9, 10]. In this paper, we present a framework for parallel mining frequent itemsets and sequential patterns based on the divide-and-conquer strategy of pattern growth. Then, we discuss the load balancing problem and introduce a sampling technique, called selective sampling, to address this problem. We implemented parallel versions of both frequent itemsets and sequential pattern mining algorithms following our framework. The experimental results show that our parallel algorithms usually achieve excellent speedups.

Parallel system is mainly composed of parallel algorithms which are cost optimal. In this paper a parallel algorithm the hash partitioned apriori (HPA) is taken into consideration. HPA partitions the candidate itemsets among processors using a hash function, like the hash join in relational databases. HPA effectively utilizes the whole memory space of all the processors, hence it works well for large scale data mining in a parallel and distributed environment. The optimization technique of dynamic data allocation is discussed for the execution of this application. This technique is applied in a parallel and distributed environment. Writing parallel data mining algorithms in a distributed environment is a non-trivial task. The main purpose of the proposed method is to meet certain challenges associated with parallel and distributed data mining such as (i) minimizing I/O (ii) Increasing processing speed (iii) Communication cost.

We live in the age of ever-increasing parallel computing resources. For a decade there has been intense research into parallel data mining algorithms. Most of this research is focused on improving the running time of existing algorithms. In contrast our focus is the improvement of the solution quality, or model accuracy. We are looking for smart strategies to invest parallel compute resources in order to achieve a better exploration of the search space by exploring several solutions in parallel, referred to as Widening. In this paper, we demonstrate the effect of neighborhood-based Widening with different types of neighborhoods for the greedy algorithm of the set cover problem.

In this work we describe a portable sequential and parallel algorithm based on Newton's method, for solving nonlinear systems. We used the GMRES iterative method to solve the inner iteration. To control the inner iteration as much as possible and avoid the oversolving problem, we also parallelized several forcing term criterions. We implemented the parallel algorithms using the parallel numerical linear algebra library SCALAPACK based on the MPI environment. Experimental results have been obtained using a cluster of Pentium II PC's connected through a Myrinet network. To test our algorithms we used three different test problems, the H-Chandrasekhar problem, computing the intersection point of several hyper-surfaces, and the Extended Rosenbrock Problem. The latter requires some improvements for the method to work with structured sparse matrices and chaotic techniques. The algorithm obtained shows a good scalability in most cases. This work is included in a framework tool we are developing where, given a problem that implies solving a nonlinear system, the best nonlinear method must be chosen to solve the problem. The method we present here is one of the methods we implemented.

Solution of large dense transportation problems using a parallel primal algorithm


 
 
 
 Apriori algorithm is a classical algorithm of association rule mining and widely used for generating frequent item sets. However, the original Apriori algorithm has some limitation such as it needs to scan the dataset many times to discover all frequent itemset and generate huge number of candidate itemset. To overcome these limitations, researchers have made a lot of improvements to the Apriori such as candidate generation, without candidate generation, transaction reduction, partitioning, and sampling. When it comes to mine massive data, these algorithms failed to prove efficiency because limitation of the processing capacity, storage capacity, and main memory constraints. Therefore, parallel and distributed algorithms are developed to perform large-scale computing in ARM on multiple processors. However, the problems with most of the parallel and distributed framework are overheads of managing distributed system, lack of high level parallel programming language, and node failures. Hadoop-MapReduce is an efficient, scalable, and simplified programming model for massive data processing and it also available on cloud environment. Cloud computing offers huge computing resources, and capacities to solve big data challenges. Recently many parallel algorithms have been proposed on Hadoop-MapReduce to enhance the performance of Apriori algorithm but there are some drawbacks: since multiple scan over the dataset is needed to generate candidate itemset, it consume more execution time. The aim of this study is to propose a parallel Transaction Reduction MapReduce Apriori algorithm (TRMR-Apriori) which is reduce unnecessary transaction values and transactions from the dataset in parallel manner to overcome above problems. The experiments show that TRMR-Apriori is able to achieve better execution time to discover frequent itemset those of previous sequential ARM algorithms such as Apriori, AprioriTid, Eclat, and FP-Growth and the previous parallel algorithms such as PApriori, MRApriori, and Modified Apriori with different condition on homogeneous computing environment using Hadoop-MapReduce platform in cloud. Overall, the TRMR-Apriori shows the strength to extract the frequent itemset from massive dataset in cloud.
 
 
 
 

Summary form only given. This article gives a brief overview of theoretical advances, computing trends, applications and future perspectives in parallel genetic algorithms. It explains basic terms and behavior of (parallel) genetic algorithms. Genetic algorithms are easily parallelized algorithms, therefore two kinds of possible parallelism, data parallelism and control parallelism, are mentioned and described towards them. Parallelism of genetic algorithms brings many advantages and gains. Classifications of these algorithms are often based on the type of computing model, a walk strategy and the used computing machinery. Afterwards significant milestones in the theory with latest advances are briefly mentioned. Then current trends in parallel computing with stress computer architectures of parallel systems, interconnection topologies, operating systems, parallel (genetic) libraries and programming paradigms are reviewed shortly. The sufficient space is devoted to the latest applications of parallel genetic algorithms. After the discussion section, perspectives of the algorithms are predicted till the year 2005. The information in the article is segregated into two periods before and after the year 2000 in all chapters. The second period is more interesting and of higher importance, because it highlights recent research efforts and gives some hints about possible future trends. That is why we devote much space to the second period. As there is no such an overview of the recent period of parallel genetic algorithms, our investigation could be appealing and useful in many aspects.

The sine cosine algorithm’s main idea is the sine and cosine-based vacillation outwards or towards the best solution. The first main contribution of this paper proposes an enhanced version of the SCA algorithm called as ESCA algorithm. The supremacy of the proposed algorithm over a set of state-of-the-art algorithms in terms of solution accuracy and convergence speed will be demonstrated by experimental tests. When these algorithms are transferred to the business sector, they must meet time requirements dependent on the industrial process. If these temporal requirements are not met, an efficient solution is to speed them up by designing parallel algorithms. The second major contribution of this work is the design of several parallel algorithms for efficiently exploiting current multicore processor architectures. First, one-level synchronous and asynchronous parallel ESCA algorithms are designed. They have two favors; retain the proposed algorithm’s behavior and provide excellent parallel performance by combining coarse-grained parallelism with fine-grained parallelism. Moreover, the parallel scalability of the proposed algorithms is further improved by employing a two-level parallel strategy. Indeed, the experimental results suggest that the one-level parallel ESCA algorithms reduce the computing time, on average, by 87.4% and 90.8%, respectively, using 12 physical processing cores. The two-level parallel algorithms provide extra reductions of the computing time by 91.4%, 93.1%, and 94.5% with 16, 20, and 24 processing cores, including physical and logical cores. Comparison analysis is carried out on 30 unconstrained benchmark functions and three challenging engineering design problems. The experimental outcomes show that the proposed ESCA algorithm behaves outstandingly well in terms of exploration and exploitation behaviors, local optima avoidance, and convergence speed toward the optimum. The overall performance of the proposed algorithm is statistically validated using three non-parametric statistical tests, namely Friedman, Friedman aligned, and Quade tests.

The focus of this paper is to propose a dynamic load balancing strategy for parallel association rule mining algorithms in the context of a Grid computing environment. This strategy is built upon a distributed model which necessitates small overheads in the communication costs for load updates and for both data and work transfers. It also supports the heterogeneity of the system and it is fault tolerant.

Most of the hospitals have accumulated a large amount of medical and treatment data after many years of operation. Moreover, data are still being generated in hospitals every day, which contain valuable information. How to extract and fully utilize the value of data from the massive historical hospital data is a key issue for both hospitals and society. With the arrival of the big data and cloud computing era, this issue has attracted considerable attention. In addition, the speed of data mining 404and analysis for large-scale data becomes a hot topic to researchers from both academia and industry. In this chapter, we focus on the optimization methods for parallel data-mining algorithms, and the applications of these algorithms in the field of large-scale hospital data processing. First, techniques of data mining and famous cloud computing platforms are considered. Then, different parallel optimization methods of the related data-mining algorithm are discussed, such as the parallel random forest algorithm based on an Apache Spark platform. Finally, applications of big data processing in hospitals are discussed, based on the optimized data-mining algorithms in the big data and cloud computing environment.