Data mining and modeling in scientific databases

Recent advances in computing, communications, digital storage technologies, and high-throughput data-acquisition technologies, make it possible to gather and store incredible volumes of data. It creates unprecedented opportunities for large-scale knowledge discovery from huge database. Data mining (DM) technology has emerged as a means of performing this discovery. There are countless researchers working on designing efficient data mining techniques, methods, and algorithms. Many data mining methods and algorithms have been developed and applied in a lot of application fields [1]. Unfortunately, most data mining researchers pay much attention to technique problems for developing data mining models and methods, while little to basic issues of data mining [2]. In this talk, some basic issues of data mining are addressed. What is data mining? What is the product of a data mining process? What are we doing in a data mining process? What is the rule we should obey in a data mining process? Through analyzing existing data mining methods, and domain-driven (or user-driven) data mining models [3-5], we find that we should take a data mining process as a process of knowledge transformation. Based on this understanding of data mining, a conceptual data mining model of domain-oriented data-driven data mining (3DM) is proposed [2]. The relationship between traditional domain-driven (or user-driven) data mining models and our proposed 3DM model is also analyzed. Some domain-oriented data-driven data mining algorithms for mining such knowledge as default rule [6], decision tree [7], and concept lattice [8] from database are proposed. The experiment results for these algorithms are also shown to illustrate the efficiency and performance of the knowledge acquired by our 3DM data mining algorithms.

This paper describes an approach to identify the state of a power system using Data Mining Algorithms. With the advent of Phasor Measurement Units (PMU), a huge amount of power system data is being stored day-to-day in the Phasor Data Collectors (PDCs). Knowledge discovery and machine learning techniques can make use of this data to extract valuable information and interesting patterns in these databases. In this paper, Symbolic Aggregate ApproXimation (SAX) is used to convert the time stamped phasor data from the PMUs into symbolic strings and Data Mining (DM) algorithms are used to predict the current state of a power system. Along with the state, the location and cause of disturbance is also identified in minimal time. The effectiveness of different DM algorithms for determining state of a power system is also shown using the measures of accuracy, precision and recall.

Preparing a data set for analysis is generally the most time consuming task in a data mining project, requiring many complex SQL queries, joining tables and aggregating columns. Existing SQL aggregations have limitations to prepare data sets because they return one column per aggregated group. In general, a significant manual effort is required to build data sets, where a horizontal layout is required. We propose simple, yet powerful, methods to generate SQL code to return aggregated columns in a horizontal tabular layout, returning a set of numbers instead of one number per row. This new class of functions is called horizontal aggregations. Horizontal aggregations build data sets with a horizontal denormalized layout (e.g. point-dimension, observation-variable, instance-feature), which is the standard layout required by most data mining algorithms. We propose three fundamental methods to evaluate horizontal aggregations: CASE: Exploiting the programming CASE construct; SPJ: Based on standard relational algebra operators (SPJ queries); PIVOT: Using the PIVOT operator, which is offered by some DBMSs. Experiments with large tables compare the proposed query evaluation methods. Our CASE method has similar speed to the PIVOT operator and it is much faster than the SPJ method. In general, the CASE and PIVOT methods exhibit linear scalability, where as the SPJ method does not. In a relational database, especially with normalized tables, a significant effort is required to prepare a summary data set that can be used as input for a data mining or statistical algorithm. Most algorithms require as input a data set with a horizontal layout, with several Records and one variable or dimension per column. That is the case with models like clustering, classification, regression and PCA; consult. Each research discipline uses different terminology to describe the data set. In data mining the common terms are point-dimension. Statistics literature generally uses observation-variable. Machine learning research uses instance-feature. This article introduces a new class of aggregate functions that can be used to build data sets in a horizontal layout (denormalized with aggregations), automating SQL query writing and extending SQL capabilities. We show evaluating horizontal aggregations is a challenging and interesting problem and we introduced alternative methods and optimizations for their efficient evaluation. II. MOTIVATION As mentioned above, building a suitable data set for data mining purposes is a time- consuming task. This task generally requires writing long SQL statements or customizing SQL Code if it is automatically generated by some tool. There are two main ingredients in such SQL code: joins and aggregations; we focus on the second one. The most widely- known aggregation is the sum of a column over groups of rows. Some other aggregations return the average, maximum, minimum or row count over groups of rows. There exist many aggregations functions and operators in SQL. Unfortunately, all these aggregations have limitations to build data sets for data mining purposes. The main reason is that, in general, data sets that are stored in a relational database (or a data warehouse) come

Use of machine learning (ML) and data mining (DM) algorithms has surfaced more often in the recent years for disease risk prediction problems in the healthcare communities. Several traditional feature selection models are combined with the DM and ML algorithms to improve accuracy of the disease risk prediction. In this study, a new Bio-inspired Ensemble Feature Selection (BEFS) model is introduced which is applied with the DM and ML algorithms. In the BEFS model, the most relevant and highly contributing features in the prediction are determined with a bio-inspired algorithm i.e., genetic algorithm, and an ensemble algorithm i.e., random forest algorithm. These important features obtained from the proposed model are then combined in various combinations and applied with the DM and ML algorithms, here logistic regression (LR) and random forest (RF), and the results obtained are promising. The experiment is executed using the famous ML language R. To accomplish this objective, the Breast Cancer Wisconsin (Diagnostic) dataset of UCI (University of California, Irvine) ML repository is utilized. In the experimental outcomes, the highest accuracy attained with the BEFS model is 96.49%, the AUC (Area Under Curve) achieved is 96%, and the sensitivity is 98.11%. These results, which greatly improve the disease risk prediction, are higher than several other existing works, while utilizing only six most relevant features out of the thirty two features of the dataset.

This paper presents a hybrid approach that integrates a genetic algorithm (GA) and data mining to produce control signatures. The control signatures define the best parameter intervals leading to a desired outcome. This hybrid method integrates multiple rule sets generated by a data mining algorithm with the fitness function of a GA. The solutions of the GA represent intersections among rules providing tight parameter bounds. The integration of intuitive rules provides an explanation for each generated control setting and it provides insights into the decision making process. The ability to analyze parameter trends and the feasible solutions generated by the GA with respect to the outcomes is another benefit of the proposed hybrid method. The presented approach for deriving control signatures is applicable to various domains, such as energy, medical protocols, manufacturing, airline operations, customer service, and so on. Control signatures were developed and tested for control of a power plant boiler. These signatures discovered insightful relationships among parameters. The results and benefits of the proposed method for the power plant boiler are discussed in the paper.

This paper reviews the progress of the research on spatial data interoperability and several methods currently used to achieving spatial data interoperability, puts the emphasis on analyzing the advantages, as well as the handicaps and existing conditions of interoperability based on shared database and the discrepancy of data models specific to different GIS, which is a main obstacle to spatial data interoperability, and brings forward two methods for achieving spatial data interoperability based on shared spatial database: one is realizing spatial data interoperability among different clients by using the same set of data model in database, the other is interoperability among clients through objects translation in a shared database. The paper also makes a comparison between these two methods in feasibility and difficulty of carrying out. Based on the analysis, this paper makes an experiment with Oracle Spatial in a local network using the second method, the emphasis of the experiment is put on the interoperability of oriented point objects and text, or annotation objects, as the two types of objects have no standard data model in Oracle Spatial and data models brought forward by different GIS vendor used to describe these two types of objects are specific to their GIS software. Two GIS software, MapInfo and Geomedia, are used as client in the experiment. The experiment shows that this method to achieve spatial data interoperability is practical and could be used inside an organization by employees in different departments, who are familiar with different software in their work, to obtain data sharing and interoperability. This method can also lower data redundancy and improve work efficiency.

With the huge amount of information available online, the World Wide Web is a fertile area for data mining research. The data mining research is at the cross road of research from several research communities, such as database, information retrieval, and within AI, especially the sub-areas of machine learning and data integrity. Every E-commerce and social website in World Wide Web uses the Classification is one of the data mining problems receiving great attention recently in the database community. Neural network is not suitable for data mining directly, because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. Different concise symbolic rules with high accuracy can be extracted from a neural network with the proposed approach. The neural network is first trained to achieve the required accuracy in data mining. In this paper we are going to combine neural network with the three different algorithms which are commonly used in data mining to improve the data mining result. These three algorithms are CHARM Algorithm, Top K Rules mining and CM SPAM Algorithm. The different datasets of online e-commerce website filpkart and Amazon are used to train the neural network and to use in data mining. The results of all three data mining algorithm with neural network techniques then tested on the available datasets and result are compared by computational complexity of the algorithm.

A fuzzy logic based resource manager (RM) has been developed that automatically allocates electronic attack resources in real-time over many dissimilar platforms. Two different data mining algorithms have been developed to determine rules, strategies, and fuzzy decision tree structure. The first data mining algorithm uses a genetic algorithm as a data mining function and is called from an electronic game. The game allows a human expert to play against the resource manager in a simulated battlespace with each of the defending platforms being exclusively directed by the fuzzy resource manager and the attacking platforms being controlled by the human expert or operating autonomously under their own logic. This approach automates the data mining problem. The game automatically creates a database reflecting the domain expert's knowledge. It calls a data mining function, a genetic algorithm, for data mining of the database as required and allows easy evaluation of the information mined in the second step. The criterion for re- optimization is discussed as well as experimental results. Then a second data mining algorithm that uses a genetic program as a data mining function is introduced to automatically discover fuzzy decision tree structures. Finally, a fuzzy decision tree generated through this process is discussed.

SummaryOne of the most important challenges of African countries is the effective use of information and communication technologies and its generalization in the different sectors (education, economic, political, and so on). This will have a great impact on different aspects of society and economic activities by making everyday procedures easier and more efficient. In this same context, we are interested in this article by the proposition of a decision support framework based on the use of data mining (DM) techniques. Using DM in e‐government is the process of translating data to appropriate knowledge which can be useful for decision‐making. We propose a framework whose goal is the generation of association rules (ARs) for better decision‐making. This framework includes two approaches: (1) the first approach applies different DM algorithms and (2) the second approach optimizes the first one by considering two different metaheuristics: the Genetic algorithm and the Cuckoo search algorithm. A new relevance measure called “Weighted Dominance” has been considered to evaluate the quality of the generated ARs. Extensive experiments have been conducted using different datasets. The results obtained demonstrated the effectiveness of combining DM and optimization algorithms compared to the first approach. Finally, a case study related to an Algerian mobile phone company has been presented illustrating the use of our framework in the decision‐making process.

The article is devoted to a research of an efficiency of high-level synthesis approach, based on Xilinx’s high-level synthesis tool - Vivado, for a hardware implementation of sorting algorithms, which are one of the key algorithms for Big Data analysis, Data Mining, Data and Management. Performance and hardware costs are the measures of the efficiency in the provided research. The research methods are simulation and comparative analysis. Efficiency of software implementation of the selected sorting algorithms, based on a universal processor, is compared with efficiency of hardware implementation of the same sorting algorithms, obtained by high-level synthesis procedure with help of Xilinx’s high-level synthesis tool. The article discusses approaches to optimize the description of the sorting algorithms and assignments in boundaries of high-level synthesis procedure to achieve optimal efficiency of the final hardware solutions. The article shows that the main efficiency gain is determinate by the internal features of the sorting algorithm, selected for hardware implementation; the ability to parallelize the processing of the source arrays, which is achieved both by the settings of the Vivado synthesis tool and description style used for source code. Article highlights research results and provide a direction for the future research works.

Data mining algorithms are often embedded in more complex systems, serving as the provider of data for internal decision making within these systems. In this paper we address an interesting problem of using data mining techniques for database query optimization. We introduce the concept of conditional cardinality patterns and design an algorithm to compute the required values for a given database schema. However applicable to any database system, our solution is best suited for data warehouse environments due to the special characteristics of both database schemata being used and queries being asked. We verify our proposal experimentally by running our algorithm against the state-of-the-art database query optimizer. The results of conducted experiments show that our algorithm outperforms traditional cost-based query optimizer with respect to the accuracy of cardinality estimation for a wide range of queries.KeywordsData WarehouseDatabase SchemaQuery OptimizationData Mining AlgorithmFact TableThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Missing values in datasets are caused by various reasons. These missing values in datasets adversely impact the performance of data mining (DM) algorithms. These values can be ignored if there are large number of instances in the dataset. However, deleting the records containing missing values in smaller sized datasets can lead to improper classification or predictions by the data mining algorithms. Several methods for finding missing values are present. Commonly used methods like replacing the missing values by the mean of the column, repeated value, are present. However, these techniques don’t approximately forecast the missing values. In our work, we have considered regression-based methodology for predicting the missing values. Results on stock dataset demonstrated that the polynomial regression (PR) model exhibited better performance when compared to linear regression (LR), quadratic regression (QR), pure-quadratic regression (PQR) and interactions regression (IR) models for prediction of missing values.KeywordsCrop yield predictionData miningDecision tree

The subfield of technologies within the discipline of computer science is known as "data mining" Moreover, statistics are put to use in order to identify trends within the database. Essentially, the primary goal of the data mining methods is to extract from the database any information that will be valuable in the future or that will be needed at some point in the future and transform it into a usable format. Understandable framework for future usage. There are several approaches to choose from, each of which may be helpful in carrying out data mining operations in an effective manner. There is some information accessible in this document concerning comparative studies or analyses of various data mining approaches and certain data mining algorithms. This information may be found in this publication. This means that reading this paper will assist us in gaining knowledge regarding data mining technologies, data mining algorithms, as well as some data of corporations that have upgraded or enhanced their profits by modifying their data mining technologies and have obtained impressive output as a result of doing so.

Recent developments in computing, communications, digital storage technologies, and high-throughput data-acquisition technologies, make it possible to gather and store incredible volumes of data. It creates unprecedented opportunities for knowledge discovery large-scale database. Data mining technology is a useful tool for this task. It is an emerging area of computational intelligence that offers new theories, techniques, and tools for processing large volumes of data, such as data analysis, decision making, etc. There are countless researchers working on designing efficient data mining techniques, methods, and algorithms. Unfortunately,most data mining researchers pay much attention to technique problems for developing data mining models and methods, while little to basic issues of data mining. What is data mining? What is the product of a data mining process? What are we doing in a data mining process? What is the rule we would obey in a data mining process? What is the relationship between the prior knowledge of domain experts and the knowledgemind from data? In this paper, we will address these basic issues of data mining from the viewpoint of informatics [1]. Data is taken as a manmade format for encoding knowledge about the natural world. We take data mining as a process of knowledge transformation. A domain-oriented data-driven data mining (3DM) model based on a conceptual data mining model is proposed. Some data-driven data mining algorithms are also proposed to show the validity of this model, e.g., the data-driven default rule generation algorithm, data-driven decision tree pre-pruning algorithm and data-driven knowledge acquisition from concept lattice.

Computer-assisted data mining algorithms (DMAs) are being studied to screen spontaneous reporting databases for signals of novel adverse events. The performance characteristics and optimum deployment of these techniques remain to be established. To explore issues in the practical evaluation and deployment of DMAs by comparing findings from an empirical Bayesian DMA with those from a traditional drug safety surveillance program. Published findings from early postmarketing safety surveillance of thalidomide were compared with findings from an empirical Bayesian DMA. Differential results were used to explore practical issues in the evaluation and deployment of DMAs. Most adverse events highlighted by each method were compatible with the product labeling or natural history/complications of reported treatment indications. Traditional surveillance highlighted 4 potentially serious and unexpected adverse events (Stevens-Johnson syndrome, toxic epidermal necrolysis, seizures, skin ulcers) warranting labeling amendments or close monitoring. None of these adverse event terms generated a signal using the DMA. The DMA would not have enhanced early postmarketing surveillance in this particular setting. While the results cannot be used to draw inferences about the global performance of DMAs, they illustrate the following: (1) DMA performance may be highly situation dependent; (2) over-reliance on these methods may have deleterious consequences, especially with so-called "designated medical events"; and (3) the most appropriate selection of pharmacovigilance tools needs to be tailored to each situation, being mindful of the numerous factors that may influence comparative performance and incremental utility of DMAs.