Astronomical Knowledge Discovery in Databases by the CoLiTec Software

The terms Data Mining (DM) and Knowledge Discovery in Databases (KDD) have been used interchangeably in practice. Strictly speaking, KDD is the umbrella of the mining process and DM is only a step in KDD. We will follow this distinction in this chapter and present a simple introduction to the Knowledge Discovery in Databases process from an optimization perspective.

Within the framework of our GLS (Global Learning Scheme) system that is a multi-strategy and cooperative KDD (Knowledge Discovery in Databases) system, this paper reports new research progress, by addressing one deeper issue concerning KDD process planning: change management, and giving our solution for it. The problem on change management can be largely solved by an incremental replanning technique. With the issue being properly handled, the GLS system is more complete in KDD process modeling, and more flexible and robust for practical use.

With the increase of data, data mining has been introducing to solve the overloading problem and to discover valid, novel, potentially useful patterns in existing data. However, to discover quantitative data, we face a sharp boundary problem that the traditional data mining techniques can not overcome. To pinpoint this problem, a lot of researches have been applied fuzzy sets to discover patterns, especially in sequential patterns . As we know, U. Fayyad et al proposed a unifying process-centric framework for Knowledge Discovery in Database (KDD) which can explain the process from top to toe for the traditional data mining problems. Unfortunately, it fails to be workable in part to describe the process for discovering sequential patterns based on fuzzy concept. Therefore, to have more general viewpoint for discovering of that sort patterns, we devote to proposing a general framework modified from the KDD process. An application is proposed to demonstrate that the framework can be a generalization of the realm problem.

SMART software for decision makers KDD experience

In recent years both the number and the size of organisational databases have increased rapidly. However, although available processing power has also grown, the increase in stored data has not necessarily led to a corresponding increase in useful information and knowledge. This has led to a growing interest in the development of tools capable of harnessing the increased processing power available to better utilise the potential of stored data. The terms Knowledge Discovery in Databases and Mining have been adopted for a field of research dealing with the automatic discovery of knowledge implicit within databases. Data mining is useful in situations where the volume of data is either too large or too complicated for manual processing or, to a lesser extent, where human experts are unavailable to provide knowledge. The success already attained by a wide range of data mining applications has continued to prompt further investigation into alternative data mining techniques and the extension of data mining to new domains. This paper surveys, from the standpoint of the database systems community, current issues in data mining research by examining the architectural and process models adopted by knowledge discovery systems, the different types of discovered knowledge, the way knowledge discovery systems operate on different data types, various techniques for knowledge discovery and the ways in which discovered knowledge is used.

Data mining, or Knowledge Discovery in Databases (KDD), is of little benefit to commercial enterprises unless it can be carried out efficiently on realistic volumes of data. Operational factors also dictate that KDD should be performed within the context of standard DBMS. Fortunately, relational DBMS have a declarative query interface (SQL) that has allowed designers of parallel hardware to exploit data parallelism efficiently. Thus, an effective approach to the problem of efficient KDD consists of arranging that KDD tasks execute on a parallel SQL server. In this paper we devise generic KDD primitives, map these to SQL and present some results of running these primitives on a commercially-available parallel SQL server.

Like other data-rich disciplines such as physics, biology, geology, and oceanography, astronomy is facing a data avalanche due to advances in telescope and detector technology, the exponential increase in computing capabilities, improvements in data-collection methods, and successful applications of theoretical simulations. As the era approaches in which data covers the full range of wavelengths from radio to gamma-rays, the expected data volumes will add up to terabytes, soon to be followed by petabytes. Proper management and processing of massive data sets requires efficient federation of database technologies. However, mining knowledge from huge data volumes is the ultimate goal and development of data-mining techniques is therefore critical. Knowledge discovery in databases (KDD) is the process of extracting useful knowledge from data. Data mining, the application of specific algorithms to discover rare or previously unknown types of object or phenomenon, is a particular step in the process.1 KDD is inherently interactive and iterative, as shown in Figure 1. Common KDD functions are classification, cluster analysis, and regression. In classification one develops a description or model for each class of data labeled with discrete integers (as opposed to cluster analysis, which is sometimes called ‘unsupervised classification’). Classification is used for the organization of future test data, better understanding of each data class, and predictions of certain properties and behaviors. It is based on spectra or images and, for example, may be used to describe galaxies by morphology. Cluster (or clustering) analysis is a multivariate procedure based on placing objects into more or less homogeneous groups such that the relationship between groups is revealed. It lacks an underlying body of statistical theory and is heuristic in nature, requiring decisions to be made by individual users (which can strongly affect results). Cluster analysis is used to classify groups or objectsmore objectively than subjectively and can help astronomers find unusual objects within a flood of data. ExamFigure 1. Knowledge discovery in databases.

Knowledge discovery in databases is a process that aims at the discovery of associations within data sets. The analysis of geo-referenced data demands a particular approach in this process. This chapter presents a new approach to the process of knowledge discovery, in which qualitative geographic identifiers give the positional aspects of geographic data. Those identifiers are manipulated using qualitative reasoning principles, which allows for the inference of new spatial relations required for the data mining step of the knowledge discovery process. The efficacy and usefulness of the implemented system — Padrão — has been tested with a bank dataset. The results support that traditional knowledge discovery systems, developed for relational databases and not having semantic knowledge linked to spatial data, can be used in the process of knowledge discovery in geo-referenced databases, since some of this semantic knowledge and the principles of qualitative spatial reasoning are available as spatial domain knowledge.

Knowledge discovery in databases, or data mining, is an important objective in the development of data- and knowledge-base systems. An attribute-oriented rough set method is developed for knowledge discovery in databases. The method integrates learning from example techniques with rough set theory. An attribute-oriented concept tree ascension technique is first applied in generalization, which substantially reduces the computational complexity of the database learning processes. Then the rough set techniques are applied to the generalized relation to derive different knowledge rules. Moreover, the approach can find all the maximal generalized rules in the data. Based on these principles, a prototype database learning system, DBROUGH, has been constructed. Our study shows that attribute-oriented induction combined with rough set techniques provide an efficient and effective mechanism for knowledge discovery in database systems.

Data mining and knowledge discovery aim at finding useful information from typically massive collections of data, and then extracting useful knowledge from the information. To date a large number of approaches have been proposed to find useful information and discover useful knowledge; for example, decision trees, Bayesian belief networks, evidence theory, rough set theory, fuzzy set theory, kNN (k-nearest-neighborhood) classifier, neural networks, and support vector machines. However, these approaches are based on a specific data type. In the real world, an intelligent system often encounters mixed data types, incomplete information (missing values), and imprecise information (fuzzy conditions). In the UCI (University of California – Irvine) Machine Learning Repository, it can be seen that there are many real world data sets with missing values and mixed data types. It is a challenge to enable machine learning or data mining approaches to deal with mixed data types (Ching, 1995; Coppock, 2003) because there are difficulties in finding a measure of similarity between objects with mixed data type attributes. The problem with mixed data types is a long-standing issue faced in data mining. The emerging techniques targeted at this issue can be classified into three classes as follows: (1) Symbolic data mining approaches plus different discretizers (e.g., Dougherty et al., 1995; Wu, 1996; Kurgan et al., 2004; Diday, 2004; Darmont et al., 2006; Wu et al., 2007) for transformation from continuous data to symbolic data; (2) Numerical data mining approaches plus transformation from symbolic data to numerical data (e.g.,, Kasabov, 2003; Darmont et al., 2006; Hadzic et al., 2007); (3) Hybrid of symbolic data mining approaches and numerical data mining approaches (e.g.,, Tung, 2002; Kasabov, 2003; Leng et al., 2005; Wu et al., 2006). Since hybrid approaches have the potential to exploit the advantages from both symbolic data mining and numerical data mining approaches, this chapter, after discassing the merits and shortcomings of current approaches, focuses on applying Self-Organizing Computing Network Model to construct a hybrid system to solve the problems of knowledge discovery from databases with a diversity of data types. Future trends for data mining on mixed type data are then discussed. Finally a conclusion is presented.

Knowledge discovery in databases (KDD) is an iterative multi-stage process for extracting useful, non-trivial information from large databases. Each stage of the process presents numerous choices to the user that can significantly change the outcome of the project. This methodology, presented in the form of a roadmap, emphasises the importance of the early stages of the KDD process and shows how careful planning can lead to a successful and well-managed project. The content is the result of expertise acquired through research and a wide range of practical experiences; the work is of value to KDD experts and novices alike. Each stage, from specification to exploitation, is described in detail with suggested approaches, resources and questions that should be considered. The final section describes how the methodology has been successfully used in the design of a commercial KDD toolkit.

In this paper we deal with association rule mining in the context of a complex, interactive and iterative knowledge discovery process. After a general introduction covering the basics of association rule mining and of the knowledge discovery process in databases we draw the attention to the problematic aspects concerning the integration of both. Actually, we come to the conclusion that with regard to human involvement and interactivity the current situation is far from being satisfying. In our paper we tackle this problem on three sides: First of all there is the algorithmic complexity. Although today’s algorithms efficiently prune the immense search space the achieved run times do not allow true interactivity. Nevertheless we present a rule caching schema that significantly reduces the number of mining runs. This schema helps to gain interactivity even in the presence of extreme run times of the mining algorithms. Second, today the mining data is typically stored in a relational database management system. We present an efficient integration with modern database systems which is one of the key factors in practical mining applications. Third, interesting rules must be picked from the set of generated rules. This might be quite costly because the generated rule sets normally are quite large whereas the percentage of useful rules is typically only a very small fraction. We enhance the traditional association rule mining framework in order to cope with this situation.KeywordsAssociation RuleMining AlgorithmFrequent ItemsetsAssociation Rule MiningGeneralize Association RuleThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Telecommunications are one of the most dynamics and strategic areas in the world. In a complex and competitive scenario with pressure for better results with fewer resources, there is a constant need in the organizations for searching new and improved forms of administration with better understanding of the processes and customers. Techniques such as knowledge discovery in databases (KDD) and data mining (DM) appear as alternatives for enabling transform raw and complex data in models that may allow better decision making. This paper presents an application of multivariate regression analysis for modeling and explaining sales in the segment of corporate telecommunications, through acting indicators. A series of comparative analyses were accomplished with intend to identify the most appropriate or impacting indicators to perform the best decisions when selling corporate services. It is also presented a systemization of KDD activities which integrates the methodologies such as CRISPDM and SEMMA in an interactive design framework. Variable selection methods and statistical validation of the model are presented. Results are presented from real databases of large Brazilian telecommunications company.

Based on an overview on recent contributions with respect to data mining applications terms like “data mining”, “data warehousing” and “knowledge discovery in databases” are related to the well-known discussion where “data analysis” and “decision support” are combined with “expert knowledge research”. Next, commercial data mining tools are compared with the help of positioning and segmentation procedures to get a feeling for the support provided by software tailored for performing tasks concerning knowledge discovery in databases. Finally, an own modification of an association rule algorithm is used to handle buying histories in the area of consumer behavior interpretation.

The thriving medical applications of data mining in the fields of medicine and public health has led to the popularity of its use in knowledge discovery in databases (KDD). Data mining has revealed novel biomedical and healthcare acquaintances for clinical decision making that has great potential to improve the treatment quality of hospitals and increase the survival rate of patients. Disease diagnosis is one of the applications where data mining tools are establishing the successful results. Data mining intends to endow with a systematic survey of current techniques of knowledge discovery in databases using data mining techniques that are in use in today’s medical research. Discussion is made to enable the disease diagnosis and the breakthrough of hidden healthcare patterns from related databases is offered. Also, the use of data mining to discover such relationships as those between health conditions and a disease is presented. It further discusses about the tools that can be used for the processing and classification of data. This paper summarizes various technical articles on medical diagnosis and prognosis. It has also been focused on current research being carried out using the data mining techniques to enhance the disease(s) forecasting process. This research paper provides future trends of current techniques of KDD, using data mining tools for healthcare. It also confers significant issues and challenges associated with data mining and healthcare in general. The research found a growing number of data mining applications, including analysis of health care centers for better health policy-making, detection of disease outbreaks and preventable hospital deaths. The root causes of all diseases get closer towards drugs i.e. the foremost risk factor of all hilarious diseases. Drug addiction using WEKA has been used that brings into light concerning majority of drug abusers started abusing drugs at age below 20yrs.It is to make aware the druggist about the various diseases that are caused with heavy or long term intake of drugs in their life. So, to make an expert system that will awake the youth about precarious use of drugs and also alert the affected person.