Recent advances in exploratory data analysis with neuro-fuzzy methods

The area of Knowledge discovery and data mining is growing rapidly. Feature Discretization is a crucial issue in Knowledge Discovery in Databases (KDD), or Data Mining because most data sets used in real world applications have features with continuously values. Discretization is performed as a preprocessing step of the data mining to make data mining techniques useful for these data sets. This thesis addresses discretization issue by proposing a multivariate discretization (MVD) algorithm. It begins withal number of common discretization algorithms like Equal width discretization, Equal frequency discretization, Naïve; Entropy based discretization, Chi square discretization, and orthogonal hyper planes. After that comparing the results achieved by the multivariate discretization (MVD) algorithm with the accuracy results of other algorithms. This thesis is divided into six chapters, covering a few common discretization algorithms and tests these algorithms on a real world datasets which varying in size and complexity, and shows how data visualization techniques will be effective in determining the degree of complexity of the given data set. We have examined the multivariate discretization (MVD) algorithm with the same data sets. After that we have classified discrete data using artificial neural network single layer perceptron and multilayer perceptron with back propagation algorithm. We have trained the Classifier using the training data set, and tested its accuracy using the testing data set. Our experiments lead to better accuracy results with some data sets and low accuracy results with other data sets, and this is subject ot the degree of data complexity then we have compared the accuracy results of multivariate discretization (MVD) algorithm with the results achieved by other discretization algorithms. We have found that multivariate discretization (MVD) algorithm produces good accuracy results in comparing with the other discretization algorithm.

The very fast technological progress provokes the creation of a big volume of the astronomical data, which is fed in the different forms. There are various directions in science that use high dimensional data to be analyzed. In this paper we presented the few aspects of the "knowledge discovery in databases" (KDD) process in astronomy, analyzed and reviewed some data mining approaches. We presented the examples of astronomical instruments, data types, astronomical sources of Big Data, processing algorithms, which are used for the knowledge discovery in databases process in astronomy. The paper deals with applying the CoLiTec (Collection Light Technology) software for the online processing of the different types of astronomical data using the knowledge discovery in databases tasks. This is achieved by using of the developed OnLine Data Analysis System (OLDAS), which helps with solving of the knowledge discovery in databases tasks, like selection, transformation, data mining, interpretation, and evaluation.

Based on an overview on recent contributions with respect to data mining applications terms like “data mining”, “data warehousing” and “knowledge discovery in databases” are related to the well-known discussion where “data analysis” and “decision support” are combined with “expert knowledge research”. Next, commercial data mining tools are compared with the help of positioning and segmentation procedures to get a feeling for the support provided by software tailored for performing tasks concerning knowledge discovery in databases. Finally, an own modification of an association rule algorithm is used to handle buying histories in the area of consumer behavior interpretation.

Data mining and knowledge discovery aim at finding useful information from typically massive collections of data, and then extracting useful knowledge from the information. To date a large number of approaches have been proposed to find useful information and discover useful knowledge; for example, decision trees, Bayesian belief networks, evidence theory, rough set theory, fuzzy set theory, kNN (k-nearest-neighborhood) classifier, neural networks, and support vector machines. However, these approaches are based on a specific data type. In the real world, an intelligent system often encounters mixed data types, incomplete information (missing values), and imprecise information (fuzzy conditions). In the UCI (University of California – Irvine) Machine Learning Repository, it can be seen that there are many real world data sets with missing values and mixed data types. It is a challenge to enable machine learning or data mining approaches to deal with mixed data types (Ching, 1995; Coppock, 2003) because there are difficulties in finding a measure of similarity between objects with mixed data type attributes. The problem with mixed data types is a long-standing issue faced in data mining. The emerging techniques targeted at this issue can be classified into three classes as follows: (1) Symbolic data mining approaches plus different discretizers (e.g., Dougherty et al., 1995; Wu, 1996; Kurgan et al., 2004; Diday, 2004; Darmont et al., 2006; Wu et al., 2007) for transformation from continuous data to symbolic data; (2) Numerical data mining approaches plus transformation from symbolic data to numerical data (e.g.,, Kasabov, 2003; Darmont et al., 2006; Hadzic et al., 2007); (3) Hybrid of symbolic data mining approaches and numerical data mining approaches (e.g.,, Tung, 2002; Kasabov, 2003; Leng et al., 2005; Wu et al., 2006). Since hybrid approaches have the potential to exploit the advantages from both symbolic data mining and numerical data mining approaches, this chapter, after discassing the merits and shortcomings of current approaches, focuses on applying Self-Organizing Computing Network Model to construct a hybrid system to solve the problems of knowledge discovery from databases with a diversity of data types. Future trends for data mining on mixed type data are then discussed. Finally a conclusion is presented.

The thriving medical applications of data mining in the fields of medicine and public health has led to the popularity of its use in knowledge discovery in databases (KDD). Data mining has revealed novel biomedical and healthcare acquaintances for clinical decision making that has great potential to improve the treatment quality of hospitals and increase the survival rate of patients. Disease diagnosis is one of the applications where data mining tools are establishing the successful results. Data mining intends to endow with a systematic survey of current techniques of knowledge discovery in databases using data mining techniques that are in use in today’s medical research. Discussion is made to enable the disease diagnosis and the breakthrough of hidden healthcare patterns from related databases is offered. Also, the use of data mining to discover such relationships as those between health conditions and a disease is presented. It further discusses about the tools that can be used for the processing and classification of data. This paper summarizes various technical articles on medical diagnosis and prognosis. It has also been focused on current research being carried out using the data mining techniques to enhance the disease(s) forecasting process. This research paper provides future trends of current techniques of KDD, using data mining tools for healthcare. It also confers significant issues and challenges associated with data mining and healthcare in general. The research found a growing number of data mining applications, including analysis of health care centers for better health policy-making, detection of disease outbreaks and preventable hospital deaths. The root causes of all diseases get closer towards drugs i.e. the foremost risk factor of all hilarious diseases. Drug addiction using WEKA has been used that brings into light concerning majority of drug abusers started abusing drugs at age below 20yrs.It is to make aware the druggist about the various diseases that are caused with heavy or long term intake of drugs in their life. So, to make an expert system that will awake the youth about precarious use of drugs and also alert the affected person.

SMART software for decision makers KDD experience

The usual approach to the interpretation of producing wells is based on mechanistic models for the simulation of steady state and transient flow regimes. However, there are significant reservations about convergence problems, computational limits, the need for extensive tuning on field data, the instability of boundary conditions, the limited applicability of existing multiphase flow models, and the uncertainties associated with choke valve models. The current industry standards are critically reviewed within this framework. The real-time monitoring of producing wells is recognised as the best way of optimising field performance. Monitoring a producing well implies the ability to track, in real-time, any changes in fluid composition, flow rates, or pressure and temperature profiles. Multiphase Flow Metering (MFM) plays a key role in this scenario. Such information, combined with the critical analysis of historical data from the well itself or from analogue wells, allows diagnosis of the system and prediction of future trends. However, field data per se ’ do not necessarily generate knowledge. This is particularly true for large databases, which are difficult to manipulate to provide suitable inputs for wellbore simulators. This paper suggests how MFM, Knowledge Discovery in Databases (KDD) and Fuzzy Logic (FL) can offer an alternative approach to the analysis of producing wells. KDD is the automated extraction of patterns representing knowledge implicitly stored in large information repositories. Distributed, ad-hoc field measurements (including MFM and downhole measurements) can be processed via data cleaning, data integration, data mining, artificial intelligence, and pattern evaluation. FL can then manage the resulting information in terms of flow assurance and production optimisation. The same techniques can also be extended to the reservoir and the production network, for an integrated approach to production system analysis.

Data mining, or Knowledge Discovery in Databases (KDD), is of little benefit to commercial enterprises unless it can be carried out efficiently on realistic volumes of data. Operational factors also dictate that KDD should be performed within the context of standard DBMS. Fortunately, relational DBMS have a declarative query interface (SQL) that has allowed designers of parallel hardware to exploit data parallelism efficiently. Thus, an effective approach to the problem of efficient KDD consists of arranging that KDD tasks execute on a parallel SQL server. In this paper we devise generic KDD primitives, map these to SQL and present some results of running these primitives on a commercially-available parallel SQL server.

Knowledge discovery in databases (KDD) is a field of research that studies the development and use of various data analysis tools and techniques. KDD research has produced an array of models, theories, functions and methodologies for producing knowledge from data. However, despite these advances, nearly two thirds of information technology (IT) managers say that data mining products are too difficult to use in a business context. This chapter discusses how advances in data mining translate into the business context. It highlights the art of business implementation rather than the science of KDD.

Knowledge Discovery in Databases (KDD) has grown a lot during the last years. But providing user support for constructing workflows is still problematic. The large number of operators available in current KDD systems makes it difficult for a user to successfully solve her task. Also, workflows can easily reach a huge number of operators(hundreds) and parts of the workflows are applied several times. Therefore, it becomes hard for the user to construct them manually. In addition, workflows are not checked for correctness before execution. Hence, it frequently happens that the execution of the workflow stops with an error after several hours runtime. In this paper we present a solution to these problems. We introduce a knowledge-based representation of Data Mining (DM) workflows as a basis for cooperative interactive planning. Moreover, we discuss workflow templates, i.e. abstract workflows that can mix executable operators and tasks to be refined later into sub-workflows. This new representation helps users to structure and handle workflows, as it constrains the number of operators that need to be considered. Finally, workflows can be grouped in templates which foster re-use further simplifying DM workflow construction.

Telecommunications are one of the most dynamics and strategic areas in the world. In a complex and competitive scenario with pressure for better results with fewer resources, there is a constant need in the organizations for searching new and improved forms of administration with better understanding of the processes and customers. Techniques such as knowledge discovery in databases (KDD) and data mining (DM) appear as alternatives for enabling transform raw and complex data in models that may allow better decision making. This paper presents an application of multivariate regression analysis for modeling and explaining sales in the segment of corporate telecommunications, through acting indicators. A series of comparative analyses were accomplished with intend to identify the most appropriate or impacting indicators to perform the best decisions when selling corporate services. It is also presented a systemization of KDD activities which integrates the methodologies such as CRISPDM and SEMMA in an interactive design framework. Variable selection methods and statistical validation of the model are presented. Results are presented from real databases of large Brazilian telecommunications company.

With the increase of data, data mining has been introducing to solve the overloading problem and to discover valid, novel, potentially useful patterns in existing data. However, to discover quantitative data, we face a sharp boundary problem that the traditional data mining techniques can not overcome. To pinpoint this problem, a lot of researches have been applied fuzzy sets to discover patterns, especially in sequential patterns . As we know, U. Fayyad et al proposed a unifying process-centric framework for Knowledge Discovery in Database (KDD) which can explain the process from top to toe for the traditional data mining problems. Unfortunately, it fails to be workable in part to describe the process for discovering sequential patterns based on fuzzy concept. Therefore, to have more general viewpoint for discovering of that sort patterns, we devote to proposing a general framework modified from the KDD process. An application is proposed to demonstrate that the framework can be a generalization of the realm problem.

The terms Data Mining (DM) and Knowledge Discovery in Databases (KDD) have been used interchangeably in practice. Strictly speaking, KDD is the umbrella of the mining process and DM is only a step in KDD. We will follow this distinction in this chapter and present a simple introduction to the Knowledge Discovery in Databases process from an optimization perspective.

The grant program for farmers is one of the government programs provided by the Cianjur Regency Agriculture Office. To accept the program, farmers submit proposals which will be reviewed by the proposal reviewer analyst. In making a decision to accept a grant proposal, it is necessary to make a quick and accurate decision. With quite a lot of proposal data that has been submitted, it can be used data patterns that have been stored with data mining methods to predict proposals that are accepted with proposals that are rejected. The algorithm used is K-Nearest Neighbor where the determination of acceptance of new proposal data is based on the closest distance to the acceptance of the previous proposal. There are six predictor variables used, namely assets, capital, business scale, length of business, cost plan and completeness of documents. The stages of data mining use the stages in Knowledge Discovery in Databases (KDD). Calculations use the weight of the assessment for each proposal criteria that have been determined. The results of the evaluation of the calculation of the kNN algorithm obtained a real accuracy of 76% and an error rate of 24%. From the calculation results, the kNN algorithm can be used as a model to determine recipients of agricultural grant programs. Program pemberian dana hibah untuk para petani merupakan salah satu program pemerintah yang diberikan oleh Dinas Pertanian Kabupaten Cianjur. Untuk menerima program tersebut petani mengajukan proposal yang akan diperiksa oleh analis pemeriksa proposal. Dalam mengambil keputusan penerimaan proposal hibah perlu adanya keputusan yang cepat dan akurat. Dengan cukup banyaknya data proposal yang telah diajukan , maka dapat digunakan pola data yang telah tersimpan dengan metode data mining untuk memprediksi proposal yang diterima dengan proposal yang ditolak. Algoritma yang digunakan yaitu K-Nearest Neighbor di mana penentuan penerimaan data proposal baru berdasarkan pada jarak terdekat dengan penerimaan proposal sebelumnya. Terdapat enam variabel prediktor yang digunakan yaitu aset, modal, skala usaha, lama usaha, rencana biaya dan kelengkapan dokumen. Tahapan data mining menggunakan tahapan dalam knowledge Discovery in Databases (KDD) . Perhitungan menggunakan bobot penilaian untuk setiap kriteria proposal yang sudah ditentukan. Hasil evaluasi perhitungan algoritma kNN diperoleh akurasi sebenar 76% dan tingkat error sebesar 24%. Dari hasil perhitungan, algoritma kNN ini dapat digunakan sebagai model untuk menentukan penerima program hibah pertanian.

Data mining or knowledge discovery in database (KDD) refers to the non-trivial process of discovering interesting, implicit, and previously unknown knowledge from databases.Such a task implies to be able to perform analyses both on high-dimensional input data and large dataset.The most popular models used in KDD are the symbolic models.Unfortunately, these models suffer of very serious limitations.Rule generation is a highly time-consuming process that generates a huge number of rules, including a large ratio of redundant rules.Hence, this prohibits any kind of rule computation and selection as soon as data are numerous and they are represented by very high-dimensional description space.This latter situation is very often encountered with documentary data.To cope with these problems, preliminary KDD trials using numerical models have been made.An algorithm for knowledge extraction from self-organizing network has been proposed in [8].This approach is based on a supervised generalized relevance learning vector quantization (GRLVQ) which is used for extracting decision trees.The different paths of the generated trees are then used for denoting rules.Nevertheless, the main defect of this method is to necessitate training data.On our own side, we have proposed a hybrid classification method for mapping an explicative structure issued from a symbolic classification into an unsupervised numerical self-organizing map (SOM) [15].SOM map and Galois lattice are generated on the same data.The cosine projection is then used for associating lattice concepts to the SOM classes.Concepts properties act as explanation for the SOM classes.Furthermore, lattice pruning combined with migration of the associated SOM classes towards the top of the pruned lattice is used to generate explanation of increasing scope on the SOM map.Association rules can also be produced in such a way.Although it establishes interesting links between numerical and symbolic worlds this approach necessitates the time-consuming computation of a whole Galois lattice.In a parallel way, in order to enhance both the quality and the granularity of the data analysis and to reduce the noise which is inevitably generated in an overall classification approach, we have introduced the multi-viewpoint analysis and multilevel clustering approach based on a significant extension of the SOM model, named MultiSOM [19][25].The viewpoint building principle consists in separating the description of the data into several sub-descriptions corresponding different property subsets or even different data subsets.In MultiSOM each viewpoint is represented by a single SOM map.www.intechopen.com