Evidence-based medicine: data mining and pharmacoepidemiology research

Data mining (DM) research has successfully developed advanced DM techniques and algorithms over the last few decades, and many organisations have great expectations to take more benefit of their data warehouses in decision making. Currently, the strong focus of most DM-researchers is still only on technology-oriented topics. Commonly the DM research has several stakeholders, the major of which can be divided into internal and external ones each having their own point of view, and which are at least partly conflicting. The most important internal groups of stakeholders are the DM research community and academics in other disciplines. The most important external stakeholder groups are managers and domain experts who have their own utility-based interests to DM and DM research results. In this paper we discuss these practice-oriented points of view towards DM research and suggest broader discussions inside the DM research community about who should do that kind of research. We bring in the discussion several topics developed in the information systems (IS) discipline and show some similarities between IS and DM systems. DM systems have also their own peculiarities and we conclude that researchers who take into account human and organisational aspects related to DM systems need to have also some understanding about DM. This makes us suggest that the research area inside the DM community should be made broader than the current heavily technology-oriented one.

Sentiment analysis has been rapidly employed for business decision support. New data mining researchers are yet to have an adequate understanding of the various applications of sentiment analysis while utilising social media data. As a result, it is critical to define the data mining and text analytics research trend holistically using existing literature. The study explores sentiment analysis research for its application in transforming social media data and identifies relevant research aspects through a comprehensive bibliometric review of 523 research articles published in the Scopus database (between 2018 and 2022) to discern the content and thematic analysis. Findings suggested that key purposes of the sentiment analysis are mainly related to innovation, transparency, and efficiency. Our review also highlights the distinctiveness of sentiment analysis for synthesising social media information to investigate various features, including the knowledge-domain map that detects author collaboration networks in the past.

Data mining is one of the most important steps of the knowledge discovery in databases process and is considered as significant subfield in knowledge management. Research in data mining continues growing in business and in learning organization over coming decades. This review paper explores the applications of data mining techniques which have been developed to support knowledge management process. The journal articles indexed in ScienceDirect Database from 2007 to 2012 are analyzed and classified. The discussion on the findings is divided into 4 topics: (i) knowledge resource; (ii) knowledge types and/or knowledge datasets; (iii) data mining tasks; and (iv) data mining techniques and applications used in knowledge management. The article first briefly describes the definition of data mining and data mining functionality. Then the knowledge management rationale and major knowledge management tools integrated in knowledge management cycle are described. Finally, the applications of data mining techniques in the process of knowledge management are summarized and discussed.

DEFINING TEXT MINING Text mining can be broadly defined as a knowledge-intensive process in which a user interacts with a document collection over time by using a suite of analysis tools. In a manner analogous to data mining, text mining seeks to extract useful information from data sources through the identification and exploration of interesting patterns. In the case of text mining, however, the data sources are document collections, and interesting patterns are found not among formalized database records but in the unstructured textual data in the documents in these collections. Certainly, text mining derives much of its inspiration and direction from seminal research on data mining. Therefore, it is not surprising to find that text mining and data mining systems evince many high-level architectural similarities. For instance, both types of systems rely on preprocessing routines, pattern-discovery algorithms, and presentation-layer elements such as visualization tools to enhance the browsing of answer sets. Further, text mining adopts many of the specific types of patterns in its core knowledge discovery operations that were first introduced and vetted in data mining research. Because data mining assumes that data have already been stored in a structured format, much of its preprocessing focus falls on two critical tasks: Scrubbing and normalizing data and creating extensive numbers of table joins. In contrast, for text mining systems, preprocessing operations center on the identification and extraction of representative features for natural language documents.

The Outlier detection is very active area of research in data mining where outlier is a mismatched data in dataset with respect to the other available data.In existing approaches the outlier detection done only on numeric dataset.For outlier detection if we use clustering method , then they mainly focus on those elements as outliers which are lying outside the clusters but it may possible that some of the unknown elements with any possible reasons became the part of the cluster so we have to concentrate on that also.The Proposed method uses hybrid approach to reduce the number of outliers.The number of outlier can only reduce by improving the cluster formulation method.The proposed method uses two data mining techniques for cluster formulation i.e. weighted k-means and neural network where weighted kmeans is the clustering technique that can apply on text and date data set as well as numeric data set.Weighted kmeans assign the weights to each element in dataset.The output of weighted k-means becomes the input for neural network where the neural network is the classification and clustering technique of data mining.Training is provided to the neural network and according to that neurons performed the testing.The neural network test the cluster formulated by weighted k-means to ensure that the clusters formulated by weighted k-means are group accordingly.There is lots of outlier detection methods present in data mining.The proposed method use Integrating Semantic Knowledge (SOF) for outlier detection.This method detects the semantic outlier where the semantic outlier is a data point that behaves differently with other data points in the same class or cluster.The main motive of this research work is to reduce the number of outliers by improving the cluster formulation methods so that outlier rate reduces and also to decrease the mean square error and improve the accuracy.The simulation result clearly shows that proposed method works pretty well as it significantly reduces the outlier.

Over the last decade data warehousing and data mining tools have evolved from research into a unique and popular applications, ranging from data warehousing and data mining for decision support to business intelligence and other kind of applications. The chapter presents and discusses data warehousing methodologies along with the main components of data mining tools and technologies and how they all could be integrated together for knowledge management in a broader sense. Knowledge management refers to the set of processes developed in an organization to create, extract, transfer, store and apply knowledge. The chapter also focuses on how data mining tools and technologies could be used in extracting knowledge from large databases or data warehouses. Knowledge management increases the ability of an organization to learn from its environment and to incorporate knowledge into the business processes by adapting to new tools and technologies. Knowledge management is also about the reusability of the knowledge that is being extracted and stored in the knowledge base. One way to improve the reusability is to use this knowledge base as front-ends to case-based reasoning (CBR) applications. The chapter further focuses on the reusability issues of knowledge management and presents an integrated framework for knowledge management by combining data mining (DM) tools and technologies with CBR methodologies. The purpose of the integrated framework is to discover, validate, retain, reuse and share knowledge in an organization with its internal users as well as its external users. The framework is independent of application domain and would be suitable for uses in areas, such as data mining and knowledge management in e-government.

As an analysis tool either of on-line analytical processing (OLAP) or data mining has its own strong points and weaknesses. It is an effective way to enhance the power and flexibility of data mining in data warehouses and large databases by integrating data mining with OLAP to offset their weaknesses. In the paper, a proposal of integrating data mining and OLAP is put forward. The mechanism of the on-line analytical data mining in data warehouses is described. Aspects which are necessary for developing a successful on-line analytical data mining system is also discussed.

Noting that several rule discovery algorithms in data mining can produce a large number of irrelevant or obvious rules from data, there has been substantial research in data mining that addressed the issue of what makes rules truly 'interesting'. This resulted in the development of a number of interestingness measures and algorithms that find all interesting rules from data. However, these approaches have the drawback that many of the discovered rules, while supposed to be interesting by definition, may actually (1) be obvious in that they logically follow from other discovered rules or (2) be expected given some of the other discovered rules and some simple distributional assumptions. In this paper we argue that this is a paradox since rules that are supposed to be interesting, in reality are uninteresting for the above reason. We show that this paradox exists for various popular interestingness measures and present an abstract characterization of an approach to alleviate the paradox. We finally discuss existing work in data mining that addresses this issue and show how these approaches can be viewed with respect to the characterization presented here.

Basically, Data Mining (DM) and Operations Research (OR) are two paradigms independent of each other. OR aims at optimal solutions of decision problems with respect to a given goal. DM is concerned with secondary analysis of large amounts of data (Hand et al., 2001). However, there are some commonalities. Both paradigms are application focused (Wu et al., 2003; White, 1991). Many Data Mining approaches are within traditional OR domains like logistics, manufacturing, health care or finance. Further, both DM and OR are multidisciplinary. Since its origins, OR has been relying on fields such as mathematics, statistics, economics and computer science. In DM, most of the current textbooks show a strong bias towards one of its founding disciplines, like database management, machine learning or statistics. Being multidisciplinary and application focused, it seems to be a natural step for both paradigms to gain synergies from integration. Thus, recently an increasing number of publications of successful approaches at the intersection of DM and OR can be observed. On the one hand, efficiency of the DM process is increased by use of advanced optimization models and methods originating from OR. On the other hand, effectiveness of decision making is increased by augmentation of traditional OR approaches with DM results. Meisel and Mattfeld (in press) provide a detailed discussion of the synergies of DM and OR.

With the huge amount of information available online, the World Wide Web is a fertile area for data mining research. The data mining research is at the cross road of research from several research communities, such as database, information retrieval, and within AI, especially the sub-areas of machine learning and data integrity. Every E-commerce and social website in World Wide Web uses the Classification is one of the data mining problems receiving great attention recently in the database community. Neural network is not suitable for data mining directly, because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. Different concise symbolic rules with high accuracy can be extracted from a neural network with the proposed approach. The neural network is first trained to achieve the required accuracy in data mining. In this paper we are going to combine neural network with the three different algorithms which are commonly used in data mining to improve the data mining result. These three algorithms are CHARM Algorithm, Top K Rules mining and CM SPAM Algorithm. The different datasets of online e-commerce website filpkart and Amazon are used to train the neural network and to use in data mining. The results of all three data mining algorithm with neural network techniques then tested on the available datasets and result are compared by computational complexity of the algorithm.

The use of set-valued objects is becoming increasingly commonplace in modern application domains, multimedia, genetics, the stock market, etc. Recent research on set indexing has focused mainly on containment joins and data mining without considering basic set operations on set-valued attributes. In this paper, we propose a novel indexing scheme for processing superset, subset and equality queries on set-valued attributes. The proposed index structure is a hybrid of itemset-transaction set tree of “frequent items” and an inverted list of “infrequent items” that take advantage of the developments in itemset research in data mining. In this hybrid scheme, the expectation is that basic set operations with frequent low cardinality sets will yield superior retrieval performance and avoid the high costs of construction and maintenance of item-set tree for infrequent large item-sets. We demonstrate, through extensive experiments, that the proposed method performs as expected, and yields superior overall performance compared to the state of the art indexing scheme for set-valued attributes, i.e., inverted lists.

One of the recent research topics in databases is Data Mining, to find, extract and mine the useful information from databases. In case of updating transactions in the database the already discovered knowledge may become invalid. So we need efficient knowledge management techniques for finding the updated knowledge from the database. There have been lot of research in data mining, but Knowledge Management in databases is not studied much. One of the data mining techniques is to find association rules from databases. But most of association rule algorithms find association rules from transactional databases. Our research is a further step of the Tree Based Association Rule Mining (TBAR) algorithm, used in relational databases for finding the association rules .In our approach of updating the already discovered knowledge; the proposed algorithm Association Rule Update (ARU), updates the already discovered association rules found through the TBAR algorithm. Our algorithm will be able to find incremental association rules from relational databases and efficiently manage the previously found knowledge.

A multi-agent system to support evidence based medicine and clinical decision making via data sharing and data privacy

Due to advances in communication and storage technologies, there is recent exponential increase in data generated and collected. Data mining has become a necessity to enable easier and efficient means of data processing. Data mining is a process of harvesting previously unknown information from existing data and utilizing such insight for business decision making. However, in sectors such as health, exposure of sensitive information is not a trivial issue, despite benefits of data mining. Hence, researchers have proposed various approaches on seeking valuable insight from data but being cautious to protect private data and information. The purpose of this paper is to provide a conceptual review of approaches by researchers in Privacy-Preserving Data Mining (PPDM). This paper further presents a classification of PPDM techniques and provide a clear description of distinctions of one class of techniques from the other, guided by discussions in literature. Lastly, an evaluation criteria which can be used as a building block towards standardizing evaluation criteria for PPDM techniques is proposed.

Data mining aims to develop a new generation of tools to intelligently assist humans in analyzing mountains of data. Over the past few years, great progress has been made in both research and applications of data mining. Data mining systems have helped many businesses by exposing previously unknown patterns in their databases, which were used to improve profits, enhance customer services, and ultimately achieve a competitive advantage. In this paper, we present our unique data mining system DM-II (Data Mining Integration and Interestingness). DM-II is a PC-based system working in Windows 95/98/NT environment. Apart from the normal components of a data mining system, DM-II has a number of unique and advanced sub-systems. These sub-systems have been applied in many real-life applications, including education applications, insurance application, accident application, disease application, drug screening application, image classification, etc. Here, we focus on discussing the following sub-systems: CBA: CBA originally stands for Classification-Based on Associations [lo]. It has now been extended with a number of other advanced features. In all, CBA has the following capabilities: Building classifiers using association rules: Traditionally, association rule mining and classifier building are regarded as two different data mining tasks. CBA unifies the two tasks. It is able to ,generate association rules and builds a classifier using a special subset of the association rules. What is significant is that CBA, in general, produces more accurate classifiers compared to the state-ofthe-art classification system C4.5 [ 161. It also helps to solve some outstanding problems with the existing classification systems. Pruning and summarizing the discovered associations: One major problem with association