Integration of Data Mining and Operations Research

Contributors

Contributors

Data mining involves extracting interesting patterns from data and can be found at the heart of operational research (OR), as its aim is to create and enhance decision support systems. Even in the early days, some data mining approaches relied on traditional OR methods such as linear programming and forecasting, and modern data mining methods are based on a wide variety of OR methods including linear and quadratic optimization, genetic algorithms and concepts based on artificial ant colonies. The use of data mining has rapidly become widespread, with applications in domains ranging from credit risk, marketing, and fraud detection to counter-terrorism. In all of these, data mining is increasingly playing a key role in decision making. Nonetheless, many challenges still need to be tackled, ranging from data quality issues to the problem of how to include domain experts' knowledge, or how to monitor model performance. In this paper, we outline a series of upcoming trends and challenges for data mining and its role within OR.

Innovative Operational Research practitioner, intellectual leader in the development of systems ideas, and founder of the field of management cybernetics. Born: September 25, 1926 in London, England. Died: August 23, 2002 in Toronto, Canada. Education: D.Sc., University of Sunderland, 2000. Key positions: Production Controller and Head of OR Group, Samuel Fox (1949–1956); Head, Operational Research and Cybernetics, United Steel (1956–1961); Managing Director, SIGMA (1961–1966); Development Director, International Publishing Corporation (1966– 1970); Consultant (1970–2002). Awards and Recognition: Silver Medal, Royal Swedish Academy for Engineering Sciences (1958); Lanchester Prize, Operations Research Society of America (1966); McCullough Plaque of the American Society for Cybernetics (1970); Norbert Weiner Gold Medal, World Organization of Systems and Cybernetics (1984); Lifetime Achievement Award, UK Systems Society (1999); Honorary degrees: Doctor of Laws, Concordia University, Montreal, Canada (1988); Doctor of Economic Sciences, University of St. Gallen, Switzerland (2000). Key OR roles: President, Operational Research Society (1970–1971); President, Society for General Systems Research (1971–1972); President, World Organization of Systems and Cybernetics (1970–the time of his death). Stafford Beer was one of the most remarkable figures that Operational Research (OR) in any country has produced. A charismatic, even flamboyant, character, he founded two major and pioneering OR groups; wrote some of the best books about the subject; and was a world leader in the development of systems ideas. He is widely acknowledged as the founder of the field of International Transactions in Operational Resarch.

PurposeThe purpose of this study is to explore to which extent data mining research would be associated with the library and information science (LIS) discipline. This study aims to identify data mining related subject terms and topics in representative LIS scholarly publications.Design/methodology/approachA large set of bibliographic records over 38,000 was collected from a scholarly database representing the fields of LIS and the data mining, respectively. A multitude of text mining techniques were applied to investigate prevailing subject terms and research topics, such as influential term analysis and Dirichlet multinomial regression topic modeling.FindingsThe findings of this study revealed the relationship between the LIS and data mining research domains. Various data mining method terms were observed in recent LIS publications, such as machine learning, artificial intelligence and neural networks. The topic modeling result identified prevailing data mining related research topics in LIS, such as machine learning, deep learning, big data and among others. In addition, this study investigated the trends of popular topics in LIS over time in the recent decade.Originality/valueThis investigation is one of a few studies that empirically investigated the relationships between the LIS and data mining research domains. Multiple text mining techniques were employed to delineate to which extent the two research domains would be associated with each other based on both at the term-level and topic-level analysis. Methodologically, the study identified influential terms in each domain using multiple feature selection indices. In addition, Dirichlet multinomial regression was applied to explore LIS topics in relation to data mining.

Data mining (DM) research has successfully developed advanced DM techniques and algorithms over the last few decades, and many organisations have great expectations to take more benefit of their data warehouses in decision making. Currently, the strong focus of most DM-researchers is still only on technology-oriented topics. Commonly the DM research has several stakeholders, the major of which can be divided into internal and external ones each having their own point of view, and which are at least partly conflicting. The most important internal groups of stakeholders are the DM research community and academics in other disciplines. The most important external stakeholder groups are managers and domain experts who have their own utility-based interests to DM and DM research results. In this paper we discuss these practice-oriented points of view towards DM research and suggest broader discussions inside the DM research community about who should do that kind of research. We bring in the discussion several topics developed in the information systems (IS) discipline and show some similarities between IS and DM systems. DM systems have also their own peculiarities and we conclude that researchers who take into account human and organisational aspects related to DM systems need to have also some understanding about DM. This makes us suggest that the research area inside the DM community should be made broader than the current heavily technology-oriented one.

Data mining is one of the most important steps of the knowledge discovery in databases process and is considered as significant subfield in knowledge management. Research in data mining continues growing in business and in learning organization over coming decades. This review paper explores the applications of data mining techniques which have been developed to support knowledge management process. The journal articles indexed in ScienceDirect Database from 2007 to 2012 are analyzed and classified. The discussion on the findings is divided into 4 topics: (i) knowledge resource; (ii) knowledge types and/or knowledge datasets; (iii) data mining tasks; and (iv) data mining techniques and applications used in knowledge management. The article first briefly describes the definition of data mining and data mining functionality. Then the knowledge management rationale and major knowledge management tools integrated in knowledge management cycle are described. Finally, the applications of data mining techniques in the process of knowledge management are summarized and discussed.

We compare person-to-person service encounters with those in which the service provider is an information system to identify the capabilities needed to personalize a service encounter. We suggest “substituting information for interaction” as a principle that unifies these different types of encounters whenever the information needed to create value in a service system accumulates incrementally through human or automated customer interactions. We review research and practice in computer science, artificial intelligence, data mining, machine learning, and information systems design to bring an interdisciplinary robustness to our conceptual proposal. Human service providers and automated service systems both need (1) a service model manager that stores information about how a customer requests a service; (2) a customer model manager that stores information about customers and preferences; (3) a recommendation system manager that uses service models, customer models, and contextual information to adapt the service at delivery time; (4) a learning system that analyzes previous service encounters to refine service and customer models, and a (5) service monitoring system that monitors the status of service delivery. The substitution concepts and mechanisms we propose highlight the range of design choices and help managers evaluate whether a human interaction or information exchange creates or undermines value in a service system.

Operations research (OR) techniques have been widely used for optimizing problems, such as manufacturing scheduling, supply chain optimization, and resource allocation. Despite its effectiveness, traditional OR, especially exact methods, often struggle with scalability, computational efficiency, and adaptability to the dynamic and uncertain environments of Industry 4.0. While machine learning (ML) advancements provide novel approaches for addressing these challenges, they also present limitations, such as the lack of guaranteeing exact solutions and the need of relevant data. Therefore, the integration of OR and ML offers a balanced solution, leveraging ML’s capability to extract patterns from large datasets and making predictive decisions and OR’s precision to enhance decision-making processes, especially in scheduling tasks withing the context of Industry 4.0. This combination not only improves solution robustness and efficiency but also mitigates individual limitations of both fields. and make predictive decisions under uncertainty complements the decision-making process of OR. This paper aims to conduct a bibliometric analysis and a brief literature review on the integration of ML and OR, focusing on their application in scheduling problems.

The Outlier detection is very active area of research in data mining where outlier is a mismatched data in dataset with respect to the other available data.In existing approaches the outlier detection done only on numeric dataset.For outlier detection if we use clustering method , then they mainly focus on those elements as outliers which are lying outside the clusters but it may possible that some of the unknown elements with any possible reasons became the part of the cluster so we have to concentrate on that also.The Proposed method uses hybrid approach to reduce the number of outliers.The number of outlier can only reduce by improving the cluster formulation method.The proposed method uses two data mining techniques for cluster formulation i.e. weighted k-means and neural network where weighted kmeans is the clustering technique that can apply on text and date data set as well as numeric data set.Weighted kmeans assign the weights to each element in dataset.The output of weighted k-means becomes the input for neural network where the neural network is the classification and clustering technique of data mining.Training is provided to the neural network and according to that neurons performed the testing.The neural network test the cluster formulated by weighted k-means to ensure that the clusters formulated by weighted k-means are group accordingly.There is lots of outlier detection methods present in data mining.The proposed method use Integrating Semantic Knowledge (SOF) for outlier detection.This method detects the semantic outlier where the semantic outlier is a data point that behaves differently with other data points in the same class or cluster.The main motive of this research work is to reduce the number of outliers by improving the cluster formulation methods so that outlier rate reduces and also to decrease the mean square error and improve the accuracy.The simulation result clearly shows that proposed method works pretty well as it significantly reduces the outlier.

With the huge amount of information available online, the World Wide Web is a fertile area for data mining research. The data mining research is at the cross road of research from several research communities, such as database, information retrieval, and within AI, especially the sub-areas of machine learning and data integrity. Every E-commerce and social website in World Wide Web uses the Classification is one of the data mining problems receiving great attention recently in the database community. Neural network is not suitable for data mining directly, because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. Different concise symbolic rules with high accuracy can be extracted from a neural network with the proposed approach. The neural network is first trained to achieve the required accuracy in data mining. In this paper we are going to combine neural network with the three different algorithms which are commonly used in data mining to improve the data mining result. These three algorithms are CHARM Algorithm, Top K Rules mining and CM SPAM Algorithm. The different datasets of online e-commerce website filpkart and Amazon are used to train the neural network and to use in data mining. The results of all three data mining algorithm with neural network techniques then tested on the available datasets and result are compared by computational complexity of the algorithm.

Operations Research applications: Opportunities and accomplishments

About Our Authors

Computational scientists must understand results from experimental, observational and computational simulation generated data to gain insights and perform knowledge discovery. As systems approach the petascale range, problems that were unimaginable a few years ago are within reach. With the increasing volume and complexity of data produced by ultra-scale simulations and high-throughput experiments, understanding the science is largely hampered by the lack of comprehensive I/O, storage, acceleration of data manipulation, analysis, and mining tools. Scientists require techniques, tools and infrastructure to facilitate better understanding of their data, in particular the ability to effectively perform complex data analysis, statistical analysis and knowledge discovery. The goal of this work is to enable more effective analysis of scientific datasets through the integration of enhancements in the I/O stack, from active storage support at the file system layer to MPI-IO and high-level I/O library layers. We propose to provide software components to accelerate data analytics, mining, I/O, and knowledge discovery for large-scale scientific applications, thereby increasing productivity of both scientists and the systems. Our approaches include 1) design the interfaces in high-level I/O libraries, such as parallel netCDF, for applications to activate data mining operations at the lower I/O layers; 2) Enhance MPI-IO runtime systems to incorporate the functionality developed as a part of the runtime system design; 3) Develop parallel data mining programs as part of runtime library for server-side file system in PVFS file system; and 4) Prototype an active storage cluster, which will utilize multicore CPUs, GPUs, and FPGAs to carry out the data mining workload.

Machine learning, data mining, and several related research areas are concerned with methods for the automated induction of models and the extraction of patterns from realistic data. Fuzzy classification is one of the important applications in fuzzy set where values ranges between 0 and 1 .Main objective of the fuzzy logic and fuzzy set to find a set of fuzzy rules that form a classification model. The set is major advantage of using fuzzy rules for classification applications to maintain transparency as well as high accuracy rate. Fuzzy pattern tree induction introduced as a novel for machine learning method for classification. Pattern tree is a hierarchical tree-like structure, whose inner nodes are marked with generalized (fuzzy) logical operators and whose leaf nodes are associated with fuzzy predicates on input attributes. Pattern tree make use of different aggregation including t-norms and t-conorms. There are two types of Fuzzy Pattern trees: (i) Bottom up induction: Information propagates from bottom to top. (ii) Top down induction: Information propagates from top to down. In this paper performance analysis and comparative study of above both approaches using number of parameters like accuracy rate, model size is mentioned. The paper is divided into five sections: section 1 is introduction of Fuzzy pattern tree, section 2 describes types of fuzzy pattern tree schemes and section 3 presents their comparative performance, section 4 does the conclusion. Section 5 includes future scope.