DATA ENGINEERING IN CRISP-DM PROCESS PRODUCTION DATA – CASE STUDY

Recent developments in computing, communications, digital storage technologies, and high-throughput data-acquisition technologies, make it possible to gather and store incredible volumes of data. It creates unprecedented opportunities for knowledge discovery large-scale database. Data mining technology is a useful tool for this task. It is an emerging area of computational intelligence that offers new theories, techniques, and tools for processing large volumes of data, such as data analysis, decision making, etc. There are countless researchers working on designing efficient data mining techniques, methods, and algorithms. Unfortunately,most data mining researchers pay much attention to technique problems for developing data mining models and methods, while little to basic issues of data mining. What is data mining? What is the product of a data mining process? What are we doing in a data mining process? What is the rule we would obey in a data mining process? What is the relationship between the prior knowledge of domain experts and the knowledgemind from data? In this paper, we will address these basic issues of data mining from the viewpoint of informatics [1]. Data is taken as a manmade format for encoding knowledge about the natural world. We take data mining as a process of knowledge transformation. A domain-oriented data-driven data mining (3DM) model based on a conceptual data mining model is proposed. Some data-driven data mining algorithms are also proposed to show the validity of this model, e.g., the data-driven default rule generation algorithm, data-driven decision tree pre-pruning algorithm and data-driven knowledge acquisition from concept lattice.

Data mining research deals with extracting useful information from large collections of data. Since data mining is a complex process that requires expertise, it is beneficial to provide it as a service on the web. On the other hand, such use of data mining services combined with data collection efforts by private and government organizations leads to increased privacy concerns. In this work, we address the issue of preserving privacy while providing data mining services on the web and present an architecture for privacy preserving sharing of data mining models on the web. In the proposed architecture, data providers use APPEL for specifying their privacy preferences on data mining models, while data collectors use P3P policies for specifying their data-usage practices. Both parties use PMML as the standard for specifying data mining queries, constraints and models.

Recent advances in computing, communications, digital storage technologies, and high-throughput data-acquisition technologies, make it possible to gather and store incredible volumes of data. It creates unprecedented opportunities for large-scale knowledge discovery from huge database. Data mining (DM) technology has emerged as a means of performing this discovery. There are countless researchers working on designing efficient data mining techniques, methods, and algorithms. Many data mining methods and algorithms have been developed and applied in a lot of application fields [1]. Unfortunately, most data mining researchers pay much attention to technique problems for developing data mining models and methods, while little to basic issues of data mining [2]. In this talk, some basic issues of data mining are addressed. What is data mining? What is the product of a data mining process? What are we doing in a data mining process? What is the rule we should obey in a data mining process? Through analyzing existing data mining methods, and domain-driven (or user-driven) data mining models [3-5], we find that we should take a data mining process as a process of knowledge transformation. Based on this understanding of data mining, a conceptual data mining model of domain-oriented data-driven data mining (3DM) is proposed [2]. The relationship between traditional domain-driven (or user-driven) data mining models and our proposed 3DM model is also analyzed. Some domain-oriented data-driven data mining algorithms for mining such knowledge as default rule [6], decision tree [7], and concept lattice [8] from database are proposed. The experiment results for these algorithms are also shown to illustrate the efficiency and performance of the knowledge acquired by our 3DM data mining algorithms.

The advanced design of a centrifugal compressor with high efficiency and wide operating range is a challenging task due to the complex flow field arising from the three-dimensional geometry, especially for the high-speed, highly loaded centrifugal compressor stage, which typically has a relatively narrow operating range. A great effort has been undertaken recently to solve the time-costly three-dimensional design problem with the assistance of a metamodel. Some effort has been done to gain insight into the design space with the assistance of the data mining method. However, the published works lack any study that systematically performs the data mining between the performance and three-dimensional geometry data due to two unsolved issues, i.e., lack of reliable systematic data mining model and unresolved high-dimensional data problem in the centrifugal compressor community. To tackle these issues, a systematic metamodel-driven data mining (MDDM) model including six general modules (i.e., problem understanding, data understating, metamodeling, data set preparation, knowledge discovery, and deployment) has been proposed and implemented to the knowledge discovery of the well-known Radiver high-speed centrifugal compressor stage. Particular attention has been paid to develop the design principle of operating range extension for the examined high-speed stage. Four specific data mining techniques, i.e., descriptive statistics, self-organization map, k–d tree, and Sobol index, were used for the statistical, correlation, cluster, and sensitivity analysis. The results showed the performance improvement probabilities, the trade-off relationships between efficiency and pressure ratio/operating range, and the characteristic variation of the performance. Specifically, the wide operating range design subspace and the narrow operating range design subspace were split away from the whole design space. In these subspaces, the two most sensitive geometry parameters that controlled the meridional curvature made a large contribution to the stage performance, especially for the meridional curvature at the shroud side near the impeller outlet. The appropriate variation ranges of the two sensitive geometry parameters were recommended, and the flow mechanism behind them was clarified. The statistical results showed that over 90% of the design stages in the recommended variation ranges had a wide operating range. A design case was chosen randomly in the recommended range to verify the performance improvement via computational fluid dynamics (CFD) simulations. The outcomes of this work are particularly relevant for the advanced design of compressors with high efficiency and a wide operating range for flexibility.

In this paper, we propose four data mining models for the Internet of Things, which are multi-layer data mining model, distributed data mining model, Grid based data mining model and data mining model from multi-technology integration perspective. Among them, multi-layer model includes four layers: 1) data collection layer, 2) data management layer, 3) event processing layer, and 4) data mining service layer. Distributed data mining model can solve problems from depositing data at different sites. Grid based data mining model allows Grid framework to realize the functions of data mining. Data mining model from multi-technology integration perspective describes the corresponding framework for the future Internet. Several key issues in data mining of IoT are also discussed.

More and more literatures have researched the application of data mining technology in customer segmentation, and achieved sound effects. One of the key purposes of customer segmentation is customer retention. But the application of single data mining technology mentioned in previous literatures is unable to identify customer churn trend for adopting different actions on customer retention. This paper focus on constructs a integrated data mining and survival analysis model to segment customers into heterogeneous group by their survival probability (churn trend) and help enterprises adopting appropriate actions to retain profitable customers according to each segment’s churn trend. This model contains two components. Firstly, using data mining clustering arithmetic cluster customers into heterogeneous clusters according to their survival characters. Secondly, using survival analysis predicting each cluster’s survival/hazard function to identify their churn trend and test the validity of clustering for getting the correct customer segmentation. This model proposed by this paper was applied in a dataset from one biggest china telecommunications company. This paper also suggests some propositions for further research.KeywordsData MiningCustomer Relationship ManagementCustomer SegmentCustomer RetentionHeterogeneous ClusterThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper is to enhance our understanding about the second-order data mining. In particularly, we examine the effect of human cognition on the creation of intelligent knowledge during the second-order data mining process. Prior studies have suggested that human cognition plays an important role in the second-order data mining process during which intelligent knowledge was discovered. Given the knowledge that no single data mining model outperforms others for all problems, a common practice in data mining projects is to run multiple data mining models at first and then invite a group of people to collaboratively make judgments on these data mining models’ performance. These judgments often diverge. Little research exists to explain why these variations of human judgments occur.

Air pollution has become one of the biggest environmental challenges worldwide, and Jakarta, the capital city of Indonesia, is no exception. Jakarta, as one of the most populous cities in the world, faces serious problems in ensuring healthy air quality for its residents. The impact of this air pollution is not only limited to human health, but also damages the environment as a whole, including plants and aquatic ecosystems. In an effort to address this issue, many studies have been conducted to develop prediction models that can provide estimates of air quality levels in Jakarta City. Two modeling models that can be used in this context are Support Vector Machine (SVM) and K-Nearest Neighbors (KNN). This study aims to compare the performance of the SVM algorithm model with the KNN algorithm in the classification of air quality in Jakarta City. Data mining is the art and science of discovering knowledge, insights, and patterns in data based on the CRISP -DM (Cross Industry Standard Process For Data Mining) methodology. The data source in this study is the Jakarta air pollution standard index (ISPU) data obtained on the satudata.jakarta.go.id website. This study compares the accuracy of the Jakarta City air quality classification on the Support vector Machine (SVM) and K-Nearest Neighbor (KNN) algorithms using python. Comparison results are determined by the level of accuracy and score of the confusion matrix. Classification results are presented in the form of a Graphic User Interface with media interface. The comparison of classification algorithms on two models in data mining shows that the Support Vector Machine (SVM) algorithm is superior to K-Nearest Neighbor (KNN). This is evident from the higher accuracy rate in SVM, especially with the use of the Rbf kernel which reaches 97.05%, compared to KNN which has an accuracy of 94.74% with parameters p = 1 and k = 5. In addition, SVM also shows a higher correct prediction value compared to KNN. In a 1-year time span the overall quality may be quite good, with moderate quality at more than equal to 50 to 100. However, control is still needed for the unhealthy category areas.

Introduction: The presence of short oligonucleotide fragments that constitute the viral genome is unique, and the variation in their abundance among different species significantly influences the pathogenicity of viral strains, evasion of the immune system, and response to antiviral treatment. Data mining and the use of weighting models are effective methods for identifying these characteristic fragments. Given the importance of oligonucleotide fragments in the genomic sequence of the hepatitis C virus (HCV) concerning resistance to interferon treatment and evasion of the host's innate immune system, this study aims to identify these characteristic fragments using weighting models applied to the HCV genomic sequence. Method: In this study, we employed ten different weighting models for the first time to identify short oligonucleotide fragments, including distinguishing dinucleotide, trinucleotide, and tetranucleotide sequences, between two groups of individuals infected with HCV who exhibit different therapeutic responses to interferon (INF) treatment. Ten weighting algorithms were executed and evaluated based on the relative abundance of short oligonucleotide fragments constituting the HCV genome in genotypes 1a, 1b, and 2b from two groups of patients resistant and sensitive to interferon treatment using Rapid Miner software . Results: Several oligonucleotide fragments, including UU, UA, and UC among dinucleotide sequences, and GUA, CUA, and CGUA among trinucleotide and tetranucleotide sequences, showed significant differences between the two groups. Previous studies indicate that the presence of these sequences plays a crucial role in the interferon response against viruses. Conclusion: The results of this study suggest that employing various weighting models in data mining can help predict important genomic indicators of the virus that are effective in evading the host's immune system and antiviral treatments at early stages. These indicators can subsequently be targeted in experimental studies for drug and vaccine design .

Data Mining: Accuracy and Error Measures for Classification and Prediction

Constrained optimization of data-mining problems to improve model performance: A direct-marketing application

The emerging standard for the platform- and system-independent representation of data mining models, PMML (Predictive Model Markup Language), is currently supported by a number of knowledge discovery support engines (KDDSE). The primary purpose of the PMML standard is to separate model generation from model storage in order to enable users to view, post-process, and utilize data mining models independently of the KDDSE that generated the model. In this chapter, an architectural framework for collaborative data mining and decision support that utilizes PMML is described. Important parts of such a general framework are visualization and evaluation methods for data mining models. Two such systems, called VizWiz and PEAR, are described in some detail.

Data privacy is a major threat to the widespread deployment of data grids in domains such as health care and finance. We propose a novel technique for obtaining knowledge - by way of a data mining model - from a data grid, while ensuring that the privacy is cryptographically secure. To the best of our knowledge, all previous approaches for solving this problem fail in the presence of malicious participants. In this paper we present an algorithm which, in addition to being secure against malicious members, is asynchronous, involves no global communication patterns, and dynamically adjusts to new data or newly added resources. As far as we know, this is the first privacy-presenting data mining algorithm to possess these features in the presence of malicious participants. Simulations of thousands of resources prove that our algorithm quickly converges to the correct result. The simulations also prove that the effect of the privacy parameter on the convergence time is logarithmic.

NMF and PLSI are two state-of-the-art unsupervised learning models in data mining, and both are widely used in many applications. References have shown the equivalence between NMF and PLSI under some conditions. However, a new issue arises here: why can they result in different solutions since they are equivalent? or in other words, their algorithm differences are not studied intensively yet. In this note, we explicitly give the algorithm differences between PLSI and NMF. Importantly, we find that even if starting from the same initializations, NMF and PLSI may converge to different local solutions, and the differences between them are born in the additional constraints in PLSI though NMF and PLSI optimize the same objective function.

With the increasing complexity of financial markets, it is gradually becoming a research hotspot in the field of accurate stock market prediction. The ant lion optimization algorithm, due to its excellent global search ability and the advantages of machine learning models in data mining, aims to further improve the accuracy of stock market prediction. Therefore, this paper will explore how to combine the ant lion optimization algorithm with machine learning models to achieve effective prediction of stock market trends. First, a stock relationship graph is constructed using graph neural networks, with stocks as nodes and relationships between stocks as edges. The node embedding technique of graph neural networks is used to extract feature representations of stocks. Then, the ant lion algorithm is used to optimize the parameters of the graph neural network, enabling it to better fit the historical data of the stock market. Finally, the trained model is used to predict the future trend of the stock market. Through experimental verification, it has been found that the proposed method has high fitness during the training process and short training time; the loss value of ant lion’s optimized machine learning algorithm is relatively small, and the model prediction accuracy is high. Applying this method to Apple, Facebook, and Tesla results in higher strategic trading returns. This method improves the accuracy of stock market prediction by optimizing machine learning algorithms with ant lions, providing investors with more reliable decision support.