Data Mining and Analytics for Construction Project Cost Data Management System

An evaluation and choice of learning algorithms is a current research area in data mining, artificial intelligence and pattern recognition, etc. Supervised learning is one of the tasks most frequently used in data mining. There are several learning algorithms available in machine learning field and new algorithms are being added in machine learning literature. There is a need for selecting the best suitable learning algorithm for a given data. With the information explosion of different learning algorithms and the changing data scenarios, there is a need of smart learning system. The paper shows one approach where past experiences learned are used to suggest the best suitable learner using 3 meta-features namely simple, statistical and information theoretic features. The system tests 38 UCI benchmark datasets from various domains using nine classifiers from various categories. It is observed that for 29 datasets, i.e., 76 % of datasets, both the predicted and actual accuracies directly match. The proposed approach is found to be correct for algorithm selection of these datasets. New proposed equation of finding classifier accuracy based on meta-features is determined and validated. The study compares various supervised learning algorithms by performing tenfold cross-validation paired t test. The work helps in a critical step in data mining for selecting the suitable data mining algorithm.

This research study analyzes the different data mining techniques used for the detection of credit card frauds. Additionally, different data mining techniques are properly described in this study. The steps of data mining are clearly explained in this study, which are helpful for recognizing the fraudulent activities. This study is discussing that Bayesian network, and decision tree are the effective techniques of data mining. The issues of data mining are also discussed in this study, which is creating performance issues, and user interaction issues in the financial institutions. Therefore, this study is providing the importance of RF algorithm in the determination of the credit card fraud. Data cleaning, and data visualization, therefore, the machine learning process is allowed to be developed with the support of the data mining process of credit cards. Clustering is the main process that is takes place in this data mining process. Additionally, the sequential pattern is being highlighted by this process, which helps to improve the fraud detection process of credit cards.

A survey on single and multi omics data mining methods in cancer data classification.

Because of huge impacts of "OMICS" technologies in life sciences, many researchers aim to implement such high throughput approach to address cellular and/or molecular functions in response to any influential intervention in genomics, proteomics, or metabolomics levels. However, in many cases, use of such technologies often encounters some cybernetic difficulties in terms of knowledge extraction from a bunch of data using related softwares. In fact, there is little guidance upon data mining for novices. The main goal of this article is to provide a brief review on different steps of microarray data handling and mining for novices and at last to introduce different PC and/or web-based softwares that can be used in preprocessing and/or data mining of microarray data. To pursue such aim, recently published papers and microarray softwares were reviewed. It was found that defining the true place of the genes in cell networks is the main phase in our understanding of programming and functioning of living cells. This can be obtained with global/selected gene expression profiling. Studying the regulation patterns of genes in groups, using clustering and classification methods helps us understand different pathways in the cell, their functions, regulations and the way one component in the system affects the other one. These networks can act as starting points for data mining and hypothesis generation, helping us reverse engineer.

This paper presents detailed technique and steps of data mining in sluice safety monitoring according to the monitoring data’s features. The tasks and objectives of data mining are defined and converted into the knowledge which can be realized by certain algorithm. The methods of data preprocessing is also discussed to eliminate or reduce redundant, noise, incomplete and inconsistent data. The examples of data preprocessing and data mining show how to apply the data mining technology in the sluice safety monitoring filed, which results indicate that the method presented in this paper is effective and can be used in other projects.

Extracting useful Web content is a major step in data mining. The Web content extraction process is very important for many technologies or uses as a preprocessing of many systems such as crawlers and indexers. Additionally, the extracted content is needed by the end users especially for blind and visually impaired users. It aims to extract useful and meaningful data from Webpages that are surrounded with various clutters such as advertisements and navigation menus. Many extraction algorithms are designed for English Language and perform less efficient and less accurate in Arabic language. In this paper, a bi-languages mining algorithm for extracting Web contents called BiLEx is presented. It extracts useful Web content from Arabic and English Webpages in the approximately same level of efficiency and accuracy. An experiment is made for 600 Webpages which are chosen randomly from 30 different Websites to test the proposed algorithm performance and efficiency. Results prove that BiLEx algorithm gives high precision, recall, and F1-measure for both Arabic and English Webpages.

Most algorithms for mining or indexing time series data do not operate directly on the original data, but instead they consider alternative representations that include transforms, quantization, approximation, and multi-resolution abstractions. Choosing the best representation and abstraction level for a given task/dataset is arguably the most critical step in time series data mining. In this paper, we investigate techniques to discover the natural intrinsic representation model, dimensionality and alphabet cardinality of a time series. The ability to discover these intrinsic features has implications beyond selecting the best parameters for particular algorithms, as characterizing data in such a manner is useful in its own right and an important sub-routine in algorithms for classification, clustering and outlier discovery. We will frame the discovery of these intrinsic features in the Minimal Description Length (MDL) framework. Extensive empirical tests show that our method is simpler, more general and significantly more accurate than previous methods, and has the important advantage of being essentially parameter-free.

Preprocessing is one of the steps in Data Mining, which involves Noise removal, Identification of outlier, Normalization, Data transformation, Handling missing values, etc. Missing value is a common problem in large datasets. Most frequently used method to handle missing values by statistical is discarding the instances with missing values. Sometime deletion of instances with missing values cause loss of essential information, which affects the performance of statistical and machine learning algorithms. This paper focuses on handling missing values using unsupervised learning techniques. Rough K-Means based missing value imputation was proposed and compared with K-Means, Fuzzy C-Means based imputation methods. The experimental analysis is carried out on two data sets Lung Cancer and Cleveland Heart data sets. The proposed method achieves the best accuracy for some of the datasets.

An incremental approach to attribute reduction from dynamic incomplete decision systems in rough set theory

An improved hybrid chameleon swarm algorithm for feature selection in medical diagnosis

Feature selection is a dimensionality reduction method known as a main step in data mining and machine learning. The aim of feature selection is to remove redundant and unrelated features. In recent years several feature selection methods based on graph theory and social networking techniques have been proposed .In this study, a feature selection approach based on multi objective PSO algorithm and social network techniques is presented. In the proposed method, Fisher score, node centrality and edge centrality are used to construct the fitness function in order to present a multi objective particle swarm optimization (PSO) approach. The proposed method run over a variety of datasets and the results are compared with the well-known filter-based feature selection methods. The results show that the proposed method is effective and the performance of the proposed method better than other methods in some cases.

Data quality has diverse dimensions, from which accuracy is the most important one. Data cleaning is one of the preprocessing steps in data mining which consists of detecting errors and repairing them. Noise is a common type of error, that occur in database. This paper proposes an automated method based on the k-means clustering for noise detection. At first, each attribute (Aj) is temporarily removed from data and the k-means clustering is applied to other attributes. Thereafter, the k-nearest neighbors is used in each cluster. After that a value is predicted for Aj in each record by the nearest neighbors. The proposed method detects noisy attributes using predicted values. Our method is able to identify several noises in a record. In addition, this method can detect noise in fields with different data types, too. Experiments show that this method can averagely detect 92% of the noises existing in the data. The proposed method is compared with a noise detection method using association rules. The results indicate that the proposed method have improved noise detection averagely by 13%.

Data clustering is an important step in data mining and machine learning. It is especially crucial to analyze the data structures for further procedures. Recently a new clustering algorithm known as ‘neutrosophic c-means’ (NCM) was proposed in order to alleviate the limitations of the popular fuzzy c-means (FCM) clustering algorithm by introducing a new objective function which contains two types of rejection. The ambiguity rejection which concerned patterns lying near the cluster boundaries, and the distance rejection was dealing with patterns that are far away from the clusters. In this paper, we extend the idea of NCM for nonlinear-shaped data clustering by incorporating the kernel function into NCM. The new clustering algorithm is called Kernel Neutrosophic c-Means (KNCM), and has been evaluated through extensive experiments. Nonlinear-shaped toy datasets, real datasets and images were used in the experiments for demonstrating the efficiency of the proposed method. A comparison between Kernel FCM (KFCM) and KNCM was also accomplished in order to visualize the performance of both methods. According to the obtained results, the proposed KNCM produced better results than KFCM.

The raw data obtained from the logs may be noisy, incomplete, and inconsistent that’s why data pre-processing is an essential step in data mining. The quality of data plays a vital role during the evaluation process. The results of the evaluation process primarily depend upon the quality of the data input. So, data pre-processing is the primary and most crucial step before knowledge discovery. This paper is based on two main steps- data pre-processing techniques and results after applied data pre-processing on mutual funds’ data. Data pre-processing transforms the raw data into a structured, understandable format. Moreover, data pre-processing performs not only the transformation of data but also makes it understandable according to need. It is mainly divided into four steps, i.e., data integration, data cleaning, data transformation, and data reduction. This paper takes the fifteen-year NAV data of twenty mutual funds for analysis propose. This paper explains several techniques of data pre-processing to transform the raw data into an understandable format.

Knowledge Discovery in Databases (KDD) is a data analysis process which, in contrast to conventional data analysis, automatically generates and evaluates very many hypotheses, deals with complex, i.e. large, high dimensional, multi relational, dynamic, or heterogeneous data, and produces understandable results for those who “own the data”. With these objectives, subgroup mining searches for hypotheses that can be supported or confirmed by the given data and that are represented as a specialization of one of three general hypothesis types: deviating subgroups, associations between two subgroups, and partially ordered sets of subgroups where the partial ordering usually relates to time. This paper gives a short introduction into the methods of subgroup mining. Especially the main preprocessing, data mining and postprocessing steps are discussed in more detail for two applications. We conclude with some problems of the current state of the art of subgroup mining.