An efficient algorithm for mining closed itemsets.

The current and future business processes would be service oriented. The required service is selected using discovery process. The discovery process uses in-memory parse tree construction or callback based document parsing. Existing service discovery systems stores both functional attributed and QoS attributed in tree data structures and they are time consuming, space consuming, requires frequent parse tree construction in various location, difficult to replicate and distribute parse tree and replicated parse trees are inconsistent. This paper proposes an efficient methodology to improve the performance of in memory parse tree discovery process over extended web service architecture. The system uses two parallel discovery algorithms, Synchronous and Asynchronous algorithms for service discovery, both algorithms uses the tree and indexed data structure for service discovery and the later also uses a bitmap table to verify functional requirements. The services are selected and ranked based on user request using Coefficient of Variance. The proposed algorithms are detailed in this paper. The experimental results are conducted on benchmark dataset are discussed. The proposed system ensures availability, consistency, effective space utilization, eases replication and distribution and improves response time for service discovery by 30%.

A scalable decision tree system and its application in pattern recognition and intrusion detection

Most retail companies have implemented computer-based information systems for recording sales transaction data. In the implementation of information systems, the data collected in the database is processed limited to making reports such as sales reports and inventory reports. Database generated from computer-based information systems can be further processed to obtain more valuable information. One strategy for using sales transaction data is to analyze consumer spending patterns. Consumer spending patterns can be in the form of associations of items that are often purchased simultaneously. The association between goods can be determined using the frequent itemset search technique. The Fp-growth algorithm is an algorithm that can be used to determine frequent itemsets in a data set. This article describes the results of implementing the FP-Growth algorithm in the consumer shopping pattern analysis application. The resulting shopping pattern is in the form of goods that are often purchased simultaneously by consumers. From the results of the application of the fp-growth algorithm, it was found that the minimum value of support had an effect, namely the smaller the input value of support, the more pairs of items were obtained. The application of the FP-Growth algorithm in determining frequent itemsets in association data mining can find customer spending habits in buying goods simultaneously.

This dissertation contributes with novel predictive modelling approaches to data-driven computational psychiatry and offers alternative analyses frameworks to the standard statistical analyses in psychiatric research. In particular, this document advances research in medical data mining, especially psychiatry, via two phases. In the first phase, this document promotes research by proposing synergistic machine learning and statistical approaches for detecting patterns and developing predictive models in clinical psychiatry data to classify diseases, predict treatment outcomes or improve treatment selections. In particular, these data-driven approaches are built upon several machine learning techniques whose predictive models have been pre-processed, trained, optimised, post-processed and tested in novel computationally intensive frameworks. In the second phase, this document advances research in medical data mining by proposing several novel extensions in the area of data classification by offering a novel decision tree algorithm, which we call PIDT, based on parameterised impurities and statistical pruning approaches toward building more accurate decision trees classifiers and developing new ensemblebased classification methods. In particular, the experimental results show that by building predictive models with the novel PIDT algorithm, these models primarily led to better performance regarding accuracy and tree size than those built with traditional decision trees. The contributions of the proposed dissertation can be summarised as follow. Firstly, several statistical and machine learning algorithms, plus techniques to improve these algorithms, are explored. Secondly, prediction modelling and pattern detection approaches for the first-episode psychosis associated with cannabis use are developed. Thirdly, a new computationally intensive machine learning framework for understanding the link between cannabis use and first-episode psychosis was introduced. Then, complementary and equally sophisticated prediction models for the first-episode psychosis associated with cannabis use were developed using artificial neural networks and deep learning within the proposed novel computationally intensive framework. Lastly, an efficient novel decision tree algorithm (PIDT) based on novel parameterised impurities and statistical pruning approaches is proposed and tested with several medical datasets. These contributions can be used to guide future theory, experiment, and treatment development in medical data mining, especially psychiatry.

Mining Frequent Pattern Using FP Tree Algorithm for Optimised Time Complexity In Different Datasets

Preparing a data set for analysis is generally the most time consuming task in a data mining project, requiring many complex SQL queries, joining tables and aggregating columns. Existing SQL aggregations have limitations to prepare data sets because they return one column per aggregated group. In general, a significant manual effort is required to build data sets, where a horizontal layout is required. We propose simple, yet powerful, methods to generate SQL code to return aggregated columns in a horizontal tabular layout, returning a set of numbers instead of one number per row. This new class of functions is called horizontal aggregations. Horizontal aggregations build data sets with a horizontal denormalized layout (e.g. point-dimension, observation-variable, instance-feature), which is the standard layout required by most data mining algorithms. We propose three fundamental methods to evaluate horizontal aggregations: CASE: Exploiting the programming CASE construct; SPJ: Based on standard relational algebra operators (SPJ queries); PIVOT: Using the PIVOT operator, which is offered by some DBMSs. Experiments with large tables compare the proposed query evaluation methods. Our CASE method has similar speed to the PIVOT operator and it is much faster than the SPJ method. In general, the CASE and PIVOT methods exhibit linear scalability, where as the SPJ method does not. In a relational database, especially with normalized tables, a significant effort is required to prepare a summary data set that can be used as input for a data mining or statistical algorithm. Most algorithms require as input a data set with a horizontal layout, with several Records and one variable or dimension per column. That is the case with models like clustering, classification, regression and PCA; consult. Each research discipline uses different terminology to describe the data set. In data mining the common terms are point-dimension. Statistics literature generally uses observation-variable. Machine learning research uses instance-feature. This article introduces a new class of aggregate functions that can be used to build data sets in a horizontal layout (denormalized with aggregations), automating SQL query writing and extending SQL capabilities. We show evaluating horizontal aggregations is a challenging and interesting problem and we introduced alternative methods and optimizations for their efficient evaluation. II. MOTIVATION As mentioned above, building a suitable data set for data mining purposes is a time- consuming task. This task generally requires writing long SQL statements or customizing SQL Code if it is automatically generated by some tool. There are two main ingredients in such SQL code: joins and aggregations; we focus on the second one. The most widely- known aggregation is the sum of a column over groups of rows. Some other aggregations return the average, maximum, minimum or row count over groups of rows. There exist many aggregations functions and operators in SQL. Unfortunately, all these aggregations have limitations to build data sets for data mining purposes. The main reason is that, in general, data sets that are stored in a relational database (or a data warehouse) come

Frequent itemset mining is an important problem in the data mining area. Extensive efforts have been devoted to developing efficient algorithms for mining frequent itemsets. However, not much attention is paid on managing the large collection of frequent itemsets produced by these algorithms for subsequent analysis and for user exploration. In this paper, we study three structures for indexing and querying frequent itemsets: inverted files, signature files and CFP-tree. The first two structures have been widely used for indexing general set-valued data. We make some modifications to make them more suitable for indexing frequent itemsets. The CFP-tree structure is specially designed for storing frequent itemsets. We add a pruning technique based on length-2 frequent itemsets to make it more efficient for processing superset queries. We study the performance of the three structures in supporting five types of containment queries: exact match, subset/superset search and immediate subset/superset search. Our results show that no structure can outperform other structures for all the five types of queries on all the datasets. CFP-tree shows better overall performance than the other two structures.

The identification of useful patterns from a medical dataset benefits early diagnosis. Classification techniques in data mining are famous for medical data prediction. Decision Tree (J48) and Navie Bayes are the most popular and frequently used algorithms in the field of prediction analysis. The efficiency of such prediction algorithms can be improved with better pre-processing approaches. This paper aims at finding optimum features using permutation combination of input data attributes for improving the classifier accuracy. The Pima Indian Diabetes Dataset from UCI repository is used for experimentation. The performance of J48 and Navie Bayes has been tested for different combination of features. Keywords: Diabetes; J48; Navie Bayes; Permutation

Diabetes is the common chronic disease and a major health challenge in all population. Gestational diabetes mellitus (GDM) is a type of diabetes developed in women at the time of pregnancy. We present a Data mining (DM) approach to identify the risk factors of Gestational diabetes mellitus (GDM) using different data mining techniques. Dataset used for analysis contains the details of the pregnant women admitted the local hospital of Mysuru, India. The data mining techniques used are k-means clustering, J48 Decision Tree, Random-Forest and Naive-Bayes classifier. Classification accuracy is enhanced by using feature subset selection wrapper approach. Data imbalanced problem is handled by using Synthetic Minority Over-sampling Technique (SMOTE). The performances of the algorithms have been measured and compared in terms of Accuracy.

Currently several thousands of objects are being tracked in the MEO and GEO regions through optical means. The problem faced in this framework is that of Multiple Target Tracking (MTT). In this context both, the correct associations among the observations and the orbits of the objects have to be determined. The complexity of the MTT problem is defined by its dimension S. The number S corresponds to the number of fences involved in the problem. Each fence consists of a set of observations where each observation belongs to a different object. The S ≥ 3 MTT problem is an NP-hard combinatorial optimization problem. There are two general ways to solve this. One way is to seek the optimum solution, this can be achieved by applying a branch-and- bound algorithm. When using these algorithms the problem has to be greatly simplified to keep the computational cost at a reasonable level. Another option is to approximate the solution by using meta-heuristic methods. These methods aim to efficiently explore the different possible combinations so that a reasonable result can be obtained with a reasonable computational effort. To this end several population-based meta-heuristic methods are implemented and tested on simulated optical measurements. With the advent of improved sensors and a heightened interest in the problem of space debris, it is expected that the number of tracked objects will grow by an order of magnitude in the near future. This research aims to provide a method that can treat the correlation and orbit determination problems simultaneously, and is able to efficiently process large data sets with minimal manual intervention.

Dengue fever is widespread disease in the tropical areas caused by bite of female Eddie mosquito. Pakistan has been victim of this rapidly growing disease since last few years. The world health organization identified the four types of dengue fever. The experts are facing the problem of misdiagnosis of dengue fever. The Tests needed for empirical Classification of Dengue Fever takes a lot of time and money especiallyin epidemic situation in a country with limited resources. Therefore, we have used data mining techniques for the efficient classification of the dengue fever Type. The decision tree learning algorithm has been used as a classification Model. We performed two experiments using decision tree. The first general experiment shows the accuracy of 99.44%. It prunes the attributes which classify the dengue fever on the basis of the values in the dataset. The Second experiment classifies the dengue fever on the basis of expert weighted attributes, which are used in the classification on the basis of Minimum Cost and resource availability. The accuracy of this Model is still high 98.62%. We compared the performance in term of Type II error. We found that the Type II error is very low in second experiment

Analytic tools are useful for detecting patterns in education data and providing insights about student performance and learning. This study compared six supervised learning algorithms (linear regression, ridge regression, the lasso, regression trees, random forests regression, gradient boosted regression) and identified features important for predicting student performance. The dataset consisted of N=1044 observations from two secondary schools in Portugal (UCI-MLR, Cortez & Silva, 2008). Performance was assessed by final grades (range: 0-20) in two courses, mathematics and Portugese. The models were fit to training data with 27 independent variables and evaluated on a testing subset. Overall, performance was lower for students in mathematics than Portugese. The models selected a similar set of variables as important for predicting performance: mother's education level, student plans for higher education, and weekly study time were positively related to predicted performance, whereas course subject, school educational support, and romantic relationships were associated with decreased student performance. The models differed in the number, weighting, order and importance given to predictor variables. Linear regression provided a model with 13 predictors. Ridge regression shrank the coefficient estimates toward zero; the lasso performed variables selection for a model with 20 predictors. There was a tradeoff between model complexity and interpretability. The single pruned regression tree provided a simple, interpretable non-linear model with four features. Random forests regression and gradient boosting reduced overfitting, but were more difficult to interpret. Advantages and limitations of the different models are discussed. Applications for educational data mining (EDM) and learning analytics (LA) are considered.

This study focuses on predicting which firms will increase capital by issuing new stocks in the near future. Many stakeholders, including banks, credit rating agencies and investors, performs a variety of analyses for firms' growth, profitability, stability, activity, productivity, etc., and regularly report the firms' financial analysis indices. In the paper, we develop predictive models for rights issues using these financial analysis indices and data mining techniques. This study approaches to building the predictive models from the perspective of two different analyses. The first is the analysis period. We divide the analysis period into before and after the IMF financial crisis, and examine whether there is the difference between the two periods. The second is the prediction time. In order to predict when firms increase capital by issuing new stocks, the prediction time is categorized as one year, two years and three years later. Therefore Total six prediction models are developed and analyzed. In this paper, we employ the decision tree technique to build the prediction models for rights issues. The decision tree is the most widely used prediction method which builds decision trees to label or categorize cases into a set of known classes. In contrast to neural networks, logistic regression and SVM, decision tree techniques are well suited for high-dimensional applications and have strong explanation capabilities. There are well-known decision tree induction algorithms such as CHAID, CART, QUEST, C5.0, etc. Among them, we use C5.0 algorithm which is the most recently developed algorithm and yields performance better than other algorithms. We obtained data for the rights issue and financial analysis from TS2000 of Korea Listed Companies Association. A record of financial analysis data is consisted of 89 variables which include 9 growth indices, 30 profitability indices, 23 stability indices, 6 activity indices and 8 productivity indices. For the model building and test, we used 10,925 financial analysis data of total 658 listed firms. PASW Modeler 13 was used to build C5.0 decision trees for the six prediction models. Total 84 variables among financial analysis data are selected as the input variables of each model, and the rights issue status (issued or not issued) is defined as the output variable. To develop prediction models using C5.0 node (Node Options: Output type = Rule set, Use boosting = false, Cross-validate = false, Mode = Simple, Favor = Generality), we used 60% of data for model building and 40% of data for model test. The results of experimental analysis show that the prediction accuracies of data after the IMF financial crisis (59.04% to 60.43%) are about 10 percent higher than ones before IMF financial crisis (68.78% to 71.41%). These results indicate that since the IMF financial crisis, the reliability of financial analysis indices has increased and the firm intention of rights issue has been more obvious. The experiment results also show that the stability-related indices have a major impact on conducting rights issue in the case of short-term prediction. On the other hand, the long-term prediction of conducting rights issue is affected by financial analysis indices on profitability, stability, activity and productivity. All the prediction models include the industry code as one of significant variables. This means that companies in different types of industries show their different types of patterns for rights issue. We conclude that it is desirable for stakeholders to take into account stability-related indices and more various financial analysis indices for short-term prediction and long-term prediction, respectively. The current study has several limitations. First, we need to compare the differences in accuracy by using different data mining techniques such as neural networks, logistic regression and SVM. Second, we are required to develop and to evaluate new prediction models including variables which research in the theory of capital structure has mentioned about the relevance to rights issue.

In this thesis the use of widefield imaging techniques and VLBI observations with a limited number of antennas are explored. I present techniques to efficiently and accurately image extremely large UV datasets. Very large VLBI datasets must be reduced into multiple, smaller datasets if today’s imaging algorithms are to be used to image them. I present a procedure for accurately shifting the phase centre of a visibility dataset. This procedure has been thoroughly tested and found to be almost two orders of magnitude more accurate than existing techniques. Errors have been found at the level of one part in 1.1 million. These are unlikely to be measurable except in the very largest UV datasets. Results of a four-station VLBI observation of a field containing multiple sources are presented. A 13 gigapixel image was constructed to search for sources across the entire primary beam of the array by generating over 700 smaller UV datasets. The source 1320+299A was detected and its astrometric position with respect to the calibrator J1329+3154 is presented. Various techniques for phase calibration and imaging across this field are explored including using the detected source as an in-beam calibrator and peeling of distant confusing sources from VLBI visibility datasets. A range of issues pertaining to wide-field VLBI have been explored including; parameterising the wide-field performance of VLBI arrays; estimating the sensitivity across the primary beam both for homogeneous and heterogeneous arrays; applying techniques such as mosaicing and primary beam correction to VLBI observations; quantifying the effects of time-average and bandwidth smearing; and calibration and imaging of wide-field VLBI datasets. The performance of a computer cluster at the Istituto di Radioastronomia in Bologna has been characterised with regard to its ability to correlate using the DiFX software correlator. Using existing software it was possible to characterise the network speed particularly for MPI applications. The capabilities of the DiFX software correlator, running on this cluster, were measured for a range of observation parameters and were shown to be commensurate with the generic performance parameters measured. The feasibility of an Italian VLBI array has been explored, with discussion of the infrastructure required, the performance of such an array, possible collaborations, and science which could be achieved. Results from a 22 GHz calibrator survey are also presented. 21 out of 33 sources were detected on a single baseline between two Italian antennas (Medicina to Noto). The results and discussions presented in this thesis suggest that wide-field VLBI is a technique whose time has finally come. Prospects for exciting new science are discussed in the final chapter.

In general, obesity is calculated using the BMI, which is found to have limitations of use when we insert only one type of non-quantitative data as a crisp value, which is commonly used, or even create an obesity forecasting model using data mining techniques. If there is an input of weight and height data as a forecasting factor in various types of data, such as crisp value, estimated value and Fuzzy Linguistic Term value, etc. The algorithm will not possibly work under the data mining technique with such input data. This research presented the solution for this problem in order to enable data mining techniques under limitations of inserted data, including crisp value, estimated value and fuzzy linguistic term value, for forecasting obesity. The researchers created a fuzzy database system in the form of Conceptual Meta Schema to support fuzzy attributed data storage. In addition, there were 3 types of fuzzy attribute matching as follows; Type 1 was matching crisp value ​​with fuzzy linguistic term value, Type 2 was matching estimated value with fuzzy linguistic term value, and Type 3 was matching fuzzy linguistic term value with fuzzy linguistic term value, respectively. The attribute node from these fuzzy attributes would be selected to be used in the work of data mining techniques. Therefore, in this research, the researchers presented and compared the fuzzy model of working under the algorithm of data mining techniques in the form called fuzzy neural network and fuzzy decision tree techniques. The research result, it was found that the appropriate models for predicting obesity were fuzzy neural network algorithm with the neural network structure being 31-3-3, momentum at 0.2, learning rate at 0.3, under dividing data to test with cross-validation folds = 10 yielded accuracy value, precision value, recall value and f-measure at 84.3%, 82.7%, 84.3%, and 82.8%, respectively. Keywords: Obesity, Fuzzy Neural Net, Fuzzy Decision Tree, Fuzzy Attribute Matching, Forecasting Model