Experimental investigation of the predictive capabilities of data driven modeling techniques in hydrology - Part 2: Application

Abstract. A comprehensive data driven modeling experiment is presented in a two-part paper. In this first part, an extensive data-driven modeling experiment is proposed. The most important concerns regarding the way data driven modeling (DDM) techniques and data were handled, compared, and evaluated, and the basis on which findings and conclusions were drawn are discussed. A concise review of key articles that presented comparisons among various DDM techniques is presented. Six DDM techniques, namely, neural networks, genetic programming, evolutionary polynomial regression, support vector machines, M5 model trees, and K-nearest neighbors are proposed and explained. Multiple linear regression and naïve models are also suggested as baseline for comparison with the various techniques. Five datasets from Canada and Europe representing evapotranspiration, upper and lower layer soil moisture content, and rainfall-runoff process are described and proposed, in the second paper, for the modeling experiment. Twelve different realizations (groups) from each dataset are created by a procedure involving random sampling. Each group contains three subsets; training, cross-validation, and testing. Each modeling technique is proposed to be applied to each of the 12 groups of each dataset. This way, both prediction accuracy and uncertainty of the modeling techniques can be evaluated. The description of the datasets, the implementation of the modeling techniques, results and analysis, and the findings of the modeling experiment are deferred to the second part of this paper.

Estimation of scour downstream of a ski-jump bucket has been a topic of research among hydraulic engineers. For estimation of scour downstream of ski jump bucket, several empirical models are in use. In recent years, there has been emphasis to develop models which are capable of producing scour with high accuracy. Use of Artificial Neural Network (ANN) approach to model depth, width and length of scour hole indicates that performance of ANN models is far better than existing empirical models. At present, use of Support Vector Machines (SVMs) and M5 Pruned Model Tree are being considered in different disciplines to further improve upon the performance of ANN models as a potential alternate. With this in view, the present study deals with the development of regression models for computing various parameters of scour hole using SVMs and M5 Model Tree. A comparative evaluation of the performance of ANN versus SVMs and M5 Model Tree clearly shows that SVMs and M5 Model Tree can prove more useful than ANN models in estimation of scour downstream of a ski jump bucket. Further, M5 model tree offers explicit expressions for use by design engineers.

Although the nonlinear power form model structure is widely accepted by practitioners in the flood regionalization modelling, there is a lack of studies on whether there is a room for further improvement, and if the answer is yes, what should be done to explore alternative model structures. A framework is proposed in this study towards investigating this issue by the following steps: (i) a universal data‐driven model is utilized to see if there is a room for improvement compared with the conventional model, and (ii) if improvement is achieved, this means that there should exist more effective model structures than the current form. However, because the universal data‐driven models are usually opaque, more explicit model structures should be explored, which are convenient for practical usage. In this study, the proposed framework is applied in a case study using the catchment characteristics from the Flood Estimation Handbook in conjunction with the gamma test, support vector machine (SVM) and genetic programming (GP). First, the gamma test is used for the purpose of input variables selection where no model structure needs to be defined as a priori, and therefore, the result can be applied to any model structures for model building. Second, an SVM, which is a powerful data‐driven nonlinear model capable of modelling a variety of nonlinear systems, is applied to the index flood model for the first time. Once the best model is determined using those two data‐driven tools, GP is employed to find an alternative model structure. As the SVM is not formulated for producing explicit model functional form, the GP offers an advantage at this point where it can infer an explicit mathematical model functional form. Copyright © 2012 John Wiley & Sons, Ltd.

In recent years, the urban impervious surface has been recognized as a key quantifiable indicator in assessing urbanization impacts on environmental and ecological conditions. A surge of research interests has resulted in the estimation of urban impervious surface using remote sensing studies. The objective of this paper is to examine and compare the effectiveness of two algorithms for extracting impervious surfaces from Landsat TM imagery; the multilayer perceptron neural network (MLPNN) and the support vector machine (SVM). An accuracy assessment was performed using the high-resolution WorldView images. The root mean square error (RMSE), the mean absolute error (MAE), and the coefficient of determination (R(2)) were calculated to validate the classification performance and accuracies of MLPNN and SVM. For the MLPNN model, the RMSE, MAE, and R(2) were 17.18%, 11.10%, and 0.8474, respectively. The SVM yielded a result with an RMSE of 13.75%, an MAE of 8.92%, and an R2 of 0.9032. The results indicated that SVM performance was superior to that of MLPNN in impervious surface classification. To further evaluate the performance of MLPNN and SVM in handling the mixed-pixels, an accuracy assessment was also conducted for the selected test areas, including commercial, residential, and rural areas. Our results suggested that SVM had better capability in handling the mixed-pixel problem than MLPNN. The superior performance of SVM over MLPNN is mainly attributed to the SVM's capability of deriving the global optimum and handling the over-fitting problem by suitable parameter selection. Overall, SVM provides an efficient and useful method for estimating the impervious surface. (C) 2011 Society of Photo-Optical Instrumentation Engineers (SPIE). [DOI: 10.1117/1.3539767]

Detection of asphyxia in infant at an early stage is crucial to reduce the rate of infant morbidity. The information regarding asphyxia can be extracted from infant cry signals using support vector machine (SVM) combined with effective feature selection methods such as principal component analysis (PCA) or orthogonal least square (OLS). The performance of SVM in recognizing infant cry with asphyxia after undergone comprehensive identification of optimal parameters at the feature extraction and classification stages has not been reported. This paper describes the two stages of optimal parameter identification; at Mel-frequency Cepstral coefficient (MFCC) analysis stage, SVM with and without employing the PCA and OLS stages, and the performance of the SVM in recognizing infant cry with asphyxia resulted from all levels of optimal parameters identification. The SVM was first optimized after performing MFCC analysis to find the optimum parameters. Two types of kernels were used, the polynomial and RBF kernels. The subsequent SVM optimizations were conducted after PCA and OLS were employed. In the PCA, the significant features were selected using eigenvalue-one-criterion (EOC), cumulative percentage variance (CPV) and the Scree test (SCREE). The SVM performance was evaluated based on classification accuracy and computation time. The experimental results have shown that the optimized SVM was able to recognize the asphyxiated infant cry with an accuracy of 94.84% and computation time of 1.98s using PCA with EOC and RBF kernel.

Support vector machine (SVM) is a statistical learning tool developed to a more complex concept of structural risk minimization. SVM is playing an increasing role in applications to detection problems in statistical signal processing and communication systems. In this paper, SVM is applied to the detection of root-mean-square-gain combining (RMSGC) diversity signals in single-input-multiple-output (SIMO) systems, in the presence of channel noise characterized as partially developed Rician multipath fading and interference noise generalized as additive color Gaussian noise (ACGN). The structure and performance of SVM in terms of the bit error rate (BER) metric is analysed for these advanced stochastic noise models. The performance of SVM is then compared to conventional SIMO signaling with optimal model-based detection. We show that the SVM performance is superior to that of the maximum likelihood detector for all the selected pre-detection diversity gain combining schemes.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Numerical and computational analyses surrounding the behavior of the bearing capacity of soils near or adjacent to slopes have been of great importance in earthwork constructions around the globe due to its unique nature. This phenomenon is encountered on pavement vertical curves, drainages, and vertical infrastructure foundations. In this work, multiple data were collected on the soil and footing interface parameters, which included width of footing, depth of foundation, distance of slope from the footing edge, soil bulk density, slope and frictional angles, and bearing capacity factors of cohesion and overburden pressure determined for the case of a foundation on or adjacent to a slope. The genetic programming (GP), evolutionary polynomial regression (EPR), and artificial neural network (ANN) intelligent techniques were employed to predict the ultimate bearing capacity of footing on or adjacent to a slope. The performance of the models was evaluated as well as compared their accuracy and robustness with the findings of Prandtl. The results were observed to show the superiority of GP, EPR, and ANN techniques over the computational works of Prandtl. In addition, the ANN outclassed the other artificial intelligence methods in the exercise.

Utilization of SVM, LSSVM and GP for Predicting the Medical Waste Generation

The prediction of wastes generated in the hospital will help their management for several activities like storage, transport and disposing. This chapter adopts Support Vector Machine (SVM), Least Square Support Vector Machine (LSSVM) and Genetic Programming (GP) in order to estimate the rate of medical waste generation. In the event of predicting the rate, type of hospital, capacity and bed occupancy has been used as inputs of SVM, LSSVM and GP. SVM is based on statistical learning theory, which provides an elegant tool for nonlinear system modeling. LSSVM is the re-formulation to the general SVM. GP, a best part of evolutionary algorithm and also the specification of Genetic Algorithm (GA). These SVM, LSSVM and GP have been used as the regression techniques. The results show the performance of the developed SVM, LSSVM and GP models were elegant and outstanding.

Lithology prediction by support vector classifiers using inverted seismic attributes data and petrophysical logs as a new approach and investigation of training data set size effect on its performance in a heterogeneous carbonate reservoir

This chapter explains the Artificial Intelligence (AI) techniques in terms of Artificial Neural Networks (ANNs), fuzzy logic, expert systems, machine learning, Genetic Programming (GP), Evolutionary Polynomial Regression (EPR), and Support Vector Machine (SVM); the AI applications in modern education; the AI applications in software engineering development; the AI applications in Content-Based Image Retrieval (CBIR); and the multifaceted applications of AI in the digital age. AI is a branch of science which deals with helping machines find the suitable solutions to complex problems in a more human-like manner. AI technologies bring more complex data-analysis features to the existing applications in various industries and greatly contribute to management's organization, planning, and controlling operations, and will continue to do so with more frequency as programs are refined.

A data-driven artificial neural network (ANN) model and a data-driven evolutionary polynomial regression (EPR) model are here used to set up two real-time crisp discharge forecasting models whose crisp parameters are estimated through the least-square criterion. In order to represent the total uncertainty of each model in performing the forecast, their parameters are then considered as grey numbers. Comparison of the results obtained through the application of the two models to a real case study shows that the crisp models based on ANN and EPR provide similar accuracy for short forecasting lead times; for long forecasting lead times, the performance of the EPR model deteriorates with respect to that of the ANN model. As regards the uncertainty bands produced by the grey formulation of the two data-driven models, it is shown that, in the ANN case, these bands are on average narrower than those obtained by using a standard technique such as the Box–Cox transformation of the errors; in the EPR case, these bands are on average larger. These results therefore suggest that the performance of a grey data-driven model depends on its inner structure and that, for the specific models here considered, the ANN is to be preferred.

Accurate estimation of river flow can have a significant importance in water resources management. In this study, Genetic programming (GP) and Support Vector Machine (SVM) methods were used to forecast daily discharge of Barandoozchay River. The daily discharge data of Barandoozchay River measured at the Dizaj hydrometric station during 2007 to 2011 was used for modeling, which 80% of the data used for training and remaining 20% used for testing of models. The results showed that in the both of considered methods, the models including discharges of one, two and three days ago had higher accuracy in verification step and the accuracy of models decreased with increasing discharge values. Comparing the performance of GP and SVM methods indicated that, however the accuracy of the GP method with the R=0.978 and RMSE=1.66 (m3/s) was slightly more than SVM method with R=0.976 and RMSE=1.80 (m3/s), but the SVM is easier than GP method. Thus, the SVM method can be used as an alternative method in forecasting daily river discharge.

The scarcity of freshwater causes the necessity for water delineation of brackish water. Reverse osmosis (RO) is one of the popular strategies characterized with lower cost and simple processing procedure compared to the other desalination techniques. The current research is conducted to investigate the efficiency the RO process based on one-week advance prediction of total dissolved solids (TDS) and permeate flow rate for Sistan and Bluchistan provinces located in Iran region. The water parameters including pH, feed pressure temperature, and conductivity are used to construct the prediction matrix. A newly hybrid data-intelligence (DI) model called multilayer perceptron hybridized with particle swarm optimization (MLP-PSO) is developed for the investigation. The potential of the proposed MLP-PSO model is validated against two predominate DI models including support vector machine (SVM) and M5Tree (M5T) models. The results evidenced the potential of the proposed MLP-PSO model over the SVM and M5T models in predicting the TDS and permeate flow rate. In addition, the proposed model attained lower uncertainty for the simulated data. Overall, the feasibility of the hybridized MLP-PSO achieved remarkable predictability for the RO process.