Surrogate based EA for expensive optimization problems

Support Vector Machine (SVM) for regression has recently attracted growing research interest due to its obvious advantage such as nonlinear function approximation with arbitrary accuracy, and good generalization ability, unique and globally optimal solutions. An overview of the basic ideas underlying SVM for regression is given in this paper. In particular, new methods such as n-SVM, LS-SVM, weighted SVM and linear SVM, and optimization algorithms including decomposition method and SMO and incremental learning with fast computational speed and ease of implementation are concentrated as well. SVM for regression is an important and promising new direction in the area of nonlinear parameter identification, forecast, modeling and control.

With the wide application of support vector machine(SVM), the algorithm of using online learning for realizing regression had been developed furtherly. After the mathematical mode of direct support vector machine (DSVM) for regression was introduced which was of the learning capacity that was similar to least squares support vector machine but less complexity of computation, according to Cholesky factorization, the algorithm of incremental learning and decremental learning were designed for DSVM in this paper, through them online learning that based on time window for regression was realized. Experimental results of simulation through Mackey-Glass chaotic time series and pseudo periodic synthetic time series data set all indicate the feasibility of the learning algorithm which will be beneficial for SVM's application in depth.

Leaf area index (LAI) is an important biophysical trait for forest ecosystem and ecological modeling, as it plays a key role for the forest productivity and structural characteristics. The ground-based methods like the handheld optical instruments for predicting LAI are subjective, pricy and time-consuming. The advent of very high spatial resolutions multispectral data and robust machine learning regression algorithms like support vector machines (SVM) and artificial neural networks (ANN) has provided an opportunity to estimate LAI at tree species level. The objective of the this study was therefore to test the utility of spectral vegetation indices (SVI) calculated from the multispectral WorldView-2 (WV-2) data in predicting LAI at tree species level using the SVM and ANN machine learning regression algorithms. We further tested whether there are significant differences between LAI of intact and fragmented (open) indigenous forest ecosystems at tree species level. The study shows that LAI at tree species level could accurately be estimated using the fragmented stratum data compared with the intact stratum data. Specifically, our study shows that the accurate LAI predictions were achieved for Hymenocardia ulmoides using the fragmented stratum data and SVM regression model based on a validation dataset (R2Val = 0.75, RMSEVal = 0.05 (1.37% of the mean)). Our study further showed that SVM regression approach achieved more accurate models for predicting the LAI of the six endangered tree species compared with ANN regression method. It is concluded that the successful application of the WV-2 data, SVM and ANN methods in predicting LAI of six endangered tree species in the Dukuduku indigenous forest could help in making informed decisions and policies regarding management, protection and conservation of these endangered tree species.

Cuckoo search algorithm (CSA) is relatively a new optimization technique with less control parameters and strong exploration ability. Due to the random search associated with CSA, it requires large number of functional evaluations for obtaining optimal solution. An improved algorithm, named as improved global-best-guided CSA, is presented here based on the best solution of previous iteration for the optimal design of multiplierless two-dimensional recursive digital filters. The most important feature of the proposed algorithm is that it is completely self-adaptive with no tuning parameters, whereas in CSA the replacement factor needs to be adjusted. The proposed algorithm exhibits 52% improvement in fitness function evaluation (for p = 2) and the execution time is reduced by 56% in comparison with the existing algorithms. Further, the proposed algorithm has been tested for several benchmark problems and found to exhibit significant performance improvement.

In this paper, we present a novel learning method for SVM (Support Vector Machine) regression ensemble used in multi-pose face detection. Firstly, several view-specific SVM classifiers are trained by using corresponding positive and negative examples. And then, an ensemble mechanism (SVM regression) is used to combine the results from the view-specific SVCs (Support Vector Classifiers). Experimental results show that the detection accuracy of the ensemble is better than the view-specific SVCs. Moreover, the SVR ensemble does not need extra pose estimation process prior to the classification; it generates pose information in addition to its detection results.

Optimization of complex mechanics simulations with object-oriented software designs

Synthetic scene generation models depend on the repeated evaluation or iteration of often computationally expensive functions to create 'texture' or structure in a modeled natural scene. The computational burden of 'texture' function evaluation is dependent upon the spatial frequency response characteristics of the optical system through which the modeled scene is to be 'propagated.' All real 'cameras' utilize detector elements of finite size to transduce the resulting MXN pixels of the image. Two general situations arise in modeling the effects of cameras (sensors) on the imaged scene. The first is the 'oversampled' case where the Nyquist frequency of the detector spacing exceeds the cutoff of the optics transfer (aperture) function. The 'undersampled' case is when such Nyquist frequency is below the aperture cutoff spatial frequency, which results in aliasing. The application of brute force in the oversampled case results in MXN scene 'texture' function evaluations, while for the undersampled case, (mM)X(nN) (m, n > 1) points are required. Several computationally efficient methods significantly reduce the stated brute force computational burden for these two cases. This paper discusses three such methods. For the oversampled case, a correct and efficient method is 2D sample 'interpolation,' in the multirate digital signal processing sense. This method expressly avoids signal aliasing caused by simpler but inappropriate bilinear interpolation of the sparser set of (mM)+(nN) (m, n fractions < 1) scene samples onto the MXN imaged scene. The second and third techniques discussed are applicable to the undersampled case. Each relies upon MXN scene 'texture' function evaluations. The second technique extrapolates the frequency spectrum of the MXN grid with a synthetic spectrum beyond Nyquist which follows the 1/f(beta ) decrease in power typical of natural (fractal) textures. The third technique, 'fractal interpolation,' operates in the spatial domain where spatial detail, generated from the computed ('texture' function) MXN grid onto a larger (nM)X(nN) grid, is synthesized at the same local fractal dimension as that of the MXN 'undersampled' data. In both cases, the synthesized frequencies above the camera Nyquist are 'folded back' in the spectral domain to approximate the aliasing of spatial frequencies into the transduced image.

Towards an effective cross-task mental workload recognition model using electroencephalography based on feature selection and support vector machine regression

Nowadays the use of surrogate models (SMs) is becoming a common practice to accelerate the optimization phase of the design of microwave and millimeter wave devices. In order to further enhance the performances of the optimization process, the accuracy of the response provided by a SM can be improved employing a suitable output correction block, obtaining in this way a so-called enhanced surrogate model (ESM). In this paper a comparative study of three different techniques for building ESMs, i.e. Kriging, Support Vector Regression Machines (SVRMs) and Artificial Neural Networks (ANNs), applied to the modelling of substrate integrated waveguide (SIW) devices, is presented and discussed.

Data-driven regression methods are adept at modeling the spindle thermal elongation for error reduction purposes, but are limited in their generalization ability for the established model to take effect under different work conditions, where one prime cause is that the measurement errors and environmental disturbances would inevitably lead to successive outliers in the experimental training data, and the regression algorithms tend to over-fit to such outliers in the thermal datasets which are usually small. Here, an approach is proposed for the first time to detect and remove the suspected outliers in the spindle thermal data while preserve most of the informative data at the same time, so that the disturbances from the outliers can be minimized. The proposed approach for outliers detection is a combination of Boosting and SVR (BSOD), where the support vector machine for regression (SVR) is used as the weak learner in Boosting. Furthermore, the SVR, which is now widely considered as the most suitable regression tool for mapping the temperature-thermal elongation relationship for a spindle, is adopted to regression model the spindle thermal elongation, where the genetic algorithm (GA) is employed to determine the optimal parameter setting for the modeling. Effectiveness of the BSOD approach was verified using four experimental datasets from two different spindles of precision boring machines under different work conditions. The results showed that the BSOD approach reduced the mean squared error MSE of the genetic algorithm optimized support vector machine estimation for unseen thermal elongation datasets in two cases by 66.09% and 49.13%, respectively. This research provides a feasible way to enhance the modeling quality for the thermal elongation of spindles during working process.

Short-term stock price forecasting using kernel principal component analysis and support vector machines: the case of Casablanca stock exchange

This paper presents an approach in which a multi-objective evolutionary algorithm (MOEA) is coupled to a surrogate method in order to explore the search space in an efficient manner. A small comparative study among three surrogate methods is conducted: an artificial neural network (ANN), a radial basis function (RBF) and a support vector machine (SVM). The winner in this comparative study was the SVM. However, our results indicated that the spread of solutions achieved by our surrogate-based MOEA was poor. Thus, we decided to introduce a second phase to the algorithm in which it is hybridized with the rough sets in order to improve the spread of solutions and help to reach the true Pareto front. We show that our proposed hybrid approach only requires 2,000 fitness function evaluations in order to solve test problems with up to 30 decision variables.

This study proposes a hybrid deep-learning-metaheuristic framework with a bi-level architecture for road network design problems (NDPs). We train a graph neural network (GNN) to approximate the solution of the user equilibrium (UE) traffic assignment problem and use inferences made by the trained model to calculate fitness function evaluations of a genetic algorithm (GA) to approximate solutions for NDPs. Using three test networks, two NDP variants and an exact solver as benchmark, we show that on average, our proposed framework can provide solutions within 1.5 % gap of the best results in less than 0.5 % of the time used by the exact solution procedure. Our framework can be utilized within an expert system for infrastructure planning to determine the best infrastructure planning and management decisions under different scenarios. Given the flexibility of the framework, it can easily be adapted to many other decision problems that can be modeled as bi-level problems on graphs. Moreover, we foreseen interesting future research directions, thus we also put forward a brief research agenda for this topic. The key observation from our research that can shape future research is that the fitness function evaluation time using the inferences made by the GNN model was in the order of milliseconds, which points to an opportunity and a need for novel heuristics that 1) can cope well with noisy fitness function values provided by deep learning models, and 2) can use the significantly enlarged efficiency of the evaluation step to explore the search space effectively (rather than efficiently). This opens a new avenue for a modern class of metaheuristics that are crafted for use with AI-powered predictors.

Local spatial properties based image interpolation scheme using SVMs

This study investigates the use of machine learning algorithms to predict the plastic rotational capacity of reinforced concrete elements, a critical factor in assessing the response and vulnerability of structures under seismic loads. Traditional mathematical models developed to evaluate the rotational capacity of concrete elements are insufficient for accurately capturing the complex nonlinear behavior of structures under seismic loads. Machine learning offers a promising alternative, given its ability to model nonlinear relationships and improve predictive accuracy based on diverse datasets. The study compares the performance of six machine learning algorithms - Linear Regression (LR), Support Vector Machine for Regression (SVR), Least Squares Support Vector Machine (LS-SVM), Decision Tree (DT), Random Forest (RF), and Extreme Gradient Boosting (XGBoost) - using datasets of reinforced concrete columns and walls. Results reveal that SVR and XGBoost provide the most accurate predictions for columns, with R² values up to 91%, while XGBoost performs best for walls with a maximum R² of 78%. However, the accuracy of these models is limited by the availability and diversity of the datasets, particularly for elements prone to shear failure. This research underscores the potential of machine learning in seismic design and structural assessment, though it also highlights the need for more comprehensive data to enhance model accuracy.