Formal convergence analysis on deterministic [formula omitted]-regularization based mini-batch learning for RBF networks

This research investigated and optimized the separation of carbon dioxide (CO2) from natural gas in an adsorption column filled with grafted-beam nanofiber adsorbent. The main purpose of using ANN and RSM models in this manuscript is to compare these two methods in predicting the CO2 adsorption capacity. In other words, it was made to find a suitable model that has the highest agreement with the experimental data. Also, the other purpose of using the RSM model is to detect the optimized empirical conditions. Moreover, two common ANN models are applied in this work, including a multilayer perceptron (MLP) and radial basis function (RBF). The novelties of this work are explained as follows: (1) detecting the optimized synthesis factors of NF-PAN/PUGMA sorbent which possess the highest CO2 adsorption capacity with the help of response surface methodology (RSM), (2) studying the simultaneous interaction of synthesis parameters on the CO2 adsorption capacity with the help of both RSM and artificial neural networks (ANNs), (3) testing two types of ANN models including multilayer perceptron (MLP) and radial basis function (RBF) to predict the effect of monomer volume percentage and irradiation dose on the CO2 adsorption capacity. Indeed, finding the best model (ANN or RSM) can help engineers in practical applications predict CO2 adsorption capacity using NF-PAN/PUGMA under different conditions without incurring high-priced chemical materials, electricity, or human resources. The validation results were examined using correlation coefficients (R2) of RSM, RBF, and MLP models. The correlation coefficients for the RSM, RBF, and MLP models were 0.9910, 0.9949, and 0.9968, respectively. Additionally, the average absolute relative deviation (AARD) values for the RBF and MLP models were 0.00046512 and 0.00045511, respectively, indicating that the MLP model is better than the RBF model. To identify the optimal network structure, the trial-and-error method was conducted for MLP and RBF models. The number of neurons was found at 12 and 45 for MLP and RBF, respectively. The optimized effective parameters were obtained using RSM: 25.80% GMA, 66.45% amine, and an irradiation intensity of 28kGy.

An accelerated proximal gradient algorithm for source-independent waveform inversion

این مطالعه به منظور بررسی پیش بینی شاخص های عملکرد تولیدی در مرغان تخم گذار با استفاده از شبکه های عصبی مصنوعی و رگرسیون غیرخطی چندگانه انجام شد. بررسی بر روی اطلاعات چهار دوره متوالی پرورش در یک واحد پرورش مرغ تخم گذار صورت گرفت. روش های داده-کاوی شامل رگرسیون خطی و غیر خطی، شبکه عصبی پرسپترون سه لایه، شبکه عصبی پرسپترون چهار لایه و شبکه عصبی با تابع پایه ای شعاعی بود. در این مدل ها از متغیرهای سن گله، میزان خوراک مصرفی و فصل تولید به عنوان متغیر پیشگو و شاخص های عملکرد تولیدی شامل درصد تخم-گذاری، وزن توده ای تخم مرغ تولیدی و ضریب تبدیل غذایی به عنوان متغیر پاسخ استفاده شد. نتایج نهایی رگرسیون های خطی نشان داد که برای تمامی متغیرهای وابسته مورد مطالعه متغیر مستقل سن گله معنی دار می باشد. بنابراین رگرسیون غیر خطی شاخص های عملکرد تولیدی در مقابل سن برای مقایسه با شبکه های عصبی مختلف مورد بررسی قرار گرفت و برای مقایسه کلیه مدل ها از ضریب تعیین (R2) و میانگین قدر مطلق خطا (MAE) استفاده شد. نتایج نشان داد بین شبکه های عصبی مصنوعی مختلف مورد مطالعه، شبکه با تابع پایه ای شعاعی بهتر از سایر مدل های در پیش‌بینی شاخص های عملکرد تولیدی مرغان تخم گذار عمل می کند.

Efficient global optimization of aerodynamic shapes with the high-fidelity method is of great importance in the design process of modern aircrafts. In this study, modifications are made to the particle swarm optimization (PSO) algorithm and the radial basis function (RBF) model for further improvements of efficiency and accuracy of optimization. Specifically, a PSO algorithm with randomly distributed cognitive and social parameters, exponential decrease of maximum velocity and inertia weight, and periodic mutation of particle position is proposed. Furthermore, a nonuniform shape parameters strategy is introduced for the RBF surrogate model. Validations on test functions show that the new PSO has remarkable speed of convergence, and the new RBF model has superior approximation accuracy. The PSO algorithm and RBF model are then combined to construct the computational fluid dynamics (CFD)–based optimization framework. Finally, optimizations of transonic airfoil and supersonic launch vehicle are perfo...

Distributed constrained optimization for multi-agent networks with nonsmooth objective functions

With the accumulating of the strong earthquakes records, it becomes practicable to achieve the more accurate attenuation relationships. Based on the seismic records of West American, the Radial Basis Function (RBF) and Back Propagation (BP) artificial neural networks model are respectively constructed for three-dimensional seismic parameters attenuation relationship. The RBF model is nice fitting for the training data, although it has great errors on other tested points. While the BP model is not good than the RBF model for the training data, it possesses a better consecutive property in the whole area. It is a proper neural network model for the problem. After training with the selected records, the Neural Networks (NN) shows a good fitting with the training records. And it is easy to construct three-dimensional model to predict the attenuation relationship. In order to demonstrate the efficiency of the presented methodology, the contrast is discussed for the results of the BP model and three typical traditional attenuation formulae.

In this paper, a particle swarm optimization (PSO) algorithm is developed for allocation and sizing of multiple active power-line conditioners (APLCs) in power systems. The utilized objective function comprises four factors as total harmonic distortion, total size of active power-line conditioners (APLCs), harmonic transmission-line loss, and motor load loss. To evaluate the capability of the proposed method, the IEEE 18-bus test system is employed and investigated in three different cases. These cases are based on the assumption of continuous/discrete and limited/unlimited size for APLCs, requiring the optimization method to solve discrete/nondiscrete nonlinear problems. Therefore, in addition to PSO, an integer nonlinear optimizer (discrete PSO called DPSO) algorithm is also developed. Simulation results are compared with results obtained by genetic algorithm as well as with nonlinear programming (discrete nonlinear programming). It is demonstrated that analytical methods enjoy higher accuracy in smooth objective functions while they are not accurate enough in practical situations where nonsmooth objective functions are involved. If improper initial solutions are provided, achieving optimized results cannot be guaranteed by using analytical approaches. It is also shown that heuristic algorithms due to their randomness characteristics are more useful in practical cases where a number of local minima are present. Simulation results confirmed the capability and effectiveness of the proposed PSO-based algorithm in the allocation and sizing of multiple APLCs in a test power system compared with analytical methods and other heuristic algorithms.

This paper discusses the Improvement of Concave-Convex procedure, where the objective function in optimization problem can be decomposed into a convex function minus a generalized differential function. While preserving the property of monotonic decreasing for optimization objective function, the convergence conditions of this procedure and the scope it can be applied to were also improved greatly. Use the properties of sub-gradient and of convex function to prove thess procedures are globally descent convergent. The optimization problem it solved can be smooth or non-smooth objective functions. Meanwhile, the global convergence of this procedure can be used for analyzing the stability of Hopfield neural networks. Also it can be used both as a new way to understand existing optimization algorithms and as a procedure for generating new algorithms.

A study is performed on the effectiveness of assessing accuracies of radial basis functions (RBFs) at design (or sampling) points. The RBF has been shown to be appropriate for creating approximation functions for linear, quadratic, and higher‐order nonlinear responses even with a relatively small size of design samples. One favourable feature of RBFs is that all RBF models pass exactly through design points; therefore, an RBF model has no error at design points. However, this feature prevents the use of statistical parameters, which are commonly used in polynomial‐regression approximations, to obtain model accuracy based on evaluations at design points. Since obtaining additional samples for many complex engineering applications is expensive, it is desirable to have some knowledge of the model accuracy based on evaluations at design points so that the best RBF model(s) can be selected. In this study, the effectiveness of using statistical parameter R2 for prediction (R) is evaluated for two highly accurate RBFs using mathematical functions and real engineering applications. The results show that R is inappropriate for RBF models, and should be used cautiously for highly nonlinear responses with a small number of sampling points and/or a large number of design variables. Copyright © 2006 John Wiley & Sons, Ltd.

In this paper, we describe a wavelet-based approach to multiresolution stochastic image modeling. The basic idea here is that a complex random field, e.g., one with long range and nonlinear spatial correlations, can be decomposed into several less complex random fields. This is done by defining a random field in each resolution of a wavelet expansion. Experiments, performed for the multiresolution AR (autoregressive) and RBF (radial basis function) models, have produced promising results. Specifically, the wavelet-AR model captures long range correlation better than the single resolution AR model, and for both the wavelet AR and RBF models, random fields in the wavelet domain do appear to be simpler to model than those on the finest resolution.

Strongly sub-feasible direction method for constrained optimization problems with nonsmooth objective functions

This paper develops a novel algorithm that tackles a general distributed convex optimization problem with a nonsmooth objective function and a series of coupling nonlinear inequality and linear equality constraints. To address such a problem, we first design a distributed proximal algorithm for problems with coupling equality constraints only, and then extend this algorithm to problems with both inequality and equality constraints by incorporating a virtual-queue-based method. The resulting algorithm, referred to as Proximal Primal-Dual Algorithm (PPDA), is shown to achieve O(1/k) rates of convergence with respect to both optimality and feasibility. Compared to the alternative methods in the literature, PPDA eliminates their assumptions on the smoothness of the objective function and achieves a stronger convergence rate result. In addition, PPDA exhibits faster convergence in a couple of numerical examples.

This paper develops a new surrogate-assisted evolutionary programming (EP) algorithm for computationally expensive constrained black-box optimization. The proposed algorithm, TRICEPS (Trust Regions In Constrained Evolutionary Programming using Surrogates) builds surrogates for the black-box objective function and inequality constraint functions in every generation of the EP and uses a trust-region-like approach to refine the best solution at the end of each generation. Each parent produces a large number of trial offspring in each generation, and then the surrogates are used to identify promising trial offspring, which become the actual offspring where the objective and constraint functions are evaluated. After the function evaluations at these offspring, TRICEPS finds a minimizer of the surrogate of the objective function within a trust region centered at the current best solution and subject to surrogate inequality constraints with a small margin and with a distance requirement from previously evaluated points. The trust region is either expanded or reduced depending on whether the subproblem solution turned out to be feasible and whether the ratio of the actual improvement to the predicted improvement exceeds or falls below certain thresholds. TRICEPS is implemented using a cubic radial basis function (RBF) model with a linear polynomial tail and is compared to an RBF-assisted EP called CEP-RBF (Regis 2014b) and to other alternatives on 18 benchmark problems and on an automotive application with 124 decision variables and 68 black-box inequality constraints. Performance and data profiles show that TRICEPS is a substantial improvement over CEP-RBF and it is much better than the other alternatives on the test problems used.

We have developed a new derivative-free algorithm based on Radial Basis Functions (RBFs). Derivative-free optimization is an active field of research and several algorithms have been proposed recently. Problems of this nature in the industrial setting are quite frequent. The reason is that in a number of applications the optimization process contains simulation packages which are treated as black boxes. The development of our own algorithm was originally motivated by an application in biomedical imaging: the medical image registration problem. The particular characteristics of this problem have incited us to develop a new optimization algorithm based on trust-region methods. However it has been designed to be generic and to be applied to a wide range of problems. The main originality of our approach is the use of RBFs to build the models. In particular we have adapted the existing theory based on quadratic models to our own models and developed new procedures especially designed for models based on RBFs. We have tested our algorithm called BOOSTERS against state-of-the-art methods (UOBYQA, NEWUOA, DFO). On the medical image registration problem, BOOSTERS appears to be the method of choice. The tests on problems from the CUTEr collection show that BOOSTERS is comparable to, but not better than other methods on small problems (size 2-20). It is performing very well for medium size problems (20-80). Moreover, it is able to solve problems of dimension 200, which is considered very large in derivative-free optimization. We have also developed a new class of algorithms combining the robustness of derivative-free algorithms with the faster rate of convergence characterizing Newtonlike-methods. In fact, they define a new class of algorithms lying between derivative-free optimization and quasi-Newton methods. These algorithms are built on the skeleton of our derivative-free algorithm but they can incorporate the gradient when it is available. They can be interpreted as a way of doping derivative-free algorithms with derivatives. If the derivatives are available at each iteration, then our method can be seen as an alternative to quasi-Newton methods. At the opposite, if the derivatives are never evaluated, then the algorithm is totally similar to BOOSTERS. It is a very interesting alternative to existing methods for problems whose objective function is expensive to evaluate and when the derivatives are not available. In this situation, the gradient can be approximated by finite differences and its costs corresponds to n additional function evaluations assuming that Rn is the domain of definition of the objective function. We have compared our method with CFSQP and BTRA, two gradient-based algorithms, and the results show that our doped method performs best. We have also a theoretical analysis of the medical image registration problem based on maximization of mutual information. Most of the current research in this field is concentrated on registration based on nonlinear image transformation. However, little attention has been paid to the theoretical properties of the optimization problem. In our analysis, we focus on the continuity and the differentiability of the objective function. We show in particular that performing a registration without extension of the reference image may lead to discontinuities in the objective function. But we demonstrate that, under some mild assumptions, the function is differentiable almost everywhere. Our analysis is important from an optimization point of view and conditions the choice of a solver. The usual practice is to use generic optimization packages without worrying about the differentiability of the objective function. But the use of gradient-based methods when the objective function is not differentiable may result in poor performance or even in absence of convergence. One of our objectives with this analysis is also that practitioners become aware of these problems and to propose them new algorithms having a potential interest for their applications.

In this paper, a time-varying channel model is proposed according to radial basis function (RBF) artificial neural networks (ANN). The channel models including path loss plus shadow fading, number of paths, direction-of-arrival (DoA), etc are developed by RBF based on channel measurements performed in mm-wave 26 GHz band in an outdoor microcell. Results show that the RBF models can accurately playback the measured data, and the parameters extracted from the RBF model can be used in time-varying channel simulation practically. Therefore, the model is expected to overcome the shortcomings in existing statistical modeling approach and match with real measured environment, which is useful for 5G system and link level simulations.