Control of Educational Processes Using SPSA

The minimization of a convex function defined over the grid Z/sup p/ is considered. A few relevant mathematical devices such as integer convexity, Markov chain Monte Carlo (MCMC) methods, including stochastic comparison (SC), and simultaneous perturbation stochastic approximation (SPSA) are summarized. A truncated fixed gain SPSA method is proposed and investigated in combination with devices borrowed from the MCMC literature. The main contribution of the paper is the development and testing a number of devices that may eventually improve the convergence properties of the algorithm, such as various truncation techniques, averaging and choices of acceptance probabilities. The basis for comparison of performances is accuracy vs. number of function evaluations. We present experimental evidence for the superiority of an SC method allowing moves in wrong directions with small probability, where the underlying method is an SPSA method using averaging and adaptive truncation.

A neural control scheme for variable air volume (VAV) air-conditioning system is proposed. The neural network is trained online by the simultaneous perturbation stochastic approximation (SPSA) method instead of the standard back-propagation algorithm. The closed-loop stability of the proposed control scheme is guaranteed based on the conic sector theory. The new control scheme provides the desired functionality as well as the adaptation of the VAV control system for a wide range of disturbances and parameter changes. To demonstrate the applicability of the proposed method, real-time experiments were carried out on a pilot VAV air-conditioning system and good testing results are obtained.

Various aspects of the neutron spectroscopy problem are studied in this research. Theoretically, the neutron emission problem transforms into a limited angle Radon transform problem. In order to solve this ill posed problem, various algorithms are tested spanning two different techniques; algebraic reconstruction as well as Monte-Carlo methods. The developed algorithms are Stochastic Gradient Approximation (SGA) method, Simultaneous Perturbation Stochastic Approximation (SPSA) method, and Time of Flight (TOF) method.

The most widely used method applied in the context of off-line dynamic demand calibration is Simultaneous Perturbation Stochastic Approximation (SPSA). In the research following the SPSA approach single origin-destination (O-D) demand components were mostly considered as calibration parameters. However, basic SPSA, especially in high dimensions, shows convergence issues, as proven by various authors. To overcome this drawback, some authors suggested modifications of basic SPSA to improve its performance. In this paper, we investigate various techniques and approaches to improve the SPSA performance, and overcome, or at least alleviate, its shortcomings. We concentrate our analysis mostly on SPSA coefficients and gradient control. The comparison of investigated settings is conducted on a real-world network. This establishes a path to identify critical aspects that influence the calibration process and suggests an optimal SPSA configuration for practice. The contribution of this paper is to provide a detailed analysis of the SPSA behavior in cases its configuration is subject to various modifications. The findings are primarily intended for the offline context. However, the insights can also be used for the selection of the most efficient SPSA configuration given time constraint, particularly suitable for on-line applications.

Daniel C. ChinThe Johns Hopkins University AppliedPhysics Laboratory11100 Johns Hopkins RoadLaurel, Maryland 20723-6099E-mail: daniel.chin@jhuapl.eduAbstract. Extracting a multivariate nonlinear physical model from a setof satellite images is considered as a multivariate nonlinear regressionproblem. Multiple local solutions often prevent gradient type algorithmsfrom obtaining global optimal solutions. A method of solving this problemis presented based on the simultaneous perturbation stochastic approxi-mation (SPSA) algorithm. The method is applied to a problem of estimat-ing the distribution of energetic ion populations in the magnetospherefrom global images of the magnetosphere. The approach uses multipleobjective functions: single image errors and the summation of squareimage errors. The algorithm is demonstrated on simulated energetic-neutral atom (ENA) images. Within a reasonable number of functionevaluations, the process converges and reconstructs the images with amean square error less than or equal to 0.1% of the original image. Also,the SPSA method is compared with results obtained from simulated an-nealing (SAN) in a single objective function setting. In the comparisonstudy, SPSA has a 3:1 advantage over SAN in both accuracy and effi-ciency measures.

The Dutch continental shelf model (DCSM) is a shallow sea model of entire continental shelf which is used operationally in the Netherlands to forecast the storm surges in the North Sea. The forecasts are necessary to support the decision of the timely closure of the moveable storm surge barriers to protect the land. In this study, an automated model calibration method, simultaneous perturbation stochastic approximation (SPSA) is implemented for tidal calibration of the DCSM. The method uses objective function evaluations to obtain the gradient approximations. The gradient approximation for the central difference method uses only two objective function evaluation independent of the number of parameters being optimized. The calibration parameter in this study is the model bathymetry. A number of calibration experiments is performed. The effectiveness of the algorithm is evaluated in terms of the accuracy of the final results as well as the computational costs required to produce these results. In doing so, comparison is made with a traditional steepest descent method and also with a newly developed proper orthogonal decomposition-based calibration method. The main findings are: (1) The SPSA method gives comparable results to steepest descent method with little computational cost. (2) The SPSA method with little computational cost can be used to estimate large number of parameters.

We propose certain discrete parameter variants of well known simulation optimization algorithms. Two of these algorithms are based on the smoothed functional (SF) technique while two others are based on the simultaneous perturbation stochastic approximation (SPSA) method. They differ from each other in the way perturbations are obtained and also the manner in which projections and parameter updates are performed. All our algorithms use two simulations and two-timescale stochastic approximation. As an application setting, we consider the important problem of admission control of packets in communication networks under dependent service times. We consider a discrete time slotted queueing model of the system and consider two different scenarios - one where the service times have a dependence on the system state and the other where they depend on the number of arrivals in a time slot. Under our settings, the simulated objective function appears ill-behaved with multiple local minima and a unique global minimum characterized by a sharp dip in the objective function in a small region of the parameter space. We compare the performance of our algorithms on these settings and observe that the two SF algorithms show the best results overall. In fact, in many cases studied, SF algorithms converge to the global minimum.

This study proposes one novel data‐driven control strategy based upon the simultaneous perturbation stochastic approximation method with adaptive weighted gradient estimation for general discrete non‐linear systems. A function approximator is used to construct the controller, and here, it is fixed as a neural network (NN), whose structure is fixed previously, while allowing its connecting weights to be updated. The control parameters are then the connecting weights of the NN controller. The biggest advantage of this data‐driven control approach is that it can generate a control signal to affect system's future performance without establishing the plant's mathematical model first. In this novel approach, to improve the control ability and accuracy, an adaptive weighted gradient estimation method is designed to do the parametric estimation with convergence analysis. Non‐linear tracking problems for typical discrete‐time non‐linear plants are introduced for simulation comparison tests, and the convergence and feasibility of this newly proposed data‐driven control strategy are well demonstrated through the simulation results. Finally, empirical study on a simulated wastewater treatment system is carried out to further illustrate the effectiveness of this newly proposed approach.

The urban rail transit system faces huge passenger flow pressure, especially at peak hours. This paper focuses on the passenger flow control of urban railway lines at peak hours. A comparative study is conducted between the mathematical programming model and the simulation experiments. The results reveal a huge difference between the optimized results based on the mathematical programming model and the simulation results. To address this, a simulation-based optimization framework is proposed based on an improved simultaneous perturbation stochastic approximation (SPSA) method. Xi’an Metro Line 2 is selected as a case study, and the results show that the proposed framework can reduce the average passenger waiting time by 64.6%.

In recent years, simulation optimization has attracted a great deal of attention because simulation can model the real systems in fidelity and capture complex dynamics. Among numerous simulation optimization algorithms, Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm is an attractive approach because of its simplicity and efficiency. Although SPSA has been applied in several problems, it does not converge for some. This research proposes Augmented Simultaneous Perturbation Stochastic Approximation (ASPSA) algorithm in which SPSA is augmented to include presearch, ordinal optimization, non-uniform gain, and line search. Performances of ASPSA are tested on complex discrete supply chain inventory optimization problems. The tests results show that ASPSA not only achieves speed up, but also improves solution quality and converges faster than SPSA. Experiments also show that ASPSA is comparable to Genetic Algorithms in solution quality (6% to 15% worse) but is much more efficient computationally (over 12x faster).Request access from your librarian to read this chapter's full text.

Calibration and validation have long been a significant topic in traffic model development. In fact, when moving to dynamic traffic assignment (DTA) models, the need to dynamically update the demand and supply components creates a considerable burden on the existing calibration algorithms, often rendering them impractical. These calibration approaches are mostly restricted either due to non-linearity or increasing problem dimensionality. Simultaneous perturbation stochastic approximation (SPSA) has been proposed for the DTA model calibration, with encouraging results, for more than a decade. However, it often fails to converge reasonably with the increase in problem size and complexity. In this paper, we combine SPSA with principal components analysis (PCA) to form a new algorithm, we call, PC–SPSA. The PCA limits the search area of SPSA within the structural relationships captured from historical estimates in lower dimensions, reducing the problem size and complexity. We formulate the algorithm, demonstrate its operation, and explore its performance using an urban network of Vitoria, Spain. The practical issues that emerge from the scale of different variables and bounding their values are also analyzed through a sensitivity analysis using a non-linear synthetic function.

B034 PARALLEL WELL LOCATION OPTIMIZATION USING STOCHASTIC ALGORITHMS ON THE GRID COMPUTATIONAL FRAMEWORK 1 Hector Klie 1 Wolfgang Bangerth 1 2 Mary F. Wheeler 1 Manish Parashar 3 and Vincent Matossian 3 1 Center for Subsurface Modeling The University of Texas at Austin Austin TX. 2 Institute for Geophysics The University of Texas at Austin Austin TX. 3 The Applied Software Systems Laboratory Rutgers University Piscataway NJ. Abstract The determination of optimal well locations is a challenging problem in oil production since it depends on geological and fluid properties as well as on economic parameters. This work addresses the efficient

This paper addresses optimization for hybrid electric vehicle (HEV). This project is using a single agent method to optimize the power losses under a specific driving cycle which is simultaneous perturbation stochastic approximation (SPSA) based method. For optimization process, four gain are added in four main parts of the HEV system. Those main parts are engine, motor, generator and battery. These four gain is controlled the output for each component to give the minimum power losses. The design method is applied to free model of HEV by using Simulink/MATLAB software while M-File/MATLAB is used to apply the SPSA method. The result from design method achieved a minimum reduction of power losses compared to original system. Thus, the comparison of result has been done to show the different before and after optimization.

This paper describes the implementation of a self- optimizing embedded control scheme for an induction motor drive. The online design problem is formulated as a search problem and solved with a stochastic optimization algorithm. The objective function takes in account the tracking error, and is directly measured on the hardware bench. The online optimization is performed with the simultaneous perturbation stochastic approximation (SPSA) algorithms, which offer a very effective tradeoff between simplicity of implementation, speed of convergence and quality of the final solutions. Among the known SPSA algorithms considered in this paper, we also propose a novel variant inspired to the concept of elitism frequently used in evolutionary computation. To assess the relative performances of the various algorithms, the paper carries out a comprehensive analysis of a control scheme for an induction motor drive subject to time-varying load disturbances.

This study proposes a data-driven PID tuning for liquid slosh suppression based on enhanced stochastic approximation. In particular, a new version of Simultaneous Perturbation Stochastic Approximation (SPSA) based on memory type function is introduced. This memory-based SPSA (M-SPSA) algorithm has the capability to obtain a better optimization accuracy than the conventional SPSA since it is able to keep the best design parameter during the tuning process. The effectiveness of this algorithm is tested to data-drive PID tuning for liquid slosh problem. The achievement of the M-SPSA based algorithm is assessed in terms of trajectory tracking of trolley position, slosh angle reduction and also computation time. The outcome of this study shows that the PID-tuned M-SPSA is able to provide better control performance accuracy than the other variant of SPSA based method.