Density Stochastic Approximation of Initial Buffer Decision Problem in Continuous Production Lines

We obtain new transport-entropy inequalities and, as a by-product, new deviation estimates for the laws of two kinds of discrete stochastic approximation schemes. The first one refers to the law of an Euler like discretization scheme of a diffusion process at a fixed deterministic date and the second one concerns the law of a stochastic approximation algorithm at a given time-step. Our results notably improve and complete those obtained in [Frikha, Menozzi, 2012]. The key point is to properly quantify the contribution of the diffusion term to the concentration regime. We also derive a general non-asymptotic deviation bound for the difference between a function of the trajectory of a continuous Euler scheme associated to a diffusion process and its mean. Finally, we obtain non-asymptotic bound for stochastic approximation with averaging of trajectories, in particular we prove that averaging a stochastic approximation algorithm with a slow decreasing step sequence gives rise to optimal concentration rate.

In this article we present and analyse new multilevel adaptations of classical stochastic approximation algorithms for the computation of a zero of a function $$f:D \rightarrow {{\mathbb {R}}}^d$$ defined on a convex domain $$D\subset {{\mathbb {R}}}^d$$ , which is given as a parameterised family of expectations. The analysis of the error and the computational cost of our method is based on similar assumptions as used in Giles (Oper Res 56(3):607–617, 2008) for the computation of a single expectation. Additionally, we essentially only require that f satisfies a classical contraction property from stochastic approximation theory. Under these assumptions we establish error bounds in pth mean for our multilevel Robbins–Monro and Polyak–Ruppert schemes that decay in the computational time as fast as the classical error bounds for multilevel Monte Carlo approximations of single expectations known from Giles (Oper Res 56(3):607–617, 2008). Our approach is universal in the sense that having multilevel implementations for a particular application at hand it is straightforward to implement the corresponding stochastic approximation algorithm.

This paper extends the link between stochastic approximation (SA) theory and randomized urn models developed in Laruelle, Pages (2013), and their applications to clinical trials introduced in Bai, HU (1999,2005) and Bai, Hu, Shen (2002). We no longer assume that the drawing rule is uniform among the balls of the urn (which contains d colors), but can be reinforced by a function f. This is a way to model risk aversion. Firstly, by considering that f is concave or convex and by reformulating the dynamics of the urn composition as an SA algorithm with remainder, we derive the a.s. convergence and the asymptotic normality (Central Limit Theorem, CLT) of the normalized procedure by calling upon the so-called ODE and SDE methods. An in-depth analysis of the case d=2 exhibits two different behaviors: A single equilibrium point when f is concave, and when f is convex, a transition phase from a single attracting equilibrium to a system with two attracting and one repulsive equilibrium points. The last setting is solved using results on non-convergence toward noisy and noiseless ``traps in order to deduce the a.s. convergence toward one of the attracting points. Secondly, the special case of a Polya urn (when the addition rule is the identity matrix) is analyzed, still using result from SA theory about ``traps''. Finally, these results are applied to a function with regular variation and to an optimal asset allocation in Finance.

Limited by the computing capacity of current computer and difficulty in inversion of large quantity of reservoir parameters for large scale reservoir history matching, the current automatic history matching methods are still hard to be applied in real fields. A parameterization method based on matrix decomposition is proposed to reduce the computational cost and improve the efficiency of history matching without decreasing the precision by mapping the original optimization problem from high dimensional space to low dimensional space. Base on the updated reservoir model after history matching, a reservoir production optimization control model is established. Considering the complexity of gradient calculation for history matching and production optimization objective function, a new general upscaling stochastic approximation (GUSA) algorithm is given based on the approximation perturbation gradient algorithm. The GUSA algorithm aims to get maximum increment of an objective function in unit step by obtaining an optimal lower triangular matrix in each iterative step. Toy problem results show that the calculation efficiency is improved much compared with the simultaneous perturbation stochastic approximation (SPSA) algorithm. For application of a real reservoir, a good history matching result is obtained for historic production data that verifies the validity of parameterization method and GUSA algorithm. An optimal control for production optimization performs an increase of oil production and a decrease of water cut. Besides, the optimal production scheme is accordant with the practical characteristics of dynamic production, which also proves the availability of GUSA algorithm.KeywordsHistory matchingParameterization methodProduction optimal controlApproximate gradientOptimization algorithm

The use of neural networks (NNs) in controlling a nonlinear stochastic system with unknown process equations is considered. The NN is used to model the resulting unknown control law. The approach is based on using the output error of the system to train the NN controller without the need to construct a separate model (NN or other type) for the unknown process dynamics. To implement such an approach, it is required that connection weights in the NN be estimated while the system is being controlled. As a result of the feedback of the unknown process dynamics, however, it is not possible to determine the gradient of the loss function for use in standard (backpropagation-type) weight estimation algorithms. Therefore, the authors consider the use of a new stochastic approximation algorithm for this weight estimation that is based on a simultaneous perturbation gradient approximation that only requires the system output error. It is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation algorithms based on finite-difference gradient approximations. >

Considering the problem of the highly dependence on the accuracy model information of the industrial control process, an online PID parameter tuning method is proposed based on the simultaneous perturbation stochastic approximation (SPSA) algorithm and the traditional PID control only by the input and output data of the system. Interfering with each component of the PID controller parameters simultaneously, and the gradient information of the control performance loss function is estimated via the output value after disturbance by the global optimization method of stochastic approximation algorithm. The PID controller parameter is iterative updated according to the gradient estimation value. Finally, we applied this method to the control process of cement grinding particle size, and the simulation results by MATLAB shown it can lead a high control precision which proves the feasibility and effectiveness of this adaptive PID control method.

This paper considers a stochastic approximation algorithm, with decreasing step size and martingale difference noise. Under very mild assumptions, we prove the nonconvergence of this process toward a certain class of repulsive sets for the associated ordinary differential equation (ODE). We then use this result to derive the convergence of the process when the ODE is cooperative in the sense of Hirsch [SIAM J. Math. Anal. 16 (1985) 423–439]. In particular, this allows us to extend significantly the main result of Hofbauer and Sandholm [Econometrica 70 (2002) 2265–2294] on the convergence of stochastic fictitious play in supermodular games.

In this paper, we propose an automatic selection of the bandwidth of the recursive kernel estimators of a regression function defined by the stochastic approximation algorithm. We showed that, using the selected bandwidth and the stepsize which minimize the mean weighted integrated squared error, the recursive estimator will be better than the non‐recursive one for small sample setting in terms of estimation error and computational costs. We corroborated these theoretical results through simulation study and a real dataset.

The discrete-time first-order multi-agent networks with communication noises are under consideration. Based on the noisy observations, the consensus control is given for networks with both fixed and time-varying topologies. The states of agents in the resulting closed-loop network are updated by a stochastic approximation (SA) algorithm, and the consensus analysis for networks turns to be the convergence analysis for SA. For networks with fixed topologies, the proposed consensus control leads to consensus of agents with probability one if the graph associated with the network is connected. In the case of time-varying topologies, the similar results are derived if the graph is jointly connected in a fixed time period. Compared with existing results, the networks considered here are in a more general setting under weaker assumptions and the strong consensus is established by a simpler proof.

Our main result is to prove almost-sure convergence of a\nstochastic-approximation algorithm defined on the space of measures on a\nnon-compact space. Our motivation is to apply this result to measure-valued\nP\\'olya processes (MVPPs, also known as infinitely-many P\\'olya urns). Our main\nidea is to use Foster-Lyapunov type criteria in a novel way to generalize\nstochastic-approximation methods to measure-valued Markov processes with a\nnon-compact underlying space, overcoming in a fairly general context one of the\nmajor difficulties of existing studies on this subject.\n From the MVPPs point of view, our result implies almost-sure convergence of a\nlarge class of MVPPs, this convergence was only obtained until now for specific\nexamples, with only convergence in probability established for general classes.\nFurthermore, our approach allows us to extend the definition of MVPPs by adding\n"weights" to the different colors of the infinitely-many-color urn. We also\nexhibit a link between non-"balanced" MVPPs and quasi-stationary distributions\nof Markovian processes, which allows us to treat, for the first time in the\nliterature, the non-balanced case.\n Finally, we show how our result can be applied to designing\nstochastic-approximation algorithms for the approximation of quasi-stationary\ndistributions of discrete- and continuous-time Markov processes on non-compact\nspaces.\n

This research was driven by the need for a more efficient production scheduling system in a consumable liquid product division of a large consumer products company. The manufacturing process under inspection consists of continuous and discrete elements, on both production and packaging lines. The production lines are split into continuous production lines and batch production lines which produce the product in fixed batch amounts. Then there are several bottling lines, some of which package a particular bottle size and others that can package multiple bottle sizes. The main objective of this research was to reduce the amount of time it takes for the client to create production and bottling schedules. An optimization model was developed to automate this process and provide the client with the best possible schedule. The objective of the model is to minimize cost by minimizing the number of switches across the production and bottling lines, as well as minimizing the amount of overproduction. Inputs into the model include model parameters, like the number of shifts to schedule, and monthly demand numbers for each stock keeping unit (SKU). The variables being solved for are the amount of each flavor to be produced across the production lines during each shift, and the number of bottles of each SKU to be bottled across the bottling lines during each shift. Due to the unique constraints and resources of the client, a custom formulation using mixed integer programming was necessary to achieve these objectives. Overall, our model fell short in some areas but succeeded in others. Our analysis showed that the model had a 13% average decrease in production switches but an 87% average increase in bottling switches compared to the current manual scheduling system. However, the ability of our system to create a good enough initial schedule reduces the time it takes expert human schedulers to develop a final schedule by up to 85%. Runtime and computational constraints barred us from creating an optimal, cost-minimized solution for our client, and future work can be directed toward solving these issues.

Stochastic approximation algorithms with non-additive noise are discussed. In studying strong convergence of such algorithms, traditionally one assumes that the iterates return to a bounded or compact set infinitely often, or that the function under consideration grows with certain rate. The usual projection algorithms require that the bounded projection region is known beforehand. It is desirable to weaken these ‘boundedness’ conditions. By introducing randomly varying truncations, Chen and Zhu (1986) achieved this for stochastic approximation algorithms with additive noise. Here, we extend their result to a more general setting.

Production of functional products using local raw materials with a high content of biologically active substances is the main trend in the production of non-alcoholic beverages. In this regard, creation of a technological line that ensures not only the quality of the finished product, but also the safety of the functional properties of the feedstock is of great interest. A computer model of a functional soft drink production line, developed using the MasterSCADA 4D software created by a leading domestic company has been. The model emulates the entire production process including the stage of filling the finished product. It is possible to control a real production line provided that communication with an industrial controller, sensors and actuators is installed and configured. A set of technological equipment has been selected for each production stage, from processing raw materials and on down to functional soft drink packaging. Recommendations have been given for water treatment, including a number of technological processes, such as rough water purification and disinfection. There are instructions for preparing sugar syrup using hot method. The characteristics of the recommended plant for hydrodynamic extraction from plant raw materials has been presented. A factor intensifying the extraction process, in the pulsating action of the extractant between the solid and liquid phases. Production of a functional drink implies using of water as an extractant, so additional methods of intensifying mass transfer processes have been considered in order to increase the percentage of extraction of target components. The use of the effect of ultrasonic, electric, pulse and discrete-pulse fields on raw materials is considered to be the most promising method for the extraction process intensification.

Intelligent reflecting surface (IRS)-aided wireless relaying system has aroused great interest recently due to its potential to improve the relaying performance. However, most existing works only consider the decode-and-forward (DF) relay and ignore the base station (BS) to user direct link. In this paper, we focus on an IRS-aided full-duplex (FD) amplify-and-forward (AF) relay system and study the beamforming optimization problem. Specifically, we aim to maximize the transmission rate when deploying the IRS near the relay by jointly optimizing the IRS reflection coefficients and transmit powers at the BS and relay. Then, a virtual stochastic successive convex approximation (VSSCA) algorithm is developed to solve our formulated de-terministic optimization problem efficiently. Finally, numerical results are provided to demonstrate the effectiveness of our proposed algorithm as compared to various benchmark schemes.

The mathematical models of production lines and its stability analysis