Auto Tuning PID Using Deep Q-Learning

Design of a Data-Driven PID Controller using Operating Data

Recently intelligent control has received increased attention in various applications. With PID controllers, it is difficult to obtain gain suitable to the load because of disturbances, parameter variations and measurement noise. In order to reduce mechanical parameter variations, and to obtain good dynamic performance, PID controllers and neural network controllers for AC servomotor control are investigated. The microprocessor provides an output to the difference in command. The servo system improves the characteristics of speed control. When the motor is running at the same speed as set by the reference signal, the speed encoder also provides a signal of the same frequency. From the viewpoint of wide control range, a good dynamic performance and a cost effective approach, the NNE of an adaptive neural network controller is used to identify the parameters and characteristics of the AC servomotor. To train the controller, the weights are dynamically adjusted using the backpropagation algorithm. Neural networks are used so PID control can control, efficiently, the speed of the AC servomotor. Finally experimental results prove the excellent performance of this control system.

Under the research context of two axis digital control turntable, this paper has set up a mathematical model of control system of turntables. Proceeding from such model, it has made an analysis on the deficiencies of classical PID control during turntable control. Against the feature of PID parameters difficult to be tuned, the neural network is thus introduced in the optimizing process of turncontrol PID parameters, during which PID controller is designed based on BP network. In addition, the PID algorithm control based on BP network is simulated with results indicating PID controller designed so featuring high tracking accuracy, stable performance and strong robustness, which provides a solution to the problem of inadequate precision required by classic PID control as regards to the control precision.

This paper proposes a new offline learning algorithm for a data-driven proportional-integral-derivative (DD-PID) controller based on an extended fictitious reference iterative tuning (E-FRIT). PID controllers have been used in many process systems. However, it is difficult to maintain a good control performance using PID controllers with fixed control parameters because of the nonlinearity of the systems. The DD-PID controller has been proposed as an effective control system for nonlinear systems. This controller can tune its PID parameters adaptively at each equilibrium point of the system output. In the conventional DD-PID controller, the PID parameters are learned so as to minimize a criterion of the FRIT method. However, the FRIT method is based on minimization of the error in system output, and therefore, the criterion is insufficient for systems, such as chemical process systems, for which the stability of a closed loop system is essential. In order to solve this problem, the E-FRIT method has been proposed; a penalty for input variation is incorporated in the criterion for this method. In the present study, an offline learning rule of PID parameters was derived based on the E-FRIT criterion. According to the rule, the PID parameters that are taken into stability can be calculated. The effectiveness of the proposed method was evaluated by simulations of a polystyrene reactor system. The simulation confirmed that the proposed DD-PID controller yields better control result than the conventional learning method.

In view of the surplus torque and complexity of controlled plant in passive electrical load system, a novel approach based on single neuron PID adaptive control and repetitive control for repetitive periodic load control system is proposed. Radial basis function (RBF) neural network is used to identify the system on-line for the single neuron PID controller to adjust its weights and PID parameters by self- learning and self-adapting based on the desired output. The dynamic state performance can be improved by the single neuron adaptive PID control and the steady state performance is also improved by modified repetitive control. Computer simulation results show that the force/position hybrid control system can effectively reduce the surplus torque and improve the loading precision, and also it has fine dynamic and steady state performance and good robustness. The reliability of whole system is further improved.

The dangers of poor pilot performance as well as time and place conditions, low altitude and climate, damage critical aircraft control system. The use of Unmanned Aircraft Vehicle or (UAV) in sensitive and important Operation is required and there is a need for autopilot that can tune the MAV in different environment conditions. Furthermore, designing the controller system is one of the main discussed Dynamic issues in Flying Objects. In this paper an attempted is made to determine the optimal coefficients of PID controller that can reject disturbances and still operate the MAV in stable positions. Basic PID controller is designed and is adopted to control the MAV, a modified techniques incorporating ISA-PID is designed to reject disturbances. The PID parameters are determined to be reduce the rise time less than 3 seconds, settling time to be less than 8 seconds and overshoot to be less than 5%. The developed model is suitable for PID controller in autopilot.

This paper considers USV heading controller using Adaptive PID and LQG controller. PID is a conventional controller and the most industry controller. However, PID parameters are selected by trial and error. That problem proposed Adaptive PID how to tune PID parameters. In this paper, PID parameters are estimated by RLS method. The second controller for comparison is LQG. LQG controller is robust controller. A gain regulator estimation method, Kalman Filter, is used as LQG controller. The computational results show the LQG controller has better performance than Adaptive PID.

This paper deals with the adaptive nonlinear identification and trajectory tracking via dynamic multilayer neural network (NN) with different timescales. Two NN identifiers are proposed for nonlinear systems identification via dynamic NNs with different timescales including both fast and slow phenomenon. The first NN identifier uses the output signals from the actual system for the system identification. In the second NN identifier, all the output signals from nonlinear system are replaced with the state variables of the NNs. The online identification algorithms for both NN identifier parameters are proposed using Lyapunov function and singularly perturbed techniques. With the identified NN models, two indirect adaptive NN controllers for the nonlinear systems containing slow and fast dynamic processes are developed. For both developed adaptive NN controllers, the trajectory errors are analyzed and the stability of the systems is proved. Simulation results show that the controller based on the second identifier has better performance than that of the first identifier.

The acronym PID stands for Proportion-Integral-Differential control. Each of these, the P, the I and the D are terms in a control algorithm, and each has a special purpose. Conventional PID controller has a simple structure and can provide satisfactory performance for many control problems. For process control systems, in particular, more than 95% of the controllers used are of PID type (Ogata, 2002). This is because it is easily applicable to field use, and also easily adjustable. The process of stabilization or improving the performance of the system is achieved by attaching a controller or compensator to the closed loop system. There are different types and structures of controllers. However, the controller cannot deal fully with nonlinear and/or time varying systems. In the design of a PID controller, the main difficulty has been how the control gains should be selected for good control performance, especially where the process to be controlled is of high order with complex nonlinear dynamics. This problem is made more difficult by the need to keep these gains tuned, during operations, to ensure robust performance in the face of plant parameter variations, load or environment disturbances and measurement noise. Genetic algorithms (GAs) are one of the efficient tools that are employed in solving optimization problems (Rothlauf, 2006). Because of the simplicity and robustness, PID controllers are frequently the used controllers in industries (Almeida et al., 2005). Parameters adjustment of PID controllers is an old challenge in the field of control system design. Some of methods have been proposed to select the PID coefficients, but they are not completely systematic methods and result in a poorly tuned controller that needs some trail and error. So far, finding new methods to automatically select PID parameters was interest of researches (Ho, 2003), (Almeida et al., 2005), (Nasri et al., 2007), and (Mansour, 2011). However in this chapter, the selection of optimal PID controller coefficients are introduced and applied based on genetic algorithms technique. Therefore it is possible to find genetically the optimal controller coefficients, Kp, Ki, and Kd with the constraint of minimizing absolute or square error signal to achieve system stability and to enhance system performance. However in feedback control systems the most important requirement is the stability of the whole system (Bartoszewicz, 2011). Another requirement is the speed, i.e. in the case of a changing reference value or a disturbance; the control error should be eliminated as soon as possible. Finally, simulation results will be implemented using MATLAB/SIMULINK to test system performance and stability. Due to the fact that in the last two decades MATLAB became the standard computing environment for engineers from different fields of science, a

For the stability problem of the active magnetic levitation ball, the application of the PID controller on the system is determined. Firstly, the linearized mathematical model of the active magnetic levitation ball system is established. Then the PID parameters are adjusted by the trial method. The influence of three PID parameters on the system is introduced, and a PID controller is designed for a set of actual systems. The performance is simulated and experimentally studied. The research results show that the active electromagnetic suspension ball system based on PID controller has high anti-interference performance and stability.

One of the difficulties encountered in the application of reinforcement learning methods to real-world problem is the generalization of large-scale or continuous state space. In order to solve the curse of dimensionality problem caused by discretizing continuous state space, a kind of Q-learning method for continuous state space based on a dynamic Elman neural network was proposed in this paper. The inputs and the output of Elman network are the system state-action pair and the corresponding Q-value. That is, Elman network is used to estimate the Q-value of state-action pair on-line. Eligibility trace for connecting weights is introduced by borrowing ideas from the eligibility trace mechanism of state in temporal difference algorithm to improve the learning speed of neural network. Computer simulations on mountain car control illustrate the performance and applicability of the proposed Q-learning scheme.

Indonesian Abu Robot Contest (KRAI) in 2018 with the theme "Throwing a Blessing Ball". The main purpose of this robot is to be able to navigate automatically in an area that is bordered by walls to complete the mission. The main problem with the robot is the navigation system. The application of PID control in the wall following system has been able to make robot movements smooth, responsive, and fast. In this study, PID control aims to smooth the movement of the robot while walking along the wall in the race arena. The PID parameter is obtained from the results of tuning with the trial and error method, the values of KP = 3, KI = 0, and KD = 5. At the PWM 150 set point the value of the ultrasonic sensor distance reading to the object in the form of a wall with an average error of 4.4. cm. At the PWM 200 set point the value of the ultrasonic sensor distance reading to the object in the form of a wall with an average error of 0.4 cm. At the PWM 250 set point the value of the ultrasonic sensor distance reading to the object in the form of a wall with an average error of 0.8 cm. This error does not have an effect on the performance of the wall following system, because the system only uses the distance value reading data with a decimal value in front of the comma. So it can be concluded that the wall following system which is designed using ultrasonic sensors with measurement error that occurs is zero.

The article is devoted to the problem of optimization of search request ranking algorithms in the digital information retrieval system. The algorithm of functioning of the neural network ranking unit based on Hopfield neural network is built. The ability to generate a ranked list of pages found as a result of the request in the digital information retrieval system can be provided by solving two problems of integer optimization: the problem of assignment of combinatorial sets of criteria for assessing the relevance of web page search and the problem of sorting of numbers—relevance values. The architecture of the neural network model based on the dynamic Hopfield neural network with binary output function designed for combinatorial optimization of the final list of documents found in the digital information retrieval system was synthesized. Promising variants of neural network models with binary output function of neurons for synthesis of the optimal evaluation plan with a combinatorial set of criteria by solving the problem of assignment were built. It has been proven that the built models differ in the rules for determining the coefficients of synaptic connections and external shifts; each of the created rules can be used independently or in different combinations with one another. In the course of analytical research, it was found that the optimization formulation of the problem of sorting of relevance values of search pages is identical to the problem of assignment of combinatorial groups of evaluation criteria provided that the elements of the performance matrix of the latter are defined as linear combinations of relevance values.

An optimal PID controller with incomplete derivation is proposed based on fuzzy inference and the genetic algorithm, which is called the fuzzy-GA PID controller with incomplete derivation. It consists of the off-line part and the on-line part. In the off-line part, by taking the overshoot, rise time, and settling time of system unit step response as the performance indexes and by using the genetic algorithm, a group of optimal PID parameters K*p, T*i, and T*i are obtained, which are used as the inital values for the on-line tuning of PID parameters. In a on-line part, based on K*p, T*i, and T*d and according to the current system error e and its time derivative, a dedicated program is written, which is used to optimize and adjust the PID parameters on line through a fuzzy inference mechanism to ensure that the system response has optimal dynamic and steady-state performance. The controller has been used to control the D. C. motor of the intelligent bionic artificial leg designed by the authors. The result of computer simulation shows that this kind of optimal PID controller has excellent control performance and robust performance.

PID control schemes have been widely employed for most process systems represented by chemical processes. However, it is a very important problem how to tune PID parameters, because these parameters have a great influence on the stability and the performance of the control system. On the other hand, lots of works for the robust control have been carried out to cope with system uncertainties. Then, some PID parameter tuning methods have been proposed based on the robust stability. However, if the range of uncertainties is very large, the control performance becomes quite conservative. By the way, the support vector machine (SVM) has been proposed as one of the pattern recognition methods and gets lots of attention in last decades. The main motivation in this paper is to present a design scheme of controllers with the switching structure, in which some robust PID controllers are suitably switched using the SVM. Finally, the proposed control scheme is numerically evaluated on a simulation example.