Mode-free altitude control for airship based on Q-learning and CMAC network

Cerebellar Model Articulation Controller (CMAC) is type of neural network inspired from part of the brain named cerebellum. It has long history in control applications, in simulated or in real-time implementations. MATLAB/Simulink is used for rapid building of CMAC neural network controller prototypes than can run in real-time. To run the controller in the same personal computer, QuaRC is used as real-time operating system that will execute controller task along Windows operating system. Hardware-in-the-loop architecture was used to test CMAC controller in real ball-and-beam physical plant from Quanser. Library of developed Simulink functional blocks enable easy exploration of different structural aspects of CMAC neural network controller, including selection of receptive field types, receptive field widths, Albus overlays, and fully interconnected multi-dimensional receptive fields. Same CMAC controller can be used also for controlling model of the plant, which can serve as means for controller to acquire knowledge before operating with real plant. CMAC controller learned to control the plant, and progressively became main control signal generator, while control signal from primary proportional-derivative controller almost vanished. Developed library can serve as easy startup for working with CMAC networks also for other types of applications.

A hierarchical cerebellar model articulation controller (CMAC) architecture suitable for context-dependent function approximation is proposed. The objective is to approximate several distinct nonlinear functions, one for each of several contexts. The active context is determined from the values of context variables, and smooth interpolation between different contexts is possible. The learning algorithms used can be similar to those of the hierarchical mixtures of experts (HME) as CMAC networks are linear in parameters. The proposed architecture converges quickly and has very low computational requirements when first order learning algorithms are used. The effectiveness of the architecture is demonstrated on a composite nonlinear regression task involving three Gaussian functions.

Based on cerebellar model articulation controller (CMAC), this paper attempts to propose a new chromosome representation scheme for representing real number parameters in genetic algorithms (GAs), which is termed CMAC-based GAs. The central idea of CMAC-based GAs is that each memory unit in a CMAC network is regarded as a real-valued gene in GAs. By this way, a chromosome is represented as those memory units addressed by a specific state and each gene involves partial information about a potential solution, not a potential solution as the conventional GAs. In addition, the corresponding crossover and mutation operators are also derived for the proposed GAs. The proposed CMAC-based GAs is applied to optimize the parameters of the proportional-integral-derivative controller. Simulation results are compared with several previous findings to demonstrate the search performance of the proposed method.

Vision has extensively expanded robots' capabilities, making the robot control problem more complex. To track a target with a robot arm in a three-dimensional space involves the use of precise commands. We propose to insert a hierarchical neurocontroller based on CMAC (cerebellar model articulation controller) networks in a visual servoing loop. This hierarchical structure splits the robot's workspace and assigns different CMAC controllers imposing thus a specialized region CMAC. Compared to a single CMAC with the same number of weights, the neurocontroller's sensitivity and precision is increased. Robot positioning and target tracking with visual feedback can then be done with a better precision.

This paper proposed an adaptive three-dimensional (3D) path-following control design for a robotic airship based on reinforcement learning. The airship 3D path-following control is decomposed into the altitude control and the planar path-following control, and the Markov decision process (MDP) models of the control problems are established, in which the scale of the state space is reduced by parameter simplification and coordinate transformation. To ensure the control adaptability without dependence on an accurate airship dynamic model, a Q-Learning algorithm is directly adopted for learning the action policy of actuator commands, and the controller is trained online based on actual motion. A cerebellar model articulation controller (CMAC) neural network is employed for experience generalization to accelerate the training process. Simulation results demonstrate that the proposed controllers can achieve comparable performance to the well-tuned proportion integral differential (PID) controllers and have a more intelligent decision-making ability.

A learning control technique which uses an extension of the CMAC (cerebellar model articulation controller) network developed by J.S. Albus is discussed, and the results of real-time control experiments which involved learning the dynamics of a five-axis industrial robot during high-speed, nonrepetitive movements are presented. During each control cycle, a training scheme was used to adjust the weights in the network in order to form an approximate dynamic model of the robot in appropriate regions of the control space. Simultaneously, the network was used during each control cycle to predict the actuator drives required to follow a desired trajectory, and these drives were used as feedforward terms in parallel to a fixed gain linear feedback controller. Trajectory tracking errors were found to converge to low values within a few training trails for both repetitive and nonrepetitive operations.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A control scheme of ANN-based PID controller is developed here to reach high precision tracking control for an electro-hydraulic servo system. The PID controller is used as a feedback controller to guarantee the system stability. The cerebellar model articulation controller (CMAC) neural network is used as a feed-forward compensator to identify the inverse system dynamics model. The CMAC and the PID controller are connected in parallel. The outputs of this paralleled controller are summed up as the total control action. A nonlinear tracking differentiator (NTD) is presented to yield high quality differential signals for the PID controller. The main task of this control algorithm is to make the error between the total control action and the output of the CMAC minimize by the CMAC learning algorithm. Thus the control action is formed by the CMAC. Numerical simulation results show comparing with conventional PID control strategy this proposed control scheme has an excellent system performance including high precision trajectory tracking ability and rejection of disturbance.

Vanadium Redox Battery (VRB) has the advantages of large capacity, long life, safety and environmentally friendly. This paper proposes a double closed-loop charge-discharge control strategy of VRB based on cerebellar model arithmetic computer (CMAC) to ensure safe charge-discharge and shorten the charge-discharge time. The proposed control strategy consists of two main control loops: an outer voltage loop and an inner current loop. The outer control loop is designed as a voltage loop with CMAC controller which uses kernel voltage for feedback, and the inner current loop adopts PI regulator with amplitude limiting. In this paper, we first build a mathematical model of VRB of 5 kW/30kW· h. Then, the proposed control strategy and conventional PID control strategy are respectively applied to VRB model and comparisons are made. Results show that under the proposed control strategy, charging time of VRB shortened by 3.03%, discharging time of VRB shortened by 9.37% compared with conventional PID control strategy, and the terminal voltage can be guaranteed in a safe range, thus realizing fast and safe charge-discharge of VRB.

In this paper, a novel learning scheme is proposed to speed up the learning process in cerebellar model articulation controllers (CMAC). In the conventional CMAC learning scheme, the correct numbers of errors are equally distributed into all addressed hypercubes, regardless of the credibility of the hypercubes. The proposed learning approach uses the inverse of learned times of the addressed hypercubes as the credibility (confidence) of the learned values, resulting in learning speed becoming very fast. To further demonstrate online learning capability of the proposed credit assigned CMAC learning scheme, this paper also presents a learning robust controller that can actually learn online. Based on robust controllers presented in the literature, the proposed online learning robust controller uses previous control input, current output acceleration, and current desired output as the state to define the nominal effective moment of the system from the CMAC table. An initial trial mechanism for the early learning stage is also proposed. With our proposed credit-assigned CMAC, the robust learning controller can accurately trace various trajectories online.

There is an increasing trend in using exoskeleton to walk or carry payload. In the process of human-exoskeleton interaction, since the user's intention is required accurately, an important problem is human-robot collaborative control. A precise position control strategy will significantly enhance collaboration between the exoskeleton and the user. This paper proposes a novel method combining sliding mode control (SMC) with Cerebellar model articulation controller (CMAC). The proposed method is applied in a lower extremity exoskeleton to track a desired trajectory. The switching surface in sliding mode control was determined by tracking error. A CMAC was designed as a compensator which considers uncertainty of the dynamical model. The dynamic of the lower extremity is derived by using Euler-Lagrange Equation. The proposed control strategy was proved stable with Lyapunov analysis and used for trajectory tracking in comparison with a conventional sliding mode control. Results with Matlab simulation shows that the proposed controller can achieve better tracking capability and robust performance.

This paper discusses adaptive control method used in high performance servo system based on Cerebellar Model Articulation Controller(CMAC). It analyzes the uncertain and nonlinear factors of the servo system, studies the modeling method using the frequency property of the system, and then uses an additional item to instead of the non-modeling factor. In the proposed control scheme, the CMAC controller is able to learn the unknown model dynamics, parameter variation and disturbance of the system on line. The model reference adaptive control (MRAC) scheme is used to give better solutions with online adaptation. It contemporarily improved the conception mapping algorithm of the Cerebellar Model Articulation Controller Neural Network(CMACNN), which gave a determined expression of the physical memory size and designed a physical memory address function. Experiment results verify that the proposed strategy can achieve high tracking precision for real-time position close-loop servo system.

Single-input CMAC control system

Towards enhanced threat modelling and analysis using a Markov Decision Process

Invisible stock loss often occurs in inventory management systems, which is usually caused by quality deterioration, theft, and misplacement and makes inventory record inaccurate. Inaccurate inventory record will bring significant negative impact on the performance of inventory control and leads to severe stock-out. To eliminate this negative impact, in this paper, we develop a Markov decision process (MDP) model for an inventory control system with invisible stock loss, inaccurate record, and replenishment lead-time. Based on the MDP model, a robust inventory control policy is constructed by using the policy iteration technique. From numerical experiments, we find that the robust policy still has a reorder point/order-up-to level structure i.e., an (s, S) structure, and can significantly reduce the impact of inventory inaccuracy.

Faustmann's formula gives the land value, or the forest value of land with trees, under deterministic assumptions regarding future stand growth and prices, over an infinite horizon. Markov decision process (MDP) models generalize Faustmann's approach by recognizing that future stand states and prices are known only as probabilistic distributions. The objective function is then the expected discounted value of returns, over an infinite horizon. It gives the land or the forest value in a stochastic environment. In MDP models, the laws of motion between stand-price states are Markov chains. Faustmann's formula is a special case where the probability of movement from one state to another is equal to unity. MDP models apply whether the stand state is bare land, or any state with trees, be it even- or uneven-aged. Decisions change the transition probabilities between stand states through silvicultural interventions. Decisions that maximize land or forest value depend only on the stand-price state, independently of how it was reached. Furthermore, to each stand-price state corresponds one single best decision. The solution of the MDP gives simultaneously the best decision for each state, and the forest value (land plus trees), given the stand state and following the best policy. Numerical solutions use either successive approximation, or linear programming. Examples with deterministic and stochastic cases show in particular the convergence of the MDP model to Faustmann's formula when the future is assumed known with certainty. In this deterministic environment, Faustmann's rule is independent of the distribution of stands in the forest. FOR. SCI. 47(4):466–474.