• Home
  • Search
  • Asymptotically Efficient Adaptive Choice of Control Laws inControlled Markov Chains
  • Cite Icon139
  • https://doi.org/10.1137/s0363012994275440Copy DOI Icon

Asymptotically Efficient Adaptive Choice of Control Laws inControlled Markov Chains

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

We consider a controlled Markov chain on a general state space whose transition probabilities are parameterized by an unknown parameter belonging to a compact metric space. There is a one-step reward associated with each pair of control and the following state of the process. Given a finite set of stationary control laws, under each of which the Markov chain is uniformly recurrent, an optimal control law is this set is one that maximizes the long-run average reward. In ignorance of the parameter value, we construct an adaptive control rule which uses the optimal control law(s) at a relative frequency of 1 - O(n-1 log n) and show that this relative frequency gives an asymptotically optimal balance between the control objective and the amount of information needed to learn about the unknown parameter. The basic idea underlying this construction is to introduce suitable "uncertainty adjustments" via sequential testing theory into the certainty-equivalence rule, thus resolving the apparent dilemma between control and information.

Similar Papers
  • Research Article
  • Citations32

Strong consistency of a modified maximum likelihood estimator for controlled Markov chains

  • Sep 01, 1980
  • Journal of Applied Probability
  • Bharat Doshi +1
  • Supplementary Content
  • Citations35

Distributed estimation and control in networked systems

  • Jan 01, 2007
  • Vijay Gupta
  • Book Chapter
  • Citations6

Adaptive Control of Markov Chains

  • Jan 01, 1979
  • V Borkar +1
  • Research Article
  • Citations126

Invariant Distributions and the Limiting Behavior of Markovian Economic Models

  • Mar 01, 1982
  • Econometrica
  • Carl A Futia
  • Research Article
  • Citations49

Design of an intelligent optimal neural network-based tracking controller for nonholonomic mobile robot systems

  • Nov 18, 2016
  • Neurocomputing
  • Mohamed Boukens +1
  • Research Article
  • Citations13

Nonlinear optimal tracking control of spacecraft formation flying with collision avoidance

  • Jun 11, 2018
  • Transactions of the Institute of Measurement and Control
  • Ruixia Liu +2
  • Research Article
  • Citations2

Synthesis of Type-1 Optimal Digital Servosystems

  • Jan 01, 1989
  • Transactions of the Institute of Systems, Control and Information Engineers
  • Kazunori Yasuda +2
  • Conference Article
  • Citations37

Hybrid Model Predictive Control Application Towards Optimal Semi-Active Suspension

  • Jan 01, 2005
  • N Giorgetti +3
  • Research Article
  • Citations13

The minimum-time thrust-vector control law in the Apollo lunar-module autopilot

  • Sep 01, 1970
  • Automatica
  • W.S Widnall
  • Conference Article
  • Citations1

Design and Performance Analysis of Automatic Control Systems for Underwater Vehicles

  • Jun 12, 1989
  • H Sehhoglu +1
  • Research Article

Online learning algorithms : For passivity-based and distributed control

  • May 03, 2016
  • Research Repository (Delft University of Technology)
  • Subramanya Nageshrao
  • Research Article
  • Citations2

The Minimum-Time Thrust-Vector Control Law in the Apollo Lunar-Module Autopilot

  • Mar 01, 1970
  • IFAC Proceedings Volumes
  • William S Widnall
  • Conference Article

Scheduling Strategy of Multiple Semi-Active Controllers With Information on the Disturbance and the Structural Response

  • Jul 17, 2016
  • Kazuhiko Hiramoto +2
  • Book Chapter
  • Citations2

M arkov Chains and M arkov Processes

  • Dec 01, 2014
  • Wiley StatsRef: Statistics Reference Online
  • Martin Jacobsen
  • Research Article
  • Citations17

Solar Radiation Pressure–Based Orbit Control with Application to Small-Body Landing

  • Nov 27, 2019
  • Journal of Guidance, Control, and Dynamics
  • Kenshiro Oguri +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.