• Home
  • Search
  • Partitioning the state space by critical states
  • Cite Icon5
  • https://doi.org/10.1109/bicta.2009.5338123Copy DOI Icon

Partitioning the state space by critical states

  • Oct 1, 2009
  • Zhao Jin +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

For scaling up reinforcement learning to large and complex problems, we propose an approach to partition the larger state space into multiple smaller state spaces based the critical states for decomposing learning task. During learning process, we record every training episode, and eliminate the state loops existed in it. We find some states have high probability (even to 1) appeared in all these acyclic episodes. We call these states critical states. That means, if agent wants to reach the goal state, then it will have high probability to pass these critical states according to the learned experience. So the critical states can be used to partition the state space for accomplishing learning task by stages. We also prove that the optimal policy found in the partitioned smaller state space is equivalent to the optimal policy found in the original state space. The experiment comparisons between Q-learning and Q-learning with critical states demonstrate our approach more effective. The more important is that our approach brings the light of how agent can use its own experience to plan learning for better performance.

Similar Papers
  • Research Article

State-Partition-Based Control of Discrete Event Systems for Enforcement of Regular Language Specifications

  • Jan 01, 2014
  • IFAC Proceedings Volumes
  • Antoine Butez +2
  • Book Chapter
  • Citations4

Embedding Theory: Introduction and Applications to Time Series Analysis

  • Jan 01, 2002
  • F Strozzi +1
  • Research Article
  • Citations44

Manifold-Based Reinforcement Learning via Locally Linear Reconstruction.

  • Jan 27, 2016
  • IEEE Transactions on Neural Networks and Learning Systems
  • Xin Xu +3
  • Book Chapter
  • Citations27

Symmetry and Symmetry-Breaking in Thermodynamic and Epistemic Engines: A Coupling of First and Second Laws

  • Jan 01, 1990
  • P N Kugler +1
  • Research Article
  • Citations38

Are extreme dissipation events predictable in turbulent fluid flows?

  • Apr 17, 2019
  • Physical Review Fluids
  • Patrick J Blonigan +2
  • Research Article
  • Citations3

Robust Fuzzy Model Predictive Control with time delays for Nonlinear Systems

  • Jan 01, 2021
  • Procedia Computer Science
  • Sofiane Bououden +3
  • Book Chapter
  • Citations1

Jump Longer to Jump Less: Improving Dynamic Boundary Projection with h-Scaling

  • Jan 01, 2022
  • Francesca Randone +2
  • Book Chapter
  • Citations9

A Nested Depth First Search Algorithm for Model Checking with Symmetry Reduction

  • Jan 01, 2002
  • Dragan Bošnački
  • Conference Article
  • Citations83

Learned Critical Probabilistic Roadmaps for Robotic Motion Planning

  • Jan 04, 2020
  • Brian Ichter +3
  • Research Article
  • Citations5

Safe deployment of a reinforcement learning robot using self stabilization

  • Nov 01, 2022
  • Intelligent Systems with Applications
  • Nanda Kishore Sreenivas +1
  • Conference Article
  • Citations24

A novel orthogonal NMF-based belief compression for POMDPs

  • Jun 20, 2007
  • Xin Li +3
  • Research Article
  • Citations16

Functional safety analysis of safety-critical system using state transition diagram

  • May 18, 2022
  • RADIOELECTRONIC AND COMPUTER SYSTEMS
  • Leonid Ozirkovskyy +4
  • Conference Article
  • Citations6

A reinforcement learning using adaptive state space construction strategy for real autonomous mobile robots

  • Aug 05, 2002
  • T Kondo +1
  • Conference Article
  • Citations10

Autonomous reconstruction of state space for learning of robot behavior

  • Oct 31, 2000
  • T Yairi +2
  • Research Article
  • Citations2

Reduction of state space in reinforcement learning by sensor selection

  • Jan 31, 2013
  • Artificial Life and Robotics
  • Yasutaka Kishima +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.