• Home
  • Search
  • A clustering-based graph Laplacian framework for value function approximation in reinforcement learning.
  • Cite Icon38
  • https://doi.org/10.1109/tcyb.2014.2311578Copy DOI Icon

A clustering-based graph Laplacian framework for value function approximation in reinforcement learning.

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In order to deal with the sequential decision problems with large or continuous state spaces, feature representation and function approximation have been a major research topic in reinforcement learning (RL). In this paper, a clustering-based graph Laplacian framework is presented for feature representation and value function approximation (VFA) in RL. By making use of clustering-based techniques, that is, K-means clustering or fuzzy C-means clustering, a graph Laplacian is constructed by subsampling in Markov decision processes (MDPs) with continuous state spaces. The basis functions for VFA can be automatically generated from spectral analysis of the graph Laplacian. The clustering-based graph Laplacian is integrated with a class of approximation policy iteration algorithms called representation policy iteration (RPI) for RL in MDPs with continuous state spaces. Simulation and experimental results show that, compared with previous RPI methods, the proposed approach needs fewer sample points to compute an efficient set of basis functions and the learning control performance can be improved for a variety of parameter settings.

Similar Papers
  • Research Article
  • Citations44

Manifold-Based Reinforcement Learning via Locally Linear Reconstruction.

  • Jan 27, 2016
  • IEEE Transactions on Neural Networks and Learning Systems
  • Xin Xu +3
  • Research Article

Online Attentive Kernel-Based Off-Policy Temporal Difference Learning

  • Nov 28, 2024
  • Applied Sciences
  • Shangdong Yang +2
  • Book Chapter
  • Citations1

A Survey of Linear Value Function Approximation in Reinforcement Learning

  • Jan 01, 2022
  • Shicheng Guo +5
  • Book Chapter
  • Citations9

Batch Reinforcement Learning with State Importance

  • Jan 01, 2004
  • Lihong Li +2
  • Supplementary Content

A New Method of Model Selection for Value Function Approximation in Reinforcement Learning

  • Jan 01, 2008
  • JSAI Technical Report, Type 2 SIG
  • Hirotaka Hachiya +2
  • Book Chapter
  • Citations5

Revisiting the Cross Entropy Method with Applications in Stochastic Global Optimization and Reinforcement Learning

  • Jan 01, 2016
  • Frontiers in artificial intelligence and applications
  • Joseph Ajin George +1
  • Conference Article
  • Citations21

Symbolic method for deriving policy in reinforcement learning

  • Dec 01, 2016
  • Eduard Alibekov +2
  • Conference Article
  • Citations10

A Deep Reinforcement Learning Approach to Asset-Liability Management

  • Oct 01, 2019
  • Alan Fontoura +2
  • Research Article
  • Citations5

Relative Q-Learning for Average-Reward Markov Decision Processes With Continuous States

  • Oct 01, 2024
  • IEEE Transactions on Automatic Control
  • Xiangyu Yang +2
  • Dissertation

The effectiveness of numerical approximation for dynamic programming problems

  • Nov 07, 2019
  • Wyatt Jones
  • Research Article
  • Citations4

Active Inference and Reinforcement Learning: A Unified Inference on Continuous State and Action Spaces Under Partial Observability.

  • Sep 17, 2024
  • Neural computation
  • Parvin Malekzadeh +1
  • Book Chapter

Behavior Learning of Autonomous Agents in Continuous State Using Function Approximation

  • Jan 01, 2004
  • Min-Kyu Shon +1
  • Research Article
  • Citations145

A Generic Markov Decision Process Model and Reinforcement Learning Method for Scheduling Agile Earth Observation Satellites

  • Sep 25, 2020
  • IEEE Transactions on Systems, Man, and Cybernetics: Systems
  • Yongming He +5
  • Research Article
  • Citations1

Off-Policy Temporal Difference Learning with Bellman Residuals

  • Nov 18, 2024
  • Mathematics
  • Shangdong Yang +2
  • Conference Article
  • Citations3

Relevance Vector Sampling for Reinforcement Learning in Continuous Action Space

  • Dec 01, 2016
  • Minwoo Lee +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.