• Home
  • Search
  • Off-Policy Temporal Difference Learning with Bellman Residuals
  • Cite Icon1
  • https://doi.org/10.3390/math12223603Copy DOI Icon

Off-Policy Temporal Difference Learning with Bellman Residuals

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In reinforcement learning, off-policy temporal difference learning methods have gained significant attention due to their flexibility in utilizing existing data. However, traditional off-policy temporal difference methods often suffer from poor convergence and stability when handling complex problems. To address these issues, this paper proposes an off-policy temporal difference algorithm with Bellman residuals (TDBR). By incorporating Bellman residuals, the proposed algorithm effectively improves the convergence and stability of the off-policy learning process. This paper first introduces the basic concepts of reinforcement learning and value function approximation, highlighting the importance of Bellman residuals in off-policy learning. Then, the theoretical foundation and implementation details of the TDBR algorithm are described in detail. Experimental results in multiple benchmark environments demonstrate that the TDBR algorithm significantly outperforms traditional methods in terms of both convergence speed and solution quality. Overall, the TDBR algorithm provides an effective and stable solution for off-policy reinforcement learning with broad application prospects. Future research can further optimize the algorithm parameters and extend its application to continuous state and action spaces to enhance its applicability and performance in real-world problems.

Similar Papers
  • Conference Article
  • Citations3

Relevance Vector Sampling for Reinforcement Learning in Continuous Action Space

  • Dec 01, 2016
  • Minwoo Lee +1
  • Research Article
  • Citations5

Representation learning for continuous action spaces is beneficial for efficient policy learning

  • Dec 16, 2022
  • Neural Networks
  • Tingting Zhao +5
  • Conference Article
  • Citations3

A reinforcement learning algorithm developed to model GenCo strategic bidding behavior in multidimensional and continuous state and action spaces

  • Apr 01, 2013
  • Alfred Yong Fu Lau +2
  • Research Article
  • Citations38

A clustering-based graph Laplacian framework for value function approximation in reinforcement learning.

  • Apr 25, 2014
  • IEEE transactions on cybernetics
  • Xin Xu +3
  • PDF
  • Research Article
  • Citations17

Cooperative Path-Planning for Multi-Vehicle Systems

  • Nov 17, 2014
  • Electronics
  • Qichen Wang +1
  • Research Article
  • Citations1

DDPG Agent to Swing Up and Balance Cart- Pole System

  • Apr 09, 2021
  • International Journal of Advanced Research in Science, Communication and Technology
  • Buvanesh Pandian V
  • Conference Article
  • Citations363

PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-Based Planning

  • May 01, 2018
  • Aleksandra Faust +6
  • Research Article
  • Citations4

Active Inference and Reinforcement Learning: A Unified Inference on Continuous State and Action Spaces Under Partial Observability.

  • Sep 17, 2024
  • Neural computation
  • Parvin Malekzadeh +1
  • Research Article
  • Citations68

A data-driven output voltage control of solid oxide fuel cell using multi-agent deep reinforcement learning

  • Sep 09, 2021
  • Applied Energy
  • Jiawen Li +2
  • Conference Article
  • Citations36

Beating the Stock Market with a Deep Reinforcement Learning Day Trading System

  • Jul 01, 2020
  • Leonardo Conegundes +1
  • Dissertation

The effectiveness of numerical approximation for dynamic programming problems

  • Nov 07, 2019
  • Wyatt Jones
  • Research Article
  • Citations8

A Study of Continuous Maximum Entropy Deep Inverse Reinforcement Learning

  • Jan 01, 2019
  • Mathematical Problems in Engineering
  • Xi-Liang Chen +4
  • Conference Article
  • Citations8

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

  • Jul 01, 2022
  • Zhixuan Liang +4
  • PDF
  • Research Article
  • Citations30

Optimal dispatch of PV inverters in unbalanced distribution systems using Reinforcement Learning

  • Oct 13, 2021
  • International Journal of Electrical Power & Energy Systems
  • Pedro P Vergara +3
  • Conference Article
  • Citations1

Moving in a Simulated Environment Through Deep Reinforcement Learning

  • Sep 07, 2022
  • Javier Esarte +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.