• Home
  • Search
  • On Exact Embedding Framework for Optimal Control of Markov Decision Processes
  • Cite Icon3
  • https://doi.org/10.1109/tac.2023.3301818Copy DOI Icon

On Exact Embedding Framework for Optimal Control of Markov Decision Processes

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

This article deals with the embedding framework of Markov decision processes (MDPs) with discrete state and action space to find optimal actions. The optimal control problem of MDPs can be efficiently tackled by restructuring the same into an equivalent linearly-solvable Markov decision processes (LMDPs) through the method called embedding. However, state costs under the embedding may not exactly match the original costs and even assume unrealistic values. In this work, we derive a constructive sufficient condition to devise an exact embedding solution rendering the embedded state cost to match the original system. Furthermore, since, in this case, the embedding implies a transition from the discrete to continuous action space, the correlation between the obtained continuous action and an equivalent desired discrete action is investigated using a maximum a posteriori probability-based method. Finally, some examples, including mammalian cell-cycle network, are presented to demonstrate the effectiveness of the proposed method.

Similar Papers
  • Book Chapter
  • Citations1

A Robust Reinforcement Learning System Using Concept of Sliding Mode Control for Unknown Nonlinear Dynamical System

  • Apr 11, 2011
  • Masanao Obayashi +5
  • PDF
  • Research Article
  • Citations7

Utilizing Human Feedback in Autonomous Driving: Discrete vs. Continuous

  • Jul 26, 2022
  • Machines
  • Maryam Savari +1
  • Research Article
  • Citations3

Three‐Dimensional Trajectory Tracking Control for Stratospheric Airship Based on Deep Reinforcement Learning

  • Jan 01, 2025
  • International Journal of Aerospace Engineering
  • Xixiang Yang +3
  • Research Article
  • Citations4

Deterministic Policy Optimization by Combining Pathwise and Score Function Estimators for Discrete Action Spaces

  • Apr 29, 2018
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Daniel Levy +1
  • Research Article
  • Citations18

Markov chain optimization of repair and replacement decisions of medical equipment

  • May 24, 2021
  • Resources, Conservation and Recycling
  • Hao-Yu Liao +2
  • Conference Article
  • Citations1

Approximate policy improvement for continuous action set-point regulation problems

  • May 04, 1998
  • A.O Esogbue +1
  • Conference Article
  • Citations9

Intelligent Resource Allocations for IRS-Assisted OFDM Communications: A Hybrid MDQN-DDPG Approach

  • May 16, 2022
  • Wei Wu +5
  • Conference Article
  • Citations2

Application of Single Neuron Model to Motion Planning and Control of Under-actuated Robot by MDP Frame Work

  • Jun 27, 2005
  • H Kawano
  • Book Chapter
  • Citations1

Improving Human Behavior Using POMDPs with Gestures and Speech Recognition

  • Sep 04, 2018
  • João A Garcia +1
  • Research Article

Advancing additive manufacturing: a Markov decision process approach for real-time quality assurance

  • Aug 11, 2025
  • Boletim da Sociedade Paranaense de Matemática
  • Muhammad Shoaib +4
  • Conference Article
  • Citations10

Increasing the throughput of HARQ via multi-packet transmission

  • Dec 01, 2014
  • Aata El Hamss +2
  • Research Article
  • Citations6

Policy Derivation Methods for Critic-Only Reinforcement Learning in Continuous Action Spaces

  • Jan 01, 2016
  • IFAC PapersOnLine
  • Eduard Alibekov +2
  • Research Article

Domains as objectives: Domain-uncertainty-aware policy optimization through explicit multi-domain convex coverage set learning

  • Aug 19, 2025
  • The International Journal of Robotics Research
  • Wendyam Eric Lionel Ilboudo +2
  • Research Article
  • Citations15

The approximation of GSTAR model for discrete cases through INAR model

  • Jan 01, 2021
  • Journal of Physics: Conference Series
  • N M Huda +2
  • Book Chapter
  • Citations5

A Neuro-fuzzy Learning System for Adaptive Swarm Behaviors Dealing with Continuous State Space

  • Jan 01, 2008
  • Takashi Kuremoto +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.