• Home
  • Search
  • Threshold UCT: Cost-Constrained Monte Carlo Tree Search with Pareto Curves
  • https://doi.org/10.1609/aaai.v39i25.34858Copy DOI Icon

Threshold UCT: Cost-Constrained Monte Carlo Tree Search with Pareto Curves

Show More
  • Abstract
  • Literature Map
  • Similar Papers
Abstract

Constrained Markov decision processes (CMDPs), in which the agent optimizes expected payoffs while keeping the expected cost below a given threshold, are the leading framework for safe sequential decision making under stochastic uncertainty. Among algorithms for planning and learning in CMDPs, methods based on Monte Carlo tree search (MCTS) have particular importance due to their efficiency and extendibility to more complex frameworks (such as partially observable settings and games). However, current MCTS-based methods for CMDPs either struggle with finding safe (i.e., constraint-satisfying) policies, or are too conservative and do not find valuable policies. We introduce Threshold UCT (T-UCT), an online MCTS-based algorithm for CMDP planning. Unlike previous MCTS-based CMDP planners, T-UCT explicitly estimates Pareto curves of cost-utility trade-offs throughout the search tree, using these together with a novel action selection and threshold update rule to seek safe and valuable policies. Our experiments demonstrate that our approach significantly outperforms state-of-the-art methods from the literature.

Similar Papers
  • Conference Article
  • Citations3

Monte Carlo Tree Search with Variable Simulation Periods for Continuously Running Tasks

  • Nov 01, 2019
  • Seydou Ba +4
  • Dissertation

Integrating Domain knowledge into Monte Carlo Tree Search for Real-Time Strategy Games

  • Jun 01, 2022
  • Zuozhi Yang +1
  • Research Article

Hedging of financial derivative contracts via Monte Carlo tree search

  • Jan 01, 2023
  • The Journal of Computational Finance
  • Oleg Szehr
  • Research Article
  • Citations3

Evolving the MCTS Upper Confidence Bounds for Trees Using a Semantic-Inspired Evolutionary Algorithm in the Game of Carcassonne

  • Sep 01, 2023
  • IEEE Transactions on Games
  • Edgar Galván +2
  • Research Article
  • Citations10

Collective intelligence evolution using ant colony optimization and neural networks

  • Apr 21, 2021
  • Neural Computing and Applications
  • Xiaoya Qi +6
  • PDF
  • Research Article
  • Citations328

Monte Carlo Tree Search: a review of recent modifications and applications

  • Jul 19, 2022
  • Artificial Intelligence Review
  • Maciej Świechowski +3
  • Research Article

Fuegito: an Educational Software Package for Game Tree Search

  • Jan 01, 2013
  • University of Alberta Library
  • Colin Hunt +1
  • Research Article
  • Citations7

Incentive Learning in Monte Carlo Tree Search

  • Dec 01, 2013
  • IEEE Transactions on Computational Intelligence and AI in Games
  • Kuo-Yuan Kao +3
  • PDF
  • Research Article
  • Citations7

Scalable Online Planning for Multi-Agent MDPs

  • Mar 10, 2022
  • Journal of Artificial Intelligence Research
  • Shushman Choudhury +3
  • Research Article
  • Citations5

삼목 게임에 적용된 몬테카를로 트리탐색

  • Jun 30, 2014
  • Journal of Korea Game Society
  • Byung-Doo Lee
  • Research Article
  • Citations75

Real-Time Monte Carlo Tree Search in Ms Pac-Man

  • Sep 01, 2014
  • IEEE Transactions on Computational Intelligence and AI in Games
  • Tom Pepels +2
  • Book Chapter

Parallel Monte Carlo Tree Search in General Video Game Playing

  • Jan 01, 2020
  • Luis G S Centeleghe +2
  • Book Chapter
  • Citations2

Efficient Real-Parameter Single Objective Optimizer Using Hierarchical CMA-ES Solvers

  • Nov 11, 2017
  • Madalina M Drugan
  • PDF
  • Research Article

Automating the static and seismic design of 2-D multistorey reinforced concrete structures by using Monte Carlo Tree Search and Genetic Algorithm

  • Nov 03, 2025
  • AI in Civil Engineering
  • Leonardo Rossi +1
  • Conference Article

Point-Based Incremental Pruning for Monte-Carlo Tree Search

  • Jul 01, 2017
  • Bo Wu +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.