• Home
  • Search
  • ROLeR: Effective Reward Shaping in Offline Reinforcement Learning for Recommender Systems
  • Cite Icon6
  • https://doi.org/10.1145/3627673.3679633Copy DOI Icon

ROLeR: Effective Reward Shaping in Offline Reinforcement Learning for Recommender Systems

  • Oct 21, 2024
  • Yi Zhang +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Offline reinforcement learning (RL) is an effective tool for real-world recommender systems with its capacity to model the dynamic interest of users and its interactive nature. Most existing offline RL recommender systems focus on model-based RL through learning a world model from offline data and building the recommendation policy by interacting with this model. Although these methods have made progress in the recommendation performance, the effectiveness of model-based offline RL methods is often constrained by the accuracy of the estimation of the reward model and the model uncertainties, primarily due to the extreme discrepancy between offline logged data and real-world data in user interactions with online platforms. To fill this gap, a more accurate reward model and uncertainty estimation are needed for the model-based RL methods. In this paper, a novel model-based Reward Shaping in Offline Reinforcement Learning for Recommender Systems, ROLeR, is proposed for reward and uncertainty estimation in recommendation systems. Specifically, a non-parametric reward shaping method is designed to refine the reward model. In addition, a flexible and more representative uncertainty penalty is designed to fit the needs of recommendation systems. Extensive experiments conducted on four benchmark datasets showcase that ROLeR achieves state-of-the-art performance compared with existing baselines. Source code can be downloaded at this address.

Similar Papers
  • Research Article
  • Citations13

Reinforcement Learning for Clinical Applications.

  • Feb 08, 2023
  • Clinical Journal of the American Society of Nephrology
  • Kia Khezeli +5
  • PDF
  • Research Article
  • Citations7

Improving Model-Based Deep Reinforcement Learning with Learning Degree Networks and Its Application in Robot Control

  • Mar 04, 2022
  • Journal of Robotics
  • Guoqing Ma +3
  • Research Article
  • Citations4

Dyna-style Model-based reinforcement learning with Model-Free Policy Optimization

  • Jan 29, 2024
  • Knowledge-Based Systems
  • Kun Dong +8
  • Book Chapter
  • Citations5

Knowledge-Based Reinforcement Learning for Data Mining

  • Jan 01, 2009
  • Daniel Kudenko +1
  • Conference Article
  • Citations2

Context-dependent meta-control for reinforcement learning using a Dirichlet process Gaussian mixture model

  • Jan 01, 2018
  • Dongjae Kim +1
  • Conference Article
  • Citations1

Comparison of Reinforcement Learning Methods for Production Control in Discrete Manufacturing Systems

  • Jun 12, 2023
  • Lingxiang Yun +3
  • Conference Article
  • Citations3

Reward Shaping in Episodic Reinforcement Learning

  • May 08, 2017
  • Marek Grześ
  • PDF
  • Research Article
  • Citations59

Reinforcement Learning for Pick and Place Operations in Robotics: A Survey

  • Sep 13, 2021
  • Robotics
  • Andrew Lobbezoo +2
  • Conference Article
  • Citations1

Mismatched No More: Joint Model-Policy Optimization for Model-Based RL

  • Jan 01, 2022
  • Benjamin Eysenbach +3
  • Conference Article
  • Citations5

Learning Locomotion Skills via Model-based Proximal Meta-Reinforcement Learning

  • Oct 01, 2019
  • Qing Xiao +2
  • Research Article
  • Citations20

Heterogeneous relational reasoning in knowledge graphs with reinforcement learning

  • Jul 13, 2022
  • Information Fusion
  • Mandana Saebi +5
  • Research Article

Reward Model Evaluation via Automatically-Ranked Policy Alignment

  • Mar 14, 2026
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Aoran Wang +3
  • Research Article
  • Citations39

Navigating complex decision spaces: Problems and paradigms in sequential choice.

  • Jan 01, 2014
  • Psychological Bulletin
  • Matthew M Walsh +1
  • PDF
  • Research Article
  • Citations32

Parallel model-based and model-free reinforcement learning for card sorting performance

  • Sep 22, 2020
  • Scientific Reports
  • Alexander Steinke +2
  • Research Article
  • Citations22

A Deep Ensemble Method for Multi-Agent Reinforcement Learning: A Case Study on Air Traffic Control

  • May 17, 2021
  • Proceedings of the International Conference on Automated Planning and Scheduling
  • Supriyo Ghosh +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.