• Home
  • Search
  • Open Benchmarking for Click-Through Rate Prediction
  • Open Access IconOpen Access
  • Cite Icon102
  • https://doi.org/10.1145/3459637.3482486Copy DOI Icon

Open Benchmarking for Click-Through Rate Prediction

  • Oct 26, 2021
  • Jieming Zhu +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Click-through rate (CTR) prediction is a critical task for many applications, as its accuracy has a direct impact on user experience and platform revenue. In recent years, CTR prediction has been widely studied in both academia and industry, resulting in a wide variety of CTR prediction models. Unfortunately, there is still a lack of standardized benchmarks and uniform evaluation protocols for CTR prediction research. This leads to non-reproducible or even inconsistent experimental results among existing studies, which largely limit the practical value and potential impact of their research. In this work, we aim to perform open benchmarking for CTR prediction and present a rigorous comparison of different models in a reproducible manner. To this end, we ran over 7,000 experiments for more than 12,000 GPU hours in total to re-evaluate 24 existing models on multiple dataset settings. Surprisingly, our experiments show that with sufficient hyper-parameter search and model tuning, many deep models have smaller differences than expected. The results also reveal that making real progress on the modeling of CTR prediction is indeed a very challenging research task. We believe that our benchmarking work could not only allow researchers to gauge the effectiveness of new models conveniently but also make them fairly compare with the state of the arts. We have publicly released the benchmarking tools, evaluation protocols, and experimental settings of our work to promote reproducible research in this field.

Similar Papers
  • Research Article
  • Citations40

Field-Aware Neural Factorization Machine for Click-Through Rate Prediction

  • Jan 01, 2019
  • IEEE Access
  • Li Zhang +4
  • Conference Article
  • Citations60

MiNet

  • Oct 19, 2020
  • Wentao Ouyang +7
  • Conference Article
  • Citations95

An Embedding Learning Framework for Numerical Features in CTR Prediction

  • Aug 14, 2021
  • Huifeng Guo +5
  • PDF
  • Research Article
  • Citations1

Feature-Interaction-Enhanced Sequential Transformer for Click-Through Rate Prediction

  • Mar 26, 2024
  • Applied Sciences
  • Quan Yuan +4
  • Conference Article
  • Citations1

Context-Aware Modeling via Simulated Exposure Page for CTR Prediction

  • Jul 18, 2023
  • Xiang Li +6
  • PDF
  • Research Article
  • Citations11

XGBDeepFM for CTR Predictions in Mobile Advertising Benefits from Ad Context

  • Apr 27, 2020
  • Mathematical Problems in Engineering
  • Han An +1
  • Conference Article

PRECTR: A Synergistic Framework for Integrating Personalized Search Relevance Matching and CTR Prediction

  • May 08, 2025
  • Rong Chen +4
  • Research Article
  • Citations14

A Hierarchical Extreme Learning Machine Algorithm for Advertisement Click-Through Rate Prediction

  • Jan 01, 2018
  • IEEE Access
  • Sen Zhang +2
  • Conference Article
  • Citations5

AIE: Auction Information Enhanced Framework for CTR Prediction in Online Advertising

  • Oct 08, 2024
  • Yang Yang +8
  • Conference Article
  • Citations80

Improving Ad Click Prediction by Considering Non-displayed Events

  • Nov 03, 2019
  • Bowen Yuan +6
  • Conference Article
  • Citations1

ELEC: Efficient Large Language Model-Empowered Click-Through Rate Prediction

  • Jul 13, 2025
  • Rui Dong +2
  • PDF
  • Research Article
  • Citations4

A Dual Adaptive Interaction Click-Through Rate Prediction Based on Attention Logarithmic Interaction Network

  • Dec 15, 2022
  • Entropy
  • Shiqi Li +2
  • Research Article
  • Citations7

A knowledge distillation-based deep interaction compressed network for CTR prediction

  • Jun 08, 2023
  • Knowledge-Based Systems
  • Fei Guan +2
  • Conference Article
  • Citations17

Regularized Adversarial Sampling and Deep Time-aware Attention for Click-Through Rate Prediction

  • Nov 03, 2019
  • Yikai Wang +7
  • Research Article
  • Citations74

Deep Match to Rank Model for Personalized Click-Through Rate Prediction

  • Apr 03, 2020
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Ze Lyu +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.