• Home
  • Search
  • PaddleBox: Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising
  • Cite Icon3
  • https://doi.org/10.1109/bigdata55660.2022.10021133Copy DOI Icon

PaddleBox: Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising

  • Dec 17, 2022
  • Weijie Zhao +5 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Click-through rate (CTR) prediction is one of the most crucial components in the online advertising industry. In order to produce a personalized CTR prediction, an industry-level CTR prediction model commonly takes a high-dimensional (∼ 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">12</sup> ) sparse vector (that is encoded from query keywords, user portraits, etc.) as input. As a result, the model requires Terabyte scale parameters to embed the high-dimensional input. Hierarchical distributed GPU parameter server has been developed at Baidu to enable GPU with limited memory to train the massive network by leveraging CPU main memory and SSDs as secondary storage. In this work, we identify two major challenges in the existing GPU training framework for massive-scale ad models and propose a collection of optimizations to tackle these challenges: (a) the GPU, CPU, SSD rapidly communicate with each other during the training. The connections between GPUs and CPUs are non-uniform due to the hardware topology. The data communication route should be optimized according to the hardware topology; (b) GPUs in different computing nodes frequently communicates to synchronize parameters. It is thus required to optimize the communications so that the distributed system can become scalable. In this paper, we propose a hardware-aware training workflow that couples the hardware topology into the algorithm design. To reduce the extensive communication between computing nodes, we introduce a k-step model merging algorithm for Adam and provide its convergence rate in non-convex optimization. To the best of our knowledge, this is the first application of k-step adaptive optimization method in industrial CTR model training. Experiments on commercial search ads data confirm the effectiveness of our proposed training framework.

Similar Papers
  • Conference Article
  • Citations1

STEC: See-Through Transformer-based Encoder for CTR Prediction

  • Apr 05, 2025
  • Serdarcan Dilbaz +1
  • Conference Article
  • Citations79

Improving Ad Click Prediction by Considering Non-displayed Events

  • Nov 03, 2019
  • Bowen Yuan +6
  • PDF
  • Research Article
  • Citations4

A Dual Adaptive Interaction Click-Through Rate Prediction Based on Attention Logarithmic Interaction Network

  • Dec 15, 2022
  • Entropy
  • Shiqi Li +2
  • Conference Article
  • Citations4

AIE: Auction Information Enhanced Framework for CTR Prediction in Online Advertising

  • Oct 08, 2024
  • Yang Yang +8
  • Conference Article
  • Citations93

An Embedding Learning Framework for Numerical Features in CTR Prediction

  • Aug 14, 2021
  • Huifeng Guo +5
  • Conference Article
  • Citations1

ELEC: Efficient Large Language Model-Empowered Click-Through Rate Prediction

  • Jul 13, 2025
  • Rui Dong +2
  • Research Article

Beyond KAN: Introducing KarSein for Adaptive High-Order Feature Interaction Modeling in CTR Prediction

  • Feb 28, 2026
  • ACM Transactions on Information Systems
  • Yunxiao Shi +4
  • PDF
  • Conference Article
  • Citations240

Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising

  • Jan 01, 2018
  • Junwei Pan +6
  • Conference Article
  • Citations26

ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Prediction

  • May 13, 2024
  • Jianghao Lin +9
  • Conference Article
  • Citations59

Deep Spatio-Temporal Neural Networks for Click-Through Rate Prediction

  • Jul 25, 2019
  • Wentao Ouyang +6
  • Conference Article
  • Citations8

Position Awareness Modeling with Knowledge Distillation for CTR Prediction

  • Sep 13, 2022
  • Congcong Liu +7
  • Conference Article
  • Citations1

Graph Isomorphism Network-Based Cohort Modeling In Click-Through Rate Prediction

  • Jul 13, 2025
  • Xuan Ma +8
  • Research Article

MLIM: A CTR prediction model describing evolution law of user interest

  • May 17, 2022
  • Web Intelligence
  • Zilong Jiang +2
  • Research Article
  • Citations1

A Review of Click-Through Rate Prediction Using Deep Learning

  • Sep 21, 2025
  • Electronics
  • Shuaa Alotaibi +1
  • Conference Article
  • Citations6

Discrete Semantic Tokenization for Deep CTR Prediction

  • May 13, 2024
  • Qijiong Liu +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.