• Home
  • Search
  • Modeling low- and high-order feature interactions with FM and self-attention network
  • Cite Icon25
  • https://doi.org/10.1007/s10489-020-01951-6Copy DOI Icon

Modeling low- and high-order feature interactions with FM and self-attention network

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Click-Through Rate (CTR) prediction has always been a very popular topic. In many online applications, such as online advertising and product recommendation, a small increase in CTR will bring great returns. However, CTR prediction has always faced several challenges. A large number of users and items and the different sizes of the feature space of different data types lead to high-dimensional and sparse input, and high-order feature interactions rely too much on expert knowledge and are very time-consuming. In this paper, we build a novel model called multi-order interactive features aware factorization machine (MoFM) for CTR prediction. To effectively capturing both low-order and high-order interactive features, three different types of prediction models are integrated, of which logistic regression (LR) and factorization machine (FM) model the original features and 2-order interactive features respectively, and a multi-head self-attention network with residual connections is used to automatically identify high-value high-order feature combinations. There is also an embedding layer in the model to realize a unified embedding processing of different data types, avoiding diversification, sparsity, and high dimensionality of features. Since, feature engineering is not required, we can carry out end-to-end model learning. Experiments on three public datasets show the superiority of the proposed model over the state-of-the-art models, and the flexibility and scalability of the model structure have also been verified.

Similar Papers
  • Research Article
  • Citations7

A knowledge distillation-based deep interaction compressed network for CTR prediction

  • Jun 08, 2023
  • Knowledge-Based Systems
  • Fei Guan +2
  • Research Article
  • Citations2

Deep context interaction network based on attention mechanism for click-through rate prediction

  • Dec 16, 2021
  • Journal of Intelligent & Fuzzy Systems
  • Ling Yuan +4
  • Research Article
  • Citations15

A joint learning model for click-through prediction in display advertising

  • Mar 02, 2021
  • Neurocomputing
  • Mengjuan Liu +5
  • PDF
  • Research Article
  • Citations4

A Dual Adaptive Interaction Click-Through Rate Prediction Based on Attention Logarithmic Interaction Network

  • Dec 15, 2022
  • Entropy
  • Shiqi Li +2
  • Conference Article
  • Citations36

Deep Interaction Machine

  • Oct 19, 2020
  • Feng Yu +5
  • Research Article
  • Citations2

Feature refinement and attention enhancement for click-through rate prediction

  • Jan 09, 2025
  • The Computer Journal
  • Sumin Li +2
  • Conference Article
  • Citations3

FINET: Fine-grained Feature Interaction Network for Click-through Rate Prediction

  • Aug 01, 2020
  • Zhibin Lian +1
  • Conference Article
  • Citations10

Multi-head Self-attention Recommendation Model based on Feature Interaction Enhancement

  • May 16, 2022
  • Yunfei Yin +3
  • Research Article
  • Citations14

Multi-scale and multi-channel neural network for click-through rate prediction

  • Jan 20, 2022
  • Neurocomputing
  • Jinjin Zhang +4
  • Research Article
  • Citations24

CAN: Effective cross features by global attention mechanism and neural network for ad click prediction

  • Feb 01, 2022
  • Tsinghua Science and Technology
  • Wenjie Cai +3
  • Conference Article
  • Citations5

A Novel Ensemble Strategy Combining Gradient Boosted Decision Trees and Factorization Machine Based Neural Network for Clicks Prediction

  • Jun 01, 2018
  • Feng Zhou +5
  • Research Article
  • Citations69

HoAFM: A High-order Attentive Factorization Machine for CTR Prediction

  • Jul 22, 2019
  • Information Processing & Management
  • Zhulin Tao +4
  • Conference Article
  • Citations791

Field-aware Factorization Machines for CTR Prediction

  • Sep 07, 2016
  • Yuchin Juan +3
  • Conference Article

Multi-Kernel-FM: A Multi-Embedding & Kernelization Factorization Machine Framework for CTR Prediction

  • Nov 01, 2022
  • Yijun Wang +1
  • Conference Article
  • Citations20

An Ad CTR Prediction Method Based on Feature Learning of Deep and Shallow Layers

  • Nov 06, 2017
  • Zai Huang +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.