• Home
  • Search
  • A Modular Machine Learning System for Flow-Level Traffic Classification in Large Networks
  • Open Access IconOpen Access
  • Cite Icon102
  • https://doi.org/10.1145/2133360.2133364Copy DOI Icon

A Modular Machine Learning System for Flow-Level Traffic Classification in Large Networks

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The ability to accurately and scalably classify network traffic is of critical importance to a wide range of management tasks of large networks, such as tier-1 ISP networks and global enterprise networks. Guided by the practical constraints and requirements of traffic classification in large networks, in this article, we explore the design of an accurate and scalable machine learning based flow-level traffic classification system, which is trained on a dataset of flow-level data that has been annotated with application protocol labels by a packet-level classifier. Our system employs a lightweight modular architecture , which combines a series of simple linear binary classifiers, each of which can be efficiently implemented and trained on vast amounts of flow data in parallel, and embraces three key innovative mechanisms, weighted threshold sampling, logistic calibration , and intelligent data partitioning , to achieve scalability while attaining high accuracy. Evaluations using real traffic data from multiple locations in a large ISP show that our system accurately reproduces the labels of the packet level classifier when runs on (unlabeled) flow records, while meeting the scalability and stability requirements of large ISP networks. Using training and test datasets that are two months apart and collected from two different locations, the flow error rates are only 3% for TCP flows and 0.4% for UDP flows. We further show that such error rates can be reduced by combining the information of spatial distributions of flows, or collective traffic statistics , during classification. We propose a novel two-step model, which seamlessly integrates these collective traffic statistics into the existing traffic classification system. Experimental results display performance improvement on all traffic classes and an overall error rate reduction by 15%. In addition to a high accuracy, at runtime, our implementation easily scales to classify traffic on 10Gbps links.

Similar Papers
  • Book Chapter
  • Citations10

On Addressing the Imbalance Problem: A Correlated KNN Approach for Network Traffic Classification

  • Jan 01, 2014
  • Di Wu +5
  • Research Article

DDos Attacks-Applications for Traffic Classification using ML Algorithms

  • Dec 01, 2023
  • INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • Prof.Sameera Sultana
  • Conference Article
  • Citations7

Scaling data-plane logging in large scale networks

  • Nov 01, 2011
  • Ahsan Arefin +3
  • Research Article
  • Citations11

Optimizing Node Capacity in Multilayer Networks

  • May 01, 2011
  • IEEE Communications Letters
  • Iyad Katib +1
  • PDF
  • Research Article
  • Citations4

Simple regression models as a threshold for selecting AFLP loci with reduced error rates

  • Oct 16, 2012
  • BMC Bioinformatics
  • David L Price +1
  • Conference Article
  • Citations42

Boosting the scalability of botnet detection using adaptive traffic sampling

  • Mar 22, 2011
  • Junjie Zhang +5
  • Research Article
  • Citations170

Cryptic species of planktonic foraminifera: their effect on palaeoceanographic reconstructions.

  • Apr 15, 2002
  • Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences
  • Michal Kucera +1
  • Research Article
  • Citations6

Traffic classification and scheduling in ATM networks

  • Dec 01, 1993
  • Telecommunication Systems
  • Sanjay Gupta +1
  • PDF
  • Research Article
  • Citations18

An Enhanced Deep Learning Approach in Forecasting Banana Harvest Yields

  • Jan 01, 2019
  • International Journal of Advanced Computer Science and Applications
  • Mariannie A Rebortera +1
  • Research Article
  • Citations3

Error Tolerant DNA Self-Assembly Using (<formula formulatype="inline"><tex> $\hbox{2}k-\hbox{1}$</tex></formula>)<formula formulatype="inline"><tex>$\times$</tex></formula>( <formula formulatype="inline"><tex>$\hbox{2}k-\hbox{1}$</tex></formula>) Snake Tile Sets

  • Mar 01, 2008
  • IEEE Transactions on NanoBioscience
  • Xiaojun Ma +2
  • Research Article
  • Citations38

Impact of an intervention to reduce prescribing errors in a pediatric intensive care unit

  • Jun 12, 2012
  • Intensive Care Medicine
  • Amalia Martinez-Anton +2
  • Addendum
  • Citations4

Retraction Note to: Traffic identification and traffic analysis based on support vector machine

  • Sep 09, 2021
  • Neural Computing and Applications
  • Youchan Zhu +1
  • Abstract
  • Citations1

G217(P) Quality Improvement Project: ‘Safe Prescribing’ in St Georges Hospital paediatric department

  • Apr 01, 2014
  • Archives of Disease in Childhood
  • M Leach +2
  • Research Article
  • Citations64

Effect of Barcode-assisted Medication Administration on Emergency Department Medication Errors

  • Aug 01, 2013
  • Academic Emergency Medicine
  • Joseph Bonkowski +7
  • Research Article
  • Citations5

A long lifetime, low error rate RRAM design with self-repair module**Project supported by the New Century Excellent Talents in University (No. NCET-12-0165) and the National Natural Science Foundation of China (Nos. 61472123, 61272396).

  • Nov 01, 2016
  • Journal of Semiconductors
  • Zhiqiang You +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.