• Home
  • Search
  • Information Bottleneck and its Applications in Deep Learning
  • Cite Icon9
  • https://doi.org/10.7508/jist.2018.03.001Copy DOI Icon

Information Bottleneck and its Applications in Deep Learning

  • Dec 29, 2018
  • حسن حافظ کلاهی +1 more
Show More
  • Abstract
  • Literature Map
  • Citations
  • Similar Papers
Abstract

Information Theory (IT) has been used in Machine Learning (ML) from early days of this field. In the last decade, advances in Deep Neural Networks (DNNs) have led to surprising improvements in many applications of ML. The result has been a paradigm shift in the community toward revisiting previous ideas and applications in this new framework. Ideas from IT are no exception. One of the ideas which is being revisited by many researchers in this new era, is Information Bottleneck (IB); a formulation of information extraction based on IT. The IB is promising in both analyzing and improving DNNs. The goal of this survey is to review the IB concept and demonstrate its applications in deep learning. The information theoretic nature of IB, makes it also a good candidate in showing the more general concept of how IT can be used in ML. Two important concepts are highlighted in this narrative on the subject, i) the concise and universal view that IT provides on seemingly unrelated methods of ML, demonstrated by explaining how IB relates to minimal sufficient statistics, stochastic gradient descent, and variational auto-encoders, and ii) the common technical mistakes and problems caused by applying ideas from IT, which is discussed by a careful study of some recent methods suffering from them.

Similar Papers
  • Book Chapter
  • Citations14

Application of Deep Learning and Machine Learning in Pattern Recognition

  • Jan 01, 2022
  • E Fantin Irudaya Raj +1
  • Research Article
  • Citations11

A review on the applications of machine learning and deep learning to groundwater salinity modeling: present status, challenges, and future directions

  • Feb 27, 2025
  • Discover Water
  • Dilip Kumar Roy +4
  • Book Chapter
  • Citations1

Machine and Deep Learning Applications

  • Jan 01, 2022
  • Uday Shankar Shanthamallu +1
  • PDF
  • Research Article
  • Citations47

Deep Ensemble Learning Approaches in Healthcare to Enhance the Prediction and Diagnosing Performance: The Workflows, Deployments, and Surveys on the Statistical, Image-Based, and Sequential Datasets.

  • Oct 14, 2021
  • International Journal of Environmental Research and Public Health
  • Duc-Khanh Nguyen +2
  • Research Article
  • Citations9

Application of Deep Learning in Microbiome

  • Jun 01, 2020
  • Journal of Artificial Intelligence for Medical Sciences
  • Qiang Zhu +4
  • Book Chapter
  • Citations17

Introduction and Literature Review of the Application of Machine Learning/Deep Learning to Load Forecasting in Power System

  • Jan 01, 2021
  • Arash Moradzadeh +4
  • Research Article
  • Citations7

Application of machine learning and deep learning in metabolic dysfunction-associated steatotic liver disease: a systematic review and meta-analysis.

  • May 01, 2026
  • Journal of advanced research
  • Huan Zhang +12
  • Research Article

Deep Learning-Based Synthesis, Classification and Analysis of Sedimentation Boundaries in Analytical Centrifugation Experiments

  • Mar 20, 2026
  • Machine Learning and Knowledge Extraction
  • Moritz Moß +9
  • Conference Article
  • Citations5

Comparative Analysis of Deep Learning and Machine Learning Techniques for Power System Fault type Classification and Location Prediction

  • Jun 01, 2022
  • Sivaramarao Bodda +2
  • Research Article
  • Citations5

Applications of machine learning and deep learning in musculoskeletal medicine: a narrative review

  • May 15, 2025
  • European Journal of Medical Research
  • Martina Feierabend +5
  • PDF
  • Research Article
  • Citations164

Intelligent Health Care: Applications of Deep Learning in Computational Medicine.

  • Apr 12, 2021
  • Frontiers in Genetics
  • Sijie Yang +4
  • Research Article

From Prediction to Decision: A Survey of Machine Learning Applications in Quantitative Finance

  • Feb 24, 2026
  • Advances in Economics, Management and Political Sciences
  • Kintat Ng
  • Book Chapter
  • Citations2

A Survey of Machine Learning and Deep Learning Applications in Genome Editing

  • Dec 10, 2021
  • Talal Almutiri +2
  • Dissertation

Exploring the applicability of machine learning based artificial intelligence in the analysis of cardiovascular imaging

  • Dec 05, 2023
  • Jan-Walter Benjamins
  • PDF
  • Research Article
  • Citations80

Chemist versus Machine: Traditional Knowledge versus Machine Learning Techniques

  • Nov 09, 2020
  • Trends in Chemistry
  • Janine George +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.