• Home
  • Search
  • Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges
  • Open Access IconOpen Access
  • Cite Icon179
  • https://doi.org/10.1109/tai.2021.3067574Copy DOI Icon

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN). These methods play a crucial role in the success or failure of the DNN for most problems and applications. Evolutionary algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimization of DNNs. Neuroevolution is a term, which describes these processes of automated configuration and training of DNNs using EAs. While many works exist in the literature, no comprehensive surveys currently exist focusing exclusively on the strengths and limitations of using neuroevolution approaches in DNNs. Absence of such surveys can lead to a disjointed and fragmented field preventing DNNs researchers potentially adopting neuroevolutionary methods in their own research, resulting in lost opportunities for wider application within real-world deep learning problems. This article presents a comprehensive survey, discussion, and evaluation of the state-of-the-art in using EAs for architectural configuration and training of DNNs. This article highlights the most pertinent current issues and challenges in neuroevolution and identifies multiple promising future research directions. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>Impact Statement—</i>The concept of deep learning originated from the study of artificial neural networks (ANNs). ANNs have achieved extraordinary results in a variety of diverse application areas. Numerous methods have been applied to the architectural configuration and learning or training of artificial DNN and these methods play a crucial role in the success or failure of the DNN for most problems and applications. Recently, EAs have been gaining momentum as a computationally feasible method (called neuroevolution) for the automated configuration and learning or training of DNNs. This article reviews over 170 recent scientific papers describing how major EAs paradigms are being applied by researchers to the configuration and optimization of multiple DNNs. By articulating a clear understanding of the context, state-of-the-art, and feasibility of Neuroevolution, researchers in AI, EAs, and DNN will benefit from this article. The impact of this article comes from contributing toward enhancing research capacity, knowledge, and skills for researchers currently working in neuroevolution and actively engaging those considering becoming involved in this area.

Similar Papers
  • Research Article
  • Citations6

A Guessing Entropy-Based Framework for Deep Learning-Assisted Side-Channel Analysis

  • Jan 01, 2023
  • IEEE Transactions on Information Forensics and Security
  • Ziyue Zhang +2
  • Conference Article
  • Citations8

A Framework for Distributed Deep Neural Network Training with Heterogeneous Computing Platforms

  • Dec 01, 2019
  • Bontak Gu +3
  • PDF
  • Research Article
  • Citations58

A new deep neural network for forecasting: Deep dendritic artificial neural network

  • Jun 11, 2024
  • Artificial Intelligence Review
  • Erol Egrioglu +1
  • Conference Article
  • Citations3

AccDP: Accelerated Data-Parallel Distributed DNN Training for Modern GPU-Based HPC Clusters

  • Dec 01, 2022
  • Nawras Alnaasan +4
  • Conference Article
  • Citations4

MalleTrain: Deep Neural Networks Training on Unfillable Supercomputer Nodes

  • May 07, 2024
  • Xiaolong Ma +6
  • Research Article
  • Citations7

A convergence analysis of Nesterov’s accelerated gradient method in training deep linear neural networks

  • Sep 05, 2022
  • Information Sciences
  • Xin Liu +2
  • Conference Article
  • Citations3

ADA-GP: Accelerating DNN Training By Adaptive Gradient Prediction

  • Oct 28, 2023
  • Vahid Janfaza +3
  • Research Article
  • Citations59

HNPU: An Adaptive DNN Training Processor Utilizing Stochastic Dynamic Fixed-Point and Active Bit-Precision Searching

  • Mar 24, 2021
  • IEEE Journal of Solid-State Circuits
  • Donghyeon Han +6
  • Research Article
  • Citations52

TxSim: Modeling Training of Deep Neural Networks on Resistive Crossbar Systems

  • Mar 31, 2021
  • IEEE Transactions on Very Large Scale Integration (VLSI) Systems
  • Sourjya Roy +3
  • Conference Article
  • Citations2

Cannikin: Optimal Adaptive Distributed DNN Training over Heterogeneous Clusters

  • Dec 02, 2024
  • Chengyi Nie +2
  • Research Article
  • Citations36

Shallow and deep neural network training by water wave optimization

  • Aug 07, 2019
  • Swarm and Evolutionary Computation
  • Xiao-Han Zhou +5
  • Conference Article
  • Citations7

Comparison of Loss Functions for Training of Deep Neural Networks in Shogi

  • Nov 01, 2018
  • Hanhua Zhu +1
  • Research Article
  • Citations2

Accelerating deep network training for radar identification using batch normalization

  • Nov 25, 2024
  • Defence Science Journal
  • Preeti Gupta +2
  • Research Article
  • Citations1

STADIA: Photonic Stochastic Gradient Descent for Neural Network Accelerators

  • Sep 09, 2023
  • ACM Transactions on Embedded Computing Systems
  • Chengpeng Xia +3
  • Conference Article
  • Citations36

Gradient Compression Supercharged High-Performance Data Parallel DNN Training

  • Oct 26, 2021
  • Youhui Bai +7
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.