• Home
  • Search
  • Trade-Off Between Memory Usage and Optimization Speed in a Simple Machine Learning Task
  • https://doi.org/10.1109/dcna67486.2025.11199920Copy DOI Icon

Trade-Off Between Memory Usage and Optimization Speed in a Simple Machine Learning Task

  • Sep 7, 2025
  • Oleg Goryunov +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

In modern machine learning, the efficiency of training is heavily influenced by the choice of optimization algorithm, with preconditioned methods offering significant speed improvements at the cost of increased memory usage. This study explores the trade-off between memory consumption and optimization performance by generalizing the Shampoo preconditioning algorithm to support arbitrarily sized preconditioning matrices, rather than being limited by tensor dimensions. We evaluate this approach on a perceptron trained for a simple regression task, measuring how different preconditioner sizes affect the training speed. Our results reveal a strong positive correlation: larger preconditioners consistently lead to faster learning, although the performance gains scale sublinearly with memory. Our findings provide practical insights into balancing computational resources and training efficiency in adaptive optimization.

Similar Papers
  • Research Article
  • Citations9

The initial effect of noise on a simple vibrotactile learning task

  • Dec 01, 1967
  • Perception & Psychophysics
  • D D Diespecker +1
  • Research Article
  • Citations16

The impact of touchscreen interactivity on U.S. toddlers’ selective attention and learning from digital media

  • Jul 14, 2021
  • Journal of Children and Media
  • Heather L Kirkorian +3
  • Research Article
  • Citations25

Effectiveness and Adoption of a Drawing-to-Learn Study Tool for Recall and Problem Solving: Minute Sketches with Folded Lists.

  • Jun 01, 2017
  • CBE—Life Sciences Education
  • Paul D Heideman +3
  • Research Article
  • Citations7

Individual differences in learning ability are negatively linked to behavioural plasticity in a frequency-dependent game

  • Dec 09, 2019
  • Animal Behaviour
  • Marie Barou Dagues +2
  • PDF
  • Research Article
  • Citations2

Appetitively motivated tasks in the IntelliCage reveal a higher motivational cost of spatial learning in male than female mice

  • Feb 29, 2024
  • Frontiers in Behavioral Neuroscience
  • Martina Nigri +3
  • Research Article

The impact of simulated cataract on face learning

  • Dec 02, 2025
  • Scientific Reports
  • Abuk Akech +1
  • Research Article

How the worm changes its tastes

  • Jul 17, 2014
  • Science
  • Peter Stern
  • Research Article
  • Citations191

Effects of ICT: Do we know what we should know?

  • May 11, 2007
  • Education and Information Technologies
  • Margaret J Cox +1
  • Research Article
  • Citations12

Ambiguity produces attention shifts in category learning.

  • Mar 15, 2016
  • Learning & Memory
  • Miguel A Vadillo +3
  • Abstract

P.6.a.001 The difference of decision-making pattern under ambiguity and risk between alcohol dependence and schizophrenia

  • Oct 01, 2007
  • European Neuropsychopharmacology
  • Y.T Kim +6
  • Research Article
  • Citations6

Rule fitness and pathology in learning classifier systems.

  • Mar 01, 2004
  • Evolutionary computation
  • Tim Kovacs
  • Research Article

Feedback Specificity Requirements for Learning in Younger and Older Adults: The Role of Cognitive Resources and Task Demand

  • Oct 01, 2009
  • Proceedings of the Human Factors and Ergonomics Society Annual Meeting
  • Christopher M Kelley +1
  • Research Article
  • Citations40

Population coupling predicts the plasticity of stimulus responses in cortical circuits.

  • Apr 21, 2020
  • eLife
  • Yann Sweeney +1
  • Research Article
  • Citations90

Pre- and post-training lesions of the intermediate medial hyperstriatum ventrale and passive avoidance learning in the chick

  • Jan 01, 1990
  • Experimental Brain Research
  • T.A Patterson +2
  • Research Article
  • Citations3

Evolutionary learning in reaction-diffusion neurons

  • Feb 01, 1991
  • Applied Mathematics and Computation
  • Kevin G Kirby
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.