• Home
  • Search
  • Review of Mixed-Precision Optimization Methods for Tensor Computations on GPUs
  • https://doi.org/10.70315/uloap.ulete.2025.0204012Copy DOI Icon

Review of Mixed-Precision Optimization Methods for Tensor Computations on GPUs

Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

The article presents a comprehensive analysis of existing methods for optimizing tensor computations on graphics processing units (GPU) when using mixed-precision modes. The relevance of the study is determined by the computational complexity of modern neural network architectures and the need to ensure higher energy efficiency of computations while maintaining the required level of numerical accuracy. The scientific novelty of the study lies in the consistent systematization of approaches to the use of FP8 and BF16 formats on NVIDIA Ampere and Hopper architectures, as well as in the formulation of an adaptive strategy for selecting data representation precision depending on the nature of the computational workload. Within the framework of the study, architectural and hardware features of tensor cores are considered, and algorithms for dynamic loss scaling and stochastic rounding are analyzed, which determine the behavior of numerical errors under reduced precision. Particular emphasis in the work is placed on ensuring numerical stability during quantization of transformer models, where the combination of deep architecture and long chains of matrix operations makes the system sensitive to error accumulation. The aim of the study is to identify preferable combinations of data representation formats for various classes of tensor operations. To achieve this aim, methods of comparative analysis of the existing literature and theoretical modeling are used. The final part of the article presents a hybrid-precision scheme oriented toward practical application in high-performance computing systems and intended for specialists engaged in the development and study of deep learning methods.

Similar Papers
  • PDF
  • Research Article
  • Citations21

Novel Parallel Algorithms for Fast Multi-GPU-Based Generation of Massive Scale-Free Networks

  • Mar 01, 2019
  • Data Science and Engineering
  • Maksudul Alam +2
  • Conference Article
  • Citations48

Breaking the GPU programming barrier with the auto-parallelising SAC compiler

  • Jan 23, 2011
  • Jing Guo +2
  • PDF
  • Research Article
  • Citations14

Accelerating SAR Image Registration Using Swarm-Intelligent GPU Parallelization

  • Jan 01, 2020
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Yingbing Liu +5
  • Research Article
  • Citations183

GPU-based fast Monte Carlo simulation for radiotherapy dose calculation

  • Oct 21, 2011
  • Physics in Medicine and Biology
  • Xun Jia +4
  • PDF
  • Research Article
  • Citations29

WFA-GPU: gap-affine pairwise read-alignment using GPUs.

  • Nov 17, 2023
  • Bioinformatics
  • Quim Aguado-Puig +6
  • Research Article
  • Citations18

Dynamic voltage and frequency scaling framework for low-power embedded GPUs

  • Oct 11, 2012
  • Electronics Letters
  • Daecheol You +1
  • Book Chapter
  • Citations49

CAMPARY: Cuda Multiple Precision Arithmetic Library and Applications

  • Jan 01, 2016
  • Mioara Joldes +3
  • Research Article
  • Citations11

Enhancing Design Space Exploration by Extending CPU/GPU Specifications onto FPGAs

  • Feb 17, 2015
  • ACM Transactions on Embedded Computing Systems
  • Muhsen Owaida +9
  • Research Article
  • Citations1

Прийняття рішень в системі стратегічного управління бізнес-процесами як інструмент підви-щення ефективності і конкурентоспроможності підприємств в умовах сталого розвитку

  • Nov 19, 2025
  • Ukrainian Journal of Applied Economics and Technology
  • Olena Arefieva +1
  • Research Article
  • Citations3

Genetic algorithm based estimation of non–functional properties for GPGPU programs

  • Dec 07, 2019
  • Journal of Systems Architecture
  • Adrian Horga +3
  • Research Article
  • Citations35

Scalability of Self-organizing Maps on a GPU cluster using OpenCL and CUDA

  • Feb 09, 2012
  • Journal of Physics: Conference Series
  • Sabine Mcconnell +4
  • Conference Article
  • Citations22

Graphics Peeping Unit: Exploiting EM Side-Channel Information of GPUs to Eavesdrop on Your Neighbors

  • May 01, 2022
  • Zihao Zhan +4
  • Conference Article
  • Citations12

A Data-Driven Frequency Scaling Approach for Deadline-aware Energy Efficient Scheduling on Graphics Processing Units (GPUs)

  • May 01, 2020
  • Shashikant Ilager +3
  • Conference Article
  • Citations9

FCM

  • May 18, 2016
  • Jiachen Song +5
  • Research Article
  • Citations117

Understanding GPU Power

  • Sep 16, 2016
  • ACM Computing Surveys
  • Robert A Bridges +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.