• Cite Icon65
  • https://doi.org/10.1145/2925426.2926256Copy DOI Icon

BLASX

  • Jun 1, 2016
  • Linnan Wang +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Basic Linear Algebra Subprograms (BLAS) are a set of low level linear algebra kernels widely adopted by applications involved with the deep learning and scientific computing. The massive and economic computing power brought forth by the emerging GPU architectures drives interest in implementation of compute-intensive level 3 BLAS on multi-GPU systems. In this paper, we investigate existing multi-GPU level 3 BLAS and present that 1) issues, such as the improper load balancing, inefficient communication, insufficient GPU stream level concurrency and data caching, impede current implementations from fully harnessing heterogeneous computing resources; 2) and the inter-GPU Peer-to-Peer(P2P) communication remains unexplored. We then present BLASX: a highly optimized multi-GPU level-3 BLAS. We adopt the concepts of algorithms-by-tiles treating a matrix tile as the basic data unit and operations on tiles as the basic task. Tasks are guided with a dynamic asynchronous runtime, which is cache and locality aware. The communication cost under BLASX becomes trivial as it perfectly overlaps communication and computation across multiple streams during asynchronous task progression. It also takes the current tile cache scheme one step further by proposing an innovative 2-level hierarchical tile cache, taking advantage of inter-GPU P2P communication. As a result, linear speedup is observable with BLASX under multi-GPU configurations; and the extensive benchmarks demonstrate that BLASX consistently outperforms the related leading industrial and academic implementations such as cuBLAS-XT, SuperMatrix, MAGMA.

Similar Papers
  • Conference Article

CIM-BLAS: Computing-in-Memory Accelerator for BLAS

  • Jun 22, 2025
  • Rui Liu +5
  • Book Chapter
  • Citations1

Optimization of Triangular Matrix Functions in BLAS Library on Loongson2F

  • Jan 01, 2010
  • Yun Xu +2
  • Research Article
  • Citations210

GEMM-based level 3 BLAS

  • Sep 01, 1998
  • ACM Transactions on Mathematical Software
  • Bo Kågström +2
  • Conference Article

A Flexible-blocking Based Approach for Performance Tuning of Matrix Multiplication Routines for Large Matrices with Edge Cases

  • Dec 01, 2018
  • Md Mosharaf Hossain +5
  • Research Article

Algorithm 1005

  • Mar 20, 2020
  • ACM Transactions on Mathematical Software
  • Kristjan Jonasson +4
  • Research Article
  • Citations1

Parallelized Simulation of a Finite Element Method in Many Integrated Core Architecture

  • Feb 07, 2017
  • Journal of Engineering Materials and Technology
  • Moonho Tak +1
  • PDF
  • Research Article
  • Citations1

Accelerating the computation for real-time application of the sinc function using graphics processing units

  • Feb 15, 2020
  • Journal of Analytical Science and Technology
  • Sangwoo Kim +1
  • Conference Article
  • Citations90

High Performance Linear Algebra Operations on Reconfigurable Systems

  • Nov 12, 2005
  • Ling Zhuo +1
  • Research Article
  • Citations6

Fast Parallel Algorithm for Polynomial Evaluation

  • Dec 01, 2003
  • Parallel Algorithms and Applications
  • Przemysław Stpiczyński
  • Conference Article
  • Citations4

Visualization imaging software design for Electrical Tomography

  • May 01, 2011
  • Chengyi Yang +2
  • Research Article
  • Citations7

VOLSCAT2.0: The new version of the package for electron and positron scattering off molecular targets

  • Sep 24, 2018
  • Computer Physics Communications
  • N Sanna +3
  • Research Article

Timing results for BLAS (Basic Linear Algebra Subprograms) libraries in R

  • Jun 15, 2016
  • Zenodo (CERN European Organization for Nuclear Research)
  • Andrzej Wójtowicz
  • Research Article
  • Citations192

CheMPS2: A free open-source spin-adapted implementation of the density matrix renormalization group for ab initio quantum chemistry

  • Feb 25, 2014
  • Computer Physics Communications
  • Sebastian Wouters +3
  • Conference Article

Accelerated MR physics simulations on multi-GPU systems

  • Nov 01, 2013
  • Christos G Xanthis +2
  • Conference Article
  • Citations2

Evaluation of Open-Source Linear Algebra Libraries in Embedded Applications

  • Jun 01, 2019
  • Christian Fibich +7
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.