• Cite Icon3
  • https://doi.org/10.1145/1356058.1356087Copy DOI Icon

Code optimization of parallel programs

  • Apr 6, 2008
  • Vivek Sarkar
Show More
  • Abstract
  • Literature Map
  • Citations
  • Similar Papers
Abstract

Code optimization has a rich history that dates back over half a century. Over the years, it has contributed deep innovations to address challenges posed by new computer system and programming language features. Examples of the former include optimizations for improved register utilization, instruction-level parallelism, vector parallelism, multiprocessor parallelism and memory hierarchy utilization. Examples of the latter include optimizations for procedural, object-oriented, functional and domain-specific languages as well as dynamic optimization for managed runtimes. These optimizations have contributed significantly to programmer productivity by reducing the effort that programmers need to spend on hand-implementing code optimizations and by enabling code to be more portable, especially as programming models and computer architectures change. While compiler frameworks are often able to incorporate new code optimizations in an evolutionary manner, there have been notable periods in the history of compilers when more revolutionary changes were necessary. Examples of such paradigm shifts in the history of compilers include interprocedural whole program analysis, coloring-based register allocation, static single assignment form, array dependence analysis, pointer alias analysis, loop transformations, adaptive profile-directed optimizations, and dynamic compilation. The revolutionary nature of these shifts is evidenced by the fact that production-strength optimization frameworks (especially those in industry) had to be rewritten from scratch or significantly modified to support the new capabilities. In this talk, we claim that the current multicore trend in the computer industry is forcing a new paradigm shift in compilers to address the challenge of code optimization of parallel programs, regardless of whether the parallelism is implicit or explicit in the programming model. All computers --- embedded, mainstream, and high-end --- are now being built from multicore processors with little or no increase in clock speed per core. This trend poses multiple challenges for compilers for future systems as the number of cores per socket continues to grow, and the cores become more heterogeneous. In addition, compilers have to keep pace with emerging parallel programming models embodied in a proliferation of new libraries and new languages.To substantiate our claim, we examine the historical foundations of code optimization including intermediate representations (IR's), abstract execution models, legality and cost analyses of IR transformations and show that they are all deeply entrenched in the von Neumann model of sequential computing. We discuss ongoing evolutionary efforts to support optimization of parallel programs in the context of existing compiler frameworks, and their inherent limitations for the long term. We then outline what a revolutionary approach will entail, and identify where its underlying paradigm shifts are likely to lie. We provide examples of past research that are likely to influence future directions in code optimization of parallel programs such as program dependence graphs, partitioning and scheduling of lightweight parallelism, synchronization optimizations, communication optimizations, transactional memory optimizations, code generation for heterogeneous accelerators, impact of memory models on code optimization, and general forms of data and computation alignment. Finally, we briefly describe the approach to code optimization of parallel programs being taken in the Habanero Multicore Software Research project at Rice University.

Similar Papers
  • Book Chapter
  • Citations7

Challenges in Code Optimization of Parallel Programs

  • Jan 01, 2009
  • Vivek Sarkar
  • Research Article

Research of Parallel Program Performance Analysis and Optimization on Multicore Platform

  • Aug 16, 2013
  • Yingyan Teng +1
  • Conference Article
  • Citations9

Towards Constructing the SSA form using Reaching Definitions Over Dominance Frontiers

  • Sep 01, 2019
  • Abu Naser Masud +1
  • Research Article
  • Citations10

More precise construction of static single assignment programs using reaching definitions

  • Apr 06, 2020
  • Journal of Systems and Software
  • Abu Naser Masud +1
  • Conference Article
  • Citations8

A type system equivalent to static single assignment

  • Jul 10, 2006
  • Yutaka Matsuno +1
  • Research Article
  • Citations8

Using static single assignment form in a code optimizer

  • Jun 01, 1992
  • ACM Letters on Programming Languages and Systems
  • Carl Mcconnell +1
  • Research Article
  • Citations3

PPModel: a modeling tool for source code maintenance and optimization of parallel programs

  • Sep 12, 2012
  • The Journal of Supercomputing
  • Ferosh Jacob +4
  • Conference Article
  • Citations87

Sparse representation of implicit flows with applications to side-channel detection

  • Mar 17, 2016
  • Bruno Rodrigues +2
  • Book Chapter
  • Citations4

Using the SSA-Form in a Code Generator

  • Jan 01, 2014
  • Benoît Dupont De Dinechin
  • Research Article
  • Citations132

Practical improvements to the construction and destruction of static single assignment form

  • Jul 10, 1998
  • Software: Practice and Experience
  • Preston Briggs +3
  • Book Chapter
  • Citations18

Incremental computation of static single assignment form

  • Jan 01, 1996
  • Jong -Deok Choi +2
  • Conference Article
  • Citations92

Dependence flow graphs: an algebraic approach to program dependencies

  • Jan 01, 1991
  • Keshav Pingali +4
  • Research Article
  • Citations70

From control flow to dataflow

  • Jun 01, 1991
  • Journal of Parallel and Distributed Computing
  • Micah Beck +2
  • Conference Article
  • Citations94

May-happen-in-parallel analysis of X10 programs

  • Mar 14, 2007
  • Shivali Agarwal +3
  • Research Article
  • Citations3

Structural Encoding of Static Single Assignment Form

  • Nov 25, 2005
  • Electronic Notes in Theoretical Computer Science
  • Andreas Gal +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.