Hmesh: A vlsi architecture for parallel processing

Thanks to their massive computational power and their SIMT computational model, Graphics Processing Units (GPUs) have been successfully used to accelerate a wide variety of regular applications (linear algebra, stencil computations, image processing and bioinformatics algorithms, among others). However, many established and emerging problems are based on irregular data structures, such as graphs. Examples can be drawn from different application domains: networking, social networking, machine learning, electrical circuit modeling, discrete event simulation, compilers, and computational sciences. It has been shown that irregular applications based on large graphs do exhibit runtime parallelism; moreover, the amount of available parallelism tends to increase with the size of the datasets. In this work, we explore an implementation space for deploying a variety of graph algorithms on GPUs. We show that the dynamic nature of the parallelism that can be extracted from graph algorithms makes it impossible to find an optimal solution. We propose a runtime system able to dynamically transition between different implementations with minimal overhead, and investigate heuristic decisions applicable across algorithms and datasets. Our evaluation is performed on two graph algorithms: breadth-first search and single-source shortest paths. We believe that our proposed mechanisms can be extended and applied to other graph algorithms that exhibit similar computational patterns.

We use the computing technology digital image processing (DIP) to enhance the teaching of linear algebra so as to make the course more visual and interesting. Certainly, this visual approach by using technology to link linear algebra to DIP is interesting and unexpected to both students as well as many faculty.

Matrix multiplication is an essential part of many applications, such as linear algebra, image processing and machine learning. One platform used in such applications is TensorFlow, which is a machine learning library whose structure is based on dataflow programming paradigm. In this work, a method for multiplication of medium-density matrices on multicore CPUs using TensorFlow platform is proposed. This method, called tbt_matmul, utilizes TensorFlow built-in methods tf.matmul and tf.sparse_matmul. By partitioning each input matrix into four smaller sub-matrices, called tiles, and applying an appropriate multiplication method to each pair depending on their density, the proposed method outperforms the built-in methods for matrices of medium density and matrices of significantly uneven distribution of non-zeros.

Image modeling in terms of physical/mathematical formation and the human visual system plays an important role in developing new image processing techniques. Parameters such as scale, orientation, texture, morphology, and color must be taken into consideration in order to solve complicated image processing problems efficiently. Although the classical linear image processing (CLIP) approach has played a central role in image processing, it is not necessarily the best and even the right choice. General linear image processing (GLIP) approaches based on different image modeling techniques have been studied to overcome some of the problems associated with CLIP. Indeed, using general linear algebra, it is possible to develop entirely new general linear operators and transforms. It is even possible to define entirely new general linear operations (addition, scalar multiplication, convolution, etc.) in order to describe how images are combined, amplified, transformed, analyzed, compared, and measured.

Numerical linear algebra, digital signal processing, and parallel algorithms are three disciplines with a great deal of activity in the last few years. The interaction between them has been growing to a level that merits an Advanced Study Institute dedicated to the three areas together. This volume gives an account of the main results in this interdisciplinary field. The following topics emerged as major themes of the meeting: - Singular value and eigenvalue decompositions, including applications, - Toeplitz matrices, including special algorithms and architectures, - Recursive least squares in linear algebra, digital signal processing and control, - Updating and downdating techniques in linear algebra and signal processing, - Stability and sensitivity analysis of special recursive least squares problems, - Special architectures for linear algebra and signal processing. This book contains tutorials on these topics given by leading scientists in each of the three areas. A consider- able number of new research results are presented in contributed papers. The tutorials and papers will be of value to anyone interested in the three disciplines.

With the introduction of more powerful and massively parallel embedded processors, embedded systems are becoming HPC-capable. Heterogeneous on-chip systems (SoC) that couple a general-purposehost processor to a many-core accelerator are becoming more and more widespread, and provide tremendous peak performance/watt, well suited to execute HPC-class programs. The increased computation potential is however traded off for ease programming. Application developers are indeed required to manually deal with outlining code parts suitable for acceleration, parallelize them efficiently over many available cores, and orchestrate data transfers to/from the accelerator. In addition, since most many-cores are organized as a collection ofclusters, featuring fast local communication but slow remote communication (i.e., to another cluster's local memory), the programmer should also take care of properly mapping the parallel computation so as to avoid poor data locality. OpenMP v4.0 introduces new constructs for computation offloading, as well as directives to deploy parallel computation in a cluster-aware manner. In this paper we assess the effectiveness of OpenMP v4.0 at exploiting the massive parallelism available in embedded heterogeneous SoCs, comparing to standard parallel loops over several computation-intensive applications from the linear algebra and image processing domains.

This paper derives tradeoffs between three basic costs of a parallel algorithm: synchronization, data movement, and computational cost. These tradeoffs are lower bounds on the execution time of the algorithm which are independent of the number of processors, but dependent on the problem size. Therefore, they provide lower bounds on the parallel execution time of any algorithm computed by a system composed of any number of homogeneous components, each with associated computational, communication, and synchronization payloads. We employ a theoretical model counts the amount of work and data movement as a maximum of any execution path during the parallel computation. By considering this metric, rather than the total communication volume over the whole machine, we obtain new insights into the characteristics of parallel schedules for algorithms with non-trivial dependency structures. We also present reductions from BSP and LogP algorithms to our execution model, extending our lower bounds to these two models of parallel computation. We first develop our results for general dependency graphs and hypergraphs based on their expansion properties, then we apply the theorem to a number of specific algorithms in numerical linear algebra, namely triangular substitution, Gaussian elimination, and Krylov subspace methods. Our lower bound for LU factorization demonstrates the optimality of Tiskin's LU algorithm answering an open question posed in his paper, as well as of the 2.5D LU algorithm which has analogous costs. We treat the computations in a general manner by noting that the computations share a similar dependency hypergraph structure and analyzing the communication requirements of lattice hypergraph structures.

The Society for Industrial and Applied Mathematics (SIAM) and the SIAM Activity Group in Linear Algebra in conjunction with the International Linear Algebra Society (ILAS) held the SIAM Conference on Linear Algebra and its applications on October 23-26, 2000 at the McKimmon Conference Center on the campus of North Carolina State University in Raleigh, North Carolina. The goals of this conference were to highlight the central role of linear algebra in many problems of mathematics and the applied sciences, including engineering problems in systems and control, signal processing and coding, economic and business problems, and problems from biology and geophysics. Particular consideration in this conference was given to applications in image processing, information retrieval and management (such as the performance of search engines on the Internet), aircraft manufacturing and design, industrial optimization problems, and assessing the economic cost of linear algebra in industry. With the development of high performance computers and new parallel architectures, computational linear algebra is in a state of rapid development. There are grand challenges requiring the development of efficient methods that solve truly large-scale problems by exploiting the ever-increasing computational power. One of the primary goals of this conference was to bring researchers and practitioners in these various areas together for exchange of information and ideas. In particular, the collaboration with ILAS was an important factor in bringing about fruitful interaction among researchers in theory, computation, and applications. There were 250 total attendees with 17% coming from industry and government. In addition, there were 27 students who attended.

Chapter 7 - Complex Numbers

Investigation of flow structures and transport phenomena in bubble columns using particle image velocimetry and miniature pressure sensors

Speckle is caused by the illumination of an optically rough surface by coherent light [1] which can cause problems in digital holography. Speckle in a reconstruction reduces visibility of details and may impede important measurements of an object. Speckle can be reduced by capturing multiple holograms and summing the reconstructed intensities [2-4]. Digital signal processing methods, which are applied after capture, have also been proposed [5–8].

This book focuses the solutions of linear algebra and matrix analysis problems, with the exclusive use of MATLAB. The topics include representations, fundamental analysis, transformations of matrices, matrix equation solutions as well as matrix functions. Attempts on matrix and linear algebra applications are also explored.

Matrix computations lie at the heart of many scientific computations. While sophisticated algorithms have been established for various numerical linear algebra problems such as the solution of linear simultaneous equations and eigenvalue problems, they require considerable modification with the advent of exaFLOPS- scale supercomputers, which are expected to have a huge number of computing cores, deep memory hierarchy, and increased probability of hardware errors. In this chapter, we discuss projected hardware characteristics of exaFLOPS machines and summarize the challenges to be faced by numerical linear algebra algorithms in the near future. Based on these preparations, we present a brief survey of recent research efforts in the field of numerical linear algebra targeted at meeting these challenges.

Brent’s scheduling principle provides a general simulation scheme when fewer processors are available than specified by the fastest parallel algorithm. Such a scheme preserves, under slow-down, the actual number of executed operations, also called work. In this paper we take the complementary viewpoint, and rather than consider the work-preserving slow-down of some fast parallel algorithm, we investigate the problem of the achievable speedups of computation while preserving the work of the best-known sequential algorithm for the same problem. The proposed technique, eminently applicable to problems of matrix-computational flavor, achieves its result through the interplay of two algorithms with significantly different features. Analogous but structurally different “interplays” have been used previously to improve the algorithmic efficiency of graph computations, selection, and list ranking. We demonstrate the efficacy of our technique for the computation of path aigebras in graphs and digraphs and various fundamental computations in linear algebra. Some of the fundamental new algorithms may have practical value; for instance, we substantially improve the algorithmic performance of the parallel solution of triangular and Toeplitz linear systems of equations and the computation of the transitive closure of digraphs.

A fundamental mathematical field with many applications in science and engineering is linear algebra. This paper investigates the various applications of linear algebra in the fields of traffic flow analysis, electric circuits, cubic spline interpolation, and genetics. This research delves into individual applications while emphasizing cross-disciplinary insights, fostering innovative solutions through the convergence of genetics, cubic spline interpolation, circuits, and traffic flow analysis. The research employs specific methodologies in each application area to demonstrate the practical integration of linear algebra in genetics, cubic spline interpolation, electric circuits, and traffic flow analysis. In genetics, linear algebra techniques are utilized to represent genetic data using matrices, analyze genotype distributions across generations, and identify genotype-phenotype associations. For cubic spline interpolation, linear algebra is employed to construct smooth interpolating curves, involving the derivation of equations for spline functions and the determination of coefficients using boundary conditions and continuity requirements. In electric circuit analysis, linear algebra is crucial for modeling circuit elements, formulating systems of linear equations based on Kirchhoff's laws, and solving for voltage and current distributions in circuits. In traffic flow analysis, linear algebra techniques are used to represent traffic movement in networks, formulate systems of linear equations representing traffic flow dynamics, and solve for traffic flow solutions to optimize transportation networks. By addressing contemporary challenges, emerging research frontiers, and future trajectories at the intersection of linear algebra and diverse domains, this study underscores the profound impact of mathematical tools in advancing understanding and resolving complex real-world problems across multiple fields.