Compiling functional languages for SIMD architectures

FFT algorithms for SIMD parallel processing systems

Parallel processing with minicomputers

The parallel implementation aspects of the Viterbi algorithm, which gives the maximum likelihood solution to decoding conventional codes, are studied. The interconnection of the trellis diagram of a binary convolutional code is similar to the shuffle exchange. This similarity is exploited to present a parallel version of the Viterbi algorithm. The proposed algorithm avoids tasks such as appending and exchanging the survivor lists that are considered pragmatic issues to be accommodated in implementing the Viterbi algorithm. This is done by introducing an additional back-tracking procedure which is almost identical to the forward path where branch and path metrics are computed. The parallel Viterbi algorithm is implemented on the connection machine which is a SIMD (single instruction stream and multiple data stream) parallel machine. >

The Multiple Associative Computing (MASC) parallelmodel is a generalization model of an AssociativeComputing (ASC) parallel model designed to supportmultiple ASC data parallel threads by using controlparallelism.

The authors present a software tool for mapping neural network computations onto mesh-connected SIMD (single instruction stream multiple data stream) computers. The tool, called NetMap, contains complete mapping programs for built-in network models and a library of elemental routing routines. For built-in models the user is required to provide only minimum information about the network. For example, for a backpropagation network with fully connected layers, it is enough to provide the number of neurons in each layer of the network. If the network has been pruned or structured in some other way, so that the layers are no longer fully interconnected, a complete connectivity matrix needs to be provided. Thus, NetMap can be used for networks of arbitrary topology. >

We have been investigating the efficiency of genetic algorithms (GA) for solving for a variety of real problems. During our investigations we have concluded that the large amount of computational time required to find GA based solutions on conventional computers is restrictive. We are therefore developing an innovative new computer architecture, suitable for the solution of large scale problems using GAs. In this paper we introduce the SIMD-GA (Single Instruction stream Multiple Data stream Genetic Algorithm), and discuss its hardware design and implementation. By taking advantage of the recent advances is HDLs (Hardware Description Language) and FPGAs (Field Programmable Gate Array) we have been able to quickly develop and prototype a PE (Processing Element) for a SIMD-GA. This approach allows us to build a cost-effective parallel processing architecture to overcome the problem of the computational time required for traditional sequential GA implementation.

Due to the high-sped development in semiconductor technology, general-purpose computers can process digital video, audio and graphics easily. Nowadays, the similar functionality has shifted from the general-purpose computer to the so-called media-processor one. Since the processor is targeted on mass-produced consumer electronic devices, the price and the ease of use are the main concern. The latest MPEG-4 video coding standard, which consumes relatively low bit-rate but yields acceptable quality, meets the target exactly. In this paper, by realizing an MPEG-4 codec using both general-purpose processor and media-processor, we investigated some issues related to the use of a media-processor for satisfying various multimedia compression and processing requirements. The investigated issues include the speedup of the MPEG-4 decoder based on a media-processor DSP chip, a multimedia co-processor or hardware accelerator, the very long instruction word (VLIW) and the single instruction multiple data stream (SIMD) programming techniques. Moreover, a media-processor based MPEG-4 solution for set-top box application is also included.

Sparse matrix-vector multiplication (SpMV) is a challenging computational kernel in linear algebra applications, like data mining, image processing, and machine learning. The performance of this kernel is greatly dependent on the size of the input matrix and the underlying hardware features. Various sparse matrix storage formats referred to commonly as sparse formats have been proposed in the literature to reduce the size of the matrix. In modern multi-core and many-core architectures, the performance of the kernel is mainly dependent on memory wall and power wall problem. Normally review on sparse formats is done with specific architecture or with specific application. This chapter presents a comparative study on various sparse formats in cross platform architecture like CPU, graphics processor unit (GPU), and single instruction multiple data stream (SIMD) registers. Space complexity analysis of various formats with its representation is discussed. Finally, the merits and demerits of each format have been summarized into a table.

A prototype OCR is constructed using a very compact parallel processing unit. This unit, designed for interactive character recognition applications, is equipped with a hand-scanner for input and a personal computer for word and/or image processing. The heart of this unit is a bit-serial Single Instruction Multiple Data stream (SIMD) array processor constructed with four identical cellular array LSIs (AAP2). The processor is fully programmable and the complex pro-cess of Japanese character recognition can be carried out with a single program package. Its architecture permits flexible and high-speed SIMD operations to process bitline data such as local fields of scanned documents. The processor components were integrated into one board and confirmed to be more than ten times faster than present image processors of the same size through various image processing tests. High character recognition performance is obtained at a reading speed of 8 Japanese characters per second which is sufficient for hand-scanning data input operations. The recognition rate is higher than 98% for about 3,300 Japanese characters.

In cellular communications, recently Multi In Multi Out (MIMO) and Massive MIMO research is getting attention for the need of high data rates in Long Term Evolution Advanced(LTE-A) and 5G Communications. In MIMO baseband signal processing at physical layer, both the channel estimation and the detection algorithms play a crucial role. In this paper it is discussed the estimation algorithms least square (LS) and minimum mean square error (MMSE) and the channel detection algorithms Zero Forcing (ZF) and MMSE. Currently none of the channel estimation algorithms of LTE-A offers twin advantages of low battery consumption and very low latency, which is a key requirement of 5G. It is expected the massive MIMO with 128 or more antennas will be a norm at 5G base stations. To achieve the ultra-low latency, the matrix computations for massive MIMO are the very big bottleneck in realizing the channel estimation and massive MIMO detection algorithms. For the optimization of the 5G Massive MIMO channel estimation and detection algorithms, the prerequisite is massive complex matrix inversion speed. In this paper, a parallel processing based coding scheme is proposed by using Gauss-Jordan elimination kernel algorithm on a single instruction multiple data (SIMD) stream vector processor to realize a complex matrix inverse with optimum speed which is the need of 5G channel estimation and detection.

DNA computing has a rich history of computing paradigms with great expressive power. However, far less expressive power is needed for data manipulation. Indeed, the relational algebra, the yardstick of database systems, is expressible in first-order logic, and thus less powerful than Turing-complete models. Turing-complete DNA computing models have to account for many and varied scenarios. A DNA implementation of data manipulations might be nimbler and perform its operation faster than a Turing-complete DNA computing model. Hence, we propose a restrictive model for implementing data manipulation operations, focused on implementability in DNA. We call this model the sticker complex model. A forte of the sticker complex model, is its ability to detect when hybridization becomes an uncontrolled chain reaction. Such chain reactions make hybridization less predictable and thus less attractive for deterministic computations. Next, we define a query language on sticker complexes, called DNAQL. DNAQL is a typed, applicative functional programming language, powerful enough to simulate the relational algebra on sticker complexes. The type system enjoys a number of desirable properties such as soundness, maximality, and tightness.

Direct particle simulation on the Connection Machine CM-2

We consider an extension of modal logic with an operator for constructing inflationary fixed points, just as the modal µ-calculus extends basic modal logic with an operator for least fixed points. Least and inflationary fixed point operators have been studied and compared in other contexts, particularly in finite model theory, where it is known that the logics IFP and LFP that result from adding such fixed point operators to first order logic have equal expressive power. As we show, the situation in modal logic is quite different, as the modal iteration calculus (MIC) we introduce has much greater expressive power than the µ-calculus. Greater expressive power comes at a cost: the calculus is algorithmically much less manageable.

This paper explores the theory and implementation of ray tracing in OCaml. We discuss the mathematical concepts and algorithms that drive the rendering process, followed by implementation using OCaml’s functional programming paradigm. Our analysis compares the merits and challenges of functional versus object-oriented (OO) approaches, highlighting the advantages of code clarity, static type safety, and ease of debugging in a functional style. With the development of functional programming module, it has been established that the debugging cost is higher in object oriented module. In this context the functional module can identify the errors at the time of compilation and we establish this fact with the development of code using OCaml. We also address performance considerations related to recursive function calls and state management complexities. Ultimately, we advocate for implementing a basic ray tracer in a functional style, demonstrating that this approach enhances code quality while aligning with the principles of immutability and strong typing.

This paper shows how to compile sparse array programming languages. A sparse array programming language is an array programming language that supports element-wise application, reduction, and broadcasting of arbitrary functions over dense and sparse arrays with any fill value. Such a language has great expressive power and can express sparse and dense linear and tensor algebra, functions over images, exclusion and inclusion filters, and even graph algorithms. Our compiler strategy generalizes prior work in the literature on sparse tensor algebra compilation to support any function applied to sparse arrays, instead of only addition and multiplication. To achieve this, we generalize the notion of sparse iteration spaces beyond intersections and unions. These iteration spaces are automatically derived by considering how algebraic properties annotated onto functions interact with the fill values of the arrays. We then show how to compile these iteration spaces to efficient code. When compared with two widely-used Python sparse array packages, our evaluation shows that we generate built-in sparse array library features with a performance of 1.4× to 53.7× when measured against PyData/Sparse for user-defined functions and between 0.98× and 5.53× when measured against SciPy/Sparse for sparse array slicing. Our technique outperforms PyData/Sparse by 6.58× to 70.3×, and (where applicable) performs between 0.96× and 28.9× that of a dense NumPy implementation, on end-to-end sparse array applications. We also implement graph linear algebra kernels in our system with a performance of between 0.56× and 3.50× compared to that of the hand-optimized SuiteSparse:GraphBLAS library.