FFT algorithms for SIMD parallel processing systems

The parallel implementation aspects of the Viterbi algorithm, which gives the maximum likelihood solution to decoding conventional codes, are studied. The interconnection of the trellis diagram of a binary convolutional code is similar to the shuffle exchange. This similarity is exploited to present a parallel version of the Viterbi algorithm. The proposed algorithm avoids tasks such as appending and exchanging the survivor lists that are considered pragmatic issues to be accommodated in implementing the Viterbi algorithm. This is done by introducing an additional back-tracking procedure which is almost identical to the forward path where branch and path metrics are computed. The parallel Viterbi algorithm is implemented on the connection machine which is a SIMD (single instruction stream and multiple data stream) parallel machine. >

Parallel processing with minicomputers

Spatial and temporal data such as multiple data streams often have concept drift problems, which refers to changes of the data distributions over time. Once concept drift occurs, a stationary machine learning predictor will probably be invalid because the testing data have a different data distribution from that of the training data. Recent studies in data streams aim to address this issue by concept drift adaptation techniques. Concept drift adaptation methods update the predictor by time and have been validated to provide accurate real-time prediction for a single data stream. However, handling multiple relevant data streams is still an unsolved challenge in this area, considering many real-world applications generate multiple data streams that are simultaneously evolving. To fill this gap, we here present a predicting network for multiple data streams, named MuNet. MuNet leverages the dependency between data streams to efficiently lower down the computational cost of real-time predictions for multiple data streams. In MuNet, an online-learned Bayesian neural network (BNN) is designed as a connector between streams. The BNN-connector can continuously use the real-time information from only one base stream to correct the stationary predictor for the other streams, which avoids the high cost caused by repeated adaptation.

On Factors Limiting the Generation of Efficient Compiler-Parallelized Programs

The Multiple Associative Computing (MASC) parallelmodel is a generalization model of an AssociativeComputing (ASC) parallel model designed to supportmultiple ASC data parallel threads by using controlparallelism.

Horizon is a scalable shared-memory Multiple Instruction stream - Multiple Data stream (MIMD) computer architecture independently under study at the Supercomputing Research Center (SRC) and Tera Computer Company. It is composed of a few hundred identical scalar processors and a comparable number of memories, sparsely embedded in a three-dimensional nearest-neighbor network. Each processor has a horizontal instruction set that can issue up to three floating point operations per cycle without resorting to vector operations. Processors will each be capable of performing several hundred Million Floating Point Operations Per Second (FLOPS) in order to achieve an overall system performance target of 100 Billion (1011) FLOPS.This paper describes the architecture of the processor in the Horizon system. In the fashion of the Denelcor HEP, the processor maintains a variable number of Single Instruction stream - Single Data stream (SISD) processes, which are called instruction streams. Memory latency introduced by the large shared memory is hidden by switching context (instruction stream) each machine cycle. The processor functional units are pipelined to achieve high computational throughput rates; however, pipeline dependencies are hidden from user code. Hardware mechanisms manage the resources to guarantee anonymity and independence of instruction streams.

The authors present a software tool for mapping neural network computations onto mesh-connected SIMD (single instruction stream multiple data stream) computers. The tool, called NetMap, contains complete mapping programs for built-in network models and a library of elemental routing routines. For built-in models the user is required to provide only minimum information about the network. For example, for a backpropagation network with fully connected layers, it is enough to provide the number of neurons in each layer of the network. If the network has been pruned or structured in some other way, so that the layers are no longer fully interconnected, a complete connectivity matrix needs to be provided. Thus, NetMap can be used for networks of arbitrary topology. >

A major impediment to the wider proliferation of Single-Instruction, Multiple Datastream (SIMD) (3) architectures rests in the unsuitability of sequential, scalar, languages for programming data-parallel systems. Existing languages lack sufficient expressive power to describe data-parallel computation. A functional language, extended with data-parallel primitives, provides a powerful abstraction of the capabilities of SIMD architectures. Such a language conveys the following benefits: greater expressive power, a rich set of data-types, transparent access to data-parallelism, amenability to program transformation and consistency with the functional style. A compilation strategy onto an abstract SIMD architecture is presented. >

9 - Parallel Processors

We have been investigating the efficiency of genetic algorithms (GA) for solving for a variety of real problems. During our investigations we have concluded that the large amount of computational time required to find GA based solutions on conventional computers is restrictive. We are therefore developing an innovative new computer architecture, suitable for the solution of large scale problems using GAs. In this paper we introduce the SIMD-GA (Single Instruction stream Multiple Data stream Genetic Algorithm), and discuss its hardware design and implementation. By taking advantage of the recent advances is HDLs (Hardware Description Language) and FPGAs (Field Programmable Gate Array) we have been able to quickly develop and prototype a PE (Processing Element) for a SIMD-GA. This approach allows us to build a cost-effective parallel processing architecture to overcome the problem of the computational time required for traditional sequential GA implementation.

Due to the high-sped development in semiconductor technology, general-purpose computers can process digital video, audio and graphics easily. Nowadays, the similar functionality has shifted from the general-purpose computer to the so-called media-processor one. Since the processor is targeted on mass-produced consumer electronic devices, the price and the ease of use are the main concern. The latest MPEG-4 video coding standard, which consumes relatively low bit-rate but yields acceptable quality, meets the target exactly. In this paper, by realizing an MPEG-4 codec using both general-purpose processor and media-processor, we investigated some issues related to the use of a media-processor for satisfying various multimedia compression and processing requirements. The investigated issues include the speedup of the MPEG-4 decoder based on a media-processor DSP chip, a multimedia co-processor or hardware accelerator, the very long instruction word (VLIW) and the single instruction multiple data stream (SIMD) programming techniques. Moreover, a media-processor based MPEG-4 solution for set-top box application is also included.

Sparse matrix-vector multiplication (SpMV) is a challenging computational kernel in linear algebra applications, like data mining, image processing, and machine learning. The performance of this kernel is greatly dependent on the size of the input matrix and the underlying hardware features. Various sparse matrix storage formats referred to commonly as sparse formats have been proposed in the literature to reduce the size of the matrix. In modern multi-core and many-core architectures, the performance of the kernel is mainly dependent on memory wall and power wall problem. Normally review on sparse formats is done with specific architecture or with specific application. This chapter presents a comparative study on various sparse formats in cross platform architecture like CPU, graphics processor unit (GPU), and single instruction multiple data stream (SIMD) registers. Space complexity analysis of various formats with its representation is discussed. Finally, the merits and demerits of each format have been summarized into a table.

A prototype OCR is constructed using a very compact parallel processing unit. This unit, designed for interactive character recognition applications, is equipped with a hand-scanner for input and a personal computer for word and/or image processing. The heart of this unit is a bit-serial Single Instruction Multiple Data stream (SIMD) array processor constructed with four identical cellular array LSIs (AAP2). The processor is fully programmable and the complex pro-cess of Japanese character recognition can be carried out with a single program package. Its architecture permits flexible and high-speed SIMD operations to process bitline data such as local fields of scanned documents. The processor components were integrated into one board and confirmed to be more than ten times faster than present image processors of the same size through various image processing tests. High character recognition performance is obtained at a reading speed of 8 Japanese characters per second which is sufficient for hand-scanning data input operations. The recognition rate is higher than 98% for about 3,300 Japanese characters.

As the complexity of speech recognition systems increases, conventional computers are unable to perform all the operations quickly enough to process the speech input in real time. This paper examines the use of a single instruction stream - multiple data stream (SIMD) parallel architecture to reduce the computation time for a dynamic time warping template matching based isolated word recognition system. Each of the components of the recognition system was written as an SMD parallel algorithm. The component SIMD algorithms and the complete word recognition system were simulated. The simulations showed that with 100 processing elements, each an 8 MHz MC68000, the speech system can perform isolated word recognition over a large vocabulary in real time. This real-time speech system used a 20 KHz sampling rate, 16 bits per sample, 100 samples per frame, 8 LPC coefficients per frame, 16 bits per LPC coefficient, and a 1,000 word vocabulary.

Three-dimensional image registration is computationally intensive and therefore slow. Parallelizing the primitive computation processes involved in registration can decrease the time. Our approach for improving image processing speed is to utilize a massively parallel single instruction stream -- multiple data stream (SIMD) computer. This paper presents the performance results comparing a massively parallel architecture to a four processor vector/parallel system and a conventional serial computer.