Modeling and optimization of buffering trade-offs for hardware implementation of image processing applications

Handbook of Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing shows you how robust subspace learning and tracking by decomposition into low-rank and sparse matrices provide a suitable framework for computer vision applications. Incorporating both existing and new ideas, the book conveniently gives you one-stop access to a number of different decompositions, algorithms, implementations, and benchmarking techniques. Divided into five parts, the book begins with an overall introduction to robust principal component analysis (PCA) via decomposition into low-rank and sparse matrices. The second part addresses robust matrix factorization/completion problems while the third part focuses on robust online subspace estimation, learning, and tracking. Covering applications in image and video processing, the fourth part discusses image analysis, image denoising, motion saliency detection, video coding, key frame extraction, and hyperspectral video processing. The final part presents resources and applications in background/foreground separation for video surveillance. With contributions from leading teams around the world, this handbook provides a complete overview of the concepts, theories, algorithms, and applications related to robust low-rank and sparse matrix decompositions. It is designed for researchers, developers, and graduate students in computer vision, image and video processing, real-time architecture, machine learning, and data mining.

The first quarter of 2007 started as an active season for the real-time image processing community. The annual SPIE Conference on Real-Time Image Processing, which was held in San Jose, CA, highlighted recent developments in this growing branch of image processing. As one of the few regular events with its focus matching that of JRTIP, one witnessed the growth in applications of real-time image and video processing. The conference served as an effective instrument to foster the dissemination of recent results and to leverage the cohesion of the real time image processing community. The preparations for the next SPIE Conference on Real-Time Image Processing in 2008 are already progressing and its call-for-papers is included at the end of this issue. In 2007, JRTIP is going to seek involvement of more scientists in the field of real-time image and video processing. After 1 year of operation, the editors-in-chief have started to recruit additional associate editors to enlarge the expertise base of the editorial board, to solicit manuscript submissions from potential authors, and to make the reviewing process more efficient. Interested colleagues should contact the editors-in-chief expressing their Associate Editor candidature in support of one or more fields within the focus of JRTIP. In addition, they wish to encourage researchers to act as guest editors for special issues on any real-time image or video processing related topics of interest to them. It is possible to schedule two special issues in 2007, provided that a draft call-for-papers is sent to the editors-in-chief as soon as possible. The third issue of JRTIP presents four articles, two are survey papers and two are original research papers. These articles address different real-time image processing perspectives ranging from algorithmic to hardware issues. The first survey paper deals with the real-time filtering in 2D and 3D using ordered statistics that are supported by a rich set of experimental results derived from several algorithmic variations. Performance measurements are used to demonstrate the effect of noise suppression and preservation of fine image structures and to show the DSP implementation of denoising filters. The second survey paper introduces a non-invasive analysis of dynamic processes in living cells using microscopic computer vision techniques for understanding the interactions between cellular and molecular functional units and for providing novel insights into the cell biology. The third article is an original research paper covering a machine vision application involving vision-based position sensing for aerial refueling of unmanned aerial vehicles in real-time. A full image processing chain is investigated and tested in a MATLAB/Simulink soft real-time environment as well as in a Linux/RTAI hard real-time environment. The feasibility of the real-time implementation is shown using off-the-shelf commercial hardware. The fourth article is another original research paper describing a hardware architecture for real-time fractal coding for image compression. The introduced parallel hardware architecture contains units for domain M. F. Carlsohn (&) Ingenieurberatung fur Computer Vision und Bildkommunikation, Am Heiddamm 36g, 28355 Bremen, Germany e-mail: matthias.carlsohn@t-online.de

The objective of this work is to design and implement an Image and Video Processing Platform (IVPP) on FGPAs using PICO based HLS. This hardware/software codesign platform has been implemented on a Xilinx Virtex-5 FPGA. The video interface blocks are done in RTL and the initialization phase is done using a MicroBlaze processor allowing the support of multiple video resolutions. This paper discusses the architectural building blocks showing the flexibility of the proposed platform. This flexibility is achieved by using a new design flow based on PICO. IVPP allows custom-processing blocks to be plugged-in to the platform architecture without modifying the front-end (capturing video data) and back-end (displaying processed output). This paper presents several examples of video processing applications, such as a Canny edge detector, motion detector and object tracking that have been realized using IVPP for real-time video processing.

As wireless video products evolve, they demand more sophisticated processing at higher resolutions and frame rates. Computational performance and energy efficiency have become critical design issues. This paper presents the Quantized Color Pack eXtension (QCPX) combined with a loop unrolling (LU) technique to improve execution performance and energy efficiency of color image and video processing applications. QCPX applied to a 32-bit datapath processor supports parallel operations on two packed 16-bit YCbCr (Y: luminance, Cr and Cb: chrominance) color pixels, providing greater subword-level parallelism by increasing the number of smaller color pixels packed into a word. Instruction-level parallelism can be further enhanced through loop unrolling. These techniques provide greater performance and efficiency for multimedia workloads on mobile systems. Experimental results on a set of media benchmark applications indicate that the LU plus QCPX-optimized version achieves a speedup ranging from 3.8 to 7.9 while reducing the energy consumption from 76% to 87% over the baseline version on identically configured, dynamically scheduled ILP superscalar processors. The LU plus QCPX-optimized version also outperforms the LU plus MDMX-like (MIPS’s multimedia extension) version.

Wavelet transform-based methods are currently used in a variety of image and video processing applications and are popular candidates for future image and video processing standards. Very little, however, has been done to develop efficient and simple stochastic models for wavelet image data. In this paper we review some existing modeling approaches for wavelet image data. Inspired by our recent estimation-quantization image coder, we introduce an efficient graphical stochastic model for wavelet image coefficients. Specifically, we propose to model wavelet image coefficients as Gaussian random variables with parameters determined by an underlying hidden Markov-type process. This stochastic model is defined using a factor graph framework. We test our model for denoising images corrupted by additive Gaussian noise. Our results are among the state-of-the-art in the field and they indicate the promise of the proposed model.

In this paper, we propose an efficient memory partitioning algorithm for parallel data access via data reuse. We found that for most of the applications in image and video processing, a large amount of data can be reused among different iterations in a loop nest. Motivated by this observation, we propose to cache these reusable data by on-chip registers. The on-chip registers used to cache the re-fetched data can be organized as chains of registers. The non-reusable data are then partitioned into several memory banks by a memory partition algorithm. We revise the existing padding method to cover cases occurring frequently in our method that some components of partition vector are zeros. Experimental results have demonstrated that compared with the state-of-the-art algorithms the proposed method can reduce the required number of memory banks by 59.8% on average. The corresponding resources for bank mapping is also significantly reduced. The number of LUTs is reduced by 78.6%. The number of Flip-Flops is reduced by 66.8%. The number of DSP48Es is reduced by 41.7%. Moreover, the storage overheads of the proposed method are zeros for most of the widely used access patterns in image filtering.

Wavelet packets provide a flexible representation of data, which has been proved to be very useful in a lot of applications in signal, image and video processing. In particular, in image and video coding, their ability to best capture the input content features can be exploited by designing appropriate optimization criteria. In this paper, we introduce joint wavelet packets for groups of frames, allowing to provide a unique best basis representation for several frames and not one basis per frame, as classically done. Two main advantages are expected from this joint representation. On the one hand, bitrate is spared, since a single tree description is sent instead of 31 per group of frames (GOP) - when a GOP contains, for example, 32 frames. On the other hand, this common description can characterize the spatio-temporal features of the given video GOP and this way can be exploited as a valuable feature for video classification and video database searching. A second contribution of the paper is to provide insight into the modifications necessary in the best basis algorithm (BBA) in order to cope with biorthogonal decompositions. A computationally efficient algorithm is deduced for an entropy-based criterion.

Many modern applications in important application domains, as communication, image and video processing, multimedia, etc. involve much data-level parallelism (DLP). Therefore, adequate exploitation of DLP is highly relevant. This paper focuses on effective and efficient exploitation of DLP for the synthesis of vector VLIW ASIP processors. We propose analytical energy models in order to rapidly estimate the energy consumption of a nested loop executed on a VLIW ASIP with respect to different vector widths. The models perform a rapid and relatively accurate energy consumption estimation through combining the relevant information on the application and implementation technology. The analytical energy models are experimentally validated and the validation results are discussed.

This paper presents the derivation of the Duality Property for the DCT-IV, which is a finite-length discrete transform having applications in Digital Signal, Image and Video Processing. This property can be applied for finding the temporal domain signal from its frequency domain counterpart or mutatis mutandis if an existing DCT-IV pair is known to exist, thus receding considerable computational gruntwork and disbursement substantially.

Multidimensional Signal, Image, and Video Processing and Coding

Window-based operations such as two dimensional (2-D) convolution operations are commonly used in image and video processing applications. In this paper, a new design technique that considers the neighboring pixels within the window to detect and eliminate redundant or unnecessary computations for power reduction is presented. A novel on-chip detection technique is developed for the proposed neighborhood dependent approach (NDA) to reduce computations. In addition, data partitioning methodology is employed in the on chip buffer design support real-time operations. This NDA method is applied to different window buffering schemes and experimental results are presented.

In this chapter applications of image and video processing to navigation of mobile robots are presented. During the last years some impressive real time applications have been showed to the world, such as the NASA missions to explore the surface of Mars with autonomous vehicles; in those missions, video and image processing played an important role to rule the vehicle. Algorithms based on the processing of video or images provided by CCD sensors or video cameras have been used in the solution of the navigation problem of autonomous vehicles. In one of those approaches, a velocity field is designed in order to guide the orientation and motion of the autonomous vehicle. A particular approach to the solution of the navigation problem of an autonomous vehicle is presented here. In the first section of this introduction a state of the art review is presented, after it, the proposed algorithm is summarized; the following sections present the procedure. Finally, some experimental results are shown at the end of the chapter.

. The development of an Image Processing (IP) application is a complex activity, which can be greatly alleviated by user-friendly graphical programming environments. Our major objective is to help IP experts reuse parts of their applications. A first work towards knowledge reuse has been to propose a suitable representation of the strategies of IP experts by means of IP plans (trees of tasks, methods and tools). This paper describes the CBR module of our interactive system for the development of IP plans. After a brief presentation of the overall architecture of the system and its other modules, we explain the distinction between an IP case and an IP plan, and give the selection criteria and functions that are used for similarity calculation. The core of the CBR module is a search/adaptation algorithm, whose main steps are detailed: retrieval of suitable cases, recursive adaptation of the selected one and memorization of new cases. The system’s implementation is presently completed; its functioning is described in a session showing the kind of assistance provided by the CBR module during the development of a new IP application.

Epitomic analysis, a recent statistical approach to form a generative model, has been applied to image, video and audio processing applications. We apply the epitomic analysis to motion capture data and define it as a motion epitome, a probabilistic model representing a finite set of primitive movements which retain various lengths of local dynamics. We review the generation, inference and learning procedures of an epitome, adapt them for motion capture data and utilize the epitomic analysis to detect unnatural movements given only positive (natural) training data. We introduce a multi-resolution of motion epitomes as well as a full body and an ensemble of epitomes, then present experimental results and compare the performance with other conventional classification methods, including Hidden Markov Models and Switching Linear Dynamic Systems.

PLX is a concise instruction set architecture (ISA) that combines the most useful features from previous generations of multimedia instruction sets with newer ISA features for high-performance, low-cost multimedia information processing. Unlike previous multimedia instruction sets, PLX is not added onto a base processor ISA, but designed from the beginning as a standalone processor architecture optimized for media processing. Its design goals are high performance multimedia processing, general-purpose programmability to support an ever-growing range of applications, simplicity for constrained environments where low power and low cost are paramount, and scalability for higher performance in less constrained multimedia systems. Another design goal of PLX is to facilitate exploration and evaluation of novel techniques in instruction set architecture, microarchitecture, arithmetic, VLSI implementations, compiler optimizations, and parallel algorithm design for new computing paradigms. Key characteristics of PLX are a fully subword-parallel architecture with novel features like wordsize scalability from 32-bit to 128-bit words, a new definition of predication, and an innovative set of subword permutation instructions. We demonstrate the use and high performance of PLX on some frequently-used code kernels selected from image, video, and graphics processing applications: discrete cosine transform, pixel padding, clip test, and median filter. Our results show that a 64-bit PLX processor achieves significant speedups over a basic 64-bit RISC processor and over IA-32 processors with MMX and SSE multimedia extensions. Using PLX's wordsize scalability feature, PLX-128 often provides an additional 2× speedup over PLX-64 in a cost-effective way. Superscalar or VLIW (Very Long Instruction Word) PLX implementations can also add additional performance through inter-instruction, rather than intra-instruction parallelism. We also describe the PLX testbed and its software tools for architecture and related research.