NeNEB-an application adjustable single chip neural network processor for mobile real time image processing

The first quarter of 2007 started as an active season for the real-time image processing community. The annual SPIE Conference on Real-Time Image Processing, which was held in San Jose, CA, highlighted recent developments in this growing branch of image processing. As one of the few regular events with its focus matching that of JRTIP, one witnessed the growth in applications of real-time image and video processing. The conference served as an effective instrument to foster the dissemination of recent results and to leverage the cohesion of the real time image processing community. The preparations for the next SPIE Conference on Real-Time Image Processing in 2008 are already progressing and its call-for-papers is included at the end of this issue. In 2007, JRTIP is going to seek involvement of more scientists in the field of real-time image and video processing. After 1 year of operation, the editors-in-chief have started to recruit additional associate editors to enlarge the expertise base of the editorial board, to solicit manuscript submissions from potential authors, and to make the reviewing process more efficient. Interested colleagues should contact the editors-in-chief expressing their Associate Editor candidature in support of one or more fields within the focus of JRTIP. In addition, they wish to encourage researchers to act as guest editors for special issues on any real-time image or video processing related topics of interest to them. It is possible to schedule two special issues in 2007, provided that a draft call-for-papers is sent to the editors-in-chief as soon as possible. The third issue of JRTIP presents four articles, two are survey papers and two are original research papers. These articles address different real-time image processing perspectives ranging from algorithmic to hardware issues. The first survey paper deals with the real-time filtering in 2D and 3D using ordered statistics that are supported by a rich set of experimental results derived from several algorithmic variations. Performance measurements are used to demonstrate the effect of noise suppression and preservation of fine image structures and to show the DSP implementation of denoising filters. The second survey paper introduces a non-invasive analysis of dynamic processes in living cells using microscopic computer vision techniques for understanding the interactions between cellular and molecular functional units and for providing novel insights into the cell biology. The third article is an original research paper covering a machine vision application involving vision-based position sensing for aerial refueling of unmanned aerial vehicles in real-time. A full image processing chain is investigated and tested in a MATLAB/Simulink soft real-time environment as well as in a Linux/RTAI hard real-time environment. The feasibility of the real-time implementation is shown using off-the-shelf commercial hardware. The fourth article is another original research paper describing a hardware architecture for real-time fractal coding for image compression. The introduced parallel hardware architecture contains units for domain M. F. Carlsohn (&) Ingenieurberatung fur Computer Vision und Bildkommunikation, Am Heiddamm 36g, 28355 Bremen, Germany e-mail: matthias.carlsohn@t-online.de

An analog Cellular Neural Network (CNN) architecture employing quantum dots to realize various real time image processing applications such as edge and line detections and motion estimation is proposed. In order to obtain programmability to switch between applications, memristive connections between neighboring cells, and for signal amplification and locking resonant tunneling diodes (RTDs) are utilized. Simulations are carried out on a 2D array of the proposed cell structure to demonstrate edge detection and line detection tasks. This work also provides analytical models and simulation results to prove above mentioned real time image processing functionalities.

Now a day's various real time applications and image processing applications requires hardware that can perform various complex arithmetic operations. These operations can be performed by using binary logarithmic number system. This paper includes binary logarithmic circuit based on FPGA. Above architecture uses combinational logic circuit elements and fixed point data path number format. The architecture is able to calculate the logarithm of integer number, fractional number and integer fractional number. This architecture is designed in Xilinx Virtex-5 device. This architecture consumes minimal FPGA resources that are shown by device utilization summary. Finally error analysis is done which shows that architecture has minimal number of errors considering fractional number and fixed point numbers.

The present of Speckle noise affects the exploitation of useful information in images, especially, in the ultrasound and SAR image. It is desirable to remove speckle noise in real time processing without damaging the image features. In this paper, we propose a new method to suppress the speckle noise in real- time by using Cellular Neural Network (CNN) technology. Section 2, CNN and Cellular Neural Network Universal Machine (CNN-UM) will be briefly introduced, which are the optimized technology for real time image processing applications. Section 3, a few mathematical models of anisotropic diffusion methods for reducing speckle noise on the SAR images will be surveyed and analyzed. In section 4, is also the main part, presented how to design a speckle denoising filter to eliminate the speckle noise and enhance the image features by converting the conventional denoising algorithm to templates, that make CNN chip understandable and processes as a real-time speckle reducing filter. The experiment's results are given in section 5 and some conclusions and future works are summarized in the last section.

Cellular neural networks show high performance capabilities in real time image processing applications. For this reason, their use in biomedical image analysis can be a useful aid to the doctor in clinical diagnosis. In this research area the improvements in systems for clinical specular microscopy in vivo made a strong contribution to the study and the comprehension of the physiopathology of corneal endothelium. The more recent systems allow acquisition of the images and morphometric analysis. Nevertheless, the results (i.e. the automated reconstruction of the endothelium cell borders) are often inaccurate. Moreover, they do not allow the correct recognition of the cell shapes. On the other hand, even if the semiautomatic systems allow an effective evaluation of the cell shape, they are highly time consuming and provide results that could be affected by the criterion used by the operator in the cell corner detection. In this paper a software tool for the full automated morphometric analysis of corneal endothelium images is presented. The tool makes use of an analogue cellular neural network algorithm that allows both cell shape recognition and endothelial cell area measurement.

In real-time image processing, the demand for efficient solutions has surged with the proliferation of applications spanning from autonomous vehicles to medical diagnostics. This study addresses the imperative need for accelerated machine learning algorithms to enhance the processing speed of image-related tasks. The research focuses on leveraging Field-Programmable Gate Arrays (FPGAs) to implement hardware acceleration, exploiting their parallel computing capabilities. The advent of machine learning in image processing has revolutionized various industries, yet real-time applications encounter computational bottlenecks. This research delves into hardware acceleration using FPGAs to overcome these constraints, offering a novel approach to expedite machine learning algorithms. Traditional software implementations of machine learning algorithms often fall short in meeting real-time processing requirements. This research aims to bridge this gap by exploring FPGA-based hardware acceleration, addressing the performance limitations hindering the seamless integration of machine learning into real-time image processing systems. While existing literature acknowledges the potential of FPGA-based acceleration, a comprehensive exploration of its application for real-time image processing is lacking. This research fills the void by presenting a detailed method and empirical results, contributing to the limited body of knowledge on FPGA-accelerated machine learning in the of image processing. The study employs a systematic approach, integrating machine learning algorithms onto FPGAs through hardware description languages. The implementation is optimized to exploit parallelism inherent in FPGAs, resulting in a tailored hardware solution for real-time image processing. Comparative analyses against software implementations provide insights into the performance gains achieved. The experimental results demonstrate a significant enhancement in processing speed, validating the efficacy of FPGA-based hardware acceleration for machine learning algorithms in real-time image processing applications.

There is a dearth of high quality reconfigurable hardware for DWT which can be extensively used in various real time signal and image processing applications. In this article, we have focused on proposing a dedicated customizable hardware for DWT applicable to 1D/2D signal processing. We have proposed bit-serial Distributed Arithmetic (DA) based VLSI architectures for ID/2D DWT. Exploitation of DA enables us to make our designs Multiplierless, thereby consuming less area. Bit serial configuration of DA is also exploited to introduce modularity and pipelining in the proposed convolution DWT based ID/2D architectures. Though the speed of the proposed designs may not be suitable for certain applications, a number of parallel channels can be introduced for the same. The provision of mode selection can efficiently be used to realize 9/7 and also 5/3 DWT filters in the same proposed architecture. In the proposed 1D/2D architecture, we have achieved about 50% memory size reduction, 30% reduction in area consumption and comparable speed in comparison to the other latest notable DWT architecture so far. We have verified the viability of our proposed 2D architecture by utilizing it in real time image decomposition.

In this paper we introduce a new real-time parallel image processing system based on an IEEE1394-based PC-cluster. PC-clusters are becoming standard tools in the field of computer vision, especially for real-time multi-view image processing. To reduce the cost of system construction, we have employed IEEE1394 bus as a base network of the PC-cluster. IEEE1394 bus has the throughput of up to 400Mbps, which is large enough to transfer uncompressed video data, and supports not only asynchronous data transfer but isochronous one, which is inevitable for real-time data transfer. In addition, since we have several products of IEEE1394-based digital cameras, we can easily integrate real-time image processing system including camera systems. This paper presents an overview of our real-time parallel image processing system on an IEEE1394based PC-cluster, referring to basic features of IEEE1394. We also show some experimental results to evaluate the performance of the PC-cluster, comparing it with the performance of our previous Myrinet-based PC-cluster. Then we show a prototypical application of the system, which is real-time volume reconstruction from multi-view images by visual cone intersection method.

Higher throughput is always desired in real time image processing applications. There are many ways to achieve higher throughput. However, if we have additional resources and memory bandwidth available, parallelism can be applied to achieve it. In this work, we have presented two image scanning methods that carry out parallelism to double the throughput of any architecture. Partitioned image scanning method divides the input image into two or more sub-images and then applies the desired filter function on each sub-image in parallel. Adjacent row scanning method provides the input to each filter alternatively, one accessing odd rows and the other even rows of the input image. From the results, it is evident that the resources utilized by partitioned image scanning method are more than that of adjacent row scanning method. The results show that adjacent row scanning method require approximately 20% lesser resources than that required by partitioned image scanning method.

Compressed Sensing (CS) methods impact is important in the health care systems. The acquisition and the processing speed of many medical imaging applications is highly improved using this technique. Orthogonal Matching Pursuit (OMP) is one of the widely used image reconstruction algorithms in CS techniques. Traditionally, OMP is divided on two main tasks: optimization stage and least square problem (LSP) resolution stage. The LSP resolution is the most complex and time consuming step in OMP. QRD is one of the main techniques used to solve the LSP in a very short time. However, the increasing image and video resolutions (UHDTV) makes the real time constraints very tight. In this paper, a novel implementation of QRD on VLIW DSP architecture with an efficient memory access approach is introduced. An instruction, data and loop levels parallelism (ILP, DLP and LLP) based Givens QRD kernel is designed. A robust memory access management based on intelligent loading/storing strategies and proficient data alignment techniques is adopted for the proposed parallel implementation. Therefore, memory read misses and CPU stalls are significantly diminished. As a final result the processing time is greatly reduced. The proposed scheme is implemented on C6678 DSP and reaches 2.22 GFLOPS. The performances achieved are 7.4, 3.83 and 2 times faster than standard Givens, Texas Instrument optimized QRD and state of the art, respectively.

Reverse Engineering on image contours plays a significant role in data reduction, transformation and deformation of shapes without the loss of reasonable quality. The mathematical basis of captured contours are used in various applied areas such as facial modeling, animations, real time image processing applications, automatic sketching, shape matching, capturing hand drawn images and designing civil engineering, bio-medical and mechanical engineering objects etc. The authors do hope that this article can effectively provides algorithmic review for approximating outlines of planar images. Furthermore, it offers a sound analysis of pros and cons of specified techniques. It also delivers enhancements and optimizations via exploring the weaknesses of existing algorithms and attacking them through hybrid solutions.

Binary morphological operations are an important means for real time image processing applications. While fixed size structuring elements are sufficient for usual image processing steps, more sophisticated algorithms require structuring elements of variable shape. These operations can be realized by using Boolean logic in connection with dead time elements, which leads to a straightforward implementation if using FPGAs.

In this paper we present a methodology and tool for rapid prototyping of real time image processing applications. We describe our design flow of multiprocessor system on chip (MPSoC) architectures based on hardware/software components. This methodology provides automated methods to specify, generate the hardware, software, and the architectural interfaces between them. Our methodology starts from system level specification of the application with parallel processes described in C-code. The processes communicate through an abstract channel called streams. We describe also the solution that we proposed to synthesize a custom bus architecture for the reconfigurable computing applications, which therefore allows minimizing hardware cost in the FPGA. On the other hand, this methodology allows generating the communication architecture based on the needs of the application. Finally, we demonstrate the effectiveness of our approach through an image processing application.

In this paper, a method for Autonomous Vehicle is presented based on real time image processing. The system detects the road by sobel edge detection, and it recognizes the obstacles, humans, and traffic lights by heuristic techniques. For this aim, some features are defined based on key points. One important technique is that every frame is divided in some sections which significantly affect time of processing. The system is able to analyze 30 frames per second to get the best decision for controlling the vehicle. The achieved structure is optimized on accuracy and the number of logic cells. The algorithms completely describes in hardware by VHDL, then implemented on DE1-SOC board which uses cyclone V FPGA.

Commercial Off The Shelf (COTS) Chip Multi-Processor (CMP) systems are for cost reasons often used in industry for soft real-time stream processing. COTS CMP systems typically have a low timing predictability, which makes it difficult to develop software applications for these systems with tight temporal requirements. Restricting the way applications use the hardware and Operating System (OS) might alleviate this difficulty, so that certain types of applications could be run on COTS CMP systems with statistically verified temporal requirements. In this thesis we restrict the application domain to soft real-time medical image processing applications, which have a much more ‘stable’ usage of hardware resources than applications in general. Techniques at the application level are employed to improve the reproducibility (i.e. to reduce the variance) of the end-to-end latency of these imaging processing systems. Firstly, we study the effectiveness of a number of scheduling heuristics that are intended to improve the reproducibility of a stream processing application that is executed on COTS multiprocessor systems. Experiments show that the proposed heuristics can reduce the end-to-end latency with almost 60%, and reduce the variation in the latency with more than 90%, when compared with a naive scheduling heuristic that does not consider execution times, dependencies and the memory hierarchy. Secondly, we want to be able to integrate multiple real-time and best-effort applications on a single COTS CMP system without reducing the reproducibility of the real-time application too much. For this we examined the first component that is shared between different applications running on separate cores, the shared cache and in particular the bandwidth in the cache. We propose a technique that implements cache bandwidth reservation in software. This is achieved by dynamically duty-cycling best-effort applications based on their cache bandwidth usages measured with processor performance counters. With this technique we can control the latency increase of real-time applications that is caused by best-effort applications. Thirdly, we introduce the Probabilistic Time Triggered System (PTTS) model to analyze and optimize the end-to-end latency of a complete system that contains multiple time triggered interfaces. Our case study demonstrates the applicability of the PTTS model and the corresponding analysis techniques for an interventional X-ray system. We expect that the PTTS model is also applicable for other systems than medical image processing systems.