A Design of Real-Time Image Processing Platform Based on TMS320C6678

In this paper we introduce a new real-time parallel image processing system based on an IEEE1394-based PC-cluster. PC-clusters are becoming standard tools in the field of computer vision, especially for real-time multi-view image processing. To reduce the cost of system construction, we have employed IEEE1394 bus as a base network of the PC-cluster. IEEE1394 bus has the throughput of up to 400Mbps, which is large enough to transfer uncompressed video data, and supports not only asynchronous data transfer but isochronous one, which is inevitable for real-time data transfer. In addition, since we have several products of IEEE1394-based digital cameras, we can easily integrate real-time image processing system including camera systems. This paper presents an overview of our real-time parallel image processing system on an IEEE1394based PC-cluster, referring to basic features of IEEE1394. We also show some experimental results to evaluate the performance of the PC-cluster, comparing it with the performance of our previous Myrinet-based PC-cluster. Then we show a prototypical application of the system, which is real-time volume reconstruction from multi-view images by visual cone intersection method.

The first quarter of 2007 started as an active season for the real-time image processing community. The annual SPIE Conference on Real-Time Image Processing, which was held in San Jose, CA, highlighted recent developments in this growing branch of image processing. As one of the few regular events with its focus matching that of JRTIP, one witnessed the growth in applications of real-time image and video processing. The conference served as an effective instrument to foster the dissemination of recent results and to leverage the cohesion of the real time image processing community. The preparations for the next SPIE Conference on Real-Time Image Processing in 2008 are already progressing and its call-for-papers is included at the end of this issue. In 2007, JRTIP is going to seek involvement of more scientists in the field of real-time image and video processing. After 1 year of operation, the editors-in-chief have started to recruit additional associate editors to enlarge the expertise base of the editorial board, to solicit manuscript submissions from potential authors, and to make the reviewing process more efficient. Interested colleagues should contact the editors-in-chief expressing their Associate Editor candidature in support of one or more fields within the focus of JRTIP. In addition, they wish to encourage researchers to act as guest editors for special issues on any real-time image or video processing related topics of interest to them. It is possible to schedule two special issues in 2007, provided that a draft call-for-papers is sent to the editors-in-chief as soon as possible. The third issue of JRTIP presents four articles, two are survey papers and two are original research papers. These articles address different real-time image processing perspectives ranging from algorithmic to hardware issues. The first survey paper deals with the real-time filtering in 2D and 3D using ordered statistics that are supported by a rich set of experimental results derived from several algorithmic variations. Performance measurements are used to demonstrate the effect of noise suppression and preservation of fine image structures and to show the DSP implementation of denoising filters. The second survey paper introduces a non-invasive analysis of dynamic processes in living cells using microscopic computer vision techniques for understanding the interactions between cellular and molecular functional units and for providing novel insights into the cell biology. The third article is an original research paper covering a machine vision application involving vision-based position sensing for aerial refueling of unmanned aerial vehicles in real-time. A full image processing chain is investigated and tested in a MATLAB/Simulink soft real-time environment as well as in a Linux/RTAI hard real-time environment. The feasibility of the real-time implementation is shown using off-the-shelf commercial hardware. The fourth article is another original research paper describing a hardware architecture for real-time fractal coding for image compression. The introduced parallel hardware architecture contains units for domain M. F. Carlsohn (&) Ingenieurberatung fur Computer Vision und Bildkommunikation, Am Heiddamm 36g, 28355 Bremen, Germany e-mail: matthias.carlsohn@t-online.de

In real-time image processing, the demand for efficient solutions has surged with the proliferation of applications spanning from autonomous vehicles to medical diagnostics. This study addresses the imperative need for accelerated machine learning algorithms to enhance the processing speed of image-related tasks. The research focuses on leveraging Field-Programmable Gate Arrays (FPGAs) to implement hardware acceleration, exploiting their parallel computing capabilities. The advent of machine learning in image processing has revolutionized various industries, yet real-time applications encounter computational bottlenecks. This research delves into hardware acceleration using FPGAs to overcome these constraints, offering a novel approach to expedite machine learning algorithms. Traditional software implementations of machine learning algorithms often fall short in meeting real-time processing requirements. This research aims to bridge this gap by exploring FPGA-based hardware acceleration, addressing the performance limitations hindering the seamless integration of machine learning into real-time image processing systems. While existing literature acknowledges the potential of FPGA-based acceleration, a comprehensive exploration of its application for real-time image processing is lacking. This research fills the void by presenting a detailed method and empirical results, contributing to the limited body of knowledge on FPGA-accelerated machine learning in the of image processing. The study employs a systematic approach, integrating machine learning algorithms onto FPGAs through hardware description languages. The implementation is optimized to exploit parallelism inherent in FPGAs, resulting in a tailored hardware solution for real-time image processing. Comparative analyses against software implementations provide insights into the performance gains achieved. The experimental results demonstrate a significant enhancement in processing speed, validating the efficacy of FPGA-based hardware acceleration for machine learning algorithms in real-time image processing applications.

Time is the best important resource in real-time video image processing system. Only few structure or algorithm can be used in real-time video image processing system because of time limitation. Establish a high speed data-access is a way to this problem. It transfer the video signal between output ports and kernel processor as less time delay. The mode, look-up-table, other analysis, adjustment or calculation rule will be applied first by medium or low data-access. We can choose different data-access according to data relation level. Data which has closer correlation will be treated through low-speed-access, which has loser correlation will be treated through medium-speed-access. Theory analysis and experiment showed that it is a high effective structure for real-time video image processing. The feature of video image and real-time processing system is presented in this paper. The basic theory of data-access is introduced also. The status and relation of high, medium, low data-access is discussed and the rules of data distribution in different access is analyzed. Some examples with datas-accesses are given in the paper (TV tracking and measuring, laser signal alarming system).

Real-Time Image Processing is the key work in robot vision. With the limitation of the hardware technique, many algorithm-oriented firmware systems were designed in the past. But their architectures were not flexible enough to achieve a multi- algorithm development system. Because of the rapid development of microelectronics technique, many high performance DSP chips and high density FPGA chips have come to life, and this makes it possible to construct a more flexible architecture in real-time image processing system. In this paper, a Double DSP General Image Processing System (DDGIPS) is concerned. We try to construct a two-DSP-based FPGA-computational system with two TMS320C6201s. The TMS320C6x devices are fixed-point processors based on the advanced VLIW CPU, which has eight functional units, including two multipliers and six arithmetic logic units. These features make C6x a good candidate for a general purpose system. In our system, the two TMS320C6210s each has a local memory space, and they also have a shared system memory space which enable them to intercommunicate and exchange data efficiently. At the same time, they can be directly interconnected in star- shaped architecture. All of these are under the control of FPGA group. As the core of the system, FPGA plays a very important role: it takes charge of DPS control, DSP communication, memory space access arbitration and the communication between the system and the host machine. And taking advantage of reconfiguring FPGA, all of the interconnection between the two DSP or between DSP and FPGA can be changed. In this way, users can easily rebuild the real-time image processing system according to the data stream and the task of the application and gain great flexibility.

A unique real-time hybrid optical-digital image processing system has been developed to perform analysis of low contrast visual and infrared images in radon-space. This system functions by implementing the forward radon transform (a mathematical tomographic transform of image data from two-dimensional image-space to one-dimensional radon-space) in a front-end optical processor, and a digital processing subsystem operating in radon-space instead of the more traditional image-space. The system works by optically converting the two-dimensional image-space data into a series of one-dimensional projections. All further processing is performed digitally in radon-space on the one-dimensional projections. Applications of interest such as the objective minimum resolvable temperature difference measurement of thermal imagers and automatic pattern recognition are discussed. Also, this paper discusses the topic of real-time object-moment analysis in radon-space which can be used for target identification under conditions of certain image distortions (size, rotation, translation, and contrast). Radon-space object-moments can be calculated using significantly less image data and fewer digital operations than those in image-space.

The unmanned aerial vehicle (UAV)-borne video synthetic aperture radar (SAR) possesses the characteristic of having high-continuous-frame-rate imaging, which is conducive to the real-time monitoring of ground-moving targets. The real-time imaging-processing system for UAV-borne video SAR (ViSAR) requires miniaturization, low power consumption, high frame rate, and high-resolution imaging. In order to achieve high-frame-rate real-time imaging on limited payload-carrying platforms, this study proposes a miniaturization design of a high-integration UAV-borne ViSAR real-time imaging-processing component (MRIPC). The proposed design integrates functions such as broadband signal generation, high-speed real-time sampling, and real-time SAR imaging processing on a single-chip FPGA. The parallel access mechanism using multiple sets of high-speed data buffers increases the data access throughput and solves the problem of data access bandwidth. The range-Doppler (RD) algorithm and map-drift (MD) algorithm are optimized using parallel multiplexing, achieving a balance between computing speed and hardware resources. The test results have verified that our proposed component is effective for the real-time processing of 2048 × 2048 single-precision floating-point data points to realize a 5 Hz imaging frame rate and 0.15 m imaging resolution, satisfying the requirements of real-time ViSAR-imaging processing.

A new design method for the real-time image processing system based on field programmable gate array (FPGA) and digital signal processing (DSP) structure is presented in the paper. In the practical system, FPGA is used to be logic unit for sensor interface protocol and complete image preprocessing and display of PAL video; DSP is applied as the main processing unit which is extendable for better performance; Modules are connected by user-configurable I/O interface, external memory interface (EMIF) and host-post interface (HPI), which configure the system to be a loosely coupled and cascaded multi-DSP system. Finally, the design approach shows good performance in the real-time image processing system with the capability of real-time processing, extendibility and multi-interface, which is significant for practical application.

In the weld image acquisition system, real-time image processing has been a difficult design bottleneck to break through, especially for the occasion of large data processing capability and more demanding real-time requirements, in which the traditional MCU can not adapt, so using high-performance FPGA as the core of the high speed image acquisition and processing card, better meets the large amount of data in most of the image processing system and high demanding real-time requirements. At the same time, system data collection, storage and display were implemented by using Verilog, and in order to reducing the influence of edge detection noise, the combination of image enhancement and median filtering image preprocessing algorithm was used. Compared to the pre-processing algorithm of the software implementation, it has a great speed advantage, and simplifies the subsequent processing work load, improves the speed and efficiency of the entire image processing system greatly. So it proves that the system has strong ability of restraining the noise of image, and more accurate extracted edge positioning, it can be applied in the seam tracking field which need higher real-time requirements.

CAS-computer assisted surgery-is a new imaging method supporting skull base surgeons. Support is granted not only for preoperative planning of therapy but also for pathfinding during surgery itself and in the postoperative phase as therapy control. The CAS-system consists of high technology items such as -a digital image generation system (CT, MR) -a real-time image processing system -a 3 D position recognition system. Robotics are not incorporated in this system but a hand-guided manipulator houses the surgeon's instrument. Accuracy of the method has been experimentally determined to be within 1 mm. Follow up systems are under development to permit microsurgery support as well.

In order to speedup image processing in embedded environment, we construct a parallel processing system of images. The parallel processing system includes a parallel hardware, a new defined language for parallel image processing and a set of software tools. From the view of high performance, low power dissipation and the characteristics of image processing, we construct a SIMD coprocessor as an image processing accelerator. Using a RISC host processor manages the whole system. The SIMD processor is scalable. We also define a new language which is extended from standard C. A new data type and a new keyword kernel are added to the language to explicitly describe parallelism. For the new hardware and language of parallel image processing, we also research software tools for this parallel system. The software tools maps programs to code that runs on the new hardware. For example develop a scheduler to transfer stream data between memories and register files. From the analysis we found that the parallel image processing system can not only match image applications' characteristics but also easy to implement by using VLSI technology. With new language and software tools supporting, an embedded real-time parallel image processing system becomes available for programmers.

A system design for performing low-level image processing tasks in real time is presented. The design is based on large processor-per-pixel arrays implemented using integrated circuit technology. Two integrated circuit architectures are summarized: an associative parallel processor and a parallel processor employing DRAM cells. In both architectures, the layout pitch of one-bit-wide logic is matched to the pitch of memory cells to form high-density processing element arrays. The system design features an efficient control path implementation, providing high processing element array utilization without demanding complex controller hardware. Sequences of array instructions are generated by a host computer before processing begins, then stored in a simple controller. Once processing begins, the host computer initiates stored sequences to perform pixel-parallel operations. A programming framework implemented using the C++ programming language supports application development. A prototype system employs associative parallel processor devices, a controller, and the programming framework. Three sample applications, smoothing and segmentation, median filtering, and optical flow, establish the suitability of the system for real-time image processing.

In order to solve the problem that all levels of civil aviation meteorological centers are unable to provide all the possible meteorological information services that may reach the low-altitude airspace for general aviation flight, this paper proposes a real-time low-altitude meteorological intelligence image collection and processing method. Based on this method, a real-time low-altitude meteorological intelligence image collection system can be designed and implemented. The real-time low-altitude meteorological intelligence image collection system is an information system that provides real-time low-altitude airspace visibility and weather condition information services for image-based flight photography, transmission, processing, and meteorological information dissemination. It includes the image collection subsystem and image processing subsystem, meteorological information publishing Web Server. Among them: The image collection subsystem is mainly used for automatically taking images of the surrounding environment at regular intervals, and transmitting the photographs and their time and place information to the image processing system installed in the low-altitude navigational information service department through the wireless communication device; the image processing subsystem collects images and related information for further processing; the meteorological information publication Web server mainly provides general airline users with services such as consultation, inquiry, and submission of low-altitude meteorological intelligence information of relevant airspace related to images.

Image edge detection significantly preserves the important structural properties in an image, while reducing the amount of data and filters out useless information. Since edge detection is in the forefront of image processing for object detection, it is crucial to have a good understanding of edge detection algorithms. This paper introduces the Canny edge detection algorithm and the implementation of the Canny edge-detection algorithm on an FPGA. Custom implementation in FPGAs, using a combination of hardware and software co-design, allows real-time processing, providing a good trade-off between performance and flexibility. The results show that FPGAs are good alternatives, which can be used to off-load these computationally-intensive and repetitive functions as co-processors and the capability of supporting canny edge detection for real-time image processing.

In this article, hardware and software tools is proposed to process image in satellite image processing systems so that the parameters can be calculated and make image viewable according to user requirements. This is a basic approach to solve real-time problems for water bodies like oceans. The extraction of Water bodies is difficult because of mountain and building shadow. There are three water indices for water bodies extraction namely, Normalized Difference Water Index (NDWI), Modified Normalized Difference Water Index (MNDWI) and Automated Water Extraction Index (AWEI) that extract water in real-time with onboard image processing system using FPGA. By selecting any input image there are given three options of NDWI, MNDWI and AWEI. When any one option is selected then the output is obtained according to selected indices. The proposed system of onboard real-time image processing using hardware software combination reduces total time to procure satellite image and extraction of water bodies.