Real-time microscopic video shooting using a view-expanded microscope system

In this study, a real-time image stitching method is proposed for cost-effective high resolution wide angle video shooting. In the first stage, the images taken from more than one camera with fixed position are stitched with a classical algorithm. After the parameters calculated in the first stage are stored, the image pixels are mapped using the stored parameters and ArUco markers. The mapping process is used for real-time panoramic video shooting after it has been calculated for that particular camera setup once. The images taken from the multi camera are combined by remapping with a GPU based approach. Therefore, registered mapping can be used in different environments without changing the position and lenses of the cameras. As a case study, real-time panoramic video is shot with two cost-effective cameras in football matches. Deep-learning based autonomous pilot video shooting is then performed on the high resolution panoramic video obtained. In experiments, 36 FPS speed has been reached by using a standard desktop computer and it has been seen that image quality measurements are at reasonable levels.

A real-time recursive back projection (BP) imaging algorithm is presented in this paper. Real time imaging is an intense demand but a challenging task in impulse surface penetrating radar (ImpSPR)'s application. Based on 'delay-sum' operation in time domain, BP imaging algorithm can precisely focus the scattering intensity and obtain high quality subsurface profile. But it's heavy computation burden restricts it's application in ImpSPR's real-time processing. By minutely analyzing it's procedure, a recursive model of BP imaging algorithm is established and real-time BP imaging algorithm is educed subsequently. The computation complexity of both non real time BP imaging algorithm and real time BP imaging algorithm are analyzed. Through processing the experimental data obtained by a ImpSPR system- RadarEye, the imaging algorithm validates its capability at the aspect of ImpSPR's real time processing.

In order to achieve faster and more dexterous manipulations, we propose a strategy called holding. In the dynamic holding condition, the object is held in a stable condition while moving at high-speed. In this paper, we first explain the concept of dynamic holding, then we describe an experiment of dribbling by a robot as an example of dynamic holding using a high-speed multifingered hand and a high-speed vision system.

We developed a novel real-time motion blur compensation system for the blur caused by high-speed one-dimensional motion between a camera and a target. The system consists of a galvanometer mirror and a high-speed color camera, without the need for any additional sensors. We controlled the galvanometer mirror with continuous back-and-forth oscillating motion synchronized to a high-speed camera. The angular speed of the mirror is given in real time within 10 ms based on the concept of background tracking and rapid raw Bayer block matching. Experiments demonstrated that our system captures motion-invariant images of objects moving at speeds up to 30 km/h.

When designing a system capable of capturing and displaying three-dimensional (3-D) moving images in real time by the integral imaging (II) method, one challenge is to eliminate pseudoscopic images. To overcome this problem, we propose a simple system with an array of three convex lenses. This paper first describes by geometrical optics the lateral magnification of the elemental optics and expansion of an elemental image, confirming that the elemental optics satisfies the conditions under which pseudoscopic images can be avoided. By the II method, adjacent elemental images must not overlap, a condition also satisfied by the proposed optical system. Next, the paper describes an experiment carried out to acquire and display 3-D images. The real-time system we have constructed comprises elemental optics array with 54(H) x 59(V) elements, a CCD camera to capture a group of elemental images created by the lens array, and a liquid crystal panel to display these images. The experiment results confirm that the system produces orthoscopic images in real time and so is effective for real-time application of the II method.

Time is the best important resource in real-time video image processing system. Only few structure or algorithm can be used in real-time video image processing system because of time limitation. Establish a high speed data-access is a way to this problem. It transfer the video signal between output ports and kernel processor as less time delay. The mode, look-up-table, other analysis, adjustment or calculation rule will be applied first by medium or low data-access. We can choose different data-access according to data relation level. Data which has closer correlation will be treated through low-speed-access, which has loser correlation will be treated through medium-speed-access. Theory analysis and experiment showed that it is a high effective structure for real-time video image processing. The feature of video image and real-time processing system is presented in this paper. The basic theory of data-access is introduced also. The status and relation of high, medium, low data-access is discussed and the rules of data distribution in different access is analyzed. Some examples with datas-accesses are given in the paper (TV tracking and measuring, laser signal alarming system).

The first quarter of 2007 started as an active season for the real-time image processing community. The annual SPIE Conference on Real-Time Image Processing, which was held in San Jose, CA, highlighted recent developments in this growing branch of image processing. As one of the few regular events with its focus matching that of JRTIP, one witnessed the growth in applications of real-time image and video processing. The conference served as an effective instrument to foster the dissemination of recent results and to leverage the cohesion of the real time image processing community. The preparations for the next SPIE Conference on Real-Time Image Processing in 2008 are already progressing and its call-for-papers is included at the end of this issue. In 2007, JRTIP is going to seek involvement of more scientists in the field of real-time image and video processing. After 1 year of operation, the editors-in-chief have started to recruit additional associate editors to enlarge the expertise base of the editorial board, to solicit manuscript submissions from potential authors, and to make the reviewing process more efficient. Interested colleagues should contact the editors-in-chief expressing their Associate Editor candidature in support of one or more fields within the focus of JRTIP. In addition, they wish to encourage researchers to act as guest editors for special issues on any real-time image or video processing related topics of interest to them. It is possible to schedule two special issues in 2007, provided that a draft call-for-papers is sent to the editors-in-chief as soon as possible. The third issue of JRTIP presents four articles, two are survey papers and two are original research papers. These articles address different real-time image processing perspectives ranging from algorithmic to hardware issues. The first survey paper deals with the real-time filtering in 2D and 3D using ordered statistics that are supported by a rich set of experimental results derived from several algorithmic variations. Performance measurements are used to demonstrate the effect of noise suppression and preservation of fine image structures and to show the DSP implementation of denoising filters. The second survey paper introduces a non-invasive analysis of dynamic processes in living cells using microscopic computer vision techniques for understanding the interactions between cellular and molecular functional units and for providing novel insights into the cell biology. The third article is an original research paper covering a machine vision application involving vision-based position sensing for aerial refueling of unmanned aerial vehicles in real-time. A full image processing chain is investigated and tested in a MATLAB/Simulink soft real-time environment as well as in a Linux/RTAI hard real-time environment. The feasibility of the real-time implementation is shown using off-the-shelf commercial hardware. The fourth article is another original research paper describing a hardware architecture for real-time fractal coding for image compression. The introduced parallel hardware architecture contains units for domain M. F. Carlsohn (&) Ingenieurberatung fur Computer Vision und Bildkommunikation, Am Heiddamm 36g, 28355 Bremen, Germany e-mail: matthias.carlsohn@t-online.de

Ultrasound image fusion for external beam radiotherapy for prostate cancer

The mechanism for motion detection in a fly's vision system, known as the Reichardt correlator, suffers from a main shortcoming as a velocity estimator: low accuracy. To enable accurate velocity estimation, responses of the Reichardt correlator to image sequences are analyzed in this paper. An elaborated model with additional preprocessing modules is proposed. The relative error of velocity estimation is significantly reduced by establishing a real-time response-velocity lookup table based on the power spectrum analysis of the input signal. By exploiting the improved velocity estimation accuracy and the simple structure of the Reichardt correlator, a high-speed vision system of 1 kHz is designed and applied for robot yaw-angle control in real-time experiments. The experimental results demonstrate the potential and feasibility of applying insect-inspired motion detection to robot control.

In recent years, high-speed vision systems are increasingly used for robot control. However, robot systems developed to date have not sufficiently utilized rapid responses of high-speed vision due to poor performance of control methods. In order to realize high-speed robot operation, it is necessary to design a control method to use the maximum torque generated by the robot. In this paper, we propose a control method integrating high speed visual processing and real time optimum control. This control method is based on nonlinear model predictive control (NMPC), and optimizes the response of the position of the robot arm so as to accurately follow the target trajectory. In this paper, we describe system design to integrate high speed visual processing and NMPC, taking high-speed operation of air hockey robot as an example. Finally, we present experimental data and verify the effectiveness of the system.

The main advantage of confocal microscopes over their conventional counterparts arises from their ability to optically 'section' nearly transparent materials; the thin image slices thus obtained can be used to reconstruct three-dimensional images, a capability which is particularly useful for the study of biological specimens. Confocal microscopes have previously used either a single laser-illuminated point-source and single point-detector (which are scanned in tandem across the object) or white-light illumination with multiple point-sources and detectors. Single-point-source systems, however, do not usually form images in real time and are restricted to using available laser wavelengths. Multiple-point-source systems, on the other hand, produce images in real time but use light very inefficiently--typically 1% or less is used for imaging. Here we demonstrate a white-light, multiple-point-source method which can in principle produce images in real time with light efficiencies as high as 50%. This system is likely to find broad practical application, particularly in the imaging of weakly reflecting or weakly fluorescent specimens.

Uncertainty propagation: Avoiding the expensive sampling process for real-time image-based measurements

We are studying electronic holography and have developed a real-time color holography system for live scene which includes three functional blocks, capture block, processing block, and display block. In this paper, we will introduce our developed system after describing basic idea that quickly calculates hologram from IP image. The first block, capture block, uses integral photography (IP) technology to capture color 3-D objects under natural light in real time. The second block, processing block, consists of four general personal computers to generate holograms from IP images in real time. Three half-zone-plated holograms for red, green and blue (RGB) channels are generated for all captured IP images by using fast Fourier Transform. The last block, display block, mainly consists of three liquid crystal displays to display the holograms and three laser sources for RGB to reconstruct the color 3-D objects. All blocks work in real time, i.e., in 30 color frames per second.

Interest in 3D medical imaging continues to increase. However, in ultrasound, real-time imaging is an indispensable strength; and real-time 3D ultrasonic imaging is not practical when conventional steered, focused beam techniques are used. This is because the speed of sound severely limits the size of the volume that can be imaged in real time. For real-time 3D imaging, approaches like simultaneous multiple beams or holography have been considered but never commercially implemented for, in part, the following reasons: A new 3D ultrasound technology should provide the convenience of a hand-held scan head, should yield real-time 3D images, and should provide 2D images with quality equal to, or greater than, presently available 2D ultrasound images. Convenient size and a reasonable price are also requirements. In this paper, a 3D ultrasonic imaging method with the potential to meet the above criteria is described. It may also provide even higher quality 2D ultrasound images than are presently available. The new method relates more closely to computed tomography than to focused steered beams. It, however, uses projections and back-projections over 3D ellipsoids rather than straight lines; and it does this in a simple straight forward manner. Implementation in software of filtered ellipsoidal back-projection is described, resolution and side lobes are discussed, and examples of the 3D point image (re. point spread function) are given.

Laser aided deposition is a material additive based manufacturing process by metallurgically bonding the deposited material to the substrate. For a precise study of the process, a mathematical model was established to investigate thermal and mass transportation phenomena, which includes substrate melting and solidification as well as the powder heating process. The detailed process together with the temperature distribution is simulated. The simulation results of the melt pool length, width and peak temperature are validated for the laser deposition process through the measurements on the melt pool geometry and surface peak temperature by the CMOS camera and the dual-wavelength temperature sensor. A smart embedded vision system using CMOS camera is used to measure the size of the melt pool. The length and width of the melt pool are measured in real time using CMOS camera with an on-board DSP processing system which processes the images in real time and provides the output through a serial port to a real-time controller. A dual wavelength non-contact temperature sensor is used for measuring the temperature of the melt pool in real time. The comparison of the melt pool length and peak temperature was conducted between simulation and experiment and shows a good agreement.Laser aided deposition is a material additive based manufacturing process by metallurgically bonding the deposited material to the substrate. For a precise study of the process, a mathematical model was established to investigate thermal and mass transportation phenomena, which includes substrate melting and solidification as well as the powder heating process. The detailed process together with the temperature distribution is simulated. The simulation results of the melt pool length, width and peak temperature are validated for the laser deposition process through the measurements on the melt pool geometry and surface peak temperature by the CMOS camera and the dual-wavelength temperature sensor. A smart embedded vision system using CMOS camera is used to measure the size of the melt pool. The length and width of the melt pool are measured in real time using CMOS camera with an on-board DSP processing system which processes the images in real time and provides the output through a serial port to a real-t...