Fast Tracking System for Multi-colored Pie-shaped Markers

The mechanism for motion detection in a fly's vision system, known as the Reichardt correlator, suffers from a main shortcoming as a velocity estimator: low accuracy. To enable accurate velocity estimation, responses of the Reichardt correlator to image sequences are analyzed in this paper. An elaborated model with additional preprocessing modules is proposed. The relative error of velocity estimation is significantly reduced by establishing a real-time response-velocity lookup table based on the power spectrum analysis of the input signal. By exploiting the improved velocity estimation accuracy and the simple structure of the Reichardt correlator, a high-speed vision system of 1 kHz is designed and applied for robot yaw-angle control in real-time experiments. The experimental results demonstrate the potential and feasibility of applying insect-inspired motion detection to robot control.

This paper proposes a camera system that can realize high-brightness video recording of fast-moving objects without motion blurs. The objective of this paper is to enhance the motion-blur-free camera system based on intermittent frame-by-frame tracking which was previously proposed by our research group. The proposed camera system consists of a real-time high-speed vision system and a high-speed 2-DOF piezo-driven mirror. The performance of the developed motion-blur free camera system is verified through video recording experiments on fast-moving objects without decreasing the exposure time of the vision system.

A high-speed vision system can be applied to color-histogram-based tracking at 2000 fps by hardware-implementing an improved CamShift algorithm. In the improved CamShift algorithm, the size, position, and orientation of a color-patterned object to be tracked in an image can be simultaneously extracted using only the hardware implementation of a color-histogram circuit module for calculating moment features of binary images quantized by 16 hue-based color bins. By hardware-implementing color-histogram circuit modules on a high-speed vision platform, IDP Express, the improved CamShift algorithm enables color-histogram-based tracking at 2000 fps for 512×511 pixel images in real-time. By installing our tracking system on a two-axis mechanical active vision system, we demonstrate the effectiveness of 2000 fps color-histogram-based tracking by performing several experiments of color-patterned objects, which are always tracked in the camera views even when they move rapidly under complicated backgrounds.

In recent years, high-speed vision systems are increasingly used for robot control. However, robot systems developed to date have not sufficiently utilized rapid responses of high-speed vision due to poor performance of control methods. In order to realize high-speed robot operation, it is necessary to design a control method to use the maximum torque generated by the robot. In this paper, we propose a control method integrating high speed visual processing and real time optimum control. This control method is based on nonlinear model predictive control (NMPC), and optimizes the response of the position of the robot arm so as to accurately follow the target trajectory. In this paper, we describe system design to integrate high speed visual processing and NMPC, taking high-speed operation of air hockey robot as an example. Finally, we present experimental data and verify the effectiveness of the system.

We developed a super-low-latency telemanipulation system using a high-speed vision system and a high-speed robot hand with a high-speed tactile sensor. This system does not require users to wear sensors. Also, since it has latency lower than the sampling rate of visual recognition by humans, the latency is not recognizable by humans. Low-latency telemanipulation systems are needed to perform tasks that require high speed, such as catching falling objects or fast-moving objects. We evaluated the latency of our telemanipulation system and successfully demonstrated catching of falling objects using our system, in contrast to a conventional vision system operating at 30 fps, which failed the task.

In order to achieve faster and more dexterous manipulations, we propose a strategy called holding. In the dynamic holding condition, the object is held in a stable condition while moving at high-speed. In this paper, we first explain the concept of dynamic holding, then we describe an experiment of dribbling by a robot as an example of dynamic holding using a high-speed multifingered hand and a high-speed vision system.

There is a need for compact, high-speed, and low-power vision systems for enabling real-time mobile autonomous applications. The best approach to achieve this is to implement the bulk of the application in hardware. Reconfigurable hardware meet these requirements without the limitation of fixed functionality that accompanies application-specific circuits. Resource constraints of reconfigurable hardware calls for optimized implementations i terms of resource usage with maintained performance. The research group in Robotics at Malardalen University is moving toward the completion of a reconfigurable hardware-platform for stereo vision, coupled with a compact embedded computer. This system will incorporate hardware-based preprocessing components enabling visual perception for autonomous machines. This thesis covers the reconfigurable hardware section of the vision system concerning the realization of scene depth extraction. It shows the advantages of image preprocessing in hardware and propose a resource optimized approach to stereo matching. The work quantifies the impact of reduced resource utilization and a desire for increased accuracy in disparity estimation. The implemented stereo matching approach performs on par with recent similar implementations in terms of accuracy, but excels in terms of resource utilization and resource sharing, as the external memory requirement is removed for larger images. Future work aims to further include processes for navigation, and structure and object recognition. Furthermore, the system will be adapted to real world scenarios, both indoors and outdoors.

The paper is devoted to the problem of formalization of software-hardware solutions in designing real-time computer vision systems. The main attention is paid to the methods of implementation of low-level operations that find features (simple elements) in the image input to the system. Algorithmic types of detectors of simple elements in images are analyzed from the point of view of hardware organization in computer vision systems. In this connection, the necessary performance and memory resources are estimated. The capabilities of parallel and pipeline execution of detector algorithms are investigated. A method of using a field programmable gate array and a digital signal processor in solving the problem of image processing in real-time computer vision systems is considered in detail.

Programming a Pipelined Image Processor

This study proposes a real-time video shooting system using a view-expanded microscope. The proposed system composed of a microscope, a galvanometer mirror and a high-speed vision system is capable of obtaining a 16,000×2,000 pixel microscopic image in real time at 25 fps by combining multiple visual fields. The effectiveness of our system is demonstrated through video shooting experiments of moving sea urchin embryos in a microchannel.

This paper introduced a high-speed vision platform, JDP Express, which can record image sequences of 512×512 pixel at 2000fps as well as real-time image processing. IDP Express was also applied to a high-speed active vision system for target tracking to capture image sequences at high resolution, which were magnified by tracking a target in an image. Several experiments were actually executed for automatic high-frame-rate video recording by using image-based trigger, and the videos captured at a high frame rate were also shown to verify its effectiveness.

Describes a parallel-processor LSI chip (the Integrated Memory Array Processor, IMAP) and a compact real-time vision system (RVS-2). The IMAP integrates 64 8-bit processors, which operate in a SIMD manner, and 2-Mbit image memory on a single chip, and has peak performance of 3.84 GIPS. The RVS-2 consists of 8 IMAPs, a video interface, a control LSI chip (the Real-time Vision System Controller, RVSC) and a host workstation. RVSC is a 16-bit processor which carries out global data operations as well as providing an instruction stream to IMAP processors. In the RVS-2 system, the IMAP processors accomplish data-parallel operations, the RVSC applies global data operations to the results, and the host workstation carries out higher-level recognition tasks using the results obtained by the IMAPs and the RVSC. The peak performance of the RVS-2 is 30 GIPS and most of the basic image processing is carried out in 0.1 to 0.7 ms, which is 50 to 300 times faster the video rate.

Experimental analysis on relationship between roughness parameters and texture features of 6082T6 sandblasted components

In recent years, the need for computer vision systems is increasing in various fields, such as security and visual inspection. It is crucial there to realize simple and high-speed practical vision systems. The present paper addresses the author's theoretical research and its applications developed thus far in working toward this goal. First, the problem of the conventional approach is pointed out, and the general framework of pattern recognition, in particular the feature extraction theory, is referred to as the theoretical foundation. Next, a scheme of adaptive vision system with learning capability is presented, which comprises two stages of feature extraction, namely, Higher-order Local Auto-Correlation and multivariate data analysis. Several applications are demonstrated, showing the flexible and effective performance of the proposed scheme.

In this paper, we propose a stereo vision system for observing continuously a quickly moving object. The system consists of two high-speed gaze controllers by rotational mirrors, and capture stereo images while gaze controllers track the object. Temporally continuous 3D shapes of the object are reconstructed from the stereo images, and the system measures high-speed motion and deformation of the object. We discuss settings of the system considering characteristics of the gaze controller by rotational mirrors. The experimental results show that the system can capture stereo images tracking a quickly and irregularly moving object, and can reconstruct temporally continuous high-resolution 3D shapes of the object.