1A1-A11 二眼駆動鏡面式視線制御による高速ステレオビジョンシステム(3次元計測/センサフュージョン(1))

The mechanism for motion detection in a fly's vision system, known as the Reichardt correlator, suffers from a main shortcoming as a velocity estimator: low accuracy. To enable accurate velocity estimation, responses of the Reichardt correlator to image sequences are analyzed in this paper. An elaborated model with additional preprocessing modules is proposed. The relative error of velocity estimation is significantly reduced by establishing a real-time response-velocity lookup table based on the power spectrum analysis of the input signal. By exploiting the improved velocity estimation accuracy and the simple structure of the Reichardt correlator, a high-speed vision system of 1 kHz is designed and applied for robot yaw-angle control in real-time experiments. The experimental results demonstrate the potential and feasibility of applying insect-inspired motion detection to robot control.

A high-speed vision system can be applied to color-histogram-based tracking at 2000 fps by hardware-implementing an improved CamShift algorithm. In the improved CamShift algorithm, the size, position, and orientation of a color-patterned object to be tracked in an image can be simultaneously extracted using only the hardware implementation of a color-histogram circuit module for calculating moment features of binary images quantized by 16 hue-based color bins. By hardware-implementing color-histogram circuit modules on a high-speed vision platform, IDP Express, the improved CamShift algorithm enables color-histogram-based tracking at 2000 fps for 512×511 pixel images in real-time. By installing our tracking system on a two-axis mechanical active vision system, we demonstrate the effectiveness of 2000 fps color-histogram-based tracking by performing several experiments of color-patterned objects, which are always tracked in the camera views even when they move rapidly under complicated backgrounds.

We developed a super-low-latency telemanipulation system using a high-speed vision system and a high-speed robot hand with a high-speed tactile sensor. This system does not require users to wear sensors. Also, since it has latency lower than the sampling rate of visual recognition by humans, the latency is not recognizable by humans. Low-latency telemanipulation systems are needed to perform tasks that require high speed, such as catching falling objects or fast-moving objects. We evaluated the latency of our telemanipulation system and successfully demonstrated catching of falling objects using our system, in contrast to a conventional vision system operating at 30 fps, which failed the task.

In recent years, high-speed vision systems are increasingly used for robot control. However, robot systems developed to date have not sufficiently utilized rapid responses of high-speed vision due to poor performance of control methods. In order to realize high-speed robot operation, it is necessary to design a control method to use the maximum torque generated by the robot. In this paper, we propose a control method integrating high speed visual processing and real time optimum control. This control method is based on nonlinear model predictive control (NMPC), and optimizes the response of the position of the robot arm so as to accurately follow the target trajectory. In this paper, we describe system design to integrate high speed visual processing and NMPC, taking high-speed operation of air hockey robot as an example. Finally, we present experimental data and verify the effectiveness of the system.

In order to achieve faster and more dexterous manipulations, we propose a strategy called holding. In the dynamic holding condition, the object is held in a stable condition while moving at high-speed. In this paper, we first explain the concept of dynamic holding, then we describe an experiment of dribbling by a robot as an example of dynamic holding using a high-speed multifingered hand and a high-speed vision system.

Stereo Measurement System for High-speed Motion and Deformation of Objects Using Two Gaze Controllers by Rotational Mirrors

This paper introduced a high-speed vision platform, JDP Express, which can record image sequences of 512×512 pixel at 2000fps as well as real-time image processing. IDP Express was also applied to a high-speed active vision system for target tracking to capture image sequences at high resolution, which were magnified by tracking a target in an image. Several experiments were actually executed for automatic high-frame-rate video recording by using image-based trigger, and the videos captured at a high frame rate were also shown to verify its effectiveness.

There is a need for compact, high-speed, and low-power vision systems for enabling real-time mobile autonomous applications. The best approach to achieve this is to implement the bulk of the application in hardware. Reconfigurable hardware meet these requirements without the limitation of fixed functionality that accompanies application-specific circuits. Resource constraints of reconfigurable hardware calls for optimized implementations i terms of resource usage with maintained performance. The research group in Robotics at Malardalen University is moving toward the completion of a reconfigurable hardware-platform for stereo vision, coupled with a compact embedded computer. This system will incorporate hardware-based preprocessing components enabling visual perception for autonomous machines. This thesis covers the reconfigurable hardware section of the vision system concerning the realization of scene depth extraction. It shows the advantages of image preprocessing in hardware and propose a resource optimized approach to stereo matching. The work quantifies the impact of reduced resource utilization and a desire for increased accuracy in disparity estimation. The implemented stereo matching approach performs on par with recent similar implementations in terms of accuracy, but excels in terms of resource utilization and resource sharing, as the external memory requirement is removed for larger images. Future work aims to further include processes for navigation, and structure and object recognition. Furthermore, the system will be adapted to real world scenarios, both indoors and outdoors.

In recent years, the need for computer vision systems is increasing in various fields, such as security and visual inspection. It is crucial there to realize simple and high-speed practical vision systems. The present paper addresses the author's theoretical research and its applications developed thus far in working toward this goal. First, the problem of the conventional approach is pointed out, and the general framework of pattern recognition, in particular the feature extraction theory, is referred to as the theoretical foundation. Next, a scheme of adaptive vision system with learning capability is presented, which comprises two stages of feature extraction, namely, Higher-order Local Auto-Correlation and multivariate data analysis. Several applications are demonstrated, showing the flexible and effective performance of the proposed scheme.

Experimental analysis on relationship between roughness parameters and texture features of 6082T6 sandblasted components

In this study, we develop a real-time high-speed color vision system to track multi-colored pie-shaped markers in 512 × 512 images at 2000 fps. Our color vision system has six cell-based labeling circuits that can calculate the 0th and 1st moment features of color-labeled regions during a single scan of a 512 × 512 image divided into 4096 8 × 8 cells. Our system examines the geometric distribution of five color regions in each pie-shaped marker using their 0th and 1st moment features, and it can simultaneously extract the unique marker codes, locations, and orientations of multiple pie-shaped markers in a 512 × 512 image at 2000 fps. To evaluate the performance of our system, we performed several experiments to track multi-colored pie-shaped markers printed on high-speed rotating objects.

Flies achieve supreme flight maneuverability through a small set of miniscule steering muscles attached to the wing base. The fast flight maneuvers arise from precisely timed activation of the steering muscles and the resulting subtle modulation of the wing stroke. In addition, slower modulation of wing kinematics arises from changes in the activity of indirect flight muscles in the thorax. We investigated if these modulations can be described as a superposition of a limited number of elementary deformations of the wing stroke that are under independent physiological control. Using a high-speed computer vision system, we recorded the wing motion of tethered flying fruit flies for up to 12 000 consecutive wing strokes at a sampling rate of 6250 Hz. We then decomposed the joint motion pattern of both wings into components that had the minimal mutual information (a measure of statistical dependence). In 100 flight segments measured from 10 individual flies, we identified 7 distinct types of frequently occurring least-dependent components, each defining a kinematic pattern (a specific deformation of the wing stroke and the sequence of its activation from cycle to cycle). Two of these stroke deformations can be associated with the control of yaw torque and total flight force, respectively. A third deformation involves a change in the downstroke-to-upstroke duration ratio, which is expected to alter the pitch torque. A fourth kinematic pattern consists in the alteration of stroke amplitude with a period of 2 wingbeat cycles, extending for dozens of cycles. Our analysis indicates that these four elementary kinematic patterns can be activated mutually independently, and occur both in isolation and in linear superposition. The results strengthen the available evidence for independent control of yaw torque, pitch torque, and total flight force. Our computational method facilitates systematic identification of novel patterns in large kinematic datasets.

This paper deals with deflection sensing for robotic control via high speed vision system. Marker is attached to the end effector of the 3-DoF robot arm and the robot is moved to satisfy a line trajectory of the end effector. The motion of the marker attached to the end effector is recorded using the high speed vision system and actual position of the end effector is calculated via image processing of recorded high-frame rate images. We found maximum 1.8 mm position error from the image processing results. We verify that deflection and vibration can be measured by comparing the actual position obtained from image and the ideal position of trajectory planning.

This study proposes a real-time video shooting system using a view-expanded microscope. The proposed system composed of a microscope, a galvanometer mirror and a high-speed vision system is capable of obtaining a 16,000×2,000 pixel microscopic image in real time at 25 fps by combining multiple visual fields. The effectiveness of our system is demonstrated through video shooting experiments of moving sea urchin embryos in a microchannel.

In this paper, using a high-speed vision system, we propose high-speed autofocus methods to acquire clear images of vibrating objects in the focal direction, in order to realize in-line precision visual inspection. These methods include a single-process method with a single camera, and a multi-process method with a laser displacement meter, to estimate focal lengths. We evaluated the high-speed autofocus performance for vibrating objects in the focal direction using a vibration tester. In addition, since the vibration is not only in the focal direction but generally in three dimensions, we established a method to track this three-dimensional high-speed vibrating object and acquire clear images at high speed. Compared to the ideal autofocus performance defined in this study, high-quality results of 110% for simple harmonic motion and 146% for rotational motion were obtained, proving that the system is highly practical.