Measuring the Angular Velocity of a Propeller with Video Camera Using Electronic Rolling Shutter

Due to the electronic rolling shutter, high-speed Complementary Metal-Oxide Semiconductor (CMOS) aerial cameras are generally subject to geometric distortions, which cannot be perfectly corrected by conventional vision-based algorithms. In this paper we propose a novel approach to address the problem of rolling shutter distortion in aerial imaging. A mathematical model is established by the coordinate transformation method. It can directly calculate the pixel distortion when an aerial camera is imaging at arbitrary gesture angles. Then all pixel distortions form a distortion map over the whole CMOS array and the map is exploited in the image rectification process incorporating reverse projection. The error analysis indicates that within the margin of measuring errors, the final calculation error of our model is less than 1/2 pixel. The experimental results show that our approach yields good rectification performance in a series of images with different distortions. We demonstrate that our method outperforms other vision-based algorithms in terms of the computational complexity, which makes it more suitable for aerial real-time imaging.

The star tracker is widely used in attitude control systems of spacecraft for attitude measurement. The attitude update rate of a star tracker is important to guarantee the attitude control performance. In this paper, we propose a novel approach to improve the attitude update rate of a star tracker. The electronic Rolling Shutter (RS) imaging mode of the complementary metal-oxide semiconductor (CMOS) image sensor in the star tracker is applied to acquire star images in which the star spots are exposed with row-to-row time offsets, thereby reflecting the rotation of star tracker at different times. The attitude estimation method with a single star spot is developed to realize the multiple attitude updates by a star image, so as to reach a high update rate. The simulation and experiment are performed to verify the proposed approaches. The test results demonstrate that the proposed approach is effective and the attitude update rate of a star tracker is increased significantly.

We demonstrate a system for capturing multi-thousand frame-per-second (fps) video using a dense array of cheap 30 fps CMOS image sensors. A benefit of using a camera array to capture high-speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. This allows us to record unpredictable events, in which nothing occurs before the event of interest that could be used to trigger the beginning of recording. Synthesizing one high-speed video sequence using images from an array of cameras requires methods to calibrate and correct those cameras' varying radiometric and geometric properties. We assume that our scene is either relatively planar or is very far away from the camera and that the images can therefore be aligned using projective transforms. We analyze the errors from this assumption and present methods to make them less visually objectionable. We also present a new method to automatically color match our sensors. Finally, we demonstrate how to compensate for spatial and temporal distortions caused by the electronic rolling shutter, a common feature of low-end CMOS sensors.

Artefacts that are present in images taken from a moving rolling shutter camera degrade the accuracy of absolute pose estimation. To alleviate this problem, we introduce an addition linear velocity in the camera projection matrix to approximate the motion of the rolling shutter camera. In particular, we derive a minimal solution using the Grobner Basis that solves for the absolute pose as well as the motion of a rolling shutter camera. We show that the minimal problem requires 5-point correspondences and gives up to 8 real solutions. We also show that our formulation can be extended to use more than 5-point correspondences. We use RANSAC to robustly get all the inliers. In the final step, we relax the linear velocity assumption and do a non-linear refinement on the fuli motion, i.e. linear and angular velocities, and pose of the rolling shutter camera with all the inliers. We verify the feasibility and accuracy of our algorithm with both simulated and real-world datasets.

Star identification robust to angular rates and false objects with rolling shutter compensation

In this work, we present a novel camera array that exploits the electronic rolling shutter to achieve high speed compressive temporal imaging. Traditional compressive temporal imaging makes use of mechanical coded apertures, despite implementation and calibration challenges. Instead, we propose to model the inherent spatial and temporal coding provided by the distinctive rolling shutter sampling from each camera of the array as a compressive temporal imaging system matrix. Thus, we can recover a high speed video from a set of snapshots from the camera array by using compressive sensing reconstruction algorithms. We present both simulation and experimental results for a 4-camera array system with different orientation angles, reconstructing up to 56 high-speed sub-frames from a set of simultaneously triggered snapshots from the array, achieving a compression rate of up to 14X.

Structured light range sensors (SLRS) like the Microsoft Kinect have electronic rolling shutters (ERS). The output of such a sensor while in motion is subject to significant motion blur (MB) and rolling shutter (RS) distortion. Most robotic literature still does not explicitly model this distortion, resulting in inaccurate camera motion estimation. In RGBD cameras, we show via experimentation that the distortion undergone by depth images is different from that of color images and provide a mathematical model for it. We propose an algorithm that rectifies for these RS and MB distortions. To assess the performance of the algorithm we conduct an extensive set of experiments for each step of the pipeline. We assess the performance of our algorithm by comparing the performance of the rectified images on scene-flow and camera pose estimation, and show that with our proposed rectification, the performance improvement is significant.

Experimental study of oblique impacts with initial spin

In the camera shooting process, video jitter caused by camera motion results to instability of the image information and the fuzzy which is not conducive for people to observe, and reduces the compression ratio. To solve this problem, we propose an electronic image stabilization algorithm using 3D rotational motion model. 3D camera motion is calculated from image sequence. The image can be mapped on the virtual sphere where the optical center of the camera is in the sphere center. We match the feature points using optical flow method, and then calculate rotation matrix using Spherical SFM algorithm between previous and current frame image. The rotation matrix contains the rotation and translation of the camera. The motion trajectory of the camera can be expressed by the 3D rotation matrix sequence. We adopt the manifold structure of the rotation matrices sequence, and we smooth the 3D rotation matrix sequences based on geodesic distance on a non-linear manifold. We formulate the video stabilization problem as objective function of smoothed rotation, and use steepest gradient descent and Armijo rules to estimate the smoothed rotation. Finally, the current frame is compensated by calculating the difference between the smoothed path and the original path to suppress the influence of the camera translation on the obtained rotation matrix. Experimental results demonstrated that the proposed algorithm does can effectively reduce random jitter without special hardware, which achieved the effect of image stabilization. Compared to the image stabilization algorithm based on 2D model, the proposed image stabilization algorithm greatly reduced the jitter and image distortion.

PURPOSE: To compare the youth pitching biomechanics with that of elite adult pitchers and to identify differences that might explain common injuries seen in youth pitcher populations. METHODS: Twenty-five professional and 14 youth baseball pitchers were filmed at 120 Hz using two high-speed video cameras. The locations of 21 body landmarks were digitized and their three-dimensional locations were calculated using the DLT method (Abdel-Aziz & Karara, 1971). Three-dimensional position data were filtered using a fourth order Butterworth filter with a cutoff frequency of 13 Hz. From the marker coordinates, the kinematics of the pitching elbow and shoulder were calculated using a standard technique (Fleisig et al., 1996; Werner et al., 1993). Kinematic data were normalized in time to facilitate comparisons and so group mean values could be calculated. RESULTS: The peak shoulder internal rotation and elbow extension angular velocities of the youth pitchers were greater than those of the professional pitchers. Peak shoulder external rotation in youth pitchers was 166 +/− 9 degrees, compared with 182 +/− 13 degrees in professionals (p<0.001). The most striking difference between the groups was in trunk motion. The peak rotational velocities of both the upper trunk and pelvis were greater in youth baseball pitchers than in professional pitchers. Peak upper trunk rotational velocity was 2102+/− 324 deg/s in youth pitchers and 1193 +/− 176 deg/s in professional pitchers (p<0.001). CONCLUSIONS: The data support the observation that youth baseball pitchers control their trunk motion in a less efficient way than elite pitchers (Ireland & Hutchinson, 1995). Peak pelvis rotation velocity occurred near the time of stride foot contact in the professionals, while it occurred much later in the youth pitchers. Proper timing of pelvis and upper trunk rotation are necessary to effectively transfer energy from the trunk to the throwing arm (Fleisig et al., 1996). Therefore, increased trunk rotational velocity may be a compensation for improper timing of segment rotations or insufficient muscle strength in youth pitchers. Improper energy transfer from the trunk to the upper extremity may lead to the increased shoulder internal rotation and elbow extension velocities in youth pitchers compared to the professionals.

This paper proposes a fully automated post image processing scheme based on facial feature detection to correct the horizontal temporal shear or rolling shutter distortion. This distortion occurs when obtaining images or video sequences from a CMOS camera with a rolling shutter whenever there is relative horizontal movement between the sensor and the object being imaged during the integration time of the image frame. Unlike CCD sensors, such as the interline CCD, which provides an electronic shutter mechanism called a global shutter in which the light collection starts and ends at exactly the same time for all pixels, CMOS sensors can not hold and store all the pixels at the same time. Each scanline is exposed, sampled, and stored in sequence, resulting in the rolling shutter effect or temporal distortion of the image that will cause inaccurate facial recognition results. Facial feature detection is performed using correlation based methods with low computational complexity. The location of key facial feature points is then used to calculate the temporal horizontal shear or the distortion of the image. This information can then be used to remove the temporal horizontal shear distortion from the detected face or the entire image. We present experimental results on controlled data sets and real scenes to show that the proposed method yields excellent results in reversing the temporal horizontal shear caused by the CMOS rolling shutter sensor and significantly improves the accuracy of our facial recognition algorithm.

For the Inverse Synthetic Aperture Radar (ISAR) imaging, the ISAR image obtained by the Range-Doppler (RD) or time-frequency analysis methods can not display the target's real shape due to its azimuth relating to the target Doppler frequency, thus the cross-range scaling is required for ISAR image. In this paper, a fast cross-range scaling method for ISAR is proposed to estimate the Rotational Angular Velocity (RAV). Firstly, the proposed method utilizes efficient Pseudo Polar Fast Fourier Transform (PPFFT) to transform the rotational motion of two ISAR images from two different instant time into translation in the polar angle direction. Then, a new cost function called integrated correction is defined to obtain the RAV coarse estimation. Finally, the optimal RAV can be estimated using the Bisection method to realize the cross-range scaling. Compared with the available algorithms, the proposed method avoids the problems of precision loss and high computational complexity caused by interpolation operation. The results of computer simulation and real data experiments are provided to demonstrate the validity of the proposed method.

This paper addresses the problem of estimating, analyzing and tracking objects moving with spatio-temporal rotational motion (spin or orbit). It is assumed that the digital signals of interest are acquired from a camera and structured as digital image sequences. The trajectories in the signal are two-dimensional spatial projections in time of motion taking place in a three-dimensional space. The purpose of this work is to focus on the rotational motion, i.e. estimate the angular velocity. In natural scenes, rotational motion usually composes with translational or accelerated motion on a trajectory. This paper shows that trajectory parameters and rotational motion can be efficiently estimated and tracked either simultaneously or separately. The final goal of this work is to provide selective reconstructions of moving objects of interest. This paper constructs new continuous wavelet transforms that can be tuned to both translational and rotational motion. The parameters of analysis that are taken into account in these rotational wavelet transforms are space and time position, velocity, spatial scale, angular orientation and angular velocity. The continuous wavelet functions are finally discretized for signal processing. The link between rotational motion, symmetry and critical sampling is also presented. Applications are presented with tracking and estimation.

To remove the vibration in video content, a video stabilization approach is proposed in this paper. Video stabilization is completed by local rotational motion model in our approach. The local rotational motion is derived from rotation-based block matching method. The global rotational motion is identified by a local rotational motion model. Particularly, global rotational motion is derived by computing the variance of local rotational motion and its direction. Finally, the global rotational motion is smoothed by calculating the motion of neighboring frames. The experimental results demonstrate that the proposed video stabilization approach can produce well smooth videos with smooth rotational motion in various videos. The comparison also shows our superior performance on video stabilization.

Rotating machinery speed extraction through smartphone video acquisition from a radial viewpoint