Video Stabilization for Fast Moving Camera Based on Feature Point Classification

This paper describes a new method for the moving object detection using the IMU sensor and instance image segmentation. In the proposed method, the feature points are extracted by the detector, and the initial fundamental matrix is calculated from the IMU data. Next, the epipolar line is used to classify the extracted feature points. From the background feature point matching, fundamental matrix is calculated iteratively to minimize the error of classification. After the feature point classification, image segmentation is used to enhance the quality of the classification result. The proposed method is implemented and tested with real-world driving videos, and compared with the previous works.

Through automated agricultural inspection, farmers can potentially achieve better productivity and accurately predict yields and crop quality. A variety of sensors can be used for agricultural inspection, but the cheapest and most information-rich is the video camera. We collect data in the field from a monocular camera fixed to a mobile inspection platform. For purposes of pineapple crop mapping and yield prediction, we propose an image processing framework for in-field fruit detection, tracking, and 3D reconstruction. We perform a series of experiments on feature point extraction using Harris, SIFT, and SURF features, feature point description using SIFT and SURF descriptors, feature point classification using SVMs, fruit region tracking using blob tracking, and 3D reconstruction using structure from motion and robust ellipsoid estimation techniques. We find that SURF feature points and descriptors provide the best tradeoff between processing time and classification accuracy and that the method is sufficiently accurate for fruit region detection. Our preliminary results for fruit region tracking and 3D fruit reconstruction are promising. We plan further work towards development of a useful aid to help farmers manage their farms.

For the extraction of paddy rice seedling centerline, this study proposed a method based on Fast-SCNN (Fast Segmentation Convolutional Neural Network) semantic segmentation network. By training the FAST-SCNN network, the optimal model was selected to separate the seedling from the picture. Feature points were extracted using the FAST (Features from Accelerated Segment Test) corner detection algorithm after the pre-processing of original images. All the outer contours of the segmentation results were extracted, and feature point classification was carried out based on the extracted outer contour. For each class of points, Hough transformation based on known points was used to fit the seedling row centerline. It has been verified by experiments that this algorithm has high robustness in each period within three weeks after transplanting. In a 1280×1024-pixel PNG format color image, the accuracy of this algorithm is 95.9% and the average time of each frame is 158ms, which meets the real-time requirement of visual navigation in paddy field.

In order to improve the accuracy and rapidity of feature line extraction from point clouds, the work proposed a feature line extraction method based on geometric structure of point space. Firstly, a spatial grid dynamic division method is designed to locate the feature region of the model. A new feature points detection operator based on the linear intercept ratio is proposed according to the geometric information of points. Then, the feature points are refined by the Laplacian operator. Finally, the refined feature points are connected into the characteristic curve by the improved method of polyline growth. Compared with the feature points detection method based on surface variation (MSSV) or the angle of normal vector (SM-PD), the proposed method has low rate of error recognition with the increased noise intensity. Meanwhile, the computation time is 224.42 ms for the standard Armadillo model, less than 530.23 ms of the MSSV and 350.75 ms of the SM-PD. The experimental results show that the proposed method can accurately extract the feature points, with good noise immunity, especially suitable for the massive point cloud model.

To achieve a higher compression ratio for very low bit rate (VLBR) video compression, motion estimation has to be more reliable and easier to code. Many segmentation based techniques try to approximate several motion vectors by a single global motion vector so that fewer vectors are required. However, a global motion vector may not be a good representation for the original vectors and the bit saving is not worth the loss. This paper proposes a new scheme for motion vector representation. Many candidates from full search block matching are selected for merging possibility. Blocks which experience similar displacement are combined to form a single object. Upon compensation for object translation or global compensation, accurate motion of each block can be identified by local compensation within an object. Instead of presenting object movement as a vector, a specific index is assigned to reduce the overhead bits. Based on extensive simulation, the proposed method with an appropriate threshold shows an improvement over standard block based hybrid schemes.

Multi-slip surfaces searching method for earth slope with weak interlayer based on local maximum shear strain increment

Information about the emotional state of a person can be inferred from facial expressions. Emotion recognition has become an active research area in recent years in various fields such as Human Robot Interaction (HRI), medicine, intelligent vehicle, etc., The challenges in emotion recognition from images with pose variations, motivates researchers to explore further. In this paper, we have proposed a method based on geometric features, considering images of 7 yaw angles (−45°,−30°,−15°,0°,+15°,+30°,+45°) from BU3DFE database. Most of the work that has been reported considered only positive yaw angles. In this work, we have included both positive and negative yaw angles. In the proposed method, feature extraction is carried out by concatenating distance and angle vectors between the feature points, and classification is performed using neural network. The results obtained for images with pose variations are encouraging and comparable with literature where work has been performed on pitch and yaw angles. Using our proposed method non-frontal views achieve similar accuracy when compared to frontal view thus making it pose invariant. The proposed method may be implemented for pitch and yaw angles in future.

The rotation characteristics of the composition Laguerre–Gaussian (LG) beams during the propagation are numerically analyzed. A simple rotation angle measure method based on imaging technology is proposed and demonstrated. First, the global feature points of the composite vortex pattern is extracted by the 8-connected boundary tracking algorithm. Then, the position coordinates of each feature point are calculated by using the weighting method. The rotation angle of each feature point relative to the center point is calculated under the coordinate system. Finally, the relative rotation angles of the composite vortex pattern at different positions relative to the original position are obtained. The results show that the measurement error reaches 10[Formula: see text] for composite beams with different sizes and beam waist, which is comparable to that of the theoretical value. The proposed method can be used to effectively and accurately measure the Gouy phase of the composite LG beams. We expect the proposed method will be benefit of the further research and application of Gouy phase.

This paper presents a fast image sequence stabilization system that uses one bit transform (IBT) matching based global motion estimation and Kalman filtering based global motion correction. In order to facilitate a fast motion estimation process, image frames are initially converted into a single bit-plane representation using the IBT, and then motion vectors are evaluated matching the IBTs of two consecutive frames using Boolean operations. Because a binary representation is established, the computational complexity of motion estimation is significantly reduced providing a proportionate reduction in power consumption of custom hardware. The global motion vector is computed using local motion vectors of sub-image blocks to further increase the motion estimation speed. For the motion correction part, global motion vectors are accumulated to yield absolute frame displacements and Kalman filtering is employed to achieve stabilization.

In this paper a new iterative motion-based segmentation algorithm is presented for object-based video coding. At first, dense motion fields between image pairs are obtained by a multiresolution motion estimation algorithm. Based on these motion fields a set of global motion vectors are determined by an iterative vector quantisation procedure. Using the global motion vectors, the images are segmented into homogenous regions. Covered and uncovered areas of the background and objects are properly recognized by examining two consecutive motion fields. Experimental results for artificial as well for real-life video have shown that the proposed algorithm results in good quality segmentation label fields while it requires only moderate computations.

A method of feature line extraction from triangular mesh model is presented. Firstly, initial feature point is manually specified based on feature points extraction from triangular mesh model. Then begin with initial feature point, Principal Component Analysis (PCA) method is applied to analyze primary direction of special feature region of feature points. Search the feature point which is furthest from the center of mass in the primary direction and it is chosen as subsequent feature point. Iteratively, record the feature point sequence in order. Finally, use three order non-uniform B-spline curve to fit and construct feature line. The experimental results demonstrate that the proposed method in this paper is efficient.

컴퓨터비전에서 안정적으로 대응점을 획득하는 것은 매우 중요한 일이다. 그러나 이들은 스케일, 조명, 시점 등이 변하는 환경에서 정확한 대응점을 찾는 과정은 쉽지 않다. SIFT 알고리즘은 객체의 모서리나 꼭지점으로부터 추출한 특징벡터를 사용하므로 스케일링, 회전, 조명변화를 가지는 영상에서도 뛰어난 매칭을 수행한다. 그러나 SIFT는 엣지에 의해 특징점을 추출하므로 엣지가 존재하지 않는 영역에서는 원하는 대응점을 찾을 수 없다. 본 연구는 SIFT에 의한 대응 특징점 추출과 매칭 성능을 향상시키기 위한 마커 모양 및 배치 방법을 제안한다. 제안 방법에서 사용한 마커의 모양은 부착 방향에 따라 SIFT 알고리즘에 의해 한 방향으로 우세한 벡터를 검출할 수 있는 반원형(SemiCircle)으로 구성한다. 그리고 대응점 매칭의 성능을 향상시키기 위하여 마커의 방향 배치는 드 브루인 수열(De Bruijn Sequence)을 이용한다. 실험을 통해 제안한 방법이 기존의 방법보다 더 정확한 특징점 검출과 매칭에 효과적임을 증명하였다. In computer vision, it is very important to obtain reliable corresponding feature points. However, we know it is not easy to find the corresponding feature points exactly considering by scaling, lighting, viewpoints, etc. Lots of SIFT methods applies the invariant to image scale and rotation and change in illumination, which is due to the feature vector extracted from corners or edges of object. However, SIFT could not find feature points, if edges do not exist in the area when we extract feature points along edges. In this paper, we present a new placement method of marker to improve the performance of SIFT feature detection and matching between different view of an object or scene. The shape of the markers used in the proposed method is formed in a semicircle to detect dominant direction vector by SIFT algorithm depending on direction placement of marker. We applied De Bruijn sequence for the markers direction placement to improve the matching performance. The experimental results show that the proposed method is more accurate and effective comparing to the current method.

A new pattern recognition method is proposed for handwritten alphabet and digit characteristics. The feature point distribution of a standard pattern is mapped onto that of a distorted pattern, through a modified self-organizing map (SOM). The distorted pattern is recognized based on similarity between both feature point distributions. The modified SOM has the following advantages. First, the number of feature points is small, and these are classified into several groups. Second, the mapping is carried out in the variable ring shape region to find a suitable pairing of the feature points. Third, the feature points are selected from both the standard and the distorted patterns to avoid any vibration. Finally, neighborhoods are selected along lines of the patterns. These improvements can provide stable and fast feature point mapping. Computer simulations demonstrated that the proposed method can adapt to a variety of pattern distortions. >

The feature extraction of a point cloud fragment model is the basis of fragment splicing, which provides the technical support for research on the segmentation, splicing, and restoration of fragment surfaces. High-quality feature extraction, however, is a complicated process due to the diversity of the surface information of a fragment model. For this subject, a high-efficient point cloud feature extraction method was proposed to address a new method for extracting feature lines. First, the projection distance feature of the point cloud model was calculated to identify the potential feature points. Furthermore, the local information of the possible feature points was used to construct the adaptive neighborhoods for identifying the feature points based on neighborhoods of the model. The clustering fusion of the feature points was proposed according to the discrimination threshold values of the feature points. Finally, the Laplace operator was utilized to refine and connect the feature points to form smooth feature lines. The experimental results showed that the proposed method was automatic, highly efficient, and with good adaptability that could effectively extract the detailed features and construct the complete feature lines. Moreover, results showed that the provided framework could extract the features of simple structure models and be feasible to a certain extent for fragment models with abundant features.

Feature matching is a core step in multi-source remote sensing image registration approaches based on feature. However, for existing methods, whether traditional classical SIFT algorithm or deep learning-based methods, they essentially rely on generating descriptors from local regions of feature points, which can lead to low matching success rates due to various challenges, including gray-scale changes, content changes, local similarity, and occlusions between images. Inspired by the human approach of finding rough corresponding regions globally and then carefully comparing local regions, and the excellent global attention property of transformers, the proposed feature matching network adopts a coarse-to-fine matching strategy that utilizes both global and local information between images to predict corresponding feature points. Importantly, the network has great flexibility of matching corresponding points for any feature points and can be effectively trained without strong supervised signals of corresponding feature points and only require the true geometric transformation between images. The qualitative experiment illustrate the effectiveness of the proposed network by matching feature points extracted by SIFT or sampled uniformly. In the quantitative experiments, we used feature points extracted by SIFT, SuperPoint, and LoFTR as the keypoints to be matched. We then calculated the mean match success ratio (MSR) and mean reprojection error (MRE) of each method at different thresholds in the test dataset. Additionally, boxplot graphs were plotted to visualize the distributions. By comparing the MSR and MRE values as well as their distributions with other methods, we can conclude that the proposed method consistently outperforms the comparison methods in terms of MSR at different thresholds. Moreover, the MSR of the proposed method remains within a reasonable range compared to the MRE of other methods.