FAOF: a feature aggregation method based on optical flow for gangue detection on production environment

Notice of Violation of IEEE Publication Principles<br><br> "Motion vector estimation of video image by Pyramidal implementation of Lucas Kanade Optical flow"<br> by Sukeshini N.Tamgade and Vibha R. Bora<br> in the Proceedings of the 2nd International Conference on Emerging Trends in Engineering and Technology (ICETET), January 2009, pp.914-917<br><br> After careful and considered review of the content and authorship of this paper by a duly constituted expert committee, this paper has been found to be in violation of IEEE's Publication Principles.<br><br> This paper duplicates content from the paper cited below. The original content was copied without attribution (including appropriate references to the original author(s) and/or paper title) and without permission.<br><br> "Pyramidal Implementation of the Affine Lucas Kanade Feature Tracker Description of the Algorithm"<br> by Jean-Yves Bouguet<br> in Intel Corporation-Microprocessor Research Labs Report, January 2001<br><br> <br/> Motion vector estimation is an important parameter for video segmentation. Effective video compression can be achieved by choosing a correct approach for the calculation of motion vector. Here in this paper we propose an optical flow motion vector estimation through iterative Lucas-Kanade Pyramidal implementation for both large & small motion In image pyramid representation a group of pixel information is gradually reduced to a value of one pixel information for both current & reference frame. The velocity & displacement at each pixel is obtained by using Lucas-Kanade equations. The original image is recovered by warping reference frame towards current frame using flow vectors i.e. velocity & displacement by using image warping techniques. The process is repeated until convergence. An iterative implementation is shown which successfully computes the optical flow for a number of synthetic image sequences.

Current Moving Object Detectors in airborne Region of Interest (ROI) coding systems for police surveillance applications used on-board of UAVs are often based on Global Motion Estimation (GME) techniques. Since in these scenarios the camera is moving, simple background removal approaches cannot be applied without a Global Motion Compensation (GMC). Common GMC algorithms assume the ground to be planar, allowing the pixels of the previous frame to be motion compensated into the current frame by applying a projective transformation. The difference image between the compensated frame and the current frame emphasis regions containing possible motion. Such moving object detectors are great in terms of run-time efficiency but are known to lack in terms of accuracy - especially for unstructured regions of moving objects - as well as the robustness against noise. Superpixel segmentation was recently proposed to overcome the issue of the imprecise region cuts given by the difference image. It provides a greatly improved true positive detection rate, but unintentionally also increases the area of false positives. This paper proposes the use of a mesh-based GME and GMC to detect the moving object regions wherein a cluster filter eliminates errors in the optical flow by assuming a smooth vector field as the global motion model. In doing so we improve the coding efficiency of the fully automatic ROI coding system by more than 24% for moving object areas conserving the detection benefits of the integration of superpixel segmentation.

Fish segmentation in underwater videos can be used to accurately determine the silhouette size of fish objects, which provides key information for fish population monitoring and fishery resources survey. Some researchers have utilized underwater optical flow to improve the fish segmentation accuracy of underwater videos. However, the underwater optical flow is not evaluated and screen in existing works, and its predictions are easily disturbed by motion of non-fish. Therefore, in this paper, by analyzing underwater optical flow data, we propose a robust underwater segmentation network, RUSNet, with adaptive screening and fusion of input information. First, to enhance the robustness of the segmentation model to low-quality optical flow inputs, a global optical flow quality evaluation module is proposed for evaluating and aligning the underwater optical flow. Second, a decoder is designed by roughly localizing the fish object and then applying the proposed multidimension attention (MDA) module to iteratively recover the rough localization map from the spatial and edge dimensions of the fish. Finally, a multioutput selective fusion method is proposed in the testing stage, in which the mean absolute error (MAE) of the prediction using a single input is compared with that obtained using multisource input. Then, the information with the highest confidence is selected for predictive fusion, which facilitates the acquisition of the ultimate underwater fish segmentation results. To verify the effectiveness of the proposed model, we trained and evaluated it using a publicly available joint underwater video dataset and a separate DeepFish public dataset. Compared with the advanced underwater fish segmentation model, the proposed model has greater robustness to low-quality background optical flow in the DeepFish dataset, with the mean pixel accuracy (mPA) and mean intersection over union (mIoU) values reaching 98.77% and 97.65%, respectively. On the joint dataset, the mPA and mIoU of the proposed model are 92.61% and 90.12%, respectively, which are 0.72% and 1.21% higher than those of the advanced underwater video object segmentation model MSGNet. The results indicate that the proposed model can adaptively select the input and accurately segment fish in complex underwater scenes, which provides an effective solution for investigating fishery resources.

This study addresses the challenges of selecting a suitable visual tracking method for real-time mobile robot applications, particularly in scenarios where the target is moving on the ground. The primary research problem addressed is the need for a flexible, computationally efficient tracking method that does not rely on pre-existing labelled datasets, as is often required by deep learning approaches. Unsupervised methods can overcome this problem by utilizing object motion information in each image frame without prior training. With many unsupervised tracking methods available, choosing an appropriate algorithm that can perform efficiently under dynamic conditions becomes a critical problem. The study compares the performance of three unsupervised visual tracking methods: particle filter, optical flow, and channel and spatial reliability tracker (CSRT) under various tracking conditions. The dataset used includes challenges such as moving target variations, changes in object scale, viewpoint changes, suboptimal lighting, image blurring, partial occlusions, and abrupt movements. Evaluation criteria include tracking accuracy, resistance to occlusion, and computational efficiency. The particle filter with ORB and a constant velocity model achieves a root mean square error (RMSE) of 36.47 pixels at 13 frames per second (fps). Optical flow performs best with an RMSE of 10.79 pixels at 30 fps, while CSRT shows an RMSE of 252.35 pixels at 4 fps. These findings highlight the effectiveness of optical flow for real-time applications, making it a promising solution for mobile robot visual tracking in challenging situations.

International audience

This paper proposes a new algorithm for abnormal behavior detection in surveillance video. Firstly, the motion information image of each frame is constructed by calculating the optical flow size and the angle difference between the optical flow vectors between consecutive frames, and then the obtained motion image information is input into the convolutional neural network (CNN) for training, and used for video abnormal behavior detection. In the algorithm, the motion information image generated based on optical flow information can provide the motion information features in the video image more accurately, which makes it easier to distinguish the normal behavior and abnormal behavior of the video. The experiment of this algorithm is carried out on the commonly used data set PETS 2009. Experimental results show that the proposed method and other algorithms have a significant improvement in the accuracy of abnormal behavior detection.

With the development of convolutional neural network, video super-resolution algorithm has achieved remarkable success. Because the dependence between frames is complex, traditional methods lack the ability to model the complex dependence, and it is difficult to estimate and compensate the motion accurately in the process of video super-resolution reconstruction. Therefore, a reconstruction network based on optical flow residuals is proposed. In low resolution space, the dense residual network is used to obtain the complementary information of adjacent video frames, and then the optical flow of high-resolution video frames is predicted through the pyramid structure, and then the low resolution video frames are transformed into high-resolution video frames through the sub-pixel convolution layer, The high-resolution video frame is compensated with the predicted high-resolution optical flow. Finally, it is input into the super-resolution fusion network to get better effect. A new loss function training network is proposed to better constrain the network. Experimental results on public data sets show that the reconstruction effect is improved in PSNR, structural similarity and subjective visual effect.

This paper presents a new automatic image segmentation method for segmenting moving object in complex environment by combining the motion information with edge information. We propose an adaptive optical flow method based on the Horn-Schunck algorithm to estimate the optical flow field. Our method puts different smoothness constraints on different directions and optical flow constraint is used according to the gradient magnitude. Canny edge detector can obtain the most edge information but miss some pixels. In order to restore these missing pixels the edge has a growing based on the continuity of optical flow field. Next, by remaining the block that has the longest edge could delete the noise in the background, and then the last segmentation result is obtained. The experimental result demonstrates that this method can segment the moving object in complex environment precisely.

We propose a novel background subtraction algorithm for the videos captured by a moving camera. In our technique, foreground and background appearance models in each frame are constructed and propagated sequentially by Bayesian filtering. We estimate the posterior of appearance, which is computed by the product of the image likelihood in the current frame and the prior appearance propagated from the previous frame. The motion, which transfers the previous appearance models to the current frame, is estimated by nonparametric belief propagation; the initial motion field is obtained by optical flow and noisy and incomplete motions are corrected effectively through the inference procedure. Our framework is represented by a graphical model, where the sequential inference of motion and appearance is performed by the combination of belief propagation and Bayesian filtering. We compare our algorithm with the existing state-of-the-art technique and evaluate its performance quantitatively and qualitatively in several challenging videos.

6D Object Pose Tracking with Optical Flow Network for Robotic Manipulation

Dense optical flow estimation is complex and time consuming, with state-of-the-art methods relying either on large synthetic data sets or on pipelines requiring up to a few minutes per frame pair. In this paper, we address the problem of optical flow estimation in the automotive scenario in a self-supervised manner. We argue that optical flow can be cast as a geometrical warping between two successive video frames and devise a deep architecture to estimate such transformation in two stages. First, a dense pixel-level flow is computed with a projective bootstrap on rigid surfaces. We show how such global transformation can be approximated with a homography and extend spatial transformer layers so that they can be employed to compute the flow field implied by such transformation. Subsequently, we refine the prediction by feeding a second, deeper network that accounts for moving objects. A final reconstruction loss compares the warping of frame X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sub> with the subsequent frame X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t+1</sub> and guides both estimates. The model has the speed advantages of end-to-end deep architectures while achieving competitive performances, both outperforming recent unsupervised methods and showing good generalization capabilities on new automotive data sets.

Is power everything? What can we learn from large data sets

Convolutional Neural Network (CNN) based methods have shown significant performance gains in the problem of visual tracking in recent years. Due to many uncertain changes of objects online, such as abrupt motion, background clutter and large deformation, the visual tracking is still a challenging task. We propose a novel algorithm, namely Deep Location-Specific Tracking, which decomposes the tracking problem into a localization task and a classification task, and trains an individual network for each task. The localization network exploits the information in the current frame and provides a specific location to improve the probability of successful tracking, while the classification network finds the target among many examples generated around the target location in the previous frame, as well as the one estimated from the localization network in the current frame. CNN based trackers often have massive number of trainable parameters, and are prone to over-fitting to some particular object states, leading to less precision or tracking drift. We address this problem by learning a classification network based on 1 × 1 convolution and global average pooling. Extensive experimental results on popular benchmark datasets show that the proposed tracker achieves competitive results without using additional tracking videos for fine-tuning. The code is available at https://github.com/ZjjConan/DLST

An indoor service robot can establish a 3D map of the environment by a visual sensor. When the robot uses the established map again, there is a situation that the robot's view is in a new position and the relative position of the map can not be obtained. A relocalization algorithm based on multi-dimensional map information is proposed to solve the problem. Based on the established 3D environment map, the algorithm samples a 2D map and matches the map with the 2D observation information of the current frame based on the particle filter algorithm to obtain the robot's initial position. Then, the target point cloud corresponding to the current frame is segmented in the 3D map, and the exact pose relationship between the current frame cloud and the target point cloud is calculated by the iterative closest point algorithm. The SVD decomposition and g2o optimization are used in the method to improve the computational efficiency and accuracy. Finally, the relocalization experiments under the wide range of robot perspective are carried out in the indoor environment. The results verify the effectiveness and robustness of the algorithm.

Traditionally, video segmentation usually extracts object using low-level features such as color, texture, edge, motion, and optical flow. This paper originally proposes that the connectivity of object motion is an advanced feature of video moving object because it can reflect semantic meanings of object to some extent. And it can be fully represented on cumulated difference image which is the combination of a certain number of interframe difference images. Based on this principle, a novel system is designed to extract initial moving object automatically. The system includes 3 key innovations: 1) System is applied on cumulated difference image which can make object more prominent than background noise. Object extraction is based on the connectivity of object motion and it can guarantee the integrity of the extracted object while eliminate big background regions which cannot be removed by conventional change detection methods, for example, intense-noise regions and shadow regions that are not connected tightly to object. 2) Video sequence analysis is performed ahead of video segmentation. Proper object extraction methods are adopted according to the characteristics of background noise and object motion. 3) The adaptive threshold is automatically determined on cumulated difference image after acute noises is removed. The threshold determined here is more reasonable. And with it, most noise can be eliminated while small-motion regions of object are preserved. Results show that this system can extract object in different kinds of sequences automatically, promptly and properly. Thus, this system is very suitable for real time video applications.