Target Detection Method of Underwater Moving Image Based on Optical Flow Characteristics

Enhanced river suspended sediment concentration identification via multimodal video image, optical flow, and water temperature data fusion

We propose the fast optical flow extractor, a filtering scheme that recovers artifact-free optical flow fields from HEVC-compressed video. To extract accurate optical flow fields, we form a regularized optimization problem that considers the smoothness of the solution and the pixelwise confidence weights of the artifact-ridden HEVC motion field. Solving such an optimization problem is slow, so we first convert the problem into a confidence-weighted filtering task. We then use the extracted optical flow fields for fast synthetic refocus of video using camera motion. By leveraging the already-available HEVC motion information, we achieve a 10-fold speed-up in the running times compared to similar methods. Our fast optical flow extractor is useful since video signals encountered in real life are typically available already in coded formats. Our fast optical flow extractor is general, and works on motion fields of any video coder.

Motion and structure estimation are elementary problems of computer vision. These are active areas of research, even though the first methods were proposed several decades ago. We develop new approaches for motion and structure estimation for autonomous driving. An autonomous vehicle requires an accurate model of its environment, wrong decisions made by an autonomous car can have severe consequences. We assume the monocular setup, where only a single camera is mounted on the car. Outdoor traffic sequences are challenging for optical flow estimation. The high speed of the car causes large displacements in the optical flow field, the lighting conditions are unstable and there can be strong distortions due to reflections and difficult weather conditions. We propose new discrete methods, which determine optical flow as optimal configuration of probabilistic graphical models. The first approach selects sparse locations in the reference frame, and matches them across the second image. The best correspondences, which match constraints from a multiple view configuration, are considered motion vectors in a graphical model. In a second approach, we solve for dense optical flow by approximating the original infeasible graphical model with a sequence of reduced models. The monocular configuration poses challenges to the estimation of scene structure, camera positions and scene parameters need to be estimated jointly. The geometry of multiple views creates blind spots in the images, and adds a scale ambiguity, which both to not exist in a setup with multiple cameras. We propose two methods for structure estimation. The first approach determines the energy optimal camera track, given optical flow and depth observations. A further approach estimates camera motion and a piecewise planar scene description jointly from a single optical flow field. The scene description contains depth and plane normal information. We evaluate our approaches for motion and structure estimation on different real world and rendered datasets. In addition to evaluation on publicly available evaluation data, we evaluate on a new rendered dataset with ground truth plane normals.

Current action recognition algorithms in ice hockey do not fully exploit the temporal cues available in video. To solve this challenge, we introduce a two-stream network utilizing player pose sequences and optical flow features for recognizing hockey actions. Player pose sequences are compact representations consisting of frame by frame human and stick joint locations and angles between joints. The optical flow features are obtained by a state-of-the-art optical flow algorithm. The player pose sequences are processed by a two-layered Long short-term memory (LSTM) network. The LSTM output is fused with optical flow features processed by a convolutional neural network (CNN). Experimental results demonstrate the efficacy of the method by achieving 90.48% test accuracy on the HARPET (Hockey Action Recognition Pose Estimation, Temporal) dataset thus surpassing current benchmark by 5%. The network performs better than the current benchmark in segregating similar classes like passing and shooting. It achieves a 90% reduction in parameters and 80% reduction in floating point operations per second (FLOPs) than the benchmark on the HARPET dataset, thus furthering the effectiveness of the network.

Human motion recognition is traditionally approached by either recognizing basic motions from features derived from video input or by interpreting complex motions by applying a high-level hierarchy of motion primitives. The former method is usually limited to rather simple motions while the latter requires human expert knowledge to build up a suitable hierarchy. In this paper we propose a new approach that uses the strength of both methods while overcoming their respective limitations. Our approach is able to recognize the motion units within complex motion sequences. The recognition process applies Hidden Markov Models (HMM) based on features consisting of optical flow gradient histograms. For each primitive motion unit we train one HMM and then concatenate these primitive motion units to form complex motion sequences. Modeling sequences with HMMs allows for a very flexible combination of motion units into motion sequences. They can either be combined in a restrictive rule-based formulation using predefined grammars or be more flexibly combined using a statistical model of sequence probabilities. In this paper we are mainly interested in the comparison of the optical flow features with marker-based features, therefore we do not use a motion grammar. We apply our approach to 24 motion units forming five complex motion sequences as they appear in a real-world kitchen tasks. The results show that the proposed approach allows for a very fast low-level recognition of human motion units without the need for any complex reconstruction, post processing or pose estimation. Straight-forward characteristic flow fields in combination with HMM sequence modeling are sufficient to reliably recognize complex motions even with an unrestricted search. Our results show that this search already achieves 13.1 % recognition error rate. We compare HMM models based on the optical flow features to those derived from a marker-based system. Our recognition results indicate that optical flow features achieve a competitive performance.

This paper proposes a novel particle image velocimetry (PIV) technique to generate an instantaneous two-dimensional velocity field for sediment-laden fluid based on the optical flow algorithm of ultrasound imaging. In this paper, an ultrasonic PIV (UIV) system is constructed by integrating a medical ultrasound instrument and an ultrasonic particle image velocimetry algorithm. The medical ultrasound instrument with a phased sensor array is used to acquire acoustic echo signals of particles in water and generate two-dimensional underwater ultrasound images. Based on the optical flow field, the instantaneous velocity of the particles in water corresponding to the pixels in the ultrasonic particle images is derived from the grayscale change between adjacent images under the L-K local constraint based on the optical flow field, and finally, the two-dimensional flow field is obtained. Through multiple sets of experiments, the proposed algorithm is verified. The experimental results are compared with those of the conventional cross-correlation algorithms. The results show that the L-K optical flow method can not only obtain the underwater velocity field accurately, but also has the advantages of good smoothness and extensive suitability, especially for the flow field measurement in sediment-laden fluid than conventional algorithms.

The computation of optical flow can play an important role in background characterization and target detectability algorithms since motion is an important cue feature for target detection. A multiresolution gradient-based motion estimation algorithm is implemented based on Horn's approach, where the optical flow field is iteratively determined at each level of the multiresolution pyramid. Different masks and kernels are tested, and an adaptive error minimization approach is taken to deal with areas in the image that violate the optical flow constraint equation.

Multi-resolution optical flow is the main trends of the current optical flow calculation. Wavelet transform and Gaussian filter are widely used to build multi-resolution structure of the image. In this paper, LK optical flow method mixed with pyramid was used to study the optical flow field. The different effects of wavelet transform and Gaussian filter in the flow distribution was also discussed. Experimental results show that the selected image decomposition method has a significant role in optical flow field calculation.

The changes in an image arising from the motion of the environment have been intensively studied over the past few years. These changes, termed optical flow, depend on both the motion and the shape of the environment. The central problem in the study of optical flow is to find the best way of recovering information about the motion and shape of the environment from the associated optical flow field. The author has previously shown (1986, 1987) that some parameters of the motion can be recovered with greater accuracy than the remaining parameters, provided the optical flow field arises from an irregular moving rigid surface. A surface is classed as irregular if the set of inverse distances to points on the surface do not possess a good linear approximation. The algorithm in question is based on a least-square error function, epsilon ( nu ). In the case of an optical flow field arising from an irregular moving rigid surface, there in a simple approximation to epsilon ( nu ) which can be used to predict the performance of the algorithm in situations of practical interest. Experimental results show that the approximation to epsilon ( nu ) is astonishingly accurate. In this paper the approximation to epsilon ( nu ) is given and some experiments to check this approximation are described.

Detection of cow mounting behavior using region geometry and optical flow characteristics

The optical flow techniques have been developed for more than one decade. Once the optical flow field is computed accurately, this measurement of image velocity can be used widely in many tasks in computer vision area. Current computer vision techniques require that the relative errors in the optical flow be less than 10%. However, to reduce error in determination of the optical flow is still a difficult problem. In this paper, firstly, errors occurring in the correlation-based approaches to optical flow computation are analyzed. Through understanding how the errors arise, we developed a new approach to computation of optical flow named the correlation-feedback approach. It is based on the idea of feedback and the correlation-based approach. In this approach, a virtual continuous image is obtained by a bilinear interpolation applied to a digital image. The idea of feedback is used so that errors in determining optical flow are reduced considerably in the iterative procedure. It is proved that the algorithm is convergent generally. Several experiments working on real image sequences in the laboratory demonstrate that our correlation-feedback algorithm performs better than the gradient-based and correlation-based algorithms in terms of accuracy.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

In this paper, we present a CNN-based fully unsupervised method for motion segmentation from optical flow. We assume that the input optical flow can be represented as a piecewise set of parametric motion models, typically, affine or quadratic motion models. The core idea of our work is to leverage the Expectation-Maximization (EM) framework in order to design in a well-founded manner a loss function and a training procedure of our motion segmentation neural network that does not require either ground-truth or manual annotation. However, in contrast to the classical iterative EM, once the network is trained, we can provide a segmentation for any unseen optical flow field in a single inference step and without estimating any motion models. We investigate different loss functions including robust ones and propose a novel efficient data augmentation technique on the optical flow field, applicable to any network taking optical flow as input. In addition, our method is able by design to segment multiple motions. Our motion segmentation network was tested on four benchmarks, DAVIS2016, SegTrackV2, FBMS59, and MoCA, and performed very well, while being fast at test time.

Obstacle detection by evaluation of optical flow fields from image sequences

Dense matching plays an important role in many fields, such as DEM (digital evaluation model) producing, robot navigation and 3D environment reconstruction. Traditional approaches may meet the demand of accuracy. But the calculation time and out puts density is hardly be accepted. Focus on the matching efficiency and complex terrain surface matching feasibility an aerial image dense matching method based on optical flow field is proposed in this paper. First, some high accurate and uniformed control points are extracted by using the feature based matching method. Then the optical flow is calculated by using these control points, so as to determine the similar region between two images. Second, the optical flow field is interpolated by using the multi-level B-spline interpolation in the similar region and accomplished the pixel by pixel coarse matching. Final, the results related to the coarse matching refinement based on the combined constraint, which recognizes the same points between images. The experimental results have shown that our method can achieve per-pixel dense matching points, the matching accuracy achieves sub-pixel level, and fully meet the three-dimensional reconstruction and automatic generation of DSM-intensive matching’s requirements. The comparison experiments demonstrated that our approach’s matching efficiency is higher than semi-global matching (SGM) and Patch-based multi-view stereo matching (PMVS) which verifies the feasibility and effectiveness of the algorithm.

Video snapshot compressive imaging (SCI) samples 3D high-speed video frames with temporally varying spatial modulation and compresses them into a single 2D measurement, and the SCI reconstruction algorithm aims to recover the original high-speed frames from the measurement. However, conventional video SCI systems encounter challenges when raising the frame rate of the reconstructed video. To achieve higher frame rates, the modulation mask's rate must be increased, which in turn leads to an increase in the associated hardware expenses. In this paper, we propose a deep unfolding-based reconstruction framework with optical flow for video SCI. The framework recovers both observed and unobserved frames from measurements, resulting in increased video frame rate. To estimate the optical flow, we transform the video features of the network into optical flow features during the iteration. Then, we design a deep denoiser and an optical flow-based motion regularizer combined with the voxels of coarse reconstructed frames to update the observed and unobserved frames. To improve the performance, we employ group convolution in the network and fuse the optical flow information from different phases to reduce the information loss. We further extend the proposed deep unfolding framework to the reconstruction of color SCI videos. Extensive experiments on benchmark data and real data prove that our proposed method has state-of-the-art reconstruction performance and visual effects.