A method of color optical flow computation with local and global model combined

The use of image processing technique for cardiac motion analysis has been an active research in the past decade. The estimation of myocardial motion eases the cardiologist in diagnosing cardiac abnormalities. In term of movement analysis, optical flow is the most popular technique that has been used by researchers. This paper describes the implementation and evaluation of three optical flow computation techniques to estimate the myocardial motion using echocardiographic images. The three techniquesare the global smoothness method (GSM), the local smoothness method (LSM) and warping technique (WT). Optical flow field is computed based on healthy cardiac video on parasternal short axes view. These techniques look promising since the optical flow fields can be utilized to estimate the myocardial movement and comply with its true movement. The performances of each technique in terms of the direction, homogeneity and computing time, are also discussed.

The optical flow techniques have been developed for more than one decade. Once the optical flow field is computed accurately, this measurement of image velocity can be used widely in many tasks in computer vision area. Current computer vision techniques require that the relative errors in the optical flow be less than 10%. However, to reduce error in determination of the optical flow is still a difficult problem. In this paper, firstly, errors occurring in the correlation-based approaches to optical flow computation are analyzed. Through understanding how the errors arise, we developed a new approach to computation of optical flow named the correlation-feedback approach. It is based on the idea of feedback and the correlation-based approach. In this approach, a virtual continuous image is obtained by a bilinear interpolation applied to a digital image. The idea of feedback is used so that errors in determining optical flow are reduced considerably in the iterative procedure. It is proved that the algorithm is convergent generally. Several experiments working on real image sequences in the laboratory demonstrate that our correlation-feedback algorithm performs better than the gradient-based and correlation-based algorithms in terms of accuracy.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A novel approach that integrates multigrid computation and Harris corner extraction is proposed for solving the problems of high complexity and low speed in conventional optical flow computation. Firstly, Harris corners are extracted from grabbed frame images of real-time video. Then corner images are generated on the basis of corner coordinates. Secondly, an efficient computational technique based on multigrid is adopted to optimize the optical flow estimation on the consecutive corner images. Therefore, real-time movement can be detected by optical flow vector field. After many experiments on videos with moving objects in different backgrounds and rates of frame grabbing, experimental results demonstrate the higher computational speed. This approach is applied to detect moving objects in real-time videos and the results are very encouraging. The approach aims at accelerating optical flow computation, also being robust to disturbance of complex background due to only adopting corner images.

In order to make optical flow methods suitable for real-time applications, this paper presents a rapid optical flow computation method based on multigrid. Multigrid computation method is well known for being the fastest numerical method when solving elliptic boundary-value problems, so we integrate multigrid into Horn-Schunck algorithm in order to obtain rapid computation. Firstly, we analyze the convergence process of traditional methods for optical flow computation; secondly, we combine the multigrid method with Horn-Schunck method, and in this way optical flow can be calculated efficiently; finally, the experiments show that our rapid optical flow computation method is efficient for target detection and video tracking systems.

Template matching has been widely used in image processing for visual inspection of complicated, patterned surfaces. The currently existing methods such as golden template matching and normalized cross correlation are very sensitive to the displacement even the object under test is carefully aligned with respect to the template. This paper proposes a dissimilarity measure based on the optical-flow technique for surface defect detection, and aims at light-emitting diode (LED) wafer die inspection. The dissimilarity measure of each pixel derived from the optical flow field does not represent the true translation distance, but is reliable enough to indicate the degree of difference between an image pair. It is well tolerated to misalignment and random product variation. The integral image technique is applied to replace the sum operations in optical flow computation, and speeds up the intensive computation. We also point out the pitfall of the Lucas-Kanade optical flow when it is applied for defect detection, and propose a swapping process to tackle the problem. The experiment on LED wafer dies has shown that the proposed method can achieve a 100% recognition rate based on a test set of 357 die images.

Despite progress made in the accuracy and robustness of optical flow in past years, the problem of over-segmentation and the blurring of image edge and motion boundary caused by the illumination change, complex texture, large displacement, and motion occlusion still remain. Recently, we developed a guided filtering scheme for flow field estimation, which is implemented as an add-on optimal operation during the coarse-to-fine optical flow computation. In this paper, we first review the research progress in optical flow computation and discuss limitations of the currently popular median filtering heuristic for a flow field optimization. We then introduce a general formulation of the guided filtering and provide the detailed illustration. Furthermore, we explore the potential of the guided filtering optimization for the flow field estimation under the coarse-to-fine computing scheme. Finally, we modify some typical and state-of-the-art optical flow methods by applying the proposed guided filtering operation to the baseline models, and test the performances of the basic and developed models through the Middlebury, MPI-Sintel, and KITTI data. The experimental results demonstrate that the guided filtering scheme is able to preserve the image edges and motion boundaries, and to improve the accuracy and robustness of optical flow estimation.

An investigation of a fatigue model with two competing failure mechanisms

In this paper, we consider a motion sense problem motivated by the International Aerial Robotics Competition Mission-7, where an aerial robot is required to provide detection and estimation about mobile vehicles. Dense optical flow computing is employed first to provide a velocity field from image sequences. Then, region growing based on the optical flow field is used to extract moving objects on the background, and motion estimation is eventually achieved while both camera and objects are moving. In addition, classical optical flow techniques do not work in the competition since there may be illumination changes, such as flashlights and reflections in the arena. To deal with this problem, the procedures of the brightness constancy relaxation and intensity normalization are combined in the optical flow algorithm. Experimental results have demonstrated the robustness against varying illumination. The proposed approach can provide motion estimation results of acceptable accuracy for several benchmark data sets and image sequences generated with micro aerial vehicles.

Just-in-time software defect prediction (JIT-SDP) is an active topic in software defect prediction, which aims to identify defect-inducing changes. Recently, some studies have found that the variability of defect data sets can affect the performance of defect predictors. By using local models, it can help improve the performance of prediction models. However, previous studies have focused on module-level defect prediction. Whether local models are still valid in the context of JIT-SDP is an important issue. To this end, we compare the performance of local and global models through a large-scale empirical study based on six open-source projects with 227417 changes. The experiment considers three evaluation scenarios of cross-validation, cross-project-validation, and timewise-cross-validation. To build local models, the experiment uses the k-medoids to divide the training set into several homogeneous regions. In addition, logistic regression and effort-aware linear regression (EALR) are used to build classification models and effort-aware prediction models, respectively. The empirical results show that local models perform worse than global models in the classification performance. However, local models have significantly better effort-aware prediction performance than global models in the cross-validation and cross-project-validation scenarios. Particularly, when the number of clusters k is set to 2, local models can obtain optimal effort-aware prediction performance. Therefore, local models are promising for effort-aware JIT-SDP.

In response to the deficiencies in visible-near infrared spectral identification of coal gangue under different working conditions, this paper employs multiple algorithms to construct local and global recognition models and conducts analysis. Simulating five different working conditions in the laboratory, including different detection angles, illumination angles, detection distances, coal powder content, and moisture content. Each working condition is further divided into four levels. Collecting spectral data of coal and gangue samples. Constructing global and local datasets, which are respectively derived from all levels and single levels under certain working conditions. Training local and global models to identify samples based on machine learning algorithms such as Lightgbm and 1DCNN. The results of the experiment indicate that there are variations in the sample spectra and recognition accuracy based on different working conditions. Furthermore, the local model exhibits lower recognition accuracy and inadequate robustness for other level samples under the same working condition. The global model is more robust and practical, as it achieves higher or equal recognition accuracy for local data compared to the local model. Among the global models, Lightgbm has the best overall performance, with an average F1-score of 99.34% and 97.9% for local and global datasets, respectively. It also has fast training time of 0.75s and recognition time of 1 ms. The Lightgbm global model can accurately identify coal and gangue in various working conditions, making it a reliable solution.

Multi-resolution optical flow is the main trends of the current optical flow calculation. Wavelet transform and Gaussian filter are widely used to build multi-resolution structure of the image. In this paper, LK optical flow method mixed with pyramid was used to study the optical flow field. The different effects of wavelet transform and Gaussian filter in the flow distribution was also discussed. Experimental results show that the selected image decomposition method has a significant role in optical flow field calculation.

Motion and structure estimation are elementary problems of computer vision. These are active areas of research, even though the first methods were proposed several decades ago. We develop new approaches for motion and structure estimation for autonomous driving. An autonomous vehicle requires an accurate model of its environment, wrong decisions made by an autonomous car can have severe consequences. We assume the monocular setup, where only a single camera is mounted on the car. Outdoor traffic sequences are challenging for optical flow estimation. The high speed of the car causes large displacements in the optical flow field, the lighting conditions are unstable and there can be strong distortions due to reflections and difficult weather conditions. We propose new discrete methods, which determine optical flow as optimal configuration of probabilistic graphical models. The first approach selects sparse locations in the reference frame, and matches them across the second image. The best correspondences, which match constraints from a multiple view configuration, are considered motion vectors in a graphical model. In a second approach, we solve for dense optical flow by approximating the original infeasible graphical model with a sequence of reduced models. The monocular configuration poses challenges to the estimation of scene structure, camera positions and scene parameters need to be estimated jointly. The geometry of multiple views creates blind spots in the images, and adds a scale ambiguity, which both to not exist in a setup with multiple cameras. We propose two methods for structure estimation. The first approach determines the energy optimal camera track, given optical flow and depth observations. A further approach estimates camera motion and a piecewise planar scene description jointly from a single optical flow field. The scene description contains depth and plane normal information. We evaluate our approaches for motion and structure estimation on different real world and rendered datasets. In addition to evaluation on publicly available evaluation data, we evaluate on a new rendered dataset with ground truth plane normals.

We propose the fast optical flow extractor, a filtering scheme that recovers artifact-free optical flow fields from HEVC-compressed video. To extract accurate optical flow fields, we form a regularized optimization problem that considers the smoothness of the solution and the pixelwise confidence weights of the artifact-ridden HEVC motion field. Solving such an optimization problem is slow, so we first convert the problem into a confidence-weighted filtering task. We then use the extracted optical flow fields for fast synthetic refocus of video using camera motion. By leveraging the already-available HEVC motion information, we achieve a 10-fold speed-up in the running times compared to similar methods. Our fast optical flow extractor is useful since video signals encountered in real life are typically available already in coded formats. Our fast optical flow extractor is general, and works on motion fields of any video coder.

Bai, Y., 2019. Target detection method of underwater moving image based on optical flow characteristics. In: Guido-Aldana, P.A. and Mulahasan, S. (eds.), Advances in Water Resources and Exploration. Journal of Coastal Research, Special Issue No. 93, pp. 668–673. Coconut Creek (Florida), ISSN 0749-0208.The detection and recognition ability of underwater moving image target is improved, and an automatic target detection method based on optical flow feature is proposed. The image acquisition model of underwater moving image target detection is constructed, and the local information entropy feature of underwater moving target image is detected by optical flow field scale decomposition method. The contrast feature of the distribution area of the optical flow field of the underwater moving image is extracted, and the information fusion model of the underwater moving target image is constructed according to the statistical value of the inter-frame difference of the underwater moving image. The center position of the candidate target is obtained, and the adaptive feature extraction of the underwater moving image target is carried out by using the correlation fusion method between the pixel neighborhoods, and the automatic detection of the underwater moving image target is realized according to the optical flow feature distribution. The simulation results show that the proposed method has good adaptability, high accuracy and strong anti-optical flow interference ability in underwater moving image target recognition.

The changes in an image arising from the motion of the environment have been intensively studied over the past few years. These changes, termed optical flow, depend on both the motion and the shape of the environment. The central problem in the study of optical flow is to find the best way of recovering information about the motion and shape of the environment from the associated optical flow field. The author has previously shown (1986, 1987) that some parameters of the motion can be recovered with greater accuracy than the remaining parameters, provided the optical flow field arises from an irregular moving rigid surface. A surface is classed as irregular if the set of inverse distances to points on the surface do not possess a good linear approximation. The algorithm in question is based on a least-square error function, epsilon ( nu ). In the case of an optical flow field arising from an irregular moving rigid surface, there in a simple approximation to epsilon ( nu ) which can be used to predict the performance of the algorithm in situations of practical interest. Experimental results show that the approximation to epsilon ( nu ) is astonishingly accurate. In this paper the approximation to epsilon ( nu ) is given and some experiments to check this approximation are described.