Optical Flow Detection Algorithm and Application Based on Accelerated Function

Despite progress made in the accuracy and robustness of optical flow in past years, the problem of over-segmentation and the blurring of image edge and motion boundary caused by the illumination change, complex texture, large displacement, and motion occlusion still remain. Recently, we developed a guided filtering scheme for flow field estimation, which is implemented as an add-on optimal operation during the coarse-to-fine optical flow computation. In this paper, we first review the research progress in optical flow computation and discuss limitations of the currently popular median filtering heuristic for a flow field optimization. We then introduce a general formulation of the guided filtering and provide the detailed illustration. Furthermore, we explore the potential of the guided filtering optimization for the flow field estimation under the coarse-to-fine computing scheme. Finally, we modify some typical and state-of-the-art optical flow methods by applying the proposed guided filtering operation to the baseline models, and test the performances of the basic and developed models through the Middlebury, MPI-Sintel, and KITTI data. The experimental results demonstrate that the guided filtering scheme is able to preserve the image edges and motion boundaries, and to improve the accuracy and robustness of optical flow estimation.

In order to make optical flow methods suitable for real-time applications, this paper presents a rapid optical flow computation method based on multigrid. Multigrid computation method is well known for being the fastest numerical method when solving elliptic boundary-value problems, so we integrate multigrid into Horn-Schunck algorithm in order to obtain rapid computation. Firstly, we analyze the convergence process of traditional methods for optical flow computation; secondly, we combine the multigrid method with Horn-Schunck method, and in this way optical flow can be calculated efficiently; finally, the experiments show that our rapid optical flow computation method is efficient for target detection and video tracking systems.

(1) Background: The process of epiretinal membrane peeling (MP) requires precise intraoperative visualization to achieve optimal surgical outcomes. This study investigates the integration of preoperative Optical Coherence Tomography (OCT) images into real-time surgical video feeds, providing a dynamic overlay that enhances the decision-making process during surgery. (2) Methods: Five MP surgeries were analyzed, where preoperative OCT images were first manually aligned with the initial frame of the surgical video by selecting five pairs of corresponding points. A homography transformation was then computed to overlay the OCT onto that first frame. Subsequently, for consecutive frames, feature point extraction (the Shi-Tomasi method) and optical flow computation (the Lucas-Kanade algorithm) were used to calculate frame-by-frame transformations, which were applied to the OCT image to maintain alignment in near real time. (3) Results: The method achieved a 92.7% success rate in optical flow detection and maintained an average processing speed of 7.56 frames per second (FPS), demonstrating the feasibility of near real-time application. (4) Conclusions: The developed approach facilitates enhanced intraoperative visualization, providing surgeons with easier retinal structure identification which results in more comprehensive data-driven decisions. By improving surgical precision while potentially reducing complications, this technique benefits both surgeons and patients. Furthermore, the integration of OCT overlays holds promise for advancing robot-assisted surgery and surgical training protocols. This pilot study establishes the feasibility of real-time OCT integration in MP and opens avenues for broader applications in vitreoretinal procedures.

The computation of optical flow has been proposed as a pre-processing step for many high-level vision algorithms. One of the main approaches to the optical flow problem is the gradient-based approach which differentiates the image intensity to compute the optical flow. Often motion vectors computed using various approaches are not reliable. Spatial smoothing of the image sequence is advisable in order to improve velocity estimates in the presence of noise. That is, the application of some kind of linear operation to the images in the sequence before solving for the flow. The pre-processing typically takes the form of some type of spatial Gaussian smoothing or scale specific band-pass filtering of the input images. The main objectives of such filtering have been to lessen the effects of noise, to isolate image structure of interest and to attenuate temporal aliasing and quantization effects in the input images.This paper investigates the effect of presmoothing on this computation of optical flow. The well known method of Horn and Schunck as well as our own finite element method are implemented and tested for improvements due to presmoothing. Discussions are provided on the effects of presmoothing for a variety of image sequences. In our experiments, smoothing is carried out on a selection of well known sequences in both time and space. Improvements are shown using presmoothing.KeywordsFinite Volume MethodsInverse Finite Element MethodsMotion TrackingOptical FlowSpatial SmoothingTemporal Smoothing

Motion estimation (or optical flow estimation) is an interesting problem in computer vision. However this task is computationally intensive for conventional processors. In this work an investigation into an FPGA-based hardware architecture for real-time motion estimation is carried out. The algorithm used in this paper is a gradient based inverse finite element method for optical flow computation. It carries out the motion estimation by calculating the gradient, Laplacian, and subsequently the velocities of each pixel in a parallel design which aids in the speed of the computation. The algorithm has been benchmarked against many of the well known algorithms cited within the field. A literature review is carried out and the algorithm is discussed. An FPGA design is presented. Preliminary simulation results are shown and discussed.

UAV Video is rapidly emerging as a widely used source of imagery for many applications in recent years. This paper presents our research on the mosaic of UAV video for the purpose of harbor surveillance. First, one new framework on estimating video frame transformation with Optical flow is presented in this paper. For this new framework, fewer number of Gaussian pyramid is created for implementing for the multiresolution approach and thus, more details for optical flow computation is well kept; Second, we make a discussion on using Fourier-Mellin Transformation in image frequency domain to estimate initial motion parameter of adjacent video frames and with those initial motion parameters, small displacements for optical flow computation can be achieved; The experimental results demonstrated that the mosaic image generated from aerial video shows satisfied visual quality and its surveillance application for fast response to time-critical event, e.g., flood, is descried.

We present in this paper a motion computation and interpretation framework for oceanographic satellite images. This framework is based on the use of a non quadratic regularization technique in optical flow computation that preserves flow discontinuities. We also show that using an appropriate tessellation of the image according to an estimate of the motion field can improve optical flow accuracy and yields more reliable flows. This method defines a non uniform multiresolution scheme that refines mesh resolution only in the neighborhood of moving structures. The second part of the paper deals with the interpretation of the obtained displacement field. We use a phase portrait model with a new formulation of the approximation of an oriented flow field. This allows us to consider arbitrary polynomial phase portrait models for characterizing salient flow features. This new framework is used for processing oceanographic and atmospheric image sequences and presents an alternative to the very complex physical modelling techniques.

In this paper, we propose the combined use of complementary concepts for 3D tracking: region fitting on one side and dense optical flow as well as tracked SIFT features on the other. Both concepts are chosen such that they can compensate for the shortcomings of each other. While tracking by the object region can prevent the accumulation of errors, optical flow and SIFT can handle larger transformations. Whereas segmentation works best in case of homogeneous objects, optical flow computation and SIFT tracking rely on sufficiently structured objects. We show that a sensible combination yields a general tracking system that can be applied in a large variety of scenarios without the need to manually adjust weighting parameters.

Template matching has been widely used in image processing for visual inspection of complicated, patterned surfaces. The currently existing methods such as golden template matching and normalized cross correlation are very sensitive to the displacement even the object under test is carefully aligned with respect to the template. This paper proposes a dissimilarity measure based on the optical-flow technique for surface defect detection, and aims at light-emitting diode (LED) wafer die inspection. The dissimilarity measure of each pixel derived from the optical flow field does not represent the true translation distance, but is reliable enough to indicate the degree of difference between an image pair. It is well tolerated to misalignment and random product variation. The integral image technique is applied to replace the sum operations in optical flow computation, and speeds up the intensive computation. We also point out the pitfall of the Lucas-Kanade optical flow when it is applied for defect detection, and propose a swapping process to tackle the problem. The experiment on LED wafer dies has shown that the proposed method can achieve a 100% recognition rate based on a test set of 357 die images.

The halftone image of a scene can be described by giving the gray value as a function of the image plane coordinate vector x, i.e., the picture function g(x). In the case of relative motion between the camera and the depicted scene, the picture function will depend not only on the image plane coordinate vector x, but in addition on the time t. During a short enough time interval, △t, the resulting temporal change of the picture function can be approximated by a local shift of spatial gray value structures. This apparent shift can be described by an optical flow vector field, u(x,t). It maps the gray value g(x–u△t,t) recorded at time t at the image plane location x–u△t into the gray value g(x,t + △t) recorded at location x at time t + △t. In most cases, this optical flow field is a good approximation to the temporal displacement of the image of a depicted surface element between time t and time t + △t.

A shipborne infrared surveillance system, which is tasked with detecting real time target trajectories, needs a kinematic evaluation of the cloudy background. In effect, false alarms appear at the cloud edges; these false alarms reducing the effectiveness of the surveillance system. The elimination of clutter by means of static criteria on each frame is not possible because of the spatial coherence of these edges. The interest of this study is to bring together complementary information that concerns the temporal evolution of the 2D infrared signal. Thus the aim is to evaluate the kinematics of cloudy backgrounds. The originality of this paper resides in the use of the Generalized Hough Transform associated with an affine transform method using optical flow computation. The first transform gives an average displacement for a set of pels, while the second extracts more accurately the motion information by means of differential analysis based on the optical flow constraint equation. The complementary aspect of these two methods of motion estimation is shown.

To estimate the motion and deformation of nonrigid objects accurately from image sequences, this paper proposes a method for nonrigid motion tracking based on smoothness constraints, which have an effect on noise reduction in optical flow computation by a gradient‐based method. The motion of nonrigid objects generally is assumed to be smooth according to such physical properties as inertia and elasticity. Therefore, such smoothness constraints are defined in space (in frame) and time (between frames) as an active net model in formulating an energy functional and Lagrange equation, and the optical flow field derived from image sequences by shape preservation and natural deformation is modified. In addition, applying continuous Kalman filtering theory for long sequences of image frames, it is possible to estimate the temporal continuous deformation of elastic objects. The nonrigid motion tracking is demonstrated by experiments involving the modification of the shape distortion yielded by noise and the reconstruction of the real motion and deformation.

The optical flow techniques have been developed for more than one decade. Once the optical flow field is computed accurately, this measurement of image velocity can be used widely in many tasks in computer vision area. Current computer vision techniques require that the relative errors in the optical flow be less than 10%. However, to reduce error in determination of the optical flow is still a difficult problem. In this paper, firstly, errors occurring in the correlation-based approaches to optical flow computation are analyzed. Through understanding how the errors arise, we developed a new approach to computation of optical flow named the correlation-feedback approach. It is based on the idea of feedback and the correlation-based approach. In this approach, a virtual continuous image is obtained by a bilinear interpolation applied to a digital image. The idea of feedback is used so that errors in determining optical flow are reduced considerably in the iterative procedure. It is proved that the algorithm is convergent generally. Several experiments working on real image sequences in the laboratory demonstrate that our correlation-feedback algorithm performs better than the gradient-based and correlation-based algorithms in terms of accuracy.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

To the breakage warning problem of grid fence, a grid breakage detection algorithm based on improved Harris corner is proposed. As traditional Harris corner extraction algorithm is very time-consuming, the parameter of gray “similarity” is introduced to demonstrate the gray similarity between pixel and its ambient pixels, through which to filter pseudo corners, reducing the Harris corner extraction time. Then after analyzing the corners' distribution, the breakage areas can be locked. Breakage detection experiments are taken to various fence images taken by robot, and the results indicate that the proposed algorithm is effective, and can meet the real-time requirements of fence breakage detection.

New applications in fields such as augmented or virtualized reality have created a demand for dense, accurate real-time stereo reconstruction. Our goal is to reconstruct a user and her office environment for networked tele-immersion, which requires accurate depth values in a relatively large workspace. In order to cope with the combinatorics of stereo correspondence we can exploit the temporal coherence of image sequences by using coarse optical flow estimates to bound disparity search ranges at the next iteration. We use a simple flood fill segmentation method to cluster similar disparity values into overlapping windows and predict their motion over time using a single optical flow calculation per window. We assume that a contiguous region of disparity represents a single smooth surface which allows us to restrict our search to a narrow disparity range. The values in the range may vary over time as objects move nearer or farther away in Z, but we can limit the number of disparities to a feasible search size per window. Further, the disparity search and optical flow calculation are independent for each window, and allow natural distribution over a multi-processor architecture.We have examined the relative complexity of stereo correspondence on full images versus our proposed window system and found that, depending on the number of frames in time used to estimate optical flow, the window-based system requires about half the time of standard correlation stereo. Experimental comparison to full image correspondence search shows our window-based reconstructions compare favourably to those generated by the full algorithm, even after several frames of propagation via estimated optical flow. The result is a system twice as fast as conventional dense correspondence without significant degradation of extracted depth values.KeywordsAugmented RealityTemporal CoherenceFull ImageFull FrameStereo SystemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.