Dynamic Illumination Optical Flow Computing for Sensing Multiple Mobile Robots From a Drone

Temporal segmentation of a video sequence into different shots is fundamental to a number of video retrieval and analysis applications. Motion estimation has been wildly used in many applications of video processing, since it provides the most essential information for an image sequence. In this paper, we explore the possibility to exploit motion and illumination estimation in a video sequence to detect various types of shot changes. Optical flow is the motion vector computed at each pixel in an image sequence from intensity variation. Traditionally, optical flow computational algorithms were derived from the brightness constancy assumption. In this paper, we employ a generalized optical flow constraint that includes an illumination parameter to model local illumination changes. An iterative optical flow and illumination estimation algorithm is developed in this paper to refine the generalized optical flow constraints step by step, thus leading to a very accurate estimation of the optical flow and illumination parameters. Two robust measures are defined from the mean and standard deviation of the estimated intensity compensation values for all the blocks in the same image. Either of these two measures corresponds significantly to various types of shot changes. We show the usefulness of these two measures through experiments.

Motion and structure estimation are elementary problems of computer vision. These are active areas of research, even though the first methods were proposed several decades ago. We develop new approaches for motion and structure estimation for autonomous driving. An autonomous vehicle requires an accurate model of its environment, wrong decisions made by an autonomous car can have severe consequences. We assume the monocular setup, where only a single camera is mounted on the car. Outdoor traffic sequences are challenging for optical flow estimation. The high speed of the car causes large displacements in the optical flow field, the lighting conditions are unstable and there can be strong distortions due to reflections and difficult weather conditions. We propose new discrete methods, which determine optical flow as optimal configuration of probabilistic graphical models. The first approach selects sparse locations in the reference frame, and matches them across the second image. The best correspondences, which match constraints from a multiple view configuration, are considered motion vectors in a graphical model. In a second approach, we solve for dense optical flow by approximating the original infeasible graphical model with a sequence of reduced models. The monocular configuration poses challenges to the estimation of scene structure, camera positions and scene parameters need to be estimated jointly. The geometry of multiple views creates blind spots in the images, and adds a scale ambiguity, which both to not exist in a setup with multiple cameras. We propose two methods for structure estimation. The first approach determines the energy optimal camera track, given optical flow and depth observations. A further approach estimates camera motion and a piecewise planar scene description jointly from a single optical flow field. The scene description contains depth and plane normal information. We evaluate our approaches for motion and structure estimation on different real world and rendered datasets. In addition to evaluation on publicly available evaluation data, we evaluate on a new rendered dataset with ground truth plane normals.

This paper proposes a novel particle image velocimetry (PIV) technique to generate an instantaneous two-dimensional velocity field for sediment-laden fluid based on the optical flow algorithm of ultrasound imaging. In this paper, an ultrasonic PIV (UIV) system is constructed by integrating a medical ultrasound instrument and an ultrasonic particle image velocimetry algorithm. The medical ultrasound instrument with a phased sensor array is used to acquire acoustic echo signals of particles in water and generate two-dimensional underwater ultrasound images. Based on the optical flow field, the instantaneous velocity of the particles in water corresponding to the pixels in the ultrasonic particle images is derived from the grayscale change between adjacent images under the L-K local constraint based on the optical flow field, and finally, the two-dimensional flow field is obtained. Through multiple sets of experiments, the proposed algorithm is verified. The experimental results are compared with those of the conventional cross-correlation algorithms. The results show that the L-K optical flow method can not only obtain the underwater velocity field accurately, but also has the advantages of good smoothness and extensive suitability, especially for the flow field measurement in sediment-laden fluid than conventional algorithms.

Optical flow can greatly improve the robustness of visual tracking algorithms. While dense optical flow algorithms have various applications, they can not be used for real-time solutions without resorting to GPU calculations. Furthermore, most optical flow algorithms fail in challenging lighting environments due to the violation of the brightness constraint. We propose a simple but effective iterative regularisation scheme for real-time, sparse optical flow algorithms, that is shown to be robust to sudden illumination changes and can handle large displacements. The algorithm proves to outperform well known techniques in real life video sequences, while being much faster to calculate. Our solution increases the robustness of a real-time particle filter based tracking application, consuming only a fraction of the available CPU power. Furthermore, a new and realistic optical flow dataset with annotated ground truth is created and made freely available for research purposes.

Towards computer-assisted neurosurgery, scene understanding algorithms for microscope video data are required. Previous work utilizes optical flow to extract spatiotemporal context from neurosurgical video sequences. However, to select an appropriate optical flow method, we need to analyze which algorithm yields the highest accuracy for the neurosurgical domain. Currently, there are no benchmark datasets available for neurosurgery. In our work, we present an approach to generate synthetic data for optical flow evaluation on the neurosurgical domain. We simulate image sequences and thereby take into account domainspecific visual conditions such as surgical instrument motion. Then, we evaluate two optical flow algorithms, Farneback and PWC-Net, on our synthetic data. Qualitative and quantitative assessments confirm that our data can be used to evaluate optical flow for the neurosurgical domain. Future work will concentrate on extending the method by modeling additional effects in neurosurgery such as elastic background motion.

Two-step phase extraction and random phase shift estimation in phase-shifting profilometry based on least-squared optical flow method

A number of proposed optical flow estimation algorithms for image motion analysis use only a few subsequent frames from an image sequence to perform calculation. Others, that use many frames, as is case in frequency and energy based, as well as in stochastic optical flow algorithms, are better suited for off-line work. We present our approach in adopting a classic gradient optical flow algorithm for on-line work over extended image sequence. Our experiments on the usual optical flow testing image sequences, consisting of few dozens of image frames show significant improvements in accuracy of the estimated velocity flow fields and in speed of computation. This is achieved by modification and integration of three basic processing phases: preparation of a limited spatio-temporal volume, estimation of velocity flow fields and post filtering of the flow fields. In the preparation phase we use efficient spatio-temporal filters for image smoothing. In the estimation phase a modified iterative solution that propagates results from previous calculation steps is used. Linear and nonlinear post filtering additionally improve optical flow estimates. The whole process is less sensitive to the aperture problem and produces more accurate flow field estimates than the original algorithm. It better copes with motion discontinuities as well. We evidence our achievements by a number of experiments on extended image sequences.

The use of image processing technique for cardiac motion analysis has been an active research in the past decade. The estimation of myocardial motion eases the cardiologist in diagnosing cardiac abnormalities. In term of movement analysis, optical flow is the most popular technique that has been used by researchers. This paper describes the implementation and evaluation of three optical flow computation techniques to estimate the myocardial motion using echocardiographic images. The three techniquesare the global smoothness method (GSM), the local smoothness method (LSM) and warping technique (WT). Optical flow field is computed based on healthy cardiac video on parasternal short axes view. These techniques look promising since the optical flow fields can be utilized to estimate the myocardial movement and comply with its true movement. The performances of each technique in terms of the direction, homogeneity and computing time, are also discussed.

Obstacle detection by evaluation of optical flow fields from image sequences

This paper describes a resource-efficient vision system for the mini-robot “Khepera”. It is implemented to perform a fundamental approach of robotic navigation, which is obstacle detection by using an optical flow algorithm. This is inspired by visual perception of insects. The optical flow field is evaluated by implementing the sum of absolute differences (SAD) operation of block matching while the mobile mini-robot moves on texture plane. The SAD is realized in an additional processing module based on a Field Programmable Gate Array (FPGA) solution which can perform various tasks in parallel; for example, image processing, camera control and symbolic protocol transmission to the mini-robot. As a result, the Khepera equipped with 2D camera and the additional module is able to autonomously move avoiding collisions with obstacles, called ego-motion. The 30 × 16 SAD operation is processed by computing 16 pairs of pixels in each operation, which accomplishes the real-time approach. The proposed implementation reduces processing time and power consumption.

Global optical flow estimation methods contain a regularization parameter (or prior and likelihood hyper‐parameters if we consider the statistical point of view) which control the tradeoff between the different constraints on the optical flow field. Although experiments (see e.g. Ng et al. [Ng and Solo(1997)]) indicate the importance of the optimal choice of the hyper‐parameters, only little attention has been focused on the optimal choice of these parameters in global motion estimation techniques in literature so far (the authors are only aware of one contribution [Ng and Solo(1997)] which attempts to estimate only the prior hyper‐parameter whereas the likelihood hyper‐parameter needs to be known). We adapt the marginalized maximum a posteriori (MMAP) estimator proposed in [Mohammad‐Djafari(1995)] to simultaneously estimating hyper‐parameters and optical flow for global motion estimation techniques. Experiments demonstrate the performance of this optimization technique and show that the choice of the regularization parameter/hyper‐parameters is an essential key‐point in order to obtain precise motion estimates.

The optical flow techniques have been developed for more than one decade. Once the optical flow field is computed accurately, this measurement of image velocity can be used widely in many tasks in computer vision area. Current computer vision techniques require that the relative errors in the optical flow be less than 10%. However, to reduce error in determination of the optical flow is still a difficult problem. In this paper, firstly, errors occurring in the correlation-based approaches to optical flow computation are analyzed. Through understanding how the errors arise, we developed a new approach to computation of optical flow named the correlation-feedback approach. It is based on the idea of feedback and the correlation-based approach. In this approach, a virtual continuous image is obtained by a bilinear interpolation applied to a digital image. The idea of feedback is used so that errors in determining optical flow are reduced considerably in the iterative procedure. It is proved that the algorithm is convergent generally. Several experiments working on real image sequences in the laboratory demonstrate that our correlation-feedback algorithm performs better than the gradient-based and correlation-based algorithms in terms of accuracy.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The robustness of the human visual system recovering motion estimation in almost any visual situation is enviable, performing enormous calculation tasks continuously, robustly, efficiently, and effortlessly. There is obviously a great deal we can learn from our own visual system. Currently, there are several optical flow algorithms, although none of them deals efficiently with noise, illumination changes, second-order motion, occlusions, and so on. The main contribution of this work is the efficient implementation of a biologically inspired motion algorithm that borrows nature templates as inspiration in the design of architectures and makes use of a specific model of human visual motion perception: Multichannel Gradient Model (McGM). This novel customizable architecture of a neuromorphic robust optical flow can be constructed with FPGA or ASIC device using properties of the cortical motion pathway, constituting a useful framework for building future complex bioinspired systems running in real time with high computational complexity. This work includes the resource usage and performance data, and the comparison with actual systems. This hardware has many application fields like object recognition, navigation, or tracking in difficult environments due to its bioinspired and robustness properties.

Enhanced river suspended sediment concentration identification via multimodal video image, optical flow, and water temperature data fusion

We propose the fast optical flow extractor, a filtering scheme that recovers artifact-free optical flow fields from HEVC-compressed video. To extract accurate optical flow fields, we form a regularized optimization problem that considers the smoothness of the solution and the pixelwise confidence weights of the artifact-ridden HEVC motion field. Solving such an optimization problem is slow, so we first convert the problem into a confidence-weighted filtering task. We then use the extracted optical flow fields for fast synthetic refocus of video using camera motion. By leveraging the already-available HEVC motion information, we achieve a 10-fold speed-up in the running times compared to similar methods. Our fast optical flow extractor is useful since video signals encountered in real life are typically available already in coded formats. Our fast optical flow extractor is general, and works on motion fields of any video coder.