Optical flow based super-resolution: A probabilistic approach

Optical flow based super-resolution

Distance estimation from visually simulated self-motion is imprecise. Depending on the evaluation method, travelled distance can be under- or over-estimated. One particular method consists of asking a stationary observer, exposed to an immersive optical flow, simulating forward self-motion, to indicate when s/he thinks s/he has reached the remembered position of a previously seen distant target. In this case, subjective evaluation of travelled distance is generally overestimated (i.e., the subject undershoots the target). Recent studies suggest that a translational optical flow with biological additional oscillations (simulating the optical effects of natural locomotion) would increase the sensation of walking and improve spatial perception, as compared to a purely translational optical flow. In the present study we tested this hypothesis, by measuring travelled distance estimation, according to two conditions of visual simulation of forward self-motion, at constant speed, in a CAVE setup: (1) an optical flow simulating pure forward translation (2) an optical flow with added "biological" oscillations, reproducing the optical effects of the natural motion of the head during walking. Our results show that an optical flow containing additional biological information enhances the accuracy of travelled distance estimation. The perceptual advantage provided by the biological oscillations in the optical flow is discussed.

Describes an FPGA based system for real time vision applications at a robotics vision laboratory. It contains an ALTERA FPGA (20K100), an interface with a digital camera, three VRAM memories to contain the data input and some output memories (a VRAM and a EDO) to contain the results. The aim of this system is to develop and test vision algorithms, such as image compression, optical flow calculation with differential and correlation methods and to hold applications that use previous calculated optical flow, like image invariants detection. The designed system is able to connect the digital camera, or the FPGA output (results of algorithms) to a PC, through its ports USB, IEEE 1394 or CENTRONICS. It also has an expansion connector to make possible the expansion of the system, with some elements of the process, for instance one system would calculate the optical flow of the input images and another system would calculate the focus of expansion in the image and the time to impact in the scene.

Super-resolution imaging techniques reconstruct a high resolution image from a set of low resolution images that are taken from almost the same point of view. The problem can be subdivided into two main parts: an image registration part where the different input images are aligned with each other, and a reconstruction part, where the high resolution image is reconstructed from the aligned images. In this paper, we mainly consider the first step: image registration. We present three frequency domain methods to accurately align a set of undersampled images. First, we describe a registration method for images that have an aliasing-free part in their spectrum. The images are then registered using that aliasing-free part. Next, we present two subspace methods to register completely aliased images. Arbitrary undersampling factors are possible with these methods, but they have an increased computational complexity. In all three methods, we only consider planar shifts. We also show the results of these three algorithms in simulations and practical experiments.

The computation of optical flow has been proposed as a pre-processing step for many high-level vision algorithms. One of the main approaches to the optical flow problem is the gradient-based approach which differentiates the image intensity to compute the optical flow. Often motion vectors computed using various approaches are not reliable. Spatial smoothing of the image sequence is advisable in order to improve velocity estimates in the presence of noise. That is, the application of some kind of linear operation to the images in the sequence before solving for the flow. The pre-processing typically takes the form of some type of spatial Gaussian smoothing or scale specific band-pass filtering of the input images. The main objectives of such filtering have been to lessen the effects of noise, to isolate image structure of interest and to attenuate temporal aliasing and quantization effects in the input images.This paper investigates the effect of presmoothing on this computation of optical flow. The well known method of Horn and Schunck as well as our own finite element method are implemented and tested for improvements due to presmoothing. Discussions are provided on the effects of presmoothing for a variety of image sequences. In our experiments, smoothing is carried out on a selection of well known sequences in both time and space. Improvements are shown using presmoothing.KeywordsFinite Volume MethodsInverse Finite Element MethodsMotion TrackingOptical FlowSpatial SmoothingTemporal Smoothing

The performance of three methods for evaluation of motion on synthesized 2-D echo image sequences with features similar to real ones are examined. The selected techniques based on the computation of optical flow are of the differential type and assume that the image brightness pattern is constant over time. They differ in the choice of the smoothing term and in the local or global treatment of the domain. The images were synthesized by simulating the process of echo formation, considering the interaction between ultrasonic fields and human tissues. Moreover, two different approaches were followed to generate the sequences: 1) a known motion field was applied to the intensity distribution of the synthesized images; 2) a known motion field was applied directly to the point scatterer distribution of the tissue. Favorable results were obtained by applying Lucas-Kanade and Horn-Schunck techniques to the sequences of the first type, while all the techniques produced large errors when applied to the other type of sequences. A discussion about the suitability of the above-mentioned techniques for evaluation of motion on real echocardiographic images is also presented together with some results.

In this paper, investigations of brain activity response from experimental works, based on optic flow and wavelet using near infrared spectroscopy are presented. The outcomes of our research project showed that optical flow is a proper technology for the investigation of brain activity based on near-infrared spectroscopy (NIRS). Our research also showed that the results in the temporal domain, spatial domain, and wavelet domain are coherently supporting the analyses of our experimental results. Our wavelet analysis can obtain the majority of the brain activity for the spatiotemporal distribution. Based on the spatiotemporal analysis, we can clearly observe the existence of the dominant channel (DC), which plays a critical role in the brain activity. The spatial distribution of the cortical activity origin can be described by the hemodynamic response in the cerebral cortex after evoked stimulation using near infrared spectroscopy. Further application is expected in the next research outcomes.

Joint direct estimation of 3D geometry and 3D motion using spatio temporal gradients

Vision-based control is being aggressively pursued for autonomous systems. Such control is particularly valuable for path planning to achieve mission objectives like target tracking and obstacle avoidance. This paper presents a multi-rate strategy that utilises a fast-rate optic flow approach and a slow-rate scene reconstruction approach. The vehicle uses scene reconstruction to generate an accurate map for path planning; however, optic flow is used to avoid obstacles while the scene reconstruction is computed. A switch element is used in the feedback path to determine whether information relating to the reconstructed map or the optical flow should be used for navigation. The resulting controller is able to generate flight trajectories and perform obstacle avoidance within a computational cost which is reasonable given performance demands and computational resources on a wide range of aircraft. A simulation demonstrates the performance of an aircraft that uses the multi-rate controller to avoid an obstacle which is only observed after a turn. Essentially, the fast-rate optic flow indicates the presence of the obstacle during the time that slow-rate scene reconstruction is being performed. The resulting flight path is able to follow mission objectives and avoid a collision.

The fly visual system, although tiny when compared to the mammalian visual system, can still perform highly sophisticated spatial tasks like collision avoidance, landing on objects, pursuit of prey, etc. Flies outperform human-made autonomous flying systems in solving such spatial tasks by a long way. This is partly due to their ability to perceive and respond to optical flow generated by motion in the environment. They are also known to take actions that alter the image flow on their eyes. Higher level neurons in the fly visual system respond to different types of complex optical flows due to rotation and translation, by pooling information from local motion detectors called the elementary motion detectors (EMDs) in the lower level. In this sense, neuronal responses (spikes) from these optical flow detectors in the fly carry highly encoded signals: a single spike can represent a complex dynamical pattern of movement in the visual field. In this paper, we investigate how such highly encoded signals can be interpreted and utilized in the fly's brain, while solely operating on the encoded internal spike patterns within their brain and no direct external (unencoded) sensory information, i.e. a form of grounding. With a computational model of the optical flow detectors based on those in the fly, we show that specific pattern of action (or coordinated motor output) is the only way that a fly can decode its internal spikes and generate meaningful, relevant behavior based on that.

A novel approach for the computation of flow based on an L' type minimization is presented. It is shown that the approach has inherent advantages since it does not smooth the flow-velocity across the edges andhence preserves edge information. A numerical approach based on computation of evolving curves is proposedfor computing the flow field. Computations carried out on a number of real image sequences in order toillustrate the theory as well as the numerical approach.Keywords: curve evolution; flow; motion; scale spaces; gradient descent 1 INTRODUCTION Optical flow field is defined as the velocity vector field of apparent motion of brightness patterns in a sequenceof images. Relative motion between a camera and objects being imaged (or relative motion among the objectsbeing imaged) are very common cases which give rise to flow.One common simplifying assumption is the constancy of intensity at any point in the image and gives rise tothe optical flow constraint (OFC). It can be expressed as the following linear equation in the unknown variablestt

Superresolution (SR) image reconstruction is a technique to obtain a high-resolution (HR) image from a set of low-resolution images. Compact High Resolution Imaging Spectrometer (CHRIS)/Project for On-Board Autonomy (Proba) is a transitional hyperspectral-oriented satellite which acquires multiple angular images of the same scene. The angular images acquired within a very short period of time are ideal for SR operation. Recent developments point to the possibility of a SR-enhanced CHRIS data set at higher resolution. Apparently, the additional details are valuable information for many applications. This letter presents the preliminary evaluation of the SR CHRIS images for ecotope mapping and subpixel classification of sealed surface using two different scenes in Belgium. Accuracy obtained from SR CHRIS images is comparable to that of the original CHRIS, but with significantly more detail in the final classification map. In view of the demands for HR hyperspectral data sets, SR operation can be an interesting option to mitigate the lower spatial resolution of the current and future spaceborne hyperspectral images. Properties such as quick revisiting time and angular acquisition of a hyperspectral satellite are important for the success of SR operations.

In this paper, trade-off between effect of registration error and number of images used in the process of super resolution image reconstruction is studied. Super Resolution image reconstruction is three phase process, of which registration is of at most importance. Super resolution image reconstruction uses set of low resolution images to reconstruct high resolution image during registration. The study demonstrates the effects of registration error and benefit of more number of low resolution images on the quality of reconstructed image. Study reveals that the registration error degrades the reconstructed image and without better registration methodology, a better super resolution method is still not of any use. It is noticed that without further improvement in the registration technique, not much improvement can be achieved by increasing number of input low resolution images.

Despite progress made in the accuracy and robustness of optical flow in past years, the problem of over-segmentation and the blurring of image edge and motion boundary caused by the illumination change, complex texture, large displacement, and motion occlusion still remain. Recently, we developed a guided filtering scheme for flow field estimation, which is implemented as an add-on optimal operation during the coarse-to-fine optical flow computation. In this paper, we first review the research progress in optical flow computation and discuss limitations of the currently popular median filtering heuristic for a flow field optimization. We then introduce a general formulation of the guided filtering and provide the detailed illustration. Furthermore, we explore the potential of the guided filtering optimization for the flow field estimation under the coarse-to-fine computing scheme. Finally, we modify some typical and state-of-the-art optical flow methods by applying the proposed guided filtering operation to the baseline models, and test the performances of the basic and developed models through the Middlebury, MPI-Sintel, and KITTI data. The experimental results demonstrate that the guided filtering scheme is able to preserve the image edges and motion boundaries, and to improve the accuracy and robustness of optical flow estimation.

In this paper, we proposed improved single image super resolution using multiple re-evaluation Process model for use in various image processing fields. The proposed method generates the first super resolution using the input image, and analyzes the change for each region by comparing the features of previous image and super resolution result. According to the analyzed features, the feature map for generate n-th super resolution was selected for improved detail. After then, next generate super resolution using previous super resolution result as input image. This process is repeated for final result. The existing single image super resolution method has two areas to be improved. First, it minimizes artifacts or staircases, which are unnecessary details that can be created during the super resolution process. Second, it is necessary to consider the input image because it affects the result depending on the quality of input image used in the super resolution process. Therefore, in order to minimize unnecessary details, the proposed method analyzed the feature map from the generated super resolution result and applied it according to the amount of change. In addition, aimed to gradually improve the input image to be used in the super resolution process by using the super resolution generated in the previous step. By comparing and evaluating the proposed method with the conventional single image super resolution method with PSNR and SSIM, it is improved by about 3%.