Bayesian video matting using learnt image priors

We propose a novel computationally efficient correspondence algorithm to identify the correspondence of similar features of the Reference frame to Search frame in the dynamic image sequence analysis for face tracking. The correspondence of the features between the set of images is the central problem in Computer Vision(CV), Image Analysis(IA) and Pattern Recognition(PR). In these areas Correspondence of features is the combinatorial explosion problem or NP-hard, because of their exhaustive searching for features in the sequence of image frames. This correspondence has three major steps such as segmentation, feature extraction and matching. In this paper we propose block-based segmentation in which the frame which contains the face to be track can be segmented as window of size 20*20 which covers the face. Here we consider the window trace mean is the feature since this is rotation invariant and it covers all the rows and columns of the window and it reduces the dimensionality and finally for matching we used the minimum absolute difference method which does not involves more computations since no multiplication operation is involved. We restrict the searching space to [-3,+3] pixels horizontally and vertically in the search image. Experimental results show that this novel algorithm could achieve much higher computational reduction as compared with full search (FS), Diamond search (DS)[6], Cross Diamond Search (CDS)[7,8] and Area Based Correspondence[14] algorithm for face tracking image sequence while similar prediction accuracy is maintained and it is especially suitable for video conferencing and slow moving dynamic image sequence.

The simultaneous recovery of three-dimensional (3D) structure from motion (SfM) for the sequences of images, is one of the more difficult problems in computer vision. Classical approaches to the problem rely on using algebraic techniques to solve for these unknowns given two or more image. Motion analysis and 3D shape estimation based on the estimated motion is an important problem in computer vision. The correspondence problem is an important tool in SfM where in this paper a general 3D motion based on a simple rotation, tilt, roll and translation is proposed, and then is used for 3D shape estimation. The current work expands the 2D motion estimation (shift, rotation) to accommodate general 3D motion (shift, rotation, tilt and roll). The proposed work in this paper of motion estimation of moving object is based on Fourier Descriptor Transformation (FDT) analysis before and after motion. The FDT is also used to resolve the correspondence problem in a sequence of images. We test our method on several large-scale photo collections, show the efficacy of the introduced approach to improve reconstruction accuracy.

Many problems in computer vision and machine learning area can be formulated as image set representation and learning. In real application, image set data often contains various kinds of noises or missing value corruptions, and are also usually sampled from nonlinear manifolds. These make the representation and learning process of image set data more challengeable. This paper proposes a robust manifold regularized low-rank approximation (MLRA) method for image set recovery and representation. MLRA provides an effective low-rank representation for image set whose elements are sampled from nonlinear manifolds. Comparing with original observed image set, MLRA of image set is generally noiseless and more regular, which can obviously encourage the robust learning and recognition process. We evaluate our method on several datasets to demonstrate the benefits of the method.

Disparity estimation belongs to the most important, but difficult, problems in image processing and computer vision. Its importance stems from a wide range of applications, while its difficulty is related to ill-posedness. To date, numerous disparity estimation algorithms have been developed. In this paper, we consider a particular case of disparity estimation based on two views and a known alpha channel partitioning each view into foreground and background. The main idea is to use this partitioning in order to enhance disparity estimation in the foreground object close to its boundary. We propose a block-based disparity model with two alpha-channel constraints: a photometric one, disabling invalid intensity/color matches, and a geometric one, preventing disparity smoothing between foreground and background. We incorporate these constraints into a Bayesian framework using the maximum a posteriori probability criterion. We experimentally demonstrate improvements in the estimated disparities at foreground object boundaries, and show examples of image relighting using these disparities

A sequential dual-energy subtraction technique for image-guided radiation therapy (IGRT) was developed. Here, we report on a computerized method for creating sequential soft-tissue images and the accuracy of tracking targets on the images obtained, in comparison to conventional fluoroscopic images. Two sets of sequential chest images during respiration of a normal subject were obtained with X-rays of different energy separately with a flat-panel detector (FPD). Sequential soft-tissue images were created from the two sets of sequential images consisting of real-time images and reference template images, respectively. The creation of sequential soft-tissue images consisted of three steps: one-to-one image correspondence of the two sequential images, image registration, and image subtraction in each frame. Motion tracking of lung vessels was then performed by the template-matching technique. For evaluation of the accuracy of motion tracking on the sequential soft-tissue images, the results were compared with those on the original sequential images. Sequential soft-tissue images provided more accurate tracking than the original images (P < 0.01). There was no significant error throughout all frames in the soft-tissue images, whereas the rib shadow introduced a tracking error in the original images. The maximum errors were 4.1 +/- 0.3 mm in the sequential soft-tissue images and 28.1 +/- 20.0 mm in the original images. In conclusion, sequential soft-tissue images were helpful for tracking of a target affected by respiratory motion. Dual-energy subtraction has the potential to improve the accuracy of IGRT without implanted markers.

Estimating 3-D structure and camera motion from 2-D image sequences is an important problem in computer vision. In this paper we present an effective approach to tracking and recovery of ego-motion from an image sequence acquired by a single camera attached to a pedestrian. Our approach consists of two stages. In the first phase, human gait analysis is performed and human gait parameters are estimated by frame-by-frame analysis utilising a generalised least squares technique. In the second phase, the gait model is employed within a “predict-correct” framework using a maximum a posteriori expectation maximisation strategy to recover ego-motion and scene structure, while continuously refining the gait model. Experiments on synthetic and real image sequences confirm that the use of the gait model allows for effective tracking while also reducing the computational complexity.KeywordsRoll AngleGait ParameterGeneralise Little SquareFeature TrackingStructure From MotionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

It is a fundamental goal in many modalities of cardiac imaging to measure the global and regional function of the left ventricle through wall motion, thickening and strain, in an effort to isolate the location, severity and extent of ischemic or infarcted myocardium. However, proper characterization of the heart complex motion still remains an open and challenging research problem. Optical flow (OF), the velocity vector field corresponding to the observed motion of brightness patterns in sequences of images, is an important quantity in a variety of problems in computer vision. In magnetic resonance images of the heart this field may provide diagnostic information concerning cardiac muscle motion. A common basic problem in estimating OF concerns how to compute derivative approximations from discrete data. It is, however, well known that derivative estimation is not a well-posed problem. Derivative estimators usually enhance the noise and some sort of smoothing in the data is necessary. This work describes a method to compute OF in 3D using the scale-space approach. The method was applied to cine MR images. A set of ten normal subjects was analyzed in terms of myocardial kinetic energy and ejection fraction.

It is a fundamental goal in many modalities of cardiac imaging to measure the global and regional function of the left ventricle through wall motion, thickening and strain, in an effort to isolate the location, severity and extent of ischemic or infarcted myocardium. However, proper characterization of the heart complex motion still remains an open and challenging research problem. Optical flow (OF), the velocity vector field corresponding to the observed motion of brightness patterns in sequences of images, is an important quantity in a variety of problems in computer vision. In magnetic resonance images of the heart this field may provide diagnostic information concerning cardiac muscle motion. A common basic problem in estimating OF concerns how to compute derivative approximations from discrete data. It is, however, well known that derivative estimation is not a well-posed problem. Derivative estimators usually enhance the noise and some sort of smoothing in the data is necessary. This work describes a method to compute OF in 3D using the scale-space approach. The method was applied to cine MR images. A set often normal subjects was analyzed in terms of myocardial kinetic energy and ejection fraction.

A typical robot vision scenario might involve a vehicle moving with an unknown 3D motion (translation and rotation) while taking intensity images of an arbitrary environment. This paper describes the theory and implementation issues of constructing a sequence of tracked images in which a desired point in the environment is tracked using an image sequence with arbitrary motion. This method is performed fully in software without any need to mechanically move the camera relative to the vehicle. This tracking technique is simple and inexpensive. Furthermore, it does not use either optical flow or feature correspondence. Instead, the spatio- temporal gradients of the input intensity images are used directly. We have also introduced the gradient maps as an intuitive way of visually representing spatio-temporal brightness gradients. We used such gradient maps to qualitatively examine the quality of our constructed image sequence. The experimental results presented support the idea of tracking in software. The final result is a sequence of tracked images where the desired point is kept stationary in the images independent of the nature of the relative motion that occurred.© (1993) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Recognizing activities in image sequences is an open problem in computer vision. In this paper we present a method to extract the most significant frames from an activity sequence. We name these frames as the keyframes. Moreover, we describe a pre-processing stage in order to build a robust representation for different human movements. Using this representation, we build an activity eigenspace that is used to obtain a probability measure. We use this measure to develop a method to select the activity keyframes automatically.KeywordsActivity SequenceActivity RecognitionHuman MovementForeground ObjectAutomatic SelectionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Weakly supervised easy-to-hard learning for object detection in image sequences

Recovering intrinsic images from natural photos is one of the foundational problems in computer vision. This mission always falls into an ill-posed problem. In order to attain reasonable estimations, one strategy is to use multiple images of the scene under various lightings so as to narrow the solution space, whereas another is to utilize priori knowledge as constraints. In this paper, we present an approach to deriving intrinsic images (including illumination images and reflectance images) that employs both strategies. Specifically, the Total Variation (TV) constraint is imposed because of its excellent edge preservation ability and simple parameter settings. To solve this objective function efficiently, we propose using the Alternating Direction Method of Multipliers (AD-MM) to build an iterative numerical scheme. Experimental results illustrate the effectiveness of the proposed model and the numerical scheme.

Automatic spectral video matting

Extraction of man-made objects in images is an important problem in computer vision. In this paper a general procedure to extract man-made objects in colour images of terrains using ‘Background Discriminant Transformation (BDT)’ is presented. BDT is simple to use, robust and application oriented. The procedure does not require extensive knowledge of the terrain. To illustrate the procedure an example is shown from an airborne colour image. However, the process has been also tested, and reported elsewhere, on spaceborne multispectral images such SPOT images. At this stage the process is semi-automatic.

In the areas of computer vision and computer graphics, analyzing information about three-dimensional objects from two-dimensional images is essential. Shape from shading is a very fundamental problem in computer vision, which uses features and patterns in images to infer the shape of the object in our viewing direction. The influence of solving this problem is significant. For example, a typical application of shape from shading is in astronomy, in which we can obtain landscape features of a planet by analyzing the information from the image we get.