Rotation invariance for dense features inside regions of interest

This paper introduces a method for extracting the distinctive feature of rotation invariance from image that shows reliable matching between images of different angle. First, we use the method of Harris corner detection for finding the interest points from the two images to be matched, in which an extreme tactics is adopted for exactly determining the interest points. Follow by adopting a sub-block checking method for eliminating the cluster and reduce the number of interest points. Second, this paper describes an approach to depict the characteristics of interest points which are gained by rotating the area centered the interest point. By this means the rotation invariance of images can implement. Third, the marching method is introduced calculating the similar information among these points, for example the NPROD algorithm. Through a number of experimental images, we prove that this method is proved viable and robust.

Local feature-based multi-object recognition scheme for surveillance

Interest strength assignment to image points is important for selecting good features. Strength assignments using spatial information aim to detect interest points repeatable across different image/illumination transformations, and have been widely adopted in many interest point detectors. Recently, strength assignment schemes using discriminant information received attention, and studies showed the superiority of discriminant strength. In this paper, we introduce a strength assignment scheme integrating spatial and discriminant information, with the motivation that strong spatial information can be helpful in improving the robustness of the discriminant strength estimation, e.g., in undersampled training scenario. Our integrated strength uses a new discriminant strength assignment, so-called locality oriented Fisher criterion score. The integrated strength leads to new methods for feature selection and weighted linear dimensionality reduction. Experimental results in two case studies (embryo developmental stage classification and face recognition) show the favorable performance of the proposed methods.

In this paper we present a new method for object retrieval starting from multiple query images. The use of multiple queries allows for a more expressive formulation of the query object including, e.g., different viewpoints and/or viewing conditions. This, in turn, leads to more diverse and more accurate retrieval results. When no query images are available to the user, they can easily be retrieved from the internet using a standard image search engine. In particular, we propose a new method based on pattern mining. Using the minimal description length principle, we derive the most suitable set of patterns to describe the query object, with patterns corresponding to local feature configurations. This results in a powerful object-specific mid-level image representation. The archive can then be searched efficiently for similar images based on this representation, using a combination of two inverted file systems. Since the patterns already encode local spatial information, good results on several standard image retrieval datasets are obtained even without costly re-ranking based on geometric verification.

In this paper, a fused global saliency-based multiscale multiresolution multistructure local binary pattern (salM 3LBP) feature and local codebookless model (CLM) feature is proposed for high-resolution image scene classification. First, two different but complementary types of descriptors (pixel intensities and differences) are developed to extract global features, characterizing the dominant spatial features in multiple scale, multiple resolution, and multiple structure manner. The micro/macrostructure information and rotation invariance are guaranteed in the global feature extraction process. For dense local feature extraction, CLM is utilized to model local enrichment scale invariant feature transform descriptor and dimension reduction is conducted via joint low-rank learning with support vector machine. Finally, a fused feature representation between salM3LBP and CLM as the scene descriptor to train a kernel-based extreme learning machine for scene classification is presented. The proposed approach is extensively evaluated on three challenging benchmark scene datasets (the 21-class land-use scene, 19-class satellite scene, and a newly available 30-class aerial scene), and the experimental results show that the proposed approach leads to superior classification performance compared with the state-of-the-art classification methods.

This paper presents a novel approach for detecting affine invariant interest points. Our method can deal with significant affine transformations including large scale changes. Such transformations introduce significant changes in the point location as well as in the scale and the shape of the neighbourhood of an interest point. Our approach allows to solve for these problems simultaneously. It is based on three key ideas: 1) The second moment matrix computed in a point can be used to normalize a region in an affine invariant way (skew and stretch). 2) The scale of the local structure is indicated by local extrema of normalized derivatives over scale. 3) An affine-adapted Harris detector determines the location of interest points. A multi-scale version of this detector is used for initialization. An iterative algorithm then modifies location, scale and neighbourhood of each point and converges to affine invariant points. For matching and recognition, the image is characterized by a set of affine invariant points; the affine transformation associated with each point allows the computation of an affine invariant descriptor which is also invariant to affine illumination changes. A quantitative comparison of our detector with existing ones shows a significant improvement in the presence of large affine deformations. Experimental results for wide baseline matching show an excellent performance in the presence of large perspective transformations including significant scale changes. Results for recognition are very good for a database with more than 5000 images.

Expert system for color image retrieval

Similarity-based retrieval of images is an important task in many image database applications. A major class of users' requests requires retrieving those images in the database that are spatially similar to the query image. We propose an algorithm for computing the spatial similarity between two symbolic images. A symbolic image is a logical representation of the original image where the image objects are uniquely labeled with symbolic names. Spatial relationships in a symbolic image are represented as edges in a weighted graph referred to as spatial-orientation graph. Spatial similarity is then quantified in terms of the number of, as well as the extent to which, the edges of the spatial-orientation graph of the database image conform to the corresponding edges of the spatial-orientation graph of the query image. The proposed algorithm is robust in the sense that it can deal with translation, scale, and rotational variances in images. The algorithm has quadratic time complexity in terms of the total number of objects in both the database and query images. We also introduce the idea of quantifying a system's retrieval quality by having an expert specify the expected rank ordering with respect to each query for a set of test queries. This enables us to assess the quality of algorithms comprehensively for retrieval in image databases. The characteristics of the proposed algorithm are compared with those of the previously available algorithms using a testbed of images. The comparison demonstrated that our algorithm is not only more efficient but also provides a rank ordering of images that consistently matches with the expert's expected rank ordering.

Human activity recognition in unconstrained RGB--D videos has extensive applications in surveillance, multimedia data analytics, human-computer interaction, etc, but remains a challenging problem due to the background clutter, camera motion, viewpoint changes, etc. We develop a novel RGB--D activity recognition approach that leverages the dense trajectory feature in RGB videos. By mapping the 2D positions of the dense trajectories from RGB video to the corresponding positions in the depth video, we can recover the 3D trajectory of the tracked interest points, which captures important motion information along the depth direction. To characterize the 3D trajectories, we apply motion boundary histogram (MBH) to depth direction and propose 3D trajectory shape descriptors. Our proposed 3D trajectory feature is a good complementary to dense trajectory feature extracted from RGB video only. The performance evaluation on a challenging unconstrained RGB--D activity recognition dataset, i.e., Hollywood 3D, shows that our proposed method outperforms the baseline methods (STIP-based) significantly, and achieves the state-of-the-art performance.

This paper proposes an effective and efficient visual place recognition (VPR) approach, which can make full use of semantic, sequential and spatial geometric information in VPR tasks. Rather than previous methods focusing on extracting discriminative and compact features to represent images, we improve VPR performance from candidate selection and geometric verification. To this end, we propose a fast feature matching algorithm for real-time geometrical verification of candidate places, termed neighborhood manifold preserving matching (NMP). To generate high-quality candidates, we design a dynamic sequence partitioning strategy based on NMP, which is able to utilize the inherently sequential nature of spatial data to cluster images into places. By sequence-to-sequence matching, the ambiguity of single frame matching can be reduced. Extensive experiments demonstrate that our VPR method outperforms the current state-of-the-art methods, and our geometric verification and candidate selection strategies are easily plugged into other VPR pipelines to significantly improve the VPR performance.

Abstract. Historical aerial photographs directly provide good evidences of past times. The Research Center for Humanities and Social Sciences (RCHSS) of Taiwan Academia Sinica has collected and scanned numerous historical maps and aerial images of Taiwan and China. Some maps or images have been geo-referenced manually, but most of historical aerial images have not been registered since there are no GPS or IMU data for orientation assisting in the past. In our research, we developed an automatic process of matching historical aerial images by SIFT (Scale Invariant Feature Transform) for handling the great quantity of images by computer vision. SIFT is one of the most popular method of image feature extracting and matching. This algorithm extracts extreme values in scale space into invariant image features, which are robust to changing in rotation scale, noise, and illumination. We also use RANSAC (Random sample consensus) to remove outliers, and obtain good conjugated points between photographs. Finally, we manually add control points for registration through least square adjustment based on collinear equation. In the future, we can use image feature points of more photographs to build control image database. Every new image will be treated as query image. If feature points of query image match the features in database, it means that the query image probably is overlapped with control images.With the updating of database, more and more query image can be matched and aligned automatically. Other research about multi-time period environmental changes can be investigated with those geo-referenced temporal spatial data.

This paper presents a multitarget tracking algorithm based on a particle filter framework that exploits a sparse distributed shape model to handle partial occlusions. The state vector is composed by a set of points of interest (i.e. corners) and it enables to jointly describe position and shape of the target. An efficient importance sampling strategy is developed to limit the number of used particles and it is based on multiple Kanade-Lucas-Tomasi (KLT) feature trackers used to estimate local motion. The importance sampling strategy adaptively handles KLT failures and partial occlusions. Particles weights are computed exploiting a shape matching technique combined with object local appearance encoded in color histograms of patches centered on the points of interest constituting the state. The proposed approach does not require background subtraction techniques and overcomes several common difficulties in the tracking domain as partial occlusions, object deformations, scale changes, abrupt motion and non-static background. Extensive experimental results are provided on challenging sequences to demonstrate the robustness of the algorithm.

Modeling the Volumetric Change of Head and Neck Tumor in Response to Radiation Therapy

A video tracking method based on spatial constrained coding (SCC) is proposed in this study. To characterise local image structure information, the dense scale-invariant feature transform (SIFT) descriptor is extracted for each pixel in the image. The proposed tracking method uses SCC model which adopts a new constrained strategy – weighted code, which is achieved by considering the sum of the weighted codes based on grey values of neighbouring pixels and distances between them. The proposed model is able to obtain robust code of corresponding pixels in the frames of complex scenes by taking spatial information into account, which enhances the stability of coding and makes the tracker more robust for object tracking. Twelve challenging sequences involving partial or full occlusion, large pose variation and drastic illumination change are chosen to test the proposed method. The experimental results show the proposed method performs excellent in comparison with other previously proposed trackers.

We propose a novel scale and rotation invariant method to track a human subject's body part regions in cluttered videos. The proposed method optimizes the assembly of body part region proposals with the spatial and temporal constraints of a human body plan. This approach is invariant to the object scale and rotation changes. To enable scale and rotation invariance, the human body part graph of the proposed method has to be loopy, efficiently optimizing the body part region assembly is a great challenge. We propose a dynamic programming method to solve the problem. We devise a method that finds N-best whole body configurations from loopy structures in each video frame using dynamic programming. The N-best configurations are then used to construct trellises with which we track human body part regions by finding shortest paths on the trellises. Our experiments on a variety of videos show that the proposed method is efficient, accurate and robust against object appearance variations, scale and rotation changes and background clutter.