3D mesh interest point detection using GISIFs and heat diffusion

Several machine learning tasks require to represent the data using only a sparse set of interest points. An ideal detector is able to find the corresponding interest points even if the data undergo a transformation typical for a given domain. Since the task is of high practical interest in computer vision, many hand-crafted solutions were proposed. In this paper, we ask a fundamental question: can we learn such detectors from scratch? Since it is often unclear what points are interesting, human labelling cannot be used to find a truly unbiased solution. Therefore, the task requires an unsupervised formulation. We are the first to propose such a formulation: training a neural network to rank points in a transformation-invariant manner. Interest points are then extracted from the top/bottom quantiles of this ranking. We validate our approach on two tasks: standard RGB image interest point detection and challenging cross-modal interest point detection between RGB and depth images. We quantitatively show that our unsupervised method performs better or on-par with baselines.

The performance of matching and object recognition methods based on interest points depends on both the properties of the underlying interest points and the choice of associated image descriptors. This paper demonstrates advantages of using generalized scale-space interest point detectors in this context for selecting a sparse set of points for computing image descriptors for image-based matching. For detecting interest points at any given scale, we make use of the Laplacian $$\nabla ^2_{norm} L$$?norm2L, the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL and four new unsigned or signed Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL, $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL, $${\mathcal {D}}_{2,norm} L$$D2,normL and $$\tilde{\mathcal {D}}_{2,norm} L$$D~2,normL, which are defined by generalizing the definitions of the Harris and Shi-and-Tomasi operators from the second moment matrix to the Hessian matrix. Then, feature selection over different scales is performed either by scale selection from local extrema over scale of scale-normalized derivates or by linking features over scale into feature trajectories and computing a significance measure from an integrated measure of normalized feature strength over scale. A theoretical analysis is presented of the robustness of the differential entities underlying these interest points under image deformations, in terms of invariance properties under affine image deformations or approximations thereof. Disregarding the effect of the rotationally symmetric scale-space smoothing operation, the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL is a truly affine covariant differential entity and the Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL and $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL have a major contribution from the affine covariant determinant of the Hessian, implying that local extrema of these differential entities will be more robust under affine image deformations than local extrema of the Laplacian operator or the Hessian feature strength measures $${\mathcal {D}}_{2,norm} L$$D2,normL, $$\tilde{\mathcal {D}}_{2,norm} L$$D~2,normL. It is shown how these generalized scale-space interest points allow for a higher ratio of correct matches and a lower ratio of false matches compared to previously known interest point detectors within the same class. The best results are obtained using interest points computed with scale linking and with the new Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL, $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL and the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL being the differential entities that lead to the best matching performance under perspective image transformations with significant foreshortening, and better than the more commonly used Laplacian operator, its difference-of-Gaussians approximation or the Harris---Laplace operator. We propose that these generalized scale-space interest points, when accompanied by associated local scale-invariant image descriptors, should allow for better performance of interest point based methods for image-based matching, object recognition and related visual tasks.

Recently, the popularity of user generated videos has highlighted efficient video indexing and browsing as an urgent problem. Points of interest (POI) detection is a technique to address this issue by establishing the implicit relationship among different media resources. The majority of existing studies detect POI by visual similarity, leveraging computer vision techniques. However, these methods suffer from high computational complexity when processing large-scale video sets and are challenging due to the sparse visual correlation among different consumer videos. The advent of geo-referenced videos provides an opportunity to detect POIs in an efficient manner. In this work, we first propose a probability model to formulate the capture intention distribution for video frames. Second, we detect the POIs by considering the contributions from multiple videos. Evaluations demonstrate that our algorithm successfully detects POIs with a reduced error distance from several dozens to a few meters.

In this paper, we compare the results of five 3D interest point detection techniques to the interest points marked by human subjects. This comparison is used to quantitatively evaluate the interest point detection algorithms. We asked human subjects to look at a number of 3D models, and mark interest points on the models via a web-based interface. We propose a voting-based method to construct ground truth out of humans' selections of interest points. Evaluation measures, namely False Positive and False Negative Errors, are then defined based on the geodesic distance between the interest points detected by a particular algorithm and the human-generated ground truth.

In this study, a novel interest point detection algorithm which combines image intensity variation and edge contour information is proposed. Firstly, the Canny edge contour detector is used to extract the edge map. Secondly, the imaginary parts of multi‐scale Gabor filters are applied to smooth input image and then the normalised information entropies at various scales can be acquired. Finally, multiplication of different normalised information entropies will be served as a new measure for interest point that can also be used for interest point detection. This method has two advantages: on the one hand, detection accuracy is greatly improved because combination information is adopted to extract interest points including contour shape information, grey variation of edge pixels and their neighbours; on the other hand, non‐interest points can be well inhabited due to multi‐scale product in detector is served as an interest point measure. Also, desirable noise robustness and time efficiency are validated through experiments. Compared with four other state‐of‐art methods, the proposed method shows excellent performance in terms of geometric transformations and localisation accuracy of repeated interest points.

Research concerning the detection of interest points features for local query is particularly rich and many methods have been proposed in the literature. Matching of interest points is an essential step in the interest point detectors evaluation process. There are many existing methods for interest points matching and most of them are related to the detectors parameters. In this paper, we present a new matching method based on the prediction validation principle by matching interest points with a local description and with adding spatial constraints. Harris [4], SIFT [6] and SURF [1] were used to detect interest-point features from satellite images. This paper presents a short evaluation of these detectors on satellite images with different spatial resolutions, describes the experimental setup and the results of the detected interest points used for the comparison of the detection rate and the repeatability. Finally, we determine which detector leads to the best results for local query on satellite images.

Local feature-based multi-object recognition scheme for surveillance

Many perception problems in robotics such as object recognition, scene understanding, and mapping are tackled using scale-invariant interest points extracted from intensity images. Since interest points describe only local portions of objects and scenes, they offer robustness to clutter, occlusions, and intra-class variation. In this paper, we present an efficient approximate algorithm to extract surface normal interest points (SNIPs) in corners and blob-like surface regions from depth images. The interest points are detected on characteristic scales that indicate their spatial extent. Our method is able to cope with irregularly sampled, noisy measurements which are typical to depth imaging devices. It also offers a trade-off between computational speed and accuracy which allows our approach to be applicable in a wide range of problem sets. We evaluate our approach on depth images of basic geometric shapes, more complex objects, and indoor scenes.

A point of interest is the characteristic of an image which can be robustly detected due to its well-defined position. The points of interest should be easily computable and are invariant to transformations in the image domain. This paper presents a novel interest point detector based on a graph centrality measure defined over the Laplacian energy of the local image network. The proposed method considers the local circular neighborhood of each pixel and associates this with an undirected graph or network. Euclidean distance is employed to connect pixels to form graph edges. The Laplacian energy of the graph will drop when a vertex is removed from it. This relative reduction indicates the significance of that vertex in the network. Graph vertices having higher values of Laplacian centrality are identified as central most nodes. This measure of pixel significance indicates the presence of interest points. The effectiveness and robustness of the proposed approach in detecting local points of high entropy values are demonstrated by the experiments conducted on standard datasets. The visual assessment of the identified interest points correlates with the coverage value calculated. Moreover, the proposed method shows stable results in the presence of geometric transformations such as rotation, viewpoint changes, and zooms. These results provide substantial evidence of the proposed interest point detector's utility in image processing applications such as image registration, object recognition, and image matching, etc.

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

This paper proposes a detector for spatio-temporal interest points. Interest point detection is a common technique in computer vision to extract salient regions and represent them by a single point for further processing. But while many algorithms exist for static images, there is hardly any method to obtain interest points from image sequences for the representation of salient motion. Here we introduce SUSANinTime, an extension of the well known SUSAN algorithm from 2D to 2D+1D, where the third dimension is time. While SUSAN-2D extracts edge- and corner points, SUSANinTime detects basic events such as turning points of object trajectories. To find out the type and saliency of the detected events, we analyze the second order statistics of the spatio-temporal volume surrounding the interest points in real world image sequences.KeywordsImage RetrievalInterest PointSalient RegionSudden AppearanceInterest Point DetectionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A problem of interest point detection in camera images, i.e. detection of points with high information content is studied. An overview of existing techniques is given. A modified Harris-Laplace interest point detector is proposed and experimental results are given. Proposed detector generates a smaller number of detected points, resulting in less complex matching. The aim is to use interest point detectors, descriptors and matching for mobile robot navigation.

This paper presents an algorithm for automated extraction of interest points (IPs)in multispectral and hyperspectral images. Interest points are features of the image that capture information from its neighbours and they are distinctive and stable under transformations such as translation and rotation. Interest-point operators for monochromatic images were proposed more than a decade ago and have since been studied extensively. IPs have been applied to diverse problems in computer vision, including image matching, recognition, registration, 3D reconstruction, change detection, and content-based image retrieval. Interest points are helpful in data reduction, and reduce the computational burden of various algorithms (like registration, object detection, 3D reconstruction etc) by replacing an exhaustive search over the entire image domain by a probe into a concise set of highly informative points. An interest operator seeks out points in an image that are structurally distinct, invariant to imaging conditions, stable under geometric transformation, and interpretable which are good candidates for interest points. Our approach extends ideas from Lowe's keypoint operator that uses local extrema of Difference of Gaussian (DoG) operator at multiple scales to detect interest point in gray level images. The proposed approach extends Lowe's method by direct conversion of scalar operations such as scale-space generation, and extreme point detection into operations that take the vector nature of the image into consideration. Experimental results with RGB and hyperspectral images which demonstrate the potential of the method for this application and the potential improvements of a fully vectorial approach over band-by-band approaches described in the literature.

To improve the real-time performance, a quick scale-invariant interest point detecting approach based on the image color information is proposed in this paper. The approach uses the scale normalized Laplacian operator to extract the interest points in the incomplete image pyramid. A new local descriptor is presented in the approach to compute the feature vector of each interest point. The descriptor is made up with several subregions like the SIFT (Scale-Invariant Feature Transform) descriptor, meanwhile, it chooses the mean values of different color components in each subregion as the feature vector’s elements to differentiate color objects better and reduce the descriptor’s dimension. Through the experiment, the detected interest points are robust to many image transformations and the approach is indicative of needing less computation than other interest point detecting algorithms. The research discloses that the approach can obtain both superior stability and real-time performance at the same time.

Human action recognition has been a challenging task in computer vision because of intra-class variability. State-of-the-art methods have shown good performance for constrained videos but have failed to achieve good results for complex scenes. Reasons for their failing include treating spatial and temporal dimensions without distinction as well as not capturing temporal information in video representation. To address these problems we propose principled changes to an action recognition framework that is based on video interest points (IP) detection with capturing differential motion as the central theme. First, we propose to detect points with high curl of optical flow, which captures relative motion boundaries in a frame. We track these points to form dense trajectories. Second, we discard points on the trajectories that do not represent change in motion of the same object, yielding temporally localized IPs. Third, we propose a video representation based on spatio-temporal arrangement of IPs with respect to their neighboring IPs. The proposed approach yields a compact and information-dense representation without using any local descriptor around the detected IPs. It significantly outperforms state-of-the-art methods on UCF youtube dataset, which has complex action classes, as well as on KTH dataset, which has simple action classes.