Contour Detection and Hierarchical Image Segmentation

The object contour detection in natural scenes is an important research problem in computer vision. The difficulty is that the integrity of contour extracted is interfered with a large number of texture edges in the background seriously. In recent years, some researchers have introduced image contour detection method with biological visual features and achieved preferable results. Such as, based on the fact of visual outer region suppression, they can suppress a certain amount of texture edge while extracting the contour of the object, and obtain the proper contour edge. However, when this method is used for contour detection in some image which the texture is similar to the contour, the texture edge still cannot be removed well, and the result is unsatisfactory. In consideration of the merit and demerit of the feature suppressed method, we propose an improved algorithm for object contour detection in natural scene based on image saliency. Firstly, the initial contour is obtained by using the suppression effect of non-classical receptive field, and the saliency map is obtained by using the hyper-complex Fourier transform image saliency detection method, and then the initial contour and the saliency map are integrated into the final contour. Compared with the traditional contour detection method, the contour extraction algorithm based on image saliency can remove more background information and improve the accuracy and integrity of object contour extracted in natural scenes.

Contour detection via stacking random forest learning

Contour detection has been extensively investigated as a fundamental problem in computer vision. In this study, a biologically-inspired candidate weighting framework is proposed for the challenging task of detecting meaningful contours. In contrast to previous models that detect contours from pixels, a modified superpixel generation processing is proposed to generate a contour candidate set and then weigh the candidates by extracting hierarchical visual cues. We extract the low-level visual local cues to weigh the contour intrinsic property and mid-level visual cues on the basis of Gestalt principles for weighting the contour grouping constraint. Experimental results tested on the BSDS benchmark show that the proposed framework exhibits promising performances to capture meaningful contours in complex scenes.

Contour detection is a fundamental problem in computer vision. However, there is still a considerable disparity between detection results and actual contours. To detect object-level contours on the basis of comprehensive analysis of potential edges, we present a deep-learning-based approach with a conditional random fields (CRF) model. We obtain the initial edgemap with a VGGNet-based model, and establish a contour correlation graph (CCG) to describe the potential edges and their relationships. Then a CRF model is adopted to fulfill the optimization on the basis of the analysis of object-level contour characteristics and to predict the validity of candidate contours. The experiment results on various datasets demonstrate that the proposed method outperforms the existing methods by a noticeable margin.

Graph and hypergraph matching are fundamental problems in computer vision, which have been receiving a steadily growing attention in the last two decades due to the recent modern algorithms and models that are both fast and accurate. Graph and hypergraph matching are successful in tackling many real-world tasks that require 2D and 3D feature matching and alignment. Matching is a general problem in vision and graphs have always been quintessential for modeling information that has both local and longer range connections. Therefore, graph matching will continue to influence computer vision and robotics research, crossing over many different mathematical and computational models, including Bayesian networks, conditional and Markov random fields, spectral graph models, and, more recently, deep neural networks. While graph matching is limited to using pairwise relationships, hypergraph matching permits the use of relationships between sets of features of any order. Consequently, it carries the promise to make matching more robust to changes in scale, deformations, and outliers. In this chapter, we present several key ideas, computational models, and optimization methods that introduced in the literature the possibility to learn graph and hypergraph matching in a semi-supervised or even completely unsupervised fashion. We start by describing two methods for solving graph and hypergraph matching based on spectral matching and integer projected fixed point methods, introduced in Chap. 1. Then, based on the same intuition (also discussed in Chap. 1) that correct assignments form a strong cluster of agreements, we follow up by introducing the main approach to learning as it applies to graph and hypergraph matching, as well as the more general Maximum A Posteriori (MAP) inference problem in probabilistic graphical models (MRFs and CRFs). In the last part of the chapter, we demonstrate experimentally the effectiveness of the algorithms presented and, whenever possible, provide extensive comparisons to other top published methods. Moreover, we show that our learning approach is general and can significantly boost the performance of different state-of-the-art matching algorithms in the literature.

In the areas of computer vision and computer graphics, analyzing information about three-dimensional objects from two-dimensional images is essential. Shape from shading is a very fundamental problem in computer vision, which uses features and patterns in images to infer the shape of the object in our viewing direction. The influence of solving this problem is significant. For example, a typical application of shape from shading is in astronomy, in which we can obtain landscape features of a planet by analyzing the information from the image we get.

Image segmentation is a fundamental problem in image processing and computer vision. Its goal is to separate an image into a collection of distinct regions, after which other high-level tasks can be performed. Nomalized cut (Ncut) algorithm is the most popular one in image segmentation algorithms. However, the number of segmentation regions needs to be specified by users or experts before the Ncut algorithm is applied to segment the image. It is a shortcoming and also one of the most important factors of bad performance in images recognition and annotation. In this paper, we propose a novel method to adjust the number of regions with two-stage category. In first stage, an initial number is determined by the rank of image richness. In this stage, the gray-level frequency information of image, Marr wavelet transform and extremal detection are employed to get the rank. In second stage, the number of segmentation regions will be re-adjusted with the regional feature consistency measure criterion. Compared with the traditional Ncut algorithm, our method is completely automated, which greatly improves the versatility of the algorithm. The experiments are conducted on the dataset with 1000 images released by Achanta. Experimental results show that the proposed algorithm achieves marked improvements in performance and outperforms traditional Ncut algorithm.

Image segmentation is a significant problem in computer vision that aims at segmenting an image into appropriate segments based on meanings in the picture. In this work, we investigate on two advanced artificial intelligence algorithms that are deep learning architectures and some new attention mechanisms to improve effectiveness of image segmentation tasks. That is why we introduce a new architecture, called TransCNN-Seg, which utilizes the global attention mechanisms of the Transformer network and combines them with local feature extraction of the CNNs. This integration utilizes the advantages of the Transformer model which has a connection with expressive modeling of long-distance relationships and the CNN for its capacity to effectively capture detailed location features. The segmentation discussed in this paper uses a multi-stage segmentation approach and is based on Vision Transformer which is adopted with spatial-channel attention modules and an enhanced decoder design defined by the use of attention-gated skip-connections. This architecture shows desirable properties in answering complicated issues in segmentation, especially in comparably difficult fields such as segmentation of tumors in various medical imaging applications as well as segmentation of road scenes for self-driving cars. On standard performance measurement criteria, which is mean Intersection over Union (mIoU), TransCNN-Seg is able to obtain an mIoU of at least 83.7%, which is 2.8% better than previous state of the art methods, when the proposed model is tested using the Cityscapes datasets and a standardized medical imaging dataset. There are 13% and 5% improvements over pure CNN counterpart in the Boundary F1-score of the regions within and outside occlusion on average over all tested methods and 10% improvement of segmentation difference across all tested methods in Concave F1-score.

Image segmentation is one of the most important problems in computer vision. Recently, Fairfield proposed an interesting approach to image segmentation using toboggan enhancement followed by naive contrast segmentation, which is a noniterative, linear execution time method. The way it operates can be thought of as a man tobogganing in the first derivative terrain, i.e., the graph surface of a discontinuity measure computed by the first derivative of the image intensity. The segmentation results it produced appeared equal in quality to that of other complex optimal region growing methods. In this paper, an improved version of Fairfield's method, called keep-sliding toboggan segmentation, is presented. With our method, the toboggan will keep sliding on a plane in the derivative terrain, where the original toboggan method will stop sliding. Therefore, our method produces far less regions than the original. Other improvements achieved are as follows: Instead of being followed by contrast segmentation post-process, our keep-sliding tobogganing process is preceded by a prefiltering process which suppresses small fluctuations in the first derivative terrain. Because of this prefiltering operation, our tobogganing process can automatically merge the regions having small intercontrast. Also, a new discontinuity measure is proposed to allow the detection of small target regions without ever-segmenting the images. Experimental results indicate that the segmentations produced by the keep-sliding toboggan method are less noisy, and, therefore, it is more appropriate to use them as initial segmentations for higher level image segmentation techniques.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Image classification is a fundamental problem in computer vision, which deserves much attention in the past decade. To provide a comprehensive recall for the image classification task based on the deep learning algorithms, this paper first provides a brief history of computer vision and convolutional neural networks. Then, the current state of research and the direction of development of deep learning-based convolutional neural network models for image classification are examined. In addition, the basic model structure, convolution feature extraction, and pooling operations of standard and convolutional neural networks are also introduced. This paper summarizes the development of convolutional neural network models in recent years. In summary, based on the review of the development of image classification algorithms at home and abroad, the current mainstream image classification algorithms and frontier progress are summarized and analyzed, and the existing problems and future development directions of image classification are summarized and prospected. It can be concluded that deep learning-based methods has a great potential in this case.

Constrained Active Region Models for Fast Tracking in Color Image Sequences

Finding the surface of a volumetric 3D object is a fundamental problem in computer vision. Energy minimizing splines, such as active surfaces, have been used to carry out such tasks, evolving under the influence of internal and external energies until the model converges to a desired surface. The present deformable model based surface extraction techniques are computationally expensive and are generally unreliable in identifying the surfaces of noisy, high-curvature and cluttered 3D objects. This paper proposes a novel decoupled active surface (DAS) for identifying the surface of volumetric 3D objects. The proposed DAS introduces two novel aspects which leads to robust, efficient and accurate convergence. First, rather than a parameterized surface, which leads to difficulties with complex shapes and parameter singularities, the DAS uses a conforming triangular mesh to represent the surface. Second, motivated by earlier successes in two-dimensional segmentation, the DAS treats the two energy components separately and uses novel solution techniques to efficiently minimize the two energy terms separately. The performance of DAS in segmenting static 3D objects is presented using several natural and synthetic volumetric images, with excellent convergence results.

Maximum flow and minimum s-t cut algorithms are used to solve several fundamental problems in computer vision. These problems have special structure, and standard techniques perform worse than the special-purpose Boykov-Kolmogorov (BK) algorithm. We introduce the incremental breadth-first search (IBFS) method, which uses ideas from BK but augments on shortest paths. IBFS is theoretically justified (runs in polynomial time) and usually outperforms BK on vision problems.

Novel view synthesis is a classic problem in computer vision. It refers to the generation of previously unseen views of a scene from a set of sparse input images taken from different viewpoints. One example of novel view synthesis is the interpolation of views in between the two images of a stereo camera. Another classic problem in computer vision is video frame interpolation, which is important for video processing. It refers to the generation of video frames in between existing ones and is commonly used to increase the frame rate of a video or to match the frame rate to the refresh rate of the monitor that the video is being displayed on. Interestingly, off-the-shelf video frame interpolation can directly be employed to successfully perform view interpolation to address the aforementioned stereo view interpolation problem. Video frame interpolation can be seen as temporal novel view synthesis. However, this perspective is usually not considered and novel view synthesis generally concerns generating unseen views in space rather than time. For this reason, the set of sparse input images that is used for spatial novel view synthesis is commonly either captured at the same time, or it is assumed that the scene is static. This paradigm limits the applicability of novel view synthesis in real-world scenarios though. This thesis addresses three applications of novel view synthesis and provides practical solutions that do not require difficult-to-acquire multi-view imagery: video frame interpolation which performs temporal video-to-video synthesis, synthesizing the 3D Ken Burns effect from a single image which performs spatial image-to-video synthesis, synthesizing video action shots which performs spatiotemporal video-to-video and video-to-image synthesis. These applications not only explore different dimensions of time and space, they also perform novel view synthesis on everyday image and video footage. This is in stark contrast to the large body of existing work which focuses on spatial novel view synthesis while requiring multiple input views that were either captured at the same time or under the assumption of a static scene.

Over the last few years energy minimization has emerged as an indispensable tool in computer vision. The primary reason for this rising popularity has been the successes of efficient graph cut based minimization algorithms in solving many low level vision problems such as image segmentation, object reconstruction, image restoration and disparity estimation. The scale and form of computer vision problems introduce many challenges in energy minimization. In this chapter we address the problem of efficient and exact minimization of groups of similar functions which are known to be solvable in polynomial time. We will present a novel dynamic algorithm for minimizing such functions. This algorithm reuses computation from previous problem instances to solve new instances resulting in a substantial improvement in the running time. We will present the results of using this approach on the problems of interactive image segmentation, image segmentation in video, human pose estimation and segmentation, and measuring uncertainty of solutions obtained by minimizing energy functions.