Top-Down Saliency Object Localization Based on Deep-Learned Features

Motivated by the recent success of supervised and weakly supervised common object discovery, in this paper, we move forward one step further to tackle common object discovery in a fully unsupervised way. Generally, object co-localization aims at simultaneously localizing objects of the same class across a group of images. Traditional object localization/detection usually trains specific object detectors which require bounding box annotations of object instances, or at least image-level labels to indicate the presence/absence of objects in an image. Given a collection of images without any annotations, our proposed fully unsupervised method is to simultaneously discover images that contain common objects and also localize common objects in corresponding images. Without requiring to know the total number of common objects, we formulate this unsupervised object discovery as a sub-graph mining problem from a weighted graph of object proposals, where nodes correspond to object proposals, and edges represent the similarities between neighbouring proposals. The positive images and common objects are jointly discovered by finding sub-graphs of strongly connected nodes, with each sub-graph capturing one object pattern. The optimization problem can be efficiently solved by our proposed maximal-flow-based algorithm. Instead of assuming that each image contains only one common object, our proposed solution can better address wild images where each image may contain multiple common objects or even no common object. Moreover, our proposed method can be easily tailored to the task of image retrieval in which the nodes correspond to the similarity between query and reference images. Extensive experiments on PASCAL VOC 2007 and Object Discovery data sets demonstrate that even without any supervision, our approach can discover/localize common objects of various classes in the presence of scale, view point, appearance variation, and partial occlusions. We also conduct broad experiments on image retrieval benchmarks, Holidays and Oxford5k data sets, to show that our proposed method, which considers both the similarity between query and reference images and also similarities among reference images, can help to improve the retrieval results significantly.

This paper proposes a method for estimating 3D information, such as shape, orientation, size, and position of objects in a monocular image, and reproduce scenes in 3D point clouds using Convolutional Neural Network (CNN). This study proposes a network that combines depth estimation, object detection, and point cloud estimation to estimate 3D information of objects. The proposed network requires networks for object detection and segmentation, and a point cloud estimation for object shape estimation. The point cloud estimation network is robust to the reproduction of the object's surface and can deal with unknown objects through a semantic understanding of the object’s shape. In addition to these networks, we combine a depth estimation network for estimating the depth of the entire scene and the distance between the camera and object. In this paper, we consider the point cloud estimation network. We estimate the point clouds for real objects in the images of the dataset and evaluate the output point clouds.

Detection of articulated objects in images, including location and state, is an important and challenging task in many object tracking applications. Image edges have proven to be a key feature, although their quality is influenced by many factors. In this paper, we propose a novel edge gradient-based template matching method for object detection. In contrast to other methods, ours does not perform any binarization or discretization during the online matching. This is facilitated by a new continuous edge gradient similarity measure. Its main components are a novel edge gradient operator, which is applied to query and template images, and the formulation as a convolution, which can be computed very efficiently in Fourier space. Our method consists of a preprocessing stage for the template images, a simple preprocessing of the query image, and our similarity measure between template and query image, which yields a confidence map for the query image. The resulting confidence map can be used for the final object localization. We compared our method to a state-of-the-art chamfer-based matching method. The results demonstrate that our method is much more robust against weak edge response and yields much better confidence maps with fewer maxima that are also more significant. In addition, our method lends itself well to efficient implementation on GPUs: at a query image resolution of 320× 256 and a template resolution of 80× 80 we can generate about 330 confidence maps per second.

Indexing and retrieval of images by spatial constraints

Few-shot segmentation(FSS), which aims to extract never learned classes of objects from query images with a few annotated support samples, is a challenging problem especially in the cases that the appearance of objects in the support and the query images is significant different. Therefore, we propose a deep network called Pyramid Co-Attention Compare Network (PCCNet) to narrow the gap between them by introducing a Pyramid Co-attention Module (PCAM). PCAM acts as a task-specific transformer to transform the features of corresponding objects in query and support images into a space in which they are much closer by taking advantage of the underlying relation between query and support images. We also introduce a Prototypical Guide Module (PGM) which uses non-parametric metric learning to guide parametric metric learning so as to combine the advantages of them. In addition, a Superpixel Refine Module(SRM) is proposed to optimize the final output segmentation masks. Experiments conducted on Pascal-5i shows that our PCCNet achieves a mean Intersection-over-Union(mIoU) score of 63.01% for 1-shot segmentation and 64.57% for 5-shot segmentation, outperforming state-of-the-art methods by margin of 2.2% and 1.6%, respectively.

Saliency detection plays an important role in computer vision area. However, most of the previous works focus on detecting the salient regions, rather than the objects, which is more reasonable in many practical applications. In this paper, a framework is proposed for detecting the salient objects in input images. This framework is composed of two main components: (1) progressively enhanced network (PEN) for amplifying the specified layers of the network and merging the global context simultaneously; (2) object-level boundary extraction module (OBEM) for extracting the complete boundary of the salient object. Experiments and comparisons show that the proposed framework achieves state-of-the-art results. Especially on many challenging datasets, our method performs much better than other methods.

The efficient separation of coal and gangue in the mining process is of great significance for improving coal mining efficiency and reducing environmental pollution. Automatic detection of coal and gangue is the key and foundation for the separation of coal and gangue. In this paper, we proposed a hierarchical framework for coal and gangue detection based on deep learning models. In this framework, the Gaussian pyramid principle is first used to construct multi-level training data, leading to the sets of coal and gangue image features with multiple scales. Then, the coal and gangue regional proposal networks (CG-RPN) are designed to determine the candidate regions of the target objects in the image. Next, convolution neural networks (CNNs) are constructed to recognize coal and gangue objects on the basis of extracted candidate regions. We performed our method on three different datasets. Experimental results showed that the proposed method improves the detection accuracy of coal and gangue objects by 0.8% compared with the previous methods, reaching up to 98.33%. In addition, our proposed method enables the detection of multiple coal and gangue objects in an individual image and solves the problem of queuing requirements in traditional methods.

The deep convolutional neural network has been extensively applied for clinical computer-aided diagnosis. In this study, we combine deep learning feature extraction and eXtreme gradient boosting (XGBoost) classifier for predicting the risk of esophageal varices (EV). First, the quantitative deep learning features and radiomics features of the regions of interest which includes spleen, liver and esophageal are extracted and concatenated. Then, XGBoost and the Least Absolute Shrinkage and Selection Operator (LASSO) are applied for the optimal predictive features selection and prediction of EV risk. XGBoost is used to assess the significance of the extracted features and LASSO is used to select the distinctive features. Finally, random forest, XGBoost and support vector machine classification methods are applied for predicting the low-risk and high-risk of esophageal varices. We collected computed tomography images of cirrhotic patients in two hospitals as the independent training and validation sets. Experimental results show that the features of esophageal are more distinctive than that of other organs. Moreover, the combination of deep learning and radiomics features based on XGBoost algorithm has outperforming classification performance in predicting the severity of EV disease compared to existing approaches.KeywordsEsophageal varicesDeep learningRadiomicsXGBoost

We present an approach for detecting and estimating the 3D poses of objects\nin images that requires only an untextured CAD model and no training phase for\nnew objects. Our approach combines Deep Learning and 3D geometry: It relies on\nan embedding of local 3D geometry to match the CAD models to the input images.\nFor points at the surface of objects, this embedding can be computed directly\nfrom the CAD model; for image locations, we learn to predict it from the image\nitself. This establishes correspondences between 3D points on the CAD model and\n2D locations of the input images. However, many of these correspondences are\nambiguous as many points may have similar local geometries. We show that we can\nuse Mask-RCNN in a class-agnostic way to detect the new objects without\nretraining and thus drastically limit the number of possible correspondences.\nWe can then robustly estimate a 3D pose from these discriminative\ncorrespondences using a RANSAC- like algorithm. We demonstrate the performance\nof this approach on the T-LESS dataset, by using a small number of objects to\nlearn the embedding and testing it on the other objects. Our experiments show\nthat our method is on par or better than previous methods.\n

Rotation invariant HOG for object localization in web images

Contour detection serves as the basis of a variety of computer vision tasks such as image segmentation and object recognition. The mainstream works to address this problem focus on designing engineered gradient features. In this work, we show that contour detection accuracy can be improved by instead making the use of the deep features learned from convolutional neural networks (CNNs). While rather than using the networks as a blackbox feature extractor, we customize the training strategy by partitioning contour (positive) data into subclasses and fitting each subclass by different model parameters. A new loss function, named positive-sharing loss, in which each subclass shares the loss for the whole positive class, is proposed to learn the parameters. Compared to the sofmax loss function, the proposed one, introduces an extra regularizer to emphasizes the losses for the positive and negative classes, which facilitates to explore more discriminative features. Our experimental results demonstrate that learned deep features can achieve top performance on Berkeley Segmentation Dataset and Benchmark (BSDS500) and obtain competitive cross dataset generalization result on the NYUD dataset.

Along with the popularity of intelligent services and mobile services, service recommendation has become a key task, especially the task based on quality-of-service (QoS) in edge computing environment. Most existing service recommendation methods have some serious defects, and cannot be directly adopted in edge computing environment. For example, most of existing methods cannot learn deep features of users or services, but in edge computing environment, there are a variety of devices with different configurations and different functions, and it is necessary to learn deep features behind those complex devices. In order to fully utilize hidden features, this paper proposes a new matrix factorization (MF) model with deep features learning, which integrates a convolutional neural network (CNN). The proposed mode is named Joint CNN-MF (JCM). JCM is capable of using the learned deep latent features of neighbors to infer the features of a user or a service. Meanwhile, to improve the accuracy of neighbors selection, the proposed model contains a novel similarity computation method. CNN learns the neighbors features, forms a feature matrix and infers the features of the target user or target service. We conducted experiments on a real-world service dataset under a batch of cases of data densities, to reflect the complex invocation cases in edge computing environment. The experimental results verify that compared to counterpart methods, our method can consistently achieve higher QoS prediction results.

Image Indexing and Retrieval Using Object-based Point Feature Maps

IntroductionGiven some exemplars, few-shot object counting aims to count the corresponding class objects in query images. However, when there are many target objects or background interference in the query image, some target objects may have occlusion and overlap, which causes a decrease in counting accuracy.MethodsTo overcome the problem, we propose a novel Hough matching feature enhancement network. First, we extract the image feature with a fixed convolutional network and refine it through local self-attention. And we design an exemplar feature aggregation module to enhance the commonality of the exemplar feature. Then, we build a Hough space to vote for candidate object regions. The Hough matching outputs reliable similarity maps between exemplars and the query image. Finally, we augment the query feature with exemplar features according to the similarity maps, and we use a cascade structure to further enhance the query feature.ResultsExperiment results on FSC-147 show that our network performs best compared to the existing methods, and the mean absolute counting error on the test set improves from 14.32 to 12.74.DiscussionAblation experiments demonstrate that Hough matching helps to achieve more accurate counting compared with previous matching methods.

Nowadays Convolutional Neural Networks (CNNs) are being employed in a wide range of industrial technologies for a variety of sectors, such as medical, automotive, aviation, agriculture, space, etc. This paper reviews the state-of-the-art in both the field of CNNs for image classification and object detection and Autonomous Driving Systems (ADSs) in a synergetic way. Layer-based details of CNNs along with parameter and floating-point operation number calculations are outlined. Using an evolutionary approach, the majority of the outstanding image classification CNNs, published in the open literature, is introduced with a focus on their accuracy performance, parameter number, model size, and inference speed, highlighting the progressive developments in convolutional operations. Results of a novel investigation of the convolution types and operations commonly used in CNNs are presented, including a timing analysis aimed at assessing their impact on CNN performance. This extensive experimental study provides new insight into the behaviour of each convolution type in terms of training time, inference time, and layer level decomposition. Building blocks for CNN-based object detection are also discussed, such as backbone networks and baseline types, and then representative state-of-the-art designs are outlined. Experimental results from the literature are summarised for each of the reviewed models. This is followed by an overview of recent ADSs related works and current industry activities, aiming to bridge academic research and industry practice on CNNs and ADSs. Design approaches targeted at solving problems of automakers in achieving real-time implementations are also proposed based on a discussion of design constraints, human vs. machine evaluations and trade-off analysis of accuracy vs. size. Current technologies, promising directions, and expectations from the literature on ADSs are introduced including a comprehensive trade-off analysis from a human-machine perspective.