NCC-Net: Normalized Cross Correlation Based Deep Matcher with Robustness to Illumination Variations

Image-based localization, or camera relocalization, is a fundamental problem in computer vision and robotics, and it refers to estimating camera pose from an image. Recent state-of-the-art approaches use learning based methods, such as Random Forests (RFs) and Convolutional Neural Networks (CNNs), to regress for each pixel in the image its corresponding position in the scene's world coordinate frame, and solve the final pose via a RANSAC-based optimization scheme using the predicted correspondences. In this paper, instead of in a patch-based manner, we propose to perform the scene coordinate regression in a full-frame manner to make the computation efficient at test time and, more importantly, to add more global context to the regression process to improve the robustness. To do so, we adopt a fully convolutional encoder-decoder neural network architecture which accepts a whole image as input and produces scene coordinate predictions for all pixels in the image. However, using more global context is prone to overfitting. To alleviate this issue, we propose to use data augmentation to generate more data for training. In addition to the data augmentation in 2D image space, we also augment the data in 3D space. We evaluate our approach on the publicly available 7-Scenes dataset, and experiments show that it has better scene coordinate predictions and achieves state-of-the-art results in localization with improved robustness on the hardest frames (e.g., frames with repeated structures).

Image classification is a fundamental problem in computer vision, which deserves much attention in the past decade. To provide a comprehensive recall for the image classification task based on the deep learning algorithms, this paper first provides a brief history of computer vision and convolutional neural networks. Then, the current state of research and the direction of development of deep learning-based convolutional neural network models for image classification are examined. In addition, the basic model structure, convolution feature extraction, and pooling operations of standard and convolutional neural networks are also introduced. This paper summarizes the development of convolutional neural network models in recent years. In summary, based on the review of the development of image classification algorithms at home and abroad, the current mainstream image classification algorithms and frontier progress are summarized and analyzed, and the existing problems and future development directions of image classification are summarized and prospected. It can be concluded that deep learning-based methods has a great potential in this case.

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .

Co-Detection is an important problem in computer vision, which involves detecting common objects from multiple images. In this paper, we address the co-detection problem and propose an integrated deep learning model involving two networks for co-detection. Our proposed model detects the objects of individual images using a convolutional neural network by generating the saliency maps, which are passed as input in a Siamese neural network to ascertain whether the salient objects in both the images are similar or different. We have tested our model on the iCoseg dataset achieving high-quality results.

As road traffic sign recognition is a crucial component for automatic driver assistance systems, it is a key problem in computer vision as well. Therefore, in this paper, we study on the problem of road traffic sign recognition utilising the computer vision technology. The main innovation of this paper is to propose an improved convolutional neural network, and then use it to tackle the road traffic sign recognition problem. Convolutional neural network can learn features from training data set, and a convolutional network contains alternating layers of convolution and pooling. Particularly, RGB traffic images are transformed to grey scale images, and then grey scale images are input to the improved convolutional neural network. Furthermore, the fixed layers are utilised to discover region of interests, and the learnable layers are used to extract features. In general, output information of the proposed two learnable layers are input to the classifier separately, and parameters of learnable layers and the classifier are trained at the same time. Finally, GTSDB data set is chosen to make performance evaluation, among which 600 images and 300 images are regarded as training and testing data set respectively. Experimental results demonstrate that the improved CNN-based traffic sign recognition performs better than the traditional CNN.

Deep Learning methods are currently the state-of-the-art in many Computer Vision and Image Processing problems, in particular image classification. After years of intensive investigation, a few models matured and became important tools, including Convolutional Neural Networks (CNNs), Siamese and Triplet Networks, Auto-Encoders (AEs) and Generative Adversarial Networks (GANs). The field is fast-paced and there is a lot of terminologies to catch up for those who want to adventure in Deep Learning waters. This paper has the objective to introduce the most fundamental concepts of Deep Learning for Computer Vision in particular CNNs, AEs and GANs, including architectures, inner workings and optimization. We offer an updated description of the theoretical and practical knowledge of working with those models. After that, we describe Siamese and Triplet Networks, not often covered in tutorial papers, as well as review the literature on recent and exciting topics such as visual stylization, pixel-wise prediction and video processing. Finally, we discuss the limitations of Deep Learning for Computer Vision.

Scene text detection is a crucial step in end-to-end scene text recognition, a greatly challenging problem in computer vision. This paper proposes a novel scene text detection method that involves superpixel-based stroke feature transform (SSFT) and deep learning based region classification (DLRC). The SSFT is developed for candidate character region (CCR) extraction, which consists in partitioning an input image into several regions via superpixel-based clustering, removing most regions based on predefined criteria satisfied by the characters, and refining the remaining regions to obtain CCRs by computing a stroke width map. The character regions are identified from the CCRs using DLRC, in which several hand-crafted low-level features, i.e., color, texture, and geometric features, and some deep convolution neural network (CNN) based high-level features are first extracted from the regions, and then these features are fused by using two fully connected networks (FCNs) for region classification. In the DLRC step, the deep feature extraction CNN and the feature fusion FCNs are jointly trained. Next, the extracted character regions are merged to form candidate text regions, from which the final scene texts are detected. The proposed method is evaluated on three publicly available datasets: ICDAR2011, ICDAR2013, and street view text. It achieves F -measures of 0.876, 0.885, and 0.631, respectively, which demonstrate the effectiveness of the proposed scene text detection method.

Predictive analysis on cellular traffic is important for the control and monitoring of wireless networks. Cellular traffic prediction is a challenging problem due to the non- stationarity and dynamic spatial-temporal correlation of the traffic. In this paper, we address the problem of accurate traffic prediction in a base station by proposing a deep neural network called RAConv. Its structure includes residual network, attention mechanism, and deep convolutional network. In the proposed architecture, a deep 3D residual convolutional network (ResConv3D) with three residual blocks are employed to learn the local spatial-temporal features. An attention-aided convolutional long short-term memory network (AConvLSTM) is then used to capture the long-term spatial-temporal dependencies. The use of the attention modules enable the network to focus on the most important spatial-temporal information. We evaluate the performance of the proposed RAConv network using a dataset provided by a Canadian wireless service provider. We consider the traffic prediction on two time scales (i.e., hourly and daily), which exhibit different spatial-temporal dependency patterns. Experimental results show that the proposed RAConv network can achieve accurate prediction under both time scales. Results also show that our proposed network provides a lower root-mean-square error (RMSE) than the conventional ConvLSTM baseline scheme.

Object detection is a fundamental problem in computer vision that involves detecting and localizing objects within an image. In recent years, convolutional neural networks (CNNs) have become the state-of-the-art approach for object detection tasks. YOLO (You Only Look Once) is a popular real-time object detection system that uses a single CNN to predict the bounding boxes and class probabilities for objects in an image. In this paper, we present a human detection and counting system using YOLO. Our system is trained on a large dataset of annotated images and achieves high accuracy and real-time performance. We evaluate our system on a public dataset and demonstrate its effectiveness in various scenarios. This paper focuses on developing a system for human detection and counting using YOLOv4 and Flask API. The system aims to address the increasing importance of human detection and counting in various fields such as social distancing, crowd management, and surveillance.

Constrained Active Region Models for Fast Tracking in Color Image Sequences

We propose a learning method to generate synthetic CT (sCT) image for MRI-only radiation treatment planning. The proposed method integrated a dense-block concept into a cycle-generative adversarial network (cycle-GAN) framework, which is named as dense-cycle-GAN in this study. Compared with GAN, the cycle-GAN includes an inverse transformation between CT (ground truth) and sCT, which could further constrain the learning model. A 2.5D fully convolution neural network (FCN) with dense-block was introduced in generator to enable end-to-end transformation. A FCN is used in discriminator to urge the generator’s sCT to be similar with the ground-truth CT images. The well-trained model was used to generate the sCT of a new MRI. This proposed algorithm was evaluated using 14 patients’ data with both MRI and CT images. The mean absolute error (MAE), peak signal-to-noise ratio (PSNR) and normalized cross correlation (NCC) indexes were used to quantify the correction accuracy of the prediction algorithm. Overall, the MAE, PSNR and NCC were 60.9&minus;11.7 HU, 24.6&plusmn;0.9 dB, and 0.96&plusmn;0.01. We have developed a novel deep learning-based method to generate sCT with a high accuracy. The proposed method makes the sCT comparable to that of the planning CT. With further evaluation and clinical implementation, this method could be a useful tool for MRI-based radiation treatment planning and attenuation correction in a PET/MRI scanner.

Edge detection is a fundamental problem in computer vision and has been explored for many decades. Due to the rapid development of machine learning techniques and their applications to image processing, there is a proliferation of neural network-based approaches to solve the edge detection problem. These methods, such as richer convolutional features for edge detection(RCF), have good performance and even outperform human beings. Most of the existing neural network-based systems use the convolutional network or its variant. They usually produce thick edges and the application of non-maximum-suppression to suppress the edge is necessary. In this paper, we explore another type of neural network called the conditional generative adversarial network (cGAN) to address the edge detection problem. cGAN is an innovative framework to do the image synthesis task. It can generate an image close to the real one. After training, our network can produce an edge map that contains more detailed information and thinner edges compared to the state-of-the-art methods that require the well-known non-maximum-suppression for post-processing. The proposed approach is able to produce a high quality edge map directly without further processing. Our solution is computation efficient. It can achieve a speed of 59 and 26 frames per second (fps) for an image resolution of [256×256×3] and [512×512×3], respectively.

Convolutional Neural Networks (CNNs) have recently been utilized in a wide range of applications, including image and pattern identification, voice recognition, biometric embedded vision, food recognition, and video analysis for surveillance, industrial robots, and autonomous vehicles. Convolutional neural networks (CNNs) are gaining popularity for a variety of reasons. Feature extractors are created by hand in classic image recognition models. The weights of the convolutional layer utilized for feature extraction, as well as the fully connected layer, are used for classification in CNNs, and these weights are set throughout the training phase. The goal of this study is to evaluate a few convolutional neural network (CNN) learning machine approaches for image recognition. Furthermore, machine learning algorithms are used extensively in contemporary approaches to picture recognition. This article focuses on the development trend of convolution neural network (CNNs) model owing to various learning methods in image recognition during the 2000s, which is largely introduced from the angles of capturing, verification, and clustering, based on twenty five journals that have been reviewed. As a result, deep convolutional neural networks (DCNNs) have been very effective in a variety of machine learning and computer vision problems because they provide a large quality gain at a low computational cost. This training approach also enables models with many processing layers to learn different degrees of abstraction for data representation.

In this paper, we propose fully convolutional siamese fusion networks for object tracking. We adopt the fusion strategy of convolutional layers for object tracking to achieve good feature representation based on convolutional neural networks. Specifically, we fuse three convolutional layers of VGGNet based on normalized cross correlation (NCC). First, we use three convolutional layers of VGGNet as the basis for fusion, and reduce the size of the convolutional layers based on a convolution kernel. Then, we resize the convolutional layers to be the same size as the deconvolutional layers for layer fusion. Next, we fuse the three layers based on NCC between the target and search region, and produce the response map. Finally, we get the tracking result from the response map by the maximum response. Various experiments on large-scale data sets verify that the proposed method is robust to occlusion, deformation, motion blur, and background clutter as well as outperforms state-of-the-art trackers in terms of distance precision and overlap success.

Novel view synthesis is an important problem in computer vision and graphics. Over the years a large number of solutions have been put forward to solve the problem. However, the large-baseline novel view synthesis problem is far from being solved. Recent works have attempted to use Convolutional Neural Networks (CNNs) to solve view synthesis tasks. Due to the difficulty of learning scene geometry and interpreting camera motion, CNNs are often unable to generate realistic novel views. In this paper, we present a novel view synthesis approach based on stereo-vision and CNNs that decomposes the problem into two sub-tasks: view dependent geometry estimation and texture inpainting. Both tasks are structured prediction problems that could be effectively learned with CNNs. Experiments on the KITTI Odometry dataset show that our approach is more accurate and significantly faster than the current state-of-the-art. The code and supplementary material will be publicly available. Results could be found here this https URL