Adversarial Network for edge detection

Edge detection is a basic problem in computer vision and image processing. The main purpose of edge detection is to identify points with obvious brightness changes in digital images. At present, there are many good detection methods, but most of them do not consider the correctness and crispness of edges at the same time. In order to address this problem, this paper proposes a method based on a deep convolution neural network. The method is mainly based on a dense network that is then combined with the single network structure of a backward refinement path module. The former can detect and retain the feature information between different layers in an image. The latter makes full use of the extracted information so that the low-level detail features and high-level abstract features can be better integrated in the final output. We tested the method on the BIPED data set. The results show that the correctness and crispness of the edges can be balanced in the detection process, and the ODS, OIS and AP of this method reach 0.888, 0.893 and 0.916, respectively. Compared with the state-of-the-art approaches, the proposed method improves the standard evaluation by 3%-5%, and the convergence speed is also significantly improved.

Image-based localization, or camera relocalization, is a fundamental problem in computer vision and robotics, and it refers to estimating camera pose from an image. Recent state-of-the-art approaches use learning based methods, such as Random Forests (RFs) and Convolutional Neural Networks (CNNs), to regress for each pixel in the image its corresponding position in the scene's world coordinate frame, and solve the final pose via a RANSAC-based optimization scheme using the predicted correspondences. In this paper, instead of in a patch-based manner, we propose to perform the scene coordinate regression in a full-frame manner to make the computation efficient at test time and, more importantly, to add more global context to the regression process to improve the robustness. To do so, we adopt a fully convolutional encoder-decoder neural network architecture which accepts a whole image as input and produces scene coordinate predictions for all pixels in the image. However, using more global context is prone to overfitting. To alleviate this issue, we propose to use data augmentation to generate more data for training. In addition to the data augmentation in 2D image space, we also augment the data in 3D space. We evaluate our approach on the publicly available 7-Scenes dataset, and experiments show that it has better scene coordinate predictions and achieves state-of-the-art results in localization with improved robustness on the hardest frames (e.g., frames with repeated structures).

Image classification is a fundamental problem in computer vision, which deserves much attention in the past decade. To provide a comprehensive recall for the image classification task based on the deep learning algorithms, this paper first provides a brief history of computer vision and convolutional neural networks. Then, the current state of research and the direction of development of deep learning-based convolutional neural network models for image classification are examined. In addition, the basic model structure, convolution feature extraction, and pooling operations of standard and convolutional neural networks are also introduced. This paper summarizes the development of convolutional neural network models in recent years. In summary, based on the review of the development of image classification algorithms at home and abroad, the current mainstream image classification algorithms and frontier progress are summarized and analyzed, and the existing problems and future development directions of image classification are summarized and prospected. It can be concluded that deep learning-based methods has a great potential in this case.

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .

As road traffic sign recognition is a crucial component for automatic driver assistance systems, it is a key problem in computer vision as well. Therefore, in this paper, we study on the problem of road traffic sign recognition utilising the computer vision technology. The main innovation of this paper is to propose an improved convolutional neural network, and then use it to tackle the road traffic sign recognition problem. Convolutional neural network can learn features from training data set, and a convolutional network contains alternating layers of convolution and pooling. Particularly, RGB traffic images are transformed to grey scale images, and then grey scale images are input to the improved convolutional neural network. Furthermore, the fixed layers are utilised to discover region of interests, and the learnable layers are used to extract features. In general, output information of the proposed two learnable layers are input to the classifier separately, and parameters of learnable layers and the classifier are trained at the same time. Finally, GTSDB data set is chosen to make performance evaluation, among which 600 images and 300 images are regarded as training and testing data set respectively. Experimental results demonstrate that the improved CNN-based traffic sign recognition performs better than the traditional CNN.

Scene text detection is a crucial step in end-to-end scene text recognition, a greatly challenging problem in computer vision. This paper proposes a novel scene text detection method that involves superpixel-based stroke feature transform (SSFT) and deep learning based region classification (DLRC). The SSFT is developed for candidate character region (CCR) extraction, which consists in partitioning an input image into several regions via superpixel-based clustering, removing most regions based on predefined criteria satisfied by the characters, and refining the remaining regions to obtain CCRs by computing a stroke width map. The character regions are identified from the CCRs using DLRC, in which several hand-crafted low-level features, i.e., color, texture, and geometric features, and some deep convolution neural network (CNN) based high-level features are first extracted from the regions, and then these features are fused by using two fully connected networks (FCNs) for region classification. In the DLRC step, the deep feature extraction CNN and the feature fusion FCNs are jointly trained. Next, the extracted character regions are merged to form candidate text regions, from which the final scene texts are detected. The proposed method is evaluated on three publicly available datasets: ICDAR2011, ICDAR2013, and street view text. It achieves F -measures of 0.876, 0.885, and 0.631, respectively, which demonstrate the effectiveness of the proposed scene text detection method.

Object detection is a fundamental problem in computer vision that involves detecting and localizing objects within an image. In recent years, convolutional neural networks (CNNs) have become the state-of-the-art approach for object detection tasks. YOLO (You Only Look Once) is a popular real-time object detection system that uses a single CNN to predict the bounding boxes and class probabilities for objects in an image. In this paper, we present a human detection and counting system using YOLO. Our system is trained on a large dataset of annotated images and achieves high accuracy and real-time performance. We evaluate our system on a public dataset and demonstrate its effectiveness in various scenarios. This paper focuses on developing a system for human detection and counting using YOLOv4 and Flask API. The system aims to address the increasing importance of human detection and counting in various fields such as social distancing, crowd management, and surveillance.

The task of matching image patches is a fundamental problem in computer vision. When sufficiently textured patches are normalized up to similarity transformation, a simple Normalized Cross Correlation (NCC) of corresponding patches will give a high value. In practice, using it on patches per se may not perform well due to the noisy variations of pixel intensities. A more prudent approach will be to apply it to the abstract features extracted by a deep convolutional network. We study the applicability of an NCC based convolutional network for the task of Patch Matching. Further, there may be cases where the network may fail due to insufficient textures. In those cases, a simple pixel difference based method will be beneficial. To this end, we propose to improve the two basic architectures, Siamese networks and Central-Surround stream networks, using robust matching layers for learning the similarities of patches, assisted by a simple cross-entropy loss function. We empirically verify the performance of the proposed models on the challenging UBC Patches dataset and show that they are close to the state-of-the-art. Further, we evaluate their resilience to large illumination changes in two experimental scenarios: 1) by manually varying the patches of UBC Patches by an affine model 2) by using the publicly available Webcam dataset. We demonstrate that our models are indeed very resilient to illumination variations; they reduce the false positive rate to nearly 10%, and improve over the popular methods by nearly 5%. Further, we demonstrate the generalisability of the proposed NCC based matching layer by applying it to Face Recognition and show that it improves the performances of well known networks on a real-world, surveillance dataset.

Graph and hypergraph matching are fundamental problems in computer vision, which have been receiving a steadily growing attention in the last two decades due to the recent modern algorithms and models that are both fast and accurate. Graph and hypergraph matching are successful in tackling many real-world tasks that require 2D and 3D feature matching and alignment. Matching is a general problem in vision and graphs have always been quintessential for modeling information that has both local and longer range connections. Therefore, graph matching will continue to influence computer vision and robotics research, crossing over many different mathematical and computational models, including Bayesian networks, conditional and Markov random fields, spectral graph models, and, more recently, deep neural networks. While graph matching is limited to using pairwise relationships, hypergraph matching permits the use of relationships between sets of features of any order. Consequently, it carries the promise to make matching more robust to changes in scale, deformations, and outliers. In this chapter, we present several key ideas, computational models, and optimization methods that introduced in the literature the possibility to learn graph and hypergraph matching in a semi-supervised or even completely unsupervised fashion. We start by describing two methods for solving graph and hypergraph matching based on spectral matching and integer projected fixed point methods, introduced in Chap. 1. Then, based on the same intuition (also discussed in Chap. 1) that correct assignments form a strong cluster of agreements, we follow up by introducing the main approach to learning as it applies to graph and hypergraph matching, as well as the more general Maximum A Posteriori (MAP) inference problem in probabilistic graphical models (MRFs and CRFs). In the last part of the chapter, we demonstrate experimentally the effectiveness of the algorithms presented and, whenever possible, provide extensive comparisons to other top published methods. Moreover, we show that our learning approach is general and can significantly boost the performance of different state-of-the-art matching algorithms in the literature.

Novel view synthesis is an important problem in computer vision and graphics. Over the years a large number of solutions have been put forward to solve the problem. However, the large-baseline novel view synthesis problem is far from being solved. Recent works have attempted to use Convolutional Neural Networks (CNNs) to solve view synthesis tasks. Due to the difficulty of learning scene geometry and interpreting camera motion, CNNs are often unable to generate realistic novel views. In this paper, we present a novel view synthesis approach based on stereo-vision and CNNs that decomposes the problem into two sub-tasks: view dependent geometry estimation and texture inpainting. Both tasks are structured prediction problems that could be effectively learned with CNNs. Experiments on the KITTI Odometry dataset show that our approach is more accurate and significantly faster than the current state-of-the-art. The code and supplementary material will be publicly available. Results could be found here this https URL

The article considers the possibilities of using the deep learning convolutional neural network ResNet in computer vision and image classification problems. The interpretation of the ResNet network and the datasets used for its training are presented, as well as a method for training a deep convolutional neural network with stochastic depth, which allows significantly reducing errors in the test sample.

In the areas of computer vision and computer graphics, analyzing information about three-dimensional objects from two-dimensional images is essential. Shape from shading is a very fundamental problem in computer vision, which uses features and patterns in images to infer the shape of the object in our viewing direction. The influence of solving this problem is significant. For example, a typical application of shape from shading is in astronomy, in which we can obtain landscape features of a planet by analyzing the information from the image we get.

Pose estimation of a three-dimensional (3D) rigid object based on monocular vision is a fundamental problem in computer vision. Existing methods usually presume that there is a certain feature projection correspondence available a priori between the input image and object's 3D model, which substantially reduces the mathematical complexity of the problem. However, in actual applications this presumption rarely holds. How to estimate the pose of a general 3D object with no feature projection correspondence available a priori is still not well resolved. In this study, the authors propose a new 3D rigid object pose estimation method based on object's contour and non-Euclidean multi-feature distance map, which solves both the pose estimation problem and the feature projection correspondence problem simultaneously and iteratively. Experiment results show that our method also has good performance in convergence speed, convergence radius and noise robustness.

\n Hand pose estimation is a challenging problem in computer vision with a wide range of applications, especially in human-computer interface. With the development of inexpensive consumer-level depth cameras and the evolution on deep learning techniques, the current state-of-art in the problem is continuously developing and several new methods have been proposed in recent years. Those methods are mostly data-driven and reach good results in standard datasets such as NYU, ICVL and HANDS17. An application that would benefit from the use of computer vision techniques is hand occupational therapy. In chronic diseases like rheumatoid arthritis (RA), the evaluation of the hand functional state is fundamental for the treatment and prevention of finger deformities. One of the procedures for deformity diagnosis is the measurement of movement angles i.e. flexion/extension and abduction/addution, made using goniometers in a process that can be time-consuming and invasive for the patient. The main proposal of this PhD is to fill a gap in the literature by proposing and evaluating the viability of using a framework composed of a 3D low-cost sensor and a 3D hand pose estimation state-of-art method for automatic assessment of rheumatoid arthritis patients. Given depth maps as input, our framework estimates 3D hand joint positions using a deep convolutional neural network. The proposed pose estimation algorithm can be executed in real-time, allowing users to visualise 3D skeleton tracking results at the same time as the depth images are acquired. Once 3D joint poses are obtained, our framework estimates flexion/extension and abduction/adduction angles by applying computational geometry oper- ations. The absence of public datasets with RA patients in the literature makes the estimation of hand poses of patients a challenge for computer vision data-driven methods. We therefore proposed a protocol to acquire new data from groups of patients and control. We performed experiments of identification of RA patients and control sets and also performed comparison with goniometer data. Results show that a method based on Fourier descriptors is able to perform automatic discrimination of hands with Rheumatoid Arthritis (RA) and healthy pa- tients. The angle between joints can be used as an indicative of current movement capabilities and function. The acquisition is much easier, non-invasive and patient-friendly, significantly reducing the evaluation time and offering real-time data for the dynamic movement.\n

Detecting emotion from facial expression has become an urgent need because of its immense applications in artificial intelligence such as human-computer collaboration, data-driven animation, human-robot communication etc. Since it is a demanding and interesting problem in computer vision, several works had been conducted regarding this topic. The objective of this research is to develop a facial expression recognition system based on convolutional neural network with data augmentation. This approach enables to classify seven basic emotions consist of angry, disgust, fear, happy, neutral, sad and surprise from image data. Convolutional neural network with data augmentation leads to higher validation accuracy than the other existing models (which is 96.24%) as well as helps to overcome their limitations.