Parametric evaluation of observed objects from images based on perspective geometry methods and convolutional neural networks

This paper presents a first investigation on the structure from motion problem from the combination of full and weak perspective images. This problem arises in multiresolution object modeling where multiple zoomed-in or close-up views are combined with wider or distant reference views. The narrow field-of-view (FOV) images from the zoomed-in or closeup views can be approximated as weak perspective projection. Using a full perspective projection model for the narrow FOV images, although more accurate, actually leads to instabilities during the estimation process due to the non-linearities in the imaging model. The weak perspective approximation leads to more stable estimation algorithms, although at the cost of a small amount of modeling inaccuracy. Previous work in structure from motion focused either on two (or more) perspective images or on a set of weak perspective (more generally, affine) images. The main contribution of this paper is the study of the SFM problem for the much neglected case of one perspective and one (or more) weak perspective image. We show that in contrast to the case of a pair of weak perspective images, there is adequate information to recover Euclidean structure from a single perspective and a single weak perspective image. The epipolar geometry is simpler than with two perspective images leading to simpler and more stable estimation algorithms. Computer simulation shows that more stable results can be obtained with the technique presented in this paper than if two images are both considered to be full perspective.

Predicting panoramic indoor lighting from a single perspective image is a fundamental but highly ill-posed problem in computer vision and graphics. To achieve locale-aware and robust prediction, this problem can be decomposed into three sub-tasks: depth-based image warping, panorama inpainting and high-dynamic-range (HDR) reconstruction, among which the success of panorama inpainting plays a key role. Recent methods mostly rely on convolutional neural networks (CNNs) to fill the missing contents in the warped panorama. However, they usually achieve suboptimal performance since the missing contents occupy a very large portion in the panoramic space while CNNs are plagued by limited receptive fields. The spatially-varying distortion in the spherical signals further increases the difficulty for conventional CNNs. To address these issues, we propose a local-to-global strategy for large-scale panorama inpainting. In our method, a depth-guided local inpainting is first applied on the warped panorama to fill small but dense holes. Then, a transformer-based network, dubbed PanoTransformer, is designed to hallucinate reasonable global structures in the large holes. To avoid distortion, we further employ cubemap projection in our design of PanoTransformer. The high-quality panorama recovered at any locale helps us to capture spatially-varying indoor illumination with physically-plausible global structures and fine details.

Objects and processes classification is a common experimental problem. Its solution, first of all, is needed in automatic diagnosis systems, for example, to determine the equipment operation state through the diagnostic signal or to identify abnormalities with medical images. With the development of convolutional neural networks, new prospects for solving such problems have risen up. However, the classification accuracy that can be achieved on these networks is not sufficient enough for all diagnosis issues. It is subject to, for example, timely diagnosis of the onset of transient phenomena. At the same time, another type of neural network, Kohonen self-organizing maps, has a conceptual property for training on unclassified set of classes, that is, giving the opportunity to solve such an issue. Therefore, the accuracy enhancement of the classification on the basis of Kohonen networks implementation in the architecture of convolutional networks is a relevant objective and has practical significance. The article analyzes the means of improving the accuracy of convolutional neural networks and arising problems solution. The ways of increasing the proportion of correct clustering on Kohonen networks due to the ‘growth’ of its grid shape in determining new classes in the learning process are also given. It is this property that makes it possible to recognize transient phenomena. It is determined that the existing solutions of the combination of Kohonen networks and convolutional networks are aimed at improving the efficiency of only self-organizing maps, with the purpose to improve the accuracy of classification by convolutional networks, it became necessary to develop a new architecture. The paper provides a description of this issue. Since the initial information of the Kohonen networks is the weighting matrix values of the grid shape neurons, it was necessary to associate it with the representation of the images in order to process the diagnostic images. The paper proposes the concept of a built-in associative array block based on Kohonen networks. According to the proposed method, a software implementation of a hybrid neural network is developed. The formulation and results of computational experiments are presented. The efficiency of the proposed method is experimentally proved. Keywords: convolutional neural networks, CNN, Kohonen self-organizing maps, SOM, signal classification, image classification.

Sun orientation estimation from a single image using short-cuts in DCNN

Self-mixing interferometry (SMI) is superior to other laser interferometry methods due to its simplicity and compactness. However, SMI signals are often complex and difficult to process due to interference in the form of variations in the effective reflectivity of the target, noisy signals, complex signal shapes, and other dependencies. Deep neural networks have been a very popular area of research in computer artificial intelligence in recent years, allowing more implicit features to be uncovered than traditional shallow machine learning. It has been shown that convolutional and back-propagation neural networks can be used for SMI signal processing. There are also studies that have used machine learning genetic algorithms for absolute distance measurement. Based on the above, this study used convolutional neural networks to form a deep neural network for absolute distance measurement based on SMI technology. We first trained the deep convolutional neural network at different feedback strengths. The results of the Convolutional Neural Network (CNN) model showed a coefficient of determination of 0.9987. which is consistent with the required model. The trained network was then used to estimate absolute distances with and without the addition of noise. The comparison proves that the proposed method is noise-proof and has high adaptability for measurements under different conditions.

Convolutional Neural Networks (CNNs) have demonstrated impressive performance in complex machine learning tasks such as classification and regression problems. A reliable neural network structure plays a decisive role in CNN studies. Through comparing and analyzing the structure of neural networks, a model structure for better visualization performance has been discovered, and such a method supports the development of deep learning research. These studies are of particular importance in end-to-end systems for autonomous driving to imitate human driving, where the interpretability of the system is limited. Because of the uncertainty of the ground truth, for the determination of human steering in an image, it is difficult to accurately compare the visualization performance of different CNN models or different visualization methods. For practical applications, however, an objective and quantitative measure for assessing visualization performance is necessary. Therefore, a method to evaluate the visualization performance of CNN models using a driver model instead of human drivers is proposed, to generate a data set which can be used to determine the decisional point (ground truth) in the input image. Then, an exclusive method is also put forth, to quantitatively calculate the relationship between the decisional point (ground truth) and the visualization results produced by CNN models. In this paper, five CNN models as an autonomous steering controller are designed based on PilotNet, and the visualization abilities of each CNN models is compared by three evaluation indicators. By comparing the visualization performance of five different CNN models, it is shown that the proposed method can successfully assess the visualization level of the CNN model.

Sound-event recognition often utilizes time-frequency analysis to produce an image-like spectrogram that provides a rich visual representation of original signal in time and frequency. Convolutional Neural Networks (CNN) with the ability of learning discriminative spectrogram patterns are suitable for sound-event recognition. However, there is relatively little effort that CNN makes full use of the important temporal information. In this paper, we propose MCRNN, a Convolutional Recurrent Neural Networks (CRNN) architecture for sound-event recognition, the letter “M” in the name “MCRNN” of our model denotes the multi-sized convolution filters. Richer features are extracted by using several different convolution filter sizes at the last convolution layer. In addition, cochleagram images are used as the input layer of the network, instead of the traditional spectrogram image of a sound signal. Experiments on the RWCP dataset shows that the recognition rate of the proposed method achieved 98.4% in clean conditions, and it robustly outperforms the existing methods, the recognition rate increased by 0.9%, 1.9% and 10.3% in 20 dB, 10 dB and 0 dB signal-to-noise ratios (SNR), respectively.

A genetic algorithm (GA), is used to optimize the many parameters of a convolutional neural network (CNN) that control the structure of the network. CNNs are used in image classification problems where it is necessary to generate feature descriptors to discern between image classes. Because of the deep representation of image data that CNNs are capable of generating, they are increasingly popular in research and industry applications. With the increasing number of use cases for CNNs, more and more time is being spent to come up with optimal CNN structures for different applications. Where one CNN might succeed at classification, another can fail. As a result, it is desirable to more easily find an optimal CNN structure to increase classification accuracy. In the proposed method, a GA is used to evolve the parameters that influence the structure of a CNN. The GA compares CNNs by training them on images of concrete containing cracks. The best CNN after several generations of the GA is then compared to the state-of-the-art CNN for crack detection. This work shows that it is possible to generalize the process of optimizing a CNN for image classification through the use of a GA.

In the indoor environment, the activity of the pedes-trian can reflect some semantic information, for example, if a user’s activity is recognized as taking elevator, the location of the user is inferred to be in the elevator. These activities can be used as the landmarks for indoor localization. The development of sensor technology enhances the smartphone’s sensing and computational capabilities. Using the smartphones, the activity of the pedestrian can be recognized. Current methods rely on extracting complex hand-crafted features, thus leading to the incapability of real time pedestrian activities identification. In this paper, we propose a real time pedestrian activities recognition method based on deep convolutional neural network. A new deep convolutional neural (CNN) network has been designed to learn the proper features automatically. Experiments show that the proposed method achieves approximately 95.21% accuracy in about 2 seconds in identifying nine types of activities, including still, walk, upstairs, up elevator, up escalator, down elevator, down escalator, downstairs and turning. Besides, we transplant the activity recognition algorithm to smartphones using tensorflow. Moreover, we have built a pedestrian activity database, which contains more than 6 GB data of accelerometer, magnetometer, gyroscope and barometer collected with various types of smartphones. We will make it public to contribute the academic research.

Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions

Nowadays, the number of patients with brain tumors is steadily increasing, diagnosis and isolation of the tumor play an important role in the process of treatment and surgery. Due to the high error of manual segmentation of the tumor, algorithms that perform this operation with less error are of great importance. Convolutional neural networks have made great progress in the field of medical imaging. The use of imaging techniques and pattern recognition in the diagnosis and automatic determination of brain tumors by MRI imaging reduces errors, human error and speeds up detection. The artificial convolutional neural network (CNN) has been widely used in the diagnosis of intelligent cancers and has significantly reduced the error rate. Therefore, in this paper, we present a new method using a combination of convolutional and multi-scale artificial neural network that has significantly increased the accuracy of tumor diagnosis. This study presents a multidisciplinary convolution neural network (MCNN) approach to classifying tumors that can be used as an important part of automated diagnosis systems for accurate cancer diagnosis. Based on the MCNN structure, which presents the MRI image to several deep convolutional neural networks of varying sizes and resolutions, the stage of extracting classical hand-made features is avoided. This approach proposes better classification rates than the classical methods. This study uses a multi-scale convolution technique to achieve a detection accuracy of 95/4%, which shows the efficiency of the proposed method.

Despite recent advances of deep neural networks in hand vein identification, the existing solutions assume the availability of a large and rich set of training image samples. These solutions, therefore, still lack the capability to extract robust and discriminative hand-vein features from a single training image sample. To overcome this problem, we propose a single-sample-per-person (SSPP) palm-vein identification approach, where only a single sample per class is enrolled in the gallery set for training. Our approach, named MSMDGAN + CNN, consists of a multi-scale and multi-direction generative adversarial network (MSMDGAN) for data augmentation and a convolutional neural network (CNN) for palm-vein identification. First, a novel data augmentation approach, MSMDGAN, is developed to learn the internal distribution of patches in a single image. The proposed MSMDGAN consists of multiple fully convolutional GANs, each of which is responsible for learning the patch distribution within an image at a different scale and at a different direction. Second, given the resulting augmented data by MSMDGAN, we design a CNN for single sample palm-vein recognition. The experimental results on two public hand-vein databases demonstrate that MSMDGAN is able to generate realistic and diverse samples, which, in turn, improves the stability of the CNN. In terms of accuracy, MSMDGAN + CNN outperforms other representative approaches and achieves state-of-the-art recognition results.

Scene text detection is a crucial step in end-to-end scene text recognition, a greatly challenging problem in computer vision. This paper proposes a novel scene text detection method that involves superpixel-based stroke feature transform (SSFT) and deep learning based region classification (DLRC). The SSFT is developed for candidate character region (CCR) extraction, which consists in partitioning an input image into several regions via superpixel-based clustering, removing most regions based on predefined criteria satisfied by the characters, and refining the remaining regions to obtain CCRs by computing a stroke width map. The character regions are identified from the CCRs using DLRC, in which several hand-crafted low-level features, i.e., color, texture, and geometric features, and some deep convolution neural network (CNN) based high-level features are first extracted from the regions, and then these features are fused by using two fully connected networks (FCNs) for region classification. In the DLRC step, the deep feature extraction CNN and the feature fusion FCNs are jointly trained. Next, the extracted character regions are merged to form candidate text regions, from which the final scene texts are detected. The proposed method is evaluated on three publicly available datasets: ICDAR2011, ICDAR2013, and street view text. It achieves F -measures of 0.876, 0.885, and 0.631, respectively, which demonstrate the effectiveness of the proposed scene text detection method.

Colorectal cancer typically affects the gastrointestinal tract within the human body. Colonoscopy is one of the most accurate methods of detecting cancer. The current system facilitates the identification of cancer by computer-assisted diagnosis (CADx) systems with a limited number of deep learning methods. It does not imply the depiction of mixed datasets for the functioning of the system. The proposed system, called ColoRectalCADx, is supported by deep learning (DL) models suitable for cancer research. The CADx system comprises five stages: convolutional neural networks (CNN), support vector machine (SVM), long short-term memory (LSTM), visual explanation such as gradient-weighted class activation mapping (Grad-CAM), and semantic segmentation phases. Here, the key components of the CADx system are equipped with 9 individual and 12 integrated CNNs, implying that the system consists mainly of investigational experiments with a total of 21 CNNs. In the subsequent phase, the CADx has a combination of CNNs of concatenated transfer learning functions associated with the machine SVM classification. Additional classification is applied to ensure effective transfer of results from CNN to LSTM. The system is mainly made up of a combination of CVC Clinic DB, Kvasir2, and Hyper Kvasir input as a mixed dataset. After CNN and LSTM, in advanced stage, malignancies are detected by using a better polyp recognition technique with Grad-CAM and semantic segmentation using U-Net. CADx results have been stored on Google Cloud for record retention. In these experiments, among all the CNNs, the individual CNN DenseNet-201 (87.1% training and 84.7% testing accuracies) and the integrated CNN ADaDR-22 (84.61% training and 82.17% testing accuracies) were the most efficient for cancer detection with the CNN+LSTM model. ColoRectalCADx accurately identifies cancer through individual CNN DesnseNet-201 and integrated CNN ADaDR-22. In Grad-CAM's visual explanations, CNN DenseNet-201 displays precise visualization of polyps, and CNN U-Net provides precise malignant polyps.

Accurate segmentation and delineation of the sub-tumor regions are very challenging tasks due to the nature of the tumor. Traditionally, convolutional neural networks (CNNs) have succeeded in achieving most promising performance for the segmentation of brain tumor; however, handcrafted features remain very important in identification of tumor's boundary regions accurately. The present work proposes a robust deep learning-based model with three different CNN architectures along with pre-defined handcrafted features for brain tumor segmentation, mainly to find out more prominent boundaries of the core and enhanced tumor regions. Generally, automatic CNN architecture does not use the pre-defined handcrafted features because it extracts the features automatically. In this present work, several pre-defined handcrafted features are computed from four MRI modalities (T2, FLAIR, T1c, and T1) with the help of additional handcrafted masks according to user interest and fed to the convolutional features (automatic features) to improve the overall performance of the proposed CNN model for tumor segmentation. Multi-pathway CNN is explored in this present work along with single-pathway CNN, which extracts simultaneously both local and global features to identify the accurate sub-regions of the tumor with the help of handcrafted features. The present work uses a cascaded CNN architecture, where the outcome of a CNN is considered as an additional input information to next subsequent CNNs. To extract the handcrafted features, convolutional operation was applied on the four MRI modalities with the help of several pre-defined masks to produce a predefined set of handcrafted features. The present work also investigates the usefulness of intensity normalization and data augmentation in pre-processing stage in order to handle the difficulties related to the imbalance of tumor labels. The proposed method is experimented on the BraST 2018 datasets and achieved promising results than the existing (currently published) methods with respect to different metrics such as specificity, sensitivity, and dice similarity coefficient (DSC) for complete, core, and enhanced tumor regions. Quantitatively, a notable gain is achieved around the boundaries of the sub-tumor regions using the proposed two-pathway CNN along with the handcrafted features. Graphical Abstract This data is mandatory. Please provide.