Semantic segmentation of rodent burrows using deep convolutional architectures

Deep convolutional neural networks have achieved great success in recent years. They have been widely used in various applications such as optical and SAR image scene classification, object detection and recognition, semantic segmentation, and change detection. However, deep neural networks rely on large-scale high-quality training data, and can only guarantee good performance when the training and test data are independently sampled from the same distribution. Deep convolutional neural networks are found to be vulnerable to subtle adversarial perturbations. This adversarial vulnerability prevents the deployment of deep neural networks in security-sensitive applications such as medical, surveillance, autonomous driving and military scenarios. This paper first presents a holistic view of security issues for deep convolutional neural network-based image recognition systems. The entire information processing chain is analyzed regarding safety and security risks. In particular, poisoning attacks and evasion attacks on deep convolutional neural networks are analyzed in detail. The root causes of adversarial vulnerabilities of deep recognition models are also discussed. Then, we give a formal definition of adversarial robustness and present a comprehensive review of adversarial attacks, adversarial defense, and adversarial robustness evaluation. Rather than listing existing research, we focus on the threat models for the adversarial attack and defense arms race. We perform a detailed analysis of several representative adversarial attacks on SAR image recognition models and provide an example of adversarial robustness evaluation. Finally, several open questions are discussed regarding recent research progress from our workgroup. This paper can be further used as a reference to develop more robust deep neural network-based image recognition models in dynamic adversarial scenarios.

Self-mixing interferometry (SMI) is superior to other laser interferometry methods due to its simplicity and compactness. However, SMI signals are often complex and difficult to process due to interference in the form of variations in the effective reflectivity of the target, noisy signals, complex signal shapes, and other dependencies. Deep neural networks have been a very popular area of research in computer artificial intelligence in recent years, allowing more implicit features to be uncovered than traditional shallow machine learning. It has been shown that convolutional and back-propagation neural networks can be used for SMI signal processing. There are also studies that have used machine learning genetic algorithms for absolute distance measurement. Based on the above, this study used convolutional neural networks to form a deep neural network for absolute distance measurement based on SMI technology. We first trained the deep convolutional neural network at different feedback strengths. The results of the Convolutional Neural Network (CNN) model showed a coefficient of determination of 0.9987. which is consistent with the required model. The trained network was then used to estimate absolute distances with and without the addition of noise. The comparison proves that the proposed method is noise-proof and has high adaptability for measurements under different conditions.

Automated image classification is an essential task of the computer vision field. The tagging of images into a set of predefined groups is referred to as image classification. The implementation of computer vision to automate image classification would be beneficial because manual image evaluation and identification can be time-consuming, particularly when there are many images of different classes. Deep learning approaches are proven to overperform existing machine learning techniques in many fields in recent years, and computer vision is one of the most notable examples. The very deep neural network (VDCNN) is a powerful deep learning model for image classification, and this paper examines it briefly using MNIST handwritten digit dataset. This dataset is used to prove the efficacy of very deep neural networks over other deep learning models. The proposed study aims to comprehend the very deep neural network architecture used to accomplish a handwritten digit recognition task. The feasibility of the proposed model is evaluated using mean accuracy, validation accuracy, and standard deviation. The study results of the very deep neural network model are compared to a convolutional neural network and convolutional neural network with batch normalization. According to the results of the comparison study, very deep neural networks achieve high accuracy of 99.1% for a handwritten dataset. The outcome of the proposed work is used to interpret how well a very deep neural network performs when compared to the other two models of deep neural networks. This proposed architecture may be used to automate the classification of handwritten digits dataset.

Video hyperlinking represents a classical example of multimodal problems. Common approaches to such problems are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially deep autoencoders, have proven promising both for crossmodal translation and for early fusion via multimodal embedding. A particular architecture, bidirectional symmetrical deep neural networks, have been proven to yield improved multimodal embeddings over classical autoencoders, while also being able to perform crossmodal translation. In this work, we focus firstly at evaluating good single-modal continuous representations both for textual and for visual information. Word2Vec and paragraph vectors are evaluated for representing collections of words, such as parts of automatic transcripts and multiple visual concepts, while different deep convolutional neural networks are evaluated for directly embedding visual information, avoiding the creation of visual concepts. Secondly, we evaluate methods for multimodal fusion and crossmodal translation, with different single-modal pairs, in the task of video hyperlinking. Bidirectional (symmetrical) deep neural networks were shown to successfully tackle downsides of multimodal autoencoders and yield a superior multimodal representation. In this work, we extensively tests them in different settings, with different single-modal representations, within the context of video-hyperlinking. Our novel bidirectional symmetrical deep neural networks are compared to classical autoencoders and are shown to yield significantly improved multimodal embeddings that significantly (alpha=0.0001) outperform multimodal embeddings obtained by deep autoencoders with an absolute improvement in precision at 10 of 14.1% when embedding visual concepts and automatic transcripts and an absolute improvement of 4.3% when embedding automatic transcripts with features obtained with very deep convolutional neural networks, yielding 80% of precision at 10.

Microscopic surgical image analysis is very important in surgical skill analysis, workflow recognition, and autonomous robotic surgery. Semantic segmentation of microscopic image is a prerequisite. Currently, supervised deep convolutional neural network (CNN) has become the state-of-the-art image segmentation method. However, manual contouring of thousands of images is needed for training a supervised CNN. Semi-supervised or weakly supervised learning can greatly reduce the need for labeling, and the semantic segmentation results are also close to or slightly better than those of supervised learning. However, most existing semi-supervised methods have disadvantages of poor segmentation accuracy and poor robustness. The existing semi-supervised methods can only be close to the segmentation accuracy of the existing supervised methods, and can only be used in large-scale data sets. There are still many limitations to the use of the field. In this article, we proposed a novel semi-supervised learning method based on principle of cross-consistency in microscopic images segmentation. To leverage unlabeled images, we use consistency principle between main decoder and auxiliary decoder. The auxiliary decoder is a variety of interference examples generated by different interference functions, which can improve the accuracy and robustness of the overall output of the network. The network also has good task scalability, and has high segmentation accuracy in fields such as natural scenes and medical surgical scenes. The experimental results of the CATARACTS-Semantic-Segmentation 2020 data set proved the effectiveness of the deep neural network. The proposed method is better than the baseline model in terms of accuracy. The experimental setting 1 segmentation is increased by 1.71%, and the experimental setting 2 segmentation accuracy is improved by 22.34%.

Deep Convolutional Neural Networks for Large-scale Speech Tasks

Maxout neurons for deep convolutional and LSTM neural networks in speech recognition

Gait recognition is a noncontact biometric procedure that determines the identity or health status of a person by analyzing his or her walking posture and habits, including skeletal and joint movements. The most remarkable feature of this method is the possibility of conducting recognition without demanding much cooperation from participants. Therefore, this recognition technique has attracted much attention from scholars. Additionally, because of the rapid development of graphics processing unit technology, related hardware and computation performance, the applications of deep-learning technology are considerably enhanced. The objective of this study was to apply a deep neural network (DNN), which employs deep-learning technology, to achieve gait-based automatic pedestrian detection and recognition. In contrast to using wearable devices to precisely capture skeletal and joint movements, pedestrian color-image sequences were used as input in this study. Subsequently, a pretraining convolutional neural network (CNN) was employed to capture pedestrian location and extract pedestrian dense optical flow to serve as concrete low-level feature inputs. Then, a finely-tuned DNN based on the wide residual network was employed to extract high-level abstract features. In addition, to overcome the difficulty of obtaining local temporal features by using a 2D CNN, part of the 3D convolutional structure was introduced into the CNN. This design enabled use of limited memory to acquire more effective features and enhance the DNN performance. The experimental results show that the proposed method has exceptional performance for pedestrian detection and recognition.

Oil spill is considered one of the main threats to marine and coastal environments. Efficient monitoring and early identification of oil slicks are vital for the corresponding authorities to react expediently, confine the environmental pollution and avoid further damage. Synthetic aperture radar (SAR) sensors are commonly used for this objective due to their capability for operating efficiently regardless of the weather and illumination conditions. Black spots probably related to oil spills can be clearly captured by SAR sensors, yet their discrimination from look-alikes poses a challenging objective. A variety of different methods have been proposed to automatically detect and classify these dark spots. Most of them employ custom-made datasets posing results as non-comparable. Moreover, in most cases, a single label is assigned to the entire SAR image resulting in a difficulties when manipulating complex scenarios or extracting further information from the depicted content. To overcome these limitations, semantic segmentation with deep convolutional neural networks (DCNNs) is proposed as an efficient approach. Moreover, a publicly available SAR image dataset is introduced, aiming to consist a benchmark for future oil spill detection methods. The presented dataset is employed to review the performance of well-known DCNN segmentation models in the specific task. DeepLabv3+ presented the best performance, in terms of test set accuracy and related inference time. Furthermore, the complex nature of the specific problem, especially due to the challenging task of discriminating oil spills and look-alikes is discussed and illustrated, utilizing the introduced dataset. Results imply that DCNN segmentation models, trained and evaluated on the provided dataset, can be utilized to implement efficient oil spill detectors. Current work is expected to contribute significantly to the future research activity regarding oil spill identification and SAR image processing.

Deep learning models are highly parameterized, resulting in difficulty in inference and transfer learning for image recognition tasks. In this work, we propose a layered pathway evolution method to compress a deep convolutional neural network (DCNN) for classification of masses in digital breast tomosynthesis (DBT). The objective is to prune the number of tunable parameters while preserving the classification accuracy. In the first stage transfer learning, 19 632 augmented regions-of-interest (ROIs) from 2454 mass lesions on mammograms were used to train a pre-trained DCNN on ImageNet. In the second stage transfer learning, the DCNN was used as a feature extractor followed by feature selection and random forest classification. The pathway evolution was performed using genetic algorithm in an iterative approach with tournament selection driven by count-preserving crossover and mutation. The second stage was trained with 9120 DBT ROIs from 228 mass lesions using leave-one-case-out cross-validation. The DCNN was reduced by 87% in the number of neurons, 34% in the number of parameters, and 95% in the number of multiply-and-add operations required in the convolutional layers. The test AUC on 89 mass lesions from 94 independent DBT cases before and after pruning were 0.88 and 0.90, respectively, and the difference was not statistically significant (p > 0.05). The proposed DCNN compression approach can reduce the number of required operations by 95% while maintaining the classification performance. The approach can be extended to other deep neural networks and imaging tasks where transfer learning is appropriate.

Experimental validation of saliency maps for understanding deep neural networks for weld penetration prediction

Deep convolutional neural networks have become a dominant solution for numerous image classification tasks. However, a main criticism is the poor explainability due to the black-box characteristic, which hurdles the extensive usage of deep convolutional neural networks. To address this issue, this paper proposes a new evolutionary multi-objective based method, which aims to explain the behaviours of deep convolutional neural networks by evolving local explanations on specific images. To the best of our knowledge, this is the first evolutionary multi-objective method to evolve local explanations. The proposed method is model-agnostic, i.e. it is applicable to explain any deep convolutional neural networks. ImageNet is used to examine the effectiveness of the proposed method. Three well-known deep convolutional neural networks -VGGNet, ResNet, and MobileNet, are chosen to demonstrate the modelagnostic characteristic. Based on the experimental results, it can be observed that the local explanations are understandable to end-users, who need to check the sensibility of the evolved explanations to decide whether to trust the predictions made by the deep convolutional neural networks. Furthermore, the local explanations evolved by the proposed method improves the confidence of deep convolutional neural networks making the predictions. Lastly, the pareto front and convergence analyses indicate that the proposed method can form a good set of nondominated solutions.

Evaluation of the benchmark datasets for testing the efficacy of deep convolutional neural networks

(1) Background: Pyoderma gangrenosum (PG) is often situated on the lower legs, and the differentiation from conventional leg ulcers (LU) is a challenging task due to the lack of clear clinical diagnostic criteria. Because of the different therapy concepts, misdiagnosis or delayed diagnosis bears a great risk for patients. (2) Objective: to develop a deep convolutional neural network (CNN) capable of analysing wound photographs to facilitate the PG diagnosis for health professionals. (3) Methods: A CNN was trained with 422 expert-selected pictures of PG and LU. In a man vs. machine contest, 33 pictures of PG and 36 pictures of LU were presented for diagnosis to 18 dermatologists at two maximum care hospitals and to the CNN. The results were statistically evaluated in terms of sensitivity, specificity and accuracy for the CNN and for dermatologists with different experience levels. (4) Results: The CNN achieved a sensitivity of 97% (95% confidence interval (CI) 84.2−99.9%) and outperformed dermatologists, with a sensitivity of 72.7% (CI 54.4−86.7%) significantly (p < 0.03). However, dermatologists achieved a slightly higher specificity (88.9% vs. 83.3%). (5) Conclusions: For the first time, a deep neural network was demonstrated to be capable of diagnosing PG, solely on the basis of photographs, and with a greater sensitivity compared to that of dermatologists.

Over the past few decades, research has focused heavily on automatic speech recognition (ASR). Although ASR for a few languages is close to reality; ASR for the low resource languages like Malayalam is still in its infancy. Here, in this work, the authors discuss the experiment conducted on accented data of Malayalam speech using two approaches. One approach uses the spectral features for modeling using deep convolutional neural network and the other uses the influential features of speech signals for modeling using LSTM-RNN approach. The proposed methodology comprises three distinct stages; dataset preparation, feature extraction, classification, and hence the construction of deep learning models that recognize the accent-based spoken sentences in the Malayalam language. Mel-frequency cepstral coefficient (MFCC) algorithm, short-term Fourier transform (STFT), and mel spectrogram methodologies are used for feature engineering and hence the features that represent the speech signals are used for constructing the accented ASR system for the Malayalam language using long short-term memory (LSTM) a recurrent neural network (RNN). The spectrogram dataset has been constructed for the speech dataset and used for constructing the ASR model with deep convolutional neural network (DCNN). The result of the experiment shows that LSTM-based RNN outperforms DCNN for the proposed dataset that has been constructed for the experiment in the natural recording environment.