A new Feature-Fusion method based on training dataset prototype for surface defect recognition

As road traffic sign recognition is a crucial component for automatic driver assistance systems, it is a key problem in computer vision as well. Therefore, in this paper, we study on the problem of road traffic sign recognition utilising the computer vision technology. The main innovation of this paper is to propose an improved convolutional neural network, and then use it to tackle the road traffic sign recognition problem. Convolutional neural network can learn features from training data set, and a convolutional network contains alternating layers of convolution and pooling. Particularly, RGB traffic images are transformed to grey scale images, and then grey scale images are input to the improved convolutional neural network. Furthermore, the fixed layers are utilised to discover region of interests, and the learnable layers are used to extract features. In general, output information of the proposed two learnable layers are input to the classifier separately, and parameters of learnable layers and the classifier are trained at the same time. Finally, GTSDB data set is chosen to make performance evaluation, among which 600 images and 300 images are regarded as training and testing data set respectively. Experimental results demonstrate that the improved CNN-based traffic sign recognition performs better than the traditional CNN.

Deep learning assisted continuous wavelet transform-based spectrogram for the detection of chlorophyll content in potato leaves

Marine diesel engines work in an environment with multiple excitation sources. Effective feature extraction and fault diagnosis of diesel engine vibration signals have become a hot research topic. Time-domain synchronous averaging (TSA) can effectively handle vibration signals. However, the key phase signal required for TSA is difficult to obtain. During signal processing, it can result in the loss of information on fault features. In addition, frequency multiplication signal waveforms are mixed. To address this problem, a multi-scale time-domain averaging decomposition (MTAD) method is proposed and combined with signal-to-image conversion and a convolutional neural network (CNN), to perform fault diagnosis on a marine diesel engine. Firstly, the vibration signals are decomposed by MTAD. The MTAD method does not require the acquisition of the key phase signal and can effectively overcome signal aliasing. Secondly, the decomposed signal components are converted into 2-D images by signal-to-image conversion. Finally, the 2-D images are input into the CNN for adaptive feature extraction and fault diagnosis. Through experiments, it is verified that the proposed method has certain noise immunity and superiority in marine diesel engine fault diagnosis.

Human facial expressions describe a set of signals, which can be associated with mental states such as emotions depending on physiological conditions. There are many potential applications of expression recognition systems. They take into account about two hundred emotional states. Expression recognition is a challenging problem, not only due to the variety of expressions, but also due to difficulty in extraction of effective features from facial images. Depending on a 3D reconstruction technique, 3D data can be immune to a great range of illumination and texture variations, and they are no sensitive as 2D images to out-of-plane rotations. Moreover, 2D images may fail to capture subtle but discriminative change on the face if there is no sufficient change in brightness, such as bulges on the cheeks and protrusion of the lips. In fact, 3D data yield better recognition than conventional 2D data for many types of facial actions The most effective tool for solution of the problem of human face recognition is neural networks. But the result of recognition can be spoiled by facial expressions and other deviation from canonical face representation. In the proposed presentation we describe a resampling method of human faces represented by 3D point clouds. The method based on the non-rigid ICP (Iterative Closest Point) algorithm. We consider the combined using of this method and convolutional neural network (CNN) in the face recognition task. Computer simulation results are provided to illustrate the performance of the proposed algorithm.

CALENDAR

Convolutional neural networks are known to outperform all other neural network models when classifying a wide variety of 2D-shapes. This type of networks supports a massively parallel extraction of low-level features in the processed images. Especially this characteristic is assumed to impact the performance of convolutional networks in character recognition tasks - and in particular when considering scaled, rotated, translated or otherwise deformed patterns. Yet training of convolutional networks is rather time-consuming due to the relatively high complexity of the entire model. To speed-up the training process, we will propose a new variant of convolutional networks - the so-called hybrid convolutional neural network (HCNN). HCNN-networks combine the original idea of LeCun's convolutional networks with the benefits of RBF-like neurons in all the layers and with the winner-takes- all mechanism applied during recall. In the tests done so far in hand-written digit recognition, HCNN proved to be capable of considerably speeding-up the training process while maintaining roughly the same performance of the trained networks like original convolutional networks.

Robust lane line segmentation based on group feature enhancement

This paper aims to introduce a novel approach that includes three steps, namely Energy efficient clustering, Disease diagnosis, and an Alert system. Initially, energy-efficient clustering of nodes was conducted, and to render the clustering more optimal, its centroid was optimally selected by a new hybrid algorithm. In addition, this cluster formation was conducted based on constraints such as distance and energy. Further, the disease diagnosis in IoT was performed under two phases namely, “feature extraction and classification”. During feature extraction, the statistical and higher-order features were extracted. These extracted features were then classified via Optimized Deep Convolutional Neural Network (DCNN). To make the classification more precise, the weights of the DCNN were optimally tuned by a new hybrid algorithm referred to as Hybrid Elephant and Moth Flame with Adaptive Learning (HEM-AL). Finally, an alert system was enabled via proposed severity level estimation, which determined the severity of the disease, suggesting patients to visit the hospital. Lastly, the supremacy of the developed approach was examined via evaluation over the other extant techniques. Accordingly, the proposed model attained an accuracy of 0.99 for test case 1, and was 7.41%, 17.34%, and 13.41% better than traditional NN, CNN, and DCNN models.

Deep learning based methods have recently been successfully explored in hyperspectral image classification field. However, training a deep learning model still requires a large number of labeled samples, which is usually impractical in hyperspectral images. In this paper, a simple but effective feature extraction method is proposed for hyperspectral image classification. Specifically, a pretrained deep convolutional neural network based on the ImageNet dataset is used to extract spatial features of a hyperspectral image. Recently, it is easy to obtain a pretrained convolutional neural network on the Internet. Note that the pretrained models are trained by using the ImageNet dataset. This means that the proposed method does not need labeled hyperspectral samples to train the deep model. Therefore, the proposed method alleviates the problem of lacking labeled samples and avoids the artificial design of feature extraction rules. Finally, the extracted features are stacked with spectral features as the input of a support vector machine classifier. The proposed method is conducted on three widely used hyperspectral image datasets. The experimental results demonstrate that the proposed method could outperform the conventional feature extraction methods and deep learning based methods.

Identifying organs/tissue and pathology on radiological and microscopic images can be performed using convolutional neural networks (CNN). However, there are scant studies on applying CNN to post-mortem gross images of visceral organs. This proof-of-concept study used 537 gross post-mortem images of dissected brain, heart, lung, liver, spleen, and kidney, which were randomly divided into a training and teaching datasets for the pre-trained CNN Xception. The CNN was trained using the training dataset and subsequently tested on the testing dataset. The overall accuracies were >95% percent for both training and testing datasets and have an F1 score of >0.95 for all dissected organs. This study showed that small datasets of post-mortem images can be classified with a very high accuracy using a pre-trained CNN. This novel area has the potential for future application in data mining, education and teaching, case review, research, quality assurance, auditing purposes, and identifying pathology.

The extraction of speaker-related features through speaker representations or utterance embeddings has been extensively studied for years. Convolutional Neural Networks (CNNs) are the most commonly used backbone network for feature extraction, and deep Residual Networks (ResNets), which incorporate residual connectivity into CNNs, achieve effective frame-level feature extraction inside the same input samples. ResNets have demonstrated impressive performance as speaker embedding extractors. With emerging this backbone and its advanced extensions, such as Res2Net, as dominant backbone networks in speaker recognition, and despite its widespread adoption, a significant gap in current speaker embeddings is the unmodified use of these successful backbones from the visual domain, without considering the specific adaptation requirements for the speech field. Tailoring the backbone to account for the unique characteristics of speech representations ensures effective learning in Automatic Speaker Verification (ASV) systems. This thesis concentrates on the backbone network, the fundamental and most critical component of Deep Neural Networks (DNNs), responsible for extracting features from input data and mapping them into distinct feature representations. One of the key questions addressed in the thesis is the following: Which backbone is most robust and yields the most generalized speaker representation, with the potential to be applied to many existing ASV systems?. Through a strategic evaluation, the study analyzed the performance of integrating different residual-based backbones into the most popular ResNet-based fundamental baselines, ResNet-34 and ECAPA-TDNN based models, with highly different model complexity and CNN stem representations. To this end, the proposed architectures were evaluated on the publicly available VoxCeleb dataset. This thesis introduces an innovative approach to speaker modeling by incorporating scale and cardinality dimensions for the first time. Experimental findings reveal that utilizing multi-scale and multi-branch aggregated residual transformations within the Res2NeXt backbone yields considerable performance improvements. Notably, 6s8g and 8s8g variants of Res2NeXt, prove particularly beneficial in improving the representational capability of learned features for speaker embedding. The consistent superiority, excellent compatibility, and robustness of Res2NeXt compared to the current popular ResNet and Res2Net backbones across various architectures and conditions demonstrate its broad applicability and effectiveness as a backbone in speaker modeling. To further evaluate the Robustness and Generalization (R&G) capabilities of the proposed speaker embeddings, the thesis expanded its scope to cross-domain applications, specifically Speech Emotion Recognition (SER), one of the most challenging downstream tasks. A comparative analysis of the proposed speaker embeddings was conducted using two benchmark datasets, IEMOCAP and CREMA-D. The overarching goal of the thesis is to contribute to the development of general-purpose and universal speech representations that generalize effectively across multiple speech-related tasks. By examining the alignment and divergence of embeddings for ASV and SER, the study seeks to answer a fundamental question: Can a single representation framework cater to both speaker verification and emotion recognition or are task-specific embeddings inevitable?. The results confirm that speaker embeddings derived from the Res2NeXt backbone are also highly effective for the SER task, maintaining their performance across diverse evaluation protocols and data variations. This demonstrates the generalizability of Res2NeXt embeddings in the cross-domain task. The study attributes their success to the ability of Res2NeXt-based models to capture rich speech features, including tone, pitch, and phonation patterns, during speaker recognition training—features that are also vital for emotion recognition.

Fine-grained car recognition aims to recognize the category information of a car, such as car make, car model, or even the year of manufacture. A number of recent studies have shown that a deep convolutional neural network (DCNN) trained on a large-scale data set can achieve impressive results at a range of generic object classification tasks. In this paper, we propose a spatially weighted pooling (SWP) strategy, which considerably improves the robustness and effectiveness of the feature representation of most dominant DCNNs. More specifically, the SWP is a novel pooling layer, which contains a predefined number of spatially weighted masks or pooling channels. The SWP pools the extracted features of DCNNs with the guidance of its learnt masks, which measures the importance of the spatial units in terms of discriminative power. As the existing methods that apply uniform grid pooling on the convolutional feature maps of DCNNs, the proposed method can extract the convolutional features and generate the pooling channels from a single DCNN. Thus minimal modification is needed in terms of implementation. Moreover, the parameters of the SWP layer can be learned in the end-to-end training process of the DCNN. By applying our method to several fine-grained car recognition data sets, we demonstrate that the proposed method can achieve better performance than recent approaches in the literature. We advance the state-of-the-art results by improving the accuracy from 92.6% to 93.1% on the Stanford Cars-196 data set and 91.2% to 97.6% on the recent CompCars data set. We have also tested the proposed method on two additional large-scale data sets with impressive results observed.

capsule endoscopy (CE) revolutionized the study of the small intestine. Nevertheless, reviewing CE images is time-consuming and prone to error. Artificial intelligence algorithms, particularly convolutional neural networks (CNN), are expected to overcome these drawbacks. Protruding lesions of the small intestine exhibit enormous morphological diversity in CE images. This study aimed to develop a CNN-based algorithm for the automatic detection small bowel protruding lesions. a CNN was developed using a pool of CE images containing protruding lesions or normal mucosa from 1,229 patients. A training dataset was used for the development of the model. The performance of the network was evaluated using an independent dataset, by calculating its sensitivity, specificity, accuracy, positive and negative predictive values. a total of 18,625 CE images (2,830 showing protruding lesions and 15,795 normal mucosa) were included. Training and validation datasets were built with an 80 %/20 % distribution, respectively. After optimizing the architecture of the network, our model automatically detected small-bowel protruding lesions with an accuracy of 92.5 %. CNN had a sensitivity and specificity of 96.8 % and 96.5 %, respectively. The CNN analyzed the validation dataset in 53 seconds, at a rate of approximately 70 frames per second. we developed an accurate CNN for the automatic detection of enteric protruding lesions with a wide range of morphologies. The development of these tools may enhance the diagnostic efficiency of CE.

Sex determination of silkworm pupae is important for silkworm industry. Multivariate analysis methods have been widely applied in hyperspectral imaging spectroscopy for classification. However, these methods require essential steps containing spectra preprocessing or feature extraction, which were not easy determined. Convolutional neural networks (CNNs), which have been employed in image recognition, could effectively learn interpretable presentations of the sample without the need of ad-hoc preprocessing steps. The species of silkworm pupae are usually up to hundreds. Conventional classifiers based on one species of silkworm pupae could not give high performance when explored to other species that not participating in the model building, resulting in bad generalization ability. In this study, a CNN model was trained to automatically identify the sex of silkworm pupae from different years and species based on the hyperspectral spectra. The results were compared with the frequently used conventional machine classifiers including support vector machine (SVM) and K nearest neighbors (KNN). The results showed that CNN outperformed SVM and KNN in terms of accuracy when applied to the raw spectra with 98.03%. However, the performance of CNN decreased to 95.09% when combined with the preprocessed data. Then principal component analysis (PCA) was adopted to reduce data dimensionality and extract features. CNN gave higher accuracy than SVM and KNN based on PCA. The discussion section revealed that CNN had high generalization ability that could classify silkworm pupae from different species with a rather well performance. It demonstrated that HSI technology in combination with CNN was useful in determining the sex of silkworm pupae.

Automatic target recognition (ATR) has made great progress with the development of deep learning. However, the target feature in synthetic aperture radar (SAR) image is not consistent with human vision, and the SAR training samples are always limited. These hard issues pose new challenges to the SAR ATR based on convolutional neural network (CNN). In this letter, we propose an improved CNN model to solve the limited sample issue via the feature augmentation and ensemble learning strategies. Normally, the high-level features that are more comprehensive and discriminative than the middle-level and low-level features are always employed for category discrimination. In order to make up the insufficient training features in the limited sample case, the cascaded features from optimally selected convolutional layers are concatenated to provide more comprehensive representation for the recognition. To take full advantage of these cascaded features, the ensemble learning-based classifier, namely, the AdaBoost rotation forest (RoF), is introduced to replace the original softmax layer to realize a more accurate limited sample recognition. Through the AdaBoost RoF method, not only are these features further enhanced by the rotation matrix but also a strong classifier is constructed by several weak classifiers with different adjusted weights. The experimental results on MSTAR data set show that the cascaded features and ensemble weak classifiers can fully exploit effective information in limited samples. Compared with the existing CNN method, the proposed method can improve the recognition accuracy by about 20% under the condition of ten training samples per class.