Time Window Analysis for Automatic Speech Emotion Recognition

The convolutional layers of standard convolutional neural networks (CNNs) are equivariant to translation. However, the convolution and fully-connected layers are not equivariant or invariant to other affine geometric transformations. Recently, a new class of CNNs is proposed in which the conventional layers of CNNs are replaced with equivariant convolution, pooling, and batch-normalization layers. The final classification layer in equivariant neural networks is invariant to different affine geometric transformations such as rotation, reflection and translation, and the scalar value is obtained by either eliminating the spatial dimensions of filter responses using convolution and down-sampling throughout the network or average is taken over the filter responses. In this work, we propose to integrate the orthogonal moments which gives the high-order statistics of the function as an effective means for encoding global invariance with respect to rotation, reflection and translation in fully-connected layers. As a result, the intermediate layers of the network become equivariant while the classification layer becomes invariant. The most widely used Zernike, pseudo-Zernike and orthogonal Fourier-Mellin moments are considered for this purpose. The effectiveness of the proposed work is evaluated by integrating the invariant transition and fully-connected layer in the architecture of group-equivariant CNNs (G-CNNs) on rotated MNIST and CIFAR10 datasets.

Identifying sleep problem severity from overnight polysomnography (PSG) recordings plays an important role in diagnosing and treating sleep disorders such as the Obstructive Sleep Apnea (OSA). This analysis traditionally is done by specialists manually through visual inspections, which can be tedious, time-consuming, and is prone to subjective errors. One of the solutions is to use Convolutional Neural Networks (CNN) where the convolutional and pooling layers behave as feature extractors and some fully-connected (FCN) layers are used for making final predictions for the OSA severity. In this paper, a CNN architecture with 1D convolutional and FCN layers for classification is presented. The PSG data for this project are from the Cleveland Children’s Sleep and Health Study database and classification results confirm the effectiveness of the proposed CNN method. The proposed 1D CNN model achieves excellent classification results without manually preprocesssing PSG signals such as feature extraction and feature reduction.

Recognition of handwritten characters in two Indic scripts Bangla and Meitei Mayek is one of the challenging responsibilities due to intricate patterns and scarcity of standard datasets. Convolutional Neural Network (CNN) is one of the stablest well-known techniques for classifying objects in distinctive specialties as it has an extraordinary capability of discovering complex patterns. In this paper, we hook a different layout and obtain a unique CNN architecture from scratch, which has manifold advantages over classical machine learning (ML) approaches, and it has a unique ability to consolidate feature extraction and classification altogether. Further, we stretch our work to uncover the mathematical rationale for using non-linearity in the deep learning (DL) model. Our proposed CNN architecture consists of four layers, including convolutional layer (CL), nonlinear activation layer (AL), pooling layer (PL), and fully connected layer (FCL), which are used in the existing two accessible Bangla datasets named cMATERdb and ISI Bangla datasets. The identical model also validates on proposed Manipuri Character dataset, called “Mayek27”. Moreover, we perform an in-depth comparison with different batch sizes and optimization techniques over all the datasets for understanding their functionality. We conceive a novel benchmark performance that has delivered state-of-the-art decisions on two regional handwritten character identifications.

The application of deep learning technology has increased rapidly in recent years. Technologies in deep learning increasingly emulate natural human abilities, such as knowledge learning, problem-solving, and decision-making. In general, deep learning can carry out self-training without repetitive programming by humans. Convolutional neural networks (CNNs) are deep learning algorithms commonly used in wide applications. CNN is often used for image classification, segmentation, object detection, video processing, natural language processing, and speech recognition. CNN has four layers: convolution layer, pooling layer, fully connected layer, and non-linear layer. The convolutional layer uses kernel filters to calculate the convolution of the input image by extracting the fundamental features. The pooling layer combines two successive convolutional layers. The third layer is the fully connected layer, commonly called the convolutional output layer. The activation function defines the output of a neural network, such as 'yes' or 'no'. The most common and popular CNN activation functions are Sigmoid, Tanh, ReLU, Leaky ReLU, Noisy ReLU, and Parametric Linear Units. The organization and function of the visual cortex greatly influence CNN architecture because it is designed to resemble the neuronal connections in the human brain. Some of the popular CNN architectures are LeNet, AlexNet and VGGNet.

Recognition of human activity based on convolutional neural network has received the interest of researchers in recent years due to its significant improvement in accuracy. A large number of algorithms based on the deep learning approach have been proposed for activity recognition purpose. However, with the increasing advancements in technologies having limited computational resources, it needs to design an efficient deep learning-based approaches with improved utilization of computational resources. This paper presents a simple and efficient 2-Dimensional convolutional neural network (2-D CNN) architecture with very small size convolutional kernel for human activity recognition. The merit of the proposed CNN architecture over standard deep learning architectures is fewer trainable parameters and lesser memory requirement which enables it to train the proposed CNN architecture on low GPU memory-based devices and also works well with smaller as well as larger size datasets. The proposed approach consists of mainly four stages: namely (1) creation of dataset and data augmentation, (2) designing 2-D convolutional neural network (CNN) architecture, (3) the proposed 2-D CNN architecture trained from scratch up to optimum stage, and (4) evaluation of the trained 2-D CNN architecture. To illustrate the effectiveness of the proposed architecture several extensive experiments are conducted on three publicly available datasets, namely IXMAS, YouTube, and UCF101 dataset. The results of the proposed method and its comparison with other state-of-the-art methods [8-12,14,18-26,29-33] demonstrate the usefulness of the proposed method.

This paper develops a wearable sport activity classification system and its associated deep learning-based sport activity classification algorithm for accurately recognizing sport activities. The proposed wearable system used two wearable inertial sensing modules worn on athletes’ wrist and ankle to collect sport motion signals and utilized a deep convolutional neural network (CNN) to extract the inherent features from the spectrograms of the short-term Fourier transform (STFT) of the sport motion signals. The wearable inertial sensing module is composed of a microcontroller, a triaxial accelerometer, a triaxial gyroscope, an RF wireless transmission module, and a power supply circuit. All ten participants wore the two wearable inertial sensing modules on their wrist and ankle to collect motion signals generated by sport activities. Subsequently, we developed a deep learning-based sport activity classification algorithm composed of sport motion signal collection, signal preprocessing, sport motion segmentation, signal normalization, spectrogram generation, image mergence/resizing, and CNN-based classification to recognize ten types of sport activities. The CNN classifier consisting of two convolutional layers, two pooling layers, a fully-connected layer, and a softmax layer can be used to divide the sport activities into table tennis, tennis, badminton, golf, batting baseball, shooting basketball, volleyball, dribbling basketball, running, and bicycling, respectively. Finally, the experimental results show that the proposed wearable sport activity classification system and its deep learning-based sport activity classification algorithm can recognize 10 sport activities with the classification rate of 99.30%.

In deep convolutional neural networks (DCNNs), model size and computation complexity are two important factors governing throughput and energy efficiency when deployed to hardware for inference. Recent works on compact DCNNs as well as pruning methods are effective, yet with drawbacks. For instance, more than half the size of all MobileNet models lies in their last two layers, mainly because compact separable convolution (CONV) layers are not applicable to their last fully connected (FC) layers. Also, in pruning methods, the compression is gained at the expense of irregularity in the DCNN architecture, which necessitates additional indexing memory to address nonzero weights, thereby increasing memory footprint, decompression delays, and energy consumption. In this article, we propose cyclic sparsely connected (CSC) architectures, with memory/computation complexity of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(N\log {}N)$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> is the number of nodes/channels given a DCNN layer that, contrary to compact depthwise separable layers, can be used as an overlay for both FC and CONV layers of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(N^{2})$ </tex-math></inline-formula> . Also, contrary to pruning methods, CSC architectures are structurally sparse and require no indexing due to their cyclic nature. We show that both standard convolution and depthwise convolution layers are special cases of the CSC layers, whose mathematical function, along with FC layers, can be unified into one single formulation and whose hardware implementation can be carried out under one arithmetic logic component. We examine the efficacy of the CSC architectures for compression of LeNet, AlexNet, and MobileNet DCNNs with precision ranging from 2 to 32 bits. More specifically, we surge upon the compact 8-bit quantized 0.5 MobileNet V1 and show that by compressing its last two layers with CSC architectures, the model is compressed by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim 1.5\times $ </tex-math></inline-formula> with a size of only 873 kB and little accuracy loss. Finally, we design a configurable hardware that implements all types of DCNN layers including FC, CONV, depthwise, CSC-FC, and CSC-CONV indistinguishably within a unified pipeline. We implement the hardware on a tiny Xilinx field-programmable gate array (FPGA) for total on-chip processing of the compressed MobileNet that, compared to the related work, has the highest Inference/J while utilizing the smallest FPGA.

Classification performance based on ImageNet is the de-facto standard metric for CNN development. In this work we challenge the notion that CNN architecture design solely based on ImageNet leads to generally effective convolutional neural network (CNN) architectures that perform well on a diverse set of datasets and application domains. To this end, we investigate and ultimately improve ImageNet as a basis for deriving such architectures. We conduct an extensive empirical study for which we train 500 CNN architectures, sampled from the broad AnyNetX design space, on ImageNet as well as 8 additional well-known image classification benchmark datasets from a diverse array of application domains. We observe that the performances of the architectures are highlydataset dependent. Some datasets even exhibit a negative error correlation with ImageNet across all architectures. We show how to significantly increase these correlations byutilizing ImageNet subsets restricted to fewer classes. These contributions can have a profound impact on the way we design future CNN architectures and help alleviate the tilt we see currently in our community with respect to over-reliance on one dataset.

This paper proposes a convolutional neural network architecture for blood vessel segmentation in retinal images. The network structure is designed on 7 layers using MatConvNet (three convolutional layers, two pooling layers, one dropout layer and a Softmax layer). The input data, selected from the DRIVE database, of the neural network is preprocessed in Matlab on Green channel. The retinal image was partitioned in patches of dimension 27 × 27 pixels using a sliding box algorithm. The whole network was trained using a number of 400,000 patches. The CNN was implemented using GPU Programming in MATLAB. The results are promising taking into account speed of processing and the simplicity of the network.

The convolutional neural network (CNN) architecture has shown remarkable success in image classification and segmentation. Its popularity has increased promptly due to various factors such as exponential growth in computational resources, availability of benchmark datasets, supporting libraries, and open-source software. The efficiency of CNN architecture majorly depends on the complexity of the architecture, availability of datasets, and hyperparameter selection. But, due to the huge number of parameters in CNN architecture, its selection has completely remained ad hoc in the past works. In this article, a novel encoding technique has been proposed that can represent complex CNN architecture effectively. The article defines basic building blocks to represent CNN architecture such as the genesis block, transit block, agile block, and output block. This encoding structure is used to generate dynamic length chromosome structure and initialized using evolutionary algorithms. A comparative analysis is also presented that shows compared its effectiveness with existing encoding representations on the basis of the number of encoding parameters, training cost, and efficiency.

Fine-tuning of pretrained convolutional neural networks (CNNs) has been proven to be an effective strategy for remote sensing image scene classification, particularly when a limited number of labeled data sets are available for training purposes. However, such a fine-tuning process often needs that the input images are resized into a fixed size to generate input vectors of the size required by fully connected layers (FCLs) in the pretrained CNN model. Such a resizing process often discards key information in the scenes and thus deteriorates the classification performance. To address this issue, in this paper, we introduce a scale-free CNN (SF-CNN) for remote sensing scene classification. Specifically, the FCLs in the CNN model are first converted into convolutional layers, which not only allow the input images to be of arbitrary sizes but also retain the ability to extract discriminative features using a traditional sliding-window-based strategy. Then, a global average pooling (GAP) layer is added after the final convolutional layer so that input images of arbitrary size can be mapped to feature maps of uniform size. Finally, we utilize the resulting feature maps to create a new FCL that is fed to a softmax layer for final classification. Our experimental results conducted using several real data sets demonstrate the superiority of the proposed SF-CNN method over several well-known classification methods, including pretrained CNN-based ones.

In this paper, we study the performance of variants of well-known Convolutional Neural Network (CNN) architectures on different audio tasks. We show that tuning the Receptive Field (RF) of CNNs is crucial to their generalization. An insufficient RF limits the CNN's ability to fit the training data. In contrast, CNNs with an excessive RF tend to over-fit the training data and fail to generalize to unseen testing data. As state-of-the-art CNN architectures - in computer vision and other domains - tend to go deeper in terms of number of layers, their RF size increases and therefore they degrade in performance in several audio classification and tagging tasks. We study well-known CNN architectures and how their building blocks affect their receptive field. We propose several systematic approaches to control the RF of CNNs and systematically test the resulting architectures on different audio classification and tagging tasks and datasets. The experiments show that regularizing the RF of CNNs using our proposed approaches can drastically improve the generalization of models, out-performing complex architectures and pre-trained models on larger datasets. The proposed CNNs achieve state-of-the-art results in multiple tasks, from acoustic scene classification to emotion and theme detection in music to instrument recognition, as demonstrated by top ranks in several pertinent challenges (DCASE, MediaEval).

Every year, new Convolutional Neural Network (CNN) architectures appear to deal with different problems in the activity of image and video recognition. These architectures usually work along the ImageNet dataset for looking for the best performance of the CNNs without taking into account the video task where they are used. This can represent a problem if the task is Human Action Recognition (HAR) in video, since the CNN architectures are pre-trained with an image dataset that can practically contain any object, while HAR problem requires consecutive frames of people doing actions. To prove the idea that using CNNs pre-trained on an image dataset does not always achieve the best performance on a video dataset and that, therefore, it is worth comparing the performance of different CNNs under similar circumstances for the HAR problem, this work proposes an analysis between eight different CNN architectures. Each one of the CNN was exclusively trained with RGB images, which were extracted from the frames of the different classes of videos of HMDB51 dataset. To make the classification of an activity in video, we average the predictions taking into account the successes. We also made some ensembles with the best performance CNNs to measure the improvement in accuracy. Our results suggest that Xception is a strong baseline model that could be used by the community to make their comparisons of their proposals more robust.

In this paper, the Convolutional Neural Network (CNN) architecture, which is one of the deep learning architectures, is used to classify the basic circuit components drawn by hand. During the training and testing stages of the model, a new dataset containing images of 863 circuit components manually drawn by different people is created. The data set contains images of four different classes of circuit components such as resistor, inductor, capacitor and voltage source. All images have been fixed to the same size and converted to grayscale to increase recognition performance and reduce process complexity. In the study, training for four classes is performed with CNN architecture. Based on the CNN architecture, four new CNN models are employed with different the number of layers. The training and validation results of these models are compared separately, the model with the highest training and validation performance is observed with four layer CNN model (CNN-4). This model obtained 84.41% accuracy rate at classification task.

The enormous volume of data, the reduction of human error, and the high classification throughput have made the automation of blood cell cancer image classification and recognition more and more popular. This work uses a number of deep learning (DL) models from Convolutional Neural Networks (CNNs) to test the classification accuracy. Alexnet, Resnet50, Visual Geometry Group 16 (VGG16), Visual Geometry Group 19 (VGG19), and GoogLeNet are the models. Various convolutional layers, including the Fully Connected (FC), max pooling, and soft-max layers, make up the architecture of these models. Since Vgg19 model showed the highest accuracy of classification with epoch size= 30, Minimum Batch Size (MBS)= 64, learning rate= 0.001 and Sgdm optimizer, it has been modified for less complexity and higher performance than its based model. The confusion matrix is used to assess and display the performance of the base models and the modified Vgg19 model. Four types of blood cell cancered images were used in this work, these are: Eosinophil, Neutrophil, Eosinophil and Monocyte. Modified Vgg19 achieved an accuracy of 0.983 and complexity in term of FLOPs is reduced to 0.7% of the original complexity for Vgg19 based model.