Design and Implementation of a Convolutional Neural Network on an Edge Computing Smartphone for Human Activity Recognition

Every year, new Convolutional Neural Network (CNN) architectures appear to deal with different problems in the activity of image and video recognition. These architectures usually work along the ImageNet dataset for looking for the best performance of the CNNs without taking into account the video task where they are used. This can represent a problem if the task is Human Action Recognition (HAR) in video, since the CNN architectures are pre-trained with an image dataset that can practically contain any object, while HAR problem requires consecutive frames of people doing actions. To prove the idea that using CNNs pre-trained on an image dataset does not always achieve the best performance on a video dataset and that, therefore, it is worth comparing the performance of different CNNs under similar circumstances for the HAR problem, this work proposes an analysis between eight different CNN architectures. Each one of the CNN was exclusively trained with RGB images, which were extracted from the frames of the different classes of videos of HMDB51 dataset. To make the classification of an activity in video, we average the predictions taking into account the successes. We also made some ensembles with the best performance CNNs to measure the improvement in accuracy. Our results suggest that Xception is a strong baseline model that could be used by the community to make their comparisons of their proposals more robust.

Recognition of human activity based on convolutional neural network has received the interest of researchers in recent years due to its significant improvement in accuracy. A large number of algorithms based on the deep learning approach have been proposed for activity recognition purpose. However, with the increasing advancements in technologies having limited computational resources, it needs to design an efficient deep learning-based approaches with improved utilization of computational resources. This paper presents a simple and efficient 2-Dimensional convolutional neural network (2-D CNN) architecture with very small size convolutional kernel for human activity recognition. The merit of the proposed CNN architecture over standard deep learning architectures is fewer trainable parameters and lesser memory requirement which enables it to train the proposed CNN architecture on low GPU memory-based devices and also works well with smaller as well as larger size datasets. The proposed approach consists of mainly four stages: namely (1) creation of dataset and data augmentation, (2) designing 2-D convolutional neural network (CNN) architecture, (3) the proposed 2-D CNN architecture trained from scratch up to optimum stage, and (4) evaluation of the trained 2-D CNN architecture. To illustrate the effectiveness of the proposed architecture several extensive experiments are conducted on three publicly available datasets, namely IXMAS, YouTube, and UCF101 dataset. The results of the proposed method and its comparison with other state-of-the-art methods [8-12,14,18-26,29-33] demonstrate the usefulness of the proposed method.

In recent years, many researchers have focused on using convolutional neural networks to perform human activity recognition as evidenced by the emergence of a number of convolutional neural network architectures such as LeNet-5, AlexNet and VGG16 and modern architectures such as ResNet, Inception V3, Inception-ResNet, MobileNet V2, NASNet and PNASNet. The main characteristic of a convolutional neural network (CNN) is its ability to extract features automatically from input images, which facilitates the processes of activity recognition and classification. Convolutional networks indeed derive more relevant and complex features with every additional layer. In addition, CNNs have achieved perfect classification on highly similar activities that were previously extremely difficult to classify. In this paper, we evaluate modern convolutional neural networks in terms of their human activity recognition accuracy, and we compare the results with the state of the art methods. In our research, we used two public data sets, HMDB (Shooting gun, kicking, falling to the floor, punching) and the Weizman dataset (walking, running, jumping, bending, one hand waving, two-hand waving, jumping in place, jumping jack, skipping). Our experimental results indicated that the CNN with NASNet architecture achieves the best performance of the six CNN architectures on both human activity data sets (HMDB and Weizman).

Background: Memristor-based implementations of Convolutional Neural Networks (CNNs) offer the potential to reduce power consumption and improve computational efficiency significantly. However, the non-ideal characteristics of memristive devices, such as variability and limited switching behavior, can introduce nonlinearity and degrade network performance. It is therefore essential to simulate memristor-based deep learning systems and evaluate the impact of these imperfections before proceeding with hardware-level deployment. Objective: This study aims to design a lightweight memristor-based MobileNetV2 model for smoking behavior detection and analyze the influence of memristor non-idealities on the network’s detection accuracy. Methods: A lightweight MobileNetV2 network was constructed and trained on a self-developed dataset for smoking behavior detection, achieving a peak accuracy of 92.25%. Based on the trained model, a memristor-based MobileNetV2 architecture was implemented using the Mem- Torch simulation framework. Two memristor models, VTEAM and Data-Driven Verilog-A RRAM, were employed to replace the convolutional and fully connected layers with memristor crossbar arrays. The impact of non-ideal memristor properties, including conductance variability and resistance state faults, was systematically evaluated. Results: Simulation results demonstrated that the memristor-based MobileNetV2 network achieved detection accuracies of 92.11% with the VTEAM model and 91.82% with the Verilog-A RRAM model. As device non-idealities increased, the detection accuracy declined significantly, indicating a strong correlation between memristor imperfections and the performance of CNN- based target detection tasks. Conclusion: Memristor non-idealities substantially affect the performance of neuromorphic CNNs. This study provides targeted analyses and proposes mitigation strategies, such as differential encoding and improved fabrication techniques, to support the practical deployment of memristor- based neural networks in energy-efficient AI applications.

Aims: This study aimed to explore the effect of training set diversity on the performance of deep learning models for predicting early gastric cancer (EGC) under endoscopy. Methods: Images of EGC and non-cancerous lesions under narrow-band imaging (ME-NBI) and magnifying blue laser imaging (ME-BLI) were retrospectively collected. Training set 1 was composed of 150 non-cancerous and 309 EGC ME-NBI images, training set 2 was composed of 1505 non-cancerous and 309 EGC ME-BLI images, and training set 3 was the combination of training set 1 and 2. Test set 1 was composed of 376 non-cancerous and 1052 EGC ME-NBI images, test set 2 consisted of 529 non-cancerous and 71 EGC ME-BLI images, and test set 3 was the combination of test set 1 and test set 2. Three deep learning models, convolutional neural network (CNN) 1, CNN 2 and CNN 3 (CNN 1, CNN 2 and CNN 3 were independently trained using training set 1, training set 2 and training set 3, respectively), were constructed, and their performances on each test set were respectively evaluated. One hundred and thirty-eight ME-NBI videos and 17 ME-BLI videos were further collected to evaluate and compare the performance of each model in real time. Results: On the whole, the performance of CNN 3 was the best. The accuracy (Acc), sensitivity (Sn), specificity (Sp) and area under the curve (AUC) of test set 1 in CNN 3 were 87.89% (1255/1428), 90.96% (342/376), 86.79% (913/1052) and 94.60%, respectively. The Acc, Sn, Sp and AUC of test set 2 in CNN 3 were 95% (570/600), 97.92% (518/529), 73.24% (52/71) and 90.93% respectively. The Acc, Sn, Sp and AUC of test set 3 in CNN 3 were 89.99% (1825/2028), 95.03% (860/905), 85.93% (965/1123) and 94.89%, respectively. The performance of CNN 3 was also the best in videos test set. The Acc, Sn and Sp of videos test set in CNN 3 were 91.03% (142/156), 90.58% (125/138) and 94.44% (17/18), respectively. Conclusions: The deep learning model with the most diverse training data has the best diagnostic effect.

In this paper, we study the performance of variants of well-known Convolutional Neural Network (CNN) architectures on different audio tasks. We show that tuning the Receptive Field (RF) of CNNs is crucial to their generalization. An insufficient RF limits the CNN's ability to fit the training data. In contrast, CNNs with an excessive RF tend to over-fit the training data and fail to generalize to unseen testing data. As state-of-the-art CNN architectures - in computer vision and other domains - tend to go deeper in terms of number of layers, their RF size increases and therefore they degrade in performance in several audio classification and tagging tasks. We study well-known CNN architectures and how their building blocks affect their receptive field. We propose several systematic approaches to control the RF of CNNs and systematically test the resulting architectures on different audio classification and tagging tasks and datasets. The experiments show that regularizing the RF of CNNs using our proposed approaches can drastically improve the generalization of models, out-performing complex architectures and pre-trained models on larger datasets. The proposed CNNs achieve state-of-the-art results in multiple tasks, from acoustic scene classification to emotion and theme detection in music to instrument recognition, as demonstrated by top ranks in several pertinent challenges (DCASE, MediaEval).

Convolutional Neural Networks (CNNs) are used for the sound signal classification successfully due to their capabilities of deep architecture with pooling schemes. Since the pooling methods in CNN work as regularization to minimize the effect of overfitting which further improves the performance of CNN. Various pooling techniques are proposed to enhance the performance of CNN for classification, but mostly all pooling approaches are considered for the classification of images. Sound samples are of a different nature from the static images and the temporal sequences of sounds play a crucial role in the digitalization of sound samples. The presented work of this paper considered the different pooling schemes with convolutional neural networks to improve the classification accuracy for environment sounds. In this approach, two different CNN architectures are implemented and analyzed with various pooling schemes at local and global levels. Further, the paper presented a new pooling scheme at the local and global pooling aspects to improve the performance. This new pooling scheme is considered as max-min difference pooling. The simulated results are obtained on the UrbanSound8k dataset with different optimizers and proposed pooling schemes. The experimental results indicate improved performance of CNN model with max-min difference pooling with global aspect

In this paper, the Convolutional Neural Network (CNN) architecture, which is one of the deep learning architectures, is used to classify the basic circuit components drawn by hand. During the training and testing stages of the model, a new dataset containing images of 863 circuit components manually drawn by different people is created. The data set contains images of four different classes of circuit components such as resistor, inductor, capacitor and voltage source. All images have been fixed to the same size and converted to grayscale to increase recognition performance and reduce process complexity. In the study, training for four classes is performed with CNN architecture. Based on the CNN architecture, four new CNN models are employed with different the number of layers. The training and validation results of these models are compared separately, the model with the highest training and validation performance is observed with four layer CNN model (CNN-4). This model obtained 84.41% accuracy rate at classification task.

Recently, convolutional neural networks (CNNs) achieved remarkable success in various fields, especially computer vision and image processing. However, it is not known what type of CNN architecture is the best fit for the detection or classification of communication signals. In this work, we compare the three of CNN architecture in a burst signal detection task. The three CNN architectures are compared to their detection performance and computational complexity. The 9-layer CNN is shown to achieve a similar performance of 12-layer CNN on overall environments. The performance of the 7-layer CNN model is worse than that of the other two types of CNN architectures, except in terms of the computational complexity at low SNR.

Background. Modern machine vision systems largely rely on convolutional neural networks, which demonstrate high performance in image analysis due to their ability to extract hierarchical feature representations. Improvement in recognition quality is commonly achieved by increasing network depth and the number of parameters; however, this approach is associated with higher computational costs, increased sensitivity to noise, and reduced robustness to variations in scale and illumination. This circumstance highlights the need for architectural solutions that enable more efficient feature extraction without a significant increase in computational complexity. A promising direction involves the use of multiscale convolutional operations that capture both local and global image context, as well as the integration of attention mechanisms that provide adaptive concentration of computational resources on the most informative regions and features. Purpose. To formulate an architectural approach to the design of convolutional neural networks based on the combination of multiscale convolutional processing and attention mechanisms, aimed at improving the efficiency of machine vision algorithms while maintaining an acceptable level of computational cost. Materials and methods. The article examines three convolutional neural network architectures, including a baseline CNN, a multiscale CNN, and an extended model incorporating an attention mechanism. Operations of two dimensional convolution, batch normalization, and nonlinear activation are analyzed, along with methods for aggregating features obtained at different spatial scales. An analytical assessment of computational complexity is performed with consideration of the number of parameters, network depth, and the volume of operations during the forward pass. Results. The application of multiscale convolutional processing expands the receptive field of the network and improves robustness to variations in object size through simultaneous analysis of features at different levels of detail. The integration of attention mechanisms provides adaptive redistribution of feature weights, reduces the influence of noise and irrelevant components, and enhances the selectivity of image analysis.

Although font recognition is a fundamental issue in the field of document analysis and recognition, it was usually ignored in the past. With the development of optical character recognition (OCR), font recognition becomes more and more important. This paper proposed a well-designed convolutional neural network (CNN) architecture for traditional Mongolian font recognition by means of a single word. To be specific, the whole word image is regarded as input of CNN. Hence, the word images should be normalized into the same size before being inputted into CNN. By comparison, an appropriate aspect ratio for the traditional Mongolian word images has been determined. Experimental results demonstrate that the proposed CNN architecture outperforms three classic CNN architectures, including LeNet-5, AlexNet and GoogLeNet. Therefore, the proposed CNN is much more suitable for the task of the traditional Mongolian font recognition in the way of a single word.

Classification performance based on ImageNet is the de-facto standard metric for CNN development. In this work we challenge the notion that CNN architecture design solely based on ImageNet leads to generally effective convolutional neural network (CNN) architectures that perform well on a diverse set of datasets and application domains. To this end, we investigate and ultimately improve ImageNet as a basis for deriving such architectures. We conduct an extensive empirical study for which we train 500 CNN architectures, sampled from the broad AnyNetX design space, on ImageNet as well as 8 additional well-known image classification benchmark datasets from a diverse array of application domains. We observe that the performances of the architectures are highlydataset dependent. Some datasets even exhibit a negative error correlation with ImageNet across all architectures. We show how to significantly increase these correlations byutilizing ImageNet subsets restricted to fewer classes. These contributions can have a profound impact on the way we design future CNN architectures and help alleviate the tilt we see currently in our community with respect to over-reliance on one dataset.

Health monitoring and tracking using Human Activity Recognition is a burgeoning study area, especially when it comes to detecting falls in the elderly. Falls can result in injuries, bodily injury, fractures, and other problems. Visual surveillance's major goal is to replace traditional video surveillance with intelligent visual surveillance. Different feature extraction algorithms were developed to better detect the fall. Convolutional Neural Network (CNN) in Fall Detection with the Optical Flow algorithm is used to extract features to find the fall detection. Designing CNN architecture for classification purposes without overfitting is a challenging task. The goal of this work is to avoid overfitting in training by automatically optimizing the CNN architecture with fine tuned dropout rate. This helps to increase the efficiency of the model. Four different CNN architecture are developed to predict fall detection. The UP-FALL Detection Dataset is used to test the performance of the four models. The experimental results show that second model produced high accuracy compared with other models.

Substantial research has explored methods to optimize convolutional neural networks (CNNs) for tasks such as image classification and object detection, but research into the image quality drivers of computer vision performance has been limited. Additionally, there are indications that image degradations such as blur and noise affect human visual interpretation and machine interpretation differently. The general image quality equation (GIQE) predicts overhead image quality for human analysis using the National Image Interpretability Rating Scale, but no such model exists to predict image quality for interpretation by CNNs. Here, we assess the relationship between image quality variables and CNN performance. Specifically, we examine the impacts of resolution, blur, and noise on CNN performance for models trained with in-distribution and out-of-distribution distortions. Using two datasets, we observe that while generalization remains a significant challenge for CNNs faced with out-of-distribution image distortions, CNN performance against low visual quality images remains strong with appropriate training, indicating the potential to expand the design trade space for sensors providing data to computer vision systems. Additionally, we find that CNN performance predictions using the functional form of the GIQE can predict CNN performance as a function of image degradation, but we observe that the legacy form of the GIQE (from GIQE versions 3 and 4) does a better job of modeling the impact of blur/relative edge response in our experiments.

The presence of anomalies in time series, i.e., data points that deviate from the expected behavior, can be a symptom of an incoming failure that may lead to costly repair. Thus, the detection of anomalous data by means of diagnostic methodologies can both reduce maintenance actions and asset unscheduled stops. To tackle this challenge, for the first time in the literature, we exploit the capabilities of Convolutional Neural Networks (CNNs), by feeding them with images obtained from multivariate time series data, transformed by means of two different approaches, namely, Gramian Angular Summation Field (GASF) and Markov Transition Field (MTF). Two CNN architectures are investigated, i.e., VGG-19 and SqueezeNet. The performance of both CNNs fed with images is compared to that of i) a Temporal Convolutional Network (TCN) fed with time series data and ii) a Support Vector Machine (SVM) model. In this paper, we present the comprehensive framework, which starts from time series transformation, goes through CNN development and ends with anomaly detection. The framework is applied to field data taken during normal operation of ten SGT-800 gas turbines, operated in combined cycle, running at base load, and located in two different regions. The normal data covers 150 days of operation. Spike faults are implanted in two out of the twenty available measured variables, i.e., compressor discharge temperature and compressor discharge pressure, by considering nine combinations of maximum fault magnitude and number of implanted spikes in each time series. The analyses carried out in this paper demonstrate that both CNNs fed with images achieve significantly higher classification accuracy than both a TCN model fed with time series data and an SVM model. Moreover, the MTF method always proves more robust than GASF method, and also allows higher accuracy values, in the range from 0.85 to 0.99.