Radar-Based Cross-Domain Human Behavior Recognition Using Physics-Informed Data Augmentation and Multi-Source Domain Adversarial Neural Network.

To improve traditional video surveillance systems' performance on human behavior recognition, a new system has been built. Not visual camera but depth camera is chosen as sensor. To adapt to the most common forward oblique view of camera, a normalized digital elevation map, whose pixel intensity indicates the elevation of the scene, is built from the depth image. Coordinates and intensity in the digital elevation map represent the position information about the scene. Oriented templates are proposed to match and detect the human head robustly in the elevation map. As for the excellent visibility of human head in the elevation map, we track the human head to get trajectory. By combining the trajectory information of human head with the elevation map, several predefined human behaviors are recognized. Our behavior recognition method is straightforward and robust. This uniqueness has no similar with the traditional machine learning and classification framework about human behavior recognition.

Big data analysis of human behavior can provide the basis and support for the application of various scenarios. Using sensors for human behavior analysis is an effective means of identification method, which is very valuable for research. To address the problems of low recognition accuracy, low recognition efficiency of traditional human behavior recognition (HBR) algorithms in complex scenes, in this paper, we propose an HBR algorithm for Mobile Big data analytics in wireless sensor network using improved transfer learning. First, different wireless sensors are fused to obtain human behavior mobile big data, and then by analyzing the importance of human behavior features (HBF), the dynamic change parameters of HBF extraction threshold are calculated. Second, combined with the dynamic change parameters of threshold, the HBF of complex scenes are extracted. Finally, the best classification function of human behavior in complex scenes is obtained by using the classification function of HBF in complex scenes. Human behavior in complex scenes is classified according to the HBF in the feature set. The HBR algorithm is designed by using the improved transfer learning network to realize the recognition of human behavior in complex scenes. The results show that the proposed algorithm can accurately recognize up to 22 HBF points, and can control the HBR time within 2 s. The human behavior false recognition rate of miscellaneous scenes is less than 10%. The recognition speed is above 10/s, and the recall rate can reach more than 98%, which improves the HBR ability of complex scenes.

Human behaviour recognition is an important research hotspot in the field of artificial intelligence. Current behaviour recognition methods have low recognition accuracy under different viewing angles, therefore, this paper proposes a novel restricted Boltzmann machine (RBM)-based temporal-spatial correlation method for student behaviour recognition in depth video. The RBM is used to map the human behaviour from different viewing angles to the high-dimensional space. The time level pooling function is applied in the time series activated by each neuron to realise the encoding of the video time sub-series. Finally, behaviour recognition and classification experiments are conducted on different public datasets and real classroom student behaviour datasets with other methods. The results show that the proposed method improves the accuracy of depth video recognition under different viewing angles and has good generalisation performance. The data analysis of abnormal behaviour in class can play an auxiliary role in dynamic classroom management.

Deep learning has revolutionized the performance of classification and object detection, but meanwhile demands sufficient labeled data for training. Given insufficient data, while many techniques have been developed to help combat overfitting, the challenge remains if one tries to train deep networks, especially in the ill-posed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">extremely low data regimes</i> : only a small set of labeled data are available, and nothing – including unlabeled data – else. Such regimes arise from practical situations where not only data labeling but also data collection itself is expensive. We propose a deep adversarial data augmentation (DADA) technique to address the problem, in which we elaborately formulate data augmentation as a problem of training a class-conditional and supervised generative adversarial network (GAN). Specifically, a new discriminator loss is proposed to fit the goal of data augmentation, through which both real and augmented samples are enforced to contribute to and be consistent in finding the decision boundaries. Tailored training techniques are developed accordingly. To quantitatively validate its effectiveness, we first perform extensive simulations to show that DADA substantially outperforms both traditional data augmentation and a few GAN-based options. We then extend experiments to three real-world small labeled classification datasets where existing data augmentation and/or transfer learning strategies are either less effective or infeasible. We also demonstrate that DADA to can be extended to the detection task. We improve the pedestrian synthesis work by substitute for our discriminator and training scheme. Validation experiment shows that DADA can improve the detection mean average precision (mAP) compared with some traditional data augmentation techniques in object detection. Source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/SchafferZhang/DADA</uri> .

This article presents a new deep transfer learning method, named structured domain adversarial neural network (SDANN), for bearing fault diagnosis with the data collected under different working conditions. The key idea of this method is integrating the strong adaptability of domain adversarial neural network (DANN) and structured relatedness information among multiple failure modes to improve the effect of transfer learning. First, for fine-grained alignment between the data collected from different working conditions, a new loss function with a discriminative regularizer is designed for DANN by using maximum correlation entropy constraint. Second, to improve the stability of DANN on an insufficient amount of data, a relatedness matrix is introduced, and a new regularizer with symmetry constraint on this matrix is designed to capture the intrinsic similarity structure among multiple fault types. Finally, a stochastic gradient descent optimization strategy is used to train the network and establish an end-to-end diagnostic model. Comparative experiments are conducted on two widely used bearing data sets. The results show that the proposed method has good diagnosis performance on insufficient monitoring data and outperforms several state-of-the-art transfer diagnosis methods and deep learning-based diagnosis methods in terms of diagnostic accuracy and numerical stability.

To realize trans-domain behavioral targeting, which targets interested potential users of a source domain (e.g. E-Commerce) based on their behaviors in a target domain (e.g. Ad-Network), heterogeneous transfer learning (HeTL) is a promising technique for modeling behavior linkage between domains. It is required for HeTL to learn three functionalities: representation alignment, distribution alignment, and classification. In our previous work, we prototyped and evaluated two typical transfer learning algorithms, but neither of them jointly learns the three desired functionalities. Recent advances in transfer learning include a domain-adversarial neural network (DANN), which jointly learns distribution alignment and classification. In this paper, we extended DANN to be able to learn representation alignment by simply replacing its shared encoder with domain-specific types, so that it jointly learns the three desired functionalities. We evaluated the effectiveness of the joint learning of the three functionalities using real-world data of two domains: E-Commerce, which is set as the source domain, and Ad Network, which is set as the target domain.

The problem of application of deep neural networks (DNNs) for extremely small dataset is considered to find the effect of dataset size and quality on performance metrics. Classification task is considered for the set of problems where images of Lyme disease symptoms were collected and structured in the open-access dataset with the different subsets of the whole dataset: clean images only (C), dirty images only (D), and clean and dirty images in the equal proportions (C_D) (https://www.kaggle.com/yoctoman/lyme-clean-and-dirty) . Two different data preprocessing techniques were applied: normalization without data augmentation (NODA) and normalization with data augmentation (DA). The two versions of the modern NASNet DNN architectures of different sizes were used, namely NASNetLarge and NASNetMobile, with the weights obtained after training on ImageNet dataset. These DNNs were trained in two experiments, namely with frozen layers, aka transfer learning (TL), and with unfrozen layers when all weights were updated (NOTL). After fivefold cross-validated training the best results for mean validation metrics were demonstrated by the models trained in C_NOTL_NODA, C_NOTL_DA, and C_TL_NODA experiments for NASNetLarge, and in C_TL_NODA by NASNetMobile. But the testing on the new unknown subset of Lyme disease images with the equal proportion of clean and dirty images gave quite different results. The models trained in the before-mentioned experiments demonstrated poor performance in comparison with the models trained in C_NOTL_DA, D_NOTL_DA experiments, and, especially in D_C_NOTL_DA ones. The successful training strategy should include tuning ImageNet-trained weights without their freezing (NOTL) and intensive data augmentation (DA), but it would not be enough if some diverse relevant background will not be added. That is why the additional methods of “context-relevant data augmentation” (CRDA) mimicked by inclusion of D_C data configuration should be considered. These findings can be used for the more effective selection of training experiments of DNNs in the healthcare context in the view of the small number of available medical data in practice.KeywordsNeural networksDeep learningSmall dataset problemData augmentationTransfer learningHealth careLyme disease

Wearable accelerometers are widely used as an ecologically valid and scalable solution for long-term at-home sleep monitoring in both clinical research and care. In this study, we applied a deep learning domain adversarial convolutional neural network (DACNN) model to this task and demonstrated that this new model outperformed existing sleep algorithms in classifying sleep–wake and estimating sleep outcomes based on wrist-worn accelerometry. This model generalized well to another dataset based on different wearable devices and activity counts, achieving an accuracy of 80.1% (sensitivity 84% and specificity 58%). Compared to commonly used sleep algorithms, this model resulted in the smallest error in wake after sleep onset (MAE of 48.7, Cole–Kripke of 86.2, Sadeh of 108.2, z-angle of 57.5) and sleep efficiency (MAE of 11.8, Cole–Kripke of 18.4, Sadeh of 23.3, z-angle of 9.3) outcomes. Despite being around for many years, accelerometer-alone devices continue to be useful due to their low cost, long battery life, and ease of use. Improving the accuracy and generalizability of sleep algorithms for accelerometer wrist devices is of utmost importance. We here demonstrated that domain adversarial convolutional neural networks can improve the overall accuracy, especially the specificity, of sleep–wake classification using wrist-worn accelerometer data, substantiating its use as a scalable and valid approach for sleep outcome assessment in real life.

Domain-invariant representation learning using an unsupervised domain adversarial adaptation deep neural network

Comparative study on deep transfer learning strategies for cross-system and cross-operation-condition building energy systems fault diagnosis

Human behaviour recognition is one of the most fundamental tasks in Industrial Internet of Behaviour (IIoB) and is crucial for the safe and reliable IIoB. Existing methods lacks adaptability and transferability. In addition, there is a data isolation problem among different users. Therefore, there is an urgent requirement to construct a secure and adaptive human behaviour recognition model in IIoB without violating the privacy of users. Mamba, a structured state space model that integrates a selection mechanism and a scan module, is used for time series modelling tasks. To tackle the aforementioned problems, an Federated Learning-based lightweight human behaviour recognition model with selective state space models is proposed. First, we design a human behaviour recognition model integrating Mamba and residual structure to achieve lightweight and secure human behaviour modelling. In addition, considering data privacy and training efficiency, a decentralised dynamic FL framework is designed to achieve lightweight and secure model collaborative training, including: initial selection of source users, model aggregation strategy based on dynamic weighting, and fine-tuning module based on small-sample data, to improve the training efficiency of the model and the accuracy of human behaviour recognition. Extensive experiments are conducted to prove the superior performance of the proposed method.

OBJECTIVE. Deep learning applications in radiology often suffer from overfitting, limiting generalization to external centers. The objective of this study was to develop a high-quality prostate segmentation model capable of maintaining a high degree of performance across multiple independent datasets using transfer learning and data augmentation. MATERIALS AND METHODS. A retrospective cohort of 648 patients who underwent prostate MRI between February 2015 and November 2018 at a single center was used for training and validation. A deep learning approach combining 2D and 3D architecture was used for training, which incorporated transfer learning. A data augmentation strategy was used that was specific to the deformations, intensity, and alterations in image quality seen on radiology images. Five independent datasets, four of which were from outside centers, were used for testing, which was conducted with and without fine-tuning of the original model. The Dice similarity coefficient was used to evaluate model performance. RESULTS. When prostate segmentation models utilizing transfer learning were applied to the internal validation cohort, the mean Dice similarity coefficient was 93.1 for whole prostate and 89.0 for transition zone segmentations. When the models were applied to multiple test set cohorts, the improvement in performance achieved using data augmentation alone was 2.2% for the whole prostate models and 3.0% for the transition zone segmentation models. However, the best test-set results were obtained with models fine-tuned on test center data with mean Dice similarity coefficients of 91.5 for whole prostate segmentation and 89.7 for transition zone segmentation. CONCLUSION. Transfer learning allowed for the development of a high-performing prostate segmentation model, and data augmentation and fine-tuning approaches improved performance of a prostate segmentation model when applied to datasets from external centers.

Neuroscience studies have shown that incorporating gaze view with third view perspective has a great influence to correctly infer human behaviors. Given the importance of both first and third person observations for the recognition of human behaviors, we propose a method that incorporates these observations in a technical system to enhance the recognition of human behaviors, thus improving beyond third person observations in a more robust human activity recognition system. First, we present the extension of our proposed semantic reasoning method by including gaze data and external observations as inputs to segment and infer human behaviors in complex real-world scenarios. Then, from the obtained results we demonstrate that the combination of gaze and external input sources greatly enhance the recognition of human behaviors. Our findings have been applied to a humanoid robot to online segment and recognize the observed human activities with better accuracy when using both input sources; for example, the activity recognition increases from 77% to 82% in our proposed pancake-making dataset. To provide completeness of our system, we have evaluated our approach with another dataset with a similar setup as the one proposed in this work, that is, the CMU-MMAC dataset. In this case, we improved the recognition of the activities for the egg scrambling scenario from 54% to 86% by combining the external views with the gaze information, thus showing the benefit of incorporating gaze information to infer human behaviors across different datasets.

EEG-based emotion identification from nerve conduction mechanisms: A gustatory-emotion coupling model combined with multiblock attention module

Improving the prediction of extreme wind speed events with generative data augmentation techniques