Weapon Activity Recognition

Due to its important applications in industry, the research of action recognition has attracted many attention. In particular, skeleton-based human action recognition methods have evolved to become more and more competitive. Nevertheless, occlusion is still a very challenging task in human action recognition as yet. Off-the-shelf works are usually based on complete skeleton data, few people consider action recognition in occlusion. For the sake of improving the recognition accuracy on occluded skeleton data, we put forward an occlusion-aware multi-stream fusion graph convolutional network (dubbed MSFGCN). Multiple streams are comprised in MSFGCN, and different occlusion cases can be disposed by different streams. Besides, in order to construct more discriminative features, multi-modal features were extracted simultaneously, such as joint coordinates, relative coordinates, small-scale temporal differences and large-scale temporal differences. In particular, it is the first time to take advantage of motion features at large and small scales at the same time, which helps to distinguish actions with different motion amplitudes. What's more, considering the different importance of different parts of the human body for different action recognition, the content adaptation operation is used to further optimize the recognition performance. Effective training strategies are also presented to further improve the performance of the model. Experimental results show that the proposed MSFGCN has a great advantage over other methods on occluded skeleton datasets. To showcase the effectiveness of different modules, extensive ablation experiments are performed on various skeleton action recognition datasets.

The accuracy of video-based action recognition depends largely on the extraction and utilization of optical flow, especially in two-stream networks. The original intention of the introduction of optical flow is to use the time information contained in video, however, the subsequent work shows that optical flow is useful for action recognition because it is invariant to appearance. In this article, we study and discuss this point of view, and propose optical flow enhancement algorithms to improve action recognition accuracy. Our enhancement algorithms improve the invariance to appearance of the representation in optical flow without losing time information, and every action recognition network with optical flow can benefit from our algorithms. We conduct a series of experiments to validate the influence of the proposed algorithms with TSN in terms of several datasets and optical flow calculation methods. As a result, we prove that first order differential algorithms are effective, TSN with our enhancement module significantly outperform original network. Based on these experiments, we also verify the importance of invariance to appearance in optical flow, and provide a reference for the follow-up study of improving action recognition accuracy.

An action is something which is done by an agent. Most action recognition researchers merely focus on the actions to be recognized, and ignore the differences of agents. Philosophers and behaviorists discover that actions are common among many species, but are performed in different ways and with different levels of sophistication. In this paper, in order to bridge action recognition tasks between different agents, we introduce a new problem, cross-agent action recognition, i.e., recognizing action for one particular agent (target) while training from other agents (source). We model this problem under three different scenarios: single source and single target, multiple sources and single target, and multiple sources and multiple targets. To this end, corresponding methods based on transfer learning are proposed to address these problems. We further design three different strategies to model the situation when a partial labeled data is provided for the target. Experimental results show that the performances of the transfer method are generally better than those of the comparative method without transfer learning, especially when we have multiple sources. Particularly, the transfer method outperforms the others significantly when the source is a human adult. In addition, cross-agent method significantly improves the results when partially labeled data is provided for the target. These demonstrate that for action recognition, knowledge can be transferred across different agents. A straightforward application of this finding is to use human action (training data is abundant) data to enhance animal action recognition.

Inertial measurement units (IMU) are included in uncountable consumer electronic products, e.g. smart phones. IMUs are state of the art for research in human motion analysis, context and activity recognition. Sensor setups to estimate the activity of a subject, e.g. standing, sitting, walking, climbing stairs etc., typically require a multitude of sensor nodes to provide reliable estimations. In our paper, we present a sensor system for enhanced context and activity recognition while requiring a minimal set of sensor nodes. Our system combines a pressure sensor featuring high spatial resolution (1248 sensitive points) with an IMU. The pressure sensor is typically placed inside a shoe and measures the force exerted on the feet. Our system eases the detection of many situations: differentiating between sitting and standing, walking and climbing stairs etc. It is adjustable to virtually any shoe and therefore could be included in projects requiring specialized footwear (e.g. running, weight lifting or biking shoes). Data can be stored on the sensor node or it can be transferred wirelessly to a remote aggregator device, i.e. a device responsible for data stream reception, sensor control and analysis. The aggregator can be implemented in a smart phone or a PC. Our system can in real-time detect situations of sitting, standing and walking automatically. The system's low-power implementation allows for run-times exceeding 14 hours. In a multi-system setup the systems can be synchronized with a node-to-node offset in the order of 10μs. The pressure can be sampled at a maximum of 120Hz, the inertial data is available at a rate of 512 Hz. Our small-footprint real-time classifier can distinguish between the situations standing, sitting, and walking with an accuracy above 99%.

High-level and longer-term activity recognition has great potentials in areas such as medical diagnosis and human behavior modeling. So far however, activity recognition research has mostly focused on lowlevel and short-term activities. This paper therefore makes a first step towards recognition of high-level activities as they occur in daily life. For this we record a realistic 10h data set and analyze the performance of four different algorithms for the recognition of both low- and high-level activities. Here we focus on simple features and computationally efficient algorithms as this facilitates the embedding and deployment of the approach in real-world scenarios. While preliminary, the experimental results suggest that the recognition of high-level activities can be achieved with the same algorithms as the recognition of low-level activities.

Visual attention mechanism has been widely used in computer vision and plays a vital role in the research of human action recognition. In this paper, we explore a novel moving target detection mechanism for human action recognition and propose a new 3D CNN (3D Convolutional Neural Network) model, dubbed LaM-2SRN (Local Features Enhanced and Moving target detected 2Stream-ResNet) for extracting and learning attention-enhanced spatiotemporal features. The contributions of this paper are as follow: First, the traditional CAM (Class Activation Map) based visual attention algorithm is used to obtain the optical flow information of the human region, thus eliminating the influence of irrelevant optical flow information (such as background clutter). Second, the ViBe algorithm is used to identify the moving target in the continuous frame, retain the optical flow information of the moving target, and make it complement with the optical flow information of the human region, to obtain a complete motion descriptor. After the motion information is marked in the video frame, we put the marked video frames and the original video frames into the 3D CNN and 2D CNN models respectively, to acquired mixed descriptor, which serves as the basis for video classification. Different from most of the previous target detection algorithms, this paper not only detects the human target, but also detects the moving foreground targets in the video, so that the pre-trained CNN model can obtain more complete motion information from videos. We use RGB-only video data for evaluation on two benchmarks: UCF101 and HMDB51, the experimental results demonstrate that our LaM-2SRN is comparable to the previous state-of-the-art algorithms.

Recent years have seen a rise in the use of various machine learning techniques in computer vision, particularly in posing feature-based human action recognition which includes convolutional neural networks (CNN) and recurrent neural network (RNN). CNN-based methods are useful in recognizing human actions for combined motions (i.e., standing up, hand shaking, walking). However, in case of uncertainty of camera motion, occlusion, and multiple people, CNN suppresses important feature information and is not efficient enough to recognize variations for human action. Besides, RNN with long short-term memory (LSTM) requires more computational power to retain memories to classify human actions. This research proposes an extended framework based on capsule network using silhouette pose features to recognize human actions. Proposed extended framework achieved high accuracy of 95.64% which is higher than previous research methodology. Extensive experimental validation of the proposed extended framework reveals efficiency which is expected to contribute significantly in action recognition research.

Activity recognition using multiplex limited penetrable visibility graph

Facial action provides various types of messages for human communications. Recognizing spontaneous facial actions, however, is very challenging due to subtle facial deformation, frequent head movements, and ambiguous and uncertain facial motion measurements. As a result, current research in facial action recognition is limited to posed facial actions and often in frontal view.

Facial action provides various types of messages for human communications. Recognizing spontaneous facial actions, however, is very challenging due to subtle facial deformation, frequent head movements, and ambiguous and uncertain facial motion measurements. As a result, current research in facial action recognition is limited to posed facial actions and often in frontal view.

Based on 549 literatures with the theme of “sports artificial intelligence” and other keywords in web of science database since 1995, this paper uses CiteSpace V software for visualization processing and analysis, and combs the country, discipline distribution, research hotspots and evolution trend of sports artificial intelligence research in recent 25 years by means of visual knowledge mapping, and discusses its research progress and development direction. 1) The research area of sports artificial intelligence is widely distributed, among which the United States, China and Germany are in the leading position. 2) Sports artificial intelligence research involves many disciplines, mainly using and learning from the research methods and theoretical perspectives of computer science, engineering, sports science and other disciplines. 3) The frequency and centrality of keywords confirm that machine learning is the main direction in the field of sports artificial intelligence, artificial neural network is the main algorithm, and data mining is the basis of practice and research. 4) Research hotspots include simple activity recognition and energy consumption research based on wearable accelerometer technology; action analysis and damage prevention and control research based on wearable sensor; computer vision scene classification research based on convolution neural network algorithm; analysis and prediction of physical fitness and technology and tactics based on computer vision; human posture recognition technology based on computer Deep learning.KeywordsArtificial intelligenceSportsKnowledge mapMachine learningComputer visionNeural network

Research in human activity recognition (HAR) requires a huge amount of data, but it is not easy to collect such measured sensor data. Besides, there is no much application of data augmentation (DA) in HAR. This study proposes Octave Mix as a novel synthetic-style DA method for sensor-based HAR. The proposed method uses frequency decomposition to intersect low- and high-frequency waveforms. In addition, we propose a DA ensemble method and a training algorithm to ensure robustness to the original sensor data while applicable to various feature representations. To evaluate the Octave Mix method’s effectiveness, we conduct experiments using four different benchmark datasets of sensor-based HAR and achieve high estimation accuracy in our results. Furthermore, we demonstrate that ensembling two DA strategies: Octave Mix with rotation and mixup with rotation, achieves higher accuracy.

Initializing the weights of a multilayer perceptron for activity and emotion recognition

Participatory exposure research, which tracks behaviour and assesses exposure to stressors like air pollution, traditionally relies on time-activity diaries. This study introduces a novel approach, employing machine learning (ML) to empower laypersons in human activity recognition (HAR), aiming to reduce dependence on manual recording by leveraging data from wearable sensors. Recognising complex activities such as smoking and cooking presents unique challenges due to specific environmental conditions. In this research, we combined wearable environment/ambient and wrist-worn activity/biometric sensors for complex activity recognition in an urban stressor exposure study, measuring parameters like particulate matter concentrations, temperature, and humidity. Two groups, Group H (88 individuals) and Group M (18 individuals), wore the devices and manually logged their activities hourly and minutely, respectively. Prioritising accessibility and inclusivity, we selected three classification algorithms: k-nearest neighbours (IBk), decision trees (J48), and random forests (RF), based on: (1) proven efficacy in existing literature, (2) understandability and transparency for laypersons, (3) availability on user-friendly platforms like WEKA, and (4) efficiency on basic devices such as office laptops or smartphones. Accuracy improved with finer temporal resolution and detailed activity categories. However, when compared to other published human activity recognition research, our accuracy rates, particularly for less complex activities, were not as competitive. Misclassifications were higher for vague activities (resting, playing), while well-defined activities (smoking, cooking, running) had few errors. Including environmental sensor data increased accuracy for all activities, especially playing, smoking, and running. Future work should consider exploring other explainable algorithms available on diverse tools and platforms. Our findings underscore ML’s potential in exposure studies, emphasising its adaptability and significance for laypersons while also highlighting areas for improvement.

Mobile activity recognition focuses on inferring the current activities of a mobile user by leveraging the rich sensory data that is available on today’s smart phones and other wearable sensors. The state of the art in mobile activity recognition research has focused on traditional classification learning techniques. In this paper, we propose the Mobile Activity Recognition System (MARS) where for the first time the classifier is built on-board the mobile device itself through ubiquitous data stream mining in an incremental manner. The advantages of on-board data stream mining for mobile activity recognition are: i) personalisation of models built to individual users; ii) increased privacy as the data is not sent to an external site; iii) adaptation of the model as the user’s activity profile changes. In our extensive experimental results using a recent benchmarking activity recognition dataset, we show that MARS can achieve similar accuracy when compared with traditional classifiers for activity recognition, while at the same time being scalable and efficient in terms of the mobile device resources consumption. MARS has been implemented on the Android platform for empirical evaluation.KeywordsMobile DeviceActivity RecognitionWearable SensorAndroid PlatformLocomotion TaskThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.