Action Recognition in Compressed Domain Using Residual Information

A low computational complexity bandwidth extension method for mobile audio coding

The compressed domain video watermarking has been less expensive since complete decoding and then re-encoding is not required for watermark embedding. Handling synchronization error due to embedding is a challenging task, especially, due to motion compensation. In this paper, we propose an embedding scheme that can minimize desynchronization in the watermark extraction process at the decoder for embedding in compressed videos with better visual quality. Experimental results show the effectiveness of the proposed scheme.

Compressed domain speech enhancement method based on ITU-T G.722.2

Current compressed video action recognition methods are mainly based on complete data. However, in a real transmission scenario, the compressed video packets are usually disorderly received and even lost due to network jitters or congestion. To recognize actions in early phases with limited packets, e.g. for quickly forecasting possible potential risks, in this paper, we propose a Temporal Enhanced Multi-Stream Network (TEMSN) towards practical compressed video action recognition. First, we make use of three modalities in the compressed domain as complementary cues and build a multistream network to capture rich information from compressed video packets. Second, we design a temporal enhanced module based on an Encoder-Decoder structure, which is applied to each stream to infer missing packets, generating more accurate action dynamics. Thanks to the multiple modalities and their temporal enhancement, our approach better models actions with partial available compressed video packets. Experiments on the HMDB-51 and UCF-101 datasets validate its effectiveness and efficiency.

Recognizing human activity from highly sparse body sensor data is becoming an important problem in Internet of Medical Things (IoMT) industry. As IoMT currently uses batteryless or passive wearable body sensors for activity recognition, the data signals of these sensors have a high sparsity, which means that the time intervals of sensors’ readings are irregular and the number of sensors’ readings in time unit is frequently limited. Therefore, learning activity recognition models from temporally sparse data is challenging. Traditional machine learning techniques cannot be applicable in this scenario due to their focus on a single sensing modality requiring regular sampling rate. In this work, we propose an effective end-to-end deep neural network (DNN) model to recognize human activities from temporally sparse data signals of passive wearable sensors that improve the accuracy rate of human activity recognition. A dropout technique is used in the developed model to deal with sparsity problem and avoiding overfitting problem. In addition, optimization of the proposed DNN model was performed by evaluating a different number of hidden layers. Various experiments were conducted on a public clinical room dataset of sparse data signals to compare the performance of the proposed DNN model with the conventional and other deep learning approaches. The experimental results demonstrate that the proposed DNN model outperforms the existing state-of-the-art methods in terms of lower inference delay and activity recognition accuracy.

Currently, most of the researches in molecular communication (MC) domain focus on the static MC scenarios. However, some envisioned important MC applications require mobile MC system. The investigation on mobile MC, especially the signal detection of mobile MC is limited. This work considers the problem of signal detection for mobile MC scenarios where the receiver nano-machine performs random movement. Due to the random movement of the receiver, the channel impulse response (CIR) changes over time which makes the received signal stochastic and complicated. This further complicates the signal detection in mobile MC and leads to that the state-of-the-art signal detection schemes for static MC scenarios fail for the mobile MC scenarios. To solve this issue, an adaptive detection scheme has been proposed by our group previously, based on dynamic estimation of the stochastically varying distance between the transmitter and receiver and the reconstruction of CIR in each interval. However, its computational complexity is high. Limited capability of current nano-machines desire low-complexity detection algorithm. In this work, we further propose an adaptive detection scheme for mobile MC with a low computational complexity by utilizing the local convex property of the CIR. With on-off keying (OOK) modulation, the signal of symbol "1" presents local convex property while that of symbol "0" presents local concave property. The convexity extent varies with the stochastic distance. A simple indicator, local maximum convexity is proposed which adapts to the stochastic distance. By comparing the adaptive indicator with an adaptive threshold within each symbol interval, the signal is detected without the need to estimate the stochastically changing distance or to reconstruct the CIR. Therefore, the computational load is effectively reduced. Numerical simulations are performed to evaluate the proposed scheme. The results show that the proposed scheme achieves good detection accuracy with low computational complexity and it could be a promising detection scheme for mobile MC scenarios.

Local and global self-attention enhanced graph convolutional network for skeleton-based action recognition

Any action dataset may contain similar classes such as running, walking and jogging. Therefore, equivalent probabilities may be provided for different classes upon action classification. In this case, the classifier cannot indubitably assign a class to a given sample. To address this problem, we propose a new hybrid classifier to automatically compress the features and classify them using SVM with polynomial or sigmoid kernels. Furthermore, we hypothesize that motion saliency detection can strength the power of motion feature extraction in the bag of visual words framework (BoVW). To this end, we evaluate the effect of 3D-discrete wavelet transform (3D-DWT), as the preprocessing step, on motion feature extraction. The experimental results show that the proposed framework achieves promising results on KTH, Weizmann, and URADL datasets, and outperforms recent state-of-the-art approaches.

Multi-view action recognition based on action volumes, fuzzy distances and cluster discriminant analysis

In Scalable Video Coding (SVC), motion estimation and inter-layer prediction play an important role in elimination of temporal and spatial redundancies between consecutive layers. This paper evaluates the performance of widely accepted block matching algorithms used in various video compression standards, with emphasis on the performance of the algorithms for a didactic scalable video codec. Many different implementations of Fast Motion Estimation Algorithms have been proposed to reduce motion estimation complexity. The block matching algorithms have been analyzed with emphasis on Peak Signal to Noise Ratio (PSNR) and computations using MATLAB. In addition to the above comparisons, a survey has been done on Spiral Search Motion Estimation Algorithms for Video Coding. A New Modified Spiral Search (NMSS) motion estimation algorithm has been proposed with lower computational complexity. The proposed algorithm achieves 72% reduction in computation with a minimal (<1dB) reduction in PSNR. A brief introduction to the entire flow of video compression H.264/SVC is also presented in this paper.

Over the last few decades, human action recognition has become one of the most challenging tasks in the field of computer vision. Effortless and accurate extraction of 3D skeleton information has been recently achieved by means of economical depth sensors and state-of-the-art deep learning approaches. In this study, we introduce a novel bag-of-poses framework for action recognition using 3D skeleton data. Our assumption is that any action can be represented by a set of predefined spatiotemporal poses. The pose descriptor is composed of three parts. The first part is concatenation of the normalized coordinate of the skeleton joints. The second part is consisted of temporal displacement of the joints constructed with predefined temporal offset, and the third part is temporal displacement with the previous frame in the sequence. In order to generate the key poses, we apply K-means clustering over all the training pose descriptors of the dataset. SVM classifier is trained with the generated key poses to classify an action pose. Accordingly, every action in the dataset is encoded with key pose histograms. ELM classifier is used for action recognition due to its fast, accurate and reliable performance compared to the other classifiers. The proposed framework is validated with five publicly available benchmark 3D action datasets and achieved state-of-the-art results on three of the datasets and competitive results on the other two datasets compared to the other methods.

In the construction industry, construction machinery are an important factor in the overall productivity and efficiency of a worksite. Thus, emphasis is put on the monitoring of actions conducted by construction machinery. This was traditionally done manually by humans, which is a timeconsuming and laborious task. Automatic action recognition of construction machinery is therefore needed. The field of action recognition is predominantly occupied by Deep Learning approaches and several previous works focused on adapting such approaches for construction machinery. However, the issue of obtaining training data is particularly troublesome for construction machinery. Our previous work proposed a Deep Learning method for learning an action recognition model from training data generated in a simulator using video filters but the precise contributions of the introduced video filter were unclear. The purpose of this study is therefore to clarify the effects of video filters for learning an action recognition model for construction machinery from simulated training data.

Many advanced video applications require manipulations of compressed video signals. Popular video manipulation functions include overlap (opaque or semitransparent), translation, scaling, linear filtering, rotation, and pixel multiplication. We propose algorithms to manipulate compressed video in the compressed domain. Specifically, we focus on compression algorithms using the discrete cosine transform (DCT) with or without motion compensation (MC). Such compression systems include JPEG, motion JPEG, MPEG, and H.261. We derive a complete set of algorithms for all aforementioned manipulation functions in the transform domain, in which video signals are represented by quantized transform coefficients. Due to a much lower data rate and the elimination of decompression/compression conversion, the transform-domain approach has great potential in reducing the computational complexity. The actual computational speedup depends on the specific manipulation functions and the compression characteristics of the input video, such as the compression rate and the nonzero motion vector percentage. The proposed techniques can be applied to general orthogonal transforms, such as the discrete trigonometric transform. For compression systems incorporating MC (such as MPEG), we propose a new decoding algorithm to reconstruct the video in the transform domain and then perform the desired manipulations in the transform domain. The same technique can be applied to efficient video transcoding (e.g., from MPEG to JPEG) with minimal decoding. >

Activity recognition has received considerable attention in many research fields, such as industrial and healthcare fields. However, many researches about activity recognition have focused on static activities and dynamic activities in current literature, while, the transitional activities, such as stand-to-sit and sit-to-stand, are more difficult to recognize than both of them. Consider that it may be important in real applications. Thus, a novel framework is proposed in this paper to recognize static activities, dynamic activities, and transitional activities by utilizing stacked denoising autoencoders (SDAE), which is able to extract features automatically as a deep learning model rather than utilize manual features extracted by conventional machine learning methods. Moreover, the resampling technique (random oversampling) is used to improve problem of unbalanced samples due to relatively short duration characteristic of transitional activity. The experiment protocol is designed to collect twelve daily activities (three types) by using wearable sensors from 10 adults in smart lab of Ulster University, the experiment results show the significant performance on transitional activity recognition and achieve the overall accuracy of 94.88% on three types of activities. The results obtained by comparing with other methods and performances on other three public datasets verify the feasibility and priority of our framework. This paper also explores the effect of multiple sensors (accelerometer and gyroscope) to determine the optimal combination for activity recognition.

In online courses, discussion forums play a key role in enhancing student interaction with peers and instructors. Due to large enrolment sizes, instructors often struggle to respond to students in a timely manner. To address this problem, both traditional Machine Learning (ML) (e.g., Random Forest) and Deep Learning (DL) approaches have been applied to classify educational forum posts (e.g., those required urgent responses vs. that did not). However, there lacks an in-depth comparison between these two kinds of approaches. To better guide people to select an appropriate model, we aimed at providing a comparative study on the effectiveness of six frequently-used traditional ML and DL models across a total of seven different classification tasks centering around two datasets of educational forum posts. Through extensive evaluation, we showed that (i) the up-to-date DL approaches did not necessarily outperform traditional ML approaches; (ii) the performance gap between the two kinds of approaches can be up to 3.68% (measured in F1 score); (iii) the traditional ML approaches should be equipped with carefully-designed features, especially those of common importance across different classification tasks. Based on the derived findings, we further provided insights to help instructors and educators construct effective classifiers for characterizing educational forum discussions, which, ultimately, would enable them to provide students with timely and personalized learning support.