Human Activity Recognition from Environmental Background Sounds for Wireless Sensor Networks

Speech recognition can be defined as the process of converting voice signals into the ranks of the word, by applying a specific algorithm that is implemented in a computer program. The research of speech recognition in Indonesia is relatively limited. This paper has studied methods of feature extraction which is the best among the Linear Predictive Coding (LPC) and Mel Frequency Cepstral Coefficients (MFCC) for speech recognition in Indonesian language. This is important because the method can produce a high accuracy for a particular language does not necessarily produce the same accuracy for other languages, considering every language has different characteristics. Thus this research hopefully can help further accelerate the use of automatic speech recognition for Indonesian language. There are two main processes in speech recognition, feature extraction and recognition. The method used for comparison feature extraction in this study is the LPC and MFCC, while the method of recognition using Hidden Markov Model (HMM). The test results showed that the LPC method is better than MFCC in Indonesian language speech recognition.

In this work, we propose a Human Activity Prediction (HAP) system using activity sequence spanning trees constructed from a life-log created by a video sensor-based daily Human Activity Recognition (HAR) system using time-sequential Independent Component (IC)-based depth silhouette features with Hidden Markov Models (HMMs). In the daily HAR system, the IC features are extracted from the collection of the depth silhouettes containing various daily human activities such as walking, sitting, lying, cooking, eating etc. Using these features, HMMs are used to model the time sequential features and recognize various human activities. The depth silhouette-based human activity recognition system is used to recognize daily human activities automatically in real time, which creates a life-log of daily activity events. In this work, we propose a method for human activity prediction using fixed-length activity sequence spanning trees based on the life-log. Utilizing the consecutive activities recorded in an activity sequence database (i.e. life-log) for a specific period of time of each day over a period such as a month, the fixed-length spanning trees can be constructed for the sequences starting with each activity where the leaf nodes contain the frequency of the fixed-length consecutive activity sequences. Once the trees are constructed, to predict an activity after a sequence of activities, we traverse the spanning trees until a path up to the previous node of the leaf nodes is matched with the testing pattern. Finally, we can predict the next activity based on the highest frequency of the leaf nodes along the matched path. The prediction experiments over the computer simulated data which is based on the daily logs show satisfactory results. Our video sensor-based human activity recognition and prediction systems can be utilized for practical applications such as smart and proactive healthcare.

Recognition of car horns based on principal component analysis of MEL frequency Cepstral coefficients and support vector machine

In this paper, we present a home-monitoring oriented human activity recognition benchmark database, based on the combination of a color video camera and a depth sensor. Our contributions are two-fold: 1) We have created a publicly releasable human activity video database (i.e., named as RGBD-HuDaAct), which contains synchronized color-depth video streams, for the task of human daily activity recognition. This database aims at encouraging more research efforts on human activity recognition based on multi-modality sensor combination (e.g., color plus depth). 2) Two multi-modality fusion schemes, which naturally combine color and depth information, have been developed from two state-of-the-art feature representation methods for action recognition, i.e., spatio-temporal interest points (STIPs) and motion history images (MHIs). These depth-extended feature representation methods are evaluated comprehensively and superior recognition performances over their uni-modality (e.g., color only) counterparts are demonstrated.

This paper presents the implementation of a practical voice recognition system using MATLAB (R2014b) to secure a given user’s system so that only the user may access it. Voice recognition systems have two phases, training and testing. During the training phase, the characteristic features of the speaker are extracted from the speech signal and stored in a database. In the testing phase, the stored audio features of the test voice sample are compared with the voice samples in the database and determined if a match exists. For this research, Mel Frequency Cepstral Coefficients (MFCCs) were chosen to represent the feature vectors of the user’s voice as it accurately simulates the behavior of the human ear. This characteristic of the MFCCs makes them an excellent measure of speaker characteristics. The feature matching process is then performed by subjecting the MFCCs to vector quantization using the LBG (Linde-Buzo-Gray) algorithm. In practical scenarios, noise is a major factor that adversely influences a voice recognition system. The paper addresses this issue by utilizing spectral subtraction to remove environmental noise affecting the speech signal thereby increasing the robustness of the system.

Real time detection of transitions between activities based on sensor data is a valuable but somewhat untapped challenge. Detecting these transitions is useful for activity segmentation, for timing notifications or interventions, and for analyzing human behavior. In this work, we design and evaluate real time machine learning-based methods for automatic segmentation and recognition of continuous human daily activity. We detect activity transitions and integrate the change point detection algorithm with smart home activity recognition to segment human daily activities into separate actions and correctly identify each action. Experiments with on real-world smart home datasets suggest that using transition aware activity recognition algorithms lead to best performance for detecting activity boundaries and streaming activity segmentation.

Prior studies in the automatic classification of voice quality have mainly studied the use of the acoustic speech signal as input. Recently, a few studies have been carried out by jointly using both speech and neck surface accelerometer (NSA) signals as inputs, and by extracting mel-frequency cepstral coefficients (MFCCs) and glottal source features. This study examines simultaneously-recorded speech and NSA signals in the classification of voice quality (breathy, modal, and pressed) using features derived from three self-supervised pre-trained models (wav2vec2-BASE, wav2vec2-LARGE, and HuBERT) and using a support vector machine (SVM) as well as convolutional neural networks (CNNs) as classifiers. Furthermore, the effectiveness of the pre-trained models is compared in feature extraction between glottal source waveforms and raw signal waveforms for both speech and NSA inputs. Using two signal processing methods (quasi-closed phase (QCP) glottal inverse filtering and zero frequency filtering (ZFF)), glottal source waveforms are estimated from both speech and NSA signals. The study has three main goals: (1) to study whether features derived from pre-trained models improve classification accuracy compared to conventional features (spectrogram, mel-spectrogram, MFCCs, i-vector, and x-vector), (2) to investigate which of the two modalities (speech vs. NSA) is more effective as input in the classification task with pre-trained model-based features, and (3) to evaluate whether the deep learning-based CNN classifier can enhance the classification accuracy in comparison to the SVM classifier. The results revealed that the use of the NSA input showed better classification performance compared to the speech signal. Between the features, the pre-trained model-based features showed better classification accuracies, both for speech and NSA inputs compared to the conventional features. The two classifiers performed equally well for all the pre-trained model-based features for both speech and NSA signals. It was also found that the HuBERT features performed better than the wav2vec2-BASE and wav2vec2-LARGE features for both speech and NSA inputs. In particular, when compared to the conventional features, the HuBERT features showed an absolute accuracy improvement of 3%–6% for speech and NSA signals in the classification of voice quality.

We make use of information inside infant’s cry signal in order to identify the infant’s psychological condition. Gaussian mixture models (GMMs) are applied to distinguish between healthy full-term and premature infants, and those with specific medical problems available in our cry database. Cry pattern for each pathological condition is created by using adapted boosting mixture learning (BML) method to estimate mixture model parameters. In the first experiment, test results demonstrate that the introduced adapted BML method for learning of GMMs has a better performance than conventional EM-based reestimation algorithm as a reference system in multipathological classification task. This newborn cry-based diagnostic system (NCDS) extracted Mel-frequency cepstral coefficients (MFCCs) as a feature vector for cry patterns of newborn infants. In binary classification experiment, the system discriminated a test infant’s cry signal into one of two groups, namely, healthy and pathological based on MFCCs. The binary classifier achieved a true positive rate of 80.77% and a true negative rate of 86.96% which show the ability of the system to correctly identify healthy and diseased infants, respectively.

As robot assisted living is becoming increasingly important for elderly people, human daily activity recognition is necessary for human-robot interaction. In this paper, we proposed an approach to daily activity recognition for elderly people. This approach uses a single wearable inertial sensor worn on the right thigh of a human subject to collect motion data. This setup can reduce the obtrusiveness to the minimum. Human daily activities can be recognized in two steps. First, two neural networks are used to classify the basic activities. Second, the activity sequence is modeled by an HMM to consider the sequential constraints exhibited in human daily life and the modified short-time Viterbi algorithm is used for realtime daily activity recognition as the fine-grained classification. We conducted experiments in a mock apartment environment and the obtained results proved the effectiveness and accuracy of our approach.

Respiratory disease presents major global health challenges, and its early detection could save numerous lives, prompting the demand for accurate and non-intrusive diagnostic methods. This study presents a multi-model machine learning framework for classifying respiratory diseases (COPD, Bronchiolitis, Pneumonia, URTI, and Healthy) using audio data from the Respiratory Sound Database. We extract Mel frequency cepstral coefficients (MFCCs) and apply data augmentation to address class imbalance. We propose a novel hybrid model, ConvMambda, and compare its performance against other popular classification models, including 1D CNN, random forest, SVM, and LSTM. The ConvMamba model achieves the highest test accuracy (90.77%) and macro F1-score (0.8932), outperforming 1D-CNN (90% accuracy) and random forest (82% accuracy), SVM (68% accuracy), and LSTM (68% accuracy). We analyze the performance across classes, highlighting challenges with minority classes (Bronchiolitis, URTI) and propose strategies to mitigate imbalance. Our findings demonstrate the potential of audio-based diagnostics, with the ConvMamba model offering the superior performance for respiratory diseases classification.

In this paper, we propose two estimators for the autocorrelation sequence of a periodic signal in additive noise. Both estimators are formulated employing tables which contain all the possible products of sample pairs in a speech signal frame. The first estimator is based on a pitch-synchronous averaging. This estimator is statistically analyzed and we show that the signal-to-noise ratio (SNR) can be increased up to a factor equal to the number of available periods. The second estimator is similar to the former one but it avoids the use of those sample products more likely affected by noise. We prove that, under certain conditions, this estimator can remove the effect of an additive noise in a statistical sense. Both estimators are employed to extract mel frequency cepstral coefficients (MFCCs) as features for robust speech recognition. Although these estimators are initially conceived for voiced speech frames, we extend their application to unvoiced sounds in order to obtain a coherent feature extractor. The experimental results show the superiority of the proposed approach over other MFCC-based front-ends such as the higher-lag autocorrelation spectrum estimation (HASE), which also employs the idea of avoiding those autocorrelation coefficients more likely affected by noise.

Nowadays, the efficiency of multiagent systems in wireless sensor networks prompts the researchers to use these emerging mobile software packets in different simulated approaches or real-world applications. Heterogeneous and distributed wireless sensor networks could be integrated with the multiagent systems to map the real-world challenges into the artificial intelligence world. The multiagent systems have been applied from simulated approaches like object detection/tracking, healthcare, control/assistant, and security systems to real-world applications, including medical/human-care systems and unmanned aerial vehicles. Furthermore, the integration of wireless sensor networks with multiagent systems have emerged novel application, which is known as mobile robots. However, the extensive use of mobile agents in wireless sensor networks has posed different challenges for researchers, including security, resource, and timing limitation. In this work, we review recent simulated approaches and real-world applications of multiagent systems in wireless sensor networks, in which a set of common factors about the things that have been studied are extracted and compared to analyze the performance of mobile agent–based systems in the wireless sensor networks, as well. This analysis provides new research directions about multiagent systems in wireless sensor networks for interested researchers. Finally, a novel framework for dealing with the challenges of multiagent-based applications in the wireless sensor networks which have been mentioned is suggested.

In this work, we present a real-time application in the scope of human daily activity recognition for robot-assisted living as an extension of our previous work [1]. We implemented our approach using Robot Operating System (ROS) environment, combining different modules to enable a robot to perceive the environment using different sensor modalities. Thus, the robot can move around, detect, track and follow a person to monitor daily activities wherever the person is. We focus our attention mainly on the robotic application by integrating several ROS modules for navigation, activity recognition and decision making. Reported results show that our framework accurately recognizes human activities in a real time application, triggering proper robot (re)actions, including spoken feedback for warnings and/or appropriate robot navigation tasks. Results evidence the potential of our approach for robot-assisted living applications.

An automatic human action representation and recognition technique is proposed in this paper. Appearance-change problem due to human wearing dresses and body shapes is also investigated in this study for automatic human action recognition. A tuned eigenspace technique is proposed for automatic human posture and/or motion recognition that successfully overcome the preceding problems. We employ image pre-processing by Gaussian and Sobel edge filter, called the first stage tuning, for reducing a dress effect, and a mean eigenspace produced by taking a mean of the similar postures, called the second stage tuning, for avoiding the preceding problems. An eigenspace called a tuned eigenspace is obtained from the mentioned processes and it is used for further recognition of unfamiliar postures and actions. The proposed method is compared with a related technique and the robustness of this approach is presented.

Over the past decades, many WSN (Wireless Sensor Network) systems have emerged. Today, the WSN systems with different technologies have started to become parts of Internet of Things (IoT) and Cyber-Physical Systems(CPSs). This situation has also led to the emergence of heterogeneous and autonomous WSN structures. In these types of complex projects, involving different WSN systems, it is necessary to well define the structure of WSN heterogeneity in order to perform some important operations such as interoperability and resource-sharing. To this end, the paper presents a WSN heterogeneity classification model. In the classification, WSN heterogeneity is addressed from a general perspective. The classification considers not only the heterogeneity within a WSN but also the heterogeneity which can be occurred in the scope of IoT/CPS projects. As far as is known, this type of classification on WSN heterogeneity has been proposed for the first time in literature.