GAF-ConvNeXt Algorithm With Transfer Learning for Human Activity Recognition Based on Wearable Plastic Fiber Sensors

Human Activity Recognition (HAR) using built‐in sensors in wearable devices offers valuable insights for continuous behaviour monitoring and health assessment. However, existing HAR approaches face several key challenges: (1) limited capacity to capture multi‐scale temporal dependencies; (2) difficulty in recognising low‐motion or static activities when relying solely on inertial sensors such as 3D accelerometers (3D‐ACC); and (3) insufficient utilisation of auxiliary physiological signals in conventional single‐task learning frameworks. To address these limitations, we propose a Hierarchical‐Split Multi‐Scale Convolutional Network with Multi‐Task Learning for wearable HAR. The proposed model integrates a hierarchical‐split convolutional block to efficiently extract both local and global temporal features. It further employs a channel‐wise attention mechanism to adaptively fuse photoplethysmography (PPG) and 3D‐ACC signals, enabling the model to capture complementary physiological dynamics and improve the recognition of subtle or low‐motion activities. In addition, we introduce a multi‐task learning framework with an auxiliary heart rate estimation task, which enhances physiological representation learning without requiring additional annotations, thereby improving model robustness and generalisability. Extensive experiments on two public HAR datasets demonstrate that our method consistently outperforms existing approaches under both overall and cross‐subject evaluation protocols, highlighting its effectiveness in realistic, user‐independent settings.

Human Activity Recognition (HAR) plays a vital role in healthcare, smart homes, and security applications. Recent advancements in sensor technology and machine learning have significantly improved HAR’s capability to address real-world challenges, particularly in continuous monitoring of elderly individuals. This study proposes a HAR system utilizing the publicly available GOTOV dataset, which contains tri-axial accelerometer data collected from ankle, chest, and wrist sensors. The dataset comprises 16 activity classes, including several kinematically similar activities that present classification challenges. Current state-of-the-art (SOTA) methods demonstrate limited performance in distinguishing these similar activities. To overcome this limitation, we developed a novel Precision-Weighted Models Fusion (PWMF) algorithm that optimally combines predictions from multiple sensor-specific models. The time-series signals were first transformed into images using Gramian Angular Field (GAF) representation to facilitate training of Convolutional Neural Network (CNN) models for each sensor modality. Our fusion-based approach achieved an F1-score of 93% across all activity classes, demonstrating superior performance in multi-class human activity recognition compared to existing methods.

With the advancement and ubiquitousness of wearable devices, wearable sensor-based human activity recognition (HAR) has become a prominent research area in the healthcare domain and human-computer interaction. Inertial measurement unit (IMU) which can provide a wide range of information such as acceleration, angular velocity has become one of the most commonly used sensors in HAR. Recently, with the growing demand for soft and flexible wearable devices, mountable stretch sensors have become a new promising modality in wearable sensor-based HAR. In this paper, we propose a deep learning-based multi-modality HAR framework which consists of three IMUs and two fabric stretch sensors in order to evaluate the potential of stretch sensors independently and in combination with IMU sensors for the activity recognition task. Three different deep learning algorithms: long short-term memory (LSTM), convolutional neural network (CNN) and hybrid CNN-LSTM are deployed to the sensor data for automatically extracting deep features and performing activity classification. The impact of sensor type on recognition accuracy of different activities is also examined in this study. A dataset collected from the proposed framework, namely iSPL IMU-Stretch and a public dataset called w-HAR are used for experiments and performance evaluation.

This study presents a plastic optical fiber (POF) sensor for measuring the driver's gripping force applied to a steering wheel. This sensor consists of a POF wound around the steering wheel covered with a soft material. The sensing method is based on the bending radiation loss of the POF induced by the driver's gripping force applied to the steering wheel. The bending radiation loss is proportional to the gripping force, and the response time of the POF sensor is about 2 s. The experimental results show that a POF sensor can be effective for monitoring the driver gripping force, and hence driver fatigue.

Human Activity Recognition(HAR) based on Channel State Information(Csi)plays an increasingly impor-tant role in human-computer interaction. Traditional research requires a large amount of activity sample to train network model. However, collecting a great many of data causes waste of time and manpower. Some models can well identify the categories of activities in this scene, but when another scene is tested, the identification accuracy of the model will be reduced. Therefore it needs to re-collect data to retrain the model. We proposed a method which can transfer the knowledge learned from a scenario to a new scenario. It can also facilitate the model's knowledge learning from the source domain and quickly generalize to new tasks that contain only a small number of samples. Through this method, the model that maintains high accuracy and scalability to identify new category, we added attention mechanism can automatically extract features that are useful to the model and ignore some noise that negatively affects the model, meanwhile improve system stability and the effectiveness of activity recognition. We also performed the scaling and shifting(SS) transformation on the network, which could reduce the parameters of the model, improve the training speed, and avoid overfitting.

Human activity recognition is becoming the vital underpinning for a myriad of emerging applications in the field of human-computer interaction, mobile computing, and smart grid. Besides the utilization of up-to-date sensing techniques, modern activity recognition systems also require a machine learning (ML) algorithm that leverages the sensory data for identification purposes. In view of the unique characteristics of the measurement data and the ML challenges thereof, we propose a non-intrusive human activity recognition system that only uses existing commodity WiFi routers. The core of our system is a novel multiple kernel representation learning (MKRL) framework that automatically extracts and combines informative patterns from the Channel State Information (CSI) measurements. The MKRL firstly learns a kernel string representation from time, frequency, wavelet, and shape domains with an efficient greedy algorithm. Then it performs information fusion from diverse perspectives based on multi-view kernel learning. Moreover, different stages of MKRL can be seamlessly integrated into a multiple kernel learning framework to build up a robust and comprehensive activity classifier. Extensive experiments are conducted in typical indoor environments and the experimental results demonstrate that the proposed system outperforms existing methods and achieves a 98\% activity recognition accuracy.

Summary Gas-liquid flow is one of the most observed in nature and in many industrial applications. The presence of dense gas is a special case (i.e., a flow with a liquid/gas viscosity ratio of 10 or even less). Dense-gas/liquid flow is found in the Brazilian offshore production scenario, which is characterized by subsea lines operating under reservoir pressures typically ranging from 100 bar to 500 bar, temperatures around 60°C, and high CO2 content, while upstream of the choke valve pressures commonly range between 30 bar and 160 bar. Despite its relevance, the technical literature shows significant gaps. Although some studies employ dynamic pressure signals to analyze two-phase flows, most rely on numerical simulations or high-frequency experimental systems. Consequently, the analysis and classification of dense-gas/liquid flow using low-frequency differential pressure signals that are precisely the most widely available measurements in offshore operations have not been addressed. Likewise, there are no approaches that integrate multiple signal-to-image encoding techniques with advanced hybrid deep learning architectures for flow-pattern characterization from pressure-gradient signals. To address these gaps, we introduce a methodology that transforms low-frequency pressure-gradient time series into multivariate images using four complementary encoding techniques. Gramian angular fields (GAFs) [Gramian angular summation field (GASF)/Gramian angular difference field (GADF)] capture temporal correlations and angular relationships; recurrence plots (RPs) reveal recurrence patterns and nonlinear dynamic structures; and Markov transition fields (MTF) model state-transition probabilities while preserving the statistical structure of the signal. These representations are integrated into a unified 2 × 2 mosaic, generating a final image of size 2m × 2n. In addition, a hybrid deep learning model convolutional neural network (CNN)-transformer with coordinated attention is proposed, designed to leverage both local features and global dependencies. The convolutional modules extract fine spatial patterns, the Swin Transformer blocks learn long-range relationships through hierarchical self-attention, and the coordinated attention module preserves directional information by recalibrating feature responses along the horizontal and vertical axes. This combination enables the model to capture rapid local variations in the pressure gradient as well as global hydrodynamic trends, both essential in complex multiphase systems. By relying exclusively on low-frequency pressure-gradient signals, which are typically available in offshore monitoring systems, the proposed method offers strong industrial applicability. The results show that the hybrid model achieves 99% accuracy in classifying dense-gas/liquid flow patterns, significantly outperforming traditional approaches based on physical correlations or statistical analysis. These advancements position the methodology as a significant contribution to multiphase-flow characterization in the oil and gas industry.

Federated Learning (FL) has recently attracted great interest in sensor-based human activity recognition (HAR) tasks. However, in real-world environment, sensor data on devices is non-independently and identically distributed (Non-IID), e.g., activity data recorded by most devices is sparse, and sensor data distribution for each client may be inconsistent. As a result, the traditional FL methods in the heterogeneous environment may incur a drifted global model that causes slow convergence and a heavy communication burden. Although some FL methods are gradually being applied to HAR, they are designed for overly ideal scenarios and do not address such Non-IID problem in the real-world setting. It is still a question whether they can be applied to cross-device FL. To tackle this challenge, we propose ProtoHAR, a prototype-guided FL framework for HAR, which aims to decouple the representation and classifier in the heterogeneous FL setting efficiently. It leverages the global prototype to correct the activity feature representation to make the prototype knowledge flow among clients without leaking privacy while solving a better classifier to avoid excessive drift of the local model in personalized training. Extensive experiments are conducted on four publicly available datasets: USC-HAD, UNIMIB-SHAR, PAMAP2, and HARBOX, which are collected in both controlled environments and real-world scenarios. The results show that compared with the state-of-the-art FL algorithms, ProtoHAR achieves the best performance and faster convergence speed in HAR datasets.

The conventional Human Activity Recognition (HAR) systems use smartwatches, smartphones, and other wearable devices to autonomously derive consumer physical activities. However, implementing real-time HAR in resource-constrained applications is challenging. Computational offloading approaches must address network reliance, latency, and data privacy, which affect system performance. The present study proposes a low-cost, wearable HAR device that collects and transmits the Inertial Measurement Unit (IMU) telemetry signals to smartphones, enabling real-time on-device activity detection. Human actions are subjective and affected by physiological, environmental, and behavioural factors, complicating HAR systems' performance in personalized contexts. Generalized methods for existing systems reduce performance for new consumers or due to alterations in consumer locomotive signatures. Thus, our second goal is to enable the on-device personalization of wearable HAR models with minimum customer calibration. The HAR device, which is in the form of a smart-band, transfers data to and fro smartphones through Bluetooth Low Energy (BLE). A lightweight 1D convolutional neural network (CNN) is built and, using transfer learning, the network model is fine-tuned from real-time sensed data. On-device HAR inferences ensure real-time processing on smartphones without computational offloading. Smartphone-based on-device training for fine-tuning the model allows the HAR system to get customized without compromising privacy or computational costs. The experimental results show that the personalized classification achieves on average 98% accuracy when assessed using four benchmark datasets.

Enabling computer systems to respond to conversational human language is a challenging problem with wide-ranging applications in the field of robotics and human computer interaction. Specifically, in image searches, humans tend to describe objects in fine-grained detail like color or company, for which conventional retrieval algorithms have shown poor performance. In this paper, a novel approach for open vocabulary image retrieval, capable of selecting the correct candidate image from among a set of distractions given a query in natural language form, is presented. Our methodology focuses on generating a robust set of image-text projections capable of accurately representing any image, with an objective of achieving high recall. To this end, an ensemble of classifiers is trained on ImageNet for representing high-resolution objects, Cifar 100 for smaller resolution images of objects and Caltech 256 for challenging views of everyday objects, for generating category-based projections. In addition to category based projections, we also make use of an image captioning model trained on MS COCO and Google Image Search (GISS) to capture additional semantic/latent information about the candidate images. To facilitate image retrieval, the natural language query and projection results are converted to a common vector representation using word embeddings, with which query-image similarity is computed. The proposed model when benchmarked on the RefCoco dataset, achieved an accuracy of 68.8%, while retrieving semantically meaningful candidate images.

Transformer oil is widely used in high-voltage transformers as a cooling and insulation medium. Electrical and thermal stresses expose transformer oil to degradation, which may produce dissolved decay products (DDPs) that are partially adsorbed on the insulation paper and accelerate the degradation of oil and paper. An evanescent field D-shaped plastic optical fibre (POF) sensor was fabricated using the side-polishing technique to detect DDPs in transformer oil. A laser light source and power meter were used in the prototype to measure the output power of the POF sensor. The measurement of DDPs was conducted on seven samples of insulating oil collected from the field. The sensor performance was evaluated based on different sensing areas, and the sensor sensitivity was measured. Results show an excellent correlation between the output power and DDPs. A correlation analysis was conducted amongst sensor output power, transformer oil refractive index and area under the absorbance curve. The performance of the sensor demonstrates improved linearity and sensitivity of 97.49% and 26.6779 µW/RIU, respectively. The POF sensor is easily fabricated, low cost and can be used for real-time monitoring of high-voltage transformers.

Learning underlying patterns from sensory data is crucial in the Human Activity Recognition (HAR) task to avoid poor generalization when coping with unseen data. A key solution to such an issue is representation learning, which becomes essential when input signals contain activities with similar patterns or when patterns generated by different subjects for the same activity vary. To address these issues, we seek a solution to increase generalization by learning the underlying factors of each sensor signal. We develop a novel multi-channel asymmetric auto-encoder to recreate input signals precisely and extract indicative unsupervised futures. Further, we investigate the role of various activation functions in signal reconstruction to ensure the model preserves the patterns of each activity in the output. Our main contribution is that we propose a multi-task learning model to enhance representation learning through shared layers between signal reconstruction and the HAR task to improve the robustness of the model in coping with users not included in the training phase. The proposed model learns shared features between different tasks that are indeed the underlying factors of each input signal. We validate our multi-task learning model using several publicly available HAR datasets, UCI-HAR, MHealth, PAMAP2, and USC-HAD, and an in-house alpine skiing dataset collected in the wild, where our model achieved 99%, 99%, 95%, 88%, and 92% accuracy. Our proposed method shows consistent performance and good generalization on all the datasets compared to the state of the art.

Hyperparameter optimization poses a significant challenge when developing deep neural networks. Building a convolutional neural network (CNN) for implementation can be an arduous and time-intensive task. This work proposed an approach to optimize the hyperparameters of one dimensional (1D-CNN) to improve the accuracy of human activity recognition (HAR). The framework includes a parametric depiction of 1D-CNNs along with an optimization process for hyperparameters aimed at maximizing the model's performance. This work designed the method called OPTConvNet for hyperparameter optimization of 1D-CNN using Hierarchical Particle Swarm Optimization (H-PSO). The H-PSO algorithm is designed to optimize the architectural, layer and training parameters of 1D-CNN. The H-PSO optimizes the architecture of the 1D-CNN at initial level. Layer and training hyperparameters will be optimized at the next level. The proposed approach employs an exponential-like inertia weight to fine-tune the balance between exploration and exploitation of particles to prevent premature convergence to a local optimum solution in the PSO algorithm. The H-PSO- CNN is evaluated on publicly available sensor- human activity recognition (S-HAR) datasets namely, UCI-HAR, Daphnet Gait, Opportunity and PAMPA2 datasets.

In recent years, many researchers have focused on using convolutional neural networks to perform human activity recognition as evidenced by the emergence of a number of convolutional neural network architectures such as LeNet-5, AlexNet and VGG16 and modern architectures such as ResNet, Inception V3, Inception-ResNet, MobileNet V2, NASNet and PNASNet. The main characteristic of a convolutional neural network (CNN) is its ability to extract features automatically from input images, which facilitates the processes of activity recognition and classification. Convolutional networks indeed derive more relevant and complex features with every additional layer. In addition, CNNs have achieved perfect classification on highly similar activities that were previously extremely difficult to classify. In this paper, we evaluate modern convolutional neural networks in terms of their human activity recognition accuracy, and we compare the results with the state of the art methods. In our research, we used two public data sets, HMDB (Shooting gun, kicking, falling to the floor, punching) and the Weizman dataset (walking, running, jumping, bending, one hand waving, two-hand waving, jumping in place, jumping jack, skipping). Our experimental results indicated that the CNN with NASNet architecture achieves the best performance of the six CNN architectures on both human activity data sets (HMDB and Weizman).

Human Action Recognition (HAR)is a challenging time series classification problem that has received significant attention from computer vision researchers. In this paper, different techniques used for human activities are investigated, and a human action recognition approach using a Convolutional Neural Network (CNN) with Long Short Term Memory (LSTM) and generative adversarial network is proposed. The proposed research evaluates the performance of cross-entropy and adversarial loss function for HAR analysis. Two different datasets UFC101 and the classic KTH dataset, are used for experimental purposes. The UFC101 dataset contains 13k videos in which 101 human actions are included i.e., playing instrument, makeup, etc. In contrast, KTH dataset contains 600 videos containing six human activities, including walking, running, jogging, hand clapping and hand waving, performed by 25 different persons. Also, demonstrates the process of HAR by mixing both datasets and evaluate the performance. The GAN enhances the model robustness by applying adversarial training which fully discovers the underlying connections in both intra-view and cross-view aspects.