HandFi: WiFi Sensing based Hand Gesture Recognition using Channel State Information

Due to the importance of vital signs, breathing rate and heart rate have been widely used in heath care. In the past few years, various systems and approaches have been proposed to detect and monitor breath and heart beat. In this paper, we show that Wi-Fi signals can also be used to recognize and count breath and heart beats with different postures. The intuition is that human activities have different impacts on time-series of Channel State Information (CSI) values, which can be utilized to recognize macro and micro human activities. In this paper, we design a Wi-Fi signal based breath and heart beats recognition system called Wi-Health. WiHeath consists of two Commercial Off-The-Shelf (COTS) Wi-Fi devices, a sender (Wi-Fi router) and a receiver (laptop). The sender sends 802.11n packets to the receiver, while the receiver continuously receives these packets and records in real-time CSI value. The receiver analyzes the collected CSI values and determines if the human is alive and the number of heart beats and breaths.

Indoor occupancy detection (IOD) via Wi-Fi sensing capitalizes on the varying patterns in CSI (Channel State Information) to estimate the number of people in a given area. However, the precision of such systems heavily depends on the quality of the CSI data, which can be degraded by noise and environmental factors. To address this issue, In this paper, we present a CSI preprocessing method to improve the accuracy of IOD systems using Wi-Fi sensing. Unlike existing preprocessing methods that use computationally complex signal processing or statistical techniques, we expand the dimension of CSI amplitude data into a three-channel vector through nonlinear transformation to amplify subtle differences between CSI data belonging to a different number of people. By drawing clearer boundaries between CSI data distributions belonging to a different number of people in a monitored area, our method improves the people-counting accuracy of a Wi-Fi sensing system. To ensure temporal consistency and improve data quality, we discretize the CSI measurements based on their transmission periods and aggregate consecutive measurements over a given time interval. These samples are then fed into a Convolutional Neural Network (CNN) specifically trained for the IOD task. Experimental results in diverse real-world scenarios verify that compared to the traditional methods, the enhanced feature representation capability of our approach leads to more accurate and robust sensing outcomes even in the most resource-constrained environment, where a commercial off-the-shelf CSI capture machine with only one antenna is used when a Wi-Fi sender with one transmit antenna sends packets periodically to the channel with the smallest Wi-Fi channel bandwidth.

Wi-Fi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by channel state information (CSI) extracted from Wi-Fi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented, and balanced CSI samples in a new environment for adaptation algorithms, but randomly captured CSI samples can be easily collected. In this article, we first explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an annotation-efficient Wi-Fi sensing model based on a novel geometric self-supervised learning algorithm. The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in Wi-Fi sensing. The AutoFi is implemented on a pair of Atheros Wi-Fi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public data sets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar data sets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that AutoFi takes a huge step toward automatic Wi-Fi sensing without any developer engagement. Our codes have been included in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/xyanchen/Wi-Fi-CSI-Sensing-Benchmark</uri> .

Two factor authentication offers significant benefits but has had limited uptake as users find the extra step required at each authentication attempt (e.g., to enter a verification code) to be onerous. In this paper, we present a novel twofactor authentication system called Wi-Access that requires no additional effort from the user beyond typing her credentials. The first factor is established by the knowledge of the password. Wi-Access uses the unique perturbations in the ambient WiFi signals caused by the hand and finger motions of the user during password entry as a way to confirm inherence (i.e., something that the user is) as the second factor for authentication. We show that the way in which Alice moves her hands and fingers while typing her password is uniquely manifested in the Channel State Information (CSI) of ubiquitous WiFi signals. We also show that if Bob types Alice's password, then the corresponding CSI signals are significantly distinct to Alice. Wi-Access uses Principle Component Analysis to reduce the dimensionality of the CSI data. A variance-based thresholding algorithm is used to extract the CSI signals that precisely correspond to the typing action. A one-class classifier based on an autoencoder that uses specific thresholds tailored to each user's training data is used to make the authentication decision. We implement Wi-Access using offthe-shelf 802.11n devices and evaluate its performance with 10 subjects. Our tests reveal that Wi-Access can on average achieve 92% authentication accuracy with 8% false positives. Moreover, Wi-Access is robust in preventing attacks with a 91.8% attack detection accuracy.

Channel state information (CSI) estimation is one of the most fundamental problems in wireless communication systems. Various methods, so far, have been developed to conduct CSI estimation. However, they usually require high computational complexity, which makes them unsuitable for 5G wireless communications due to employing many new techniques (e.g., massive MIMO, OFDM, and millimeter-Wave (mmWave)). In this paper, we propose an efficient online CSI prediction scheme, called OCEAN, for predicting CSI from historical data in 5G wireless communication systems. Specifically, we first identify several important features affecting the CSI of a radio link and a data sample consists of the information of the features and the CSI. We then design a learning framework that is an integration of a CNN (convolutional neural network) and a long short term with memory (LSTM) network. We also further develop an offline-online two-step training mechanism, enabling the prediction results to be more stable when applying it to practical 5G wireless communication systems. To validate OCEAN's efficacy, we consider four typical case studies, and conduct extensive experiments in the four scenarios, i.e., two outdoor and two indoor scenarios. The experiment results show that OCEAN not only obtains the predicted CSI values very quickly but also achieves highly accurate CSI prediction with up to 2.650-3.457 percent average difference ratio (ADR) between the predicted and measured CSI.

Gesture recognition has been the subject of intensive research in recent years owing to its wide applications. Unlike traditional systems, which usually require wearable sensors, many recent works have achieved the desirable gesture recognition performance using wireless channel state information from commercially available WiFi devices. However, existing works generally require training new models for different locations due to the location-dependent nature of channel state information. This paper proposes a location-independent system that can recognize gestures performed in a new location without training a new model. Our approach uses disentanglement that extricates location and other extraneous information from those needed for gesture recognition. The implementation is based on an unsupervised invariance induction framework consisting of feature extraction, a multi-output latent space, gesture recognition, and decoder modules. The key idea in designing this system is to separate gesture-dependent features from location-dependent features. Specifically, the feature extraction module consisting of a long short-term memory network is employed to select representative features; it essentially serves as an encoder to generate the latent space. During the training process, the network learns to cluster features representation for the gesture recognition and decoder by minimizing the total loss of the gesture recognition and decoder modules. We test our system with a dataset collected from various subjects performing four different gestures in multiple locations in seven rooms with different layouts. The results show that our location-independent gesture recognition system can achieve 88.69% accuracy for new locations.

BackgroundOlder adults belonging to racial or ethnic minorities with low socioeconomic status are at an elevated risk of developing dementia, but resources for assessing functional decline and detecting cognitive impairment are limited. Cognitive impairment affects the ability to perform daily activities and mobility behaviors. Traditional assessment methods have drawbacks, so smart home technologies (SmHT) have emerged to offer objective, high-frequency, and remote monitoring. However, these technologies usually rely on motion sensors that cannot identify specific activity types. This group often lacks access to these technologies due to limited resources and technology experience. There is a need to develop new sensing technology that is discreet, affordable, and requires minimal user engagement to characterize and quantify various in-home activities. Furthermore, it is essential to explore the feasibility of developing machine learning (ML) algorithms for SmHT through collaborations between clinical researchers and engineers and involving minority, low-income older adults for novel sensor development.ObjectiveThis study aims to examine the feasibility of developing a novel channel state information–based device-free, low-cost Wi-Fi sensing system, and associated ML algorithms for localizing and recognizing different patterns of in-home activities and mobility in residents of low-income senior housing with and without mild cognitive impairment.MethodsThis feasibility study was conducted in collaboration with a wellness care group, which serves the healthy aging needs of low-income housing residents. Prior to this feasibility study, we conducted a pilot study to collect channel state information data from several activity scenarios (eg, sitting, walking, and preparing meals) using the proposed Wi-Fi sensing system continuously over a week in apartments of low-income housing residents. These activities were videotaped to generate ground truth annotations to test the accuracy of the ML algorithms derived from the proposed system. Using qualitative individual interviews, we explored the acceptability of the Wi-Fi sensing system and implementation barriers in the low-income housing setting. We use the same study protocol for the proposed feasibility study.ResultsThe Wi-Fi sensing system deployment began in November 2022, with participant recruitment starting in July 2023. Preliminary results will be available in the summer of 2025. Preliminary results are focused on the feasibility of developing ML models for Wi-Fi sensing–based activity and mobility assessment, community-based recruitment and data collection, ground truth, and older adults’ Wi-Fi sensing technology acceptance.ConclusionsThis feasibility study can make a contribution to SmHT science and ML capabilities for early detection of cognitive decline among socially vulnerable older adults. Currently, sensing devices are not readily available to this population due to cost and information barriers. Our sensing device has the potential to identify individuals at risk for cognitive decline by assessing their level of physical function by tracking their in-home activities and mobility behaviors, at a low cost.International Registered Report Identifier (IRRID)DERR1-10.2196/53447

IEEE 802.11 CSI randomization to preserve location privacy: An empirical evaluation in different scenarios

Due to its high-dimensional data characteristics, the channel state information (CSI) of Wi-Fi signals has become a strong candidate for use in indoor localization. In addition, machine learning techniques can improve the accuracy of indoor localization systems using multiview CSI data received at multiple access points (APs). However, in complex environments, most CSI views collected at APs in non-line-of-sight (NLoS) configurations relative to a transmitter may lose so much useful data information as to become nonsalient. In this paper, we propose a practical machine learning approach named unsupervised view-selective deep learning (UVSDL), in which only the most salient CSI view is selected in an unsupervised manner to be applied in regression for localization. In an experiment in a complex building, our variational deep learning (VDL)-based regression method with the most salient CSI view achieves a localization accuracy of 1.36 m, significantly outperforming the best-known system BiLoc by 25 %.

In this paper, a WiFi and visual fingerprint localization model based on low-rank fusion (LRF-WiVi) is proposed, which makes full use of the complementarity of heterogeneous signals by modeling both the signal-specific actions and interaction of location information in the two signals end-to-end. Firstly, two feature extraction subnetworks are designed to extract the feature vectors containing location information of WiFi channel state information (CSI) and multi-directional visual images respectively. Then, the low-rank fusion module efficiently aggregates the specific actions and interactions of the two feature vectors while maintaining low computational complexity. The fusion features obtained are used for position estimation; In addition, for the CSI feature extraction subnetwork, we designed a novel construction method of CSI time-frequency characteristic map and a double-branch CNN structure to extract features. LRF-WiVi jointly learns the parameters of each module under the guidance of the same loss function, making the whole model more consistent with the goal of fusion localization. Extensive experiments are conducted in a complex laboratory and an open hall to verify the superior performance of LRF-WiVi in utilizing WiFi and visual signal complementarity. The results show that our method achieves more advanced positioning performance than other methods in both scenarios.

This paper relates to the application method of channel state information (CSI) to the Viterbi (maximum likelihood) decoder in the digital terrestrial television broadcasting system. The proposed Viterbi decoder uses the CSI derived from the pilots inserted in the transmitter of the COFDM (coded orthogonal frequency division multiplexing) system. The CSI is calculated by interpolation using the pilots in the receiver. The active real (I) and imaginary (Q) data after equalization are transferred to the branch metric calculation block that decides the euclidean distance for soft decision decoding and also the estimated CSI values are transferred to the same block. After calculating the euclidean distance for the soft decision decoding, the euclidean distance of the branch metric is multiplied by CSI. To do so, new branch metric values that consider each carrier state information are obtained. We simulated this method in Rayleigh fading defined in the ETSI standard. From the simulation, this method has better performance of about 0.15 dB to 0.7 dB and 2.2 dB to 2.9 dB in the Rayleigh channel than that of conventional soft decision Viterbi decoding with or without a bit interleaver where the constellation is QPSK, l6-QAM and 64-QAM.

The past years have witnessed the rapid conceptualization and development of wireless sensing based onChannel State Information (CSI)with commodity WiFi devices. Recent studies have demonstrated the vast potential of WiFi sensing in detection, recognition, and estimation applications. However, the widespread deployment of WiFi sensing systems still faces a significant challenge: how to ensure the sensing performance when exposing a pre-trained sensing system to new domains, such as new environments, different configurations, and unseen users, without data collection and system retraining. This survey provides a comprehensive review of recent research efforts on cross-domain WiFi Sensing. We first introduce the mathematical model of CSI and explore the impact of different domains on CSI. Then we present a general workflow of cross-domain WiFi sensing systems, which consists of signal processing and cross-domain sensing. Five cross-domain sensing algorithms, including domain-invariant feature extraction, virtual sample generation, transfer learning, few-shot learning and big data solution, are summarized to show how they achieve high sensing accuracy when encountering new domains. The advantages and limitations of each algorithm are also summarized and the performance comparison is made based on different applications. Finally, we discuss the remaining challenges to further promote the practical usability of cross-domain WiFi sensing systems.

This paper presents a new approach for recognizing human activities that involve two humans, referred to as human-human interactions, using Wi-Fi signals. The proposed approach utilizes the Channel State Information (CSI) metric of the Wi-Fi signals to characterize the performed interactions in indoor environment. Specifically, the proposed approach analyzes the CSI data and extracts a set of time-domain and frequency-domain features that comprise salient information to distinguish between the performed interactions. The extracted features are used to construct a multi-class support vector machine classifier that can recognize the classes of the interactions comprised within the CSI data. The performance of the proposed approach was evaluated using our publicly available human-human interaction CSI dataset that contains the CSI data recorded for 40 pairs of participants while performing 13 interactions. The experimental results indicate that our proposed approach achieved an average recognition accuracy of 69.78% computed overall the 13 interactions. The reported results for each pair of participants demonstrate the feasibility of our proposed approach to recognize human-human interactions using the CSI metric of the Wi-Fi signals.

The prevalence of WiFi infrastructures has enabled the advancement of a wide range of WiFi-based intelligent applications. Behavioral sensing, as an increasingly popular example, is attracting growing research interest in developing a device-free, accurate, and privacy-friendly interaction. Recent advances in behavioral research have shown great promise, but most of existing systems fail to capture the full scope of the relationship between WiFi signals and behaviors. In this work, we propose a behavioral sensing framework by modeling channel state information (CSI) of WiFi signals. The key insight is to harness antenna diversity and model signals reflected off the human body from multiple WiFi access points (APs) to infer daily activities. To this end, a CSI tensor is first constructed by aggregating the CSI from all WiFi APs. Next, a CANDECAMP/PARAFAC (CP) approach is leveraged for factorizing this CSI tensor into several single-rank tensors as key features. Then, a features optimization strategy is applied to the extracted features for selecting those correlated and dominant features by an embedded graph learning and constraint. Finally, a recurrent neural network (RNN) model is used to take the optimized features and capture a robust mapping between the input and output for recognizing daily activities in homes without requiring a prior calibration and any explicit involvement from the users. We design and implement a behavioral sensing prototype reusing existing WiFi and then carry out comprehensive experiments in two indoor environments. Extensive experimental results suggest that our scheme can accurately and effectively recognize the users’ behaviors and also outperforms existing ones.

Recently, device-free human behavior recognition has become a hot research topic and has achieved significant progress in the field of ubiquitous computing. Among various implementation, behavior recognition based on WiFi CSI (channel state information) has drawn wide attention due to its major advantages. This paper investigates more than 100 latest CSI based behavior recognition applications within the last 6 years and presents a comprehensive survey from every aspect of human behavior recognition. Firstly, this paper reviews general behavior recognition applications using the WiFi signal and presents the basic concept of CSI and the fundamental principle of CSI-based behavior recognition. This paper analyzes the key components and core characteristics of the system architecture of human behavior recognition using CSI. Afterward, we divide the sensing procedures into many steps and summarize the typical studies from these steps, including base signal selection, signal preprocessing, and identification approaches. Next, based on the recognition technique, we classify the applications into three groups, including pattern-based, model-based, and deep learning-based approach. In every group, we categorize the state-of-the-art applications into three groups, including coarse-grained specific behavior recognition, fine-grained specific behavior recognition, and activity inference. It elaborates the typical behavior recognition applications from five aspects, including experimental equipment, experimental scenario, behavior, classifier, and system performance. Then, this paper presents comprehensive discussions about representative applications from the implementation view and outlines the major consideration when developing a recognition system. Finally, this article concludes by analyzing the open issues of CSI-based behavior recognition applications and pointing out future research directions.